Valentius Kryptix

Machine learning models involves choosing the right approach for a particular problem and evaluating whether the selected model performs reliably. As part of my Data Science internship at Valentius Kryptix, I completed the “Milestone: Model Comparison Report” to compare different supervised and unsupervised machine learning approaches on the Breast Cancer Wisconsin dataset.

The main objective of this milestone was not simply to train multiple models, but to understand how different algorithms behave, how their results should be evaluated, and what additional insights can be obtained from unsupervised learning.

Dataset and Project Objective

For this project, I used the Breast Cancer Wisconsin dataset containing diagnostic measurements derived from breast cell characteristics. The dataset contains 569 samples and 30 numerical features after removing unnecessary columns.

The target variable, diagnosis, represents two classes:

• Benign (0)
• Malignant (1)

The dataset was divided into training and testing sets using an 80/20 stratified split. Stratification was used to preserve the class distribution in both sets.

Supervised Learning Models

Two different supervised classification algorithms were selected:

Logistic Regression

Logistic Regression was used as a linear classification algorithm. Since the dataset contains features with different scales, StandardScaler was used before training the model.

Random Forest

Random Forest was used as a tree-based ensemble classification algorithm. It combines multiple decision trees to make predictions and can capture nonlinear relationships in the data.

Both models were trained using the same training data and evaluated on the same unseen test set to make the comparison consistent.

Model Evaluation

Instead of relying only on accuracy, I evaluated the models using Accuracy, Precision, Recall, F1-Score, and Confusion Matrix.

The results were:

Model | Accuracy | Precision | Recall | F1 Score
Logistic Regression | 96.49% | 97.50% | 92.86% | 95.12%
Random Forest | 96.49% | 100.00% | 90.48% | 95.00%

Both models achieved the same accuracy of 96.49%. However, the other metrics revealed differences between them.

Logistic Regression achieved higher Recall and a slightly higher F1 Score. Its confusion matrix contained 3 false negatives, while Random Forest had 4 false negatives.

Random Forest achieved 100% precision on the test set and produced no false-positive predictions, but it missed one additional malignant case compared with Logistic Regression.

This comparison demonstrates why evaluating a classification model using multiple metrics is important. Accuracy alone would not show these differences.

Unsupervised Learning with K-Means

To explore the dataset from another perspective, I also applied K-Means clustering.

Unlike supervised learning, K-Means does not use the diagnosis target while forming clusters. The features were standardized before clustering, and the algorithm was configured to create two clusters.

The resulting clusters were:

Cluster | Benign | Malignant
Cluster 0 | 339 | 36
Cluster 1 | 18 | 176

The results showed that the discovered clusters had a strong relationship with the actual diagnosis labels, even though diagnosis was not provided to K-Means during clustering.

The Silhouette Score was 0.3434, indicating moderate separation between the clusters. Therefore, K-Means was useful as an exploratory technique for discovering natural groups, but it should not be treated as a replacement for supervised classification when labelled data is available.

Key Learnings

This milestone helped me understand several practical machine learning concepts.

First, different algorithms can produce similar accuracy while behaving differently on important metrics such as precision and recall.

Second, confusion matrices provide a more detailed view of model predictions by showing true positives, true negatives, false positives, and false negatives.

Third, unsupervised learning can reveal patterns in data without using target labels. Comparing those discovered groups with known labels can provide additional insights into the structure of a dataset.

Model Recommendation

Based on the evaluated test-set results, Logistic Regression was selected as the preferred supervised model for this project. It achieved the same accuracy as Random Forest while producing a slightly higher Recall and F1 Score and one fewer false negative.

This choice is based on the evaluation results from this project and is not intended as a clinical deployment recommendation.

Conclusion

The Model Comparison Report demonstrated the importance of evaluating machine learning approaches systematically rather than focusing only on whether a model produces predictions.

By comparing Logistic Regression and Random Forest and then applying K-Means clustering, I gained practical experience with both supervised and unsupervised machine learning.

The project strengthened my understanding of model evaluation, confusion matrices, classification metrics, clustering, and data-driven model comparison using Python and Scikit-learn.

Project Repository

The complete project, including the Jupyter Notebook, dataset, README, and requirements file, is available on GitHub:

https://github.com/student-Madhumitakhatua/ML-Model-Comparison-Suite