Valentius Kryptix

Introduction

Supervised learning is one of the most widely used approaches in Machine Learning. In supervised learning, a model learns from data where the correct output or target is already known. The model studies the relationship between input features and the target variable and then uses what it has learned to make predictions on new, unseen data.

For this task, I worked on a comparative study of five different supervised learning algorithms. The main goal was not just to train different models, but to understand how different algorithm families behave when they are trained and evaluated on the same dataset using the same experimental setup.

The five algorithms used in my project were Logistic Regression, Decision Tree, K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Naive Bayes.

Why Compare Multiple Algorithms?

There is no single Machine Learning algorithm that performs best for every problem. Different algorithms make different assumptions about the structure of the data.

For example, Logistic Regression works well when the relationship between features and the target can be represented using a relatively simple decision boundary. Decision Trees can capture non-linear relationships by creating a series of decision rules. KNN makes predictions based on nearby observations, while SVM tries to find an effective boundary between classes. Naive Bayes uses probability-based assumptions to classify observations.

Comparing these algorithms on the same dataset helps us understand their strengths and limitations instead of selecting a model simply because it is popular.

Dataset and Preprocessing

For the comparison, I used a real-world dataset suitable for a classification problem. The same dataset was used for all five models so that the comparison remained consistent.

Before training the models, the data was prepared using basic preprocessing steps. Missing values were handled, the relevant features and target were separated, and the dataset was divided into training and testing sets.

I used the same train-test split for every algorithm. This is important because using different splits could make the comparison unfair.

Feature scaling was also considered where required. Algorithms such as K-Nearest Neighbors and Support Vector Machine are sensitive to the scale of numerical features, so standardized features were used for those models. The preprocessing parameters were learned from the training data and then applied to the test data.

Model Training and Results

All five algorithms were trained using the same training data and evaluated on the same test data.

The results were:

Rank	Model	Accuracy	Precision	Recall	F1 Score
1	Logistic Regression	98.25%	98.61%	98.61%	98.61%
2	Support Vector Machine	98.25%	98.61%	98.61%	98.61%
3	K-Nearest Neighbors	95.61%	95.89%	97.22%	96.55%
4	Naive Bayes	92.98%	94.44%	94.44%	94.44%
5	Decision Tree	91.23%	95.59%	90.28%	92.86%

The comparison shows that Logistic Regression and Support Vector Machine achieved the highest overall performance, with an F1 score of approximately 0.986.

KNN also performed well, but its performance was lower than the top two models. Naive Bayes achieved a moderate result, while Decision Tree produced the lowest F1 score among the five models in this experiment.

Understanding the Results

The strong performance of Logistic Regression suggests that the classes in this dataset may have a relatively clear relationship with the input features. Because Logistic Regression is designed to learn a linear decision boundary, it can perform very well when the underlying structure of the data is reasonably suitable for a linear classifier.

SVM achieved the same score in this experiment. Its ability to find a separating boundary between classes can make it effective when the classes have a well-defined structure. Feature scaling also helps SVM work effectively with numerical inputs.

KNN performed well because observations belonging to the same class may have similar feature values. However, KNN depends heavily on distances between observations, so its performance can be affected by feature distributions and the choice of neighborhood size.

Naive Bayes performed reasonably well but relies on assumptions about the relationships between features. When those assumptions do not completely match the actual dataset, performance can decrease.

Decision Tree achieved the lowest F1 score in this particular comparison. A tree can model complex non-linear relationships, but its performance depends strongly on the structure and characteristics of the data and on the chosen tree parameters. A single decision tree may also be more sensitive to the particular training data than some other approaches.

What I Learned

The most important lesson from this project was that model selection should be based on experimentation and evaluation rather than assumptions.

By training five different algorithms under the same conditions, I could directly compare their performance and understand why different algorithms produce different results. I also gained practical experience with train-test splitting, preprocessing, feature scaling, model training, and classification metrics such as accuracy, precision, recall, and F1 score.

Creating the comparison chart made the differences between the models easier to understand visually.

This project also helped me understand that a high accuracy score alone does not always tell the complete story. Looking at precision, recall, and F1 score provides a better understanding of how a classification model performs.

Conclusion

This comparative study provided practical insight into five important supervised learning algorithms. Logistic Regression and SVM performed the best on the selected dataset, while Decision Tree performed the weakest among the models tested.

More importantly, the project helped me understand how algorithm characteristics, preprocessing, feature scaling, and dataset structure can influence Machine Learning performance.

For anyone learning Machine Learning, implementing and comparing multiple algorithms on the same dataset is a useful way to move beyond simply using a library and start developing intuition about how different models actually behave.

Project: Supervised Learning Model Comparison
Algorithms: Logistic Regression, Decision Tree, KNN, SVM, Naive Bayes
Evaluation: Accuracy, Precision, Recall, and F1 Score