Machine Learning Fundamentals
Regression, metrics, regularization, and ensembles — with worked math.
Before transformers there was — and still is — classical machine learning: the models that run most real-world prediction systems and the concepts every ML interview assumes. This guide orders the fundamentals articles on this site into one path: linear regression built from scratch, the metrics that tell you whether a model is any good, regularization, and the tree ensembles that win on tabular data.
Regression from scratch
The simplest model in ML, built by hand.
01Simple Linear Regression Model
Walk through the math behind ordinary least squares: deriving slope and intercept from scratch, understanding residuals geometrically, and the assumptions that make or break the...
02Multiple Linear Regression Model
Extending OLS to multiple predictors introduces multicollinearity, interpretation traps, and matrix algebra. Walk through the normal equations and what the coefficients actually...
Measuring model fit
RSS, R², RMSE, MAE — what each metric actually tells you.
03A Beginner's Guide to Residual Sum of Squares (RSS)
RSS is the foundation of least-squares regression. What it actually measures, why it grows with bad predictions, and how minimizing it gives you the best-fit line.
04A Beginner's Guide to R-Squared (R²)
R² tells you how much variance your model explains, but it can be negative, inflated by more features, and misleading on non-linear data. Break down the variance decomposition a...
05A Beginner's Guide to Root Mean Squared Error (RMSE)
RMSE penalizes large errors harder than MAE because it squares them first. Understand the formula from first principles, its units advantage, and when to prefer it over other me...
06A Beginner's Guide to Mean Absolute Error (MAE)
MAE treats all errors equally and stays in the same units as your target, making it more interpretable than RMSE. When it's the right loss function to use, and when it isn't.
Regularization
Ridge, Lasso, and Elastic Net: taming overfitting with penalties.
07A Beginner's Guide to Ridge Regression (L2 Regularization)
L2 regularization shrinks all coefficients toward zero without eliminating them. Understand the bias-variance tradeoff Ridge introduces, how the penalty term changes the loss su...
08A Beginner's Guide to Lasso Regression (L1 Regularization)
Lasso's L1 penalty drives coefficients exactly to zero, performing automatic feature selection. Why this makes it fundamentally different from Ridge, and when sparse solutions a...
09A Beginner's Guide to Elastic Net Regression (L1 + L2 Regularization)
Elastic Net blends L1 and L2 penalties to get Lasso's feature selection with Ridge's stability on correlated predictors. Walk through the dual-penalty math and how to tune both ...
Classification & ensembles
From KNN to the gradient-boosted trees that win Kaggle.
10K-Nearest Neighbors (KNN). Part 1: Classification
KNN is simple enough to fit in one equation but deceptively tricky in production. The geometry behind distance voting, why k=1 will always overfit, and where the algorithm break...
11Decision Trees: A Complete Guide with Hand-Worked Examples
Decision trees split data by finding the best question at each node. Learn how information gain and Gini impurity work, with a full hand-worked example.
12Random Forest: A Complete Beginner's Guide
Random Forest builds hundreds of deliberately different decision trees and takes a vote. This guide explains exactly why that works, covering bootstrap sampling, feature randomn...
13XGBoost: A Complete Beginner's Guide
XGBoost dominates every structured-data leaderboard. This guide explains exactly why: gradient boosting, second-order Taylor objectives, regularisation, optimal leaf weights, an...
Habits of good models
Feature engineering, overfitting, and metrics beyond accuracy.
14Feature Engineering: Making Data Understandable for Machines
The features you feed a model matter more than the model you choose. Practical techniques for encoding, transforming, and constructing features that improve generalization on re...
15Why Overfitting Is the Real Enemy of Machine Learning
High training accuracy means nothing if the model memorized the data. Understand why overfitting happens, how to spot it in learning curves, and which regularization techniques ...
16Metrics Beyond Accuracy: Measuring What Actually Matters
Accuracy hides class imbalance. Precision, recall, F1, AUC, and MCC each expose different failure modes. Choose the wrong metric and you'll ship a model that looks good on paper...
Machine Learning Fundamentals
The same ground as a full self-paced course — statistics to capstone, with quizzes, labs, and a certificate.
Start the courseTools & resources
Free utilities that pair with this guide.
Keep going
Guides that pick up where this one ends.