Objective: Synthesize everything from this course into one self-directed project: explore a dataset, engineer a feature, train at least three different model types, evaluate them with appropriate metrics, and justify a final model choice.
Setup
Every code block below runs live in your browser using Pyodide. Click Run to execute it. We'll
use load_breast_cancer from scikit-learn for a real-world binary classification problem, the same
dataset you've used in several labs across this course, so you can focus on the pipeline rather than getting
acquainted with new data.
Task 1: Exploratory Data Analysis
Load the dataset, check its shape and class balance. This is the same first move you made in the very first lab of this course.
import pandas as pd
from sklearn.datasets import load_breast_cancer
data = load_breast_cancer(as_frame=True)
df = data.frame
print("Shape:", df.shape)
print()
print("Class balance:")
print(df["target"].value_counts())
You should see a moderate class imbalance (roughly 60/40), not severe enough to require special handling, but worth knowing about before you interpret any metric.
Task 2: Feature Engineering and Scaling
Split the data first, then fit your scaler only on the training split. Fitting it on the full dataset
would leak test-set information into preprocessing. We'll also engineer one new feature: the ratio of
mean area to mean perimeter, a simple, defensible interaction that wasn't in the
original columns.
import pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
data = load_breast_cancer(as_frame=True)
df = data.frame
df["area_perimeter_ratio"] = df["mean area"] / df["mean perimeter"]
print("Correlation of new feature with target:", df["area_perimeter_ratio"].corr(df["target"]).round(3))
X = df.drop(columns=["target"])
y = df["target"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test) # transform only, never fit, on test data
print("Scaled training shape:", X_train_scaled.shape)
Task 3: Train Three Model Types
Loop over a small dictionary of models so you don't repeat boilerplate three times: a logistic regression (linear), a decision tree (single tree-based model), and a random forest (ensemble).
import pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
data = load_breast_cancer(as_frame=True)
df = data.frame
X = df.drop(columns=["target"])
y = df["target"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
models = {
"Logistic Regression": LogisticRegression(max_iter=5000),
"Decision Tree": DecisionTreeClassifier(max_depth=5, random_state=42),
"Random Forest": RandomForestClassifier(n_estimators=100, random_state=42),
}
results = []
for name, model in models.items():
model.fit(X_train_scaled, y_train)
y_pred = model.predict(X_test_scaled)
results.append({
"model": name,
"accuracy": accuracy_score(y_test, y_pred),
"precision": precision_score(y_test, y_pred),
"recall": recall_score(y_test, y_pred),
"f1": f1_score(y_test, y_pred),
})
results_df = pd.DataFrame(results)
print(results_df)
Tree-based models don't strictly need scaled features to perform well (splits are based on thresholds, not distances), but it's harmless to feed them the scaled data here for pipeline simplicity, and it keeps the comparison clean since every model sees identical inputs.
Task 4: Pick a Winner
Resist the urge to default to whichever model has the highest accuracy without looking further. A good justification reads something like: "Random Forest had the highest F1 (0.97 vs. 0.94 for logistic regression) and the fewest false negatives in its confusion matrix, which matters most for this screening use case, even though its raw accuracy was only marginally higher." Run the block below to pull up the confusion matrix for whichever model scored highest in Task 3, and use it to write your own 2-3 sentence justification.
import pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import confusion_matrix
data = load_breast_cancer(as_frame=True)
df = data.frame
X = df.drop(columns=["target"])
y = df["target"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train_scaled, y_train)
y_pred = model.predict(X_test_scaled)
print("Confusion matrix (rows=true, cols=predicted):")
print(confusion_matrix(y_test, y_pred))
Task 5: Stretch Goal, Hyperparameter Tuning
Try improving your winning model with a small grid search over n_estimators and
max_depth, then check whether it actually improved your test F1 score. Sometimes it does, and
sometimes a sensible default was already close to as good as you can get on a clean dataset like this one. That
honest comparison, not blind optimism that tuning always helps, is the real skill this capstone is meant to
build.
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
data = load_breast_cancer(as_frame=True)
df = data.frame
X = df.drop(columns=["target"])
y = df["target"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
param_grid = {"n_estimators": [50, 100, 200], "max_depth": [3, 5, None]}
grid = GridSearchCV(RandomForestClassifier(random_state=42), param_grid, cv=5, scoring="f1")
grid.fit(X_train_scaled, y_train)
print("Best params:", grid.best_params_)
print("Best CV F1:", round(grid.best_score_, 4))
print("Test F1 with tuned model:", round(grid.best_estimator_.score(X_test_scaled, y_test), 4))