Exploratory Data Analysis: A Repeatable Process

What you will learn in this lesson:

• Why EDA should be a repeatable process, not an unstructured "look around"
• The three-stage workflow: univariate, bivariate, then target relationship
• How to spot common red flags: skew, outliers, suspicious columns, and constant features
• A full runnable walkthrough of the process on a real dataset

Why EDA Needs a Process, Not Just Curiosity

It's tempting to treat exploratory data analysis as just "poking around" a dataset before the real work starts. That approach works fine the first time, but it doesn't scale: on your tenth project, an unstructured look around means you'll forget to check something you checked automatically the first nine times. A repeatable process turns EDA into a checklist you run every time, so the things that matter, missing values, suspicious columns, target imbalance, never slip through because you were tired or in a hurry.

This lesson gives you that checklist: three stages, run in order, every time.

Stage 1: Univariate, Look at Each Column Alone

Before any column can be related to anything else, you need to know what it looks like by itself. For every column, check: its data type, its range (min/max for numeric, unique values for categorical), how many values are missing, and its distribution shape (is it symmetric, skewed, or does it have an unusual spike at one value).

This stage catches problems early. A column that's supposed to be a percentage but has a maximum of 1000 is flagged here, before it ever reaches a model. A column where 95% of rows have the exact same value is flagged here too, since a feature like that carries almost no information and may not be worth keeping.

Stage 2: Bivariate, Look at Pairs of Columns

Once you understand each column alone, look at how columns relate to each other. A correlation matrix between numeric features is the standard tool here, the same one used in the multiple regression lesson to flag multicollinearity. For categorical columns, a simple cross-tabulation (counting how often each combination of categories appears) serves the same purpose.

This stage is where you'd catch two columns that are actually duplicates of each other under different names, or two features so strongly correlated that one of them is redundant.

Stage 3: Target Relationship, the Stage That Actually Matters Most

The final stage looks specifically at how every feature relates to the target you're trying to predict. For regression, this is usually each feature's correlation with the target, the same technique from the exploration lab in Phase 1. For classification, this is usually a comparison of each feature's distribution across the different classes.

This stage is also where you should be most suspicious. If a single feature correlates almost perfectly with the target, far better than anything else, ask why. Sometimes it's a genuinely strong predictor. Sometimes it's a leakage hint, a column that's only available because of information that wouldn't exist yet at prediction time (a "days since cancellation" column predicting churn is the classic example: it's not available until after the thing you're trying to predict has already happened).

A Full Walkthrough

import pandas as pd
from sklearn.datasets import load_breast_cancer

data = load_breast_cancer(as_frame=True)
df = data.frame

print("=== Stage 1: Univariate ===")
print("Shape:", df.shape)
print("\nMissing values per column (showing only columns with any):")
missing = df.isna().sum()
print(missing[missing > 0] if missing.sum() > 0 else "None")
print("\nAny constant columns (zero variance)?")
constant_cols = [c for c in df.columns if df[c].nunique() == 1]
print(constant_cols if constant_cols else "None")

print("\n=== Stage 2: Bivariate ===")
corr = df.drop(columns=["target"]).corr()
print("Sample of the correlation matrix (first 4x4):")
print(corr.iloc[:4, :4].round(2))

print("\n=== Stage 3: Target relationship ===")
target_corr = df.corr()["target"].drop("target").sort_values(key=abs, ascending=False)
print("Top 5 features most correlated with the target:")
print(target_corr.head(5).round(3))

Run this and notice how each stage answers a different question: stage 1 asks "is anything obviously broken," stage 2 asks "are any features redundant with each other," and stage 3 asks "what's actually predictive, and is any of it suspiciously predictive." Running all three, in this order, every time, is the habit this lesson is trying to build.