What Machine Learning Actually Is

What you will learn in this lesson:

• The core idea of machine learning: learning a function from data instead of hand-coding rules, and what a "hypothesis space" actually is
• What "training" mathematically optimizes, and how that differs from inference
• Supervised, unsupervised, and reinforcement learning, each with two concrete real-world examples
• A practical method for figuring out which problem type you're actually facing
• Why getting this framing wrong can waste weeks of modeling work

Learning a Function Instead of Writing Rules

Traditional software is built by hand-coding rules. If you wanted to detect spam emails the old-fashioned way, you might write: "if the email contains the word 'lottery' and was sent from an unknown address, mark it as spam." This works for a while, but spammers adapt, language is messy, and soon you have thousands of brittle if-else statements that still miss obvious spam and flag legitimate emails.

Machine learning takes a fundamentally different approach. Instead of writing the rules yourself, you give the computer many examples of emails already labeled "spam" or "not spam," and you let an algorithm discover the pattern that separates the two. Formally, you are trying to learn a function f that maps an input x (the email's text, sender, formatting, links, etc.) to an output y (spam or not spam).

The algorithm doesn't invent this function out of thin air. It searches through a constrained space of candidate functions and picks the one that best matches the examples it was shown.

The Hypothesis Space: What the Algorithm Is Actually Searching Through

This "space of candidate functions" has a formal name: the hypothesis space (sometimes written H). Every machine learning algorithm restricts itself to a particular family of functions before it ever sees data. A linear regression model can only ever express straight lines (or flat hyperplanes in higher dimensions). Its hypothesis space is "all linear functions of the inputs."

A decision tree's hypothesis space is "all functions expressible as a sequence of if-else splits on feature values." A neural network with a given architecture has a hypothesis space defined by every possible setting of its weights.

This matters enormously in practice. If the true relationship between your inputs and outputs is a curve, but you choose linear regression, no amount of data or clever optimization will ever let the model find that curve. It simply isn't in the hypothesis space you gave it to search.

This is why "model selection" is not just a matter of taste: you are choosing the shape of the space the algorithm is allowed to search, and if the true pattern lives outside that space, you've already lost before training even starts.

Training is the process of searching this hypothesis space for the specific function (the specific line, tree, or set of weights) that best fits the data you have, according to some measure of "fit" called a loss function. Training literally means: adjust the model's internal parameters to minimize the loss function over the training examples.

For linear regression, the loss is typically the sum of squared prediction errors. For classification, it's often something like cross-entropy. Different algorithms minimize this loss differently: some have an exact closed-form solution, others use iterative numerical optimization like gradient descent. Either way, the goal is always the same: find the point in the hypothesis space with the lowest loss on the data you have.

This shift, from "humans encode the logic" to "the algorithm discovers the logic by searching a defined space of possibilities," is the single most important idea in machine learning. It is also why data quality and quantity matter so much: the function the algorithm finds can only be as good as the examples it learns from, and it can only be as expressive as the hypothesis space you gave it permission to search.

You don't need to wait until later lessons to see this in action. Here's the entire idea in eight lines of runnable code: a model is shown 6 labeled examples, searches its hypothesis space, and then predicts a label for a brand-new point it never saw during training.

from sklearn.linear_model import LogisticRegression

# 6 labeled training examples: [hours_studied, practice_tests_taken] -> passed (1) or failed (0)
X_train = [[1, 0], [2, 0], [2, 1], [5, 2], [6, 3], [8, 4]]
y_train = [0, 0, 0, 1, 1, 1]

model = LogisticRegression()
model.fit(X_train, y_train)  # this is "training": searching the hypothesis space

new_student = [[4, 1]]  # a brand-new input the model never saw
prediction = model.predict(new_student)  # this is "inference"
print("Predicted outcome for [4 hours studied, 1 practice test]:", "Pass" if prediction[0] else "Fail")
print("Learned coefficients:", model.coef_)

That's the whole loop: fit() searched logistic regression's hypothesis space for the line that best separates the labeled examples, and predict() applied the function it found to new input. Every algorithm in this course follows this same two-step shape. Only the hypothesis space and the search procedure change.

The Three Major Learning Paradigms

Machine learning problems are usually grouped into three broad categories, based on what kind of feedback the algorithm receives while learning.

Supervised Learning

In supervised learning, every training example comes with a "correct answer" attached, called a label. The algorithm's job is to learn the mapping from inputs to labels so it can predict labels for new, unseen inputs. Supervised learning splits further into:

• Classification, where the label is a category. Example one: a bank predicts whether a loan applicant will default, using historical applications labeled "defaulted" or "repaid." The label is categorical, and the decision (approve/deny) hinges directly on it. Example two: a hospital's imaging pipeline classifies a chest X-ray as showing pneumonia or not, trained on thousands of X-rays a radiologist has already labeled.
• Regression, where the label is a continuous number. Example one: a real estate platform predicts a house's sale price in dollars from its size, location, and age, trained on past sales where the actual sale price is known. Example two: a ride-sharing app predicts the expected trip duration in minutes given the pickup point, drop-off point, and time of day, trained on millions of completed historical trips with known durations.

Supervised learning is by far the most common type used in industry today, largely because labeled data, while expensive to collect, gives the algorithm a very clear, unambiguous training signal: "for this input, the answer was exactly this."

Unsupervised Learning

In unsupervised learning, there are no labels at all. The algorithm only sees the raw inputs and has to find structure, patterns, or groupings on its own.

• Clustering, example one: a retailer groups customers into segments based purely on purchasing behavior (frequency, basket size, category mix), without ever telling the algorithm what the "correct" segments are. The algorithm discovers groups like "frequent small-basket shoppers" and "occasional bulk buyers" purely from patterns in the data, and marketing then designs campaigns per segment.
• Clustering, example two: an astronomer clusters stars by their spectral and brightness measurements to discover previously uncatalogued star types, with no pre-existing labels to learn from. The groupings themselves are the discovery.
• Dimensionality reduction, example one: a genomics lab compresses thousands of gene-expression measurements per patient down to a handful of meaningful components, making the data easier to visualize and analyze without losing the dominant signal.
• Dimensionality reduction, example two: a recommendation engine compresses millions of sparse user-item rating vectors into a small number of "taste" dimensions, which is what makes techniques like matrix factorization for movie recommendations computationally feasible.

Unsupervised learning is especially useful for exploration: when you suspect there is structure in your data but don't yet know what it looks like, or when labels are too expensive, too slow, or simply impossible to collect at scale.

Reinforcement Learning

Reinforcement learning is different from both of the above. There is no fixed dataset of correct input-output pairs. Instead, an agent interacts with an environment, takes actions, and receives rewards or penalties based on the outcomes of those actions. Over many attempts, the agent learns a strategy (called a policy) that maximizes its cumulative reward over time.

• Example one: a program learns to play chess or Go by playing millions of games against itself, receiving a reward of +1 for a win, -1 for a loss, and nothing in between. It must work out for itself which intermediate moves actually contributed to winning, a problem called "credit assignment."
• Example two: a data-center cooling system learns to adjust fan speeds and chiller settings, receiving a reward signal based on energy saved while staying within safe temperature ranges, gradually discovering control strategies no human engineer explicitly programmed.

Reinforcement learning is powerful for sequential decision-making problems where actions have delayed consequences, but it is generally harder to train, more sample-hungry, and less common in everyday business applications than supervised learning.

Training and Inference

Training is the process of showing the algorithm your historical data so it can search the hypothesis space and settle on the function (the specific parameter values) that minimizes the loss. This is the "learning" phase, and it can take anywhere from milliseconds to weeks depending on the model and dataset size.

Inference is what happens after training is complete: you give the trained model a brand-new input it has never seen before, and it produces a prediction by simply evaluating the already-fixed function. When a spam filter checks an incoming email against the model trained last week, that is inference. Inference is typically far faster than training because the parameters are already fixed. The model just applies the function it learned, rather than searching for it.

A useful mental model: training is a student studying for an exam using practice questions with answer keys. Inference is the student sitting the actual exam, where the questions are new but the underlying concepts are the same ones they practiced.

Recognizing Which Problem Type You're Facing

Given a real scenario, work through these questions in order:

1. Do you have historical examples with known correct answers? If yes, you likely have a supervised problem. If no labels exist or are far too expensive to obtain, you're looking at unsupervised learning, or you need to budget time and money to create labels before anything else.
2. If supervised, is the answer a category or a number? "Will this customer churn, yes or no" is classification. "How many days until this customer churns" is regression. These call for different loss functions, different evaluation metrics, and sometimes entirely different algorithm families.
3. Is there a sequence of actions with delayed consequences, where the "correct" action at each step isn't directly labeled? That's the signature of a reinforcement learning problem. Most business problems are not actually like this, even though "the model decides what to do" can sound similar on the surface.
4. Are you trying to discover structure rather than predict a specific target? If your actual goal is "find natural groupings" or "find the dominant patterns," that's unsupervised learning, even if you eventually use the discovered structure as an input to a later supervised model.

Why Problem Framing Comes Before Algorithm Choice

A common beginner mistake is to jump straight to "which algorithm should I use?" before clearly defining the problem. But the type of problem you have almost always determines which family of techniques is even applicable. Choosing the algorithm before framing the problem is like picking a tool before you know what you're building.

Suppose you want to predict customer churn. Is this supervised (you have historical labels of who churned) or could it be framed as unsupervised (clustering customers by risk profile without historical labels)? Is "churn" a yes/no classification, or do you actually care about the probability of churning, or even the expected time until churn (a survival-analysis-flavored regression problem entirely different from plain classification)?

Getting this framing wrong means you could spend weeks tuning a sophisticated algorithm that is answering the wrong question, while a much simpler model matched to the correct framing would have worked far better with far less effort. A team that builds an elaborate classifier for "will churn in the next 30 days" when the business actually needed "expected lifetime value remaining" has wasted real engineering time solving an adjacent but different problem.

Before opening any modeling library, always ask: what exactly am I trying to predict or discover, what data do I actually have (labeled or unlabeled), and what does a "correct" answer even look like for this problem?

The rest of this course builds on the assumption that you can answer these questions clearly before reaching for any specific algorithm. The next lesson turns to the statistical vocabulary, mean, variance, distributions, and the bias-variance tradeoff, that you need before fitting your first real model.