AI & ML Glossary
Terms from the courses, in one place. Search for what you need, and tap + Flashcard on anything you want to remember. It'll come back in your daily review until it sticks.
All 92 terms
Nothing matches “”.
Try a shorter word — searching “grad” finds every gradient-related term.
A
-
a -
A neuron's activation: the output after applying a non-linear activation function to
z. - Adjusted R²
- R² penalized for the number of features used, so it doesn't automatically rise when you add useless features.
-
α(alpha) - The regularization strength in ridge, lasso, and Elastic Net. Larger α means a stronger penalty on coefficient size.
- Attention weights
- A softmax distribution over input positions, derived from query/key similarity, used to weight a combination of values.
B
-
b0, b1, ..., bn(orβ0, β1, ..., βn) -
Intercept and per-feature coefficients in multiple linear regression:
y = b0 + b1·x1 + ... + bn·xn. - Backpropagation through time (BPTT)
- Backpropagation applied to an unrolled recurrent network, propagating gradients backward across time steps as well as layers.
- Backpropagation
- Computing the gradient of the loss with respect to every weight by applying the chain rule backward from the output layer to the input layer.
- Bagging
- Bootstrap aggregating: training many models on random subsamples of the data (drawn with replacement) and averaging their predictions. The basis of random forests.
- Batch normalization
- Normalizing activations within each mini-batch to stable mean and variance, reducing internal shift between layers and stabilizing training.
- Batch size
- The number of training examples used to compute one gradient update.
- Bayes' theorem
-
P(class | features) = P(features | class) · P(class) / P(features). Lets you flip a conditional probability around. - Bias
- Error from a model being too simple to capture the true pattern. Systematic and repeatable, regardless of which training sample it saw.
- Boosting
- Training models sequentially, where each new model corrects the residual errors of the ensemble built so far. The basis of gradient boosting and XGBoost.
C
-
C(SVM) - Controls how heavily margin violations are penalized. Large C means a narrower, stricter margin; small C means a wider, more tolerant one.
- Cell state (LSTM)
- A separate memory channel in an LSTM, regulated by gates, that lets information persist over much longer sequences than a vanilla RNN's hidden state alone.
- Centroid
- The mean position of all points currently assigned to a cluster in K-means.
- Conditional independence (Naive Bayes)
- The assumption that every feature is independent of every other feature, given the class. Rarely exactly true, but often harmless for ranking classes.
- Cross-entropy loss
- The standard classification loss, paired with a softmax output. Its gradient stays informative even for confidently wrong predictions.
- Curse of dimensionality
- As feature count grows, distances between points become less meaningful, weakening any purely distance-based method.
D
- Data augmentation
- Applying realistic transformations (crops, flips, rotations, etc.) to training examples to expose the model to more variation without collecting new data.
- Dead neuron
- A ReLU neuron stuck outputting zero for every input, because its weights drove it permanently into the negative, zero-gradient region.
- Dropout
- Randomly zeroing a fraction of activations on each forward pass during training, so the network can't over-rely on any single neuron.
E
- Early stopping
- Halting training once validation performance stops improving, keeping the weights from before the model began overfitting.
- Encoder-decoder
- An architecture where an encoder compresses an input sequence into a representation that a decoder uses to generate an output sequence, possibly of a different length.
- Entropy
-
−Σ pᵢ log₂(pᵢ). An information-theoretic alternative to Gini impurity, used to compute information gain. - Epoch
- One full pass through the entire training dataset.
- Explained variance ratio
- The fraction of a dataset's total variance a single principal component captures on its own.
- Exploding gradient
- Gradients growing without bound as they're propagated backward, causing unstable, divergent updates.
F
-
f/ŷ("y-hat") - The model's prediction.
- F1 score
-
The harmonic mean of precision and recall:
2 · (Precision · Recall) / (Precision + Recall). High only when both are reasonably high. - Feature map
- The grid of outputs produced by sliding one filter across an input; high values mark where that filter's pattern was detected.
- Fine-tuning
- Continuing to train a pretrained model's weights (often with a small learning rate, sometimes with early layers frozen) on a new, usually smaller, task-specific dataset.
- Forget / input / output gate
- The three learned gates in an LSTM that control what is discarded from, added to, and read out of the cell state at each time step.
- Forward propagation
- Computing a network's output by passing an input through each layer's weights, bias, and activation in sequence.
G
- Gini impurity
-
1 − Σ pᵢ². Roughly the probability that two randomly chosen points from a group have different labels. 0 means perfectly pure. - GRU
- Gated Recurrent Unit: merges the cell and hidden state into one and uses two gates instead of three, a simpler alternative to the LSTM.
H
-
H -
The hypothesis space: the family of functions an algorithm is allowed to search through. Training means finding the
f ∈ Hthat makesf(x)as close toyas possible. - He initialization
- Like Xavier, but scaled up to compensate for ReLU zeroing out roughly half its inputs.
- The vector an RNN carries forward between time steps, summarizing everything the network has processed in the sequence so far.
I
- Inertia
- The total squared distance from every point to its assigned centroid. Always decreases as K increases; the elbow method looks for where it stops decreasing fast.
K
-
K(K-means) - The number of clusters K-means is told to find. Chosen up front, often via the elbow method.
-
K(KNN) - The number of nearest neighbors consulted when classifying a new point. Small K means high variance; large K means high bias.
- K-fold cross-validation
- Rotating which portion of the data is used for validation across K rounds, so every sample contributes to both training and validation without ever doing both at once.
- Kernel / filter
- A small grid of learned weights that slides across an input, computing a weighted sum of each local patch it covers.
- Kernel trick
- Computing what a dot product would be in a higher-dimensional space, without explicitly building that space, letting SVM handle non-linear boundaries.
L
- L1 penalty (lasso)
- Penalizes the sum of absolute coefficient values. Can shrink coefficients to exactly zero, which performs feature selection.
-
l1_ratio - In Elastic Net, the mix between L1 and L2 penalty, from 0 (pure ridge) to 1 (pure lasso).
- L2 penalty (ridge)
- Penalizes the sum of squared coefficients. Shrinks every coefficient toward zero smoothly, but rarely to exactly zero.
-
Learning rate (
η) -
The step size used in a gradient descent update:
w ← w − η·∇w. - Learning rate decay
- Shrinking the learning rate over the course of training, taking large early steps and smaller, more precise steps later.
- Loss function
- A number measuring how wrong a model's predictions are. Training minimizes this number.
M
-
m,b -
Slope and intercept of a simple linear regression line:
y = mx + b. - MAE
-
Mean Absolute Error: the average of
|actual − predicted|. Weighs every error linearly. - Margin (SVM)
- The distance between a decision boundary and the closest training points on either side. SVM maximizes this.
- Mixed precision
- Using lower-precision number formats (e.g. float16) for most computation, combined with float32 for numerically sensitive steps, to speed up training.
- Momentum
- A running average of past gradients added to the current update, smoothing noisy steps and accelerating progress in consistent directions.
- MSE / RMSE
- Mean Squared Error and its square root. Squares errors before averaging, so large errors count disproportionately more.
- Multi-head attention
- Running several attention computations in parallel, each with its own learned projections, so different heads can specialize in different relationships.
- Multicollinearity
- When two or more features are highly correlated, making individual coefficients unstable to interpret even though overall predictions stay accurate.
N
-
n_estimators,learning_rate,max_depth - The three core gradient-boosting hyperparameters: number of trees, how much each tree's correction counts, and how deep each tree is allowed to grow.
O
- Overfit-a-tiny-batch test
- A debugging sanity check: a correctly wired model should be able to drive its loss near zero on a handful of examples it should easily memorize.
- Overfitting
- Training error keeps falling while validation error rises. The model has started fitting noise instead of signal.
P
- Padding
- Adding a border (usually zeros) around an input before convolving, used to control the output's spatial size.
- Pooling (max / average)
- Downsampling a feature map by keeping only the max (or average) value within each small local region.
- Positional encoding
- A signal added to input embeddings that injects information about token order, since self-attention has no inherent sense of position.
- Precision
-
TP / (TP + FP). Of everything flagged positive, how much actually was. - Principal component
- A new axis found by PCA, aligned with a direction of variance in the data. Ordered from most to least variance captured.
Q
- Query, key, value (Q, K, V)
- The three projections of an input used in attention: a query is compared against keys to compute weights, which combine the values.
R
- R²
-
Coefficient of determination:
1 − (Sum of Squared Residuals / Total Sum of Squares). The fraction of variance in the target explained by the model, relative to always guessing the mean. - Recall
-
TP / (TP + FN). Of everything actually positive, how much was caught. - ReLU
-
max(0, z). The default hidden-layer activation in most modern networks; cheap to compute and avoids vanishing gradients for positive inputs. - Residual / skip connection
- A shortcut that adds a layer's input directly to its output, letting gradients flow around the layer and making very deep networks easier to train.
- Residual
-
The gap between an actual value and a model's predicted value:
actual − predicted. -
ρ(rho) -
The correlation between trees in an ensemble. The variance of an average of
ntrees isρσ² + (1−ρ)σ²/n, soρσ²is a variance floor more trees alone can't beat.
S
- Self-attention
- Attention where queries, keys, and values all come from the same sequence, letting every position relate to every other position.
- SGD
- Stochastic gradient descent: updating weights using the gradient from one example or a small mini-batch, rather than the full dataset.
- Sigmoid / tanh
- Smooth, saturating activations bounded in (0, 1) or (−1, 1). Prone to vanishing gradients in deep networks since their derivative shrinks toward zero at the extremes.
- Softmax
- Converts a vector of raw scores (logits) into a probability distribution: exponentiate each value, then normalize so they sum to 1.
- Stride
- The step size a filter moves between positions. Larger strides produce smaller output feature maps.
- Support vectors
- The training points closest to the decision boundary. They're the only points that determine where the boundary sits.
T
- TP, FP, TN, FN
- True/false positive, true/false negative: the four cells of a confusion matrix.
- Train / validation / test split
- Three distinct data subsets: one to fit parameters, one to make decisions about the model, and one held back untouched for a single final honest check.
U
- Underfitting
- Both training and validation error are high and close together. The model is too constrained to capture the true pattern.
- Unrolling
- Representing an RNN's repeated application across time steps as one long computational graph with shared weights, so backpropagation can be applied across the whole sequence.
V
- Vanishing gradient
- Gradients shrinking toward zero as they're propagated backward across many layers or time steps, stalling learning in early layers.
- Variance
- Error from a model being too sensitive to the specific training sample it happened to see. Retrain on different data, get a noticeably different model.
W
-
w,b -
A neuron's weights and bias. Its pre-activation output is
z = w·x + b. - Warmup
- Starting training with a small learning rate and gradually increasing it, to avoid destabilizing updates while weights are still freshly initialized.
X
-
x - An input: a single example's features.
- Xavier / Glorot initialization
- Scales initial random weights based on layer size to keep activation variance roughly stable across layers, suited to symmetric activations like tanh.
Y
-
y - The true label or target value for an example.
Z
-
z - A neuron's pre-activation value, before an activation function is applied.
✓
Saved to your flashcards — review today's cards
Removed from your flashcards —