Check your understanding of this module before moving on.
?
0/3
Checkpoint quiz 3 questions
A quick check on this module's key ideas before you move on
Question 01
What is the main practical difference between batch gradient descent and stochastic gradient descent (SGD)?
A is correct. Batch gradient descent gives an exact gradient but requires a full pass over the data per update, which is slow on large datasets. SGD (and mini-batch SGD) trades some gradient noise for the ability to make many more updates per epoch.
Question 02
How does momentum help gradient descent in practice?
A is correct. Momentum keeps a moving average of recent gradient directions and adds it to the current update. This damps oscillations in noisy directions and builds speed in consistent directions, like a ball rolling downhill with inertia.
Question 03
Why might mini-batch gradient descent (e.g. batches of 32 or 64 examples) be preferred over both full-batch and single-example SGD in most deep learning practice?
B is correct. Mini-batches give a gradient estimate that's far less noisy than a single example but far cheaper to compute than the full dataset, and the batch size maps well onto GPU parallelism, which is why it's the default in practice.