Check your understanding of this module before moving on.
?
0/3
Checkpoint quiz 3 questions
A quick check on this module's key ideas before you move on
Question 01
What problem does attention solve relative to a basic encoder-decoder model that uses one fixed context vector?
A is correct. Instead of compressing the whole input into one vector, attention lets the decoder compute, at each step, a weighted combination of all encoder outputs, where the weights reflect how relevant each input position is to the current output step.
Question 02
In the query/key/value formulation of attention, what determines how much weight is given to a particular value?
A is correct. Attention computes a similarity score between the query and each key, then applies softmax to turn those scores into a probability distribution over positions, and uses those probabilities to weight the corresponding values in the final output.
Question 03
Why is attention described as letting a model learn 'soft' alignment rather than hard alignment between input and output positions?
A is correct. A softmax distribution over positions means every position contributes something (even if small), rather than the model being forced to commit to one single input position per output step, which is what makes the alignment 'soft.'