Check your understanding of this module before moving on.
?
0/3
Checkpoint quiz 3 questions
A quick check on this module's key ideas before you move on
Question 01
What is the key architectural difference between transformers and RNNs for processing sequences?
A is correct. Instead of a sequential recurrence that processes one time step at a time, transformers use self-attention to let every position attend to every other position simultaneously, which enables much more parallel computation during training.
Question 02
Since self-attention has no inherent notion of sequence order, how do transformers represent the position of each token?
A is correct. Without positional encodings, self-attention would treat a sequence as an unordered set, since it computes similarity between tokens regardless of their position. Adding positional encodings gives the model the information it needs to distinguish 'order matters' patterns.
Question 03
Why do transformer blocks typically use multiple attention 'heads' rather than a single attention computation?
A is correct. Splitting attention into multiple heads, each with its own learned query/key/value projections, lets different heads specialize in capturing different kinds of relationships between tokens, which a single attention computation could not capture as flexibly.