Attention
Also known as self-attention
A mechanism that computes weighted combinations of information according to its relevance.
In the real world
Self-attention can connect a pronoun to earlier context in a sequence.
Also known as self-attention
A mechanism that computes weighted combinations of information according to its relevance.
Self-attention can connect a pronoun to earlier context in a sequence.