Transformers are formed of Encoder and Decoder
For language conversion , Encoders are fed one language and decoder outputs another language.
Encoders are used to understand the context while decider are used to generate context.
Encoders and decoders contains a number of itself within them and there is no hard and fast rule to how many we can implement.
Each encoder is broken down into two sub layers
Decoder also has both of these layers but there is an extra layer called “Encoder Decoder Attention” which helps decoder focus on relevant part of the input sentence.
The words are embedded into vectors which are the passed to the encoders.
The embedding only happens in the bottom layer , and vector size is taken to be 512