Prior to the publication of the 2017 Transformer paper, researchers had identified attention mechanisms as a method to parallelize training and communicate information across longer sequences

Opening sign-in…