The transformer architecture integrated previously known techniques, including attention mechanisms and residual networks