TorchTPU's DeferAndFuse execution mode yields over 50% speed improvement compared to Strict Eager mode by fusing eager operators before dispatch to TPUs.

From the articleGoogle Open-Sources TorchTPU, a Native PyTorch Backend for Cloud TPUsAlso in Switch Your PyTorch Model From Nvidia to Google TPUs With One Line: TorchTPU Is Now Open Source, Switch Your PyTorch Model From Nvidia to Google TPUs With One Line: TorchTPU Is Now Open Source, Google Open-Sources TorchTPU, a Native PyTorch Backend for Cloud TPUs