9.8 Transformers TBD

Attention Is All You Need https://arxiv.org/abs/1706.03762

Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention)

What is a Transformer?

The Illustrated Transformer

The Annotated Transformer.ipynb

9.8.1 Example transformer for time series forecasting

Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case

(Wu et al. 2020)

References

Wu, Neo, Bradley Green, Xue Ben, and Shawn O’Banion. 2020. “Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case.” arXiv Preprint arXiv:2001.08317.