Attention Is All You Need

2017 paper by Ashish Vaswani and seven Google colleagues introducing the transformer, a network built on attention that became the basis of large language models.

Read the paper ↗ arXiv

Part of the thread Hand-built meaning or learned meaning proposes

influenced

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding writing 2018 [src]
• Large language model concept [src]

authored by

• Ashish Vaswani person [src]