BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

2018 paper by Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova: a transformer pre-trained on large amounts of text that set new results on many language tasks; Google applied it to Search in 2019.

Read the paper ↗ arXiv

Also at blog.google

authored by

• Jacob Devlin person [src]
• Kenton Lee person [src]
• Kristina Toutanova person [src]
• Ming-Wei Chang person [src]

influenced by

• Attention Is All You Need writing 2017 [src]