Returning to Google DeepMind

Returning to Google and recounting my experiences as a startup co-founder.

Yi Tay 7/16/24 Yi Tay 7/16/24

What happened to BERT & T5? On Transformer Encoders, PrefixLM and Denoising Objectives

A Blogpost series about Model Architectures Part 1: What happened to BERT and T5? Thoughts on Transformer Encoders, PrefixLM and Denoising objectives

Yi Tay 3/6/24 Yi Tay 3/6/24

Chronicles of training strong LLMs from scratch in the wild

Yi Tay 3/30/23 Yi Tay 3/30/23

Documenting my 3.3 years at Google Research and Brain.

Yi Tay 3/3/23 Yi Tay 3/3/23

Releasing the new open source Flan-UL2 20B model.

Yi Tay 1/3/23 Yi Tay 1/3/23

Here are some of the best language AI / NLP papers of 2022!

Yi Tay 11/16/22 Yi Tay 11/16/22

Some thoughts on emergent abilities and scaling language models.