Hi! I am currently senior staff research scientist at Google DeepMind. I am working on LLM and AI research.
Previously, I was co-founder/chief scientist at Reka AI. Before being a startup co-founder, I was senior research scientist at Google Brain where I worked on industry defining LLMs such as PaLM-2, Flan-2 and UL2.
Returning to Google DeepMind
Returning to Google and recounting my experiences as a startup co-founder.
What happened to BERT & T5? On Transformer Encoders, PrefixLM and Denoising Objectives
A Blogpost series about Model Architectures Part 1: What happened to BERT and T5? Thoughts on Transformer Encoders, PrefixLM and Denoising objectives
Training great LLMs entirely from ground up in the wilderness as a startup
Chronicles of training strong LLMs from scratch in the wild
2022 in Review: Top language AI research papers + interesting papers to read
Here are some of the best language AI / NLP papers of 2022!
On Emergence, Scaling and Inductive Bias
Some thoughts on emergent abilities and scaling language models.