After 2 years of work by 442 contributors across 132 institutions, I am thrilled to announce that the github.com/google/BIG-bench paper is now live: arxiv.org/abs/2206.04615. BIG-bench consists of 204 diverse tasks to measure and extrapolate the capabilities of large language models.