BREAKING NEWS The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Physics to John J. Hopfield and Geoffrey E. Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.”

BREAKING NEWS The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Chemistry with one half to David Baker “for computational protein design” and the other half jointly to Demis Hassabis and John M. Jumper “for protein structure prediction.”

Combining discrete and continuous data is an important capability for generative models. To address this for protein design, we introduce Multiflow, a generative model for structure and sequence generation. Preprint: arxiv.org/abs/2402.04997 Code: github.com/jasonkyuyim/multi… 1/8

Just a reminder that RFdiffusion is free and publicly available, and thanks to @sokrypton, accessible through Google Colab! He's done an incredible job making it user friendly - no need to shell out the big $$$ colab.research.google.com/gi…

Announcing the Generative and Experimental Perspectives for Biomolecular Design workshop at #iclr2024! We hope to bring together researchers in ML and experimental biology to accelerate progress on real-world applications. Website: gembio.ai/ Paper deadline: Feb 3

Researchers from @MIT_CSAIL developed “FrameDiff,” a computational tool that uses generative AI to craft new protein structures, aiming to accelerate drug development and improve gene therapy. news.mit.edu/2023/generative…

Today we're making RF Diffusion, our guided diffusion model for protein design with potential applications in medicine, vaccines & advanced materials, free to use. The software has proven much faster and more capable than prior protein design tools. bakerlab.org/2023/03/30/rf-d…

Combining discrete and continuous data is an important capability for generative models. To address this for protein design, we introduce Multiflow, a generative model for structure and sequence generation. Preprint: arxiv.org/abs/2402.04997 Code: github.com/jasonkyuyim/multi… 1/8

Introducing ESM Cambrian. Unsupervised learning can invert biology at scale to reveal the hidden structure of the natural world. We’ve scaled up compute and data to train a new generation of protein language models. ESM C defines a new state of the art for protein representation learning.

Out today: the final version of our paper showing that CNNs are competitive to transformers as pretrained protein language models. with @alexijielu and @nfusi

Live! Keynote talk by Arnaud Doucet From Diffusion Models to Schrödinger Bridges West Exhibition Hall C, B3 neurips.cc/virtual/2024/invi…

New paper out! We introduce “Generator Matching” (GM), a method to build GenAI models for any data type (incl. multimodal) with any Markov process. GM unifies a range of state-of-the-art models and enables new designs of generative models. arxiv.org/abs/2410.20587 (1/5)

Digital art techniques can now devise custom, working biomolecules on demand. These proteins could form the basis for vaccines, therapeutics and biomaterials. Read the full story: nature.com/articles/d41586-0…

Directed evolution is key for unlocking new protein function But is difficult and time consuming So how can we accelerate protein design by 10-100x? With AI! Now introducing EVOLVEpro, an LLM-based model for evolving proteins rapidly and efficiently biorxiv.org/content/10.1101/…

Diffusion/flow models on discrete spaces would like guidance too. Learn how to discretely guide your model from the fabulous triplet team of co-authors @HNisonoff , @junhaobearxiong, and Stephan Allenspach! arxiv.org/abs/2406.01572

DALL-E’s amazing images are popping up all over the web. That software uses something called a diffusion model, which is trained to remove noise from static until a clear picture is formed. Turns out diffusion models can design proteins too!

Super excited to preprint our work on developing a Biomolecular Emulator (BioEmu): Scalable emulation of protein equilibrium ensembles with generative deep learning from @MSFTResearch AI for Science. #ML #AI #NeuralNetworks #Biology #AI4Science biorxiv.org/content/10.1101/…

🧬🤖 Introducing RNA-FrameFlow –– an unconditional generative model for 3D RNA backbone design! 📑: arxiv.org/abs/2406.13839 🧰: github.com/rish-16/rna-backb… Our method generates ≥ 40% self-consistent *all-atom* RNA backbones that are globally and locally realistic 💪🏻 1/9

Introducing AlphaQubit: our AI-based system that can more accurately identify errors inside quantum computers. 🖥️⚡ This research is a joint venture with @GoogleQuantumAI, published today in @Nature → goo.gle/3ZflWMn

The widely-used FAPE loss of AlphaFold2 actually suffers from a gradient vanishing problem! By fixing it with geodesic terms with ⭐Frame Aligned Frame Error⭐，we can fix it and improve performance on immune complexes by a large margin! Presenting at ICML TODAY!😊 (1/4)

We’re presenting AlphaProteo: an AI system for designing novel proteins that bind more successfully to target molecules. 🧬 It could help scientists better understand how biological systems function, save time in research, advance drug design and more. 🧵 dpmd.ai/3XuMqbX

Really cool to see the huge engagement surrounding AlphaFold 3 and the structures scientists are posting. We’re working to release the AF3 model (incl. weights) in the next 6 months for academic use, so it won’t depend on our research infra. Also the AFServer job limit is now doubling to 20 per day.

We’re excited to share our preprint where we show, for the first time, the atomically accurate design of VHH antibodies!

NeurIPS 2024 will have a track for papers from high schoolers.

I'm excited to announce that I'm part of the team at Xaira Therapeutics! This project has been some time in the making. I'm convinced that generative modelling, and ML for biology more generally, will play a pivotal role in the next generation of therapeutics. 1/2

Nearly all existing protein-based therapeutics are created from a fraction of possible protein concepts. This is about to change. We are excited to share a publication in @Nature describing Chroma, an AI model that can program novel proteins. generatebiomedicines.com/new…

New paper: how to do flow matching on discrete data. Flows give a simple generative framework and better performance than discrete diffusion models. Discrete flows are easily combined with continuous flow matching for multimodal models. arxiv.org/abs/2402.04997 A thread 1/7

Papers are important but so is sleep. We're extending the paper submission deadline to February 7th 11:59PM UTC-0. We look forward to your submissions! #ICLR2024

Two remarkable new papers @NatureMedicine on foundation models #AI for pathology nature.com/articles/s41591-0… 100,000 whole slide images w/ >100 million path images nature.com/articles/s41591-0… Multimodal of ~1.2 million images and text pairs @AI4Pathology @richardjchen @MYLu97 @DFKW_MD

Excited to share Discrete Flow Matching! A discrete flow framework that yields state-of-the-art non-autoregressive modeling. E.g., on code tasks (Pass@1): HumanEval 6.7/11.6, MBPP 6.7/13.1 w/ Tal Remez, @shaulneta, @FelixKreuk, @RickyTQChen, @syhw, @adiyossLC, @lipmanya

Introducing Sora, our text-to-video model. Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions. openai.com/sora Prompt: “Beautiful, snowy Tokyo city is bustling. The camera moves through the bustling city street, following several people enjoying the beautiful snowy weather and shopping at nearby stalls. Gorgeous sakura petals are flying through the wind along with snowflakes.”

Introducing FoldFlow-2, a new SOTA sequence-conditioned protein generative model! work w/ @DreamFoldAI James V @FatrasKilian Eric TL @PabloLemosP @riashatislam @ChengHaoLiu1 @jarridrb @tara_aksa @mmbronstein @AlexanderTong7 @bose_joey Arxiv: arxiv.org/abs/2405.20313 1/8 🧵

# RLHF is just barely RL Reinforcement Learning from Human Feedback (RLHF) is the third (and last) major stage of training an LLM, after pretraining and supervised finetuning (SFT). My rant on RLHF is that it is just barely RL, in a way that I think is not too widely appreciated. RL is powerful. RLHF is not. Let's take a look at the example of AlphaGo. AlphaGo was trained with actual RL. The computer played games of Go and trained on rollouts that maximized the reward function (winning the game), eventually surpassing the best human players at Go. AlphaGo was not trained with RLHF. If it were, it would not have worked nearly as well. What would it look like to train AlphaGo with RLHF? Well first, you'd give human labelers two board states from Go, and ask them which one they like better: Then you'd collect say 100,000 comparisons like this, and you'd train a "Reward Model" (RM) neural network to imitate this human "vibe check" of the board state. You'd train it to agree with the human judgement on average. Once we have a Reward Model vibe check, you run RL with respect to it, learning to play the moves that lead to good vibes. Clearly, this would not have led anywhere too interesting in Go. There are two fundamental, separate reasons for this: 1. The vibes could be misleading - this is not the actual reward (winning the game). This is a crappy proxy objective. But much worse, 2. You'd find that your RL optimization goes off rails as it quickly discovers board states that are adversarial examples to the Reward Model. Remember the RM is a massive neural net with billions of parameters imitating the vibe. There are board states are "out of distribution" to its training data, which are not actually good states, yet by chance they get a very high reward from the RM. For the exact same reasons, sometimes I'm a bit surprised RLHF works for LLMs at all. The RM we train for LLMs is just a vibe check in the exact same way. It gives high scores to the kinds of assistant responses that human raters statistically seem to like. It's not the "actual" objective of correctly solving problems, it's a proxy objective of what looks good to humans. Second, you can't even run RLHF for too long because your model quickly learns to respond in ways that game the reward model. These predictions can look really weird, e.g. you'll see that your LLM Assistant starts to respond with something non-sensical like "The the the the the the" to many prompts. Which looks ridiculous to you but then you look at the RM vibe check and see that for some reason the RM thinks these look excellent. Your LLM found an adversarial example. It's out of domain w.r.t. the RM's training data, in an undefined territory. Yes you can mitigate this by repeatedly adding these specific examples into the training set, but you'll find other adversarial examples next time around. For this reason, you can't even run RLHF for too many steps of optimization. You do a few hundred/thousand steps and then you have to call it because your optimization will start to game the RM. This is not RL like AlphaGo was. And yet, RLHF is a net helpful step of building an LLM Assistant. I think there's a few subtle reasons but my favorite one to point to is that through it, the LLM Assistant benefits from the generator-discriminator gap. That is, for many problem types, it is a significantly easier task for a human labeler to select the best of few candidate answers, instead of writing the ideal answer from scratch. A good example is a prompt like "Generate a poem about paperclips" or something like that. An average human labeler will struggle to write a good poem from scratch as an SFT example, but they could select a good looking poem given a few candidates. So RLHF is a kind of way to benefit from this gap of "easiness" of human supervision. There's a few other reasons, e.g. RLHF is also helpful in mitigating hallucinations because if the RM is a strong enough model to catch the LLM making stuff up during training, it can learn to penalize this with a low reward, teaching the model an aversion to risking factual knowledge when it's not sure. But a satisfying treatment of hallucinations and their mitigations is a whole different post so I digress. All to say that RLHF *is* net useful, but it's not RL. No production-grade *actual* RL on an LLM has so far been convincingly achieved and demonstrated in an open domain, at scale. And intuitively, this is because getting actual rewards (i.e. the equivalent of win the game) is really difficult in the open-ended problem solving tasks. It's all fun and games in a closed, game-like environment like Go where the dynamics are constrained and the reward function is cheap to evaluate and impossible to game. But how do you give an objective reward for summarizing an article? Or answering a slightly ambiguous question about some pip install issue? Or telling a joke? Or re-writing some Java code to Python? Going towards this is not in principle impossible but it's also not trivial and it requires some creative thinking. But whoever convincingly cracks this problem will be able to run actual RL. The kind of RL that led to AlphaGo beating humans in Go. Except this LLM would have a real shot of beating humans in open-domain problem solving.

Working with brilliant interns is one of the best things about the job. Really enjoyed supervising this project led by @json_yim!

Tired: poolside PMs updating Jira from their laptops Wired: cyborg PMs updating Jira tix from their Apple Vision Pro while on the go