Virtual Cell wrangler @genbioai | PhD @CMUCompBio | Creator contextualized.ml | context-adaptive models, disease simulators, personalized medicine
San Francisco, CA
- Tweets 606
- Following 378
- Followers 871
- Likes 1,662
Pinned Tweet
Honored to share a major thread of my PhD research, out now in PNAS. We address a core issue with how models are used for scientific discovery.
Models are so important that they define the entire scientific process... 1/n
8
46
317
61,730
My team at genbio.ai/ is hiring PhD interns this Summer to work on sequence-based foundation models (DNA, RNA, protein) as part of an AI-Driven Digital Organism (AIDO).
To apply, please send your CV, github, and website (if applicable) to caleb.ellington@genbio.ai
4
24
133
14,080
Pho Tran.
Thanh Vi.
Pho Shizzle.
Spring Kitchen.
Long ago, the four nations lived together in harmony. Then, everything changed when Pho Shizzle attacked.
1
16
60
Good software should be fast, reliable, reusable, and maintainable. A lot of BioML benchmarking is uh… not.
But biology doesn’t standardize to a few data types like language, audio, or images. We’re constantly inventing new ways to measure life... 1/n
1
13
60
5,387
Attending my first conference next week -- and I'll be giving a talk! If you'll be at CSHL Biological Data Science, look for our work on inferring sample-specific gene regulatory networks for 7000 tumors. Lots to come, but what a fun start to this project.
#biodata22
2
10
39
Beyond excited to announce the first release of Contextualized (v0.1.0), a statistical machine learning toolbox for estimating models, distributions, and functions with context-dependent behavior. Check out our demos for examples with code!
contextualized.ml/
4
6
36
I'm sorry to inform my followers that I no longer have a hot foreign gf BECAUSE NOW SHES MY HOT AMERICAN CITIZEN GF
1
1
29
NEW PREPRINT
Human decisions are nuanced and hard to quantify. Accurate models are too complex to interpret, and interpretable models are too simple to be accurate. But decision models must be both accurate and interpretable to support real decisions!
arxiv.org/abs/2310.07918
12
27
5,708
Bringing this back in honor of UT vs UW tonight
1
28
3,858
I'm a big admirer of what @manntis4 and @CorinWagen have built at rowansci.com/ in such a short time. Now they've released their first FMs as NNPs seem like they're at an inflection point. Really excited to dig in on Egret-1 and the future of NNPs. Come learn with us!
📡🧬 Join us for the next Foundation Models for Biology Seminar Series (#FM4Bio) session on June 18 at 9 AM PT, featuring @manntis4.
Eli will present "Egret-1: Pretrained Neural Network Potentials for Efficient and Accurate Bioorganic Simulation." Simulating atomic systems with both speed and accuracy could reshape molecular and materials design. Egret-1 offers a promising new path.
Save your spot → x.genbio.ai/seminar
1
2
30
3,303
The Contextualized Machine Learning White Paper
arxiv.org/abs/2310.11340
w/ @ben_lengerich
Intuition, applications, algorithms, and extensions for contextualized models: models that understand heterogeneity in real data, adapt to new environments, and are explainable by design.
8
22
4,635
AIDO models are getting bigger, stronger, and more multi-modal, and AIDO.ModelGenerator is making it all possible.
Today's release includes the new AIDO.Tissue, AIDO.StructurePrediction, and AIDO.Protein-RAG models, plus new tutorials, benchmarks, and open-source integrations.
1/ We’ve updated AIDO.ModelGenerator, our open-source framework, which enables scientists and engineers to integrate their own data with all publicly available data using state-of-the-art (SOTA) foundation models. AIDO.ModelGenerator can be used for adapting, benchmarking, and applying biological foundation models across many modalities and scales. 🧬🦠
New foundation models, datasets, and tutorials are now available on @HuggingFace and @GitHub.
8
26
1,796
Tomorrow at NeurIPS I'll be presenting my work on sample-specific graphical models, and using them to model the heterogeneous and patient-specific pathology of cancer with observational data.
Come find with me at the Generative AI and Biology workshop, or ping me to chat
2
8
22
2,588
Love visiting grandpa because I tell him about Hulu and he tells me which companies the mob used to own
1
22
In the PNAS article, we conduct the largest and most comprehensive evaluation of Contextualized models, producing sample-specific models of 7997 tumors, generating models for unseen disease types, and revealing new prognostic tumor types.
pnas.org/doi/10.1073/pnas.24…
1
24
1,760
Today I'll be taking over @CMUPittCompBio! If you're interested in doing a PhD, want to know about The Burgh, or simply want to chat about computational biology, I'll be answering questions all day
Meet Caleb (@probablybots) our final social media curator taking over X and Instagram! He is working tirelessly at @ericxing Lab to advance personalized medicine with ML/AI tools. Also, catch him scaling cliffs and leading as CPCBGSA’s President! Don't miss his takeover!
4
19
1,517
If you're interested in using or developing contextualized models, reach out to myself or @ben_lengerich. Ben just started his new lab at UW Madison focusing on language as context and LLMs, and I'm working on multi-modal bio contexts and bio FMs with @genbioai.
2
20
2,288
TX Energy: blackouts were caused by low energy yield & high demand
Abbott: so there were too many people who wanted power
TX Energy: more like not enough power for everyone
Abbott: so this wouldn't happen if TX had fewer people
TX Energy: I think you've got the wrong idea
1
19
These questions drove us to create Contextualized models. In addition to learning the traditional way, Contextualized models also learn from the addition of new study conditions.
contextualized.ml/
1
18
1,971
rejected from all my fellowships but another person said I look like timothee chalamet so it's all good
1
18
Typically, science is done by (1) defining a set of experimental conditions to study and (2) gathering many samples for these conditions. This enables model learning, because learning only works on large and controlled datasets.
3
18
2,376
moving in together is just taking turns saying "so you're telling me nobody ever taught you how to ___"
16
This means we can push the boundaries of study design, accounting for thousands of conditions simultaneously (such as individual genetics or medical history) and even producing models for a single sample while being far more accurate than classical approaches.
1
17
1,805
But then how do we produce insights for
- systems which are extremely variable, and cannot be captured by a single set of experimental conditions (i.e. complex and heterogeneous diseases like cancer)?
- systems that can't be observed many times (i.e. rare diseases)?
1
15
2,100
If you're apartment hunting, considering new students, switching jobs, or facing any other high-value decision in a competitive market, here's a bit of solace for you (and me).
cnellington.github.io/blog/2…
15
942
AIDO.ModelGenerator v0.1.2 is now on PyPI
Use the mgen CLI for no-code inference, embedding, and finetuning for the new SOTA AIDO models (Tissue, StructurePrediction, Protein-RAG), as well as ESM, Enformer, Borzoi, Geneformer, and scFoundation models.
pip install modelgenerator
1
6
16
1,596
Recently, we used contextualized.ml/ to go beyond physical limits in biology and medicine, inferring n-of-1 models for 7997 of patients and generating models of unseen diseases on-demand. This hints at how to develop accurate and personalized biological simulators like AIDO.
1/ How can we model each tumor’s unique biology instead of relying on one-size-fits-all approaches?
In our latest blog post, we highlight findings from the recent PNAS paper with GenBio AI Research Scientist @probablybots and Co-Founder and Chief Scientist @ericxing, showing how contextualized networks pave the way for an interactome module in AIDO.
1
6
16
2,299
earlier this year I spent 2 weeks crossing 100 miles of the Alaskan bush but the night I just spent in the Miami-Dade International Airport might be the hardest thing I've ever done
14
If you're at @SymposiumML4H, I'll be at poster 32 presenting our work on modeling and interpreting medical decisions with a novel interpretability mechanism, the context-specific policy, which achieves SOTA accuracy and exact interpretability.
#ml4h2023
arxiv.org/abs/2310.07918
1
13
1,154
Contextualized modeling is the culmination of over a decade of effort and multiple PhDs in CMU's SAILING Lab with @ericxing , and MIT's CSAIL Lab with @manoliskellis. Incredibly grateful to @ben_lengerich, @alshedivat, Mladen Kolar, and @itsrainingdata for early contributions,
1
14
1,532
Y’all my girl’s addicted to pills 😔 she takes them every day but each month she tries to quit for a week and gets super angry someone help
1
2
13
We're 2+ years into the pandemic, why don't we have a clearer understanding of COVID-19 treatment? In our new publication, we use contextualized.ml to explore how common COVID-19 treatments have variable effects that depend on personal risk factors.
tinyurl.com/covid-htes
2
2
13
I'm sure everyone's wondering how @Rubatious_ has been saving the world lately, so I'd like to present her recent news feature:
wpxi.com/news/top-stories/co…
1
1
12
Contextualized ML v0.2.7 -- Linear Encoders, API Reference, and Developer Resources
github.com/cnellington/Conte…
1
3
12
1,853
New lab, new logo, new ideas, come join the party
🚀I’m excited to share that we’re launching OpenContext, a new Slack group growing out of the ContextualizedML project! What started as a simple open-source repo (led by @probablybots) has blossomed into a community of developers and researchers united by an interest in context-adaptive statistical modeling. DM me or @probablybots for an invite to the Slack group.
Our first initiative is an open, collaborative review paper on Context-Adaptive Inference. This paper will offer a perspective on some timely questions in statistical learning like how foundation models can be used as context. We're looking to include insights from across disciplines, so please join us. If you’re interested in contributing or just staying updated, get involved directly on Github:
github.com/LengerichLab/cont…
2
12
1,184
1/n I feel like I often miss out on really great things when they don't make the end-of-year top-10 lists, because every year I rediscover amazing things/art/ideas from the past that I wish I'd found sooner because they make my year fun, make me think more, or inspire me
1
1
12
AI is cool because you can make life-altering systems without ever understanding concepts like 'linear regression' or 'ethics'
1
12
as well as my co-authors @ben_lengerich, @tbkwatkins, @yangjiekun, @abhinadduri, @SazanMahbub, Hanxi Xiao, @manoliskellis, and @ericxing, and collaborators bringing Contextualized modeling into new areas @JannikDe12, Ethan Wu, @SazanMahbub, and @yingtaoluo.
1
11
1,650
Metaverse is cool but it will never replace going to a house party and stealing small irreplaceable objects
12
At @genbioai, it was obvious from day 1 that we couldn’t rely on universal datatypes for benchmarking. We needed new tools to build, test, and productionize the multi-scale and multi-modal biological simulator we’re building.
Hence, AIDO.ModelGenerator
biorxiv.org/content/10.1101/…
1
11
449
We love papers with code. Super scalable mutual information estimator from Gokul Gowri. Lots of fun applications to network inference and other high-dimensional dependency structures in biology.
latentmi.readthedocs.io/en/l…
#MLCB2024
7
342
I wrote a short blog post on my new site 👀 outlining some of my recent work unifying a lesser-known area of statistics that seems extremely useful for biologists and medical researchers.
Would love some help getting this to its target audience
cnellington.github.io/blog/2…
2
10
Incredibly proud to call this my first public paper! By estimating sample-specific causal networks, we reveal dynamic systems that rewire at per-sample resolution. Our experiments show how these networks inform finance, sports, medicine, and single-cell analysis,
1
9
the only valid anti-vax take is that we're currently living the mass vaccination/infection part of I Am Legend and in a month we'll all be zombies
1
9
Contextualized v0.2.3 release:
- Bugfixes to predict_params in the SKLearn-style easy modules
- New kwargs in the easy modules to improve early stopping
pypi.org/project/contextuali…
11
This new ML paradigm, contextualized modeling, understands heterogeneity in real data, adapts to new environments, and is explainable by design.
We apply it to show that 1000s of clinical/molecular factors alter tumor pathology by changing gene regulation on a per-patient basis.
1
10
436
If you're around #ISMB2024, DM me and let's chat about bio LLMs/FMs, generative models, heterogeneous diseases, and next gen medicines. We're actively recruiting scientists, bioinformaticians, and data engineers at all levels for an ambitious project with @ericxing and @dasongle.
Headed to #ISMB2024 in Montreal this weekend, along with AttentionPert lead author Ding Bai. Catch his keynote talk Monday at 3:20pm!
doi.org/10.1093/bioinformati…
2
10
2,150
I spend all day looking for 2 semicolons and then bake something with vinegar instead of buttermilk and it turns out fine, no compiler error or anything
9
Student who comes to the last 5 minutes of office hours to ask if all of their answers are correct
1
8
Talking about dad habits is funny until you catch yourself wrist deep in the olive jar with a glass of milk at 9pm
9
Really enjoyed doing this last year, and appreciated having a moment to pick through all the noise and showcase a few things that stuck with me into 2022. Seeing it now is even better; it feels like a time capsule. So here’s the follow-up for 2022’s best-of.
1/n I feel like I often miss out on really great things when they don't make the end-of-year top-10 lists, because every year I rediscover amazing things/art/ideas from the past that I wish I'd found sooner because they make my year fun, make me think more, or inspire me
1
8
1,689
While accuracy and interpretability are often considered mutually exclusive in modeling, this is incorrect! In our work at ICML now, we show that models can enjoy the best of both by being context-aware and highly personalized (Session 6, 1:30pm CET, icml.cc/virtual/2024/poster/…).
1
4
9
1,390
Contextualized v0.2.2 is released with some major updates!
- Check us out on PyPI
pip install contextualized-ml
- New contextualized models (Correlation & Markov Graphs)
- sklearn style wrappers (import-fit-predict)
- New contextualized.ml/ with docs & tutorials
1
1
9
I saw Dustin from Stranger Things today and let me tell you his character isn't even acting, that's 100% raw unfiltered DUSTIN
8
Contextualized v0.2.6: Consistency Tests and P-values
Contextualization adds flexible modeling components to capture heterogeneity in data, but flexible models can overfit. In this update, we add tools to test when contextual effects are meaningful.
github.com/cnellington/Conte…
3
9
1,504
I'll be honest, I was trying to break chatGPT by asking it about matrix math and it helped me find an mistake in my own work
1
8
@Rubatious_ made me throw out the quesadilla maker, I wonder if our relationship can take this
1
7
Jaynes' 1957 Information Theory and Statistical Mechanics
the entropy perspective on likelihood makes an important connection to physics and explains a bit of why ML is the way it is
bayes.wustl.edu/etj/articles…
8
even if my new manuscript doesn't get accepted at least it got two thumbs up reacts in the group chat
7
Making an educational game for preschoolers called "shapes and colors" where kids earn points by matching new shapes and colors! Users are capped at 5 shapes per day, but they can buy loot crates to keep going and have a 2% chance to unlock legendary shapes like the octagon
2
8
today I will go on social media for 6 hours and think about how stupid other people are while I go on social media for 6 hours
8
The delta airlines site has this cool new recommendation feature where you go to select seats and it recommends you stay in your assigned seat by crashing your whole browser
8
342
