📣 We are excited and thrilled to announce APOLLO, a healthcare system-scale multimodal temporal foundation model for virtual patient representations. Trained on 25 billion clinical events from 7.2 million patients across 33 years and 28 modalities, APOLLO learns a unified atlas of medicine. Turning labs, notes, pathology images, medications, and diagnoses into coherent, computable longitudinal trajectories. APOLLO is disease-agnostic by design, a single model that learns the shared structure underlying human health and disease across every specialty, modality, and stage of care. The possibilities are enormous: earlier risk prediction, treatment response modeling, clinical trial matching, biomarker discovery, and a new generation of agentic systems built on rich patient representations. Read the pre-print: arxiv.org/pdf/2604.18570 Read our blog post about the work: linkedin.com/pulse/apollo-mu… 👏 🎉Huge congratulations to Andrew Zhang , @TongDing99, Sophia J. Wagner, and the rest of the team.

⚡️🔬📣 Excited to share our new @Nature article building and evaluating PathChat, a multimodal generative AI copilot and chatbot for human pathology. Article: nature.com/articles/s41586-0… Open Access Link: rdcu.be/dKC0r We leverage our previous success in building foundation models for computational pathology such as UNI / CONCH and combine it with the advancements of large vision language models and generative AI to enable PathChat to answer diverse pathology-related queries. We assessed PathChat using both multiple choice diagnostic questions and open-ended questions. Congratulations to @MYLu97 @chenbowen118 @DFKW_MD @richardjchen and everyone else who contributed to this work. Also see blog post from @MYLu97 about this work: linkedin.com/pulse/towards-m… , also teasing the development and preview of PathChat 2, a successor to PathChat 1 bringing new capabilities and substantially improved performance to the state-of-the-art.

Excited to share our @nature paper showing AI-driven computational pathology can be used to predict origins for complicated metastatic and unknown primary cancers. #digitalpathology #DeepLearning #AI Link: rdcu.be/cj1Ai @BrighamResearch @broadinstitute @harvardmed 1/2

Excited to share our @Cancer_Cell article presenting a pan-cancer integrative histology-genomic analysis, multimodal integration improves prognostic models, discovers molecular & morphologic correlates of prognosis. bit.ly/3PbnUVM @harvardmed @BWHPath @broadinstitute 1/2

⚡️🔬📣Excited to share our two new @NatureMedicine articles, we develop computational pathology foundation models, 1. UNI, a self-supervised computational pathology model trained on 100 million pathology images from 100k+ slides. 2. CONCH, a vision-language model for computational pathology trained on 1.17 million pathology image-text pairs. Access the articles @NatureMedicine UNI: nature.com/articles/s41591-0… CONCH: nature.com/articles/s41591-0… Access the code, models: UNI: github.com/mahmoodlab/UNI CONCH: github.com/mahmoodlab/CONCH Interesting aspects: - Both models are evaluated on a host of different clinically relevant tasks for WSI classification, ROI classification, segmentation, image retrieval, image-to-text retrieval, text-to-image retrieval, in 0-shot, few-shot and supervised settings. These adaptations encompass the utility of large public datasets and evaluations on independent test cohorts. - Both models exclude commonly used public computational pathology benchmarks from pre-training allowing for a much more holistic evaluation. Some limitations: Both UNI and CONCH represent early developments in foundation models for pathology. More data, and additional evaluation is needed to realize the full potential of these models. Nevertheless, we show the models capabilities on a variety of different benchmarks with several demonstrating state-of-the-art performance. Future work and insights: While these developments are exciting, they represent work we did about a year ago when the pre-prints were made available, since then we have been busy collecting significantly larger datasets and hope to make larger models available in the future. We have also used UNI and CONCH as the backbone for our Pathology specific chatbot, PathChat (arxiv.org/abs/2312.07814), which is further trained on hundreds of thousands of pathology specific Q-A instructions. We are also excited to see foundation models for several other areas of biomedicine including for single cell data (nature.com/articles/s41592-0…), radiology (nature.com/articles/s42256-0…) and the general trajectory towards general purpose AI for biomedicine. Congratulations to our superstar leaders @richardjchen @MYLu97 @DFKW_MD @TongDing99, Bowen Chen and everyone else who contributed to these studies @GuillaumeJaume @GreatAndrew90 @sharifa_sahai @Aparwani_dpath and others.

We are excited to announce PathChat - a vision-language AI assistant for #Pathology that can analyze histology images and answer diverse pathology-related queries. Co-led by our superstars @MYLu97 @chenbowen118 @DFKW_MD Preprint: arxiv.org/abs/2312.07814 Demo below,

I shared my thoughts in a @NatureMedicine comment on the urgent need for more standardized benchmarks in #AI for biomedicine. nature.com/articles/s41591-0…

Excited to share our @Cancer_Cell review on #AI-driven multimodal data integration in #oncology. We discuss methods, and app. of multimodal data fusion, interconnection, assoc. disc. & more. @BWHPath @MGHPathology @harvard_data @broadinstitute Link: bit.ly/3Cshmhc 1/2

⚡️📣After the success of our previous pathology foundation models UNI (rdcu.be/dBMgh) and CONCH (rdcu.be/dBMf6), we are now announcing TITAN (arxiv.org/abs/2411.19666), a new state-of-the-art whole slide level foundation model trained on >330k pathology slides using a diverse set of neoplastic, infectious, and inflammatory cases and corresponding captions synthetically generated via PathChat (nature.com/articles/s41586-0…) our generative AI co-pilot for pathology. Preprint: arxiv.org/pdf/2411.19666 Code: github.com/mahmoodlab/TITAN Download Model: huggingface.co/MahmoodLab/TI… Blog from @TongDing99: lnkd.in/eRdbpjAM Congratulations to our superstar trainees: @TongDing99 @sophiajwagner @GreatAndrew90 @richardjchen

What would you do with 1000+ spatial transcriptomics samples with corresponding H&E-stained whole-slide images? Meet HEST-1k, a collection of 1,108 ST samples assembled from 131 public and internal cohorts encompassing 25 organs, 2 species. HEST-1k includes over 1.5 million expression–morphology pairs. 🔍Explore 3 use-cases for HEST-1k: - HEST-Benchmark: Evaluate gene expression prediction from histology across 10 organs and 9 cancer types, testing multiple foundation models for pathology including UNI, and GigaPath. - HEST for discovery: Explore our proof-of-concept for multimodal biomarker characterization using Xenium breast cancer samples. - HEST for fine-tuning pathology foundation models: See how HEST-1k can enhance foundation models for histology with expression-guided fine-tuning. 📄Preprint: arxiv.org/pdf/2406.16192 👩‍💻 Code and Data access: github.com/mahmoodlab/HEST Congratulations to @guillaumejaume, @pauldoucet, and everyone else who contributed to this work. Huge thanks to everyone who helped curate the dataset. #SpatialTranscriptomics #ComputationalPathology #CancerResearch #Bioinformatics

The in-print version of the PathChat @Nature article is now available online with open access nature.com/articles/s41586-0… learn more about next steps and how PathChat is further being developed at @modella_ai

✨📣Introducing THREADS: a multimodal foundation model for pathology trained on paired histology and genomic data 🔬+🧬 We show that: (a) THREADS achieves SOTA performance on >50 tasks in oncologic pathology with much less pre-training data than other models, highlighting the importance of multimodal foundation models (b) THREADS particularly does better on more difficult tasks such as treatment response prediction highlighting how capturing the molecular landscape underlying morphological patterns is important. See the pre-print and read the blog for additional insights: Preprint: arxiv.org/pdf/2501.16652 Blog post: linkedin.com/pulse/threads-m… Code & Model: Coming soon keep an eye out at github.com/mahmoodlab Congratulations to @anurag_vaidya7 @GuillaumeJaume @aspartate_ai and everyone else who contributed to this work.

⚡️🔬📣Excited to share our new @NatureMedicine article, examining disparities in pathology AI models, assessing how modeling choices impact disparities, and evaluating the potential of self-supervised foundation models in mitigating these disparities. nature.com/articles/s41591-0… See very informative N&V from @2plus2make5 @rajivmovva @PangWeiKoh nature.com/articles/s41591-0… Also see, very exciting parallel work from @s0f1ra @sgowal @alan_karthi @GoogleDeepMind showing the use of synthetic data to reduce disparities nature.com/articles/s41591-0… Congratulations to @anurag_vaidya7 @richardjchen @DFKW_MD @GreatAndrew90 @GuillaumeJaume @tom_hartvigsen @MYLu97 @tiffanyytchen @jana_lipkova and everyone else who contributed to this study.

⚡️📣Today we are tremendously excited to announce @modella_ai the first startup from Mahmood Lab, based on an array of foundation models and generative AI tools including our recent PathChat article in @Nature (nature.com/articles/s41586-0…) @modella_ai will actualize these exciting developments and put them in the hands of pathologists, clinicians, researchers, and trainees. See our announcement below and sign up for the PathChat 2 waitlist at modella.ai Congratulations to the entire team and especially @richardjchen @jill_stefanelli @MYLu97 @kuanchen22 @chenbowen118, Long Le, and everyone else, stay tuned for exciting additional announcements.

So exciting to see that PathChat published in @Nature last summer (nature.com/articles/s41586-0…) has now received FDA Breakthrough Device Designation. Likely one of the first generative AI tools for pathology to receive the designation. We are well on our quest to having a single MLLM answer any question about human pathology! Congratulations to our superstars who worked on this project @richardjchen @MYLu97 @DFKW_

Here's our latest, fast and scalable whole slide image retrieval, WSI search at speeds indep. of rep.size, common & rare disease subtype retrieval, similar morphology retrieval etc. @natBME @harvardmed @broadinstitute @BWHPath @MGHPathology Paper: rdcu.be/cXglV 1/2

Thank you @Cancer_Cell for featuring our AI-driven multimodal data integration study on this month's cover. Thanks to Katie Yost for the beautiful cover illustration. Article: cell.com/cancer-cell/fulltex… @harvardmed @broadinstitute @BWHPath @MGHPathology @harvard_data

⚡️📣Delighted to announce MMP, a prototype-based multimodal framework combining histology and transcriptomics for cancer outcome prediction, to appear in #ICML 2024 @icmlconf. Congratulations to our superstar postdoc @GreatAndrew90 and rest of the team who helped the study. Paper: arxiv.org/pdf/2407.00224 Code: github.com/mahmoodlab/MMP This represents the latest iteration of the multimodal fusion frameworks our lab has investigated since Pathomic Fusion by @richardjchen in 2019. Few interesting facts to know about MMP - Multimodal extension of PANTHER (CVPR 2024), combining morphological prototypes and transcriptomic prototypes (pathways) - Outperforms other multimodal baselines with ~10x less computation - Intuitive prototype-oriented cross-modal interpretability analyses #ComputationalPathology #DigitalPathology #ICML2024 #MultimodalFusion

Based on numerous requests, we are providing the open ShareIT link for UNI and CONCH. Please access it below: Open ShareIT Read Links: UNI: rdcu.be/dBMgh CONCH: rdcu.be/dBMf6 Journal Links for complete pdf: UNI: nature.com/articles/s41591-0… CONCH: nature.com/articles/s41591-0…

⚡️📣👇Tremendously excited to share our new @CellCellPress article, where we develop TriPath, a method for analyzing 3D pathology samples using weakly supervised AI. Article: authors.elsevier.com/a/1j3Ri…. TriPath enables 3D computational pathology via 3D multiple instance learning allowing AI models to capture intricate morphological details from pathology volumes. Code: github.com/mahmoodlab/TriPat… Blog post: linkedin.com/pulse/towards-3… Tested on two different imaging modalities, and patient cohorts from two institutions. Our superstar @GreatAndrew90 put in a monumental effort of leading the study, in a fantastic collaboration with @jonliu123 at @UW . Interesting aspects: - Utilizing the whole tissue volume and leveraging 3D deep learning enable superior risk prediction performance compared to 2D deep learning baselines based on a few sampled tissue sections that emulate standard clinical practice. This indicates TriPath can harness additional information provided by 3D tissue morphology. - The performance is also superior to clinical baselines from a reader study that involved six expert pathologists. - The morphologically heterogeneous tissue volume could lead to opposing patient-level outcome predictions, dependent on which portion of the tissue volume is used. This concurs with current clinical literature warning that tissue sampling bias can lead to misdiagnosis. Some limitations: - While the 3D pathology cohort size is unprecedented, it is smaller than typical 2D pathology cohorts. Further large-scale studies will be required for validation. Nevertheless, we believe that this study will initiate a positive cycle, encouraging academic institutions and pharmaceutical companies to contribute large banks of human tissue blocks with paired clinical outcomes, thus speeding up advancements in 3D computational pathology. Concluding insights: We believe that 3D pathology is just around the corner - It has the huge potential to not only augment/improve the current clinical practice centered around 2D examination of human tissue, but also help reveal novel biomarkers for prognosis and therapeutic response.. @harvardmed @harvard_data @MassGenBrigham @broadinstitute

⚡🎉 We are thrilled to introduce VORTEX, an AI-powered computational framework for predicting 3D Spatial Transcriptomics (ST) using 3D tissue images and minimal 2D ST! 🧬 By combining cutting-edge 3D non-destructive tissue imaging with AI, VORTEX imputes the 3D molecular landscape of large tissue samples in a cost-effective and scalable manner. 🧠💡Our approach: By pretraining on diverse 3D morphology–2D transcriptomic pairs from heterogeneous tissue samples, and then fine-tuning on minimal 2D ST data from a volume of interest, VORTEX leverages both generic tissue-specific and sample-specific morphomolecular correlates to predict 3D ST. Congratulations to our superstar co-leads @Criis_perez99 and @GreatAndrew90, this was an exciting collaboration with @jonliu123 @SizunJ @ABashashati. Preprint: arxiv.org/pdf/2502.17761 Demo: vortex-demo.github.io/ Read the excellent blog from our superstar grad student @Criis_perez99: lnkd.in/eYQzpyPk Also see our previous work on 3D Computational Pathology from @GreatAndrew90 published in Cell last year: lnkd.in/eBvrVf-3 Stay tuned for more to come.

Excited to share our @NatureMedicine article showing deep learning-enabled assessment of cardiac allograft rejection from EMBs. #digitalpathology #DeepLearning #AI @harvardmed @broadinstitute @BWHPath Journal: nature.com/articles/s41591-0… Demo: crane.mahmoodlab.org 1/2

⚡️🔬📣 Here are our two latest preprints on how AI for Pathology can advance pre-clinical drug safety and toxicity assessment. Work led by our superstar postdoc @GuillaumeJaume: Deep Learning-based Modeling for Preclinical Drug Safety Assessment 📄 Preprint: biorxiv.org/content/10.1101/… 🔍 Demo: mahmoodlab.github.io/tox-fou… 🌟 Insights: We trained a Vision Transformer model (TRACE) on H&E-stained whole-slide images from 150+ preclinical toxicity studies. We showed that TRACE can assist and augment pathological assessment for lesion detection, quantification and automatic dose-response characterization. TRACE was also evaluated alongside ten expert pathologists and showed better agreement with the consensus. AI-driven Discovery of Morphomolecular Signatures in Toxicology 📄 Preprint: biorxiv.org/content/10.1101/… 🔍 Demo: mahmoodlab.github.io/tox-dis… 🌟Insights: We developed GEESE, an AI model trained to predict gene expression of 1,500+ targets from histology. We showed that GEESE can reveal molecular signatures associated with distinct morphologies and toxicity mechanisms that are preserved across multiple compounds and species. Congrats to Thomas Peeters, Simone de Brot @GreatAndrew90 and everyone involved! Stay tuned for more coming soon..

Here's our latest - visual language pretrained multiple instance zero-shot transfer for histopathology images accepted at #CVPR2023 led by our superstars @MYLu97 and @bowen_chen_118 Paper: arxiv.org/pdf/2306.07831.pdf Video: piped.video/x8Ch5wsCJRw Code: github.com/mahmoodlab/MI-Zer… If you're at @CVPR check out our paper on Thu Jun/22 10:30a.m. PDT Exhibit halls ABC # 312

Thanks for all the interest and requests, open access link for our Cell @CellCellPress article introducing TriPath a method for weakly supervised AI on 3D pathology samples is available here: authors.elsevier.com/a/1j3Ri… Also see, Code: github.com/mahmoodlab/TriPat… Explainer video: piped.video/watch?v=JQh5FFmc… Blog post from @GreatAndrew90: lnkd.in/dXbUjYuX @Harvard @harvardmed @BrighamResearch @MGH_RI @UW @broadinstitute

Thrilled to share our new @natBME review article discussing algorithmic #fairness in #AI for medicine and healthcare. We discuss sources of algorithmic biases in healthcare, and emerging methods for mitigating biases. Led by our superstar grad student @richardjchen Journal Link: nature.com/articles/s41551-0…

⚡️🔬📣 We are excited to announce our new #ECCV 2024 @eccvconf paper "Multistain Pretraining for Slide Representation Learning in Pathology" Led by @GuillaumeJaume & @anurag_vaidya7 this work is the latest iteration of our efforts on whole slide representation learning for pathology our lab started with HIPT (CVPR’22) and recently Tangle (CVPR’24). We pretrain new models using cross-stain alignment between H&E slides and widely deployed immunohistochemistry and special stains slides. 📄 Paper: arxiv.org/abs/2408.02859 🔍 Code: github.com/mahmoodlab/MADELE… #ComputationalPathology #DigitalPathology

Today we are tremendously excited to announce Judith, a generative AI powered agent for continuous biomedical discovery.

👉📣Introducing KRONOS, a panel-agnostic foundation model purpose-built for multiplex spatial proteomics! Access the pre-print here: arxiv.org/pdf/2506.03373 Trained self-supervised on 47 million single-marker patches spanning 175 protein markers, 16 tissue types, 8 imaging platforms and 5 institutions. Its architecture couples a shared channel-wise stem with sinusoidal marker-identity embeddings, making it natively compatible with high-dimensional multiplex data. 🔬 Biology at scale: label-efficient cell phenotyping, unsupervised tissue phenotyping, tissue/region classification, artefact detection, patient stratification, and spatial biomarker discovery. 🔎 Spatial search engine: drop in any cell, patch, or region of interest and KRONOS retrieves morphologically or immunologically similar areas across multi-cohort databases—supporting both exploratory and hypothesis-driven queries. 💻 Code, pretrained models & tutorials: github.com/mahmoodlab/KRONOS Exciting collaboration with my buddy @SizunJ's lab. Congratulations to @mshaban_ai, Yuzhou Chang and all the other co-authors. Read the blog post from Sizun: lnkd.in/dwrZpYkP Read the blog post from Shaban: lnkd.in/dwBsnX7u

⚡️Announcing UNI 2: Over the past nine months its been humbling to see how UNI (rdcu.be/dBMgh) and CONCH (rdcu.be/dBMf6) our two foundation models for computational pathology have been downloaded >1 Million times and used in >400 studies. Today we are excited to release UNI 2, a new state-of-the-art foundation model trained on over 200 million pathology H&E and IHC images deduplicated for diversity from >350k diverse whole slide images across neoplastic, infectious and inflammatory disease. Benchmarks and download models: github.com/mahmoodlab/UNI UNI Article @NatureMedicine: rdcu.be/dBMgh Blog: linkedin.com/pulse/announcin… Also, see our recent announcement on TITAN (arxiv.org/pdf/2411.19666), our multimodal slide level foundation model (github.com/mahmoodlab/TITAN). Congratulations to @TongDing99 @MYLu97 @richardjchen

Can pathology image + gene expression improve whole slide-level representation learning? Learn more about our work on TANGLE at our superstar postdoc @GuillaumeJaume's oral talk at @CVPR #CVPR2024 Oral Session: Orals 3C Medical and Physics-based vision Time: Thu, 20 Jun, 9:54 - 10:12 a.m. PDT Location: Summit Flex Hall C Poster Session: Time: Thu, 20 Jun, 10:30 a.m. PDT — noon PDT Location: Arch 4A-E, Poster #175 Paper: arxiv.org/pdf/2405.11618 Code: github.com/mahmoodlab/TANGLE @anurag_vaidya7 @richardjchen @DFKW_MD @GreatAndrew90 .

Congratulations 🎉 to Dr. @richardjchen for completing his PhD @HarvardDBMI, and special thanks to everyone on the committee @VanAllenLab @marinkazitnik @njdurr @alexander_baras Some key work from Richard's PhD includes Pathomic Fusion (IEEE TMI, 2020), MCAT (ICCV, 2021), HIPT (CVPR, 2022), Porpoise (Cancer Cell, 2022), UNI.

Its been a month since we released UNI and CONCH, a vision only and a vision-language foundation model for computational pathology w/ @NatureMedicine articles. The models have been downloaded over 233k times on @HuggingFace we are analyzing the early impact, see where people are using the models. The significant usage also underscores the breadth of utility such models offer and the growing interest in computational pathology.. Articles UNI: rdcu.be/dBMgh CONCH: rdcu.be/dBMf6 Code and models UNI: github.com/mahmoodlab/UNI CONCH: github.com/mahmoodlab/CONCH

⚡️🔬📣Thrilled and excited to share that we will be presenting three articles at @CVPR 2024 related to whole slide level representation learning, multimodal contrastive learning and multimodal fusion. #CVPR2024 #ComputationalPathology 1. TANGLE: Transcriptomics-guided Slide Representation Learning in Computational Pathology (Oral) Paper: arxiv.org/pdf/2405.11618v1 Code: github.com/mahmoodlab/Tangle 2. PANTHER: Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology Paper: arxiv.org/pdf/2405.11643 Code: github.com/mahmoodlab/Panthe… 3. SurvPath: Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Prediction Paper: arxiv.org/pdf/2304.06819 Code: github.com/mahmoodlab/SurvPa… Also, check out our keynote at the Workshop on Foundation Models in Medical Vision (fmv-cvpr24workshop.github.io…), Demos for TriPath (authors.elsevier.com/a/1j3Ri…) and PathChat, and several posters at the CV4Science workshops (sites.google.com/nyu.edu/com…). See you at #CVPR2024 @GreatAndrew90 @GuillaumeJaume @richardjchen @MYLu97 @TongDing99

Excited to report that our widely used data-efficient weakly supervised computational pathology method CLAM is now out in print at Nature BME @natBME nature.com/articles/s41551-0… Code: github.com/mahmoodlab/CLAM Demo: clam.mahmoodlab.org/ @BrighamResearch @broadinstitute

Based on numerous requests we are providing the open access ShareIT link for our @NatureMedicine article on identifying and mitigating disparities in pathology AI models. Open read link: rdcu.be/dFdMS Journal link: nature.com/articles/s41591-0…

We have six open postdoctoral positions, if you are interested in working at the interface of machine learning + health, and have a background in computer vision, machine learning or medical image analysis send us an application with you CV, statement of research interests and contact information for three references. If you're interested in meeting current postdocs and students several will be at NeurIPS this week including @MYLu97 @GreatAndrew90 @GuillaumeJaume #ml4h

We’re building multimodal all-of-patient foundation models and AI Agents that integrate the patients entire record w temporal alignment to identify cancer resistance traits, predict treatment resp, and discover new biomarkers as part of the ADAPT program. arpa-h.gov/news-and-events/a…

For the past 5 years, our CLAM toolbox has been widely used by the community for whole-slide image (WSI) processing. For scalability, today we are excited to announce the release of two major internally developed open libraries for large scale batch processing of pathology WSIs and foundation model benchmarking: Trident (github.com/mahmoodlab/triden…) and Patho-Bench (github.com/mahmoodlab/patho-…). Read the excellent blog from our superstar grad student Andrew Zhang @aspartate_ai explaining why we built these tools and how you can benefit and contribute: lnkd.in/e5HUMW3U Trident includes, - Computationally efficient tools for handling very large scale batch processing of WSI maximizing GPU and CPU compute. - Support for all SOTA patch-level and slide level foundation models, including latest foundation models from our group UNI 2 (github.com/mahmoodlab/UNI) and TITAN (github.com/mahmoodlab/TITAN). Patho-Bench includes, - A large benchmark to standardize evaluation of pathology FMs curated from publicly available data, featuring 42 clinically relevant tasks (including disease subtyping, grading, treatment response, survival/outcome prediction, IHC scoring, biomarker and molecular alteration prediction). - Experiment handlers for running all combination of modeling choices and tasks including linear probing, survival, etc. - The ability to run hundreds of experiments using a single command, with automatic GPU load-balancing, experiment monitoring, and results collection. Preprint: arxiv.org/abs/2502.06750 Congrats to @aspartate_ai, @GuillaumeJaume , @anurag_vaidya7, and the rest of the team! Stay tuned for more to come soon.

If you're at #ICML2024 @icmlconf come check out our work on multimodal prototypes and say hi to our superstar @GreatAndrew90 Paper: arxiv.org/pdf/2407.00224 Poster #202: Multimodal Prototyping for cancer survival prediction When/Where: 7/24 (Wed) 11:30AM ~ 1PM CEST Hall C 4-9

Our work on #pathology foundation models highlighted by @harvardmed news. hms.harvard.edu/news/researc…

We are celebrating the lab's fifth anniversary this week but more importantly we are celebrating the careers of over 50 trainees, graduate students, postdocs who worked in the lab during this time and contributed to each others success.

Mahmood Lab dinner at #CVPR We are hiring at all levels if you’re interested come see us at one of our talks, posters or demos.

Mahmood Lab will be presenting three articles (TANGLE - oral, Panther, and SurvPath) at the @CVPR #CVPR2024 main conference, two demos (PathChat and TriPath) and talks at the workshop on foundation models for medical vision, and computer vision for science. We are also recruiting at all levels come talk to us about opportunities. @GreatAndrew90 @richardjchen @MYLu97 @GuillaumeJaume @TongDing99 will be there.

This article (arxiv.org/pdf/2408.15823) from @jnkath is the most comprehensive effort to compare comp path foundation models to date. Showing that multimodal FMs like CONCH outperform unimodal FMs on unimodal tasks, and that the diversity of data >> quantity of data.

Thanks for all the interest in our PathChat @Nature article (nature.com/articles/s41586-0…), if you are at #CVPR and would like to try out PathChat please visit our demo later this week, Date, time: Thu 20 Jun 10:30 a.m. PDT — 6:45 p.m. PDT Location: Arch 4CDE, Demo #8

One year since releasing UNI and CONCH, our computational pathology models have 1.4M+ downloads, have been used by hundreds of studies, and unlocked unique capabilities. Update here: linkedin.com/pulse/one-year-…

Big day for the Mahmood Lab at @CVPR #CVPR2024 thanks for all the interest. @GreatAndrew90 @GuillaumeJaume @MYLu97 @richardjchen @TongDing99 @anurag_vaidya7

Are you interested in working at the interface of machine learning and pathology? We're hiring postdocs, software engineers etc. See examples of our recent work here: bit.ly/3QwWg7o More at: mahmoodlab.org @BWHPath @MGHPathology @broadinstitute @harvard_data

Our @TheLancet perspective w/ @EricTopol highlights key findings from our recent @NatureMedicine article about AI-driven heart transplant rejection assessment. thelancet.com/journals/lance…

Thrilled to announce CONCH, we build on our CVPR 2023 MI-Zero study by training a vision-language model for computational pathology with a significantly larger dataset (1.17 M histology image-text pairs) and adapt it to several downstream zero-shot and few shot tasks. Read more: arxiv.org/abs/2307.12914

Congratulations to our superstar Max Lu @MYLu97 on defending his PhD @MITEECS Max was one of the first students to join the lab, key work from his PhD includes CLAM (Nature BME, 2021); TOAD (Nature, 2021); CONCH (Nature Medicine, 2024); MI-Zero (CVPR, 2023) and most recently PathChat (Nature, 2024). Very proud of everything you accomplished and excited to see what you do at @modella_ai

Thrilled to announce that we just received the NoA for our @NIGMS R35 MIRA, a five year $2.2M grant to support our work on #computationalpathology I am so grateful to everyone in our lab and their hard work to help make this happen.

News feature from @NatureBiotech highlights our work on 3D Spatial Transcriptomics - VORTEX, read our pre-print here: arxiv.org/pdf/2502.17761 nature.com/articles/s41587-0…

Happy holidays from the Mahmood Lab!

Very exciting work from our superstar postdoc @GreatAndrew90 , weakly-supervised deep learning for 3D Pathology samples. Very interesting collaboration with @jonliu123, @alexander_baras and @Aparwani_dpath. Pre-print: arxiv.org/pdf/2307.14907.pdf Code: github.com/mahmoodlab/mamba Demo: mamba-demo.github.io/demo/

Great 🧵about 🧵THREADS, our new multimodal histology + genomic foundation model from @simocristea

Thanks for all the interest in our recent Cell article introducing TriPath, our deep learning package for 3D Computational Pathology (cell.com/cell/fulltext/S0092…) If you are at #CVPR and would like to try out TriPath in action please visit our demo tomorrow, Thu 20 Jun 10:30 a.m. - 6:45 p.m. PDT Location: Arch 4CDE, Demo #16 @CellCellPress @CVPR @GreatAndrew90 .

Excited to see our work highlighted by @Meta @AIatMeta! Including UNI (nature.com/articles/s41591-0…) and PathChat (nature.com/articles/s41586-0…). ai.meta.com/blog/mahmood-lab…

Always grateful for my academic family.

Here's our latest - scaling vision transformers to gigapixel images via hierarchical self-supervised training accepted at #CVPR2022 (oral) from our superstar grad student @richardjchen Paper: arxiv.org/pdf/2206.02647.pdf Code: github.com/mahmoodlab/HIPT Oral: piped.video/watch?v=cABkB1J-…

Thanks for the overwhelming response, for inviting me to speak and for organizing such a nice workshop. #CVPR

Our work on Pathomic Fusion for integrating histology and genomics for diagnosis, prognosis with multimodal interpretability published at IEEE TMI. Paper: ieeexplore.ieee.org/document… Preprint: arxiv.org/abs/1912.08937 Code: github.com/mahmoodlab/Pathom… Talk: piped.video/TrjGEUVX5YE

Always reminded that how lucky I am to have such an incredibly talented and wonderful group of people in my lab!

📣 Excited to share our new ICML 2025 Spotlight article, “Do Multiple Instance Learning Models Transfer?” – addressing a foundational question for building robust and generalizable MIL models. Read the article: arxiv.org/pdf/2506.09022 👉Enhanced Performance & Robustness: Pretrained MIL models consistently lead to improved performance even when the pre-training data comes from a different organ, site, disease model than the target task. 👉Aggregation Transfers: Transfer gains come from the MIL aggregation module, not just patch encoders. Resetting attention layers drops performance by 5–8%, showing they encode generalizable pooling logic. 👉Pancancer Generalization: Models pretrained on a more diverse and challenging data (e.g. 108-class pancancer classification task) achieve the stronger overall transfer performance. 👉Robust benefits across patch encoders: Benefits from MIL transfer are consistent across a wide range of patch encoders, from out-of-domain encoders such as ResNet50 pretrained on natural images, to in-domain encoders including Gigapath and UNIv2. This research highlights supervised pretraining as a highly accessible path to generalizable MIL models, offering a data and compute-efficient route for developing slide level encoders with flexible combination of MIL method and patch encoder. Congratulations to @Daniel__Shao @GreatAndrew90 @richardjchen and everyone else who contributed. Stay tuned for an array of pre-trained MIL models ready to transfer to any task! Visit us at @icmlconf.

Happy holidays from the Mahmood lab family 🎉🎊. This year is extra sweet, we are celebrating four of our superstar postdocs starting faculty jobs and their own research labs @jana_lipkova (UC Irvine) : @DFKW_MD (Emory University) ; @iaincar9 (UNC Chapel Hill) and @DrFabLucas (UW)

Heading to #NeurIPS? We'll be presenting our HEST article for jointly studying spatial transcriptomics and histology. @GuillaumeJaume Main conference poster session: 📅 Wed 11 Dec 11 a.m. PST — 2 p.m. PST 📍 East Exhibit Hall A-C #1104 👩‍💻 Access the HEST-Library and HEST-1k dataset on GitHub: github.com/mahmoodlab/hest 📄 Article: arxiv.org/pdf/2406.16192 🚀 HEST-1k Dataset includes 1,229 paired spatial transcriptomics samples each paired with a whole slide pathology image, assembled from 153 public and internal cohorts. 🛠️ HEST-Library includes batch effect handling, whole-slide registration, spatial-data integration, etc. ⚙️ HEST-Benchmark, a new patch level benchmark for predicting gene expression from histology, our foundation model evaluation features 11 patch encoders, including H-Optimus-0, Virchow2, and an early look at UNIv1.5 (more on this soon!).

Does context play a role in #computationalpathology prognostic models? See our N&V (go.nature.com/3dsVXeX) about TEA-graph, a very interesting GCN-based method from Sunghoon Kwon and colleagues (see their article: go.nature.com/3psK0IZ). @GreatAndrew90 @GuillaumeJaume

Our comment on the exciting potential of AstroPath published in @Cancer_Cell - Multiplex computational pathology for treatment response prediction @MYLu97 @HSaterMD Read link: bit.ly/3s7zEPd Journal: cell.com/cancer-cell/fulltex…

Always grateful for my academic family!

Our new @natBME short paper demonstrates utility and identifies key challenges associated with using #synthetic data in #MachineLearning and #AI for medicine. Read link: rdcu.be/cmAfk Journal link: go.nature.com/3wtgSDr @richarizardd95

Exciting and impressive work from Gabriele Campanella et al. developing much needed benchmarks for pathology foundation models on clinically relevent diagnostic and biomarker prediction tasks. paper: arxiv.org/pdf/2407.06508v1 Very comprehensive analysis enhancing our understanding around scaling laws for path foundation models showing data diversity is much more imp than data quantaty, excited to see how UNI trained on just 100M path patches using diverse 100k WSIs spanning neoplastic, infectious, inflamatory diseases and normal tissue fairs against models trained on much larger and often oncology-only datasets. Hopefully we will see many more benchmark studies further analyzing scaling laws around data and model size for comp path.

🎉🎊Happy Holidays from our academic family to yours! We’re celebrating an incredible year. Thank you to our amazing team, collaborators, and supporters who made this year so productive and inspiring. Stay tuned for what we have in store next year—we can’t wait to share the exciting developments ahead! 🥂

Thanks to @taryn_plumb for covering our PathChat @Nature article (nature.com/articles/s41586-0…) in @VentureBeat - New medical LLM, PathChat, can talk to pathologists about tumors, offer diagnoses venturebeat.com/ai/new-medic…

Congratulations 🎉 to Dr. @sharifa_sahai for completing her PhD @HMS_SysBio, and special thanks to everyone on the committee @deboramarks @nmrajpoot @GeorgKGerber1.

Excited to share our new preprint - Deep Learning-based Computational Pathology Predicts Origins for Cancers of Unknown Primary. arXiv: arxiv.org/abs/2006.13932 Demo: toad.mahmoodlab.org Origin prediction without IHC! #ComputationalPathology #DigitalPathology #CUP .. 1/n

Exciting work from our superstar graduate students @richardjchen @MYLu97 and @DFKW_MD A general purpose self-supervised model for Computational Pathology.

We have several open #computationalpathology positions at @harvardmed @broadinstitute @BWHPath for postdocs, research associates, software engineers etc. Consider joining our team: mahmoodlab.org

Thanks to @brittanytrang for covering our PathChat @Nature article (nature.com/articles/s41586-0…) in @statnews - AI for biopsies can analyze microscopy images, talk to pathologists about your tumor statnews.com/2024/06/12/path… @Harvard @harvardmed @BrighamWomens

We had an exciting time at #DiscoverBrigham with 8 posters and @DFKW_MD @iaincar9 & Bowen Chen receiving the Excellence in Research Awards. @BrighamResearch @BWHPath

Are you interested in working at the interface of computer vision, artificial intelligence, and pathology? Our group is looking for postdoctoral fellows (limited to recent graduates), and research software engineers. See more at mahmoodlab.org

It was a pleasure speaking with @AndrewLBeam and @arjunmanrai @NEJM_AI

Its been a crazy year, it was great to see the team together in person again! @MYLu97 @richarizardd95 @schaumberg_a @DFKW_MD @tiffanyytchen @sharifa_sahai @kuanchen22

Lab celebration of all the hard work these amazing people put into their research. As always, reminded how incredibly lucky I am to be working with such amazing colleagues.

DOD is Digitizing the World's Largest Pathological Sample Repository governmentciomedia.com/dod-d…

From our superstars @tiffanyytchen and @jana_lipkova - Two Backgrounds, One Purpose bwhclinicalandresearchnews.o…

Excited to share our recent preprint - Federated Learning for Computational Pathology on Gigapixel Whole Slide Images #federatedlearning for #computationalpathology arxiv.org/pdf/2009.10190.pdf 1/2

Looking forward to seeing eveyone at #CVPR2025 this week.

Are you interested in working at the interface of machine learning and pathology? Consider joining @DFKW_MD's lab at Emory.

Thank you @pathologistmag for including me in the pathology #PowerList 2022.

Wrapping up #NeurIPS2024 w/ @GreatAndrew90 @MYLu97 @GuillaumeJaume

Are you interested in working at the interface of machine learning and pathology? We're hiring postdocs, software engineers etc. More at: mahmoodlab.org @BWHPath @harvardmed

Exciting collaboration with @SizunJ out in print @NatureComms Article: nature.com/articles/s41467-0… Code: github.com/mahmoodlab/MAPS

📣 If you're at @icmlconf ICML 2025, our superstar graduate student @Daniel__Shao will be presenting our work on transferability of MIL models and supervised foundation models for computational pathology. Poster session: 11 am - 1:30 pm PST at West Exhibition Hall B2-B3 #W-314 Read the article: arxiv.org/pdf/2506.09022 Code and models: github.com/mahmoodlab/MIL-La…

This week we are reaching far and wide with @richardjchen @anurag_vaidya7 and I at the #AACR Cancer Health Disparities conference in Philly; @GreatAndrew90 @iaincar9 at #MICCAI22 in Singapore and @sharifa_sahai at the #Banff @cst_transplant conference! #DigitalPathology

Join us for the @NIH_CommonFund PRIMED-AI workshop next week.

We are delighted to be hosting Prof. @nmrajpoot of @TIAwarwick for @BrighamWomens Pathology #grandrounds today!

Interesting 🧵from @MYLu97 @richardjchen showcasing how the community is using our pathology foundation models UNI (nature.com/articles/s41591-0…) and CONCH (nature.com/articles/s41591-0…).

Happy 4th of July 🇺🇸 Celebrating our ideals of freedom, unity & opportunity that bind us all. To our service members, veterans & first responders who safeguard our freedoms, thank you for your dedication. Wishing everyone a safe, joyful Independence Day! #FourthOfJuly