Student Researcher @GoogleDeepMind PhD-ing @WisconsinCS | MS at @uwcse/@RAIVNLab | Ex @AIatMeta @nvidiaai and @samsungresearch. He/Him
Madison, WI
- Tweets 1,175
- Following 458
- Followers 1,056
- Likes 8,180
I'm going to be research interning at @Meta @RealityLabs in Redmond this summer!!! 😭❤️🏔️
7
300
19,069
With all the recent hype around 🪆Matryoshka Representation Learning 🪆(Thanks @OpenAI !), I finally put my longstanding plan of writing a detailed blog about MRL to action
aniketrege.github.io/blog/20…
This blog is NOT a paper walkthrough (see @RitvikRastogi19 for that!)
(1/7)
6
24
171
40,987
Really clean + simple video walking through 🪆MRL🪆
(incoming shameless self plug)
For some more background into representation learning and why we created MRL, checkout my blog from earlier this year!
aniketrege.github.io/blog/20…
Matryoshka Representation Learning (MRL) is a super exciting approach to improving the quality and efficiency of embedding models and strategies ✨
MRL allows models to store more information in the earlier dimensions of their data vectors. This method not only boosts performance in tasks like classification and retrieval, but is also a super cool compression technique!
Paper: arxiv.org/pdf/2205.13147
For compression: weaviate.io/blog/openais-mat…
It’s been so much fun learning and making this video with @DanielW966, thanks for all your help!
1
17
111
12,418
Happy and proud to see our MRL work (arxiv.org/abs/2205.13147, NeurIPS '22) recognized and used by @OpenAI to power their latest flexible embedding models!
Blog post: openai.com/blog/new-embeddin…
One of those great grad student days :')
1
2
53
5,086
Super happy to share that I'm joining the @UWMadison CSE #PhD program this Fall, focusing on #MachineLearning and #computervision research. Huge thanks to all my collaborators and mentors (esp. @adityakusupati) for all their help and guidance through this super chaotic process!
5
2
56
3,292
Got this *bangin* gongura ramen from salt and pepper story (ft @darthdevi) today!
Sooo delish 🤤 10/10 would order again
Doing an Andhrasian popup delivery lunch with Chef Puja from The Salt&Pepper Story this weekend! We have gone mad and this is the result 😂♥️
Orders can be placed for the dish of your choice up till this Thursday, 27th June 2024. Deliveries will happen on 29th June 2024, this Saturday.
Contact Misha on +91 98452 07778 to place orders or simply DM @/thesaltandpepperstory on IG :)
You can also tweet here and we’ll be in touch.
1
2
42
4,242
Video of this talk is up on YouTube!
piped.video/watch?v=IbfdvzPw…
I walk through our Matryoshka Representation Learning 🪆 (MRL) and Adaptive Web-Scale Semantic Search 💃(AdANNS) works
#ml twitter pls feel free to share your thoughts, always happy to chat! 😄
Hi Twitter! I'm giving a talk on the recently popular Matryoshka Representation Learning and followup web-scale semantic search work at @UWMadisonECE MLOPT Seminar
mlopt.ece.wisc.edu/idea-semi…
Time: April 26, 2024, 12:30 PM – 1:30 PM CT
Everyone is welcome to join on Zoom! :D
1/n
2
5
39
6,747
🚨 Super stoked that my joint first-author work on adaptive semantic nearest neighbors search (AdANNS) is accepted at #NeurIPS2023 !!
Apply to your #LLM of choice for vector search today 😁
Paper: arxiv.org/abs/2305.19435
Code: github.com/RAIVNLab/AdANNS/
Introducing💃AdANNS: A Framework for Adaptive Semantic Search🕺
TL;DR: Up to 90× faster nearest neighbor retrieval and 2× lower memory cost for web-scale search.
Applies to vector search at scale & improves all "retrieval" augmented models!
arxiv.org/abs/2305.19435
[1/8]
2
1
42
3,611
Hi Twitter! I'm giving a talk on the recently popular Matryoshka Representation Learning and followup web-scale semantic search work at @UWMadisonECE MLOPT Seminar
mlopt.ece.wisc.edu/idea-semi…
Time: April 26, 2024, 12:30 PM – 1:30 PM CT
Everyone is welcome to join on Zoom! :D
1/n
3
6
36
13,049
Well done to the states of Alabama, Georgia, Kentucky, Missouri, and Ohio who have somehow managed to build a time machine to the 1920s: old white men deciding what rights women have over their own bodies #abortionisahumanright
2
12
32
To appear at #iclr 2025!!
I'm excited to talk about how to build generative models that are more representative of the diverse, heterogeneous human values present across global cultures - happy to chat!
Happy to introduce 🤝PAL🤝 , a simple framework for Pluralistic ALignment
Tl;dr: simple and cheap alignment to diverse human preferences for LLMs and text-to-image models
Project Page: pal-alignment.github.io/
Paper: arxiv.org/abs/2406.08469
(1/12)
2
2
31
2,345
I LOVE Studio Ghibli films and consider their art and Miyazaki's vision timeless, haunting, thought-provoking, and deeply, uniquely human.
I'm also a PhD student working on text-to-image models.
Join me on a short philosophical journey as I break down my inner conflict
🧵 1/n
5
5
32
4,634
Stop by CVPR poster #330 in Hall D for a chat about heterogenous and culturally representative generative models!
Training text-to-image models?
Want your models to represent cultures across the globe but don't know how to systematically evaluate them?
Introducing ⚕️CuRe⚕️ a new benchmark and scoring suite for cultural representativeness through the lens of information gain
(1/10)
3
30
1,155
🪆MRL blog updates🪆 because of all the excellent recent work from the community!
Blog: aniketrege.github.io/blog/20…
1. Why does MRL work? primer on nested vector subspaces ft. @ZainHasan6 and @nomic_ai
2. Adaptive Retrieval ft @ujjwalm29 , @xenovacom, and @supabase
(1/n)
1
4
28
3,149
Training text-to-image models?
Want your models to represent cultures across the globe but don't know how to systematically evaluate them?
Introducing ⚕️CuRe⚕️ a new benchmark and scoring suite for cultural representativeness through the lens of information gain
(1/10)
1
8
32
10,305
For anyone who is interested in 🪆 Matryoshka embeddings 🪆, I'll be presenting our follow-up work on adaptive semantic search (NeurIPS '23) next week at the @_superAGI Leap Summit! 😄
Paper: arxiv.org/abs/2305.19435
Teaser from my blog: aniketrege.github.io/blog/20…
Meet @wregss from @WisconsinCS at the AGI Leap Summit'24, where he'll be presenting his research on "AdANNS: A Framework for Adaptive Semantic Search"
Register now: superagi.com/agi-leap-summit…
#AGI #LeapSummit
1
21
2,551
Had a fantastic time at #CVPR2025 and my spotlight talk on culturally representative T2I models at the DemoDiv workshop was definitely the highlight
Thanks @polkirichenko and all the organizers, attendees and panelists for a super engaging & thought provoking workshop!
🚨 I'll be giving a short contributed talk about CuRe at CVPR 's DemoDiv workshop today (06/11) at 10 AM!
Please drop by, I'd love to chat 😁
1
1
22
1,722
Omgggg I got another TA'ship for the winter quarter at @UW 😭😭😭
Idk who is blessing me with all this luck but thank you I can continue to recklessly drop $75 each time I hit up Trader Joe's for another 3 months
1
21
So exciting to see my joint first-author work (#MRL🪆 & AdANNS -arxiv.org/abs/2205.13147, TBD) recognized as important algorithms research in #ML efficiency and search by @GoogleAI
Talked about in the last paragraph!
@adityakusupati @BhattGantavya @RAIVNLab @uwcse
As #DeepLearning models become more widely used, it is increasingly important that they be both robust and efficient. Today we summarize some of our many efforts to improve #ML efficiency through algorithms research. → goo.gle/3I6asCj
1
22
2,674
Presenting our Matryoshka Representation Learning work with @adityakusupati @adityaasinha at @NeurIPSConf Hall J #640 starting in in 15 minutes! Missing @BhattGantavya :’( #NeurIPS2022
19
In the months of May and June every year, the blood of all Maharashtrians is replaced with aamras #marathi
17
Hey twitter fam! I co-authored this work with @adityakusupati and @BhattGantavya in collab with @uwcse/@RAIVNLab and @GoogleAI!
[1/2]
Introducting🪆Matryoshka Representations for Adaptive Deployment🪆
TL;DR: up to 14× lower real-world classification & retrival costs at web-scale at no loss in accuracy & w/o any overhead across setups.
Paper: arxiv.org/abs/2205.13147
Code: github.com/RAIVNLab/MRL
[1/11]
1
3
16
C'mon @zomatoin If you're going to send me these notifications at least do it properly, "Bone Appetit" was riiight there!
1
15
Happy to introduce 🤝PAL🤝 , a simple framework for Pluralistic ALignment
Tl;dr: simple and cheap alignment to diverse human preferences for LLMs and text-to-image models
Project Page: pal-alignment.github.io/
Paper: arxiv.org/abs/2406.08469
(1/12)
1
2
14
2,977
Super cool use-case of MRL by @vespaengine - shortlist with small embeddings and rerank with large embeddings!
MRL means the small embedding is just a slice of the large one - no need for any bells and whistles!
For people who read in code:
x_small = x_large [:k]
I’m excited to share this comprehensive exploration of OpenAI’s Matryoshka 🪆embeddings and how to make the most out of them with Vespa by @andreer!
It's great to see another use case for Vespa’s phased retrieval and ranking support.
blog.vespa.ai/matryoshka-emb…
1
16
3,234
I and @BhattGantavya will be presenting and talking about MRL tomorrow with @latentspacepod 😁
Register if you'd like to discuss + chat flexible representation learning / vector search!
We r so back lol - pretty phenomenal lineup this week for the @latentspacepod discord's paper club
We've got @wregss and @BhattGantavya presenting on their paper Matryoshka Representation Learning which Open AI uses under the hood for their new embedding model
We've got @aimuggle presenting on representation learning and how to control LLMs with just feature vectors
4
15
12,744
It's been one week of grad school and man has it kicked my ass so far! Feels super weird to be doing homework (how old am I again) but alsoooo overall it's been super nice? Idk, thoughts
3
14
Yet another piece of the puzzle 🧩
Simple and elegant follow up to our previous MRL🪆 work - 100s of transformers for free with mix n match routing for truly elastic deployment!
Shameless plug: for more on elastic search, also check out our work AdANNS💃 at NeurIPS '23
Announcing MatFormer - a nested🪆(Matryoshka) Transformer that offers elasticity across deployment constraints.
MatFormer is an architecture that lets us use 100s of accurate smaller models that we never actually trained for!
arxiv.org/abs/2310.07707 1/9
2
14
2,947
Just realised I've been listening to my ipad in this flight with ear plugs+speakers instead of airpods for the last 45 minutes... hope everyone enjoys jujutsu kaisen
13
Drop by our #ICLR2025 poster to hear about compute-optimal alignment to diverse, pluralistic preferences with both LLMs and T2I models!
Want to align LLMs and text-to-image models to heterogeneous preferences?
Check out our #ICLR2025 paper on learning personalizable rewards (PAL) in a sample efficient way.
iclr.cc/virtual/2025/poster/…
When: Friday Apr 25th from 10 am
Where: Hall 3 + Hall 2B #196, Poster Session 3
14
6,115
(1/n) Inspired by @abolijoshi and my own boredom, here are some of my culinary experiments during this period of self-isolation.
First up, this classic cheesecake (Tasty) generously topped with sea salt caramel! [Mar 14]
If interested, pls suggest things you'd like to see!
2
14
More visualizations helping me understand my own paper 🤪 amazing work @ZainHasan6 !
When creating MRL, we thought about the marginal utility of increasing dims (e.g. what really changes if we jump from 64-d to 128-d? How much additional information do we get?)
(1/n)
How does the underlying data representation change as you use more and more dimensions of a Matryoshka embedding?
Below every frame is a 3d vector space that was generated using PCA on up to only a certain number of MRL vector dimensions.
Took 10k random samples from DBpedia stored in @weaviate and embedded using the openai text-embed-3-large model and then reduced them down to 2d space using PCA; started off using only 4-dims all the way up to using all 3072-dims.
You can see that by the time we get to using 512 dims, the data structure is already defined, after this dims 513+ are mainly used to tighten data representation inside the larger structure.
By the time we get to dims 2000+ we only see small jittering of the data points, probably because the contributions of the incremental dimensions are negligible and can't be seen in the first 3 principal components.
2
2
14
1,627
Put a batch of brownies in a preheated oven, and the mains went just as I turned the knob to 30 min and I tripped the entire house. hows your day going
1
11
Today I made this bomb ass beef bulgogi and I surprised myself by taking a feels trip back to my 2019 trip 😭
맛있게 드세요
mas-issge deuseyo!
And also mashed potatoes because why not
11
My personal favorite from this year so far:
The rebuttal has addressed most of my concerns, but I acknowledge other reviewers concerns and maintain my score 🫠
2
12
925
Hello to all I made this Tres Leches with dulce de Leche and fresh berries and it tasted a lot better than it looked which was still pretty good imo that'll be all have a nice day
12
Returning to Bangalore after 10 days in Paris, Helsinki, Tallinn and London, the trip hangover is reaaaal. But BJP lost in Delhi so I'M HAPPY huehuehue
1
11
You can now use MRL embeddings🪆in Sentence Transformers ft @tomaarsen , @osanseviero and @xenovacom !!
This is a really nice blog walking through how to use the flexibility of Matryoshka embeddings for similarity search 😁
More info here: aniketrege.github.io/blog/20…
🔥Sentence Transformers v2.4.0 is released! It introduces Matryoshka Embedding models (training & inference), 2 new state-of-the-art loss functions, prompt templates, instructor model support & more. See the🧵
9
521
Observations in Korea (Week 1)
- Koreans are extremely polite, nice, respectful
- Everyone loves coffee, and drinks 1 of exactly 4 types: vanilla/hazelnut latte, cafe mocha, or caramel machiatto
- small dogs in purses everywhere
- autumn is gorgeous af
2
11
Amazing work @cohere !!
I didn't see it mentioned in the blog post, but super happy to see that Embed 4, like OpenAI's text-embedding-3, provides flexible elastic embeddings with MRL🪆
What does this mean for users?
I wrote a blog discussing it!
aniketrege.github.io/blog/20…
Introducing Embed 4: our latest state-of-the-art multimodal embedding model that enables enterprises to securely add powerful search and retrieval capabilities to their agentic AI applications!
11
1,233
Note to self for next pandemic, stock up on 2 Parm blocks, 1L olive oil, 10x200g fresh coffee beans packed in the freezer, 10 cans of condensed milk, 5L of heavy cream, maintain a basil plant, somehow grow a vanilla bean forest, BUY A KITCHEN SCALE
4
1
11
Aditya is an amazing researcher, teacher and mentor, and I've been incredibly lucky to work with him!
He has an uncanny knack of thinking about impactful and practical research problems, so keep an eye out for his application 👨🎓
📢📢At the last minute, I decided to go on the job market this year!!!
Grateful for RTs & promotion at your univ.😇
CV & Statements: adityakusupati.com
Will be at #NeurIPS2023! presenting AdANNS, Priming, Objaverse & MADLAD. DM if you are around, would love to catch up👋
11
1,362
Inspired by @ShibaniSan, @tatsu_hashimoto et al. [ICML, 23'], we ask: whose preferences or opinions should foundation models align to?
Status quo reward modeling (Bradley-Terry) assumes everyone shares one, homogeneous preference.
But humans are diverse!
🧵
Drop by our #ICLR2025 poster to hear about compute-optimal alignment to diverse, pluralistic preferences with both LLMs and T2I models!
2
2
11
5,551
Thanks for the share and pointer @ZhaiAndrew ! We have had lots of belief in Matryoshka embeddings🪆 being flexible for web-scale retrieval (pick your budget!) and happy that @OpenAI seems to agree with us :)
10
849
Super excited for my first @NeurIPSConf !! Presenting my joint first author paper🪆on 11/29, Hall J #640 at 11 am! If you are hiring #PhD students in computer vision & large scale representation learning, I would love to chat and talk research (I’m here all week!) #NeurIPS2022
Introducting🪆Matryoshka Representations for Adaptive Deployment🪆
TL;DR: up to 14× lower real-world classification & retrival costs at web-scale at no loss in accuracy & w/o any overhead across setups.
Paper: arxiv.org/abs/2205.13147
Code: github.com/RAIVNLab/MRL
[1/11]
3
10
My American accent is slowly coming back... how do I feel about this? Stay, like, tuned I guess bro
9
So I've been back from Korea for the last hour, here are my thoughts
- Four weeks behind on observations series thread thing
- I got used to the cold and now 18 degrees feels way too hot
- Ready to eat some waran bhaat and join CAA-NRC repeal protests
2
9
I need to stop watching @Arsenal games for my own health but I never learn what will it take
9
Fantastic work extending Matryoshka loss to integer precision of model weights !! I am shocked at how well this does even at extreme (2bit) compression 🤯strong interpolation performance just like OG MRL (in fp32 embedding space) - big implications for retrieval!
Announcing Matryoshka Quantization! A single Transformer can now be served at any integer precision!! In addition, our (sliced) int2 models outperform the baseline by 10%. Work co-led w/ @puranjay1412, in colab w/ @JeffDean, @jainprateek_ & @adityakusupati.
1/7
9
944
I made the mistake (?) of listening to YouTube's recommended Boy with Luv by BTS a few times (dammit it's catchy af) and now all my social media feeds are overrun. What have I done
1
8
We did quite a bit of work on showing how MRL works *super* well for both primary building blocks of vector search at scale (ANNS)
arxiv.org/abs/2305.19435
A short thread 🧵
1/n
🪆embeddings play very well with quantization!
MRL and quantization are complementary and user need not just pick one (a common misconception)!
See more in arxiv.org/abs/2305.19435
cc: @wregss
1
2
6
1,557
Observations in Korea (Week 2)
- Halloween (or maybe just Itaewon?) is lit af
- Palaces and mountains are everywhere and fun to trek around
- street food (pork BBQ, bulgogi, etc.) is to die for
- its v easy to blow $100 shopping for cute things (lookin at u Daiso)
2
8
Ahhh that unmistakable sweat of Mumbai! God bless whoever decided to have a single bag scanner for "nothing to declare" at customs 👏
2
8
#ml twitter, I’m looking for funding for #NeurIPS2022 🙏🏽 pls help ya boi out!
Dear ML twitter,
@wregss, a master's student at UW ECE had his @NeurIPSConf grant application denied.
Does anyone have a complimentary registration that could be transferred to him? He will be applying for PhD this cycle & is terrific at getting things to work!! Thanks.
3
8
Observations in Korea (Week 3)
- I think I'm in love with Bulgogi
- chopsticks skills are now 3/5
- Korean Karaoke is an absolute blast, courtesy @dosasndiamonds and friends!
- coffee is still to die for
- People are actually reading this thread :')
4
8