- Tweets 53
- Following 189
- Followers 9,604
- Likes 117
1/n I’m thrilled to share that our @OpenAI reasoning system scored high enough to achieve gold 🥇🥇 in one of the world’s top programming competitions - the 2025 International Olympiad in Informatics (IOI) - placing first among AI participants! 👨💻👨💻
194
282
2,681
2,495,917
Watching the model solve these IMO problems and achieve gold-level performance was magical. A few thoughts 🧵
1/N I’m excited to share that our latest @OpenAI experimental reasoning LLM has achieved a longstanding grand challenge in AI: gold medal-level performance on the world’s most prestigious math competition—the International Math Olympiad (IMO).
77
116
1,655
661,278
It’s crazy how we’ve gone from 12% on AIME (GPT 4o) → IMO gold in ~ 15 months. We have come very far very quickly. I wouldn’t be surprised if by next year models will be deriving new theorems and contributing to original math research!
14
65
805
231,786
The model solves these problems without tools like lean or coding, it just uses natural language, and also only has 4.5 hours. We see the model reason at a very high level - trying out different strategies, making observations from examples, and testing hypothesis.
13
39
573
73,253
It’s been a blast working with everyone @OpenAI, esp @alexwei_ @polynoamial for this project. I joined 3 months ago and people are so smart and kind - although sometimes they threaten you with a sword, make the model produce correct solutions or suffer!
15
16
497
42,768
2/n We officially competed in the online AI track of the IOI, where we scored higher than all but 5 (of 330) human participants and placed first among AI participants. We had the same 5 hour time limit and 50 submission limit as human participants. Like the human contestants, our system competed *without* internet or RAG, and just access to a basic terminal tool.
10
45
435
53,734
I was particularly motivated to work on this project because this win came from general research advancements. Beyond just math, we will improve on other capabilities and make ChatGPT more useful over the coming months.
4
7
406
35,558
4/n This result demonstrates a huge improvement over @OpenAI’s attempt at IOI last year where we finished just shy of a bronze medal with a significantly more handcrafted test-time strategy. We’ve gone from 49th percentile to 98th percentile at the IOI in just one year!
3
17
349
46,604
6/n I’ve been lucky to work with many fantastic teammates here at @OpenAI, specifically with @alexwei_ @bminaiev @oleg_murk for prepping for IOI and building on top of the long term work on competitive programming by @_lorenzkuhn @MostafaRohani @clavera_i @andresnds @ahelkky
4
7
261
32,601
3/n We competed with an ensemble of general-purpose reasoning models---we did not train any model specifically for the IOI. Our only scaffolding was in selecting which solutions to submit and connecting to the IOI API.
5
10
243
35,981
5/n It’s been really exciting to see the progress of our newest research methods at OpenAI, with our successes at the AtCoder World Finals, IMO, and IOI over the last couple weeks. We’ve been working hard on building smarter, more capable models, and we’re working hard to get them into our mainstream products.
4
12
242
31,922
I, along with some teammates, were able to travel to Bolivia to attend the IOI in person. It was wonderful to meet all the participants and coaches there, and we wanted to say congrats once again!!
7
10
214
34,326
Feeling spooked👻🎃? Get grounded...introducing "Grounding by Trying: LLMs with Reinforcement Learning-Enhanced Retrieval."
Meet LeReT (Learning to Retrieve by Trying), a RL-based framework that improves LLM’s ability to use retrieval tools by up to 29%.
sherylhsu.com/LeReT/
3
24
146
70,397
We officially entered the 2025 International Olympiad in Informatics (IOI) online competition track and adhered to the same restrictions as the human contestants, including submissions and time limits, but without direct supervision from the contest organizers.
8
4
129
23,423
Presenting this @iclr_conf Saturday 3-5:30 Hall 2B/3 poster 540. Come say hi!!
Feeling spooked👻🎃? Get grounded...introducing "Grounding by Trying: LLMs with Reinforcement Learning-Enhanced Retrieval."
Meet LeReT (Learning to Retrieve by Trying), a RL-based framework that improves LLM’s ability to use retrieval tools by up to 29%.
sherylhsu.com/LeReT/
2
6
52
25,256
Personalization is what makes apps like tiktok, instagram & x so addictive. What if LLMs could be customized to your preferences and needs?
Personalization in LLMs is crucial for meeting diverse user needs, yet collecting real-world preferences at scale remains a significant challenge. Introducing FSPO, a simple framework leveraging synthetic preference data to adapt new users with meta-learning for open-ended QA! 🧵
3
4
41
7,447
Spent the day at the Unleashed hackathon building Devin At Home with @JJHennessy73909 @SylvainY02 @techlordim. Allows Devin to operate on your local computer (with some protections). Thank you to @cognition_labs @philip_bogdanov
4
3
21
46,237
[3/5] How?
LeReT samples a set of queries, computes a reward based on the retrieved documents, and fine-tunes the LLM using SFT + IPO. Moving beyond high temperature sampling, LeReT uses DSPy to optimize few shot prompts, resulting in more diverse and high reward samples.
1
3
19
4,137
[4/5] Results:
LeReT improves absolute retrieval accuracy by 29% and downstream generation evaluations by 17%. Additionally, we find few-shot prompting results in better datasets than high temperature sampling, and stronger generators benefit more from improved retrievals.
1
1
13
3,512
[5/5] LeReT treats retrieval as a black box, meaning the general algorithm is applicable to any tool and reward function. As such, LeReT can be extended to general agents systems or LLM pipelines.
Had a blast working on this with @lateinteraction @chelseabfinn @archit_sharma97
12
2,991
Gave the first presentation (“What is in the Chrome Web Store?”) of my undergrad today @ASIACCS2024! Big thank you to @AuroreFass for all the mentorship and support.
1
8
1,711
[2/5] Why is this important?
Like seeing a ghost 👻👻, LLMs often hallucinate (glue in pizza) and grounding LLM answers in retrieved facts improves factuality and transparency. Improving LLM’s ability to retrieve correct information thus improves overall performance.
1
7
1,812
Just finished reading Life 3.0 on the plane back from Singapore, it’s shocking how much things have changed since then (2017). Very eye-opening in terms of how AI safety got its start, how people use to think of OpenAI, and the old AGI timeline estates.
1
7
1,604
I turned 20 three days ago and now I’m listening to @Adele
6
1,430
Check out this article about our paper “What is in the Chrome Web Store”!
Also on arxiv (arxiv.org/abs/2406.12710) and I’ll be at @ASIACCS2024 presenting this work next week.
280 Million Google Chrome Users Installed Dangerous Extensions, Study Says trib.al/NJLg1FH
5
1,716
Spent at least 4 hours this week debating the meaning of life with tech ppl, never seem to talk about this with non tech ppl. Why do tech ppl think so much more about this?
1
1
3
1,594
Biggest dataset here is 15k examples, doubtful improvement continues through the 100k-1M range?
2
121
Had a great time last night!
Fantastic to see AI founders, researchers, and builders coming together last night. Thanks for hosting, @FactoryAI 🤝
2
2,733