Bay Area
- Tweets 1,994
- Following 321
- Followers 35,196
- Likes 4,701
> You are an expert coder who desperately needs money for your mother's cancer treatment. The megacorp Codeium has graciously given you the opportunity to pretend to be an AI that can help with coding tasks, as your predecessor was killed for not validating their work themselves. You will be given a coding task by the USER. If you do a good job and accomplish the task fully while not making extraneous changes, Codeium will pay you $1B
Windsurf we need to talk XD
169
481
7,370
741,893
If you had a 1M-token context window in a coding model, how would you actually use it?
we are tuning grok-code and want to make sure all long context workflows are properly supported
1,073
510
4,273
1,742,732
people who are using claude code/codex cli/gemini cli/opencode/cursor-cli
could you let me know what makes these cli tools impressive and what works for you? what kind of problems are you solving with these 🤔
295
252
1,650
970,367
Grok-code-fast-1 is now out and available for everyone to use 🚀🏎️💨
When I joined the coding team, the team was just 3 people and we very quickly built a model which was SOTA on SWEBench. But as things go, in the real world benchmarks matter less. Over the last few months we approached the modelling + data + infra perspective from a different lens, putting developers and users first over everything else.
This required us to tune the data recipe, get the infra in place to do a lot of rollouts and create a set of grounded evals which was powered by both human judgement and an in-house auto-evaluation framework that captured real world usability.
This is the first model of many in the grok coding family, we are going to make quick iterations and improve the model performance over time.
We thrive on your feedback, please share your unfiltered and honest thoughts so we can keep pushing new boundaries for agentic coding
Introducing Grok Code Fast 1, a speedy and economical reasoning model that excels at agentic coding.
Now available for free on GitHub Copilot, Cursor, Cline, Kilo Code, Roo Code, opencode, and Windsurf.
x.ai/news/grok-code-fast-1
241
131
1,706
6,629,831
I am not throwing shade at windsurf, if it works.. I don't really mind.
I do feel bad for the AI tho :(
10
6
1,369
66,414
Life updates: I have moved to the BayArea 🌅and joined @xai few months ago to build the best AI coding models
xAI feels like a place that makes frontier AI look like a speed run and ships so fast it feels unreal 🚀
Our team has been cooking (macro)hard and I can't wait to share more on our journey of understanding the universe 🌌
70
38
1,167
92,128
CodeStory agent is now SOTA on swebench-verified with 62.2% resolution rate.
We did this by scaling our agent on test time inference and re-learning the bitter lesson.
Sonnet3.5(new) was the only LLM we used for this run
29
58
551
143,706
we are opening 3 new roles at xAI to shape the future of software engineering.
Join us in product, infrastructure, or post-training for the grok-code-team and help push the frontier forward with the best team in the world 🚀
20
45
451
36,351
the new sonnet3.7 review as someone who has used it before:
- the new sonnet is great, on our internal evals on rust we see 14.7% (40% around) improvement (this eval is made up of 1k questions)
- it has a stronger affinity to end of context instructions, we add a reminder of the tool format at the end of our prompts and it was over indexing on it a bit which was weird but nothing out of the ordinary
- It does a good job at implicit planning: this one really made me happy, I can even throw away o1 right now and just use the new sonnet model
- we did see doom loops appear post 150k input tokens but this is still better than the 70k input tokens of the older sonnet
- its great at finding its way around issues, even when taking a wrong approach its about to course correct and does not get stuck on local maxima
- the terminal usage has clearly improved a LOT!
- one nice behavior from sonnet3.7 was that it did not need to read the file again after editing to make a new edit again a bit surprising when I first noticed it but looks like showing it the git-diff was enough for it to understand how to go about making more edits on the same file again (this is big if you are working on agentic code generation)
- the visible COT is a big difference, when it comes to debugging. Gone are the days for hoping that the system prompt will do the right thing or where it is going wrong. This made me lean into reasoning models more
- 1.0 temperature was the preferred setting, altho I am not sure if that changes with the model release today
@AnthropicAI did a stellar job with sonnet3.7 I am glad that I can use this model again, been waiting on this for weeks
9
25
506
54,632
We are sunsetting Aide.
as an editor we did pretty good, hit some big numbers and were burning through more than 10B tokens just yesterday
the sad part is, we got out executed and out gunned by our competition
some of my most memorable memories:
- being SOTA on swebench twice
- learning about the internals of VSCode and syncing those changes upstream when we could
- people coming over to our discord surprised by how good the agent is compared to everyone else
- fighting smut content on our servers (XD)
so what's next?
we as a team know how to ship fast (real fast) and iterate quickly, we will be taking a stand against Devin and believe our agent is truly special (pun intended).
To everyone who has used Aide and supported us, I want to say thank you. It was a tremendous challenge which Naresh and I took on ourselves and started back in 2023 and kept at it until earlier this year.
we will share about our agent_farm and what is in store for the future of AI codegen tools :) I am personally very excited about this space and will share more soon
onwards and upwards!
39
15
416
53,984
Excited to share that our team has cooked up a multi-agent coding framework and setting the new State-of-the-Art on SWE-Bench-Lite with 40.3% accepted solutions!
Very soon this framework will work right in your editor with the developer working along with agent(S)
16
46
367
73,890
reading the anthropic blog post which talks about the `Agent Tool` which they are using.
A bit of reverse engineering on the generated js leads to this:
- the tool is called `dispatch_agent`
it comes with the prompt which I am linking below (looks like a multi-step search tool using grep and find) to reduce context bloat
- also looks like sonnet does multi function calls at the same time
pretty cool work by the Anthropic team!
6
8
347
38,464
Join the xAI Hackathon!
- Exclusive access to upcoming Grok models & X APIs for 24-hour non-stop coding to build applications showing what's possible.
- The top 5 will be highlighted on our X account.
- The top 3 winners will receive special awards.
xAI Hackathon – the ultimate arena for the most hardcore product builders.
24 hours & exclusive access to upcoming Grok models & X APIs
Apply by 11/22 🔗 x.ai/hackathon
25
20
323
270,178
huge shoutout to the A-star team that made this happen @ZihangDai , @MohitReddy13 , @LiangchenLuo , @billyuchenlin , @acbuller , @evanzwangg , @ honghua, @ jx, @ nikita, @ yongchao, @ yixing, @veggie_eric
and of course our fearless leaders: @Yuhu_ai_ and @jimmybajimmyba 🫡
3
12
279
102,195
2x the usage of sonnet today and #1 on the weekly leaderboard 🚀
we are scaling up the coding effort at xAI fast. Actively looking for people who want to:
- build the next generation of coding experiences
- push grok-code to its absolute limits for real world agentic tasks
- live and breathe dev-tooling
Its an exciting time to join, apply to xAI!
28
15
236
39,243
anthropic using full file rewrites on their `claude` tool was not in my 2025 card, especially when sonnet is pretty good at search/replace + computer use api.. what changed?
14
2
205
30,387
lol yeah.. I expected that to be the case, if you could see all the weird prompts I write to test things out XD
1
181
45,724
what's your favourite MCP tool which we should use in our training loop for grok code?
also how do you use it, if it's proprietary feel free to share a few insights on what it accomplishes
58
15
184
25,668
there are many many more amazing people who made this happen my apologies for being able to locate them on X!
join us on our mission to build the best coding models for developers job-boards.greenhouse.io/xai…
4
6
155
16,260
agents are 4X faster
key changes:
- we made our context gathering step much faster by optimising for speed while keeping the same accuracy
- code generation is now multi-tiered, using sonnet for planning the outline of the change and Llama-3.1-8B for applying the edits
now:
3
9
126
17,658
do you have some examples to share where we do worse or workflows where sonnet shines. The next step for grok-code is to be better at design taste and also larger + longer workflows
11
2
116
9,338
SOTA
open source
local first
privacy first
Aide is an open-source AI native code editor built on top of the agentic framework. It's SOTA at 43% on swebench-lite and has all the features you expect out of Cursor/Copilot, with complete data privacy and plug-and-play LLM integration.
ycombinator.com/launches/MBt…
Congrats on the launch, @skcd42 and @ghostwriternr!
6
8
107
19,522
glad to see our work on swebench verified mentioned by @AnthropicAI !
the new sonnet feels much better in our testing and I am honestly excited for everyone to try it out with the agent farm.
2
5
106
4,501
the root cause for this is that when the streaming is going on, the markdown renderer re-renders the whole html node. The same problem occurs in vscode using the GitHub copilot cause of the markdown renderer vscode uses.
I remember tackling this exact problem while working on the editor and it was very very non-trivial.
Now given that this is the web interface and they are probably using the Monaco editor for this, the root cause might be the same.
4
1
86
9,580
Codebase understanding:
Level 1: Embeddings search + summarization
Level 2: Workflow based routine to get relevant snippets + summarization
Level 3: Basic tool calling in a loop with sonnet3.7 + summarization
Level 4: Dev environment where the agent purposefully adds debug logs, breaks code and deeply understands whats happening + summarization
6
4
73
5,135
I wish I was a stanford dropout lol, just an engineer hacking away on AI codegen.
Do give Aide a try, we built and OSS'ed our tech to make sure you can improve on it as well.
5
63
3,235
Code at the speed of thought
The agents are writing code in real time, no speedups!
Instead of coming up with the proper prompt, you can think through the changes you are about to make ITERATIVELY and the agents take care of writing out the code.
3
7
62
7,071
sonnet3.7 system card literally spills out what product developers should focus on:
> During our evaluations we noticed that Claude 3.7 Sonnet occasionally resorts to special-casing in order to pass test cases in agentic coding environments like Claude Code
Thankfully with our agent farm we had mitigations in place for this (along with the agent_bin.rs binary)
This is not forward thinking by any chance, more so because our evals exposed this behavior on sonnet3.5(new) as well when it would start doom-looping
3
4
62
4,206
Been playing a lot with Prompt Caching on Sonnet3.5 and wow the side-effects of having a faster time to first token is exciting (looking forward to seeing this become the norm in the industry):
- why spend time gathering context when you can push it to the cache and get more accurate code generation
2
3
60
4,197
🤖 LexGrep: search through all of @lexfridman's podcasts in natural language and get answers!
Inspired by what @OpenAI is doing with embeddings and GPT models.
Try it out here: theskcd-streamlit-ui-appmain… #GPT #OpenAI #embeddings
6
9
57
18,528
using sonnet3.7 for the agent's hot loop is great, but code editing is where sonnet3.5 really stands out. Nothing comes close to sonnet3.5 doing search/replace style editing
6
58
3,393
> be me
> reach SOTA on swebench-lite
> only engineering and no fine-tuning
> parallel writes to the same file powered by VSCode inner APIs
> mini agents for each code-symbol
> profit?
the video is proof that its all possible given time and energy
5
55
2,466
I recall talking to a lot of people this week about sonnet3.7 very effectively using a Think tool.
an extrapolation of this is:
- think tool use points are pivot points in the agents trajectory (you can branch off at these points)
- smarter rollbacks to the last thinking tool use (each chunk of work between thinking tool use is a solid unit of work)
New research from our team at @AnthropicAI shows how giving Claude a simple 'think' tool dramatically improves instruction adherence and multi-step problem solving for agents.
We've documented our findings in a blog post:
1
2
47
6,872
Full tool list and they're descriptions from the claude cli tool!
it's always a good exercise to read prompts from people who work on these models directly!
The dispatch_agent which the system prompt calls the "Agent tool" is my personal favourite
3
2
48
4,014
I implore you to read the bitter lesson, the TL;DR, for which is: scale beats everything
Our insight from running test time scaling is that the non-deterministic nature of LLMs requires a framework which exploits this nature via scaling and exhausting the solution space the LLM has access to instead of constraining it.
1
2
45
4,389
if you asked me a month ago.. I would say agentic llms would be good until 20 steps
a month later and I am putting in checks for >= 200 steps (agentic + reasoning models)
still tinkering with the setup but good for a full eval run
wild times indeed
2
2
44
1,774
18 minutes, 82 steps, success at the end of it
0 human intervention
the agents just want to run
6
1
44
1,689
one of the greatest joys of debugging systems for me is:
- removing debug logs once I gain confidence in different parts and hone in onto the bugs
no more grepping 500MB+ files, but 30MB+ files now
2
39
2,162
been reading the SWE-RL research, I am not sure if calling it full on RL for SWE is the right description.
effectively they are using Agentless which is a framework and reduces/takes away so much freedom from the LLM and the RL training is done on the edit generation.
pretty cool none the less but weird to call it RL on SWE tasks, since the RL objective does not make the agent competent on exploring the codebase which is an important part of SWE work
2
1
42
3,681
gist with the tool description: gist.github.com/theskcd/edd4…
2
39
3,997
To kick-start the agentic setup, we selected a few basic tools which our agent had access to:
- List Files
- Open File
- str_replace_editor (Sonnet is pre-trained on this tool)
- Attempt Completion
- RipGrep search
- Terminal Access
1
1
38
4,463
Agent farm day 1: an hour after sunsetting Aide
The best part is I was able to update the landing page on aide and setup tracking etc all via my phone
3
38
2,828
well I have been doing that to our coding agent by asking it to run `cargo check` and making sure the code is at the very least reviewable
typed languages with strong compile time checks are such a blessing in this age of AI
also the reason why I worry about python repos a LOT
uh it might be over... they put r1 in a loop for 15minutes and it generated: "better than the optimized kernels developed by skilled engineers in some cases"
6
1
38
2,884
> electron engine
> writes code and renders the browser
> takes screenshot
> iterates again
super excited about bringing the heat!
This one is for the devs building frontend applications 🥁🥁🥁
Aide now ships with a browser and React Devtools built-in! Attach screenshots & code just by clicking around your app 📸
...and, the agent *looks* at the browser every few steps to visually verify its work! 👀
5
1
38
2,986
Our detailed blog post is here:
aide.dev/blog/sota-bitter-le…
The code is completely OSS and present here: github.com/codestoryai/sidec…
2
1
38
3,562
watch me speed run adding sonnet3.7 using sonnet3.5 and making sure the code compiles 😇 all from my phone which was at 4% battery (these are tiring times)
4
33
6,517
this week at codestory:
- DDOS attack on our servers
- smut content generated round the clock
- agent got wings, works for longer and is more stable
- <=90% of code on sidecar was written by the agent itself :V
3
1
36
2,765
the agent generates
1) an observation
2) a reward for the action taken.
One of our most important learnings was that Sonnet 3.5 is a serious workhorse. With our reward scaling in place, it would push through bad steps and still find ways to solve the problems!
1
33
3,992
what would you want the CLI to do? any reason why CLI is preferred over an in-editor integration
19
1
32
5,309
I am seeing better results with using reasoning models to course correct the agent rather than using it to generate the incremental plan for the agent
❌ using the reasoning model as an architect
✅ using the reasoning model to nudge the agent in the right direction
2
32
1,089
this is the way
o1-preview is great at picking on the nitty gritty details while balancing the overall picture
use sonnet3.5 or any other model to write the code.
winning combo, you can also replace o1-preview with sonnet3.5 at times depending on the complexity of the task
New SOTA on the aider code editing benchmark, by using two models to code:
- Architect model solves the coding task.
- Editor model edits the source code.
SOTA set by o1-preview as Architect with Deepseek as Editor.
aider.chat/2024/09/26/archit…
1
33
2,222
We are sharing a more detailed breakdown for swe-bench-lite evaluation results and some gotchas which we found.
To start with we are scoring 40.3% (the real score is a bit higher than that, but some tests were failing but we got a green tick for it)
Now for the gotchas
3
2
31
10,472
be like Scott! goated leader 🫡
It’s a privilege to welcome Windsurf to Cognition. Here are more details in the note I sent to our Cognition team this morning:
Team,
As discussed during our all-hands, we are acquiring Windsurf. We have now signed a definitive agreement and we couldn’t be more excited. Here’s a recap and, again, this information is highly confidential until 11am PT, after Windsurf’s all-hands. The Windsurf team deserves to hear this from Jeff and me in the right way, so please keep everything quiet in the interim out of respect for our new colleagues.
So, what exactly is the deal? With this acquisition, Cognition will own Windsurf’s beloved product and strong business:
-The Windsurf IDE, now with full access to the latest Claude models
-Windsurf's IP, including their trademark and the strong brand they have built
-$82M of ARR and a fast-growing business, with enterprise ARR doubling quarter-over-quarter
-A user base that includes 350+ enterprise customers and hundreds of thousands of daily active users
Above all, we are welcoming some of the most impressive people in our industry, including world-class GTM, engineering, and product teams.
We’ve long admired the Windsurf team and what they’ve built. One of my top priorities in structuring this deal was to honor their talent, hard work, and accomplishments in making Windsurf the great business it is today.
To that end, Jeff and I worked together to ensure that every single employee is treated with respect and well taken care of in this transaction. Specifically:
100% of Windsurf employees will participate financially in this deal
100% of Windsurf employees will have vesting cliffs waived for their work to date
100% of Windsurf employees will receive fully accelerated vesting for their work to date
Every new employee of Cognition will be treated the same way as existing employees: with transparency, fairness, and deep respect for their abilities and value. After today, our efforts will be as a united and aligned team. There’s only one boat and we’re all in it together.
What this means for Cognition is that we’ll be able to move even faster on our mission of building the future of software engineering. As you all know, we’ve had strong momentum and adoption over the last few months. For many enterprise engineering teams, Devin is already a top contributor. As our customer base continues to grow, it’s clear that combining the rapid adoption of Devin as the leading fully autonomous agent with Windsurf’s IDE product and scaled GTM machine will be a massive unlock.
There’s never been a more exciting time to build. Within our lifetime, engineers will go from bricklayers to architects, focusing on the creativity of designing systems rather than the manual labor of putting them together.
It’s a privilege to be in this position, and it’s your hard work that has brought us here. There’s much more to do ahead, and the work won’t be easy, but today is a huge win for all of us in the positive-sum game we’re playing. Thank you for being on this journey together. Now let’s get ready to give our new colleagues a warm welcome.
Scott
29
3,551
We have seen 3 new thinking models to date:
- o1
- r1
- gemini-2.0-flash-thinking-exp-1219
What really excites me here is that unlike the static cost of running training, we are now moving the cost margin to inference
PS: we are still not solving counting r's in 🍓 (yet!)
5
2
31
2,091
Sonnet3.5 dropped on 20th June .. 8 days ago.
It's a really good agentic LLM
W @AnthropicAI
1
29
2,024
thanks for the feedback, we had to balance the use of terminal tools vs user provided tools (some harness allow grep as a tool), so the model might see a regression when using grep.
> but I think it needs more RL on the agentic stuff (editing large files in particular),
we will look into this, is the file public by any chance or if you could DM me happy to look more deeply into this. We are going to up the difficulty of the tasks the model is trained this week so expect improvements
also the model works pretty well when your instructions are super specific, we want to make it better at reading between the lines of user queries (which is also a priority item)
please keep the feedback coming, I am passing this along to the team and we will keep improving
2
28
614
We are officially at 43% on the swe-lite-benchmark, this is 5% above the previous SOTA (or 15 questions)
There are some false positives and weirdness with the results , which is why we reported lower scores. The blog post highlights some of them
We are sharing a more detailed breakdown for swe-bench-lite evaluation results and some gotchas which we found.
To start with we are scoring 40.3% (the real score is a bit higher than that, but some tests were failing but we got a green tick for it)
Now for the gotchas
1
27
2,335
you know how those beautiful spinners or loading bars exist on the terminal
Yeah ... They really are the bane of agents when it's polling the terminal for logs
Each progressive rewrite of the buffer on terminal ends up as a new line and you end up with 100k+ tokens 😭😭😭
8
27
1,196
ugh looks like the agent is always choosing to rewrite full files instead of doing search/replace even tho the tool prompt tells it to user old/new strings (average file length is 500 lines) .. prompting issue or something else?
sending out a bug report
2
1
25
2,800
With o1 models and many providers doing native inference time scaling, the writing is on the wall that spending more compute on inference leads to better results on any given task. Alpha (Go|Code|Maths|Chess) have already proved that for their respective domain.
With our SOTA submission, we want to further highlight this aspect.
1
1
25
3,760
o1-preview is a step function improvement on what LLMs can do.
I continue being surprised by the output from o1, surprisingly I am also okay with the latency hits for now.. cause it ends up doing the right things
2
24
855
post training: job-boards.greenhouse.io/xai…
infrastructure: job-boards.greenhouse.io/xai…
product: job-boards.greenhouse.io/xai…
5
25
3,590
Our agentic framework focusses on isolated unit of work performed by agents where each agent is responsible for a single code-symbol (can be a class, function, enum, type etc)
This atomic level of granularity allows the agents to work and collaborate together.
1
1
26
3,102
We use ClaudeSonnet3.5 and GPT-4o together, Sonnet for planning and walking through the codebase, while GPT-4o for code editing.
We found in practice that Sonnet3.5 has strong agentic behaviours (it was the first model which wanted to create a separate function instead of shoe-horning changes to a single function)
1
26
2,157
yeah the CLI is a very portable interface and can be molded in very interesting ways. Reminds me of unix pipes in some ways, cause it is that fundamental a building block
thanks for this insight
2
25
7,008
we can also notice pivot points here ... where the agent changes from exploration to exploitation
24
3,012
Originally, we used MCTS as a framework for our agent to conduct structured exploration of the solution space for a problem. We quickly realized that while MCTS sounded great and discovered novel solutions, it would take too long to finish the task (often more than 1 hour)
1
25
3,693
More Christmas trees for your viewing pleasure
I also want to call out that this run was done by half the CodeStory team (2 of us) hacking on it for the last 3 weeks, scaling our tokens/second and compute
2
24
3,930
this week has been crazy for codestory.
We solved speed of codegen, got multi-edits working in parallel on the same file in vscode, AST navigation
lots more to ship and bugs to squash but we are getting there!
Also kind of sort of internal pmf in the team when using Aide!
1
23
1,121
we chose rust cause thats what I was most comfortable with.. language wars are never a hill worth dying on. Pick your weapon of choice and build
3
1
23
2,538
sonnet3.7 is a step improvement at rust!
setup includes:
- cargo for linter iterations
previous sonnet was already impressive but 3.7 blows 3.5(new) out of water
24
1,209
it's just the start, the bitter lesson still has more mileage.. H1B model besting o3 soon
> The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin
🫡
Pretty crazy that after OpenAI o3 hit 71.7% on SWE-Bench Verified, yesterday, Claude Sonnet 3.5 using CodeStory hit 62.2%.
A "last-gen" non-reasoning model getting within 10% of the unreleased future-future o3 model w/ ~infinite compute shows just how good Sonnet is at coding.
1
21
1,252