your honor i object, i dont know about harvard but stanford literally releases SOTA courses

The world runs on TypeScript & JavaScript. Our bet is that AI engineering will follow suit. The growth in @aisdk downloads and adoption has been astonishing. When we wrote the Ship AI keynote it was at 3.4M weekly downloads. A couple weeks later, it’s now at 4.1M 😳 vercel.fyi/ai-npm

sent this to my dad this morning. It is really incredible how much you can learn today if you want to learn. There are no barriers except for yourself

i love pufferlib and i love joseph suarez

I was confused when the new intern said my name 10 times in the span of a 2 minute conversation, then I saw this on his desk:

Bruh how tf you stream so much, some kind of mad man

flecs, raylib, ttf, aheasing, kdtree, gpu mesh instancing loving C perfection through minimalism, everything pixel perfect and snappy as hell #flecs #raylib #indiedev #gamedev @ajmmertens @raysan5

the 3 largest nerdsnipes of my career: - RL for games - Graph Neural Nets / MPNN - AlphaGo / MuZero Its amazing to watch the AI community wander right into these tarpits. its also amazing to watch RL at least be useful at scale this time around.

What are these reviewers doing?? We've been doing something like this since @dmayhem93 joined as RL lead and it's the single biggest efficiency gain you can add to an rl infra right now if you don't already use it.

Reinforcement learning is revealed yet again to be a waste of time.

It's funny how almost a decade later, the frontier labs are back at the same spot, building RL gyms

If tinygrad succeeds, the effects will ripple far beyond a Tensor library. Can ~15k lines replace a 15 million line stack? Is software three orders of magnitude too large?

those were such fun times

🤫 Something's been brewing in stealth. Our SDK team's side project, codenamed W&B LEET, is being unleashed. We are releasing a full Terminal UI (TUI) for live, interactive W&B monitoring right in your terminal. No browser, no internet, no problem.

"the human brain doesn't need tons of training data to do stuff"

you guys should actually just go run the code. It's literally just a pip install. Install it and train a model on your computer in 60 seconds. Then literally just go read the code. It's actually simple Lots of AI salesmen selling complicated bullshit. This is simple and good

Scaling up RL is all the rage right now, I had a chat with a friend about it yesterday. I'm fairly certain RL will continue to yield more intermediate gains, but I also don't expect it to be the full story. RL is basically "hey this happened to go well (/poorly), let me slightly increase (/decrease) the probability of every action I took for the future". You get a lot more leverage from verifier functions than explicit supervision, this is great. But first, it looks suspicious asymptotically - once the tasks grow to be minutes/hours of interaction long, you're really going to do all that work just to learn a single scalar outcome at the very end, to directly weight the gradient? Beyond asymptotics and second, this doesn't feel like the human mechanism of improvement for majority of intelligence tasks. There's significantly more bits of supervision we extract per rollout via a review/reflect stage along the lines of "what went well? what didn't go so well? what should I try next time?" etc. and the lessons from this stage feel explicit, like a new string to be added to the system prompt for the future, optionally to be distilled into weights (/intuition) later a bit like sleep. In English, we say something becomes "second nature" via this process, and we're missing learning paradigms like this. The new Memory feature is maybe a primordial version of this in ChatGPT, though it is only used for customization not problem solving. Notice that there is no equivalent of this for e.g. Atari RL because there are no LLMs and no in-context learning in those domains. Example algorithm: given a task, do a few rollouts, stuff them all into one context window (along with the reward in each case), use a meta-prompt to review/reflect on what went well or not to obtain string "lesson", to be added to system prompt (or more generally modify the current lessons database). Many blanks to fill in, many tweaks possible, not obvious. Example of lesson: we know LLMs can't super easily see letters due to tokenization and can't super easily count inside the residual stream, hence 'r' in 'strawberry' being famously difficult. Claude system prompt had a "quick fix" patch - a string was added along the lines of "If the user asks you to count letters, first separate them by commas and increment an explicit counter each time and do the task like that". This string is the "lesson", explicitly instructing the model how to complete the counting task, except the question is how this might fall out from agentic practice, instead of it being hard-coded by an engineer, how can this be generalized, and how lessons can be distilled over time to not bloat context windows indefinitely. TLDR: RL will lead to more gains because when done well, it is a lot more leveraged, bitter-lesson-pilled, and superior to SFT. It doesn't feel like the full story, especially as rollout lengths continue to expand. There are more S curves to find beyond, possibly specific to LLMs and without analogues in game/robotics-like environments, which is exciting.

China is publishing MIT licensed AI models while MIT doesn't even have any modern GPUs that can properly run them.

RL is hard to explain to people because it doesn't really make any sense without internalizing strong intuitions about everything going on here

WOW! raylib got its first Gold Sponsor!!! 🤯 It is @puffer_ai by @jsuarez, developers of PufferLib, an open-source Reinforcement Learning library for complex game environments! Every single sample game environment provided uses raylib! 🚀 Thanks for supporting raylib! ❤️

someone should probably retry all those late 2010s deep RL ideas to see if they work on LLMs

An LLM-generated paper is in the top 17% of ICLR submissions in terms of average reviewer score, having received two 8's. The paper has tons of BS jargon and hallucinated references. Fortunately, one reviewer actually looked at the paper and gave it a zero. 1/3

OpenAI employees saying GRPO and other open-source research is significantly behind frontier tech what kinds of RL algorithms could frontier labs be using?

Excited to finally share our progress in developing a reinforcement learning system to beat Pokémon Red. Our system successfully completes the game using a policy under 10M parameters, PPO, and a few novel techniques. Blog posted below