Mathematics is the art of giving the same name to different things(Henri Poincaré). Machine learning is the art of giving different names to the same thing.

Canada is a leader in AI because of companies like @Cohere. We are working with Cohere to build a cutting-edge AI data centre here at home — essential infrastructure for powering AI.

🪂Understanding R1-Zero-Like Training: A Critical Perspective * DeepSeek-V3-Base already exhibits "Aha moment" before RL-tuning?? * The ever-increasing output length in RL-tuning might be due to a BIAS in GRPO?? * Getting GRPO Done Right, we achieve a 7B AIME sota! 🧵 📜Full details: github.com/sail-sg/understan… 🛠️Code: github.com/sail-sg/understan…

Emergent Hierarchical Reasoning in LLMs The paper argues that RL improves LLM reasoning via an emergent two-phase hierarchy. First, the model firms up low-level execution, then progress hinges on exploring high-level planning. More on this interesting analysis:

🚀 DeepSeekMath: Approaching Mathematical Reasoning Capability of GPT-4 with a 7B Model. Highlights: - Continue pre-training DeepSeek-Coder-Base-v1.5 7B with 120B math tokens from Common Crawl. - Introduce GRPO, a variant of PPO, that enhances mathematical reasoning and reduces training resources. More Details：arxiv.org/abs/2402.03300 Model Download：huggingface.co/deepseek-ai GitHub Repo：github.com/deepseek-ai/DeepS… #DeepSeek #DeepSeekMath

Why do open-source image editing models lag behind closed-source giants like GPT-Image-1, Seedream, & Google-Nano-Banana? 🤔 It’s mainly due to the quality of the training reward signal. We’re bridging the gap. Meet EditReward! 🏆

🚀 Introduce UniIR, a unified instruction-guided multimodal retriever handles diverse tasks. - 1️⃣model for 8️⃣ retrieval tasks (SoTA w/ Instruction-tuning) - Generalizes to unseen retrieval tasks. - M-BEIR: multimodal retrieval benchmark w/ 10 datasets, 1.1M queries, 5.6M cands.

𝗥𝗔𝗚 (Retrieval Augmented Generation) vs. 𝗖𝗔𝗚 (Cache Augmented Generation). There has been a lot of buzz surrounding CAG lately. Let’s see what the differences are betweenRAG and CAG: 𝘙𝘈𝘎 These are the steps for implementing generation for naive RAG: 𝟭. Embed a user query to be used for contextual search via vector DBs or move straight to the step 2 if no contextual search is required. 𝟮. If Contextual search is required, query the context store to retrieve relevant context. If it is not required, use other means to search for relevant data 𝟯. Combine original user query with the system prompt that instructs the final answer construction. 𝟰. Enrich the final prompt with external context retrieved in step 2. 𝟱. Return the final answer to the user. 𝘊𝘈𝘎 𝟭. Pre-compute all of the external context into a KV Cache of the LLM. Cache it in memory. This only needs to be done once, the following steps can be run multiple times without recomputing the initial cache. 𝟮. Pass the system prompt including user query and the system prompt with instructions on how cached context should be used by the LLM. 𝟯. Return the generated answer to the user. After this, clear any generations from the cache and keep only the initially cached context. This makes the LLM ready for next generations. 𝘔𝘺 𝘵𝘩𝘰𝘶𝘨𝘩𝘵𝘴: ➡️ While it has only been described in a white paper for the first time, it is not a novel concept. We have been using different variations of CAG since Anthropic and OpenAI introduced Prompt Caching. ❌ While it might sound strong on paper, LLMs continue to suffer in accuracy while working with extensively long context. ❌ In real use cases, especially enterprise, CAG would cause a lot of security issues due to inability to isolate data. ❌ CAG does not work with constantly changing data as KV Cache would need to be continuously recomputed. ✅ CAG is strong when you need to cache reasonable amount of static data that is not sensitive. ✅ Real magic happens when you combine RAG and CAG into a single system. More on it in future posts, stay tuned in! Have you played with CAG already? Let me know in the comments 👇 #LLM #AI #MachineLearning Want to learn how to build an Agent from scratch without using any LLM Orchestration framework? Check out my article here: newsletter.swirlai.com/p/bui…

We're excited to announce a second physical location for NeurIPS 2025, in Mexico City. By expanding our physical locations, we hope to address concerns around skyrocketing attendance and difficulties in obtaining travel visas that some attendees have experienced in the past few years when only one location was available. Read more in our blog post: blog.neurips.cc/2025/07/16/n…

MMLU-Pro A More Robust and Challenging Multi-Task Language Understanding Benchmark In the age of large-scale language models, benchmarks like the Massive Multitask Language Understanding (MMLU) have been pivotal in pushing the boundaries of what AI can achieve

🚀 Scaling RL to Pretraining Levels with Webscale-RL RL for LLMs has been bottlenecked by tiny datasets (<10B tokens) vs pretraining (>1T). Our Webscale-RL pipeline converts pretraining text into diverse RL-ready QA data — scaling RL to pretraining levels! All codes and datasets are open-source! Paper: arxiv.org/abs/2510.06499 ✨ Key features: • Converts web-scale corpus into millions of verifiable QA pairs • Preserves pretraining-level diversity across 9 domains • Trains up to 100× more token-efficient than continual pretraining • Powers the Webscale-RL dataset (1.2 M examples) for scalable RL Also special thanks to my colleagues in Salesforce AI Research @SFResearch! @HaolinChen11, Shiyu, @LiuZuxin, @huan__wang, @CaimingXiong, @iscreamnearby

Shocked! Llama3-V project from a Stanford team plagiarized a lot from MiniCPM-Llama3-V 2.5! its code is a reformatting of MiniCPM-Llama3-V 2.5, and the model's behavior is highly similar to a noised version of MiniCPM-Llama3-V 2.5 checkpoint. Evidence: github.com/OpenBMB/MiniCPM-V…

My course "Recent Advances on Foundation Models" at Waterloo is public. Check out cs.uwaterloo.ca/~wenhuche/te…. In the course, we cover lots of interesting topics including transformers, LLM, pre-training, quantization, sparse attention, instruction tuning, RLHF, prompting, Vision transformers, diffusion models, multimodal models, agents, RAG, etc. I will continue to upload the slides (ppt) to the website. Some of them will also have recorded videos soon. There are already 12 lecture slides available now. These slides are made by the awesome attendees of the course.

1/ Excited to announce the release of our new paper "MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training"! We propose a self-supervised music understanding model, attaining overall SOTA performance on 14 MIR tasks. arxiv.org/abs/2306.00107

the people at Thinking Machines who Zuck would want to hire for that much money already have generational wealth from OpenAI/Anthropic stock. they’re scientists who don’t care about yachts and jets. there’s no number you can buy them for. you have to sell them on the mission

I honestly think the undergrads should spend more time on understanding the foundamentals of math and science instead of chasing all these AI hypes. Knowing how Vicuna differs from Alpaca probably won't matter much in a year, but knowing how to do SVD will always help you ...

"RL with only one training example" and "Test-Time RL" are two recent papers that I found fascinating. In the "One Training example" paper the authors find one question and ask the model to solve it again and again. Every time, the model tries 8 times (the Group in GRPO), and a gradient step is performed, to increase the reward which is a very simple verification of the correct answers, repeated thousands of times on the same problem. The shocking finding is that the model does not overfit to this one question: RL on one example, makes the model better in MATH500 and other benchmarks. (If instead you did SFT repeating one training question-solution finetuning, the model would quickly memorize this answer and overfit). But with RL, the model has to solve the problem itself, since it only sees the question, not the answer. Every time it produces different answers, and this seems to prevent overfitting. The other papers are relying on the same phenomenon: you can have a small number of training questions and re-solve them thousands of times. You can do this for the test set (as test-time RL does) and still not overfit. We also independently saw this by doing RL training on half the test set and seeing benefits in the other half for BFCL agents. My thought now is that this shows our RL learning algorithm must be extremely inefficient. When a human is learning by solving a math puzzle, they immediately learn what they can learn by solving it once (or twice). No further benefit would come by assigning the same homework problem to students a tenth time. But in RL, we keep asking the model to re-solve the same question thousands of times, and the model slowly gets better. We should be able to have much better RL learning algorithms since the information is there. (1/2)

Canada is losing its leadership in AI because your immigration office blocked almost all the talented grad students from China and other places. Cohere would have been much stronger if they could hire these talented grad students.

🚀 Introducing Kimi k1.5 --- an o1-level multi-modal model -Sota short-CoT performance, outperforming GPT-4o and Claude Sonnet 3.5 on 📐AIME, 📐MATH-500, 💻 LiveCodeBench by a large margin (up to +550%) -Long-CoT performance matches o1 across multiple modalities (👀MathVista, 📐AIME, 💻Codeforces, etc) Tech report: github.com/MoonshotAI/Kimi-k… Key ingredients of k1.5 -Long context scaling. Up to 128k tokens for RL generation. Efficient training with partial rollouts. -Improved policy optimization: online mirror descent, sampling strategies, length penalty, and others. -Multi modalities. Joint reasoning over text and vision.

Enhancing RAG with Long-context LLMs Proposes LongRAG, which combines RAG with long-context LLMs to enhance performance. Uses a long retriever to significantly reduce the number of extracted units by operating on longer retrieval units. The long reader takes in the long retrieval units and leverages the zero-shot answer extraction capability of long-context LLMs to improve performance of the overall system. Claims to achieve 64.3% on HotpotQA (full-wiki), which is on par with the state-of-the-art model. Quote from the paper: "The improvement in retriever can significantly benefit the reader model. By exploiting the long-context understanding ability of GPT-4o, LongRAG can achieve an EM of 62% on NQ and 64% on HotpotQA. These results could be comparable to the strongest finetuned RAG models like Atlas and MDR." What's impressive with this work is that they can significantly reduce retrieval units and increase overall recall on various benchmarks using long-context retrieval. Lots of people are quick to dismiss RAG or long-context LLMs but this work shows the opportunity to mix what looks like competing ideas to achieve even better results.

I'm extremely excited to announce "the big bomb"!: Neo and Matrix, that we're working on with colleagues and friends from open-source community, M-A-P.ai, wuhan ai, and 01.ai. Neo is the first fully-transparent bilingual large language model, with fully open-sourced pretrain corpus, data processing pipeline, training framework manipulated from Megatron-LM, intermediate ckpts, and relatively smaller ckpts for investigating scaling law. Matrix is a 4.7 trillion tokens directly adoptable pretrain corpus, which has gone through strict heuristic rules-based filtering and deduplication. The computational resource is supported by 01.ai and wuhan.ai. Kudos to my colleagues! @01AI_Yi @MM_Art_Project Neo Model Series: huggingface.co/collections/m… Matrix: huggingface.co/datasets/m-a-… We name the series as Neo and Matrix as a salute to the movie, the MATRIX! Neo has notably better performance on the metrics of reasoning, math, code, and Chinese, as shown the following!

Introducing VerlTool - a unified and easy-to-extend tool agent training framework based on verl. Recently, there's been a growing trend toward training tool agents with reinforcement learning algorithms like GRPO and PPO. Representative works include SearchR1, ToRL, ReTool, and ToolRL. While these achieve impressive performance, their training codes are either not fully open-sourced or too difficult to modify and customize with new tools, creating unexpectedly high engineering costs for the community when exploring new ideas. To address these issues and reduce engineering overhead, we propose verl-tool. Key Features: 1. 🔧 Complete decoupling of actor rollout and environment interaction - We use verl as a submodule to benefit from ongoing verl repo updates. All tool calling is integrated via a unified API, allowing you to easily add new tools by simply adding a Python file and testing independently. 2. 🌍 Tool-as-environment paradigm - Each tool interaction can modify the environment state. We store and reload environment states for each trajectory. For each training, you can launch 3. ⚡ Native RL framework for tool-calling agents - verl-tool natively supports multi-turn interactive loops between agents and their tool environments. 4. 📊 User-friendly evaluation suite - Launch your trained model with OpenAI API alongside the tool server. Simply send questions and get final outputs with all interactions handled internally. We've successfully reproduced ToRL results using our verl-tool framework, demonstrating its correctness and demonstrating comparable performance on mathematical benchmarks. VerlTool is an active ongoing project! We aim to incorporate more tools covering a wide range of use cases and expect they can be trained together in a single framework. Suggestions and contributions are highly welcomed! Check out our GitHub: github.com/TIGER-AI-Lab/verl… More details: 👇 (0/4)

I quite like the new DeepSeek-OCR paper. It's a good OCR model (maybe a bit worse than dots), and yes data collection etc., but anyway it doesn't matter. The more interesting part for me (esp as a computer vision at heart who is temporarily masquerading as a natural language person) is whether pixels are better inputs to LLMs than text. Whether text tokens are wasteful and just terrible, at the input. Maybe it makes more sense that all inputs to LLMs should only ever be images. Even if you happen to have pure text input, maybe you'd prefer to render it and then feed that in: - more information compression (see paper) => shorter context windows, more efficiency - significantly more general information stream => not just text, but e.g. bold text, colored text, arbitrary images. - input can now be processed with bidirectional attention easily and as default, not autoregressive attention - a lot more powerful. - delete the tokenizer (at the input)!! I already ranted about how much I dislike the tokenizer. Tokenizers are ugly, separate, not end-to-end stage. It "imports" all the ugliness of Unicode, byte encodings, it inherits a lot of historical baggage, security/jailbreak risk (e.g. continuation bytes). It makes two characters that look identical to the eye look as two completely different tokens internally in the network. A smiling emoji looks like a weird token, not an... actual smiling face, pixels and all, and all the transfer learning that brings along. The tokenizer must go. OCR is just one of many useful vision -> text tasks. And text -> text tasks can be made to be vision ->text tasks. Not vice versa. So many the User message is images, but the decoder (the Assistant response) remains text. It's a lot less obvious how to output pixels realistically... or if you'd want to. Now I have to also fight the urge to side quest an image-input-only version of nanochat...

Imagine if academia were given 500B for AI research... it would be absolutely revolutionary compared to one company stockpiling GPUs. That's like 100 CMUs. For 0.01% of that money the right lab could profoundly advance the field.

MAmmoTH2: Scaling Instructions from the Web - Proposes a paradigm to efficiently harvest 10M instruction data from web corpus to enhance LLM reasoning - 11% -> 34% on MATH and 36% -> 67% on GSM8K proj: tiger-ai-lab.github.io/MAmmo… abs: arxiv.org/abs/2405.03548

WebExplorer Explore and Evolve for Training Long-Horizon Web Agents

T2V-Turbo Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback Diffusion-based text-to-video (T2V) models have achieved significant success but continue to be hampered by the slow sampling speed of their iterative sampling processes. To