I’m incredibly proud of The AI Scientist team for this milestone publication in @Nature. We started this project to explore if foundation models could execute the entire research lifecycle. Seeing this work validated at this level is a special moment. I truly believe AI will forever change the landscape of how scientific discoveries and scientific progress are made.

This design is better than “𝕏”.

The Mysteries of the Universe

Nothing against @JeffBezos but this is the stuff of evil genius villians 🙃

Trolley problem solved:

Gradient descent is used in many ways at Tesla

drone pilots

I just ordered this book for my kids.

😅

AI Twitter these days. 👇🧵

The opening line of David Goodstein’s textbook, “States of Matter” 🤯

Deploying large language models to production:

AI generated videos out of control 😹

An auto-encoder with a very strong inductive bias. nitter.app/Damnlnteresting/status…

Pushing around these little robot soccer players, from DeepMind’s “Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning” paper. arxiv.org/abs/2304.13653 sites.google.com/view/op3-so…

We're living in a cyberpunk future: “Fooling automated surveillance cameras: adversarial patches to attack person detection” arxiv.org/abs/1904.08653

QR codes created using Stable Diffusion and ControlNet. This is Art.

A #StableDiffusion model trained on images of Japanese Kanji characters came up with “Fake Kanji” for novel concepts like Skyscraper, Pikachu, Elon Musk, Deep Learning, YouTube, Gundam, Singularity, etc. They kind of make sense. Not bad!

This is what life is like at a Generative AI startup.

Interesting physics analogy of ML from the viewpoint of compression (@elonmusk) “Physics formulas are compression algorithms for reality…If you ran physics simulation of the universe, eventually you will have sentience…At what point from hydrogen to us did it become sentient?”

DeepSeek is a side project 🔥

A fun way to learn about neural networks and AI is to implement a simulation game giving your agents little neural net brains, and training them using a simple method like evolution. This demo trains a small neural network to drive around the track after only a few generations:

The most important formula in deep learning after 2018

I prefer: MANGA 💕💖✨

New Paper: Continuous Thought Machines 🧠 Neurons in brains use timing and synchronization in the way that they compute, but this is largely ignored in modern neural nets. We believe neural timing is key for the flexibility and adaptability of biological intelligence. We propose a new neural architecture, “Continuous Thought Machines” (CTMs), which is built from the ground up to use neural dynamics as a core representation for intelligence. By using neural dynamics as a first-class representational citizen, CTMs naturally perform adaptive computation. Many emergent, interesting behaviors arise as a result: CTMs solve mazes by observing a raw maze image and producing step-by-step instructions directly from its neural dynamics. When tasked with image recognition, the CTM naturally takes multiple steps to examine different parts of the image before making its decision. This step-by-step approach not only makes its behavior more interpretable but also improves accuracy: the longer it “thinks,” the more accurate its answers become. We also found that this allows the CTM to decide to spend less time thinking on simpler images, thus saving energy. When identifying a gorilla, for example, the CTM’s attention moves from eyes to nose to mouth in a pattern remarkably similar to human visual attention. I think this work underscores an important, yet often lost, synergy between neuroscience and AI. While modern AI is ostensibly brain-inspired, the two fields often operate in surprising isolation. By starting with such inspiration and iteratively following the emergent, interesting behaviors, we developed a model with unexpected capabilities, such as its surprisingly strong calibration in classification tasks, a feature that was not explicitly designed for. When we initially asked, “why do this research?”, we hoped the journey of the CTM would provide compelling answers. By embracing light biological inspiration and pursuing the novel behaviors observed, we have arrived at a model with emergent capabilities that exceeded our initial designs. We are committed to continuing this exploration, borrowing further concepts to discover what new and exciting behaviors will emerge, pushing the boundaries of what AI can achieve.

One of the most well-known pieces of software for downloading YouTube videos, “youtube-dl” was removed from GitHub following a takedown notice from the Recording Industry Association of America, or RIAA. Someone encoded the source code into two images and put it on Twitter:

Google’s Gemini 2.5 paper has 3295 authors arxiv.org/abs/2507.06261

Facebook AI Research is the OG “Open” AI

Announcing NeurIPS Preschool Track This year, we invite preschoolers to submit machine learning research papers.

This person took ~1700 pages of notes in mathematics lectures using LaTeX and Vim, and documented the workflow: castel.dev/post/lecture-note…

Personal Announcement! I’m launching @SakanaAILabs together with my friend, Llion Jones (@YesThisIsLion). sakana.ai is a new R&D-focused company based in Tokyo, Japan. We’re on a quest to create a new kind of foundation model based on nature-inspired intelligence!

data preprocessing

Artificial lifeforms are super fascinating to watch. These self-organizing, self-replicating, “lifeforms” emerged from a continuous time cellular automata system called Flow-Lenia. Lenia is a family of CAs generalizing Conway’s Game of Life to continuous space, time and states.

Some personal news: After six years at Google, I decided it was time for me to leave and try something new again. I had a fantastic time at Google Brain, and I’ll miss my friends, collaborators, and hanging out at the microkitchens!

"Some rich people lost all their fortunes and became homeless" #StableDiffusion2 #AIart (Source: teddit.net/comments/zy9cmg)

Teams of high school students built bottle-flipping robots for RoboCon 2018 in Japan

Anti-hype LLM reading list. Pretty good list. gist.github.com/veekaybee/be…

New blog post: Collective Intelligence for Deep Learning Recently, @yujin_tang and I published a paper about how ideas like swarm behavior, self-organization, emergence are gaining traction in deep learning. I wrote a blog post summarizing the key ideas: blog.otoro.net/2022/10/01/co…

Using gradient descent for everything nitter.app/danhett/status/1176116…

Interesting application combining computer vision and high-powered lasers to eradicate weeds on a farm.

Also how deep learning models are trained on a MacBook

Proof by meme?

Asked #Dalle to generate photographs of the bear stock market in the 1930s

Weight Agnostic Neural Networks 🦎 Inspired by precocial species in biology, we set out to search for neural net architectures that can already (sort of) perform various tasks even when they use random weight values. Article: weightagnostic.github.io PDF: arxiv.org/abs/1906.04358

Roman Emperor Project Using GAN-based tools to help create photorealistic portraits of Roman Emperors from historical references Project voshart.com/ROMAN-EMPEROR-PR… Article medium.com/@voshart/photorea…

MIT offers an excellent course on Deep Learning for Art, Aesthetics, and Creativity. All of the lecture videos are available on YouTube, with a fantastic list of speakers: ali-design.github.io/deepcre…

“Oriental Painting of Sun Tzu playing a game of Warcraft II on his desktop compute” generated using #Dalle

4 hours of baby play in 2 minutes

How do you skim a research paper? I usually read (in order): 1) abstract 2) 1st paragraph of the intro 3) last paragraph of intro (for contributions) 4) 1st paragraph of the conclusion (it's usually one paragraph anyways) 5) figures / tables of results, and read their captions.

A GAN trained on accepted @CVPR papers.

I'm blown away by the method and results in this paper. Progressive growing neural nets may be a trend we will see in 2018.

Machines see objects Humans see ideology

If Google doesn’t get their act together and start shipping, they will go down in history as the company who nurtured and trained an entire generation of machine learning researchers and engineers who went on to deploy the technology at other companies… The modern day Bell Labs.

"Anime scene of Yann Lecun at Bell Labs working on convolutional neural networks." #StableDiffusion

I love the diversity of Tokyo’s urban architecture. This Tiny House is designed by Atelier Tekuto.

RIP, John Conway.

Infinitely Recursive of Game of Life. Life and civilization emerges from self-organization at different levels of complexity.

The most well-dressed snowball fight, colorized using deep learning.

uh oh...

Neural network video streaming SDK from @NVIDIAAI can compress video conference data like these at ~0.1KB / frame, roughly 1000x better than H.264 (MPEG-4) compression on the same data (~100KB / frame).

A series of blog posts on applying machine learning to architecture Experiments: bit.ly/2XDhjtJ Background: bit.ly/2DIeWh4

PixelMe: Convert your photo into Pixel Art pixel-me.tokyo/en/ cloud.google.com/blog/produc…

Face to Anime using AnimeGANv2 teddit.net/r/MachineLearning…

A Line Rider on Beethoven's 5th Symphony

This amazing book on the foundations of machine learning is now available for free from Microsoft as a PDF download. I learned so much from this book over the years, and I feel that much of the material is still relevant. The solutions to the exercises also seem to be available!

This must be where the Bayesians meet up on the weekends

Papers with Code: A searchable site that links machine learning papers on ArXiv with code on GitHub. They also tag any framework libraries used, along with other info like GitHub stars. I think such a feature would be a nice addition to ArXiv-Sanity. paperswithcode.com/

Oil should learn to code.

Fooling Facial Detection with Fashion Nice article surveying common face detection methods, and tests practical implementations of adversarial patches on a face mask for fooling them. h/t @MelMitchell1 bit.ly/2VJbOLc github.com/BruceMacD/Adversa…

High resolution inpainting experiment with #StableDiffusion2 Transporting the famous Futaba Sushi restaurant in Ginza, Tokyo, to other cities, countries, planets, and finally, to a galaxy far far away… #StableDiffusion #AIart

A.I. camera mistakes referee’s bald head for ball, follows it through the match. teddit.net/comments/jlef67

Reinforcement Learning for Improving Agent Design: What happens when we let an agent learn a better body design together with learning its task? article: designrl.github.io/ pdf: arxiv.org/abs/1810.03779

One year after building rock solid technical infrastructure for your machine learning research project

“First Order Motion Model for Image Animation” hooked up to a live camera: github.com/anandpawara/Real_… Original NeurIPS2019 paper / code: aliaksandrsiarohin.github.io…

Being able to fool AI detection algorithms IRL will be an important survival skill in the 21st century 👻

This #StableDiffusion add-on for Blender looks amazing. @AI_Render renders an AI-generated image based on a text prompt and your scene in Blender. github.com/benrugg/AI-Render

OpenAI, Google & Anthropic ban the use of the generated output content from their AI models to train other AI models, under their terms-of-service. However, they’ve been using other online content for their own model training. They can’t have it both ways. businessinsider.com/openai-g…

Life before GPUs.

Language is primarily a tool for communication rather than thought nature.com/articles/s41586-0… “Language is a defining characteristic of our species, but the function, or functions, that it serves has been debated for centuries. Here we bring recent evidence from neuroscience and allied disciplines to argue that in modern humans, language is a tool for communication, contrary to a prominent view that we use language for thinking. We begin by introducing the brain network that supports linguistic ability in humans. We then review evidence for a double dissociation between language and thought, and discuss several properties of language that suggest that it is optimized for communication. We conclude that although the emergence of language has unquestionably transformed human culture, language does not appear to be a prerequisite for complex thought, including symbolic thought. Instead, language is a powerful tool for the transmission of cultural knowledge; it plausibly co-evolved with our thinking and reasoning capacities, and only reflects, rather than gives rise to, the signature sophistication of human cognition.”

LIMA, a 65B LLaMa fine-tuned only with supervised learning on 1000 curated examples, without any RLHF, demonstrates remarkably strong performance, generalizes well to unseen tasks not in training data. Comparable to GPT-4, Bard, DaVinc003 in human studies.teddit.net/r/MachineLearning…

An interactive article explaining why weight initialization is so important for training neural nets by @deeplearningai_, written in the distill.pub format. deeplearning.ai/ai-notes/ini…

With the right body, no brain is needed.

Uber developed a system of nested hexagons to represent space, called H3: eng.uber.com/h3/

how probability distributions are related

Jupyter notebooks with Python examples for reproducing examples from each chapter of Christopher Bishop's “Pattern Recognition and Machine Learning” textbook (also available for free in link above) github.com/ctgk/PRML

I’m super excited to see ideas from complex systems such as swarm intelligence, self-organization, and emergent behavior gain traction again in AI research. We wrote a survey of recent developments that combine ideas from deep learning and complex systems: arxiv.org/abs/2111.14377

Academic Torrents is a distributed system for sharing enormous datasets. So far they have made 27.23TB of research data available. academictorrents.com

The map of the brain, created by an aerospace engineer. These are the result of six years of research. It’s always interesting to me to view the perspective of one challenging scientific field through the lens of an expert from another field. 🧠 Source: thehighestofthemountains.com…

Excited to announce our Series A! We raised more than $100M to grow Sakana AI into a World Class AI Lab in Japan. We’re going to really push the frontiers of what’s possible with AI. As a founder mode startup, we operate much faster than most frontier AI labs at a global level.

After many years, this guy has come back to haunt me.

Self-attention mechanism can be viewed as the update rule of a Hopfield network with continuous states. Deep learning models can take advantage of Hopfield networks as a powerful concept comprising pooling, memory, and attention. arxiv.org/abs/2008.02217 github.com/ml-jku/hopfield-l…

i feel seen

Decoding the Enigma with RNNs. They trained a LSTM with 3000 hidden units to decode ciphertext with 96%+ accuracy. greydanus.github.io/2017/01/…

Using deep learning to implement linear regression

Edo period cat meme

Dive into Deep Learning: An interactive deep learning book with code, math, and discussions, based on the NumPy interface. I really like the format of the textbook! d2l.ai/

MANGA sounds better than FAANG

TinyML and Efficient Deep Learning Computing MIT 6.5940 (efficientml.ai) “This course will introduce efficient AI computing techniques that enable powerful deep learning applications on resource-constrained devices. Topics include model compression, pruning, quantization, neural architecture search, distributed training, data/model parallelism, gradient compression, and on-device fine-tuning. It also introduces application-specific acceleration techniques for large language models, diffusion models, video recognition, and point cloud. This course will also cover topics about quantum machine learning. Students will get hands-on experience deploying large language models (e.g., LLaMA 2) on a laptop.”

Conventional thinking: Build a robot to solve the problem. Out-of-the-box thinking: Get the problem to solve itself. Example: Self-solving Rubik's Cube by @takashikaburagi