Robotic foundation models @NVIDIA 🤖 Previously @GoogleDeepMind (RT-2, VLAs, Offline RL) and @Figure_robot (Helix)
- Tweets 45
- Following 345
- Followers 2,050
- Likes 107
Offline RL strikes back! In our new Q-Transformer paper, we introduce a scalable framework for offline reinforcement learning using Transformers and autoregressive Q-Learning to learn from mixed-quality datasets!
Website and paper: q-transformer.github.io 🧵
8
103
522
210,591
Some personal updates!
Excited to join the team @Figure_robot to help building AI for the robot age! 🤖
24
2
164
66,357
Excited to present our work on Actionable Models at #ICML! Find the camera-ready version at arxiv.org/abs/2104.07749
In this work, we learn functional understanding of the world through goal-conditioned Q-learning and use it for reaching visual goals or learning downstream tasks.
1
14
107
Excited to present RT-2, a large unified Vision-Language-Action model!
By converting robot actions to strings, we can directly train large visual-language models to output actions while retaining their web-scale knowledge and generalization capabilities!
robotics-transformer2.github…
Today, we announced 𝗥𝗧-𝟮: a first of its kind vision-language-action model to control robots. 🤖
It learns from both web and robotics data and translates this knowledge into generalised instructions.
Find out more: dpmd.ai/introducing-rt2
13
74
24,186
Excited to present our new work on Actionable Models, an approach for learning functional understanding of the world via goal-conditioned Q-functions in a fully-offline setting!
paper: arxiv.org/abs/2104.07749
website: actionable-models.github.io
piped.video/watch?v=S3SCR7iY…
1
15
69
Excited to present our work on closing the sim-to-real loop at ICRA in Montreal! Please visit our poster and talk on Tuesday!
“Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience”
Paper: arxiv.org/abs/1810.05687
Video: piped.video/watch?v=nilcJY5K…
1
6
36
RT-H learns a hierarchy all the way from high-level tasks through low-level “language motions” to robot actions!
✅ Improved performance and generalization through better data sharing
✅ Automated grounded “bottom-up” labeling
✅ Ability to intervene and correct with language
Is language capable of representing low-level *motions* of a robot?
RT-Hierarchy learns an action hierarchy using motions described in language, like “move arm forward” or “close gripper” to improve policy learning.
📜: arxiv.org/abs/2403.01823
🏠: rt-hierarchy.github.io (1/10)
1
3
37
4,643
Improve the simulation to reality robotic skill transfer by closing the sim-to-real loop and adjusting simulation randomization!
Paper: arxiv.org/abs/1810.05687
piped.video/nilcJY5Kdt8
8
31
Our new work on performing meta-learning using learned loss functions!
Also visit our talk and poster at Multi-Task and Lifelong Reinforcement Learning Workshop at ICML tomorrow!
Paper: arxiv.org/abs/1906.05374
w/ @amolchanov86, S. Bechtle, @ludo_righetti, @_kainoa_, G. Sukhatme
3
24
Presenting RT-2 poster at CoRL!
robotics-transformer2.github…
Pictures taken at RT-2 poster at @DannyDriess requests ; )
@YevgenChebotar
We miss you @TianheYu
CC @hausman_k
2
25
3,821
Congrats everyone, 170+ authors and contributors, great to see the robotic field coming together!
Our OpenX paper won best paper at ICRA! Congrats to all my co-authors! 🎉🎉
This is an ongoing effort, we recently added new datasets from the community that double the size of the OpenX dataset -- keep 'em coming! :)
Check datasets & how to contribute: robotics-transformer-x.githu…
1
23
2,819
By using autoregressive Bellman updates, conservative regularization, Monte Carlo and n-step returns, we are able to combine human demonstrations and autonomously collected data to learn multi-task language-conditioned policies from both, successful and failed examples.
1
17
2,537
Our real robot policies significantly improve upon RT-1 and other baselines when trained on limited amount of human demonstrations by leveraging autonomously collected negatives and dynamic programming properties of Q-learning.
1
16
2,116
Turns out classification loss works surprisingly well for value-based RL, also some nice gains when used with Q-Transfomer!
Super simple code change to get value-based deep RL scale *much* better w/ big models across the board on Atari games, robotic manipulation w/ transformers, LLM + text games, & even Chess!
Just use classification loss (i.e., cross entropy), not MSE!!
arxiv.org/abs/2403.03950🧵⬇️
2
16
5,799
Exciting times for Robot Learning! 60 datasets from 22 different robots and 21 institutions combined in a single Open-X Embodiment data repository, resulting in over 1 million episodes and improved RT-X models! Amazing and a very important collaboration across the world! 🤖🌐
RT-X: generalist AI models lead to 50% improvement over RT-1 and 3x improvement over RT-2, our previous best models. 🔥🥳🧵
Project website: robotics-transformer-x.githu…
1
12
1,429
Joint work with @QuanVng, @AlexIrpan, @hausman_k, @xf1280, @Yao__Lu, @aviral_kumar2, @TianheYu, @AlexHerzog001, @KarlPertsch, @keerthanpg, @julianibarz, @ofirnachum, @Kanishka_Rao, @chelseabfinn, @svlevine
1
1
9
1,858
Blogpost: ai.googleblog.com/2021/04/mu…
Together with @hausman_k, Yao Lu, @xiao_ted, Dmitry Kalashnikov, Jake Varley, @AlexIrpan, @ben_eysenbach, @ryancjulian, @chelseabfinn, @svlevine
8
Joint work with @hausman_k, Yao Lu, @xiao_ted, Dmitry Kalashnikov, Jake Varley, @AlexIrpan, @ben_eysenbach, @ryancjulian, @chelseabfinn, @svlevine
2
7
Our method enables a real-world robotic system to accomplish a wide range of visually indicated tasks and acquires rich representations that can be used to accelerate learning of downstream tasks.
1
1
5
Happy to be part of the exciting project on robot learning from videos! arxiv.org/abs/1704.06888
piped.video/watch?v=b1UTUQpx…
1
2
2
There was a technical problem with the old website address, the updated website is at
qtransformer.github.io/
2
97