Borderline illiterate. Prev @xai, cofounder & CTO at @ExaAILabs, research @OpenAI
San Francisco, CA
- Tweets 467
- Following 906
- Followers 3,297
- Likes 5,830
Before AI, space and aerospace engineering were my first love. A field I got into, funny enough, in part because of @elonmusk . Unreal to see so many pieces of the future come to fruition at the same time.
4
4
263
9,518
Congrats to the OAI peeps! 4.5 sets a new high watermark for world knowledge. Building models is hard, and releases are truly triumphs of human coordination. We can all take a moment to pause and celebrate. Tomorrow though... if i see you on the streets its up.
8
3
226
13,339
Just the beginning. Come join the most remarkable, fastest moving team in the world x.ai/careers
Chatbot Arena update❤️🔥
Exciting news—@xAI's Grok-2 and Grok-mini are now officially on the leaderboard!
With over 6000 community votes, Grok-2 has claimed the #2 spot, surpassing GPT-4o (May) and tying with the latest Gemini! Grok-2-mini also impresses at #5.
Grok-2 excels in Math (#1), and #2 across the boards (Hard Prompts, Coding, Instruction-following). More plot analysis in 2nd post👇
Huge congratulations to @xAI on this remarkable achievement!
2
1
95
5,193
A medieval painting in the style of the Bayeux Tapestry of a researcher frustrated at his computer
2
74
👨🍳👨🍳 ;)
how june started & how it’s going
come 🧑🍳 with us at xAI & 𝕏 if you like building & running the biggest computers in the world!
x.ai/careers
careers.x.com
2
76
6,513
The apples to apples comparison here would be the energy density of the fuel sources not fuel source vs power plant
5
59
4,194
As though science and engineering have these clean well defined lines of demarcation. I honestly just don’t understand the point of holding strong opinions like this. Like, materially what are you arguing for? And with whom? What does expressing fealty to this view accomplish?
2
3
46
Such sad sad news. It was such an honor working with you (again!) Igor. I literally wouldn’t be here without your support and trust. So inspiring as a leader and engineer and as a human being. Really gonna miss you 🥲
1
44
8,196
This but unironically. Something feels quite off about achieving AGI by using the RL cannon to individually tile all domains of interest. All the truly interesting things are extremely data sparse anyway.
my bar for agi is an ai that can learn to run a gas station for a year without a team of scientists collecting the Gas Station Dataset
13
1
40
4,799
Some personal news, I’m leaving @metaphorsystems . Super proud of everything we built. Really big things coming from the Metaphor team soon. Excited about exploring what’s next! 🫡
42
4,184
We all need like a week in a cabin in the woods doing nothing but catching up on ML papers and implementing toy ideas
3
2
28
3,623
Also, really makes you feel like a fucking goober complaining about work when the spaceX boys just caught a rocket with chopsticks
3
1
23
1,024
Training efficiency gains make compute advantages matter more not less
if the frontier models are commoditized, compute concentration matters even more
if you can train better models for fewer flops, compute concentration matters even more
compute is the primary means of production of the future and owning more will always be good
2
23
2,911
I have this exact thought all the time. It’s kinda terrifying the sheer volume of knowledge you forget if you’re not intentional about remembering it
Every time I fail a spaced repetition review for a card which I remember thinking was almost too trivial to write down, I become more convinced that everything I read without making cards for is a waste of time.
1
21
1,839
arxiv.org/abs/2208.07339 I think you’re thinking of the llm.int8 paper from @Tim_Dettmers
1
21
1,910
I mean, I’m curious what the alternative is supposed to be here? Having the means to eradicate material and economic suffering and choosing not to do it?
1
15
Just experienced all of this this morning. Actually kind of fascinating to watch a person make all the same kinds of errors an LLM does in realtime . Also terrifying when that person is responsible for your health.
Most doctors are like poorly trained LLMs. They naively fixate on the wrong tokens, and then are confidently incorrect as they hallucinate or confabulate, and then gaslight you based on their prior training paradigms.
2
14
1,816
if you can't explain your case succinctly to a bunch of smart people who as a matter of fact do "boring, complicated, technical" things all day, maybe it's not as sound or as certain as you might think.
12
800
For quite some time my bar for being truly impressed with robotics developments has been “can it fold laundry” . Super cool!
13
1,337
Control theory is so much more fun the second time around. Can't believe younger me thought it was boring.
1
1
14
889
Fun fact, on two separate occasions I’ve almost hit Terry Tao with my car while he was bicycling (he pulled out in front of me!) I shudder to think at the cost to humanity lol
fun story:
terry tao was on both my and my brother's committee.
he solved both our dissertation problems before we were done talking, each of us got "wouldn't it have been easier to...outline of entire proof" 🫠
1
12
1,602
I have never seen anyone so consistently elated at the mere possibility of others failing. It’d actually be kinda hilarious if it didn’t seem like such a miserable way to live. Garry is *the* ultimate hater. Like, is this his job?
OMG! The GenAI honeymoon is definitely over.
“A pharma company stopped using Microsoft's Copilot AI tool, with an exec citing high cost and low value.
The company used Office 365 Copilot for 500 staff and compared it to middle-school presentations.”
-@BusinessInsider
1
9
1,479
I don’t think it’s just you! All these explanations in the comments i think miss the point. Yeah grooves make vibrations but how *exactly* does the 3d shape of the groove/channel map to a particular frequency
2
10
2,606
Lol might just be brown noise (kinda sounds like engine noise) that’s helpful. Try working with this in the background piped.video/watch?v=-teK_6JX… I’d try that in lieu of reducing your oxygen intake like some others have suggested hahaha
123
Also have been thinking a bit about this. Seems like this kind of strategy learning should be some kind of stateful continuous running sleep-wake process. Maybe the ideal thing is to co-train this optimizer as part of post training.
2
11
5,046
We have a lot of really cool stuff in store, but I think the first step is being able to actually intelligently filter the internet at scale like this. 🤘
The latest model also has been trained to understand dates so now you can search for things like “startup announcement tweets from April 2019” or “pdfs written in the 60s about art” (5/7)
7
926
The expected range of the compression ratio isn’t as obvious to me. Whats been most surprising is all the folks making claims about “theoretical maximum” compression, as though such a thing is knowable even in principle.
1
7
265
LoRA is really great. It’s got all the hallmarks of a great paper. Effective, lucid enough to describe in a sentence or two, and simple enough to upset you that you didn’t think of it first.
9
647
Come build!!
If you were moved by this 1984 ad, you'd be training your own models.
We're hosting a hackathon this weekend (anythingbutwrappers.com) w/ @brevdev and @TheCeloCenter
The only rule is that you train your own model! Free GPUs, food & drink. So come and let it rip 🤘
8
505
Lol there’s a line in one of his books for undergrad physics that goes something like “All basis are equal, but the eigenbasis is more equal than others”
9
Whatever else may be going on, modeling the OAI folks as cogs who are motivated primarily by compensation means you understand nothing about that organization.
Salesforce will match any OpenAI researcher who has tendered their resignation full cash & equity OTE to immediately join our Salesforce Einstein Trusted AI research team under Silvio Savarese. Send me your cv directly to ceo@salesforce.com. Einstein is the most successful enterprise AI Platform completing 1 Trillion predictive & generative transactions this week! Join our Trusted AI Enterprise Revolution. ❤️
1
8
1,759
Pedagogically I suspect the row column ontology is easier to grok for the majority of people in high school /early undergrad. Einstein notation has more utility outside of that though for sure.
7
2,925
Idk understanding it from like a Fourier perspective is one thing , but seeing it’s consequences in a physical system was the “oh shit particles like frfr have wave like behavior” moment for me
5
695
Hmm I’m not sure how well this holds up. Surely human minds have many orders of magnitude more computational power than say an iPhone. The difference in speed is compensated by more parallel computation. Yet a chess engine on a phone still crushes the world’s top players.
1
7
3,627
legitimately one of the most mystical experiences of my entire life. pictures just don’t do total solar eclipses an ounce of justice.
6
871
Idk I think in the coming years we’ll see statements about being able to infinitely scale intelligence as a kind of category error. @fchollet s intuition about intelligence being a kind of conversion efficiency between prior knowledge and future OOD performance feels right. That doesn’t imply that humans are anywhere near the efficiency bounds, but it does imply that there are bounds.
Being efficiency bound, by the way, doesn’t stop you from arbitrarily scaling the tech tree since you can in principle just expend more compute to explore the search space.
Human intelligence is just a milestone on a near-infinite scale of intelligence.
Physics provides a near-infinite complexity frontier.
1
7
1,931
The next 10 years are about to be weird
Copilot moves machines closer to humans by allowing to control computer with language input.
Such an evolution started decades ago when humans upgraded punched cards with programing languages resembling English - from assembly to C to Python 🚀.
cnbc.com/2021/06/29/microsof…
6
@fchollet by far. I don't necessarily agree with him completely, but he makes a compelling case for intelligence being a kind of normalized sample efficiency. He'd make the case that deep learning as it stands won't get you the truly interesting stuff like warp drives and deep new scientific insights etc... largely because for every thing we actually care about we're deeply data limited.
5
1,027
Huh... so this is what an exponential feels like.
here is sora, our video generation model:
openai.com/sora
today we are starting red-teaming and offering access to a limited number of creators.
@_tim_brooks @billpeeb @model_mechanic are really incredible; amazing work by them and the team.
remarkable moment.
6
633
Machine learning models typically use a fixed amount of compute at inference regardless of how complex their inputs are. Can we make more effective models by deliberately leveraging more compute at test time? I explore this and related things here piped.video/8iz5v3Q0g9I
2
1
6
If you haven’t signed up, there’s still time to get in the Anything But Wrappers hackathon tomorrow! partiful.com/e/TahYqVw1D7jJU… First prize includes a week of free compute on one our 8X80GB A100 Nodes 🔥
ANNOUNCING PRIZES AND SPONSORS👇
Hackathon tomorrow where the only rule is you build your own model. Don't know how? Come anyways and we'll show you 🤙
🥇1st place🥇
👉1 week of free training on one of @metaphorsystems 80GB 8xA100 nodes
👉$2K of @brevdev credits
1
6
821
The timescales of atmospheric blowoff are so ridiculously long as to make this a complete nonissue for any civilization capable of creating at atmosphere in the first place
5
I’ve been saying something similar in private as well. I think any methods that aim for formal guarantees on model behavior are likely doomed (to be clear still worth studying!). Not quite sure if I would lump MI in this bucket though
1
5
428
Wait what do you mean took 100-1000x longer to evolve? Animals have had fine motor skills for hundreds of millions years. T-Rex’s and Raptors had hella good navigation and world manipulation ability, talking and painting are new.
1
5
The data processing inequality continues to do untold harm to our generation. When it comes to LLMs it's almost always irrelevant
2
6
979
🤯Don't mind me...just updating my timelines
Gato🐈a scalable generalist agent that uses a single transformer with exactly the same weights to play Atari, follow text instructions, caption images, chat with people, control a real robot arm, and more: dpmd.ai/Gato
Paper: dpmd.ai/Gato-paper 1/
1
5
I’ve been thinking about the same thing! I wonder if there’s a way to do it with minimal trust on either side. That is fine tune on gpt-4 using your LoRA weights without being able to recover gpt’s weights
4
114
Lol my favorite of these is "James while John had had had had had had had had had had had a better effect on the teacher"
3
101
A little more on the nose but if you take a linear probe to a LM thats been trained to predict the shortest path between U.S cities just given the name of two cities, the probe will reveal the LM has learned a map that’s isometric to the true map of the country
2
4
Jesus.. it's been like 10 minutes since the last one
Phenaki: Variable Length Video Generation from Open Domain Textual Descriptions
abs: openreview.net/forum?id=vOEX…
project page: phenaki.video/
Generating videos from text, with prompts that can change over time, and videos that can be as long as multiple minutes
4