My Notes:
- “Software 3.0”
Software 1.0 is code, 2.0 was weights of tightly scoped ML models, 3.0 is using English to program LLMs
Just like how in certain use cases software 2.0 would “eat away” at software 1.0 code, software 3.0 will now eat away at a lot of places where we would previously have used 1.0/2.0
- LLMs are like ___
LLMs are kind of like power stations: they require infrastructure, have switching stations (open router) and cause “brownouts” during disruptions in output
LLMs are unlike traditional utilities though and more like software. They can easily be duplicated and transferred.
LLMs also resemble operating systems: they memory, external devices, networking, etc
Similar to the OS space there are a few major closed providers (Mac / windows ≈ gpt / Claude) and a more widespread ecosystem of open source options (Linux ≈ llama).
- We are currently in the 1960s shared computing moment for AI
Just like traditional computing in the 60s, current frontier AI is expensive and requires immense centralized compute, accessible only over the network, and shared by others when you’re not using it.
What will the PC moment for AI look like?
- The anatomy of a successful AI empowered app
Sees humans as a fundamental part of the loop- 1000 line Devin PR still needs to be reviewed. The best AI empowered apps provide the best experience for their human users by providing: a custom GUI (escape just textual interfaces) + an autonomy “slider” (tab -> cmd K -> cmd L -> cmd I)
Really likes the Iron Man suit analogy: the suit is 2 parts:
Augmentation: giving the user strength, tools, sensors and information
Autonomy: the suit at many times has a mind of its own- taking actions without being prompted
How can we design AI products that follow these patterns?
- Rate of progress
Seemed very conservative on the rate of AI progression. Cited this example of when he rode a Waymo prototype around back in 2014 and had a 15 minute drive with zero interventions. He thought “wow we’ve done it, it’s here”. And while we’ve made a lot of progress since then, there’s still kinks to work out. And we will, but it just takes time.
Where `works` is a boolean array of situations where your product “works” or not:
A demo is simply `works.any()`, but building a real product is `works.all()`
Sees AI in a similar way: explicitly said to avoid thinking like AI 2027 or saying 2025 is “the year of agents” and instead to look at 2025-2035 as the “decade of agents”
- Vibe “coding” is the easy part and the rest is still hard
Cited example of vibe coding a simple app: cursor wrote most of the code pretty easily. But after that the AI speed ups disappeared: setting up auth, getting environment variables, deployment, registering and assigning a domain, etc
Docs are a really important part of this. Explicitly cited Clerks docs which had a 5-10 step long list of instructions for where / when to click. After showing this he literally said “What the hell?”
Cited lee rob / vercel adding curl commands to docs where “click” was before as a good first step
- AI reverse flow of adoption
Previous major advanced technologies (computing, internet, GPS) stemmed from government / large corporation use cases and slowly made their way to consumers.
AI so far has had the opposite flow. Consumers are using the tech for the most random mundane things while governments haven’t even started to adopt AI.
Wow fantastic talk from
@karpathy at startup school
Unbelievably insightful and pragmatic