A battle for AI is unfolding before our eyes.
AI compute is being pushed toward edge devices. Small models are getting better every single day. Just this week, multiple exceptional small models were released.
On the other end of the spectrum, OpenAI is pushing AI cloud compute to the point where Sam Altman says it'll be so cheap it won't be worth metering.
Chip companies are making massive gains with on-device AI that can run faster and more efficiently.
The future will be:
95% of consumer AI compute will be executed on-device. For the final 5%, the fastest and cheapest cloud AI provider will be used.
Developers will build AI apps for the next few years using the fastest and cheapest cloud AI provider.
Because LLMs will be a commodity, most AI developers won't want to spend their time building their own AI infrastructure. They will want whatever is the most efficient (cost/speed) and most consistent.
Eventually, AI developers will tap into the already available on-device AI models.
I am a firm believer in on-device AI, here's why:
* Massive algorithmic gains are happening every week (example: MoA) that allow small models to perform nearly as well as frontier models
* On-device AI is essentially free
* Running models on-device, with chips that have been honed for decades for mobile devices and thus are incredibly efficient, will reduce energy usage by orders of magnitude
* The best use case for consumer AI - AI assistants - requires giving AI access to personal information, which is the most private and secure when you don't have to send that info to 3rd party services
* Latency makes or breaks the experience with AI, especially for AI assistants. Latency will always be faster on-device because there's no requirement to ping a 3rd party server
That DOES NOT mean there won't be value for cloud AI providers. In fact, there will be tremendous value. The fastest and cheapest providers will win for the use cases that just cannot be run on-device.
For high-scale use cases, which typically are performed at the enterprise level, cloud providers will be insanely valuable. Spinning up thousands or even millions of cutting-edge AI agents to do the work of entire organizations can only be done in the cloud.
Want to know the best part of all of this?
Consumers win big :)