Building everyone's favorite AI infrastructure platform @modal
🇸🇪 → 🗽
- Tweets 13,344
- Following 3,855
- Followers 55,466
- Likes 36,617
Someone needs to write a blog post about the stacks of Chinese tech companies. Feels like there's a lot of software/techniques that are widely used in China but not so much elsewhere.
94
164
3,645
541,275
Estonia tenbagging its economy in 25 years is pretty remarkable. What’s the secret?
466
152
3,061
1,055,851
If you rank every major software innovation (OOP, containers, version control, deep learning, consensus algorithms, asymmetric encryption, whatever...) by the value it created, the relational database would come out at the top and it's not even close
101
206
3,022
Excited to announce I’m a guinea pig in an A/B experiment at Drizly. Please promote the product manager who is measuring my price elasticity for vodka at 2.30pm on a Monday.
13
59
2,530
173,304
A show I would seriously love to watch is one where a Ph.D. in queuing theory (or operations research) goes to random places like airports or hospitals or whatever and figures out tricks to optimize flow.
79
101
2,122
This country should bring in 10x more H-1Bs and also make the pathway from H-1B to permanent residency 10x easier
141
130
1,946
* There are 100M knowledge workers in the US
* That's $10T/year in salaries
* My new startup can make them 10% more productive
* Even if we only capture 10% of the value, that's $100B/year
* We're selling coffee
* Invest in my coffee startup at a $1T valuation
67
51
2,089
204,532
It’s pretty crazy how inefficient most CI setups are. We’re spending 99% of the time reinstalling 99% the same dependencies as last time and recompiling the same code as last time etc. How did we end up in this situation?
109
118
1,742
Blog formats ranked by reader friendliness:
1. Self-hosted blog
2. Github Pages
...
57. Pinning theses on a church door
...
275. Medium
33
184
1,624
This might be heresy but:
1. Code reviews are a massive productivity tax with tiny quality benefits
2. They should not be mandated
3. The author should feel free to request a review if they want it
4. If you don't trust your engineers, invest more in CI, or hire better ones
196
112
1,669
622,269
Genuine question: what happens when they run out of letters in the alphabet?
We’re excited to announce that Databricks is raising a Series K investment that values the company at >$100 billion.
We’ll use this new capital to accelerate our AI offerings for customers, including expanding Agent Bricks and investing in our new database offering, Lakebase. Our goal is to make it easier for companies to build AI apps and agents - on their data.
Thank you to our investors who continue to support our growth and vision.
databricks.com/company/newsr…
193
24
1,638
222,958
Software is over. No point coding anymore, we should all just stop.
Unrelated question: how do you promote a tweet so that all your competitors definitely see it?
38
52
1,547
85,857
Google should spin out its TPU team into a separate business, retain a big stake, and have it go public. Easy peasy way to make a bazillion dollars.
57
43
1,385
136,024
Went to a founder dinner and half the people there were ex-Palantir. What kind of founder crack do they put in the water there?
49
34
1,351
520,795
Something that old boring process literature talk about (Toyota, The Goal etc) that applies to software engineering 10000000% is that minimizing the size of the feedback loop is much more important for productivity than minimizing idleness
24
197
1,313
If the speed of light was 10x faster then we’d probably just have one big data center in the world doing all the compute and if it was 10x slower then we’d probably not have data centers at all. Thanks for attending my TED talk.
40
36
1,198
131,392
Ctrl-R (reverse-i-search) in the terminal is something that I think made me like 15% more productive as a software engineer when I learned it maybe 10 years ago. Sharing it in the hope that at least one follower learns a new trick :)
38
124
1,170
Why software projects take longer than you think – a statistical model erikbern.com/2019/04/15/why-… (blog post)
41
419
1,133
Sure, there’s H-1B abuse, but there’s a tradeoff curve here, right? Going all the way to a $100k fee feels like way past the point of curbing most abuse, and well into the territory of losing out on a lot of great talent.
477
54
1,123
175,522
Had my most genius idea ever yesterday and paid my kids $1 for every toy they got rid of. Cost me $140 but so worth it.
49
15
1,149
90,846
Twitter laying off 75% obviously doesn’t have a control group, but it strikes me as an experiment that a lot of tech execs and board members will be paying close attention to
63
70
1,063
It's true – @modal has raised a $87M Series B at a $1.1B valuation to advance the future of AI infrastructure.
Thank you to @Lux_Capital, @Redpoint, @AmplifyPartners, and others.
Now more than ever, AI demands a complete reinvention of traditional compute infrastructure
140
81
1,127
858,172
Insane flex tbh.
New personal goal: hit 3 commas so I can build a new git out of spite.
30
17
1,083
164,249
People saying don’t use deep learning when logistic regression works aren’t nearly going far enough... 90% of it should be at most a SQL query and a scatter plot, look at it and pick a cutoff or whatever, then move on to the next problem.
32
75
817
Working on some insanely fun tech right now. I go to bed thinking about it and I wake up thinking about it. It’s maybe the 2nd time in my career I’ve felt this way (1st was building Spotify’s music recommender)
19
5
846
Serious data question: how do companies deal with dashboard rot? Everywhere I've worked, 95% of all dashboards are half-broken or not trustworthy because no one has touched them for 6+ months. Are dashboards inherently ephemeral?
97
55
827
This is Spotify's Hadoop cluster in 2009 (with @jooon). My matrix factorization job would crash when the machines overheated so I fixed it by putting up a curtain. Prob my best bugfix ever.
6
180
769
A bit nervous some large % of the economy is held up by GPU depreciation napkin math
38
37
818
94,523
The funniest beneficiary of the AI boom must be the tiny island Anguilla? Napkin math, but .ai domains is something like ~25% of their GDP at this point.
13
48
769
91,961
Is it just me or is it kind of funny that LLMs throw everything about fast data structures out the window and is like… every query is O(n), let’s just slap a mega fast GPU on it and it works
41
18
767
162,243
Hypothetical question, inspired by current events: if you’re a CEO and you tell a company: don’t acquire us, just hire me and my top people for half the price. That’s a breach of fiduciary duty to the shareholders right?
28
12
781
86,930
I think this specialization of data teams into 99 different roles (data scientist, data engineer, analytics engineer, ML engineer etc) is generally a bad thing driven by the fact that tools are bad and too hard to use
58
64
723
I wrote a blog post about how to build at data team in the form of a story (and it ended up being about 4x longer than I anticipated): erikbern.com/2021/07/07/the-…
26
140
727
I’m bullish on LLMs for code but this is kind of a funny/weird counterpoint that maybe it makes developers chose worse tools
78
44
699
182,046
Bias for action is dumb advice because opportunity cost is always super high. Better to drag your feet for a bit until you know what's the 10x thing.
57
32
618
47,711
Is there any benefit of using Conda today? Pretty much everything exists as a wheel and pip is much faster.
111
25
593
246,202
I love database companies. For a lot of startups I don’t really get the technical innovation. Databases though… the whole thing is basically crazy data structures, distributed systems problems, parsers, query planners etc.
15
39
598
If I had a business doing $1B/year in revenue with thin margins and lots of competitors, I would also go to DC and beg regulators for protection
13
40
590
77,814
My new rule is you can only dunk on Python after acknowledging that (a) people pick the language that makes them the most productive (b) Python is the most widely used language in the world
44
36
567
I have a feeling most companies would benefit from thinking of data scientists not as crazy math professors but as journalists finding the scoop in the data: dumb shit the company has been dropping the ball on.
7
94
554
We hired a head of biz ops as the first non-eng hire and a few weeks later she's writing a bunch of code like the rest of us
20
11
569
170,778
My dumbest tax idea is to get lots of cats and set up a pet trust (which is a legal thing in the US) for each one of them and invest it into qualified small business stock (QSBS) which is exempt from cap gains tax.
32
42
565
Password-less login flows are 100x better UX for infrequently used websites (e.g., magic links to your email, or "enter the code we just texted you"). The world would truly be a better place if every service adopted this.
50
32
555
123,670
My advice to people thinking about building some sort of “run code in the cloud” startup: only do it if you’re genuinely excited about spending 5y+ thinking about load balancing, page faults, cloud security, DNS, …
26
25
522
150,482
Hard work is cool but do you know what really makes startup successful? A bunch of people willing to pick up whatever task has to get done that’s potentially way outside their comfort zone.
10
52
510
47,065
ScaNN (Scalable Nearest Neighbors) interesting new approximate nearest neighbor library from Google which seems to beat SoTA github.com/google-research/g…
7
109
508
One simple rule of thumb I use when I write code is that each commit is either a pure refactoring (ie no change in code behavior), or a pure change in behavior (but otherwise nothing else). But try to never mix the two. Sometimes you can apply this rule on a PR level as well.
9
46
479
Lotteries for vaccines seems like a brilliant idea. I suspect a lot of people are afraid of vaccines because they overestimate small probabilities, so let’s give them something that’s valuable to a person who overestimates small probabilities: a lottery ticket.
7
40
476
I wrote a blog post about how I think the cloud service stack will change in the next 5-10 years: erikbern.com/2021/11/30/stor…
54
78
489
A lot of the engineers with the deepest experience that I work with also have an absurdly wide span: everything from memory allocators to CSS selectors.
They just like to build cool stuff, and that’s why they picked up so many things.
"Hiring pure backend engineer and expecting them to do non backend stuff IMO is wrong"
Sigh.
Any capable intern/new grad picks up whatever new technology is needed to get the job done.
If you, as an *experienced* engineer, refuse to do so: you're less capable than an intern
19
27
495
40,774
Indexing your data sets and documenting it is another example of a problem lots of companies have but there’s no great tool for it, so everyone builds their own. Here’s Shopify’s: engineering.shopify.com/blog…
7
80
489
A lot of people are asking me what I'm working on. I'm not ready to share, but let me just mention one thing: we're spending $10,000/day on liquid nitrogen right now.
54
11
481
Why can't databases analyze queries and add indexes automatically? Feels like it could be good enough for 95%+ of all practical situations.
81
16
461
I honestly think the peak startup ability age is like 45+ and we would benefit from having this reflected in the cultural idea of what a startup looks like
People tend to think startups need a bunch of young folks but the elite move is hiring a bunch of seasoned professionals and a semi-retired industry veteran genius how knows crazy math and can tell stories of industry.
25
29
454
108,287
Me talking to VCs: Datadog is such an amazing company with a fantastic product and such an impressive ability to increase revenue
Me when I get my Datadog invoice: wtf??? who are these people running this racket???
34
12
470
157,472
The eigenvector of "Why we moved from language X to language Y"
erikbern.com/2017/03/15/the-… (blog post)
26
336
446
I’m soo psyched to announce this one. We have raised again!
Raised the bar for AI inference. By snapshotting GPU memory, we can cold start containers running vLLM or many other things extremely fast. This will be a game changer for many customers.
We just launched GPU memory snapshotting on @modal_labs in alpha. Speed up cold boots by up to 12x 😇
If you're deploying AI models, a huge amount of cold boot time comes from loading model weights into GPU memory. This makes it difficult to scale GPU resources up and down without compromising on user experience.
GPU snapshotting solves that.
14
28
477
38,102
I always fall asleep during takeoff and I just realized it’s because of high carbon dioxide levels? Mildly disconcerting.
24
25
444
For all the talks about the state of higher education in the US, the list of Nobel winners by university since 2000 is pretty impressive – clearly this country is doing *something* right.
36
39
448
Super controversial data opinion: I don’t think Python is significantly harder to learn than SQL. What’s harder is all the stuff around it (setting up a dev environment, using an editor, version control, deploying) vs in SQL all of that is mostly taken care of.
49
24
454
I just wrote a bit about what I'm working on and why I'm excited about it: erikbern.com/2022/12/07/what…
23
42
442
Playing around generating fonts with a deep neural network: erikbern.com/2016/01/21/anal… nitter.app/fulhack/status/6899762…
13
273
424
It's always confusing to me when some companies are paranoid about source code theft. I barely understand code that I wrote myself years ago. Good luck to the thief trying to make any sense out of 1M lines of Java stolen from XYZ Bigcorp Enterprises, Inc.
35
28
427
68,276
In the long run, I expect most/all of developers doing things locally will go away. Developing, testing, building, running, deploying, etc. When developers need to run things locally today it’s a sign that cloud tools aren’t there yet imo.
70
21
427
Scatterplot "trick" that I like: use the alpha channel if you have overlapping points. Makes it a lot easier to judge the density of points.
11
78
415
Vector (approximate nearest neighbor) databases are fairly underground still, but I think they could go super mainstream if you combine them with embedding models so people can just insert (multimodal) data and query without having to think about vectors.
33
41
430
Got my 2nd Moderna shot today. No side effects but I spent the whole day installing Windows XP on all my computers for some reason.
5
18
404
Looking back at my early career I think a massively formative experience was spending 1y+ solo unsupervised in my mid 20s building a complex uncertain project from scratch (Spotify's rec sys).
15
8
408
94,933
I have zero data, but it feels like there’s been a prestige shift away from technical management towards writing code in the last five years. Ambitious young people seem fairly indifferent about the management track. I think it’s mostly a good thing?
45
6
421
34,958
Up until recently it was just me hacking on prototypes, but my company is starting to feel real. We are 4 people, have a real physical office, and we just closed a seed round with @AmplifyPartners – looking forward to share a bit more about what we are working on soon!
34
6
407
Pretty crazy result from the machine learning freaks at @modal_labs: transcribe one week of audio in one minute spending one dollar!
18
22
431
35,381
Let's say you infiltrated an evil megacorp by getting hired as a CTO, and you want to thwart its output as long as you possibly can (without getting caught/fired). What would you do?
201
40
386
Thinking about tools and the value they create and imo “what use case does this unlock” is 100x more valuable to answer than “what problem does this solve”
18
45
380
A good engineer _could_ make a fortune starting a competitor to Okta ($13B market cap, mediocre product), but then they would have to dedicate 10 years of their life to SAML and that seems miserable
25
12
374
58,417
The holy trinity of serverless GPU has been achieved internally at @modal_labs:
✅ Custom filesystem optimized for container cold start
✅ CPU snapshot+restore
✅ GPU snapshot+restore
Last one is working internally but not released yet – stay tuned!
17
24
405
40,392
Spotify made a (imo) brilliant architectural decision early and made it super easy for any service to emit machine readable events with a proper schema. This was the basis of the whole analytics stack. I think this is rare though? Do we need better tooling for it?
26
37
368
I asked DALL-E to generate a picture of the most promising data infrastructure startup in the world – it's pretty creepy how accurate it is
21
14
376
When a software engineer says “that’s not necessarily true”, it means someone just said something incredibly stupid
9
15
377
21,087
Just found out there's a whole community of people writing bots that buy exclusive sneakers (there's lots of repos on Github). Very curious about it. What's the modern sneaker stack?
46
13
377
All the VC/founders musing about doing rollups and injecting AI into inefficient service jobs are going to learn this the hard way. I hope they enjoy spending 5 years learning about mortgage servicing compliance regulation or whatever!
19
20
384
132,657
Most popular workflow schedulers:
1. crontab + various scripts
2. a business analysts runs it every morning
3. Windows Task Scheduler
4. Jenkins
5. Airflow
26
30
351
One of the best and worst parts of building software is that building a prototype is like 5% of the work of building something that runs at scale in production
17
39
366
It’s impressive how eerily similar ChatGPT sounds to some toxic people I worked with – the ability to come up with a confident answer to anything on the spot.
20
17
348
A bunch of people have internalized that it's normal and cool that you have to use 1,500 different products to build your data stack. I think it's wrong!
35
30
353