Asst. Prof. in OID @Wharton @Penn. Cofounder @workhelix. Everyone can just do stuff and that's {good, bad}. I study the economics of AI.
Philadelphia, PA
- Tweets 2,785
- Following 2,212
- Followers 5,755
- Likes 15,855
Really excited to see our paper w/@manlikemishap @thankyourniceAi @sj_manning out today in @sciencemagazine. Link here: science.org/doi/10.1126/scie… tl;dr: Studying where LLMs might affect workers, we find that GPTs are GPTs! But what does that mean and how did we do it? 🧵
15
97
321
147,473
Deeply grateful to get to be part of this group and to have the chance to work on these projects! Thank you Schmidt Sciences! And a big thank you to my advisors and coauthors (esp @erikbryn!)
Nineteen people were named AI2050 fellows by Schmidt Sciences. Among them is @Wharton Professor @danielrock. Rock will measure the impact of AI on job creation and technological dissemination, which will lead to the creation of an open-sourced LLM: whr.tn/4ad9nE5
18
8
110
19,290
We're hosting a conference on AI and the Future of Work here at @Wharton! Conference is May 22-23.
If you have a paper you think is a good fit, please submit it by 2/26.
link here: ai.wharton.upenn.edu/events/…
conf details here: ai.wharton.upenn.edu/events/…
2
34
84
20,339
I don't think people are excited enough about using language models to design antibodies: biorxiv.org/content/10.1101/…
Perhaps I don't understand reasons to NOT be stoked that this is possible?
6
12
49
17,878
Using uv from @astral_sh and @charliermarsh... all I can say is wow. This would've taken forever in conda. HT @EconoBen.
cc @jeremyphoward I'm sure you're using this or something like it already but in the small chance you aren't it might make your day.
4
2
43
5,665
kicking off the AI and the Future of Work Conference here @Wharton @AIatWharton. I'll be live tweeting this afternoon's session B. First up:
Anders Humlum and Emilie Westergaard's work on "The Adoption of ChatGPT"
2
11
45
6,476
It’s unreasonable to expect the gains from AI immediately. It takes R&D! Great read from @emollick:
oneusefulthing.org/p/latent-…
1
12
44
14,146
Interested in the headlines and economics surrounding SVB, crypto, AI, and the law? Come work with @GrennanJill and me this year researching emerging technologies! We're hiring a Pre-Doc in either the SF Bay Area or Philadelphia to work with us. cc @econ_ra
2
15
39
11,933
Join us tmrw at noon ET for the ABFR Webinar. I'll present joint work with @ThankYourNiceAI @sj_manning @ManlikeMishap on the labor market impact potential of LLMs.
And a big thank you to @DrDaronAcemoglu for discussing the paper!
Link here: abfr-forum.org
1
12
34
23,987
We don't find evidence that AI is going to wipe out all labor (at least not without further breakthroughs). Think of jobs as bundles of tasks (@DAcemogluMIT @davidautor @erikbryn) We find ~80% of workers have >10% of their tasks "exposed".
1
9
32
3,585
Learning a ton at this conference! Stunned to see the amazing things ML researchers are building. @kchonyc @egrefen @drnigam @zamir_ar @EPFL_CREATE @ABosselut
We are now welcoming Prof. @danielrock from @Penn, who will be talking about the impact potential of Large Language Models on the labor market.
#AMLDGenAI23
2
7
29
5,954
Looking forward to discussing our project on the labor market impact potential of large language models.
Exciting upcoming Webinar by @danielrock @Wharton on his joint paper w/@OpenAI researchers @ThankYourNiceAI @sj_manning @ManlikeMishap
80% US workers could see 10% of job impacted by LLMs, 19% could see >50% impact #EconOfAI #EconTwitter
Mon, May 15
9:00am EST
Zoom info👇
1
7
30
10,725
This thread is so good. Qualitative researchers are a cheat code for any quantitative researcher. They’ll teach you what to study and why.
One day I will write something longer about the value of qualitative research methods for quantitative (esp. applied micro) researchers. But 👇 is a short thread
TLDR: Qual methods super useful even if you never present/analyze qual data in published work
#ResearchMethods
2
3
26
5,416
Starting a session at #AOM2024 on AI adoption and productivity. Come join us!
Marriott Grace room (4th floor)
2
24
1,525
A little positivity for this Tuesday: Really impressed with the students in my @Wharton MBA digital economics class. What an insightful group with fascinating experience.
24
1,566
Now @AIatWharton we have @ocolluphid presenting work with @malsobay @eamanjahani @daveholtz @ssuri @CNicolaides and Joe Zhang: Early Experimental Evidence on the Behavioral
Dynamics of Prompt Engineering
Do models matter more than prompts? What about prompting strategy?
2
5
24
2,262
We found researchers, scientists, and technologists had a lot of potential change. If our R&D is more productive with LLMs, it could change the direction of innovation instead of just helping us to do what we do now. We're already seeing ML tools assist scientists. It's exciting!
1
9
24
2,721
How long before spirit airlines is buying up Boeing airplanes and charging you to not sit next to the exit door
1
23
4,020
everyone should follow @BenSManning. Doing some of the coolest work around on LLMs. Maybe he'll automate our research too w/ @Kehang_Zhu @johnjhorton
4
21
1,888
For students/postdocs on the IS market, this looks like a great tool: isjobs.xyz/Database-23-24
Thanks @chzenan for a useful public good!
2
3
21
5,709
Broken recording this for sure but everyone should give the latest class of models a shot for some of their work tasks!
The methodology isn't 100% clear, but it seems Google solicited hard problems in various jobs, created & ran prompts for them on Gemini, and then asked experts how much time could be saved on the tasks based on the results. Estimates ranged from 26% to 75% storage.googleapis.com/deepm…
3
2
18
4,967
Really underrated risk as our workforce ages and retires is lost expertise. Particularly salient for areas like nuclear power technicians.
Gonna guess it’s probably not as much of an issue in high wage high prestige contexts like investment banking or management consulting.
2
1
16
1,899
One of the reasons to be excited about how LLMs can help in research is that they augment humans in doing the most important components of research work (described well by Jeremy in this thread!):
For those that hope (or worry) that LLMs will do breakthrough scientific research, I've got good (or bad) news:
LLMs are particularly, exceedingly, marvellously ill-suited to this task. (if you're a researcher, you'll have noticed this already)
Here's why🧵
4
16
2,377
NEWS: Announcing @WhartonAIAI, a new interdisciplinary initiative focused on the application of AI and analytics to fuel business innovation: whr.tn/ai-analytics
The initiative includes:
🔷 Curricular enhancements
🔷 Investments in new research
🔷 Collaboration with @OpenAI
18
1,193
What happens when a new technology helps you accelerate science?
bloomberg.com/news/articles/…
Math, chemistry, biology... lots of dominos falling.
1
2
16
4,278
Exposure means potential change, neither good nor bad without more info. Many workers will be augmented in completing tasks. A small set of tasks might be automated (more below), but we don't see many full jobs vanishing.
1
14
2,299
Our big question is whether generative AI is a "general-purpose technology" (@timobres), what economists call technologies with 1) pervasive impact, 2) improvement over time, and 3) "innovational complementarities"... they spawn and require complementary investment (more below).
1
2
13
1,964
One fun additional result is absolutely explosive growth in open source development when ChatGPT was launched. Check out the growth in forks for LLM-related repos. @raveeshmayya @doron_yev @gal_oestreicher @SarahHBana @GurbaxaniVijay
1
2
13
1,006
@BenSManning is up here talking about an instant classic paper now @AIatWharton. "Automated Social Science: Language Models as Scientist & Subjects" with @Kehang_Zhu and @johnjhorton.
benjaminmanning.io/files/rs.…
1
3
15
944
That this works is fascinating but also hugely useful.
In-between solutions probably work too. Quantize data to arbitrary complexity /
Compute trade offs —> manage accuracy and speed in vector search
TIL about binary vector search... apparently there's a trick where you can take an embedding vector like [0.0051, 0.017, -0.0186, -0.0185...] and turn that into a binary vector just reflecting if each value is > 0 - so [1, 1, -1, -1, ...] and still get useful cosine similarities!
1
2
16
2,323
We use the US govt O*NET taxonomy. It lists out ~20k tasks that people do at work, so we score those tasks according to a rubric. Basically for each task, could you cut the time it takes to do that task in half with no drop in quality?
1
1
14
1,513
Reminder: Today's the last day to submit for our Future of Work Conference!
We're hosting a conference on AI and the Future of Work here at @Wharton! Conference is May 22-23.
If you have a paper you think is a good fit, please submit it by 2/26.
link here: ai.wharton.upenn.edu/events/…
conf details here: ai.wharton.upenn.edu/events/…
6
16
3,517
Want to recommend folks follow @simonw if they don't. He regularly blogs super useful content related to LLMs. Here's an example: simonwillison.net/2024/Apr/2…
2
1
15
1,585
uhh is crowd work toast?
One of our key sources of human data is no longer fully “human"!
We estimate that 33-46% of crowd workers on MTurk used large language models (LLMs) in a text production task - which may increase as ChatGPT and the like become more popular and powerful.
arxiv.org/abs/2306.07899
3
1
11
2,066
So in a sense we're a bit stuck! We need more research to track the labor market. If AI is like other GPTs (steam, electricity, compute, etc.), large shifts are coming but we also have limited capabilities to predict them. We can measure this as it happens, even using LLMs.
1
2
13
888
Next up is @mrfrank5790 presenting work with @estebanmoro and YY Ann: "AI Exposure Predicts Unemployment Risk"
Can we find job disruption in places where there are higher levels of exposure to AI tech?
fascinating work here: arxiv.org/pdf/2308.02624
4
5
14
1,249
Which means lots of change… as much as we'd like to forecast what will be automated, such predictions are in general very difficult at this stage. As many have noted, "exposure" only gets you so far. @ProfNeilT @Fleming_Martin
1
12
941
Okay so if this works well… what if a study is far away from what LLMs predict?
🚨New WP: Can LLMs predict results of social science experiments?🚨
Prior work uses LLMs to simulate survey responses, but can they predict results of social science experiments?
Across 70 studies, we find striking alignment (r = .85) between simulated and observed effects 🧵👇
3
2
15
2,088