Professor, researcher, maker of things Book: ATLAS OF AI Latest: CALCULATING EMPIRES calculatingempires.net | MSR-NYC | @USC | Knowing Machines
- Tweets 2,258
- Following 2,560
- Followers 82,138
- Likes 26,260
Pinned Tweet
🎉 Launching online today:
"Calculating Empires: A Genealogy of Technology and Power since 1500."
Dive into our large-scale interactive visualization exploring how technical and social structures co-evolved over five centuries. 🧵 calculatingempires.net/
72
544
1,483
158,726
"If you think technology will solve your problems, you don’t understand technology — and you don’t understand your problems."
Laurie Anderson, 2020
60
988
4,564
• This AI does not "read children's emotions"
• There are not seven universal emotions
• Facial muscle data does not correlate reliably with a person's inner state
Covid is being used as a pretext to install this in remote classes, then keep it.
edition.cnn.com/2021/02/16/t…
49
1,016
2,842
ChatGPT strikes again. A journalist contacted me to research her profile on @lexfridman. ChatGPT informed her that @_KarenHao and I were his top critics. It cited articles we'd written about him, gave links, and summaries. Only problem: it's all false. Here's what she sent me:
117
659
2,913
1,584,031
Umm, anyone a little concerned that Bard is saying its training dataset includes... Gmail?
I'm assuming that's flat out wrong, otherwise Google is crossing some serious legal boundaries.
Community note
Alphabet & Google have confirmed that this is an error and Bard's dataset is not using GMAIL data.
nitter.app/GoogleWorkspac…
searchenginejournal.com/google-bard-tr…
appleinsider.com/articles/23/03
130
366
2,892
956,943
Stunning: new rigorous evidence shows that wellness programs have NO significant impact. Apart from this: they massively increase the surveillance of workers, which we show here. @iajunwa @Lawgeek nytimes.com/2018/08/06/upsho…
41
1,125
2,006
Meanwhile, a new attack uses ultrasonic commands to make Siri & Alexa do anything, but no human can hear it. fastcodesign.com/90139019/a-…
60
2,212
2,054
We saw this coming, and here it is. Endless trapdoors ahead: data inaccuracies, intentional gaming, constant intimate surveillance 24/7, data breaches that will be infinitely worse, &c...
John Hancock, one of the oldest and largest North American life insurers, will stop underwriting traditional life insurance and instead sell only interactive policies that track fitness and health data through wearable devices
43
1,125
1,688
This is HUGE. And it's all thanks to the thousands of Google workers who came together to refuse to work on AI for military programs. buzzfeed.com/daveyalba/googl…
20
610
1,669
Time for a collective eyeroll from all of us who actually research this issue. @AOC was talking about facial recognition systems. They're proven to have bias issues. But this is how it gets reported.
51
332
1,685
Thrilled to launch a big project today: ANATOMY OF AN AI SYSTEM. It's a large map & long-form essay about Amazon's Echo, and the full stack of capital, labor, and natural resources used in AI. It's a collab with @TheCreaturesLab, who is a visual genius ✨ anatomyof.ai/
31
698
1,623
Let's talk about the 10%. What if the AI surgeon was trained on data that oversampled white men (as per many controlled trials)? And it consistently produces worse outcomes for Black people and women? Seems like it matters who "you" are in this hypothetical 🤔
Suppose you have cancer and you have to choose between a black box AI surgeon that cannot explain how it works but has a 90% cure rate and a human surgeon with an 80% cure rate. Do you want the AI surgeon to be illegal?
25
307
1,499
So Facebook is deleting one billion facial recognition scans, but it's keeping DeepFace, the model that is trained on all those faces. Note that "the company has also not ruled out incorporating facial recognition into future products." Very meta. 👀
Breaking News: Facebook plans to delete face scan data from over 1 billion users, shutting down a facial recognition system that became a privacy headache. nyti.ms/3CEwF5r
34
580
1,492
This is VERY bad idea. Why should we be forced to submit to facial recognition just to buy groceries? So brands can test if their ad campaigns work? Umm no thanks. This invades your physical privacy and profiles you, for bad data. theatlantic.com/technology/a…
62
549
1,184
This tweet was 5 years in the making: #AtlasofAI is out TODAY!
It’s a book on the politics and planetary costs of artificial intelligence as an extractive industry— consuming natural resources, labor, and vast quantities of data. 👉 bookshop.org/books/atlas-of-… @yalepress (1/6)
48
281
1,222
2020 has been a strange and hard year, and like many of us, I'm caring for family who are seriously ill. After 5 years of building & co-leading AI Now, I’m stepping down to make room for my next chapter. I’m so proud of what we accomplished together and hopeful for the future. ♥️
92
16
1,166
This has to be the most stunning bookshop I’ve ever wandered into. Converted from an old theatre, the box seats become reading zones 😍
49
218
1,229
Guess what? Study shows that self-driving cars are better at detecting pedestrians with lighter skin tones.
Translation: Pedestrian deaths by self-driving cars are already here - but they're not evenly distributed. jamiemorgenstern.com/papers/…
33
665
1,142
My latest for @Nature: AI's environmental costs are soaring. The new energy-hungry models for video, text, and image could create an energy crisis - and impact drinking water reserves. We urgently need action from industry, researchers, and legislators. nature.com/articles/d41586-0…
17
529
1,073
186,024
This is about to get really weird, folks nitter.app/vicenews/status/834814…
23
699
1,035
My NIPS talk 'The Trouble with Bias' is now up on the 'tubes. About the politics of classification, and limits of focusing on allocative harms not representational harms. The desire for a quick 'technical fix' to bias could actually do more damage. piped.video/watch?v=fMym_BKW…
17
479
1,088
Wow - the human/AI interface is about to get weirder.
US Copyright Office just ruled that works "autonomously created" by algorithms cannot be copyrighted. This is the work they rejected:
46
293
1,047
These are *actual* notes from Peter Thiel's class at Stanford on how to run startups, featuring VPs from Palantir and PayPal. And it gets worse from here.
44
416
974
It happened: Facebook just went off the deep end in Australia. They are blocking *all* news content to Australians, and *no* Australian media can post news.
This is what showdowns between states and platforms look like. It's deplatforming at scale.
about.fb.com/news/2021/02/ch…
116
512
978
I have a piece in @nature today on the urgent need to regulate emotion recognition tech. During the pandemic, this tech has been pushed further into schools and workplaces. We should reject the phrenological impulse, where unverified systems are used to interpret inner states.
Comment: We can no longer allow emotion-recognition technologies to go unregulated, argues @katecrawford. go.nature.com/31TZBp2
16
328
995
OMG - my book has arrived!
These galley copies of Atlas of AI had to navigate oceans and international logistics, but here they are. It’s quite a thrill to see it in physical form at long last. 📚 🌎
43
70
926
Our new paper is out today: “DIRTY DATA, BAD PREDICTIONS.” We show serious problems with predictive policing being used where there is evidence of illegal and discriminatory police practices. By Rashida Richardson, @Lawgeek & me: papers.ssrn.com/sol3/papers.…
20
436
920
I'm so excited that my new book is out in April:
ATLAS OF AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. 📕 🔜
You can preorder from any local bookshop, eg: bit.ly/376GzyR
Featuring a beautiful cover & illustrations by @TheCreaturesLab 👇
ALT Atlas of AI (Yale, 2021)
38
184
867
Today we officially launch the @AINowInstitute at NYU! Grateful to be part of an incredible community of scholars in multiple disciplines working on the social and political implications of AI, ML and algorithmic accountability.
23
247
791
We can finally share a little secret: Anatomyof.ai has just been acquired by MoMA. Wild!
Huge thanks to the whole curatorial team at @MuseumModernArt for their support of our project, and to @TheCreaturesLab for this joyous collab ✨♥️🗺️ moma.org/collection/works/40…
31
140
808
I just published this piece in @NatureNews today. Facial recognition is being rolled out in cities worldwide with few safeguards. It's used by ICE, CBP, and across public space. Bad when it fails, bad when it works. It's time for a moratorium. nature.com/articles/d41586-0…
15
384
800
What if 'tech ethics' is a smokescreen unless it contends with the bigger issues of concentrated power, governance models, due process, and public accountability? 🤔 nitter.app/rachelcoldicutt/status…
23
219
802
Meanwhile, Amazon's latest patent is for Alexa to detect when people are sick, bored or unhappy. "Alexa would listen out for if users are crying and then class them as experiencing an “emotional abnormality.” 🙃 telegraph.co.uk/technology/2…
78
597
773
The AI Summit ends in rupture. AI accelerationists want pure expansion—more capital, energy, private infrastructure, no guard rails. Public interest camp supports labor, sustainability, shared data. safety, and oversight. The gap never looked wider. AI is in its empire era.
36
275
873
67,613
Most developers will be asked to build something unethical or dangerous. But they rarely talk about it. Read this. medium.freecodecamp.com/the-…
19
771
757
HUGE. A bug in fMRI software may invalidate 15 years of research. It was producing false positive rates of up to 70% theregister.co.uk/2016/07/03…
32
1,006
766
Wait WHAT? The US says "an algorithm" identified Eyal Weizman as a security threat and he can't enter the country. Yes, the same guy who founded Forensic Architecture at Goldsmiths, a respected researcher who visits frequently. Not good. nytimes.com/2020/02/19/arts/…
17
285
704
📣 The AI Now 2018 Report is now live! 📣
Our biggest ever report tackles the issues of AI and accountability, after a hell of a year for the tech sector. Read the report, see our 10 recommendations ainowinstitute.org/AI_Now_20…
13
411
747
Why did OpenAI use Scarlett Johansson's voice? As Jessa Lingel & I discuss in our journal article on AI agents, there's a long history of using white women's voices to “personalize” a technology to make it feel safe and non-threatening while it is capturing maximum data. 🧵
23
232
732
118,238
This story exposes the myth of control. Spotify & Netflix could read *private* FB messages. Microsoft, Amazon & Yahoo all had access far beyond what ANY user agreed to. The total lack of respect for user wishes is the infinitely repeating scandal of 2018 nytimes.com/2018/12/18/techn…
17
422
668
Thanks for this. So can I confirm that this was a Bard hallucination, and that there's no Gmail data included whatsoever in the training process?
34
31
710
125,524
Truly horrifying story.
Right now, Uyghurs are being used as lab subjects for emotion AI, under coercion, strapped into restraint chairs, and then tagged as "nervous" or "anxious" which is taken as a marker of guilt. via @hare_brain bbc.com/news/technology-5710…
64
429
669
Hidden amongst the zingers in the leaked Google memo there's this: data quality is what is making the difference now. This is an important shift away from internet-scale garbage dump datasets that have been running the show for years.
28
177
705
168,707
Finally, a paper on why "AI for good" is an empty phrase without a theory of change. What is "good" is never articulated in the rush to tech solutions, while alternative reforms are overlooked. Read @benzevgreen's piece before it blows up at @NeurIPSConf benzevgreen.com/wp-content/u…
18
226
672
So I wrote a thing for @nytimes on race, gender and AI: why billionaires fear the rise of an AI apex predator. nytimes.com/2016/06/26/opini…
40
418
674
Want to see how an AI trained on ImageNet will classify you? Try ImageNet Roulette, based on ImageNet's Person classes. It's part of the 'Training Humans' exhibition by @trevorpaglen & me - on the history & politics of training sets. Full project out soon
imagenet-roulette.paglen.com
80
262
674
Nature just published a major feature on researchers working on bias in machine learning. Features many of us, including @geomblog @mikarv @s010n @achould @b_mittelstadt @rayidghani @dgrobinson @bowlinearl - and the @AINowInstitute work on due process nature.com/articles/d41586-0…
11
364
647
WHOA. IBM Watson has been recommending 'unsafe and incorrect' cancer treatments, according to their own internal documents gizmodo.com/ibm-watson-repor…
33
388
618
Let's recap: Paris attackers were known suspects, didn't use encryption, and boasted in public about their plans. theguardian.com/commentisfre…
34
1,136
587
Whoa. Uber tracked users *after* they deleted the app from their phones nytimes.com/2017/04/23/techn…
35
722
579
In the 12hrs since Bard told me it was trained on Gmail data:
-Google replies (says it's not)
-Elon Musk replies (lol)
-Google adds a 'community note' that this is a Bard error and it's not trained on Gmail
-Some ace memes
What should happen next: Real talk about training data 🧵
18
136
589
106,638
Guess we could start calling this a 'hallucitation'? 🤷♀️
If you're curious about the article, it's here. cc @mjnblack
businessinsider.com/lex-frid…?
20
45
576
172,098
Just wow. So this happened. Thank you @DesignMuseum and all the jury for giving us the Design of the Year award.
@TheCreaturesLab and I are kinda blown away 😮 💥
Drum-roll please... | The overall winner of #BeazleyDesignsoftheYear is "Anatomy of an AI System" 🏆 by @katecrawford & @TheCreaturesLab - which charts the birth, life and death of a voice assistant and its impact on our planet. > fal.cn/35cNG @BeazleyGroup
24
82
590
Let's call the Facebook experiment what it is: a symptom of a much wider failure to think about ethics, power and consent on platforms.
46
715
556
Big news 👇👀
Breaking News: San Francisco became the first major U.S. city to ban the use of facial recognition technology by police and other municipal agencies nyti.ms/2VA92cJ
7
154
553
[Opens WSJ]
Today, a wealthy, white, ex-hedge fund guy said facial recognition is fine. He quotes cops, cites Amazon's PR that it has no racial biases, and...
[rubs eyes]
...opens with noted pioneer of eugenics and 'race betterment', Francis Galton. wsj.com/articles/have-no-fea…
24
199
524
And here it is - the leaked client list of Clearview. Remember how they said it was “strictly law enforcement only”? Not so much. Clients include Walmart and Macy’s as well as ICE and CBP.
Clearview AI has been used by more than 2,200 entities. From governments to the private sector. From ICE to Interpol. From Australia to the UAE.
Collectively, there have been nearly 500,000 searches.
buzzfeednews.com/article/rya…
11
372
529
New paper shows racial bias in automatic hate speech detection. So AI models trained to flag & remove offensive tweets risk suppressing black voices.
"African Americans are up to two times more likely to be labelled as offensive compared to others" homes.cs.washington.edu/~msa…
11
303
521
On the many problems with emotion "detection" in AI:
"A growing crowd of researchers argues that the variation is so extensive that it stretches the gold-standard idea to the breaking point. Their views are backed up by a vast literature review."
nature.com/articles/d41586-0…
15
236
531
This tweet did not age well
We use #data to change audience behaviour. Learn more about what Cambridge Analytica could do for you. goo.gl/5B2akP
15
150
483
TLDR: another article about AI in which the only risk is super-intelligence, and only dudes have opinions🤔 vanityfair.com/news/2017/03/…
25
268
488
LITIGATING ALGORITHMS is out today: our report on the current court cases about algorithmic decision making. We convened the lawyers leading the cases on criminal risk assessment, Medicaid & teacher evals. With @EFF and @RaceNYU, this is what we found out: ainowinstitute.org/litigatin…
4
268
490
In the last two days, Google and @FinancialTimes editorial board have called for a temporary moratorium on facial recognition - following in the footsteps of many researchers and civil society orgs.
The FT View: A new technology with a considerable risk of unintended harms should not be rushed out on.ft.com/2Gd4oHw
12
252
457
The numbers keep getting worse: the true energy costs of AI, connected devices, and cryptocurrency theguardian.com/environment/…
15
361
467
US job applicants are getting a single number that determines if they get a job. That number comes from a bollocks AI system that claims to rate their social media activity. And people think 'social credit scores' are just in China 🤔
19
248
446
Big news: Ohio's attorney general is suspending access to facial recognition databases for police officers following the news that federal agencies like ICE and the FBI are mining state databases without people’s consent toledoblade.com/local/politi…
2
179
431
Today @trevorpaglen & I open MAKING FACES: an installation and event about the 150 year history of facial recognition. It's an intervention staged in Paris that focuses on politics of the face as a terrain of identification, power and control.
7
98
466
OMG - IT ARRIVED 💥💥💥
Just 5 days before it goes on sale in the US, physical copies of my book are here! Exciting to see it and @TheCreaturesLab’s beautiful illustrations come to life. #atlasofAI
18
40
445
Excited to launch this new investigation w @trevorpaglen - to be read alongside ImageNet Roulette. It's about how training data works, the costs of classification, and why "removing bias" or "increasing diversity of data" isn't enough 👁️ excavating.ai
12
190
458
The UN has made a deal with Palantir to give them highly sensitive data about aid recipients in the World Food Program - part of their “very aggressive digital transformation journey.” World reacts in horror. 😱 irinnews.org/news/2019/02/05…
17
283
432
🚨 NEW RESEARCH ALERT: The diversity crisis in AI has hit a moment of reckoning: the call is coming from inside the house. Study led by @sarahbmyers shows lack of workforce diversity and bias in AI systems are connected. Read here: ainowinstitute.org/discrimin… #AIdiversitycrisis
11
218
447
"Have you ever had concerns that a prospective employer is strip-mining your social media and making assertions about your private life?"
22
272
414
AI is rapidly becoming a new, privately-owned infrastructure - and the bias and safety issues urgently need more research (my oped for @WSJ)
There are no established methods to test AI for safety, fairness or effectiveness
on.wsj.com/2gpr4uM
15
288
439
Amazon's system "penalized resumes that included the word 'women’s,' as in 'women’s chess club captain.'
Amazon scraps secret AI recruiting tool that showed bias against women reut.rs/2Pmn6zu
13
304
421
Ed tech company experiments on 9000 kids without anyone's consent or knowledge to see if they test differently when 'social-psychological' messaging is secretly inserted? HARD NO. blogs.edweek.org/edweek/Digi…
22
313
412
So I gave a talk at the @royalsociety last week. Seemed like the right time to discuss the political landscape we're in, and why the bias debate in AI is too narrow. Here's the video: "Just an Engineer: On the Politics of AI" piped.video/watch?v=HPopJb5a…
4
161
446
This story is just *so* dead on. gizmodo.com/predictim-claims…
9
171
411
There is a real problem here. Scientists and researchers like me have no way to know what Bard, GPT4, or Sydney are trained on. Companies refuse to say. This matters, because training data is part of the core foundation on which models are built. Science relies on transparency.
15
118
415
44,181
Now the dust has settled on Google's AI principles, it's time to ask about governance. How are they implemented? Who decides? There's no mention of process, or people, or how they'll evaluate if a tool is 'beneficial'. Are they... autonomous ethics? blog.google/topics/ai/ai-pri…
13
227
422
"This is the story of how affect recognition came to be part of the AI industry, and the problems that presents."
An excerpt from #AtlasofAI is out today in @TheAtlantic drawn from my chapter on emotion AI. Thanks to @AdrienneLaF for editing 🙏📚✍
theatlantic.com/technology/a…
10
163
424
IBM calls their new program ‘Not Learning From History’
SCOOP: With secret access to NYPD CCTV @IBM created software which tags people based on their skin tone + hair/clothing color. IBM gave NYPD access, then pitched them on a new AI product which identifies people on camera as "Black," "White," and "Asian":
theintercept.com/2018/09/06/…
8
166
420
Whoa, ImageNet Roulette went... nuts. The servers are barely standing. Good to see this simple interface generate an international critical discussion about the race & gender politics of classification in AI, and how training data can harm. More here: theguardian.com/technology/2…
8
147
406
Today we're sharing our Critical Dataset Studies reading list. The Knowing Machines team uses it to reflect on the growing literature on ML datasets across disciplines. Datasets powerfully construct model worldviews, so they're important to study. knowingmachines.org/reading-… (1/4)
5
153
398
Today is a fun anniversary in my music life. 20 years ago we released "2020" - an electronic album exploring ideas of gender, machines, and strange futures. ABC just published this retrospective on B(if)tek, 2020, and what we do now 👩🎤🎹 abc.net.au/news/2020-08-12/t…
38
77
406
So many important concerns have been raised about the Google AI Board, particularly the inclusion of the Heritage Foundation and a drone manufacturer. When we wrote 'Why Ethics is Not Enough' last year, this is what we meant ainowinstitute.org/AI_Now_20…
5
156
379
The current rush to "bossware" to surveil and rank workers during Covid harkens back to Ford's factories, Bentham's inspection houses, and Taylor's micromanagement of bodies. Now it's sprinkled with "AI and data science" to be more invasive, granular, and controlling.
12
187
381
We need to talk about ethics more. Because developers will be asked to do some seriously awful things in the next 4 years. (1/5)
15
341
366
🎧NEW PODCAST! 🎧 Want to understand AI with deeper context? We've launched a pod series on how AI is trained to interpret the world. Each ep has a research theme: from the social, legal, cultural, environmental & political impacts of generative AI. knowingmachines.org/podcast 👇📻
8
138
386
56,120
It's 2017, and researchers are still using Playboy's Lena centerfold as a test image. Given the gender issues in this field, maybe it's time to move on guys? 🤔
The results of this "Deep Image Prior" paper are stunning: I'd have assumed they were not possible if the authors hadn't provided code. dmitryulyanov.github.io/deep…
11
118
401
Hey London, I’m giving a lecture at the @royalsociety tomorrow on the politics of AI - and why narrow tech fixes to bias don’t work. All welcome ✨👌✨
Join @katecrawford (@ainowinstitute, @MSFTResearch) debating the biases built into machine learning, and what that means for inequality - Tuesday, 7.30pm #YouandAI ow.ly/n82P30kLpRD
16
123
371
The @EU_Commission's final proposal for an Artificial Intelligence Act is here. Some examples:
AI systems are prohibited if they violate human rights, do general social scoring for authorities, use live remote biometrics in public for policing. 👀
digital-strategy.ec.europa.e…
9
169
366
Deep support to all the women at Microsoft who were brave enough to come forward. Cultures of harrassment, exclusion and unfair compensation are unacceptable, and it's time to make change across the whole sector. 🙌
Today women from Microsoft asked CEO Satya Nadella for answers after a massive email thread where female employees shared their experiences with discrimination and harassment over the past couple weeks wired.com/story/microsoft-em…
2
81
365
Big news: LAPD will end the use of the broken predictive policing system known as PredPol, citing budget concerns under COVID-19. This is thanks in large part to community groups like @stoplapdspying pushing back against its use. latimes.com/california/story…
6
143
367
Whoa. Adobe is offering *full* legal indemnification for copyright lawsuits over generated images that enterprise users produce in Firefly. Their model is trained on licensed & out of copyright images, which others don't do - so it's a big throw-down. fastcompany.com/90906560/ado…
8
132
368
102,478
Wow. @TheCreaturesLab & I just got news that our Anatomy of an AI System has been acquired for the permanent collection of the V&A museum. The map, the newspaper, and the code are now setting up a new home. Big thanks to @V_and_A curatorial team ✨🗺️💜 collections.vam.ac.uk/search…
17
34
375
Honored to be on @TIME's list with many friends and colleagues. Glad to see it has included the work of those addressing AI's social, political & ecological effects. time.com/collection/time100-… #TIME100AI
11
47
361
49,519