Co-founder of stealth startup. Inventor of GANs. Lead author of deeplearningbook.org Founding chairman of @pubhealthaction
San Francisco, CA
- Tweets 2,997
- Following 1,361
- Followers 373,305
- Likes 9,259
I'm excited to announce that I've joined DeepMind! I'll be a research scientist in @OriolVinyalsML 's Deep Learning team.
140
223
6,958
One of my favorite samples from the Progressive GANs paper is this one from the "cat" category. Apparently some of the cat training photos were memes with text. The GAN doesn't know what text is so it has made up new text-like imagery in the right place for a meme caption.
52
2,232
5,418
I never heard back from MIT. I got rejected from CMU. I was accepted to U of T but not to work with the PI I wanted there. I got "honorable mention" for NSF GRFP but not actual money. Don't let temporary failures discourage you. nitter.app/mrtz/status/9504934338…
44
866
3,488
4.5 years of GAN progress on face generation. arxiv.org/abs/1406.2661 arxiv.org/abs/1511.06434 arxiv.org/abs/1606.07536 arxiv.org/abs/1710.10196 arxiv.org/abs/1812.04948
35
1,239
3,454
CycleGAN turning a horse video into a zebra video ( github.com/junyanz/CycleGAN )
33
1,574
2,831
More of what my colleagues and I have been working on in AI for fusion is public now
We’re announcing a research collaboration with @CFS_energy, one of the world’s leading nuclear fusion companies.
Together, we’re helping speed up the development of clean, safe, limitless fusion power with AI. ⚛️
57
123
2,791
288,511
This exciting new PyTorch library includes quite a lot of the GANs I've featured in my talks over the past few years, all in one place!
10
410
2,022
OctConv is a simple replacement for the traditional convolution operation that gets better accuracy with fewer FLOPs export.arxiv.org/abs/1904.05…
16
543
1,866
I’m in Fortune’s 40 under 40: fortune.com/40-under-40/2019…
93
98
1,735
When Schmidhuber interrupted my talk on GANs in 2016, he did so at the time I was giving credit to relevant prior work: noise contrastive estimation. Schmidhuber isn’t about proper about academic credit assignment, he’s about self-aggrandizement.
36
51
1,441
754,760
ML researchers, reviewers, and press coverage of ML need to get a lot more serious about statistically robustness of results and the effect of hyperparameters. This study shows that many papers over the last year or so were just observing sampling error, not true improvement.
Are GANs Created Equal? A Large-Scale Study
arxiv.org/abs/1711.10337
32
609
1,410
Posting a call for help: does anyone know of a good way to simultaneously treat both POTS and Ménière’s disease? Please contact me if you’re either a clinician with experience doing this or a patient who has found a good solution. Context in thread
137
324
1,378
370,416
By looking at this image, you can see how sensitive your own eyes are to contrast at different frequencies (taller apparent peaks=more sensitivity at that frequency). It's like a graph that is made by perceiving the graph itself. h/t @catherineols fourier.eng.hmc.edu/e180/lec…
24
464
1,309
Whoa! It turns out that famous examples of NLP systems succeeding and failing were very misleading. “Man is to king as woman is to queen” only works if the model is hardcoded not to be able to say “king” for the last word.
1/7 Do word embeddings really say that man is to doctor as woman is to nurse? Apparently not. Check out this thread for a description of a short paper I co-wrote with Malvina Nissim and Rob van der Goot, available here: arxiv.org/abs/1905.09866 #NLProc #bias
28
348
1,307
Schmidhuber was a reviewer of the GAN paper in 2014. The reviews are public. He is the reviewer that asked us to cite predictability minimization. He did not ask us to cite artificial creativity (his current complaint). He hadn’t thought of that even in 2016.
When Schmidhuber interrupted my talk on GANs in 2016, he did so at the time I was giving credit to relevant prior work: noise contrastive estimation. Schmidhuber isn’t about proper about academic credit assignment, he’s about self-aggrandizement.
39
50
1,317
368,135
GANs for generating images of how clothes will fit. Only two of these images are photos. qz.com/1090267/artificial-in…
19
568
1,279
ML Twitter, what are your favorite papers / other resources about the class imbalance problem?
63
203
1,190
GANs can be used to automatically design dental crowns, that are then actually manufactured and used in the physical world. Crowns need to be made specifically for each patient and need to fit correctly with the other teeth and support biting and chewing. arxiv.org/abs/1804.00064
9
373
1,176
Apple now has an AI/ML residency program! I'm looking forward to working with our first class of residents.
Thrilled to announce a new program designed to help experts in applied fields build ML-powered products and experiences. Introducing the AI/ML residency program: machinelearning.apple.com/up…
11
158
1,170
Philip Wang at Uber set up thispersondoesnotexist.com to show a new imaGANary person every time you refresh the page
44
406
1,107
Super-resolution GANs for improving the texture resolution of old games: gamespot.com/forums/pc-mac-l…
20
263
1,097
An exciting property of style-based generators is that they have learned to do 3D viewpoint rotations around objects like cars. These kinds of meaningful latent interpolations show that the model has learned about the structure of the world. piped.video/kSLJriaOumA?t=335
8
340
1,067
Today GANs got 1 of 2 test of time awards at @NeurIPSConf . Many thanks to my co-author @dwf for preparing and presenting the talk. It’s no easy task to summarize 85k papers in 12 minutes
19
32
1,030
99,096
The quiet semisupervised revolution continues
FixMatch: focusing on simplicity for semi-supervised learning and improving state of the art (CIFAR 94.9% with 250 labels, 88.6% with 40). arxiv.org/abs/2001.07685
Collaboration with Kihyuk Sohn, @chunliang_tw @ZizhaoZhang Nicholas Carlini @ekindogus @Han_Zhang_ @colinraffel
8
196
968
Yoshua, Aaron, and I have released the LaTeX template for the Deep Learning book: github.com/goodfeli/dlbook_n… Useful if you want to follow the same math notation conventions as we do or if you want to put a notation page in your document
8
256
978
This new family of GAN loss functions looks promising! I'm especially excited about Fig 4-6, where we see that the new loss results in much faster learning during the first several iterations of training. I implemented the RSGAN loss on a toy problem and it worked well.
My new paper is out! " The relativistic discriminator: a key element missing from standard GAN" explains how most GANs are missing a key ingredient which makes them so much better and much more stable! #Deeplearning #AI
ajolicoeur.wordpress.com/Rel…
arxiv.org/abs/1807.00734
7
248
970
I’m heading to Uruguay next month to teach about generative models at khipu.ai/
29
81
944
Thank you to the many people who reached out after my now-deleted tweet last week asking for help with an urgent problem. For everyone still concerned, things are under control now.
29
8
871
217,900
In the Google DeepMind fusion team, we're working with external collaborators to solve nuclear fusion power generation. Check out the team's 2022 Nature paper in collaboration with EPFL for more context on how reinforcement learning can be used to control a fusion power generator. Today, we're announcing TORAX, an open source simulator component for modeling fusion power generators.
Excited to announce the release of TORAX, a tokamak transport simulator from our @GoogleDeepMind Fusion team! #fusionenergy
- Open-source: github.com/google-deepmind/t…
- Uses JAX: fast, differentiable
- Easy coupling of ML-surrogates
Hot off the press → arxiv.org/abs/2406.06718
12
108
886
149,786
Forbes listed GANs as one of the best tech innovations of the last three years: forbes.com/sites/forbestechc…
17
186
885
I've spent several years studying machine learning security with the goal of making ML reliable before it is used in more and more important contexts. Unfortunately, ML capabilities and adoption are growing much faster than ML robustness. safe.ai/statement-on-ai-risk
85
142
842
137,654
ML paper writing pro-tip: you can download the raw source of any arxiv paper. Click on the "Other formats" link, then click "Download source". This gets you a .tar.gz with all the .tex files, all the image files for the figures in their original resolution, etc.
18
272
853
MIT Technology Review listed GANs as 1 of 10 breakthrough technologies this year: technologyreview.com/lists/t…
16
243
873
While GANs have been great at generating realistic images from a single category (one GAN for faces, another GAN for buildings) they've always struggled to fit all 1,000 classes of ImageNet with a single GAN. This ICLR submission has done it: openreview.net/pdf?id=B1QRgz…-
28
328
818
Vue.ai, a startup that uses GANs to generate retail marketing images, has raised $17M: venturebeat.com/2019/04/24/v…
14
187
778
Self-attention for GANs. No more problems with losing track of how many faces the generator has drawn on the dog. nitter.app/gstsdn/status/99908347…
13
232
750
This year I unfortunately got POTS from COVID. POTS is a failure of the body to regulate blood flow correctly, the heart has to beat extra fast to get blood to reach the head when upright. It’s much more complicated than that but that’s the defining symptom.
29
118
745
261,144
This is really cool. Some of my PhD labmates worked on ML for compression back in the pretraining era, and I remember it being really hard to get a compression advantage.
Check out our new work on face-vid2vid, a neural talking-head model for video conferencing that is 10x more bandwidth efficient than H264
arxiv arxiv.org/abs/2011.15126
project nvlabs.github.io/face-vid2vi…
video piped.video/nLYg9Waw72U
@tcwang0509 @arunmallya
#GAN
19
96
743
One of my main concerns about machine learning interpretability tools is that they will make people think they understand ML when they don't. People seem to think linear models are interpretable, but no one looks at them and has the intuition that they have adversarial examples nitter.app/superwuster/status/963…
18
211
719
NVIDIA gave me a new T-Rex, signed by Jensen. They are not even for sale yet! Thanks NVIDIA! The pic is with GAN extraordinaire Ming-Yu at the NVIDIA reception last night.
14
47
722
To gain some idea of the far future of ML security, we studied a simple toy problem called "adversarial spheres," simulating a future where advanced ML models are extremely accurate. We find that even then, an adversary can still easily fool them. arxiv.org/abs/1801.02774
13
250
708
If you’re upset that someone didn’t cite your paper, I strongly recommend contacting the authors privately before making a public complaint. Thread:
15
72
684
A math trick I like a lot is the approach to taking derivatives using hyperreal numbers. Thread:
13
158
686
It would be great to post saying “I’m a physician with #LongCovid. Here’s two cents. Dentists: please wear N95s and use air filtration in your practice so patients don’t have to choose between getting or worsening Long COVID and endocarditis, jaw necrosis, etc”
3
52
563
12,345
Updating some slides from last year. “>2,000 papers later, still not really solved” -> “>5,000 papers, still not really solved”
8
43
653
Hadrien Jean has made a series of notes on linear algebra, following the deep learning textbook: hadrienj.github.io/posts/Dee…
6
230
661
arxiv.org/pdf/1807.10875.pdf TensorFuzz automates the process of finding inputs that cause some specific testable behavior, like disagreement between float16 and float32 implementations of a neural network
Neural networks are notoriously hard to debug. @gstsdn has developed a new debugging methodology by adapting traditional coverage guided fuzzing techniques to neural networks.
5
212
651
#CVPR2018 I will teach an Introduction to GANs at 8:45 AM in Room 150-ABC at the Perception Beyond the Visible Spectrum workshop. Slides available at iangoodfellow.com/slides/
6
157
658
When I invented adversarial training as a defense against adversarial examples, I focused on making it as cheap and scalable as possible. Eric and collaborators have now upgraded the original cheap version to compete with newer, more expensive versions.
1/ New paper on an old topic: turns out, FGSM works as well as PGD for adversarial training!*
*Just avoid catastrophic overfitting, as seen in picture
Paper: arxiv.org/abs/2001.03994
Code: github.com/locuslab/fast_adv…
Joint work with @_leslierice and @zicokolter to be at #ICLR2020
9
113
634
I suspect that peer review *actually causes* rather than mitigates many of the “troubling trends” recently identified by @zacharylipton and Jacob Steinhardt: arxiv.org/abs/1807.03341
20
205
650
My GAN co-author Sherjil Ozair has written about some memories of 2012-2014 in the context of GANs winning one of this year's test of time awards, worth a read for the nostalgia if you were around back then, or for learning what it was like if you weren't
Very happy to hear that GANs are getting the test of time award at NeurIPS 2024.
The NeurIPS test of time awards are given to papers which have stood the test of the time for a decade.
I took some time to reminisce how GANs came about and how AI has evolve in the last decade.
13
57
630
88,934
The term “deep learning” reminds me of “horseless carriage.” It made sense when introduced, but now that it is the dominant paradigm, it feels quaint to specify that there is no horse. The horse here is of course the shallow model / convex cost constraint.
19
113
615
A tech report summarizing my NIPS tutorial on GANs arxiv.org/abs/1701.00160
9
291
618
The Self-Organizing Conference on Machine Learning is returning as a 100% online event for 2020. Nov 30-Dec 4. It will still be small to maintain the group discussion feel. Apply at socml.org
14
104
610
The new Justin Timberlake video is at a deep learning conference. This is just surreal to me. piped.video/watch?v=gA-NDZb2…
25
217
611
Google Brain Residency has been upgraded to Google AI Residency. Now possible to work with more AI teams at Google. research.google.com/teams/br…
3
174
611
TensorFlow launches Easy Mode... I mean, Eager Mode research.googleblog.com/2017…
7
215
614
The "personal responsibility" approach does not work. I'm probably the 2nd most COVID avoidant person I know, managed not to get infected to my knowledge until July 2024, and that one infection has totally taken me out.
25
97
588
31,005
My team at Apple is hiring in Zurich: jobs.apple.com/en-gb/details…
19
123
605
Moonlake is launching publicly today, focused on generating simulations and games. Simulations are an important application of generative models that I’ve advocated for since 2014
We raised $28M seed from Threshold Ventures, AIX Ventures, and NVentures (Nvidia's venture capital arm) —alongside 10+ unicorn founders and top AI researchers— to build reasoning models that generate real-time simulations and games.
Models are bottlenecked by practical simulations that can act as Reinforcement Learning environments. Human self-expression is bounded by tools that let us create alternate realities.
At Moonlake, we are building a future where anyone can create interactive worlds, bring their child-like wonder to life, learn within them, and most importantly, share experiences with people we care about.
More in 🧵
14
58
626
123,302
CycleGAN learns to turn horses into zebras *without supervision*: arxiv.org/pdf/1703.10593.pdf
10
273
577
I’m one of @ForeignPolicy ‘s 100 global thinkers for 2019: foreignpolicy.com/2019-globa…
39
39
555
Train ImageNet in 18 minutes for just $40. fast.ai/2018/08/10/fastai-di… By my former colleague @yaroslavvb
1
131
581
GANs for architecture: archinect.com/news/article/1…
7
150
565
The "assistant professor" title seems especially galling to me: who exactly is the assistant professor assisting? They do the full job.
Academic titles tend to begin with negative adjectives and gradually remove adjectives. "Undergraduate student" -> "graduate student", "assistant professor" -> "associate professor" -> "full professor".
20
54
552
“Be careful what you wish for”
Describe programming in only six words.
We’ll RT all the best ones.
Ours:
Turning ideas and caffeine into code.
#ProgrammingIn6Words #wednesdaywisdom
11
57
544
My account was compromised and now recovered. I have deleted a fraudulent post about a crypto token using the DeepMind name. Please do not participate if you saw that while it was up.
49
21
556
89,417
A true Don’t Die ideology would require relentless focus on stopping the spread of COVID. I say this as someone who went from a fit mountain biker to totally disabled at 38 as a result of a single COVID infection.
34
27
522
50,112
This paper shows how to make adversarial examples with GANs. No need for a norm ball constraint. They look unperturbed to a human observer but break a model trained to resist large perturbations. arxiv.org/pdf/1805.07894.pdf
7
175
512
I think changing the name of NIPS is the right thing to do. The majority of women in the poll voted for it, and moral leadership shouldn’t be driven by polls anyway.
13
64
512
The definition of "adversarial examples" I prefer these days is "Adversarial examples are inputs to machine learning models that an attacker has intentionally designed to cause the model to make a mistake" blog.openai.com/adversarial-…
15
127
499
NIPS 2017 adversarial examples challenge: kaggle.com/c/nips-2017-non-t… Test your defenses against others' adversarial examples and vice versa!
9
207
507
Check out Adversarial Logit Pairing, the new state of the art defense against adversarial examples on ImageNet, by @harinidkannan @alexey2004 and I: arxiv.org/abs/1803.06373
3
176
508
I'll present a talk called "Defense Against the Dark Arts" summarizing the state of the art and key research challenges for defenses against adversarial examples. Room 259, 1:30 PM. iangoodfellow.com/slides/201…
9
143
496
An article about GANs in the @nytimes : nytimes.com/interactive/2018…
7
163
489
David has released a new paper from an old collaboration. Glad to see it out!
Latent Adversarial Generator Code is Out!
Code: github.com/google-research/l…
Arxiv: arxiv.org/abs/2003.02365
@docmilanfar @goodfellow_ian
2
94
483
nature.com/articles/s41586-0… Deep learning for predicting aftershocks of large earthquakes. Besides offering better predictions, interpretations of the model suggest promising directions for new physical theories
4
157
478
A great Scientific American article on creativity and problem solving in machine learning by Chris Baraniuk: scientificamerican.com/artic…
2
197
484
Neural networks are notoriously hard to debug. @gstsdn has developed a new debugging methodology by adapting traditional coverage guided fuzzing techniques to neural networks.
3
154
494
I originally thought of GANs as an unsupervised learning algorithm, but so far, to create recognizable object categories, they've needed a supervision signal / labeled images. This new work shows how to get them to work well with few labels.
How to train SOTA high-fidelity conditional GANs usin 10x fewer labels? Using self-supervision and semi-supervision! Check out our latest work at goo.gl/idWNVs @GoogleAI @ETHZurich @TheMarvinRitter @mtschannen @XiaohuaZhai @OlivierBachem @sylvain_gelly
3
134
481
The arxiv of the future must have comments and open peer review: authorea.com/users/3/article…
17
167
475
It’s strange to see people defining deep learning as supervised learning via backprop, considering that the 2006 deep learning revolution was originally based on the idea that neither of those things work very well
10
108
475