Machine learning researcher @MSFTResearch (@MSRNE); adjunct professor @Stanford
- Tweets 602
- Following 279
- Followers 2,961
- Likes 1,417
Pinned Tweet
@tobias_schrdr and I are excited to share WildCat: Near-Linear Attention in Theory and Practice arxiv.org/abs/2602.10056
By attending over a spectrally-accurate optimally-weighted coreset, WildCat approximates exact attention with super-polynomial error decay in near-linear time
4
11
64
9,001
If you're a PhD student interested in interning with me or one of my amazing colleagues at Microsoft Research New England (@MSRNE, @MSFTResearch) this summer, please apply here jobs.careers.microsoft.com/g…
12
80
395
86,491
If you're a PhD student interested in interning with me or one of my amazing colleagues at Microsoft Research New England (@MSRNE, @MSFTResearch) this summer, please apply here
jobs.careers.microsoft.com/g…
(If you'd like to work with me, please include my name in your cover letter!)
8
67
424
52,213
If you're a PhD student interested in interning with me or one of my amazing colleagues at Microsoft Research New England (@MSRNE, @MSFTResearch) this summer, please apply here jobs.careers.microsoft.com/g…
6
65
308
48,081
If you're a PhD student interested in interning with me or one of my amazing colleagues at MSR New England this summer, please apply here
careers.microsoft.com/us/en/…
4
65
273
If you're a PhD student interested in interning with me or one of my amazing colleagues at MSR New England this summer, please apply here
careers.microsoft.com/us/en/…
6
79
229
If you’d like to join Microsoft Research New England as a researcher in AI / ML / statistics, please apply here: jobs.careers.microsoft.com/g… @MSFTResearch @MSRNE
1
24
129
23,200
New guarantees for approximating attention, accelerating SGD, and testing sample quality in near-linear time
ALT https://arxiv.org/abs/2502.12063
1
13
136
25,762
Call for Machine Learning, AI, & Statistics Researchers at Microsoft Research New England @MSRNE : careers.microsoft.com/us/en/…
2
39
116
Interested in helping to organize @NeurIPSConf 2024?
Let us know here: docs.google.com/forms/u/1/d/…
33
107
43,639
If you're a recent or graduating PhD student interested in postdocing with the ML / stats team at MSR New England, please apply here aka.ms/ml-postdoc-msrne
1
37
93
If you're an undergraduate interested in interning with me or one of my amazing colleagues at Microsoft Research New England (@MSRNE, @MSFTResearch) this summer, please apply here
microsoft.com/en-us/research…
4
20
76
14,884
Call for machine learning and statistics postdocs at Microsoft Research New England @MSFTResearch careers.microsoft.com/us/en/…
18
72
For me the Netflix Prize was a bellwether of many ML trends: all of the leading teams used SGD and low-rank approximation for scalable non-convex optimization, trained neural networks for both modeling and ensembling, and fit billion-parameter models to get the best performance
1
4
36
9,918
New @NatureComms work with Soukayna Mouatadid, Paulo Orenstein, @GFlaspohler, @judah47, Miruna Oprescu, Ernest Fraenkel, @UofT, @MSFTResearch, @MSRNE, @AER_inc, @MIT, @SpringerNature
Adaptive bias correction for improved subseasonal forecasting
rdcu.be/deAJW
1
6
33
6,037
You can also use “empirical Berry Esseen bounds” to get both finite sample validity and asymptotic efficiency (matching the optimal CLT-based width in the limit) arxiv.org/abs/2208.09922
1
2
29
4,470
If you're a PhD student interested in a spring internship exploring fairness in clinical trials and A/B experimentation, please apply here careers.microsoft.com/studen…
7
26
I've decided to solve the most pressing problems of our time using language models, for example,
1
20
2,163
This follow-up article also surveys recent applications in probabilistic inference, computational statistics, and machine learning
arxiv.org/pdf/2105.03481
3
19
3,010
If you’re attending @COLM_conf this week, check out Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation (w/ @ericzelikman @AdamKalai)
1
17
1,198
Check out this amazing work by Myra Cheng! (@mariadearteaga, @adamfungi, and I helped too)
📣New research📣"Social Norm Bias: Residual Harms of Fairness-Aware Algorithms” led by undergrad Myra Cheng (applying to PhDs soon!) w/ @adamfungi @lestermackey🧵
arxiv.org/abs/2108.11056
1
12
Yeah! It somehow knew that I wanted to buy a statistics for social good t-shirt, mug, sticker, and book
11
1,498
Don’t just improve your code; improve your code improver!
“Recursive self-improvement” (RSI) is one of the oldest ideas in AI. Can language models write code that recursively improves itself?
Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
w/@elianalorch, @LesterMackey, @adamfungi
(1/n)
ALT Pipeline figure for STOP. On the left, improver_0 improves itself to become improver_1, etc. until improver_T. On the right, improver_0 is expanded to visualize that improver_0, the seed improver, takes a program and returns the best improvement the language model generates.
4
11
5,787
Thanks @MayaAjmera ! #TuesdayTrailblazers
For this edition of #TuesdayTraillazers, meet 2003 Science Talent Search and International Science and Engineering Fair alum, @LesterMackey
1
11
Exciting new work with @raazdwivedi; let us know what you think!
Super excited to present “Kernel Thinning”, a new procedure for compressing a distribution more effectively than i.i.d. sampling or standard MCMC thinning. w/ @LesterMackey, today 2PM ET@#MCM, tomorrow 10AM/Noon@#COLT2021
arxiv.org/abs/2105.05842
Video: learningtheory.org/colt2021/…
1
9
My first tweet 😀
Our #ICML2021 paper unifies optimistic and delayed online learning to develop optimal algorithms with no hyperparameters to tune (w/ applications in subseasonal forecasting)
arxiv.org/abs/2106.06885
w/ @bremen79 @judah47 S Mouatadid @MirunaOprescu P Orenstein @LesterMackey
🧵
1
7
I’m pretty sure this is what they designed Sora 2 for (sound on) @raazdwivedi @AShettyV
1
8
755
Inspired by the groundbreaking work of @insu_han @daliri__majid @aminkarbasi @WentaoGuo7 @chrismdesa @afedercooper @cdomingoenrich
1
1
8
1,775
Come intern with us at @MSRNE!
The Microsoft Research Undergraduate Internship Program offers 12-week internships in our Redmond, NYC, or New England labs for rising juniors and seniors who are passionate about technology. Apply by October 6: msft.it/6015scgSJ
1
8
1,521
The competition also ushered in the new era of parallel and distributed ML. My teammates and I bought some of the first Intel quad-core processors to work on the competition, and I remember writing some of the first Spark code with @matei_zaharia to scale beyond that.
8
542
This looks great @risteski_a ! See also Minimum Stein Discrepancy Estimators (arxiv.org/pdf/1906.08283.pdf) for an analysis of score matching (and related procedures like diffusion score matching)
2
6
@immonica please see the response from @NeurIPSConf here:
NeurIPS acknowledges that the cultural generalization made by the keynote speaker today reinforces implicit biases by making generalisations about Chinese scholars. This is not what NeurIPS stands for. NeurIPS is dedicated to being a safe space for all of us. We want to address the comment made during the invited talk this afternoon, as it is something that NeurIPS does not condone and it doesn't align with our code of conduct. We are addressing this issue with the speaker directly.
NeurIPS is dedicated to being a diverse and inclusive place where everyone is treated equally.
1
6
1,410
Andrew Barbour’s generator method also gives you a practical way to create Stein operators for any distribution
6
306
Code coming soon!
github.com/microsoft/thinfor…
github.com/microsoft/khsgd
github.com/microsoft/deepctt
5
1,138
I was watching the season finale of Severance and didn’t want the screenshot noise to ruin the experience
4
321
Undergrads, get your MSR internship applications in by November 6!
Come do an undergraduate research internship at MSR!
microsoft.com/en-us/research…
3
1,037
This reminds me of the tool that Joel Tropp used to derive "intrinsic dimension" concentration inequalities for random matrices in apps.dtic.mil/sti/tr/pdf/ADA…
1
4
295
These letters often come from a PhD advisor or other collaborators or research mentors who can speak to your research experience
1
3
3,383
Reminds me of when someone opened our package of microwave popcorn and left half of the bags for us to enjoy
3
562
I still remember our conversations Franklin, and I’m glad to see all of the success the that The Black List has had!
1
211
I’d like to pretend that added it on purpose, but, in reality, I accidentally took the screenshot while pressing the mute button
1
3
356
Call for NeurIPS Ethics Reviewers
neurips.cc/Conferences/2023/…
2
1
1,506
Thanks @nancybaym! I'm looking forward to h̶a̶n̶g̶i̶n̶g̶ ̶o̶u̶t̶ working with you in person one day soon
3
In the #realworldproblem motivating this question, my only option was a Church's Chicken in Costa Rica
1
79
tl;dr: A low-cost machine learning correction for physics-based dynamical models improves subseasonal forecasting of temperature and precipitation two to six weeks ahead.
2
269
The leaf sheep finally has some competition! boredpanda.com/leaf-sheep-se…
2
The matrix Hoeffding inequality (Cor. 4.2 of arxiv.org/pdf/1201.6002.pdf) would give you P( twonorm(sum_i X_i) >= t ) <= (d+1) exp( - t^2 / sum_i c_i^2 ) . There are also ways to get rid of that d multiplier
2
628