Director of AI/ML, @NYCOfficeOfTech. ai@nyc.gov. Opinions mine
New York 🇺🇸🗽
- Tweets 21,130
- Following 2,997
- Followers 4,108
- Likes 23,740
The computational biology people got here in 2010. With PCA. Not even deep learning.
Are we to conclude that principal components analysis has an internal world model?
pubmed.ncbi.nlm.nih.gov/2019…
Do language models have an internal world model? A sense of time? At multiple spatiotemporal scales?
In a new paper with @tegmark we provide evidence that they do by finding a literal map of the world inside the activations of Llama-2!
27
95
694
186,491
I have THOUGHTS about postdoc pay. Exactly 5 years ago, I started my first true industry job as a data scientist. Then, I had 8 years of post-PhD work experience at MIT, on top of 2 years experience outside of academia working at a national R&D lab. 🧵
5
95
640
A more accurate title: what happens when companies build emotion stereotyping models without regard for neurodivergence and then we plop them into an uncritical hype machine? wired.com/story/artificial-e…
7
66
363
Looks like the Transformer authors updated the "Attention is all you need" paper with e̶m̶a̶i̶l̶s̶
arxiv.org/abs/1706.03762
7
26
276
264,171
Why does the @nytimes address @evijit as Dr. Ghosh but @ruchowdh as Ms. Chowdhury?? On a piece that is partially on AI bias and with two women on the byline @SarahFKessler @TiffKHsu, no less
1
41
224
120,092
Come on @OpenAI, we've known about chatbots and their limitations for psychotherapy since *1966*.
Just had a quite emotional, personal conversation w/ ChatGPT in voice mode, talking about stress, work-life balance. Interestingly I felt heard & warm. Never tried therapy before but this is probably it? Try it especially if you usually just use it as a productivity tool.
10
49
193
21,662
My contribution to @JuliaConOrg hackathon: Llama2.jl, a port of @karpathy's llama2.c in #JuliaLang. Simply replacing the naive matrix products with BLAS ones affords a >50% speedup on a single CPU!
gist.github.com/jiahao/07a93…
6
23
201
39,709
After almost two months, I'm so excited to finally share where my first day of work today is at!
(P.S. we're still WFH)
17
6
174
New @NeurIPSConf paper on the (limited) usefulness of bias removal methods for deep learning models with my wonderful collaborators @PurdueCS @VectorInst @UWaterloo! papers.nips.cc/paper/2021/ha…
4
22
169
So yeah, anyone who's been BOTH a postdoc AND have industry experience knows EXACTLY why postdocs are called “trainees” and why they are not payed (sic) six figure salaries. It's the invisible hand of the exploitative academic market at play! mobile.twitter.com/sebatlab/…
Anyone who has been a young PI starting a new lab with postdoc hires that are coming from a wide variety of backgrounds with limited experience in YOUR field knows EXACTLY why they are called “trainees” and why they are not payed six figure salaries.
1
6
126
So I got promoted last month to AI Research Director...
does anyone want to come work with me on Responsible AI in financial services?
Other positions open in general AI Research and other specialties like NLP. US / UK for now
Thrilled to be hiring AI Research Scientists specializing in Responsible AI! Design AI systems with fairness, ethics, explainability, robustness, and safety as first-class concerns. Please share widely! jpmc.fa.oraclecloud.com/hcmU… jpmc.fa.oraclecloud.com/hcmU…
13
14
138
Soft launch of Urban Hawker today, Anthony Bourdain's project to bring Singaporean food stateside
You know it's legit when the first words we hear coming in is "going to peng san liao"
Anyway here is some chicken chop curry rice
3
16
123
4) academics who complain about paying postdocs more than 50k/yr need to be honest about whether they're actually offshoring costs off their grant accounts onto public assistance for food and housing schemes for postdocs who can't supplant their incomes
1
7
100
Sorry but #2 is in fact false. You -can- show that training models can create new biases, which get amplified when applied at scale.
4
3
104
Imagine feeding a young family in Cambridge MA on a single postdoc income of $50k, paying full market value for rent and childcare, and you'll begin to appreciate why many postdocs are also on food stamps.
1
4
96
this is the dynamic of "soft money" that is incredibly exploitative in academia and yet rarely discussed in public. In soft money positions, universities are not obliged to pay you a single dollar! and whatever salary they promise is actually contigent on the sponsoring PI
1
5
94
Oh, you're a computer scientist?
Can you fix my printer?
@AcademicChatter
9
3
83
Thrilled to be hiring AI Research Scientists specializing in Responsible AI! Design AI systems with fairness, ethics, explainability, robustness, and safety as first-class concerns. Please share widely! jpmc.fa.oraclecloud.com/hcmU… jpmc.fa.oraclecloud.com/hcmU…
6
30
98
There was a point where MIT was several months late in processing a conference travel reimbursement and caused my bank account balance to be consistently in the red, causing me no end of headaches paying rent and utilities
1
3
92
Imagine being served steak on a tombstone and mashed potatoes in an urn. That's how culturally insensitive this photo shoot is
😱TOO stunned to see this in @guardian @ObserverUK Lunar New Year recipes. You most definitely DO NOT pair a LNY dish w joss paper 金銀衣紙 which=hell money you burn for the dead. You DO NOT talk abt death in the New Year! DM me next time if you are unsure theguardian.com/lifeandstyle…
2
32
98
Being a postdoc sucks because it's an inherently exploitative position that only came into existence when demand for TT jobs started to exceed supply, with a general turning point in many US STEM academic disciplines in the early 90s. Ymmv depending on your local market dynamics
2
2
86
Alan Edelman: Students don't often come to class. When I observe this lecture hall as a matrix, it's a sparse matrix. Today, it's a dense matrix
#JuliaCon
1
13
98
8,974
It's my great pleasure to share that I will be serving as an Ethics Review Chair at the 2023 @NeurIPSConf.
7
5
92
14,090
Edward Lee's classic "The Problem with Threads" is worth reading eecs.berkeley.edu/Pubs/TechR…
3
13
96
7,753
You can have a PhD from Harvard and if you're a Black woman, people will still find ways to drag you down. All of us who have been talked down to and had to fight to be recognized for our expertise should be taking notes.
Misogynoir is the canary and we are still in the coalmine
Harvard President Claudine Gay will resign Tuesday afternoon, bringing an end to the shortest presidency in the University's history, according to a person with knowledge of the decision.
From @HaidarEmma & @cam_kettles
l8r.it/8Rfy
8
15
77
12,443
As a postdoc, I was also not eligible for PI status at MIT, which is needed to actually put your name on research grants you apply for. Without PI status, postdocs don't get credit for funding they pull in - their only reward is to pull in funds to continue their own salaries
1
2
83
Summary: 1) @MITGradUnion @MITPDA get my unreserved support 2) if you really want a TT job, it's practically essential to get internal politics aligned in your favor 3) non-default "isms" like racism, sexism, LGBTQ+ prejudice all add up
1
2
79
Chemistry major here (BS 2002) and I have ☕️ to spill on the role organic chemistry plays in chemical education and its ubiquitous culture of bullying students as a charade for promoting academic rigor🧵
NYU dismissed a chemistry professor after students said his class was too hard, igniting a debate about the relaxing of academic standards. nyti.ms/3UVGyFp
2
21
78
LightGraphs, the @JuliaLanguage package for network analysis, beats its competition in other languages in all algorithmic tasks timlrx.com/2020/05/10/benchm…
1
23
86
Living off a postdoc salary at MIT was more difficult than living off my PhD student stipend at UIUC. As a grad student, rent was 20% of my stipend, and I could afford to travel several times a year.
1
3
76
who in practice is likely to delegate the task of writing grants to fund your own salary back to you, and they get to claim it's part of the training they provide to postdocs as trainees! How incredibly convenient
1
1
71
My brother passed his PhD oral defense today. A milestone for us first-generation college students! 🎓
3
78
[Reposted with homepage ss]
While the rest of us are struggling to survive 2020, the incredible @pumas_ai team @ChrisRackauckas @ANoackJensen @YingboMa1 @vijay_ivaturi ++ have enabled Moderna's Covid vaccine with state of the art numerical algorithms and @JuliaLanguage!!! 👏👏
2
20
77
So, so, so fed up with being told that the only problem with AI bias is that algorithms are being trained without enough Black people's faces.
2
6
66
Time and time again the two-language problem resurfaces. One of the really neat experiences with #JuliaLang is not needing to know internal C/flisp/Fortran-library implementation details to contribute.
One of the weird things about working in the #Python community is that our core technologies are mostly not written in Python.
If someone wants to contribute code, now they have two problems.
- Michael Droettboom at #SciPy2023
11
79
6,695
As a postdoc, rent was 37% of my salary and I had to worry about the $ difference between single-trip subway/bus tickets and getting a weekly or monthly pass. And I had expectations to send money home to my family as I was the only one pulling in an income at the time.
1
69
(and yes their pet pick turned out to be a blond-haired, blue-eyed European man. Quelle surprise!)
The third interview was a full two day affair, with interviews running 8-6 with about 2 dozen faculty. About 2-3 of them no-showed for interviews *at their own offices*
2
2
69
UPDATE: REPL mode added
Talk to language models from the Julia REPL!
This marks the end of the weekend project. If you like this and want more, please contribute to github.com/rai-llc/LanguageM…!
UPDATE: Prompting now works!!!
Let #JuliaLang tell you stories about dogs using nanoGPT's stories15M model
The reference implementation get 118.9 tokens/s, my Julia code gets 314.9 tokens/s
3
12
80
20,386
Most non-Chinese people have no idea how Chinese languages have been systematically oppressed in the 20th C to establish Mandarin supremacy as part of creating a pan-Chinese identity. Cantonese is a way to preserve Hong Kong identity before it gets integrated away
We can finally share our BIG $1M announcement - it was really hard to keep this a secret! Let me tell you. We worked. SO. HARD. for this, I could cry. All this Canto solidarity 🔥😭#Cantonese #SaveCantonese #Stanford #Canto #LunarNewYear #ChineseNewYear #YearOfTheTiger 🐅🥳
9
16
73
Let's add on top of ALL that the very real uncertainties of staying within the very narrow confines of academic visas. Most US postdocs are on J-1 visiting scholar visas that restrict or prevent spouses (on J-2 companion visas) from working.
2
1
70
One last thing - being brought up in a traditional Chinese household meant being indoctrinated on the importance of academic success. It's very easy to see the pursuit of an academic career as the pinnacle of that success. But few such parents understand the opportunity costs
1
3
64
"There are only two languages that I still use regularly, and @WolframResearch is one of them"
An incredible meeting of minds as Alan introduces @stephen_wolfram x @JuliaLanguage @JuliaConOrg
1
6
74
6,188
So thrilled to see @3blue1brown teaching @MIT_CSAIL Computational Thinking in @JuliaLanguage with Professor Philip the Corgi @DavidPSanders @AlanEdelman1 and James Schloss @MIT #JuliaLang piped.video/watch?v=vxjRWtWo…
1
20
72
Despite these odds I managed to land a few in-person interviews. One of them has yet to reimburse me for my travel expenses that are now 11 years old. Another told me off-the-record the only reason I was picked was so that they could justify their pet pick over "an MIT applicant"
1
1
68
My friend Samuel Watson at Brown is offering a free introductory online course on data science in R. No homework or exams; mainly labs. Mondays and Wednesdays at 10 AM, starting next week. please share prismia.chat/shared/data-sci…
38
64
Today, I agreed to be an Area Chair of @FAccTConference in the area of Data and Algorithm Evaluation! Looking forward to seeing everyone's submissions on how to evaluate bias, privacy, trust, and other risks in the area of responsible AI
2
6
68
OpenAI: matmul considered slightly conscious
DeepMind:
Today in @nature, with @EPFL, the first deep reinforcement learning system that can keep nuclear fusion plasma stable inside its tokamaks, opening new avenues to advance nuclear fusion research.
Paper: dpmd.ai/fusion-paper
1
5
70
This is also a widely-shared mistranslation of the original. It's not that Japanese ministers decided not enforce copyright on AI training, rather that they are commenting that current law does not provide a mechanism for doing so.
4
10
64
4,199
I grew up in an era in Singapore where for the sake of national unity, parents were strongly discouraged from teaching their children "dialects" like Hokkien and Cantonese in favour of Mandarin. My registered name is in pinyin and I was constantly mistaken for a mainlander
Did you know: lots of 1st gen Asian parents don't teach their kids their native language in order to make integrating at school easier. My parents never spoke Thai to me, and I didn't want to learn b/c of my own proximity to whiteness. It's a constant source of regret for me.
6
6
58
Prof. Tim Holy is one of the unsung heroes of the @JuliaLanguage community - he's contributed to just about everything! The only question is how he finds the time to write code AND run a lab #JuliaLang
Today @markkitti & @preibischs hosted @JuliaLanguage’s Tim Holy at @HHMIJanelia - It was impressive to see the power of #Julialang - Let’s make a deep dive into the language 👩🏻💻💪😀 Check out @JeffBezanson et al.’s blog post 👉 julialang.org/blog/2012/02/w…
1
12
66
5) academics who complain that grant funding needs to change before they can pay more can leverage their privilege to pressure their peers who are reviewing and advising on funding decisions into not actively driving young researchers into financial hardship
1
3
57
Tank's glorious return to @MIT_CSAIL @MIT for @JuliaConOrg @JuliaLanguage
Say hello if you see him! He loves attention
7
67
3,048
and I was better off not working there. This is the kind of knowledge afforded only to the in-group of the academy that would only be manifest way too late for an outsider like me.
1
3
61
And I have enormous respect for people who tough it out in academia anyway. I fulfilled a childhood dream of becoming a scientist at MIT! and more power to anyone who wants to fulfill similar aspirations
2
2
54
In CS for example, it's now standard practice to supplement PhD stipends with industry internships that can easily exceed the entire annual academic income of the studentship itself. But not all disciplines are so lucky - oftentimes, augmenting income means more teaching work
2
2
59
UPDATE: Prompting now works!!!
Let #JuliaLang tell you stories about dogs using nanoGPT's stories15M model
The reference implementation get 118.9 tokens/s, my Julia code gets 314.9 tokens/s
My contribution to @JuliaConOrg hackathon: Llama2.jl, a port of @karpathy's llama2.c in #JuliaLang. Simply replacing the naive matrix products with BLAS ones affords a >50% speedup on a single CPU!
gist.github.com/jiahao/07a93…
1
4
65
25,570
Research as it should be done - Terry Tao and coauthors have completely revised their original paper to include an exhaustive literature review of the eigenvalue-eigenvector identity which they had formerly claimed as known. terrytao.wordpress.com/2019/…
1
16
57
People said this about IBM Watson and many other failed AI products going back to the 1970s. Time and time again we see people fail to appreciate the complexities of domain expertise and see only simulacra of the most superficial layers
More and more, I think ChatGPT is coming for the cognitive class. It's going to replace what doctors do -- hearing symptoms and making diagnoses -- before it changes what nurses do -- helping patients get up and handle themselves in the hospital.
3
2
55
17,780
Finally meeting the one-and-only @sarahookr who's managed to multitask in New Orleans this week by attending both #NeurIPS2022 AND #workboat2022
1
3
63
An American told me to stop saying I'm from Singapore we're basically China since we speak the same language. Oddly, he wasn't happy with the comment I made about how you English people still think the British Empire rules the world
2
1
54
Front row seat for the tagging of #JuliaLang 1.0. Literal blood, sweat and tears (and m keys) were spilled over the years. So happy to witness a part of history - a programming language hitting 1.0 - with so many friends and collaborators #JuliaCon2018 github.com/JuliaLang/julia/p…
13
54
If you're running a research event and claim to value diversity and inclusion, you gotta at least try to pronounce Chinese and Indian names properly. Especially when presenting work on these people. That's literally 1/3 of the world's population whose names get mangled regularly.
5
6
45
I would never have stood a chance as a first-generation college graduate who had no idea academia was so heavily stacked in favor of children of academics and knew how to credential their applications with publications
My jaws keep dropping as I go through 70 PhD applicant files. People w/ 2 coauthored papers & an interesting solo writing sample don’t even make it to the top 10 in my pile.
The level of knowledge, research experience & passion these kids bring to the table is just remarkable! 🤩
2
6
47
6,098
I have a #JuliaLang joke, but the time to first joke is too long
2
9
54
GAYS CAN FUCK IN SINGAPORE BUT IF YOU WANNA GET MARRIED GO MIGRATE AND GET HIGH PAYING JOBS ABROAD AND WHEN YOU SUCCEED DON'T FORGET TO GUAI GUAI COME TO OVERSEAS SINGAPOREAN EVENTS WHERE WE NAG YOU TO COME BACK TO SINGAPORE MACHAM GOT FREE FOOD AND MUST COLLECT GOODIE BAG HOR
1
18
49
and not just that, I was told that I should be grateful to even get the experience to interview! My better connected friends (read: other postdocs who weren't first-generation college graduates like me, but the children of faculty) told me "that dept is known to be toxic anyway"
1
3
50
On top of the general financial misery of being a postdoc, I graduated in May 2009 at the depth of the economic crisis. There were simply no jobs to be had in industry. I was told that I was lucky that someone actually wanted to make me a postdoc offer
1
1
50
The next 3 years, I was on the TT market. Whereas there were normally ~30 positions a year available, in those 3 years there were a grand TOTAL of 3 positions advertised in the entire US. You bet I applied for everything and more besides.
1
50
this of course is not to say that industry is perfect. Petty politics is everywhere. but it's really hard to say that it's more exploitative when typical fresh-PhD industry salaries now start at the market equilibrium price of $200k, easily four times that for the academic market
1
5
47
The expectations for research and teaching statements inflated wildly in those years. Even when I sought feedback, I was told that no one had good advice on how to write them anymore because what used to be considered a solid entry just two years ago was now barely table stakes
1
45
First Kareem doesn't directly engage with my attempts to address his misconceptions. Then he attacks me claiming I'm trying to control his speech. And now the trolls like Jon Stokes are out in full force attacking me. The last thing I need is getting fired. This is my TED talk.
10
46
Ok I couldn't resist. Support for llama2 is now available!
This is pure CPU execution of llama2-13b-chat in single precision, getting a very decent 17 tokens/s. Lots of room for speedup with parallelization and quantization à la GGML
UPDATE: REPL mode added
Talk to language models from the Julia REPL!
This marks the end of the weekend project. If you like this and want more, please contribute to github.com/rai-llc/LanguageM…!
1
6
56
10,062
It's really tiring to see the umpteenth research presentation of loan applications and fairness that promptly makes unrealistic characterizations of how loans work IRL. How are we to progress on algorithmic fairness if researchers don't bother to engage domain experts????
10
48
#JuliaLang accepted to Google Summer of Code 2020! Plan your proposal and find a mentor now summerofcode.withgoogle.com/…
18
49
#JuliaLang cracks the top 20 list of programming languages on the Tiobe Index
Just a few years ago it was a big deal that we made it on the list at all
Onwards and upwards 📈
tiobe.com/tiobe-index/
11
51
4,039
At NYC @JuliaLanguage meetup. The legendary @ChrisRackauckas demonstrates algorithm development in #JuliaLang via Discourse blog post. Composition of differential equation solvers with number types with uncertainty = free error analysis
1
16
53
Just 4 more issues to #JuliaLang 1.0! As @quinn_jacobd put it, "smells like alpha time" @JuliaLanguage github.com/JuliaLang/julia/m…
2
18
45
If you have spare CPU/GPU cycles to spare, consider helping @jchodera 's lab and the @foldingathome team find candidate treatments for #COVID-19. Read about the science and the real-time updates here: foldingathome.org/2020/03/10…
19
46
Navigating #NeurIPS2023 with a service dog is like walking through honey. Every 10 steps someone wants to say hello to Tank or I. (No prizes for guessing who's more popular)
Even Tank has had enough attention for the moment
4
1
53
10,119
I'm thrilled to announce the launch of @llc_rai, my new consultancy focused on practical, yet responsible AI solutions for the evolving needs of businesses and regulators that is informed by the very latest academic research.
2
8
47
I worked hard for every dollar in university. I scrimped on dental work I couldn’t afford. I didn’t buy a car, relying on the campus bus network or walking everywhere. If my advisors preferred saving paperwork by not hiring students, I wouldn’t be here today. Insulting
4
3
43
The mass firing of what seems to be most of the Twitter META team, starting from its director, is a chilling reminder of why responsible AI in companies is difficult and why the external force of regulation is needed to keep vicissitudes in check.
1
8
45
Between BLM, Timnit and other events of the year, I've become convinced that being silent is no longer an option. We have to move beyond colorblindness firmly into antiracism, and similarly with sexism, homophobia and other forms of bias
3
3
40
I really enjoyed the efforts of #JuliaCon2020 organizers and community to reproduce interactivity online. Scheduled talk releases, Discord audio and text chats, and Gather.town socials works quite well! And enjoying the cameos on the podcast :) #JuliaLang
1
5
47
@WolframLanguage now has support for @JuliaLanguage
interop! Wow!
#julialang
reference.wolfram.com/langua…
15
47
And these bullies are everywhere. Reviewers, colleagues, bosses. Recognize the behavior and hold them accountable
"Bullying is a means for mediocre scientists to rise to the top. Some star academics reached their position because they are bullies, not in spite of it." - Excellent piece about bullying in academia (and really any other professional environment). nature.com/articles/s41562-0…
1
7
43
12,176