We help decision-makers navigate the transition to a world with advanced AI, by producing rigorous research and fostering talent.

London, United Kingdom
Today, we’re relaunching as a nonprofit. Our mission remains the same: We are building a global research community, dedicated to helping humanity transition to a world with advanced AI. But we’ll also be exploring new ways to expand our impact. governance.ai/post/the-centr…
13
34
119
What a night! We've officially launched our new London office, bringing together some of the brightest minds in AI governance from academia, industry, government, and civil society. The move from Oxford to London opens exciting new possibilities for collaboration in the UK on critical AI challenges. Thank you to everyone who joined us to celebrate this milestone - your support and insights continue to shape our work. This is just the beginning of GovAI's London chapter. We are excited to continue fostering collaboration on critical AI governance challenges within our new London community and beyond.
3
4
76
4,558
Apply for the GovAI 2025 Winter Fellowship! Fellows join GovAI to conduct research on a topic of their choice, with mentorship from leading experts in the field of AI governance. The deadline to apply is Sunday 11 August 2024. governance.ai/post/winter-fe…
3
16
59
7,322
Increased computational power is central to recent AI progress. Governing compute could be an effective way to achieve AI policy goals, but could also introduce new societal risks. We explore compute governance in a recent multi-author paper.
1
16
53
6,717
Research Fellow applications are open: We are interested in candidates who have a demonstrated ability to produce excellent research and care deeply about the long-run impacts of AI. The role would offer significant research freedom. Apply by Feb 1st: governance.ai/post/research-…
1
22
45
Join us on October 10th at 5pm BST (Midday ET, 9am PT) for a free webinar with Daron Acemoglu on his latest book 'Power and Progress: Our 1000-Year Struggle Over Technology and Prosperity', hosted by GovAI's Economics of AI Lead Anton Korinek. Sign up: governance.ai/post/upcoming-…
13
41
45,510
GovAI is hiring! You can join us as a Research Scholar for a year, or as a Research Fellow on a more permanent basis. The deadline for both applications is Sunday 20 July 2025. Learn more about these roles in the thread and here: governance.ai/opportunities 1/3
6
16
43
6,393
New GovAI Report out: @tobyordoxford considers the development of the atomic bomb, and draws insights that could be applied to similarly transformative technologies such as AI. Click the link below to read the report: governance.ai/research-paper…
2
10
37
New GovAI blog post: @sj_manning discusses what we can learn from existing approaches to predicting AI's impact on work, and proposes ways to improve these predictions through better "automation evaluations". governance.ai/post/predictin…
1
7
40
12,519
GovAI is hiring! You can join us as a visiting Research Scholar for a year, or as a Research Fellow on a more permanent basis. The deadline for both applications is 23:59 GMT Sunday 16 February 2025. Our broad topics of interest currently include — but are not limited to — frontier AI safety frameworks, AI regulation, international governance, technical governance, agent governance, economics of AI, threat modelling, and risk assessment and forecasting. More information about these two roles in the thread below. 1/3
1
5
39
4,994
GovAI is hiring a Chief of Staff The Chief of Staff would serve as the “central node” within GovAI. A range of crucial responsibilities, most of which currently sit with the Director, would be taken over by the Chief of Staff. Apply here: bit.ly/3nJYnbP
15
34
New report on the risks & benefits of open-sourcing highly-capable foundation models. @ea_seger et al. outline how such models could pose extreme risks that need to be weighed against OS benefits, and discuss less risky methods for pursuing some OS goals: tinyurl.com/2522dy7p
2
5
35
7,728
As our 2022 Summer Fellowship comes to a close, we’re proud to highlight what our Summer Fellows have been up to (🧵):
1
3
33
Applications to our Summer Fellowship 2024 are now open. For more information and to apply, click the link below. The deadline for applications is 23:59 GMT on 17 December. governance.ai/post/summer-fe…
15
31
13,486
Applications for our 2025 Summer Fellowship are open! Fellows join GovAI to conduct research on a topic of their choice, with mentorship from leading experts in the field of AI governance. The deadline to apply is 23:59 GMT on Sunday 05 January 2025. governance.ai/post/summer-fe…
1
6
30
3,221
Our 2022 Annual Report is now out. Click the link to read or download the PDF: governance.ai/post/annual-re…
5
27
11,154
GovAI is hiring! You can join our research team as a Research Fellow or spend a year with us as a Research Scholar. In our current Research Scholar round, we would especially like to highlight our interest in candidates who can conduct technical research to inform AI governance decisions, candidates who intend to focus on US policy and work primarily from Washington, DC, and candidates who intend to manage projects or launch new initiatives. The deadline for all applications is 23:59 BST Sunday 7 July 2024. More information about each role in the thread.
2
11
29
4,426
In a new paper, @jide_alaga, @jonasschuett, and @Manderljung propose a set of criteria to judge the quality of safety frameworks such as Anthropic’s Responsible Scaling Policy and OpenAI’s Preparedness Framework. governance.ai/research-paper…
3
9
28
3,069
Our 2024 Winter Fellowship recently ended. Here we share summaries of the projects our Winter Fellows worked on during their three months with us: governance.ai/post/winter-fe…
1
4
27
5,556
New paper! 📄 “Supervising Frontier AI Developers”, by GovAI Research Scholar Peter Wills. (1/3)
3
6
29
4,841
GovAI is hiring: 1/5 We have opened applications for our Research Scholar positions. These are one-year visiting positions. There are several versions of this role (see thread below).
1
8
23
3,874
Applications for our 2024 Winter Fellowship are now open. Click below for more information and to apply. Deadline for applications is midnight on July 23rd, Anywhere on Earth. governance.ai/post/winter-fe…
11
25
6,098
Applications for our Winter Fellowships are open! We encourage you to apply if you have an interest in our work and are considering using your career to study or shape the long term implications of advanced AI. (Application Deadline 1st August 2022) governance.ai/opportunities/…
1
12
26
New GovAI blog post: @bmgarfinkel @Manderljung @ohlennart @RobertTrager @imbenclifford & @ea_seger consider how the next AI Safety Summit can reinforce the world's commitment to an ambitious summit series. governance.ai/post/goals-for…
1
10
26
6,589
GovAI recently held a workshop to consider what the UK-hosted global summit on AI safety should try to accomplish. Click the link below to read our post drawing on those discussions and exploring the possible outcomes of the summit. governance.ai/post/what-shou…
4
24
1,998
New GovAI report by @jjding99 & @jennywxiao is out: "Recent Trends in China's LLM Landscape". Follow the link to read the report: governance.ai/research-paper…
5
24
2,533
A reminder that applications to our Summer Fellowship 2024 are open, and the deadline for submissions is 23:59 GMT on 17th December. For more information and to apply, follow the link below: governance.ai/post/summer-fe…
1
8
20
3,629
Apply for the 2026 Winter Fellowship! Fellows join GovAI in London for 3 months to conduct paid research on a topic of their choice, with mentorship from leading experts in the field of AI governance. Application deadline: 13 July, 2025 at 23:59 BST governance.ai/post/winter-fe…
4
22
3,695
Ben Bucknall & @RobertTrager detail the forms of access to AI systems required for different areas of safety-relevant AI research, and make some recommendations for providing greater researcher access through the development of dedicated ‘research APIs’: governance.ai/research-paper…
1
5
19
2,488
What can technical researchers do to assist AI governance efforts? In a new paper, @ben_s_bucknall and @AnkaReuel et al outline Technical AI Governance – technical analysis and tools for supporting the effective governance of AI.
Super excited to be sharing a new paper, coauthored with @AnkaReuel and many others, on Open Problems in Technical AI Governance. Summary in the thread below.
2
4
21
3,101
Apply for GovAI’s DC Fellowship! Fellows will join GovAI in Washington, DC for 3 months to conduct paid research on a topic of their choice, with mentorship from leading experts in the field of AI policy. Application Deadline: May 25, 2025 at 23:59 ET. governance.ai/post/dc-fellow…
1
7
18
3,566
We are looking for a Research Manager to join our team! The Research Manager will be responsible for managing and continually improving the systems that underlie our research pipeline. For information and how to apply: governance.ai/post/research-…
10
18
4,994
Join @GovAI_ and @aigioxford on November 26 for a live-streamed conversation with Jeff Ding about his new book: Technology and the Rise of Great Powers. For more information, follow this link: shorturl.at/VNDWX
5
18
5,470
We're hiring a Research Management Associate to help run our Seasonal Fellowships in London! Help shape the next generation of AI governance researchers, with potential to become a Research Manager. Visa sponsorship available. Apply by Sunday 9 March 2025. bit.ly/3QoP4fa
2
4
16
5,203
Frontier AI Safety Frameworks (e.g. OpenAI’s Preparedness Framework and Anthropic’s Responsible Scaling Policy) are an increasingly important tool in AI governance – even more so with 16 companies having committed to develop and implement them. We’re excited to support @AIsafetyInst in organising a conference on how these frameworks can be improved.
📢 We're hosting a conference in November on frontier AI safety commitments. With @GovAI_, we'll bring together AI companies and researchers from all over the world in San Francisco to discuss the safer development of AI.
3
16
3,242
The UK Department of Science, Innovation, and Technology has released a major new report on safety practices for frontier AI organisations. GovAI is pleased to have been among the civil society organisations that contributed to this important guide.
With the #AISafetySummit days away, leading AI companies have published their safety policies. In line with this, government has released an overview of emerging safety processes for AI organisations to keep models safe and reduce risk. Find out more 👇 gov.uk/government/news/leadi…
2
15
2,039
@RobertTrager & @ohlennart, along with 10 others from across civil society, academia and industry, released a report on International Governance of Civilian AI. Follow the link to read the report: governance.ai/research-paper…
1
5
15
1,020
Preliminary findings from our new cross-cultural survey of over 13,000 people found an overwhelming consensus for careful management of AI in Europe and the United States. Read the full post by @NoemiDreksler, David McCaffary, @Lauren_A_Kahn, @kkmays, @Manderljung, @AllanDafoe, @mchorowitz, & @baobaofzhang here: governance.ai/post/increasin… Stay tuned for more findings. Full report coming soon!
3
12
1,027
We are pleased to announce an upcoming GovAI Webinar: How Should Frontier AI Models be Regulated? For more information and to sign up, follow the link below: governance.ai/post/upcoming-…
5
13
2,231
The UK and South Korea have announced commitments from 16 global AI tech companies to a set of safety outcomes. GovAI's Director @bmgarfinkel was quoted in the press release, which can be found here: gov.uk/government/news/histo…
1
14
602
We have a new blog post up from @nikhilmulani & @jesswhittles: "Proposing a Foundation Model Information-Sharing Regime for the UK" Link below: governance.ai/post/proposing…
Replying to @RishiSunak
We'll lead on AI at home🇬🇧 @DeepMind, @OpenAI and @AnthropicAI will give us early access to their models to help us understand any potential risks. And we’re launching two new fellowships to enhance machine learning in drugs and food design, and AI scientist training.
4
14
2,102
We are looking for an Executive Assistant to our Director! This role will help the Director prioritise correctly among competing objectives and help leverage his time most effectively. The deadline to apply is Sunday 1 September 2024. shorturl.at/Yw1rP
2
11
2,231
Safety cases are gaining traction as a tool for AI governance. But what does a frontier AI safety case look like in practice? Together with the @AISafetyInst, GovAI researchers @Arthur_Goemans_, @MarieBassBuhl, & @jonasschuett developed a template: arxiv.org/abs/2411.08088
6
13
3,008
A new paper from @jide_alaga and @jonasschuett proposes an evaluation-based coordination scheme for situations in which frontier AI developers discover that their models have certain dangerous capabilities. tinyurl.com/hv4377em
4
13
1,404
Announcing the new GovAI Policy Program (GAPP): We have launched an invite-only pilot of the program this year. To find out more, and to register your interest in joining the next cohort, click the link below: governance.ai/post/announcin…
5
13
1,656
Applications for our 2023 Summer Fellowships are now open: We encourage you to apply if you have an interest in our work and are considering using your career to study or shape the long-term implications of advanced AI. governance.ai/post/summer-fe…
11
11
2,305
Upcoming webinar: Join us on November 27th at 5pm GMT (Midday EST, 9am PST), for a discussion between @Manderljung & @paul_scharre on 'Power in the Age of Artificial Intelligence'. Sign up through the link below: governance.ai/post/upcoming-…
6
12
3,921
How can we bridge the growing trust gap between the public and AI developers? @Manderljung, GovAI's Head of Policy is co-author on a new paper in @ScienceMagazine: Filling gaps in trustworthy development of AI (S. Avin et al.) science.org/doi/10.1126/scie…
1
10
Jakob Mökander, one of our 2022 Summer Research Fellows, has published his project on auditing LLMs, together with @jonasschuett @hannahrosekirk & @Floridi arxiv.org/abs/2302.08500
3
9
2,729
We are looking for a Policy Team Project Manager to join our team. As Policy Team Project Manager, you will facilitate our policy engagement, boost the Policy Team’s research output, & support its smooth functioning. See below for info. and to apply: governance.ai/post/policy-te…
4
10
1,427
A reminder that GovAI Affiliate Dr. Jeffrey Ding is speaking today at noon: If you’re in Oxford you can join us in-person at the Martin School, but the event will also be livestreamed on YouTube at piped.video/live/VhDm347BU-U
Join @GovAI_ and @aigioxford on November 26 for a live-streamed conversation with Jeff Ding about his new book: Technology and the Rise of Great Powers. For more information, follow this link: shorturl.at/VNDWX
2
11
3,408
Research Fellow Research Fellows are experienced researchers who play a central role in GovAI’s research agenda. Their contracts are full-time and last two years on a fixed-term basis, with the possibility of renewal. We welcome candidates from a wide range of academic and professional backgrounds. Successful applicants will have a demonstrated ability to produce impactful research on important AI governance issues, and a strong commitment to positively shaping the lasting impacts of AI. The role offers significant flexibility, access to a broad network of experts, and opportunities for collaboration. 📅 Deadline: 23:59 BST Sunday 20 July 2025 🔗 Learn more: governance.ai/post/research-… 2/3
2
5
11
1,920
GovAI is hiring a Research Manager They would play an indispensable role in allowing GovAI to produce a high volume of high-quality research that contributes to our fundamental mission. Apply here by 20th Feb: governance.ai/post/research-…
1
5
11
GovAI has opened applications for Research Fellow positions. Find out more and apply through the link below. Application deadline is midnight on 9th August, BST. governance.ai/post/research-…
4
9
1,508
Our AI Governance Fellowships – a 3-month research program based in Oxford – will continue. Previous Fellows have gone on to work at places like OpenAI, Stanford, Open Philanthropy, Our World in Data, and the Future of Humanity Institute. governance.ai/opportunities/…
1
1
9
GovAI Research Fellow @jonasschuett's article "Risk Management in the Artificial Intelligence Act" has been published in the European Journal of Risk Regulation @EurJRR doi.org/10.1017/err.2023.1
2
7
795
Applications are open for GovAI Research Scholars (General Track). For more information and to apply, see the link below. The application deadline is Sunday the 7th of August at 23:59 BST. governance.ai/post/research-…
5
9
How have top ML researchers' forecasts of AI changed over time? New pre-print from GovAI's @baobaofzhang @NoemiDreksler @Manderljung @AllanDafoe with co-authors @charliegiattino @Lauren_A_Kahn @mchorowitz Summary of the results here:
3
9
We have opened applications to GovAI Research Scholar positions. For more information and to apply to the Policy Track of this role, follow the link below: governance.ai/post/research-…
2
3
9
1,545
We are looking for a Survey Analyst (3-month contractor FT/PT) The successful applicant will help conduct and analyse surveys on public and expert opinion on artificial intelligence and AI governance. Application Deadline: 10th August 2022 governance.ai/post/survey-an…
2
5
8
GovAI is hiring for an Operations Manager and Operations Associate! Our operations team enables our staff to do the best work they can, and executes major programmes and events like our summer fellowships and seminar series. Deadline: 15 May at 22:00 governance.ai/post/operation…
3
9
In a new GovAI blog post "The AI Summit Series: What Should Its Niche Be?", @_LuciaVelasco considers what niche the AI Summit Series could occupy within the growing international AI governance landscape. governance.ai/post/the-ai-su…
4
8
2,449
GovAI is hiring. We are looking for a Talent Management Associate to help improve, and recruit for, GovAI's research programmes. Click the link below for more information and to apply. Applications for this position will close on May 3rd, 2023. governance.ai/post/talent-ma…
1
5
7
2,170
Research Scholar: Research Scholar is a one-year visiting position more strongly focused on career development. In this role, you will have the freedom to pursue many types of work. This could include conducting policy research, social science research, or relevant technical research; engaging with and advising policymakers; or starting and managing applied projects. Over the course of the year, you will deepen your understanding of the field, connect with a network of experts, and build your skills and professional profile, all within an institutional home that offers both flexibility and support. 📅 Deadline:  23:59 GMT Sunday 16 February 2025 🔗 Learn more: governance.ai/post/research-…
6
1,579
We’ll be hosting our inaugural Governance of AI conference in the San Francisco Bay Area, in Fall 2022. We expect to open a call for papers later this year.
1
7
Research Fellow: Research Fellow contracts are full-time and last two years fixed-term, with the possibility of renewal. We are open to candidates with a wide range of research interests and backgrounds, who have a demonstrated ability to produce excellent research and want to influence the lasting impacts of AI. The role offers significant flexibility, access to a broad network of experts, and opportunities for collaboration. 📅 Deadline: 23:59 GMT Sunday 16 February 2025 🔗Learn more: governance.ai/post/research-… 2/3
1
6
1,619
Applications are open for GovAI Research Scholars (Policy Track). For more information and to apply, see the link below. The application deadline is Sunday the 7th of August at 23:59 BST. governance.ai/post/research-…
4
7
How will transformative AI affect our economy and society, labor markets, inequality, growth, and economic policy? @akorinek, GovAI's Economics of AI Lead, recently released a free course on @coursera: The Economics of AI coursera.org/learn/economics…
2
7
Applications are open for GovAI Research Scholar (Policy Track) roles. For more information and to apply, see the link below. The application deadline is Sunday the 12th of April at midnight, Anywhere on Earth. governance.ai/post/research-…
4
6
1,507
How should we handle the risks of AI agents? In a paper at FAccT 2024, @_achan96_ and a multi-institutional team of co-authors argue that we need information about how they will be used.
AI agents, which could accomplish complex tasks with limited human supervision, are coming down the pipe. How do we manage their risks? Our new @FAccTConference paper argues that we need visibility---information about the use of agents---and investigates how to obtain it. 🧵
2
8
1,402
New GovAI blog post out: GovAI Research Scholar Elizabeth Seger @ea_seger considers various meanings of "AI democratization" - meanings which sometimes conflict. governance.ai/post/what-do-w…
2
7
756
New paper by Elizabeth Seger, Aviv Ovadya, Ben Garfinkel, Divya Siddarth, and Allan Dafoe argues that “democratising AI” is much more complex and nuanced than just open-sourcing models. governance.ai/research-paper…
3
8
963
We’ll begin offering prizes to undergraduate and graduate students who produce excellent AI governance research in 2022. governance.ai/opportunities/…
1
1
7
GovAI is searching for a Research Management Associate to join our team. For more information about the role and to apply, please follow the link below: governance.ai/post/research-…
6
624
@ohlennart, @manderljung, @emmabluemke, @roberttrager offer their perspectives and summarize the paper in this GovAI blog post: governance.ai/post/computing…
1
5
303
Research Scholar The Research Scholar role is a one-year visiting position. It is designed to support the career development of AI governance researchers and practitioners, while offering an opportunity to do impactful work. As a Research Scholar, you will have freedom to pursue many different projects. This could include conducting policy research, social science research, or technical research; advising policymakers; or starting new applied projects. Over the course of the year, you will deepen your understanding of the field, connect with a network of experts, and build your skills and professional profile within an institutional home that offers both flexibility and support. 📅 Deadline: 23:59 BST Sunday 20 July 2025 🔗 Learn more: governance.ai/post/research-… 3/3
1
4
7
1,647
They also introduce & draw on a new resource — the AI Survey Hub for Attitudes and Research Exchange (AI SHARE) database — developed at the GRAIL lab @purduepolsci, which aggregates AI public opinion survey data from over 200 studies conducted 2014-2023. governance.ai/research-paper…
4
947
In the coming year, we’ll be expanding our team. We’re opening Research Fellow positions. These roles offer research freedom, access to a network of expertise, and opportunities for collaboration. governance.ai/post/research-…
1
1
6
In April we hosted an online discussion: Paul Scharre & Helen Toner on AI Capabilities and the Nature of Warfare Video link below. piped.video/LjGwEx5Ob0U
6
Read the full paper on SSRN: papers.ssrn.com/sol3/papers.… And a summary on the GovAI blog: governance.ai/post/the-case-… (3/3)
1
3
5
1,958
GovAI is looking for a Head of Operations to shape and lead our operations team, supporting our programmes and our people, and help us accomplish our mission. For more information and to apply, please follow the link below: governance.ai/post/head-of-o…
1
3
6
1,076
@Manderljung, Joslyn Barnhart, Alexis Carlier, @NoemiDreksler, @akorinek, Anne le Roux, Robert Trager, and Eoghan Stafford make up the rest of the team. We also have a global affiliate community that is spread across leading universities, companies, and think tanks.
1
6