{"opportunitySpacesMenu":{"title":"Mathematics for Safe AI ","subTitle":"Opportunity space","exploreText":"Explore this space","image":null,"imageMobile":null,"backLink":{"isActive":false,"text":"Back to Opportunity spaces","href":"/opportunity-spaces","title":"Back to Opportunity spaces"},"signUpLink":{"isActive":false,"text":"Sign up for updates","href":"/sign-up-for-updates-qa","title":""},"items":[{"title":"Overview","link":{"isActive":true,"text":"Mathematics for Safe AI","href":"/opportunity-spaces/mathematics-for-safe-ai","title":"Mathematics for Safe AI"}},{"title":"Safeguarded AI","link":{"isActive":false,"text":"Safeguarded AI","href":"/opportunity-spaces/mathematics-for-safe-ai/safeguarded-ai","title":"Safeguarded AI"},"children":[{"title":"Meet the Creators","link":{"isActive":false,"text":"Meet the Creators","href":"/opportunity-spaces/mathematics-for-safe-ai/safeguarded-ai/meet-the-creators","title":"Meet the Creators"}},{"title":"Funding","link":{"isActive":false,"text":"Funding","href":"/opportunity-spaces/mathematics-for-safe-ai/safeguarded-ai/funding","title":"Funding"}}]},{"title":"Mathematics for Safe AI","link":{"isActive":false,"text":"Mathematics for Safe AI","href":"/opportunity-spaces/mathematics-for-safe-ai/mathematics-for-safe-ai","title":"Mathematics for Safe AI"},"children":[]}],"activeBackgroundColor":"#ffcd9b"},"modules":[{"type":"op-header","image":{"src":"/media/agphimuo/aria_safeai_image.png?rmode=pad&format=webp&quality=100","alt":"ARIA Safeai Image"},"title":"Mathematics for Safe AI ","intro":"We don’t yet have known technical solutions to ensure that powerful AI systems interact as intended with real-world systems and populations. \n\nA combination of scientific world-models and mathematical proofs may be the answer to ensuring AI provides transformational benefit without harm.\n","text":"","colour1":"orange","colour2":"purple","btnColour":"purple","breadcrumb":[{"isActive":false,"text":"Home","href":"/home","title":""},{"isActive":false,"text":"Opportunity spaces","href":"/opportunity-spaces","title":""},{"isActive":true,"text":"Mathematics for Safe AI","href":"/opportunity-spaces/mathematics-for-safe-ai","title":""}],"links":{"opLink":{"isActive":true,"text":"Mathematics for Safe AI","href":"/opportunity-spaces/mathematics-for-safe-ai","title":""},"programmeLinks":[{"isActive":false,"text":"Safeguarded AI","href":"/opportunity-spaces/mathematics-for-safe-ai/safeguarded-ai","title":""}],"seedLinks":[{"isActive":false,"text":"Mathematics for Safe AI","href":"/opportunity-spaces/mathematics-for-safe-ai/mathematics-for-safe-ai","title":""}]},"hideGradient":false},{"type":"standout-copy","title":"What is an opportunity space?","text":"

Opportunity spaces are areas of research that we believe are ripe for breakthroughs. They are defined by our Programme Directors, and must be highly consequential for society, under-explored relative to its potential impact, and ripe for new talent, perspectives, or resources to change what’s possible.

","link":null,"color":null,"isWhiteText":false,"fullWidth":false},{"type":"technical-areas","title":"Core beliefs","text":"The core beliefs that underpin this opportunity space:","items":[{"label":"1.","title":"","text":"

Future AI systems will be powerful enough to transformatively enhance or threaten human civilisation at a global scale —> we need as-yet-unproven technologies to certify that cyber-physical AI systems will deliver intended benefits while avoiding harms. 

"},{"label":"2.","title":"","text":"

Given the potential of AI systems to anticipate and exploit world-states beyond human experience or comprehension, traditional methods of empirical testing will be insufficiently reliable for certification —> mathematical proof offers a critical but underexplored foundation for robust verification of AI.

"},{"label":"3.","title":"","text":"

It will eventually be possible to build mathematically robust, human-auditable models that comprehensively capture the physical phenomena and social affordances that underpin human flourishing —> we should begin developing such world models today to advance transformative AI and provide a basis for provable safety

"}]},{"type":"spacer-comp","cssSizeClass":"medium","height":0},{"type":"rich-text-content","text":"
\n
\n
Observations
\n

Some signposts as to why we see this area as important, underserved, and ripe.

\n
\n

 

\n
\n
\n
\"Observations
\n
\n
\n
\n
\n
","mediumLayout":false,"fullWidth":false},{"type":"rich-text-content","text":"

Download as a PDF here, or the accessible version here.

","mediumLayout":false,"fullWidth":false},{"type":"spacer-comp","cssSizeClass":"medium","height":0},{"type":"media-text","mediaRight":true,"title":"Programme spotlight: Safeguarded AI","text":"

As AI becomes more capable, it has the potential to power scientific breakthroughs, enhance global prosperity, and safeguard us from disasters. But only if it’s deployed wisely.

\n

Current techniques working to mitigate the risk of advanced AI systems have serious limitations, as they can’t be relied upon in practice to ensure reliability and safety.

\n

Backed by £59m, this programme looks to combine scientific world models and mathematical proofs ARIA is looking to construct a ‘gatekeeper’ – an AI system designed to understand and reduce the risks of other AI agents. If successful, we’ll unlock the full economic and social benefits of advanced AI systems while minimising risks.

","backgroundColour":"orange","link":{"isActive":false,"text":"Discover more ","href":"/opportunity-spaces/mathematics-for-safe-ai/safeguarded-ai","title":"Explore the Safeguarded AI programme"},"mediaType":"image","video":null,"image":{"src":{"mobile":"/media/krsbyppo/davidad-workshop.jpeg?width=400&height=400&format=webp&v=1db4d770da20310","tablet":"/media/krsbyppo/davidad-workshop.jpeg?width=440&height=550&format=webp&v=1db4d770da20310","desktop":"/media/krsbyppo/davidad-workshop.jpeg?width=600&height=750&format=webp&v=1db4d770da20310"},"alt":"A photo of a group of people sitting around the table and looking at davidad writing on a whiteboard. ","title":""},"secondaryImage":null,"mediaSecondMobile":false,"caption":"","fullWidth":false,"mediaVertical":false},{"type":"spacer-comp","cssSizeClass":"medium","height":0},{"type":"team-cards","items":[{"image":{"src":"/media/4xuaytke/davidad-pd-1.jpg?width=1000&height=1000&format=webp&v=1db3acb9411d3d0","alt":"Davidad PD 1"},"name":"David 'davidad' Dalrymple","text":"davidad is a software engineer with a multidisciplinary scientific background. He’s spent five years formulating a vision for how mathematical approaches could guarantee reliable and trustworthy AI. Before joining ARIA, davidad co-invented the top-40 cryptocurrency Filecoin and worked as a Senior Software Engineer at Twitter.","position":"Programme Director"},{"image":{"src":"/media/dzcho2ye/yasir-bakki.jpg?rxy=0.4821575042765275,0.40508862206975416&width=1000&height=1000&format=webp&v=1db4d76e69b2ad0","alt":"A photo of Yasir Bakki smiling against a grey background."},"name":"Yasir Bakki","text":"Yasir is an experienced programme manager whose background spans the aviation, tech, emergency services, and defence sectors. Before joining ARIA, he led transformation efforts at Babcock for the London Fire Brigade’s fleet and a global implementation programme at a tech start-up. He supports ARIA as an Operating Partner from Pace. ","position":"Programme Specialist"},{"image":{"src":"/media/zqzhenhm/nora-ammann.jpg?rxy=0.4681235005520459,0.4680113289268752&width=1000&height=1000&format=webp&v=1db4d76de7857b0","alt":"A photo of Nora Ammann smiling at the camera against a white background."},"name":"Nora Ammann ","text":"Nora is an interdisciplinary researcher with expertise in complex systems, philosophy of science, political theory and AI. She focuses on the development of transformative AI and understanding intelligent behavior in natural, social, or artificial systems. Before ARIA, she co-founded and led PIBBSS, a research initiative exploring interdisciplinary approaches to AI risk, governance and safety.","position":"Technical Specialist"}],"title":"Meet the programme team","text":"

Our Programme Directors are supported by a Programme Specialist (P-Spec) and Technical Specialist (T-Spec); this is the nucleus of each programme team. P-Specs co-ordinate and oversee the project management of their respective programmes, whilst T-Specs provide highly specialised and targeted technical expertise to support programmatic rigour.

"},{"type":"spacer-comp","cssSizeClass":"medium","height":0},{"type":"standout-copy","title":"Explore our other opportunity spaces","text":"

Our opportunity spaces are designed as an open invitation for researchers from across disciplines and institutions to learn with us and contribute – a variety of perspectives are just what we need to change what’s possible.

","link":{"isActive":false,"text":"See our Opportunity Spaces","href":"/opportunity-spaces","title":""},"color":"ccbbed","isWhiteText":false,"fullWidth":false}],"scriptsAtTop":" \n\n\n\n\n\n\n\n\n
\n\n","scriptsAtBottom":"","id":2170,"languageCode":"en","pageName":"Mathematics for Safe AI","scrollSmoothness":0.0,"gtmCode":"GTM-T8Z5F55","topNav":{"logo":{"text":"ARIA - Home","href":"/media/hsxblxrf/logo-aria-black.svg"},"nodes":[{"text":"About us","href":null,"target":null,"id":null,"nodes":[{"text":"About us","href":"/about-aria","target":null,"id":"2207","nodes":[],"noOpener":false,"noReferrer":false},{"text":"Our team","href":"/about-aria/our-team","target":null,"id":"2159","nodes":[],"noOpener":false,"noReferrer":false},{"text":"Programme Directors","href":"/about-aria/our-team/programme-directors","target":null,"id":"2831","nodes":[],"noOpener":false,"noReferrer":false},{"text":"Reporting and policies","href":"/about-aria/reporting-and-policies","target":null,"id":"2160","nodes":[],"noOpener":false,"noReferrer":false}],"noOpener":true,"noReferrer":false},{"text":"What we do","href":null,"target":null,"id":null,"nodes":[{"text":"How we work","href":"/how-we-work","target":null,"id":"2817","nodes":[],"noOpener":false,"noReferrer":false},{"text":"Live opportunity spaces","href":"/opportunity-spaces","target":null,"id":"2215","nodes":[],"noOpener":false,"noReferrer":false},{"text":"Emerging areas","href":"/emerging-areas","target":null,"id":"2963","nodes":[],"noOpener":false,"noReferrer":false},{"text":"Activation Partners","href":"/activation-partners","target":null,"id":"2174","nodes":[],"noOpener":false,"noReferrer":false}],"noOpener":true,"noReferrer":false},{"text":"Funding","href":null,"target":null,"id":null,"nodes":[{"text":"Funding opportunities","href":"/funding-opportunities","target":null,"id":"2164","nodes":[],"noOpener":false,"noReferrer":false},{"text":"Applicant guidance","href":"/funding-opportunities/applicant-guidance","target":null,"id":"2162","nodes":[],"noOpener":false,"noReferrer":false},{"text":"Being funded by ARIA","href":"/funding-opportunities/the-creator-experience","target":null,"id":"2433","nodes":[],"noOpener":false,"noReferrer":false},{"text":"FAQs","href":"/funding-opportunities/faqs","target":null,"id":"2649","nodes":[],"noOpener":false,"noReferrer":false}],"noOpener":true,"noReferrer":false},{"text":"Careers","href":"https://aria-jobs.teamtailor.com/","target":"_blank","id":null,"nodes":[],"noOpener":false,"noReferrer":false}],"headerLinks":[{"target":"","isActive":false,"text":"Become a Programme Director","href":"/about-aria/our-team/programme-directors/cohort-3","title":""},{"isActive":false,"text":"Opportunity spaces","href":"/opportunity-spaces","title":""}],"menuLinks":[{"isActive":false,"text":"Contact us","href":"/contact","title":""},{"isActive":false,"text":"Terms of use","href":"/terms-and-conditions","title":""}],"activeItem":"2170","menuText":"Menu","menuTextClose":"Close menu","search":{"isActive":false,"text":"Search","href":"/search","title":""},"animationOffText":"Motion: Off","animationOnText":"Motion: On"},"footer":{"header":"Help us drive momentum ","text":"ARIA is a non-departmental public body, sponsored by the Department for Science, Innovation + Technology.","copyright":"Copyright 2026","headerLinks":[{"isActive":false,"text":"Contact","href":"/contact","title":""},{"target":"_blank","isActive":true,"text":"Careers","href":"https://aria-jobs.teamtailor.com/","title":""}],"socialHeader":"Follow our progress","footerLinks":[{"isActive":false,"text":"Privacy Policy","href":"/privacy-policy","title":""},{"isActive":false,"text":"Terms and Conditions","href":"/terms-and-conditions","title":""},{"isActive":false,"text":"Cookie Policy","href":"/cookie-policy","title":""},{"isActive":false,"text":"Accessibility Statement","href":"/accessibility-statement","title":""},{"isActive":false,"text":"Accessibility Support","href":"/funding-opportunities/accessibility-support","title":""},{"target":"_blank","isActive":true,"text":"Careers","href":"https://aria-jobs.teamtailor.com/","title":""},{"isActive":false,"text":"Supply to us","href":"/supply-to-us","title":""}],"socialItems":[{"link":{"target":"_blank","isActive":true,"text":"ARIA Research Twitter","href":"https://x.com/aria_research","title":""},"type":"x"},{"link":{"target":"_blank","isActive":true,"text":"LinkedIn","href":"https://www.linkedin.com/company/advanced-research-and-invention-agency","title":""},"type":"Linkedin"},{"link":{"target":"_blank","isActive":true,"text":"ARIA Research Substack","href":"https://ariaresearch.substack.com/","title":""},"type":"SubStack"}],"formText":"","formSubmit":"Sign up","formId":"","recaptchaSiteKey":"6LcuyY4bAAAAAE2OzRFfA7vDWyohfZXyl2uTuHLa","verificationToken":"","redirectUrl":null,"submitMessage":"","formTheme":""},"seo":{"title":"Mathematics for Safe AI","description":"Mathematics for Safe AI explores how combination of scientific world-models and mathematical proofs may be the answer to ensuring AI provides transformational benefit without harm.","keywords":"Mathematics for Safe AI, ARIA opportunity spaces, R&D, math research projects, mathematics research projects, UK research and development","includeInSitemapXml":true,"canonicalTag":""},"sharing":{"og":[{"property":"og:title","content":"Mathematics for Safe AI"},{"property":"og:description","content":"We don’t yet have known technical solutions to ensure that powerful AI systems interact as intended with real-world systems and populations. A combination of scientific world-models and mathematical proofs may be the answer to ensuring AI provides transformational benefit without harm."},{"property":"og:image","content":"https://aria.org.uk/media/u01hahvb/aria-web-card-15seeds2.jpg?width=600&format=webp&quality=100&v=1db669bc60a00f0"},{"property":"og:url","content":"https://aria.org.uk/opportunity-spaces/mathematics-for-safe-ai"}],"twitter":[{"name":"twitter:card","content":"summary_large_image"},{"name":"twitter:site","content":"@ARIA_research"},{"name":"twitter:title","content":"Mathematics for Safe AI"},{"name":"twitter:description","content":"We don’t yet have known technical solutions to ensure that powerful AI systems interact as intended with real-world systems and populations. A combination of scientific world-models and mathematical proofs may be the answer to ensuring AI provides transformational benefit without harm."},{"name":"twitter:image","content":"https://aria.org.uk/media/u01hahvb/aria-web-card-15seeds2.jpg?width=600&format=webp&quality=100&v=1db669bc60a00f0"},{"name":"twitter:url","content":"https://aria.org.uk/opportunity-spaces/mathematics-for-safe-ai"}]},"pageType":"opSpacePage"}