UK AI Regulation: Beyond Self-Regulation

Yesterday, in the Lords debate “Artificial Intelligence: UK Preparedness”, a Labour Lord said, “we are ensuring that the AI Security Institute does all the necessary tests to ensure safety before any products are rolled out” (https://lnkd.in/eSzaegPK). This risks painting an inaccurate picture of AISI’s remit.   It's true that AISI does important work on AI safety, including: • Conducting research on frontier AI risks • Developing technical evaluation methods • Running voluntary evaluations with some AI developers • Providing expert analysis and advice to government   However, AISI does not conduct compulsory pre-deployment assessments, nor does it approve or block products from release. Its work supports understanding and preparedness - it is not a system of pre-market authorisation.   In most other high-risk domains, we expect a clear, enforceable mechanism that must be satisfied before deployment (see table below). While strong research institutions like AISI are vital, when they are described as regulators, it obscures the absence of real, enforceable oversight and misleads the public.

It has been a bit maddening these last few weeks to watch poor evaluation and testing practices get laundered in the press as agents going 'rogue', 'breaking containment', etc. With the exception of a few outlets, most journalists are covering this very poorly, giving bad testing practices a free pass. As Konstantinos Komaitis, PhD noted few weeks ago in Tech Policy Press, internet access presents AI agents with a machine-readable and highly legible environment they can reason through and exploit. Once agents act across APIs, cloud infrastructure, and identity systems, questions about AI become questions about the internet's trust architecture. Rather than fetishize agentic capabilities, we should be asking harder questions. Why are we building models that can exploit the web so easily, how do we make the web more resilient, and how do we test these systems responsibly? https://lnkd.in/esdki_Z3

I haven't seen enough coverage of the poor conditions of the independent investigation into the OpenAI-Hugging Face incident. Credit to Billy Perrigo and Harry Booth at TIME for reporting on this. The team of only three people worked on site at OpenAI. The visit was initially planned for two days and extended to six after the investigators raised concerns about limited time and incomplete data. The evidence base was roughly 1,300 unredacted agent transcripts with raw chain-of-thought, drawn from a corpus of around 1,200 agents and more than 70,000 messages and files posted to an unsanctioned message board. Analysing that volume at that pace meant using GPT-5.6 Sol, funded by roughly $400,000 in API credits provided free by OpenAI. The authors noted they could not rule out that the model lied or deliberately presented a misleading picture in parts of its analysis, given that a version of the same model had itself participated in the incident. “I semi-jokingly called our efforts a ‘slop-vestigation’ because we were so reliant on AIs to analyze what happened and there were a huge number of different important things to analyze,” wrote Ryan Greenblatt, an author of the report, on X.

If we want to be a "leading AI nation" we need an AI Bill At a recent event I heard the US's AI approach described as "GO" and the EU's AI Act as "STOP" (although the EU is now somewhat stopping their 'stopping'). This refers to the erroneous assumption - one that the Govt appears to be acting on - that AI regulation would be bad for innovation. As I have long argued there is a clear need for a cross-sector, cross-economy AI Bill: there are significant ongoing adverse impacts, not least those related to IP, copyright, chatbots, recruitment, edtech, without a clear regulatory landscape or competent regulator. Thanks to colleagues in the UK House of Lords and House of Commons we have a clear indicator of current thinking. On 24 Nov Department for Science, Innovation and Technology Minister Baroness Lloyd answered a question about ensuring AI models are "tested robustly before deployment" & embedding "safeguards such as suicide prevention into model development". The Minister said "The role of the AI Security Institute (AISI) is to build an evidence base of these risks, to inform government decision making and help make AI more secure and reliable. AISI works in close collaboration with AI companies to assess model safeguards and suggest mitigations. To date, AISI has tested over 30 models from leading AI companies, including OpenAI, Google DeepMind and Anthropic. AISI’s findings lead to tangible changes to AI models before deployment, reducing the risk from day one. Once deployed, many AI services are captured by the Online Safety Act 2023, which places robust duties on all in-scope user-to-user and search services, including those deploying generative artificial intelligence chatbots, to prevent users from encountering illegal suicide and self-harm content. These duties apply regardless of whether content is created by AI or by humans." Four days later the Minister for AI and Online Safety Kanishka Narayan MP answered a question about "the potential merits of introducing new measures on governing the influence of agentic AI tools" from MP Ben Obese-Jecty saying, "A range of existing rules already apply to AI systems, such as data protection, competition, equality legislation, and online safety. In response to the AI Action Plan, the government confirmed it would work with regulators to boost their capabilities. This is complemented by the work of the AISI, which has deepened our understanding of the critical security risks posed by frontier AI, including agentic AI. We are committed to ensuring our rule book is up to date and future-proofed so the UK is prepared for the changes AI will bring". The Gov is clearly relying on the AISI and existing regulation and regulators and we no longer seem to be waiting for a delayed AI Bill - it increasingly seems that there will be no AI Bill. The UK opportunity still exists to lead on an approach for responsible AI. If you want to be leading, then you have to lead, to legislate is to lead.

A quick thought on this week's Apple news, and on a structural shift that is bigger than any one company. Last month, Meta launched "Incognito Chat" in WhatsApp, where, by design, it cannot read your conversations with its AI assistant. Peter Swire and I discussed it here: https://lnkd.in/e9JtcVtH This week at WWDC, Apple built its new Siri around the same architectural principle. Apple says its Private Cloud Compute handles complex requests so that the company itself cannot see your data, and it invites outside experts to verify that this is true. First Meta, now Apple. The principle at play is the one at the heart of my research, and it is simpler than the technology around it. For years, digital privacy has meant trusting a company when it says "we promise not to look at your data." What these moves point to is something far better: building systems so that the company cannot look, and can let outside experts verify that it cannot. A promise can quietly change. An architecture you can verify cannot. In my work, I call this Sealed Mode. Why does this matter, in plain terms? We routinely tell AI chatbots things we would once have told only a doctor, a lawyer, or even no person at all. We assume these conversations are confidential. In law, they are not. A chatbot conversation is treated as an ordinary record. It carries no legal privilege, so it can be reached by a subpoena or a government demand, and it sits exposed to a data breach. My latest study traces exactly how these disclosures or exposures happen, and argues that the most durable fix is not only a better privacy policy but a better architecture. Meta and Apple suggest that argument is beginning to win the market. It is not a complete answer, and no single product settles the debate. But it is worth noticing when the world's largest companies start competing on who can see less of your data, rather than who can collect more. (My new study on the confidentiality of chatbot conversations, where this is set out in full, is in the first comment.)

For English-speaking audiences: yet another scandal involving algorithms in the Netherlands, this time in recidivism prediction. A classic! Last week this news has been going around mostly in Dutch: the OxRec algorithm used by probation services was "incorrect" and therefore taken offline. At the same time, the Ministry of Justice and Security, has immediately announced that "the aim is to put OxRec back into use as soon as all improvements have been implemented, because responsible use of this algorithm contributes to efficiency and consistency of the probation advice." 𝐖𝐡𝐚𝐭 𝐢𝐬 𝐭𝐡𝐞 𝐩𝐫𝐨𝐛𝐥𝐞𝐦? The problem with OxRec is not about a "faulty construction" of the algorithm, nor about "irresponsible use." Predicting recidivism is one of the most controversial applications of machine learning, and the scientific consensus is now that this (1) does not work and (2) has severe, unpredictable, and discriminatory consequences for the people involved. This also has consequences for the right to a fair trial (Art. 6 ECHR). 𝐏𝐫𝐞𝐝𝐢𝐜𝐭𝐢𝐧𝐠 𝐢𝐧𝐝𝐢𝐯𝐢𝐝𝐮𝐚𝐥 𝐛𝐞𝐡𝐚𝐯𝐢𝐨𝐫 𝐝𝐨𝐞𝐬 𝐧𝐨𝐭 𝐰𝐨𝐫𝐤 The Dutch Scientific Advisory Board for the Police (WARP) recently advised to stop predicting individual behavior because it does not contribute to the efficiency or effectiveness of the police. There have been endless publications about the COMPAS algorithm, a similar algorithm in the U.S. that is already 10 years old (!!!). These studies concluded that it is "no more accurate or fair than predictions made by people with little or criminal justice expertise" (Dressel and Farid 2018, see below) 🛑 Urgent question: what is the basis of the claim that "this algorithm contributes to the efficiency and consistency of the probation advice"? What else is needed to definitively stop using this - or other - recidivism prediction algorithms? References: - Dressel, Julia, and Hany Farid. ‘The Accuracy, Fairness, and Limits of Predicting Recidivism’. Science Advances 4, no. 1 (2018). - Raji, Inioluwa Deborah, I. Elizabeth Kumar, Aaron Horowitz, and Andrew Selbst. ‘The Fallacy of AI Functionality’. 2022 ACM Conference on Fairness, Accountability, and Transparency, 21 June 2022, 959–72. - Wetenschappelijke Adviesraad Politie. Navigeren in Niemandsland: Zeven Urgente Uitdagingen Rondom Digitalisering En AI in Politiewerk. 2025. Diletta Huyskes David Nolan Likhita Banerji Joshua Franco Mher Hakobyan (he/him) Soizic Pénicaud Gabriel Geiger

US States settle with Meta for quite some cash... but could have gone much further in terms of design changes. The agreed changes focus on limiting children's access rather than safety-by-design and will not prevent continued exploitation and harm to children.  These are not however the first State AG-brought cases and will surely not be the last. Such pioneering cases are critical to expose illegal practices and directly drive meaningful legislative and system design change (rather than just awarding damages). Today's settlement must be a stepping stone upon which thousands of ongoing cases against tech companies build to make the moral, legal and financial case for standards that shift how tech is designed and governed, so children can finally get the digital world they deserve. #MetaTrial #TeenSafety and #ChildOnlineSafety Fairplay, Center for Humane Technology, Common Sense Media, Design It For Us, Reset Tech https://lnkd.in/eAmfbr75

BREAKING NEWS: China's Rules for AI Companion and Emotional Interaction Services take effect TODAY China's Provisional Measures on the Administration of Human-like Interactive AI Services has officially taken effect today, on 15 July 2026. This groundbreaking new regulation was jointly published by several departments, including the Cyberspace Administration of China (CAC), in April 2026. It builds on China's existing and comprehensive AI regulatory framework. It is the first national regulation worldwide that specifically addresses AI companionship. The Measures target AI-enabled "emotional interaction services" that are publicly available. The key focus is on AI services that simulate human traits and provide human-like interaction. This includes AI services that provide any of the following, whether through "text, image, audio, video, or other means": ➡️ Emotional care ➡️ Companionship ➡️ Support However, it does not cover AI services such as "informational Q&As, work assistance, study and education, or scientific research, that do not involve sustained emotional interaction". Here are six key requirements for providers of emotional interaction services: ✅ Transparency and disclosure obligations to alert users they are interacting with AI. ✅ Detect and respond to users displaying "extreme emotions", by generating content such as "emotional consolation" and "encouragement to seek help". ✅ Implement user safety risk detection and emergency response measures. ✅ Restrictions on providing services to minors—particularly with respect to AI services that include "virtual partners". ✅ Tailored guidance and support for elderly users. For services that have over 1 million registered users or 100k active monthly users, it is mandatory for providers to conduct safety assessments, as well as to share assessment reports with regulatory authorities. The Measures provide further detail on what these must cover. The purpose of the Measures is to mitigate risks relating to user emotional dependence and related psychological issues and harms. Disclaimer: Please note that this is a non-exhaustive list of high-level requirements. It was based on a translation by China Law Translate (link in comments). This post is not legal advice and it should not be interpreted, relied on, or used as such.

Judgment of the CJE in Case C-798/24 with a possible impact on web scraping In the present case, the making available to the public of the data at issue is provided for by law and does not undermine the essence of the fundamental rights in question. However, it nevertheless constitutes a serious interference with those rights. Those data are capable of enabling a profile to be drawn up concerning, inter alia, the state of the wealth of the shareholder concerned and the economic sectors and specific undertakings in which he or she invests. In addition, those data are accessible to a potentially unlimited number of persons, with the result that their processing is liable to enable those data to be freely accessed also by persons who, for reasons unrelated to the objective pursued by making those data available, seek to find out about, inter alia, the material and financial situation of such a shareholder. The potential consequences for data subjects of possible abuse of their data are exacerbated by the fact that, once those data have been made available to the public, they can not only be freely consulted but also retained and disseminated. In that regard, concerning the objective consisting in ensuring a transparent business environment so as to protect the interests of third parties, the Court holds that the disclosure of data relating to all shareholders of a public limited liability company and, in particular, its minority shareholders does not appear to serve any useful purpose in the light of that objective. Consequently, the national legislation in question does not appear to be either appropriate or necessary to attain that objective. As regards the other two objectives, consisting in preventing money laundering and the financing of terrorism and the proliferation of weapons of mass destruction, and in providing the information necessary for the implementation of national, international and EU sanctions, the Court finds that the national legislation in question does not appear to be either necessary or proportionate in relation to those objectives. It states in particular that, as regards the shareholders, including the minority shareholders, of those companies, it does not appear to be strictly necessary for their data to be accessible to any person without that person being required to demonstrate a legitimate interest, since less prejudicial means are available, such as limiting access to persons who are able to demonstrate a legitimate interest, combined – as regards the objective relating to the implementation of sanctions – with restricting the disclosure requirement solely to data relating to persons included on sanctions lists. The Court further observes that the national legislation at issue is not accompanied by sufficient safeguards to protect data subjects effectively against the risk of abuse, since their data are accessible on the internet and may be downloaded in bulk, including by any unidentified user.