Kill switches in artificial intelligence: how businesses can manage a growing risk

In 1983, a US artificial intelligence was given control of its nuclear deterrent. It's objective was to win a war with the USSR. It was designed to overcome the supposed frailties of human operators who might pause or question orders to launch their nuclear weapons. When the AI believed that the USSR was about to attack, and its human commanders refused to act, it took over control and started the countdown to fire the US missiles. The AI could not be stopped as there was no kill switch.

Thankfully, this is not true: it is the plot of the classic movie, Wargames. Skip forward 40 years and kill switches for AI are now a live problem. Last month, the US government forced Anthropic to turn off its latest AI versions – Claude Mythos and Fable – for fear that they could be weaponised, cutting off the UK from this technology without any warning. 

This article explains what a kill switch does, the governmental power to use them, and how to mitigate the consequences if IT systems are suddenly switched off.

What are kill switches and why are they used?

A kill switch can be any technical means of preventing access to technology. Given the prevalence of cloud systems, the simplest means will be disconnecting an end-user from a data centre or shutting down the data centre entirely. This will be effective in shutting down AIs which are currently nearly all hosted in third-party data centres.

Deploying kill switches in local devices and on-premise solutions is harder. By its very nature, this requires some form of access to the device from another location in order to issue the shutdown command and also relies on the end-user not jailbreaking the device to prevent the shutdown.

The idea behind a kill switch is to stop dangerous use of technology. The US government's view was that in the wrong hands Claude Mythos could be used to detect and exploit security vulnerabilities in other IT systems, allowing hackers to cause significant harm. A more dystopian angle is that super-intelligent AI could escape human control and an ultimate safeguard is needed to shut it down before that happens. These views are not coming from the fringes – an MP has proposed an amendment to upcoming UK legislation to address exactly this risk (see further below).

What is the legal basis for governments to control technology?

The grounds for governments to order the shutdown of technology are muddled and fragmented. 

The US shutdown of Claude was done using an export control order. The US government directed Anthropic to suspend all access to Mythos and Fable by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees. Without the means to separate US users from non-US users, Anthropic had to disable access for everyone.

Using export control rules in this way raises another technology hot topic – that of technological sovereignty. When a foreign government can cut off access to their technologies with the mere sending of a letter, this creates serious concerns for the UK's national security and exposes the economy to systemic risk. Combined with increasingly blurred lines between national security and economic protectionism, overreliance on foreign technology is becoming inherently risky – and that risk is most acute with AI, which is dominated by US providers. 

In the UK similar powers exist under the Export Control Act 2002. The ECA 2002 (as implemented by the Export Control Order 2008 and other secondary legislation) does not specifically mention artificial intelligence, but does apply to a range of dual-use items, software and technology that could cover artificial intelligence. Like the US, the export control rules are primarily designed to prevent the transfer of high-risk UK technology outside the UK. They would not directly prevent a UK company using UK technology, however this could still cause problems when employees of, or others working for, a UK company are located overseas – just as Anthropic encountered with its own employees.

This gap in the government's powers has led one MP to propose an amendment to the upcoming Cyber Security and Resilience Bill ^[1] to include a "kill switch". The amendment would give the government the power to order the shutdown of an AI system, or a data centre running an AI, where there are reasonable grounds to believe that the technology has been compromised and is likely to cause large-scale disruption to critical infrastructure or harm to human life. 

Similar proposals are being mooted in the US, with US tech companies seeking a new law that provides an objective and transparent framework for the use of kill switches that is not subject to the political pressures that can be seen in the use of export control rules.

The EU already has some kill switch powers in the EU AI Act which, in certain circumstances, allow authorities to order the withdrawal of access to a high-risk AI ^[2]. It is not clear how this would work in practice as the power is new and there may be jurisdictional challenges in its application – for example, where an AI is hosted outside the EU but the adverse impact of that AI is in the EU.

Impact on businesses

Where AI is used to supplement or replace human workflows, an interruption of the AI service could mean business processes coming to a stop. Businesses need to be careful if running core processes entirely by AI. Imagine an accountancy practice that uses an AI agent to onboard new clients, with the agent adding the client details to the billing and matter management systems, running compliance checks, and sending out engagement terms. The loss of the AI could prevent the onboarding of new clients – something that might be tolerated for a few days, but could be disastrous if it continued for weeks or months.

The contract with the AI provider (or the tech provider who is making use of an underlying AI model) is unlikely to offer much protection. Most AI and technology contracts do not guarantee 100% uptime and allow a provider to suspend services if the service would be unlawful, and can usually do so without liability. As an example, Anthropic's standard terms for Claude provide that:

"Anthropic may suspend Customer’s access to any portion or all of the Services if […] Anthropic's provision of the Services to Customer is prohibited by applicable law or would result in a material increase in the cost of providing the Services"; and that "Anthropic will have no liability for any damage, liabilities, losses (including any loss of data or profits), or any other consequences that Customer may incur because of a Service Suspension."

Having the AI model hosted in the UK will not prevent the use of a kill switch. Although domestic hosting is useful for a number of reasons (e.g. data protection compliance), it does not stop a tech company connecting to a data centre and shutting down services.

Furthermore, becoming overly reliant on an AI (or any emerging technology) might give rise to regulatory liabilities. Various sector and critical infrastructure regulations ^[3] place obligations on organisations to ensure operational resilience against a wide range of foreseeable risks. The recent shutdown of Claude Mythos and Fable has crystallised the use of kill switches as a foreseeable possibility that must be addressed within resiliency plans. Failure to do so could result in regulatory action, including penalties.

So what can businesses do?

1. Limit the use of AI in critical processes. This requires strong governance around the adoption of AI to ensure it is only deployed where its shutdown could not have serious adverse consequences, and to make carefully balanced judgments around the risk / reward of deploying AI.
2. Consider using older AI models. The possible need to use a kill switch increases with new and more powerful AI models. Sometimes the most powerful models are not needed to run business processes and older versions will do the job just as well (and more cheaply). This decreases the chance of them being disabled.
3. Design and test resiliency plans. Plans should be drawn up for scenarios where AI services are disabled. These should map out the consequences in terms of business impact and then put in place alternative modes of continuing to operate. This could include being able to switch to another AI provider, falling back on traditional technology, or having manual workarounds.
4. Keep off-site backups. Good cyber security practice already promotes back-ups being kept physically separated from live systems to address environmental risks (fire, flood, vandalism, etc.) or telecommunications interruption with a primary data centre. Having off-site backups would also help ensure ongoing access to the company's data if a primary IT system is disabled by a kill switch.
5. Conduct due diligence on IT suppliers. If an IT supplier is promising AI features, ask questions about the underlying AI models, KPIs, and resiliency plans for an interruption in the AI service.
6. Review contract terms with tech and AI suppliers. Try to negotiate the inclusion of remedies for suspensions in service, thereby moving some of the risk to the IT vendor. This will be difficult for low value / off-the-shelf services, but may be possible in larger outsourcing arrangements where the customer has more bargaining power.
7. Build protection into customer terms. When providing services to customers that are enabled or assisted by AI, consider whether to include suspension rights and limitations on liability in customer contract terms. Wherever possible, do not promise a higher level of service availability than can be secured from IT / AI suppliers.
8. Consider insurance. Some insurance policies, often those connected to cyber insurance cover, will pay out for business interruption costs if IT systems fail even where there has been no cyber-attack. 
9. Use UK AI models. There are very few UK providers of foundational AI models at the moment but this might be an option in the future. The UK government has recognised that reliance on US technology is a problem and wants the UK to build its own foundational AI models. This is being spearheaded by the UK Sovereign AI Fund – an investment fund backed by the UK government. However, these models are years away from commercial deployment at scale.

Footnotes

[1] The primary purpose of this bill is to ensure adequate cyber security measures are implemented by providers of the UK's critical infrastructure. It is currently progressing through Parliament.

[2] E.g. Article 79(5). The EU AI Act is currently under review and may be modified in the future. Also some provisions of the EU AI Act are yet to come into force.

[3] E.g. the Network and Information Systems Regulations 2018 that are set to be revised by the Cyber Security and Resilience Bill discussed earlier.