As artificial intelligence moves from the lab into trading floors, regulators are starting to ask a question that sounds almost mechanical: what happens when the machine runs too fast, or in the wrong direction?
Sarah Breeden, a member of the Bank of England’s policy leadership, raised that issue at an ECB conference, warning that AI-powered trading systems could amplify market swings and, in particular, encourage “herding behaviour”. The phrase is not new in finance, but the mechanism is. In traditional markets, herding emerges when investors interpret the same news through similar heuristics, or when liquidity dries up and everyone reacts to the same price signals. With AI, the speed and similarity of responses can become far more extreme—because models can be trained on overlapping data, optimized for comparable objectives, and deployed with automation that turns signals into orders in milliseconds.
Breeden’s point was not simply that AI can make markets more efficient. It was that efficiency can come with a stability cost if many participants effectively run the same playbook at the same time. When multiple systems respond to the same inputs—macro releases, volatility spikes, order-book changes, or even social and news sentiment—market dynamics can shift from “many independent decisions” to something closer to synchronized behaviour. That synchronization can create feedback loops: price moves trigger model actions, model actions deepen the move, and the deeper move triggers further actions. In such environments, the difference between a controlled adjustment and a disorderly one can be measured in seconds.
The implication, Breeden suggested, is that safeguards may need to be built into the infrastructure of trading itself. One of the most striking examples is the idea of “kill switches”—predefined mechanisms that can halt or constrain automated trading when certain conditions are met. The concept is straightforward: if a system begins to behave unexpectedly, or if its activity crosses thresholds that indicate potential harm, it should be able to stop quickly rather than continue compounding the problem.
But the real debate is more complex than whether kill switches exist. It is about what they should do, who should control them, and how they should interact with the broader market ecosystem. A kill switch that is too blunt could freeze liquidity at precisely the wrong moment. A kill switch that is too narrow might fail to prevent the kind of cascading behaviour regulators worry about. And a kill switch that is purely internal—controlled only by the firm running the model—may not be enough if the risk is systemic, shared across many firms and strategies.
To understand why Breeden’s remarks matter, it helps to look at what AI changes in trading. Traditional algorithmic trading already uses automation, but AI systems often add layers of learning and adaptation. Even when models are not fully autonomous in the human sense, they can still be adaptive in the technical sense: they update parameters, adjust risk limits, recalibrate predictions, or change how they interpret incoming data. That means the system’s behaviour is not fixed. It evolves as market conditions evolve.
Now consider what happens when many firms deploy models that are trained on similar historical periods, use similar features, and optimize for similar performance metrics. Even if the models are not identical, they can converge on similar decision boundaries. When the market enters a regime that resembles the training data—say, a volatility cluster or a liquidity shock—multiple models may interpret the same signals as “the right time to act.” If those actions are correlated, the market can experience a surge in order flow that overwhelms normal price discovery.
This is where “herding behaviour” becomes more than a metaphor. In a world of manual trading, herding is limited by human attention, differing risk appetites, and slower execution. In a world of AI-driven execution, those frictions shrink. Orders can be generated and routed instantly, and risk controls can be implemented automatically. That sounds like a safety improvement—until you realize that safety controls can also become synchronized. If many systems share similar risk triggers, they may all pull back at once, leaving the market thin. Or they may all accelerate when the same trigger suggests opportunity.
Breeden’s warning therefore lands at the intersection of two regulatory priorities: market stability and operational resilience. Regulators have long focused on whether firms can withstand shocks—cyber incidents, outages, or model failures. But AI introduces a different kind of failure mode: not just “the system breaks,” but “the system behaves coherently yet dangerously.” A model can be technically functioning while still producing outcomes that are harmful at the market level. It might be making correct predictions relative to its objective function, while the objective function itself is misaligned with systemic stability.
That is why the kill switch idea resonates. It is a recognition that stopping a malfunctioning system is not enough; sometimes the system is not malfunctioning—it is simply doing what it was designed to do, at scale, under conditions that were not fully anticipated.
Still, kill switches are not a magic button. They raise immediate questions that regulators and industry will have to answer.
First is the question of thresholds. What conditions should trigger a halt? Volatility spikes? Unusual order-book patterns? Rapid changes in spread? Divergence between predicted and realized outcomes? Excessive order-to-trade ratios? A kill switch based on one metric might miss the true risk driver. A kill switch based on many metrics might be too sensitive, triggering false positives and harming liquidity.
Second is the question of timing. In fast markets, delays of even a fraction of a second can matter. If a kill switch requires manual approval, it may be too slow. If it is fully automatic, it must be robust against adversarial or accidental triggers. The system must distinguish between a genuine emergency and a transient anomaly.
Third is the question of scope. Should kill switches stop trading entirely, or should they reduce exposure, widen quoting, or switch to a safer execution mode? A full stop might be appropriate for certain failure modes, but partial constraints could preserve some liquidity while limiting damage. Regulators may prefer graduated interventions rather than binary halts, especially in markets where liquidity provision is essential.
Fourth is the question of governance. Who decides when the kill switch is activated? If each firm controls its own kill switch, the market may still experience correlated behaviour because many firms could activate their switches simultaneously. That could reduce volatility—or it could create a sudden liquidity vacuum. If regulators want coordination, they must consider how to implement it without undermining competition or creating new points of failure.
Fifth is the question of accountability and transparency. If a kill switch is triggered, regulators will likely want to know why. That implies auditability: logs that capture model inputs, decision paths, and risk calculations. It also implies that firms must be able to explain the system’s behaviour after the fact, not just during the event. In other words, kill switches are not only operational tools; they are part of a broader accountability framework.
Breeden’s remarks also highlight a subtle but important point: the risk is not only that AI will trade incorrectly, but that AI will trade similarly. Even if each model is individually safe, the collective outcome can be unsafe. This is a classic systemic risk problem, but AI makes it more acute because it can compress decision-making and increase the probability of correlated actions.
There is another layer to this: the feedback loop between market microstructure and model training. Many AI systems are trained on historical market data, but market microstructure evolves. If a model is trained on a period with different liquidity conditions, it may behave differently in a new regime. Even if the model adapts, it may adapt in ways that are not aligned with stability. For example, a model might learn that certain order-book patterns historically preceded profitable trades, and then it might aggressively exploit those patterns when they appear again—even if the current market structure makes the exploitation destabilizing.
This is why regulators are increasingly interested in “guardrails” rather than only performance metrics. Guardrails can include kill switches, but also include circuit breakers, risk limits, throttles on order rates, restrictions on leverage, and requirements for stress testing under adverse scenarios. The kill switch is the most dramatic guardrail, but it is rarely the only one.
What makes Breeden’s comments particularly relevant is the context: she spoke at an ECB conference, which signals that the issue is not confined to the UK. European regulators have been grappling with how to supervise algorithmic and automated trading, including questions around market abuse, transparency, and resilience. AI adds new complexity to these supervisory tasks. It is harder to monitor a model’s intent than a human’s. It is harder to predict how a model will behave under novel conditions. And it is harder to ensure that the model’s behaviour does not create unintended externalities.
In that sense, kill switches can be seen as part of a broader shift in regulatory thinking: from treating trading technology as a private tool to treating it as a public-risk factor. Markets are shared infrastructure. When automation scales, the line between firm-level risk and market-level risk blurs.
Yet there is a tension regulators must manage. Overly restrictive controls could push trading activity into less transparent channels or encourage firms to design around compliance requirements. If kill switches are mandated without careful calibration, firms might implement them in ways that satisfy paperwork but do not meaningfully reduce systemic risk. Conversely, if kill switches are left entirely voluntary, firms may adopt them unevenly, leaving gaps in protection.
A unique take on the kill switch debate is to view it not as a single device but as a design philosophy: “fail safely.” In engineering, fail-safe design means that when systems encounter unexpected conditions, they move toward a safer state rather than continuing to operate in a potentially harmful mode. Applied to AI trading, fail-safe design would mean that models are not only optimized for profit but also constrained by stability-aware objectives and operational boundaries.
That could involve designing models with explicit risk constraints, using conservative execution policies under uncertainty, and ensuring that the system can degrade gracefully. For example, instead of halting completely, a system might switch to a lower-frequency strategy, reduce order size, or stop placing orders that would worsen imbalance in the order book. The