AI safety has a familiar story line: researchers push the frontier, engineers harden systems, and policymakers react once the technology is already embedded in daily life. But the headline idea behind “AI safety isn’t just a lab problem—it’s a society problem” flips that script. It argues that the most consequential work for keeping advanced AI safe will not happen only in research labs or corporate security teams. It will happen in public institutions—through laws, oversight, procurement rules, auditing requirements, and democratic accountability that can keep pace with capabilities that evolve faster than any single technical fix.
That framing matters because AI risk is not purely a question of whether a model can be made “aligned” in the abstract. Real-world harm often emerges from incentives, deployment choices, data practices, and institutional failures. A system can be technically impressive and still be unsafe if it is deployed without adequate safeguards, if its outputs are used in high-stakes contexts without meaningful verification, or if the people affected by its decisions have no voice in how those decisions are made. In other words, safety is not only a property of models; it is also a property of systems, markets, and governance.
The labs will continue to build the technology. That part is unavoidable. But the rules—who gets access, what uses are permitted, what standards must be met, how incidents are investigated, and how harms are remedied—cannot be left to the same actors who benefit from rapid deployment. If AI safety is treated as a technical afterthought, society ends up paying the price for speed. If it is treated as a public responsibility, the incentives shift toward transparency, accountability, and measurable risk reduction.
A new kind of governance challenge
AI governance is often discussed as if it were a single policy decision: pass a law, set a threshold, require compliance, move on. The reality is more complex. AI systems are diverse—ranging from narrow tools that assist doctors to general-purpose models that can generate text, code, images, and instructions. Their risks vary accordingly. Some harms are immediate and visible: fraud, misinformation, deepfakes, and automated harassment. Others are slower and structural: labor displacement without retraining pathways, discriminatory outcomes that become normalized, and the erosion of trust when citizens cannot tell what is real.
Even when a model is “safe” in a controlled environment, deployment can introduce new failure modes. A chatbot connected to customer service can become a persuasive tool for scams. A model integrated into hiring workflows can amplify bias if the training data reflects historical discrimination. A system used for content moderation can suppress legitimate speech if its evaluation criteria are opaque or misaligned with democratic values. These are not merely technical issues; they are governance issues—about how institutions decide what counts as acceptable performance and who bears responsibility when things go wrong.
This is why the argument that citizens and elected representatives must set the rules is not rhetorical. It is a recognition that safety requires legitimacy. People accept constraints more readily when they understand the trade-offs and have a channel to influence them. Without that, governance becomes either performative—full of promises but light on enforcement—or adversarial, with public trust collapsing under the weight of opaque decisions.
From “alignment” to accountability
Technical alignment aims to reduce the chance that a system behaves in ways that conflict with human intent. But accountability asks a different question: even if a system behaves unpredictably, who is responsible for preventing foreseeable misuse and for responding when harm occurs?
Consider the difference between two scenarios. In one, a lab releases a model with documentation about limitations and safety mitigations. In another, a company deploys the same model in a way that bypasses safeguards—perhaps by enabling features that increase autonomy, by lowering friction for risky requests, or by using the model as a decision-maker rather than an assistant. The second scenario may produce harm even if the underlying model was developed with caution. The safety problem shifts from “Can the model be aligned?” to “Did the deployer follow the rules, and were those rules strong enough?”
Accountability also includes the ability to audit. Auditing AI is not as simple as checking a single equation. It involves evaluating behavior across contexts, measuring performance on relevant tasks, testing for vulnerabilities, and verifying that safeguards remain effective after updates. That requires standards and resources—things that governments and independent regulators can provide, but that private actors may not prioritize if doing so slows product cycles.
A society-first approach would treat auditing and incident reporting as core infrastructure, not optional extras. It would require that high-risk deployments maintain logs sufficient for investigation, that safety evaluations be conducted against defined benchmarks, and that regulators have access to evidence when something goes wrong. This is how safety becomes enforceable rather than aspirational.
The role of citizens: more than voting once
When people hear “citizens must set the rules,” they may imagine a distant process: elections, committees, and legislation. But citizens influence AI safety in more immediate ways too—through public consultation, participation in standard-setting, scrutiny of procurement decisions, and pressure on institutions to publish clear explanations.
Public oversight can take several forms:
First, transparency expectations. Citizens cannot meaningfully evaluate safety claims if they are buried in technical jargon or locked behind corporate confidentiality. Governance can require plain-language disclosures about what an AI system does, what it is not designed to do, and what risks it carries. Transparency is not about exposing trade secrets; it is about enabling informed consent and informed oversight.
Second, participatory rulemaking. For high-impact systems—those affecting employment, credit, healthcare, education, policing, or access to services—public input should shape the boundaries of acceptable use. This is where democratic decision-making becomes practical: communities can highlight risks that technologists might overlook, such as cultural bias, language-specific failure modes, or the social consequences of automated decisions.
Third, civic monitoring. Even with laws, enforcement depends on detection. Civil society organizations, journalists, and researchers play a crucial role in identifying harmful patterns, documenting incidents, and pushing for corrective action. A society-first approach would treat this monitoring as part of the safety ecosystem, including protections for whistleblowers and researchers who test systems responsibly.
In short, citizens are not just the final authority at election time. They are the ongoing feedback loop that keeps governance grounded in lived experience.
Elected representatives: translating values into enforceable standards
Elected representatives face a difficult task: turning broad public values—fairness, safety, privacy, accountability—into specific legal obligations that can be enforced. That translation is where many AI governance efforts stumble. Vague rules create uncertainty and allow companies to comply on paper while doing little in practice. Overly technical rules can become obsolete quickly as models evolve.
A better approach is to focus on outcomes and risk tiers rather than trying to legislate every technical detail. For example, lawmakers can require that systems used in high-stakes contexts meet defined safety and reliability thresholds, undergo independent evaluation, and provide meaningful recourse to individuals harmed by errors. They can mandate that certain categories of use—such as biometric identification in sensitive settings or automated decision-making that materially affects rights—require heightened scrutiny and explicit authorization.
Risk-tiering is not a silver bullet, but it helps align regulatory effort with potential harm. Low-risk uses can be governed with lighter-touch requirements, while high-risk uses demand stronger controls. The key is that the classification must be transparent and contestable, not a loophole that companies can game.
Representatives also need to ensure that enforcement has teeth. If regulators lack funding, technical expertise, or the power to impose penalties, rules become symbolic. Effective governance includes the ability to investigate, to require remediation, and to halt deployments when safety conditions are not met.
A unique take: safety as a public utility mindset
One reason AI safety debates feel stuck is that they treat safety as a competitive advantage. Companies want to market their models as “responsible,” but the incentives to cut corners remain strong when timelines and market pressure reward speed. A society-first approach suggests a different mindset: safety should be treated like public infrastructure.
Public utility thinking means that some aspects of AI safety—auditing frameworks, incident reporting channels, standardized evaluation methods, and independent testing capacity—should be supported as shared resources. Just as societies build roads and electrical grids that no single company can fully control, they can build safety systems that make it harder for harmful deployments to slip through.
This does not mean government runs every model. It means government sets the baseline conditions under which models can be used, and it invests in the capacity to verify compliance. When safety infrastructure is shared, the burden does not fall entirely on individual firms, and the public gains confidence that safety claims are not merely marketing.
Procurement as a lever: the fastest path to safer deployment
Governments are major buyers of AI systems. Procurement rules can therefore become one of the most immediate levers for safety. If public agencies require vendors to meet strict standards—independent audits, documented limitations, robust monitoring, and clear accountability—then safer practices spread quickly through the supply chain.
Procurement can also force clarity about responsibility. Contracts can specify what happens when systems fail: who investigates, what remediation is required, what compensation is offered, and how long-term monitoring is handled. This turns safety from a one-time certification into an ongoing obligation.
Moreover, public procurement can reduce the “race to deploy” dynamic. If agencies commit to buying only systems that meet safety requirements, vendors have a strong incentive to invest in compliance and evaluation. That investment can then spill over into commercial markets.
The hardest part: governing autonomy and emergent behavior
As AI systems become more capable, the risk is not only that they will produce incorrect outputs. The risk is that they will act—taking steps toward goals, using tools, interacting with environments, and generating plans that are difficult to predict in advance. Autonomy changes the safety equation because it introduces causal impact. A wrong answer is harmful; a wrong action can be catastrophic.
Governing autonomy requires rules that address both capability and context. A system that is safe when used as a passive assistant may become unsafe when given permissions to execute