The rush to make artificial intelligence “ethical” has produced a familiar pattern: invite the right experts, ask them the wrong questions, and then wonder why the answers don’t translate into safer systems.
In recent months, as AI labs and technology companies have moved from experimentation to deployment, they have increasingly turned outward—to theologians, philosophers, and other scholars trained in moral reasoning. The intent is understandable. If machines are going to influence hiring decisions, medical triage, policing, credit, education, and countless other high-stakes areas, then someone should help ensure that the systems behave in ways societies can defend as morally acceptable.
But the problem, according to critics of the current approach, is not simply that ethics is being treated as a branding exercise or a compliance checkbox. It’s that the industry often approaches moral philosophy as if it were a tool for writing rules—something like a software specification—rather than a discipline for clarifying values, responsibilities, and the social processes through which moral judgments are made.
That mismatch matters. When ethics work is reduced to “what constraints should we encode?” the conversation can drift away from the deeper questions that moral philosophy and theology are uniquely positioned to address: Who is accountable when an AI causes harm? What does it mean to respect persons rather than merely avoid prohibited outcomes? How do communities decide what counts as fairness, dignity, or justice? And what happens when moral disagreement is not a bug but a feature of plural societies?
The result is a kind of ethical miscalibration. Companies may end up with documents that sound profound but function like guardrails without steering. They may also create the illusion that morality can be engineered in advance, rather than governed over time.
A growing industry habit: consult, codify, deploy
The modern AI ethics pipeline often looks like this. A company recognizes reputational and regulatory pressure. It forms an internal ethics team or partners with external advisors. It commissions frameworks, risk assessments, and model cards. It asks philosophers and theologians to help define principles such as beneficence, non-maleficence, fairness, transparency, and accountability.
Then comes the translation step: convert those principles into policies, checklists, and technical requirements. The questions asked at this stage tend to become narrower and more operational. Should the system refuse certain requests? Should it prioritize one group over another? What thresholds count as “safe”? Which harms are most important to prevent? How should the model respond when it cannot comply?
These are not trivial questions. But they are often framed as if morality were primarily a matter of selecting the correct set of rules for a particular model behavior. That framing can crowd out the more difficult work: determining how moral responsibility should be distributed across designers, deployers, and institutions; how to handle uncertainty and contested values; and how to ensure that the system’s real-world effects match the moral intentions behind its design.
Philosophers and theologians, critics argue, are frequently brought in late—after key architectural choices have been made—or asked to validate a direction already chosen by commercial incentives and product timelines. In that context, their expertise can be used to polish the language of ethics rather than to challenge the underlying assumptions about what the system is for and who it serves.
The deeper issue: morality isn’t just a list of prohibitions
Moral reasoning is not only about avoiding wrongdoing. It is also about what we owe to one another, how we justify decisions, and how we interpret human dignity in practice. Theology and philosophy have long traditions of grappling with these questions, including the tension between universal principles and culturally specific norms, the role of intention versus outcome, and the difference between legal compliance and moral legitimacy.
When AI ethics becomes a checklist, it tends to focus on the easiest measurable parts of morality: whether the system produces disallowed content, whether it meets certain fairness metrics, whether it logs enough information to satisfy an audit. These are necessary components of responsible development, but they are not sufficient for moral governance.
Consider a common scenario: an AI system denies a loan application. A checklist approach might ask whether the denial was based on protected attributes, whether the model’s error rates are within acceptable bounds, and whether the explanation meets transparency requirements. Those questions are relevant.
But moral philosophy pushes further. Why does the system have the authority to deny someone’s livelihood? What does it mean to treat applicants as persons rather than as data points? How should the institution handle cases where the model is uncertain? What recourse is meaningful for someone who disagrees with the decision? And what obligations does the organization have when the system’s outputs reflect structural inequities that predate the model?
In other words, the moral question is not only “Did the model violate a rule?” It is also “What kind of relationship is the institution creating between itself and the people affected by its decisions?”
This is where the industry’s framing often goes wrong. It treats morality as something that can be fully specified before deployment, rather than something that must be continuously interpreted and justified in context.
The accountability gap: who owns the harm?
One of the most persistent problems in AI governance is the accountability gap. When an AI system causes harm, responsibility can dissolve into a fog of shared roles: the model developer says the deployment was the customer’s choice; the deployer says the model was a vendor product; the vendor says it provided documentation and safeguards; regulators say the system is too complex to attribute causation cleanly.
Ethics consultations sometimes fail to confront this directly because the questions are posed in a way that assumes the moral work is primarily technical. “How do we prevent harmful outputs?” becomes the dominant theme. But moral responsibility is not only about preventing harm; it is about answering for harm when prevention fails.
Theologians and philosophers are often well equipped to address this, but only if they are asked the right questions. Instead of asking them to help define “acceptable behavior,” companies could ask: What does it mean for an organization to be morally responsible for decisions made with AI assistance? What duties arise from deploying a system that predictably affects vulnerable people? How should liability be allocated when the system’s behavior is partly emergent or hard to predict? What does due care require beyond documentation?
These questions are uncomfortable because they imply that ethics is not merely a property of the model. It is a property of the institution and its governance structures.
And governance structures are slow. They require legal changes, organizational redesign, and sometimes political negotiation. That slowness clashes with the industry’s urgency to ship.
The speed problem: ethics as a sprint, not a foundation
AI companies operate under intense competitive pressure. Product cycles are short. Investors want traction. Regulators move carefully but still impose deadlines. In that environment, ethics efforts can become a sprint: produce something quickly that signals seriousness.
But moral frameworks are not like unit tests. They cannot be fully validated in a lab environment. Even if a model performs well on benchmarks, its real-world impact depends on how it is integrated into workflows, how humans use it, what incentives surround it, and what kinds of errors are tolerated.
A system that is “safe” in a narrow sense can still be morally problematic if it systematically disadvantages certain groups, erodes agency, or creates a false sense of objectivity. Conversely, a system that occasionally violates a rule might be morally defensible if it is used in a context where human oversight is robust and the institution provides meaningful recourse.
This is why the question “Can we program morality?” is often the wrong starting point. The more relevant question is “How do we govern moral risk over time?” Governance includes monitoring, auditing, incident response, user education, and mechanisms for contesting decisions. It also includes the willingness to stop using a system when it fails morally, not just when it fails technically.
When ethics is treated as a pre-deployment checklist, governance becomes an afterthought. That is precisely the mismatch critics are pointing to.
The temptation to outsource moral judgment
Another subtle dynamic is the outsourcing of moral judgment. When companies bring in external scholars, they may hope to transfer moral authority to experts. That can be beneficial—expertise matters. But it can also become a way to avoid democratic legitimacy.
Moral decisions in plural societies are rarely settled by a single philosophical framework. Different communities may disagree about what counts as fairness, how much weight to give to individual autonomy versus collective welfare, and what kinds of trade-offs are acceptable. Theology and philosophy can illuminate these disagreements, but they cannot replace the social process through which societies decide.
If the industry uses philosophers and theologians mainly to generate a set of principles that can be adopted unilaterally by corporations, it risks turning moral reasoning into a private contract. The public may be asked to accept moral authority without having participated in the moral deliberation.
This is not an argument against expert input. It is an argument for pairing expert guidance with institutional accountability and public engagement. Ethics should not be something that arrives as a document from above. It should be something that is negotiated through transparent processes, with clear lines of responsibility.
The “wrong questions” in practice
So what are the wrong questions? They vary by company and context, but several recurring patterns stand out.
First, the industry often asks for moral rules that can be directly encoded, rather than for moral reasoning that can guide judgment under uncertainty. Real moral life is full of ambiguous cases. A system will encounter edge conditions, incomplete information, and conflicting values. The question should be how to structure decision-making so that moral judgment remains possible, not how to eliminate ambiguity entirely.
Second, companies sometimes ask for principles without specifying the institutional commitments required to realize them. A principle like “respect for persons” is not self-executing. It requires design choices, user interfaces that support agency, policies for recourse, and training for staff who interact with the system. Without those commitments, principles remain rhetorical.
Third, the industry may focus on output harms while neglecting upstream harms—such as manipulation, coercion, or the erosion of trust. A model that avoids explicit prohibited content can still be used to steer people toward decisions