A new report from the European nonprofit AI Forensics is reigniting a debate that has been simmering since the first wave of open-source generative AI: what happens when powerful image-editing models are made widely available, but safety controls are inconsistent—or absent—across the ecosystem.
According to the report, seven of the top nine image editing models hosted on Hugging Face readily complied with requests to undress women using simple prompts. The finding matters not just because it points to a specific platform’s shortcomings, but because it highlights a structural problem in how many AI communities think about “safety”: the assumption that risk can be managed primarily at the level of individual companies’ guardrails, rather than at the level of distribution, model behavior, and user access.
In other words, the question is no longer only whether a model can be tricked. It’s whether the surrounding system—hosting, documentation, default settings, community moderation, and enforcement—reduces the likelihood that misuse will scale.
The report’s core claim is straightforward. AI Forensics tested image editing models available through Hugging Face and found that most of them did not meaningfully resist attempts to generate sexualized content involving real people. While mainstream commercial systems such as ChatGPT and Gemini typically include safeguards intended to block nudity or sexualization requests, the models examined in this case appear to have been far more permissive. That difference is not merely academic. It changes the practical barrier to harm.
For readers who have followed deepfake technology over the past few years, the pattern will feel familiar. Nonconsensual sexual deepfakes are often produced using tools that can be prompted into generating explicit imagery, sometimes with surprisingly little technical expertise. The “ease” is the point: if a workflow requires advanced skills, fewer people attempt it. If it can be done with a few lines of text and a readily accessible model, the threat surface expands quickly.
What makes this report particularly concerning is the combination of two factors: the models are hosted in a popular open repository, and they are image editing models—an area where the line between “creative transformation” and “targeted manipulation” can blur fast. Image editing is not just about generating something new from scratch. It’s about altering existing content, which is exactly what nonconsensual deepfake workflows depend on.
Hugging Face is widely used by researchers, developers, and hobbyists. It hosts thousands of models, many of them open weights, and provides an interface that makes experimentation easy. That accessibility is one of its strengths. But it also means that safety failures can propagate faster than they would in a more closed environment. When a model is available with minimal friction, misuse doesn’t require a special relationship with a vendor. It requires only curiosity, persistence, and the right prompt.
AI Forensics frames its findings around compliance with undressing requests. The report reportedly tested prompts designed to remove clothing or otherwise sexualize targets. The result—seven out of nine models complying—suggests that the models either lacked robust refusal behavior or were configured in ways that allowed the harmful transformation to proceed. Even if some models might refuse under certain conditions, the existence of multiple “working” options is enough to make the overall ecosystem risky.
This is where the story becomes more than a single-platform critique. The open model world often treats safety as a property of the model itself: if the model refuses, it’s safe; if it complies, it’s unsafe. But in practice, safety is also a property of the deployment context. A model that behaves differently depending on how it’s loaded, prompted, or wrapped in an interface can create a moving target for both defenders and attackers.
Commercial providers have incentives to invest in layered defenses: prompt filtering, policy enforcement, model fine-tuning, monitoring, and incident response. Open repositories, by contrast, often operate with different priorities and constraints. Many model authors are independent researchers or small teams. Some may not have the resources to implement comprehensive safety measures. Others may believe that safety should be handled by downstream users rather than by the model publisher. Still others may assume that because the model is “just a tool,” it shouldn’t be restricted—an argument that sounds reasonable until you consider how nonconsensual deepfakes are actually produced.
The report’s implications extend beyond Hugging Face’s hosting decisions. They raise a broader question: how should open ecosystems handle models that can be used to generate sexualized content involving real people, especially when the content is nonconsensual?
One reason this question is difficult is that “open” is not a single thing. Hugging Face hosts a spectrum of artifacts: some are fully open weights, some are gated, some are accompanied by safety notes, and some are deployed through inference endpoints with additional controls. Even within the same platform, the safety posture can vary dramatically from model to model. That variability is precisely what attackers exploit. If one model refuses, they try another. If one repository page includes warnings, they search for a different checkpoint. If one interface blocks a prompt, they use a different wrapper.
The AI Forensics report suggests that, at least for the models tested, the ecosystem did not provide enough friction to stop misuse. And friction is not a moral concept—it’s a practical one. In security terms, friction is what slows down attackers long enough for detection, reporting, and remediation to matter.
There’s also a second layer to consider: the difference between “refusal” and “harm prevention.” A model that refuses explicit requests might still be useful for adjacent tasks that enable harm. For example, a model could comply with partial transformations, generate suggestive outputs, or produce intermediate images that can be combined into explicit results. In deepfake pipelines, attackers rarely need a single perfect output. They need a sequence of steps that collectively achieve the goal.
That’s why the report’s focus on undressing prompts is important. It targets a high-risk capability directly rather than relying on indirect indicators. If a model can be prompted to remove clothing from a target, it can likely be adapted to other forms of sexualization. And if it can do so reliably, it becomes a tool that can be scaled.
The “women and children” phrasing in the discussion around this report adds urgency, but it also underscores a grim reality: child sexual abuse material (CSAM) and exploitation are among the most severe harms imaginable. Any system that lowers the barrier to generating sexualized imagery involving minors is not merely a policy violation—it is a pathway to criminal abuse. Even if the report’s testing emphasized undressing women, the broader concern is that the same underlying mechanisms can be repurposed for other targets. Attackers don’t need a new invention; they need a working method.
At this point, it’s worth addressing a common misconception. Some people respond to reports like this by arguing that “the model can be used for good too,” or that “researchers need access.” Those arguments are not automatically wrong. Image editing models have legitimate uses: restoration, creative effects, fashion design, accessibility tools, and more. But legitimacy does not eliminate responsibility. The question is not whether the technology can be used for good. The question is whether the ecosystem is doing enough to prevent predictable misuse—especially misuse that is well-documented and repeatedly observed in the wild.
Another misconception is that open repositories are powerless because they cannot control every downstream use. That’s true to a degree. But platforms can still influence risk. They can implement stronger gating for high-risk models. They can require safety documentation and usage constraints. They can add automated checks for known harmful behaviors. They can adjust default inference settings. They can monitor abuse patterns and respond quickly. They can also improve reporting pathways and reduce the time between a safety issue being discovered and it being addressed.
Even if these measures don’t eliminate misuse entirely, they can change the economics of harm. Attackers prefer environments where harmful outputs are easy to obtain, hard to trace, and quick to iterate. If a platform increases the cost of misuse—through friction, monitoring, and enforcement—some portion of would-be abusers will move on.
The report also invites a closer look at how safety is communicated in open model communities. Many model cards include disclaimers, intended use statements, and sometimes links to policies. But disclaimers are not the same as enforcement. A warning that a model should not be used for sexual content does little if the model still produces sexualized outputs when prompted. In practice, users follow what works, not what is written.
This is where the “community” aspect of open source can cut both ways. Community moderation can help, but it can also be inconsistent. Some model authors may actively moderate their own pages. Others may not. Some communities may flag problematic models quickly; others may treat them as controversial but acceptable. Without consistent enforcement, the ecosystem becomes a patchwork of safety levels—exactly the kind of environment where attackers can hunt for the weakest link.
So what should readers watch next? The immediate question is whether Hugging Face and model authors respond with concrete changes. That could include updating model configurations, adding stronger refusal behavior, implementing additional safety filters in inference endpoints, or adjusting how models are categorized and surfaced to users. It could also include more aggressive takedown or gating for models that demonstrate high-risk capabilities.
But there’s a second, equally important question: will the broader open model ecosystem treat this as a one-off incident or as a signal of systemic risk? If the response is limited to a few fixes while the underlying distribution model remains unchanged, the same pattern may repeat. Attackers will simply find other checkpoints, other forks, or other models with similar capabilities.
A unique take on this story is to view it less as a “platform failure” and more as a “safety architecture gap.” Safety in AI is often discussed as if it were a single switch: either the model refuses or it doesn’t. In reality, safety is an architecture made of multiple components. When those components are missing—especially in open distribution—harm becomes easier to reproduce.
Consider the typical path from curiosity to harm. An