When AI products move from demos to day-to-day usage, the conversation changes fast. The early excitement—agents that can “do things,” models that can “reason,” workflows that feel almost magical—gives way to a more grounded set of questions: How much does it cost per successful outcome? What happens when latency spikes? Can you debug failures without spelunking through logs for hours? And perhaps most importantly, how do you structure the system so that improvements in one part don’t accidentally break another?
Guillermo Rauch, CEO of Vercel, recently discussed these realities with TechCrunch, framing the next phase of AI product building as an engineering problem as much as a research one. His core point is simple but consequential: once teams optimize for production, they start looking at price/performance. That shift doesn’t just change budgets—it changes architectures. And one of the biggest architecture debates emerging right now is whether models should be split off from agents, rather than bundled together as a single monolithic “AI brain” that everything depends on.
This isn’t a purely academic distinction. In practice, “models” and “agents” represent different layers of responsibility. The model is the capability engine: it generates text, follows instructions, and produces intermediate reasoning signals. The agent is the orchestration layer: it decides what to do next, when to call tools, how to handle multi-step tasks, and how to recover from errors. When those layers are tightly coupled, teams often end up paying for flexibility they don’t need—or losing flexibility they do need.
Rauch’s comments land in the middle of a broader industry trend: developers are increasingly treating AI systems like production software systems, not like experimental notebooks. That means thinking in terms of interfaces, deployment boundaries, observability, and cost controls. It also means acknowledging that “best model” is not always the same thing as “best system.”
The price/performance pivot: why it changes everything
In the demo era, the dominant metric is usually capability. If a model can produce impressive outputs, the system feels successful—even if it’s slow or expensive. But production forces a different calculus. A feature that costs a few cents per request might be fine at small scale, but become untenable when usage grows. A workflow that works 90% of the time might be acceptable for a prototype, but in production the remaining 10% becomes a support nightmare unless you can detect, diagnose, and remediate failures quickly.
Price/performance optimization pushes teams to ask questions like:
What portion of the workload truly requires the most capable (and most expensive) model?
Can some steps be handled by smaller or cheaper models without degrading user outcomes?
How do we avoid paying “full price” for every step when many steps could be lightweight?
Where does latency come from—model inference, tool calls, retrieval, or orchestration overhead?
How do we measure performance in a way that reflects real user success, not just token-level metrics?
Once you start asking these questions, the coupling between models and agents becomes a liability. If the agent is built around a specific model’s behavior, swapping models becomes risky. If the agent logic assumes a certain style of output, a different model might break tool selection or degrade reliability. Conversely, if the model is embedded deeply into the agent’s decision-making loop, you may find yourself unable to optimize cost without rewriting orchestration logic.
That’s where the “split models from agents” idea becomes more than a buzzword. It’s a strategy for isolating change. If the model layer can be swapped, scaled, or tuned independently, teams can pursue better price/performance without destabilizing the entire system.
What “splitting” really means in production terms
It’s easy to imagine splitting as a clean separation: one service for models, another for agents. But in real systems, the separation can be more nuanced. The key is not necessarily physical deployment boundaries; it’s conceptual boundaries and operational independence.
A practical split often looks like this:
1) The agent defines the workflow and contracts.
The agent decides which tools to call, what information it needs, and how to interpret results. It operates on structured inputs and outputs wherever possible. Instead of relying on free-form text generation as the only interface, the agent uses schemas, function calls, and explicit state transitions.
2) The model becomes a replaceable component behind a stable interface.
The model is responsible for generating content or producing intermediate signals, but the agent treats it as a capability provider. The agent expects certain types of outputs (for example, JSON fields, tool-call intents, or classification labels), not a particular narrative style.
3) The system can route requests to different models based on context.
Once the model layer is abstracted, routing becomes feasible. A high-stakes step might use a stronger model; a low-risk step might use a cheaper one. The agent can request “capability level X” rather than “use model Y.”
4) Observability and cost attribution align with the architecture.
If models and agents are separated, it becomes easier to attribute cost and latency to the correct layer. You can see whether failures come from orchestration logic, tool execution, retrieval quality, or model generation.
This is the kind of separation that supports iterative improvement. Teams can refine agent policies without retraining or re-tuning the model layer. They can upgrade models without rewriting the agent’s entire workflow. And they can experiment with routing strategies to improve price/performance.
Why bundling is tempting—and why it breaks down
Bundling models and agents together is attractive because it reduces initial complexity. Early agent frameworks often encourage a tight loop: the agent prompts the model, the model responds with both reasoning and actions, and the agent executes those actions. This approach can be fast to build and easy to understand.
But bundling creates hidden dependencies. The agent’s behavior becomes entangled with the model’s quirks: how it interprets instructions, how it formats tool calls, how it handles ambiguity, and how it recovers from partial failures. When you later try to optimize cost by switching to a smaller model, you discover that the agent’s assumptions no longer hold. Suddenly, the “agent” isn’t really independent—it’s a wrapper around a specific model’s behavior.
In production, that coupling becomes expensive. Every model change triggers a cascade of regression testing. Every attempt to reduce cost risks breaking reliability. And because the system is treated as one unit, it’s harder to isolate where performance problems originate.
Splitting models from agents is essentially a way to prevent the system from becoming brittle. It’s a design choice that acknowledges a reality: models will change frequently, while agent workflows and product requirements evolve more slowly. If your architecture can’t tolerate model churn, you’ll pay for it in engineering time and operational risk.
The “next phase” of AI product building
Rauch’s framing suggests that teams are already moving into this next phase. The early phase was about proving that AI could be useful. The next phase is about making AI dependable and economically viable.
That shift is visible across the industry in several ways:
Developers are building more structured agent workflows rather than relying on purely generative loops.
Teams are investing in evaluation harnesses that measure task success, not just output quality.
Observability is becoming a first-class requirement: tracing tool calls, capturing intermediate states, and logging model inputs/outputs in a privacy-aware way.
Cost controls are being integrated into the product lifecycle, not bolted on afterward.
In that context, the model/agent split becomes a natural evolution. It aligns with how production systems are built: separate concerns, define interfaces, and allow components to evolve independently.
A unique angle: price/performance isn’t just about cheaper models
One reason this topic is getting attention is that “price/performance” can be misunderstood. People sometimes interpret it as “use smaller models.” That’s part of it, but not the whole story.
Price/performance in production is also about reducing wasted work. Many agent systems spend tokens and time on steps that don’t materially affect the final outcome. For example, an agent might generate long explanations when it only needs a short tool-call intent. Or it might repeatedly re-check information that could be cached. Or it might run expensive reasoning even when a simpler heuristic would suffice.
When models are separated from agents, it becomes easier to implement these optimizations systematically. The agent can decide when to ask for deep reasoning and when to rely on cheaper signals. The model layer can expose multiple “modes” or “capability tiers.” The system can route requests based on confidence thresholds, task complexity, or user context.
In other words, splitting enables dynamic allocation of compute. It turns cost optimization into a controllable policy rather than a static configuration.
This is also where reliability improves. If the agent can choose different model tiers based on uncertainty, it can reduce failure rates without always paying the highest cost. And if the agent can fall back to alternative strategies when a model response is malformed or unhelpful, the system becomes more resilient.
Deployment and developer tooling: why Vercel’s perspective matters
Vercel’s involvement in this conversation is notable because it sits at the intersection of AI and deployment infrastructure. Production AI isn’t just about model selection; it’s about shipping software reliably. That includes handling traffic spikes, managing cold starts, integrating with edge runtimes, and ensuring that the AI features behave consistently across environments.
From a platform perspective, splitting models from agents can also simplify deployment. If the agent orchestration logic is deployed as part of an application, while the model inference is handled through a separate layer (whether internal services or external APIs), teams can scale and update each part independently. That can reduce downtime during upgrades and make rollbacks safer.
It also helps with compliance and governance. Some organizations want tighter control over model access, data handling, or logging. If the model layer is centralized behind a stable interface, it becomes easier to enforce policies. Meanwhile, the agent layer can remain focused on workflow logic and user-facing behavior.
None of this is to say that splitting is always