In the rush to quantify what artificial intelligence can do for work, many conversations have quietly converged on a deceptively simple question: how much faster can you get the job done?
It’s an appealing metric because it feels concrete. A draft that used to take a day now takes three hours. A report that required a full afternoon of analysis can be assembled in minutes. A customer support agent can respond with less effort and fewer back-and-forth messages. In boardrooms and team stand-ups alike, “time saved” becomes the shorthand for value.
But a growing body of commentary—and a fresh piece of coverage challenging the prevailing approach—argues that this focus on speed, especially when it’s based on self-reported estimates, may be measuring the wrong thing. The problem isn’t that time-to-completion is irrelevant. It’s that it’s often treated as a proxy for productivity without accounting for what productivity actually means in real organizations: outcomes, correctness, cost, iteration, risk, and the ability to deliver consistently under messy conditions.
When AI is introduced into knowledge work, the work doesn’t just get shorter. It changes shape. And if the measurement system doesn’t reflect that change, teams can end up celebrating impressive-looking numbers that don’t translate into durable performance gains.
The hidden weakness: self-reported timelines
Self-reported estimates are convenient. They’re fast to collect, easy to compare, and they align with how people naturally describe their experience. If someone says, “I can do this in half the time now,” that statement can be turned into a productivity multiplier almost instantly.
Yet self-reports are also vulnerable to bias. Optimism is one factor. People want to believe the tools are working, particularly early in adoption when enthusiasm is high and skepticism is low. Another factor is perception: individuals may remember the “happy path” where the model behaves well, rather than the full workflow that includes prompt rewriting, fact-checking, rework, and escalation when the output is wrong or incomplete.
There’s also a more structural issue. Many tasks that AI helps with are not linear. They involve cycles: generate, review, correct, refine, and sometimes restart. A self-reported estimate might capture the time spent producing a first draft, but not the time spent ensuring the final deliverable meets standards. In other words, speed can be measured at the wrong stage of the process.
Even when people try to be honest, they may not have a shared definition of “done.” Is a document “done” when the first version is generated? Or when it’s reviewed, edited, approved, and ready to publish? Is a data analysis “done” when the model returns an answer? Or when the team verifies assumptions, checks sources, and confirms the result holds up against edge cases?
If the metric doesn’t match the operational definition of completion, the numbers can look better than reality.
Why “faster” can be misleading
The most common trap is assuming that reduced time automatically implies increased productivity. But productivity is not merely throughput; it’s throughput of useful, correct, and valuable work.
AI can accelerate certain steps while increasing others. For example, a model might generate a plausible summary quickly, but the team may spend additional time verifying accuracy, checking citations, or reconciling contradictions with internal documents. In some environments, the verification burden is manageable. In others—especially regulated industries—it can erase the time savings.
There’s also the question of quality. Faster output that requires heavy rework can still be less productive overall. If a team spends twice as long correcting AI-generated errors, the net gain might be negative even if the initial generation step is dramatically quicker.
Then there’s the “value” dimension. A task completed faster might not be the task that matters most. AI can make it easier to produce more content, more drafts, more variations. But if those outputs don’t improve decisions, reduce customer friction, or strengthen business outcomes, the organization may be optimizing for activity rather than impact.
This is why the article’s core argument resonates: speed is not always the same as value, and self-reported speed is particularly unreliable as a measure of real productivity gains.
The measurement gap: what organizations actually need
To understand why speed metrics can fail, it helps to look at how organizations operate. Most workplaces don’t run on idealized workflows. They run on constraints: limited time, shifting priorities, incomplete information, and the need to coordinate across roles.
In that environment, productivity is often determined by factors that aren’t captured by “time to first draft.” Consider a few examples:
1) Accuracy and trust
If AI outputs require frequent correction, teams may lose trust in the tool. That can slow adoption and increase cognitive load. Even if the model is fast, the human must remain vigilant. Productivity gains depend on whether the tool reduces uncertainty or simply shifts it.
2) Rework and downstream effects
A faster deliverable that triggers downstream revisions can create hidden costs. A marketing team might generate copy quickly, but if compliance review flags issues, the cycle time expands. A finance team might draft analysis quickly, but if the result fails audit checks, the rework can be extensive.
3) Consistency across contexts
AI performance can vary by domain, data availability, and prompt quality. A self-reported estimate might reflect a narrow set of tasks where the model performs well. Real productivity gains require consistent performance across the breadth of work.
4) Iteration cycles
Knowledge work often involves multiple iterations. Measuring only the initial generation time ignores the number of cycles required to reach a stable, acceptable outcome.
5) Opportunity cost
Time saved on one task can be consumed by additional tasks created by the new capacity. This is sometimes called “productivity rebound.” If AI makes it easier to produce more, teams may fill the extra time with more work rather than achieving higher outcomes. The organization might still be busy, but not necessarily more effective.
These factors suggest that productivity measurement should focus on the entire workflow, not just the fastest step.
A more meaningful way to measure AI-driven productivity
If speed is an imperfect metric, what should replace it? The answer isn’t to abandon measurement. It’s to measure the right things, in the right way, with enough context to reflect how work actually gets done.
Here are several metrics that often provide a more accurate picture of AI-driven productivity:
Outcome-based metrics
Instead of asking how quickly a task can be completed, ask whether the output achieves its intended purpose. Examples include:
– Conversion rates for marketing content
– Resolution rates and customer satisfaction for support responses
– Error rates and defect counts for technical documentation
– Approval rates for internal reviews
– Time-to-decision for business processes
Outcome metrics are harder to collect, but they align with value.
Quality and correctness metrics
Quality can be measured directly through:
– Human evaluation scores (with clear rubrics)
– Fact-check pass rates
– Citation accuracy (where applicable)
– Compliance adherence
– Regression testing results for technical outputs
If AI reduces time but increases error rates, the net productivity gain may be illusory.
Cycle time to “accepted” work
A practical improvement over self-reported speed is to measure time from request to acceptance. Acceptance should be defined operationally:
– Submitted to the reviewer
– Approved by the relevant stakeholder
– Published or deployed
– Integrated into a workflow
This captures rework and review time, which are often the real bottlenecks.
Cost per accepted deliverable
Productivity is also about efficiency. Organizations can measure:
– Labor hours per accepted output
– Total cost per deliverable (including tooling and compute)
– Cost of rework (hours spent correcting AI outputs)
This helps distinguish between “fast but expensive” and “fast and efficient.”
Iteration count and revision depth
Instead of only measuring time, track how many rounds it takes to reach acceptance. For example:
– Number of prompt revisions
– Number of editing passes
– Number of reviewer comments resolved before approval
A tool that produces a first draft quickly but requires many revisions may not be improving productivity in the way teams assume.
Consistency metrics
Measure performance variance across tasks:
– How often the model meets quality thresholds without manual intervention
– How performance changes across different topics or document types
– How frequently the workflow requires escalation to a human expert
Consistency is a major determinant of whether AI becomes a reliable productivity engine or remains a novelty.
Human effort distribution
AI can change where human effort is spent. Teams can measure:
– Time spent prompting vs. time spent reviewing
– Time spent correcting vs. time spent creating
– Cognitive load proxies (for example, number of interruptions or escalations)
This reveals whether AI is truly reducing effort or merely shifting it.
The role of experimental design: moving beyond anecdotes
One reason self-reported estimates persist is that many AI deployments are evaluated informally. Teams test the tool on a few tasks, observe improvements, and generalize.
A more rigorous approach would treat productivity measurement like an experiment:
– Define the task set and acceptance criteria
– Establish baseline performance without AI
– Run controlled trials with AI enabled
– Track the full workflow metrics (not just first-draft time)
– Compare results across different task types
This doesn’t require heavy bureaucracy. Even lightweight experiments can reduce bias and clarify what’s actually improving.
For example, a team could select a representative sample of past work items, then measure:
– Time to first draft
– Time to acceptance
– Revision count
– Quality score
– Rework rate
– Outcome impact (where feasible)
If AI improves only the first two metrics but worsens the rest, the conclusion should be nuanced. If AI improves acceptance time and quality without increasing rework, the case becomes stronger.
A unique take: productivity is a system, not a speed contest
The deeper insight behind the critique is that productivity is systemic. AI doesn’t simply compress time; it alters the workflow. That means the measurement framework must evolve too.
Think of productivity as a chain of constraints. Speed is only one link. Other links include:
– Information