Why enterprise AI stalls at the demo: Prove one workflow before scaling
Pick a workflow tied to a customer pain point, define permissions, accountability, baselines, and evaluation before building, and use real data for evidence before expanding. What a pilot leaves behind—context, metrics, and operating know-how—makes the next workflow easier to start.
- Enterprise AI gets stuck between demo and production not because models can't answer, but because no one picked a workflow with clear boundaries, agreed on evidence in advance, or tied pilot results to a concrete decision.
- The first workflow should come from where customers wait, complain, or churn. Candidates that lack real users, real data, a signable baseline, or a clear owner should be dropped before development begins.
- Set the baseline, golden set, and failure criteria before any agent is built. Then run the new approach in parallel with the old one for two to four weeks, measuring throughput, quality, and economics before deciding to continue, adjust, stop, or expand.
- Permissions and accountability must be settled before development. Only after evidence supports expansion should you harden production deployment and handoff. The context, evaluations, and operational lessons left behind mean the next workflow doesn't start from zero.
The most common illusion companies have about AI is confusing "can be demonstrated" with "is worth deploying." Over six weeks, Mark Ajzenstadt spoke with 20 operating partners at private equity firms and found that while most portfolio companies already use AI to draft emails and summarize meeting notes, few can articulate how to move it into workflows that actually touch customers and business outcomes.
This gap operates on two timescales. A polished demo can take a week to build, but proving production quality can take six months. The problem usually isn't whether the model can answer; it's that companies skip three foundational steps: they don't first choose a process with clear boundaries, fail to agree on what counts as success before starting development, and never set up pilot results to feed into an explicit go/no-go decision.
Ajzenstadt is the founder of Limestone, an AI delivery firm. His "AI transformation loop" starts with a customer pain point: pick a process with clean boundaries, agree on baseline metrics and evaluation methods before development, validate a minimal slice against real data, and only then decide whether to scale up investment. Data access and ownership must be defined from day one. Only after the evidence supports expansion should the team refine production deployment and operational handover. Each time the loop runs, the accumulated context, evaluation suite, infrastructure, and business knowledge lower the cost of the next initiative.
Step one: find a viable process where customers are waiting
Asking business teams "what AI use cases do you have?" rarely yields good answers. Operators spend their days serving customers, maintaining relationships, and handling exceptions. Customers, meanwhile, don't proactively ask for AI next quarter. Asking frontline staff to pause their work and invent an AI roadmap from thin air is the wrong starting point.
A more reliable approach is to work backwards from the customer: identify where customers wait, complain, or churn, then examine the work, data, evidence, and ownership behind that experience. The original article breaks this examination into five lenses.

- Customer interaction: Look for places that change customer outcomes. This could mean offering insights they didn't have before, compressing multi-day waits into minutes, or improving the product between releases. Ajzenstadt specifically notes that onboarding moments—account opening, document review, new-customer activation—are often undervalued; when customers can't use a service quickly, that's a signal to investigate further.
- Underlying work: Break steps into categories: deterministic, judgment-intensive, exception-driven, and relationship-dependent. Machines should handle deterministic steps and prepare materials for judgment calls; humans remain responsible for decisions, approvals, and exceptions. If most of the work depends on complex judgment or client relationships, save it for a later pilot and choose processes where acceptance criteria are easier to define up front.
- Context: Verify where facts live, who owns them, what fields actually mean, and what sensitivity levels apply. If no one internally can explain what the data means, defining the data is the first job.
- Evidence: Check whether the company can currently measure work units, volume, cycle time, hands-on time, rework, exception rates, and unit costs. Without a current baseline, there's no way to prove what changed after the pilot.
- Ownership: Pre-assign a process owner, data owner, and executive sponsor. If no one will take over running the process after the pilot, and no one is willing to make a go/no-go decision based on evidence, the process is not viable.
Companies should identify 3–5 candidate processes, rank them by value, feasibility, risk, and reuse potential, and then choose just one to start. Anything without real users, real data, a signable baseline, or clear ownership should be eliminated before development begins. Goals like "redesign the entire organization" that can't be validated in a short cycle also fail the test.
Once selected, walk through the process with the people who actually do the work. Collect representative cases, exceptions, permissions, and undocumented expert judgment. Interview first, then inventory technology; before reading code, ask who owns each dataset and what each field represents. Ajzenstadt also requires writing down a single business outcome and explicit non-goals, because teams easily expand a small pilot into a full system by week two.
Step two: agree on baseline and evaluation criteria before building the agent
The critical part of Ajzenstadt's ten-step method isn't the number ten; it's the ordering. If you pick metrics after seeing model outputs, you'll naturally favor whatever the model does well. If you build the baseline only after the pilot ends, you can't tell where changes came from. Baseline metrics, a golden set, and critical failure categories must exist before the agent does.
Both sides must agree on current workload, cycle times, hands-on processing time, quality and rework rates, exception rates, operating costs, and the measurement window. How things are done today, what performance level is being achieved, and how improvement will be calculated all need to be settled before the trial run.
The "golden set" here means a batch of real tasks whose results have been confirmed by the customer's own experts. Evaluation must also cover the failures the business cannot tolerate—wrong numbers, protected data leaks, or confident behavior when data is missing. Only when both successes and critical failures are baked into the criteria do trial results qualify for decision-making.
This preparation requires involvement from people inside the company. Under Limestone's "proof month" structure, executive sponsors dedicate 30 minutes a week and make decisions at checkpoints, process owners spend 2–4 hours weekly on walkthroughs and acceptance reviews, and data owners commit 1–3 hours at the start. A vendor can build the system, but business goals, field definitions, and correct answers must be confirmed by the company itself.
This also defines the budget boundary. Limestone reports its delivery mix as one embedded developer with an agent plus a part-time architect at US$15,000–20,000 per month; infrastructure for one operational agent project handling four groups of customer data runs about US$250 per month. These are different figures: the infrastructure bill doesn't include development, acceptance, or organizational handover costs. Their policy of no up-front diagnostic fees, monthly billing, and no payment if the first month doesn't satisfy is that vendor's commercial commitment—it can't substitute for the customer's own acceptance criteria.
Ajzenstadt adds one more discipline to the proof month: every activity must yield a decision, a tested hypothesis, a working increment, or a reusable asset. Anything that produces none of those isn't worth the team's time. This keeps walkthroughs, meetings, and development focused on actual results.
Step three: run a 2–4 week parallel trial and let evidence make the call
A pilot shouldn't start by making final decisions for people, nor should it write directly to systems of record. A safer minimal slice looks like this: the agent recommends, a named person decides, real data is used, the process runs in parallel with the old way, and every execution leaves a trace others can inspect.
These 2–4 weeks must answer three distinct questions at once: is it faster, is it better, and is it worth it once all costs are counted?
| Metric | What to actually look at | What it can't be reduced to |
|---|---|---|
| Throughput | Work completed per cycle, cycle time, hands-on time | "Running more" alone doesn't prove better delivery |
| Quality | First-pass rate, rework, human coverage, incidents, critical failure categories | Average accuracy can't hide tail risks like data leaks or false confidence when data is missing |
| Economics | Unit operating cost, plus verified value of capacity, rework avoided, incremental output, and risk-adjusted losses prevented | Hours saved don't automatically convert to cash; subtract cloud, model, support, and organizational change costs, and report as a range |
The article cites Faros AI's telemetry on 22,000 developers from 2026, noting that while task throughput rose 33.7%, defects per developer increased 54% and incidents per pull request rose 242.7%. Those numbers don't predict any single company's outcome, but they make it clear that speed and quality must be measured separately.
The pilot has four exits: continue, adjust, stop, or scale. If results don't clear the baseline agreed up front, stopping at four weeks preserves the budget for later. Adjustments should modify the implementation or the process approach, then be verified against pre-agreed criteria; lowering the bar when results disappoint isn't acceptable. If the scope of the task itself changes, the verification conditions need to be redefined—old and new results can't be treated as one comparison.
The figure places the trial run and all three measurement categories in the same phase: evidence accumulates continuously during execution, and the business and delivery teams together decide where the next investment goes.
How pilots guide the next step
Evidence before investment
Define permissions and owners at the start. A pilot has four exits.
-
Choose one workflow
Trace customer pain points to real users, data, owners, and explicit non-goals.
Output: a bounded workflow and one business outcome -
Set baselines and tests
Agree on current performance and the measurement window. Customer experts confirm the golden set and critical failure criteria.
Output: acceptance criteria set before development -
Run and measure together
Run alongside the old process for 2–4 weeks. The agent recommends; a human decides. No system-of-record writes yet.
Output: throughput, quality, economics, and readable execution logs
Decide together against agreed criteria
- Go on
- Keep validating within scope and gathering evidence.
- Adjust
- Revise the implementation or workflow, then rerun. Poor results do not justify a lower bar.
- Stop
- End the pilot, retain evidence, and stop further investment.
After evidence supports expansion, harden production deployment and the operational handover
An agent that clears the pilot still needs reliable scheduling, observability, integration, failure fallbacks, cost ceilings, and continuous evaluation before it can operate daily. Every change requires re-evaluation, and nothing should depend on one engineer's laptop. The original article's seven-level adoption ladder describes the capabilities required to move from a personal tool to an organizational system.
Moving up the ladder doesn't mean basic permission management can wait. Ajzenstadt is explicit: knowledge structures need role-based access control from day one; data owners, sensitivity levels, and pilot leaders are identified before development starts. The extra hardening comes after a pilot passes—it's about production operations and scale, not about deferring data and permission boundaries.

| Level | Added capability | Typical state |
|---|---|---|
| 1 | Tool adoption | People start AI tools one by one; automation disappears when they take leave or close their laptops |
| 2 | Context layer | Department knowledge managed in domains, queried across functions through defined interfaces, reducing semantic noise |
| 3 | Evaluation, routing, quality gates | Define standards and routing first, then choose models; swapping models doesn't require rebuilding the whole system |
| 4 | Hosted execution off personal machines | Triggered by schedule or conditions without someone logging in daily |
| 5 | Governance | Knowledge structures have role-based permissions; agents have lifecycles; unused or underperforming capabilities get retired |
| 6 | Agent-to-agent handover | One agent's output feeds the next, with quality gates in between |
| 7 | Semi- or fully automated workflows | Agents handle routine tasks at scale; people focus on exceptions and high-risk judgment |
Many companies are still at level 1 yet buying capabilities from level 7. The real bottleneck is usually level 4: moving scripts off personal computers and onto services that trigger reliably, log their activity, handle failures, and have clear ownership. Without stable hosted services, multi-agent handoffs can't become reliable daily operations either.
Governance also needs concrete retirement rules. Ajzenstadt suggests giving every agent a lifespan—enforce lifecycle management once tool count passes 100, retire anything that hasn't run in 90 days, and retire anything that falls below its evaluation threshold. These are management rules he proposes; the point is that someone must own decommissioning stale capabilities so obsolete agents don't keep valid credentials.
Finally, train the operators, hand over a runbook, and embed the capability into existing workflows. People's roles shift from starting tools one by one to accepting results, exercising judgment, and handling exceptions; every expert correction flows back into the golden set. Naming an owner is just the beginning—the operational handover is complete only when the team actually uses, evaluates, and maintains the capability.
Three cases show what the loop can do—and what it can't prove
Ajzenstadt offers three customer cases. They illustrate how the loop operates in practice but they can't by themselves prove the method will produce similar returns at every company: clients aren't named, and the original baselines, full evaluation sets, and third-party review materials haven't been published.
| Scenario | How the loop ran | Author-reported results | What can't be extrapolated |
|---|---|---|---|
| Medical billing denial queries | Built a golden set from customer's real questions and expert answers; customer's product tech lead accepted after six weeks | 59/60 on first deployment, 15/15 on denial cases; 97% accuracy maintained after online database schema migration, key questions traceable to the cent | Agent only suggests, human decides, nothing writes to the system of record; evaluation set not public |
| Logistics platform invoice rebuild | First week had no code; every feature spec'd first, human gate on every merge | In 90 days, two engineers merged 122 PRs, 104 with fewer than 5 review comments, delivered 50% earlier than customer estimate | Software has relatively clear inputs and acceptance gates; evidence depth exceeds most operational work |
| Hardware manufacturer pilot | Four-week parallel trial against another firm roughly ten times Limestone's size | Customer CTO judged Limestone superior, planned 50/50 joint delivery first, then 80/20 across ten other Scrum teams | No standardized scoring published; "performed better" is the customer CTO's assessment relayed by the author |
The article also admits plainly that no company transforms itself in one to three months. Software engineering has the deepest evidence because inputs, tests, and human gates are relatively well defined. Outside software and simple repetitive processes, there are no universally "obvious" use cases—every judgment- or relationship-intensive workflow has to establish its own baseline, golden set, and failure criteria from scratch.
These limitations don't weaken the loop; they explain its purpose. It doesn't promise answers in advance. It gives the company a controlled period to gather enough evidence to decide whether to keep investing.
The real compounding effect: the next process doesn't start from zero
The first process pays off in more than just possible savings. When the loop is run completely, the company keeps four kinds of assets that carry straight into the next pilot:
- Context layer: data definitions, owners, sensitivity levels, and cross-department boundaries are already mapped.
- Evaluation framework: the golden set, failure categories, routing logic, and quality gates can be extended.
- Hosted infrastructure: scheduling, logging, fallbacks, security review, and cost ceilings don't need to be rebuilt from nothing.
- Business ownership: runbooks, approval relationships, and owners are already embedded in daily operations; expert corrections keep flowing back into the golden set.
Models will change. Today's popular agent framework may be obsolete next quarter. But workflow knowledge, evaluation standards, hosting capabilities, and operational responsibility stay with the company. Because those assets persist, the second process starts from where the first one ended.
Find 3–5 candidate processes where customers are waiting, complaining, or churning, and use the five lenses to pick one. Define permissions, responsibility, baselines, and evaluation before development. Run a 2–4 week parallel trial on real data, and let throughput, quality, and economics jointly determine whether to continue, adjust, stop, or scale. Once the evidence supports expansion, refine production operations and handover, and carry the assets from each proven process into the next.
Before the next AI bet, show the evidence
After speaking with 20 private equity operating partners, Limestone founder Mark Ajzenstadt found a common gap: firms use AI for emails but stall on workflows that affect customers.
A demo alone doesn't justify deployment
Without clear process boundaries, preset standards, and go/no-go criteria, a pilot rarely becomes daily operations.
Two steps before building
Define access, data ownership, and accountability from the start.
01 Pick a process
Look inward where customers wait or complain; drop it if there's no real user, data, baseline, or owner.
02 Set acceptance criteria
Agree on today's performance, test window and unacceptable failures. Build a golden set of real tasks whose answers the customer's experts have confirmed.
03 Run a parallel trial for 2–4 weeks
Use real data alongside the old method. AI suggests, humans decide; don't write to the system of record yet.
Throughput
Volume and turnaround time
Faster does not always mean better
Quality
Rework and critical failures
Averages must not hide critical errors
Economics
Value minus all costs
Hours saved are not cash saved
Decide together against the pre-agreed bar
Continue
Keep scope;
gather more evidence
Adjust
Revise the approach;
trial again
Stop
Keep evidence;
end further spending
Adjust the implementation, but don't lower the bar just because results are poor.
Give the next workflow a head start
When scaling, add stable hosting, continuous evaluation, rollback and cost caps, then train operators. Reuse the context, evaluations, infrastructure and business accountability in the next pilot.
Illustrative scenario, not a real client story. Method by Mark Ajzenstadt; no universal benefit is promised.
The demo is over. The customer is still waiting.
The AI demo worked, but the client is still waiting.
Select one workflow: real users, data, an owner, and clearly stated non-goals.
Baseline · Golden set · Failure criteria
First, agree on the baseline and measurement window. The client's experts confirm the golden set and failure criteria. Permissions are clear from the start.
Throughput · Quality · Economics
Run a parallel trial for 2–4 weeks: the agent suggests, the human decides. No writes to the system of record yet.
Decide together, using the original criteria. If the results fall short, adjust the approach and re-test, not the bar.
If the evidence supports scaling: set up stable hosting, continuous evaluation, rollback, cost caps, and operator training. Carry the context, testing, and operational insights into the next workflow.
Members can copy or download the full Markdown, and let their AI subscribe and ask.
Found this useful? Leave your email — every new explainer, the moment it's out. Unsubscribe anytime.
