Why most CEOs' AI efforts are just 'fake progress': The seven decisions that determine real enterprise AI transformation

The core problem: why most CEO AI efforts are "fake progress"

Executives aren't necessarily lying. "Fake progress" means an organization picks the metrics that make it look busiest and least likely to disturb the status quo. Pilots, proofs of concept, and productivity plugins all have their uses. The problem starts when these activities become the goal itself—when companies mistake increasingly frenetic activity for a change in underlying capability.

There are five typical symptoms.

1. Confusing the number of pilots with the depth of transformation

Customer support builds a draft-reply assistant. Finance builds a Q&A bot for reports. Legal builds a contract summarizer. At the end of the year, the company tallies up dozens of AI use cases and the presentation deck looks great.

But remove those projects and the core workflows barely change: support still copies information between five systems, finance still reconciles numbers manually at month-end, and legal still relies on senior staff to judge exceptions. The more projects, the thinner the scarce engineering talent, data capabilities, and management attention get spread. Bain's assessment is blunt: this activity looks like a transformation portfolio but is really just manufacturing the appearance of progress.

2. Adding an AI button to an old process and calling it a business redesign

A lot of AI transformation is really just speeding up one step in an inefficient workflow. An email that took an employee 20 minutes to write now takes the model 2 minutes. But why the email exists in the first place, why it needs to be handed off four times, and who actually has the authority to decide—none of that gets touched.

The result is that a piece of the process gets faster without the end-to-end flow getting any faster. Upstream data is still messy, downstream approvals are still bottlenecks, and employees now have to spend time double-checking the model's output. The company has automated the most visible segment of the old process without redesigning how the work should actually be done.

3. Using short-term ROI as the only filter

Bain's CEO survey contains another "roughly 85%": that share of CEOs is primarily using AI for near-term cost cutting and efficiency gains, hoping to fund later transformation efforts. That's a different metric from the "85% executing poorly" figure—the former 85% is an investment preference, the latter a performance judgment.

Funding early wins to pay for later work isn't wrong. The fatal flaw is that when every project must prove ROI within the same year, the organization systematically gravitates toward tasks that are easiest to quantify, closest to the old way of doing things, and least likely to shift competitive position. Cross-functional overhauls of customer journeys, supply chains, or product development get killed first by the budget process precisely because they take longer and involve more complex accountability.

4. Mistaking vendor capabilities for your own

Giving everyone access to an AI-enabled SaaS suite doesn't mean the company now has AI capability. A competitor can buy the same functionality tomorrow. Worse, if data semantics, workflow logic, tool integrations, and usage feedback all live inside the vendor's platform, every use might be improving their product while leaving you with little to build on for the next deployment.

That means companies need to keep revisiting their software subscriptions. AI is now replacing a lot of the interface operations, process handoffs, and information shuffling that software used to handle. Layering an AI plugin onto every legacy system might just mean paying an extra subscription fee while the walls between those systems stay exactly where they were.

5. Ambitious strategy rhetoric, deeply conservative operating mechanisms

The most common absurdity: the CEO declares AI the company's top priority for the next three years, yet projects still have to justify their ROI individually each year. The team says it's building proprietary capability, yet key engineering is entirely outsourced. Management demands rapid iteration, yet every release has to go through months of serial approvals.

This reveals the company never really made the decision—the problem goes far beyond execution details. Bain points to a deeper root cause: over the past two decades, many large companies got used to outsourcing development and buying software to meet every need, so internal engineering capability atrophied. You can buy models and platforms, but who gets access to which data, which tools can be called, and which business rules apply are still your design decisions. Vendors can't deliver that knowledge of how work gets done.

Put all this reporting evidence together and you still have to answer two questions that can't substitute for each other: did business outcomes actually change, and did this deployment leave reusable capability for the next one? The framework below lets you test a company's existing evidence against both.

All these symptoms point to the same problem: companies treat "doing more projects" as "having stronger capability." When a project ends, failures don't become new test cases, data still speaks in different dialects, veteran judgment stays trapped in people's heads, and other teams don't learn what pitfalls were hit. The next deployment starts from scratch all over again. The more projects you roll out, the more duplicated effort you roll out along with them.

Build the "proprietary intelligence" competitors can't buy

Commodity foundation model capabilities will keep spreading, and prices and access barriers will keep falling. GPT, Claude, and Gemini will still differ in quality, speed, price, context window, compliance, and deployment options. But no company can sustain a lead simply by "using an advanced model," because competitors can get similar capability through the same market.

The real gap shows up in the specifics: when you call the same model, does your system know what just happened with the customer, how a veteran worker would typically judge this situation, which tools can be invoked, which exceptions have to go to a human, and how the last failure should change the next execution?

Together, those three capabilities—data that understands your business, methods written into the system, and a learning mechanism that corrects course using outcomes—constitute proprietary intelligence.

Pillar one: proprietary data so agents actually know your customers and business

Proprietary data isn't every historical record piling up in the warehouse. What matters is customer records, operational processes, human judgment calls, and final outcomes—things competitors can't access and that keep reflecting how the business is changing.

An insurer's edge, for example, lives in the long-run correlations between customer characteristics, claim materials, investigation paths, human judgment, and final payout outcomes. Policy counts are just the surface. For a retailer, raw sales data isn't enough either; what the model needs to understand is how price, display, promotion, weather, inventory, returns, and regional differences combine to produce results. A general-purpose model knows the generic rules of insurance and retail, but it doesn't know where this specific company has been burned by exceptions in the past.

Having unique data isn't sufficient. The same word "revenue" can mean three different things in the customer system, the finance system, and the operations system. A customer's latest status might be scattered across five places. Give an agent those fragments and it will just produce plausible-sounding answers faster. Companies need to align terminology and definitions across priority business areas first, then check whether data is timely, where it came from, and who's allowed to use it. You don't have to govern all company data upfront—just get the data straight for the business processes you're actually going to redo.

Pillar two: encoded workflows that turn veteran judgment into company capability

The scarcest knowledge in most companies lives in the judgment of senior employees. Process manuals rarely spell out: when a case can be approved outright, which combination of signals means it has to be escalated, when a customer's phrasing actually signals a completely different need, or when following the standard procedure by the book would make things worse.

Encoding a workflow means converting that judgment into steps, tools, and exception rules an agent can execute. Take corporate credit underwriting. The shallow version is just having a model prepare application materials for a human. The deeper version lets the system read internal risk records, verify evidence, flag contradictions against the company's own policies, and—when it's uncertain—hand the case to a human along with its reasoning and the relevant documents. It shouldn't just spit out a score it can't explain.

Copying the old process onto AI doesn't work either. Many steps in legacy workflows exist only to ferry information between disconnected systems or to preserve accountability structures from the past. Companies should ask: if we were building this from scratch today, would these handoffs and approvals still be necessary? Cut the redundant steps first, then decide where human judgment genuinely has to stay in the loop.

At minimum, the company needs to control who can do what. Every agent and tool should have an inventory entry, version history, tests, and an audit log. Check permissions before any tool is called. And when something goes wrong, you need to be able to trace which agent looked at what data and took what action. Handing all that control to a single vendor is outsourcing the way your company thinks.

Related readingMicrosoft CEO Satya Nadella: Companies Face a ‘Reverse Information Paradox,’ Paying for AI While Handing Over Their Internal Knowledge and ExperienceCompanies pay for intelligence twice: once in model fees, and again in the proprietary know-how required to make the model genuinely useful.→

Pillar three: a learning architecture where successes and failures both leave something behind

Most AI projects are designed to "deliver a result," not to "teach the organization more." After launch, the model gets something wrong and employees note it in a chat thread. A customer complains and the team holds a retrospective. The bug gets fixed, but the lesson stays in the ticket and in people's heads. Six months later, another team hits the same trap.

The learning mechanism has to solve two problems. First, when an agent gets something wrong, how does it avoid repeating the mistake? The system needs to record what it did, where it failed, and turn the real failure into a new test case. Second, how do other teams avoid taking the same detour? Vetted definitions, tools, and lessons should flow into a shared knowledge area so that the tenth agent can build on what the first nine already validated.

Three types of information need to stay distinct. Business data answers "what's true," current information answers "what's happening right now," and historical lessons answer "what's worked before." Mix them together and the system might treat an outdated lesson as a current fact—or start from scratch every single time.

Bain cites Shopify and Madrigal to illustrate the mechanism. According to the report, Shopify's system ran roughly 400 experiments on a process already considered highly optimized, and only 1 produced a meaningful improvement. The point isn't "400 tries for one win"—it's that a machine can search for low-probability gains at a scale a human team couldn't sustain. The Madrigal LangChain customer case describes a different loop: production failures automatically feed into the evaluation set, multiple agents pool evidence and memory in a shared workspace, and new use-case development goes from weeks down to hours.

These figures and examples come mainly from Bain, its clients, and vendors. They're not controlled experiments, so they don't prove every company will get the same results. They just demonstrate one thing: an agent doesn't automatically get better after launch. Failures need to become tests, lessons need to be vetted, and other teams need to be able to reuse them—only then does the system get progressively less error-prone.

Why does the gap widen the longer you use it?

Models don't get smarter just from being used. The advantage comes from a specific chain: business operations leave better records, agents therefore understand the field better, employees get faster and more accurate with agent help, employees then write their effective practices back into the workflow, and the improved workflow produces higher-quality records.

Each time this loop runs, the company accumulates reusable assets: more consistent data definitions, more reliable system integrations, more mature operating procedures, a fuller test suite, and employees with sharper judgment. A latecomer can buy the same models but can't buy back in a single procurement cycle the experience that was earned through real business operations.

The compounding can also run in the wrong direction. Dirty data compounds errors, low adoption cuts off feedback, wrong memories poison other agents, and market changes can invalidate the best practices of last year. The moat's quality depends on the quality of feedback signals, how fast corrections happen, the discipline of retiring outdated knowledge, and the ability to reuse across teams.

The 7 decisions that get enterprise AI transformation right

The seven decisions follow a sequence. First, decide how much the company is willing to invest and which business areas to prioritize. Then, hold onto the data and control. Finally, change how teams work, so mistakes leave lessons behind and both the money and the risk have an accountable owner.

Decision 1: Is the company truly committing, or just experimenting?

The CEO and the board first have to decide whether AI projects are just for exploration or are meant to change how the company competes over the next few years. If it's the latter, multi-year budgets, talent arrangements, approval speed, and delivery methods all have to change. Adding another committee accomplishes nothing.

The simplest test is whether the operating mechanisms match the rhetoric. Claiming to lead while requiring projects to pay back within the year. Claiming to control core capabilities while relying almost entirely on outsourced engineers. Claiming rapid learning while every adjustment goes through months of approvals. When these contradictions are present, the company has a strategic statement, not a strategic commitment.

Bain cites Bradesco as a case in point. The bank initially built its payments experience around a small number of large agents, but employee testing quickly revealed the approach was too slow, too expensive, and could not scale safely. Rather than dressing up the demo as a success, the team spent roughly five months reworking the architecture. According to the report, the resulting capabilities later made their way into multiple core banking scenarios serving 22 million customers. This case does not prove that a five-month rebuild guarantees scale, but it does illustrate a key point: when a company acknowledges a failed first design and rebuilds, it retains the engineering capability and organizational trust needed for the next round.

Decision 2: Pick a few critical business areas and see them through end to end

Spreading budget evenly across dozens of use cases looks fair, but it actually spreads failure evenly as well. The business areas that get prioritized usually meet three criteria: they directly affect competitive advantage, the company holds proprietary data, and the full workflow genuinely has room for redesign.

Bain's overview recommends focusing on 3–5 priorities, while the main text of Decision 2 narrows that to 2–3 domains. The report's own framing is not entirely consistent. Companies do not need to mechanically choose between the two numbers. The point is to concentrate most resources, the strongest talent, and the CEO's attention on a small number of business areas that can move revenue, costs, or customer experience—and to keep at it until those workflows are fully transformed.

A bank, for example, should not build a chatbot for dozens of departments simultaneously. Instead, it could focus on rebuilding fraud handling or corporate credit approval as a whole: from signal ingestion, evidence verification, risk assessment, and human escalation to outcome feedback. Only at that scale can a company accumulate data, workflows, and learning assets at the same time.

Decision 3: Get the data that decides outcomes under control first

Data initiatives are prone to a different kind of false progress: building a grand "unified data platform" and then, two years later, still being unable to say which business outcome it changed. The correct sequencing is to work backward from priority areas. Whatever business concepts agents need to understand, real-time signals they must read, data products they should call, and outcome feedback they need to see—make those pieces work first.

Agents act continuously on real-time data, so data governance has to operate in real time as well. Monthly quality checks cannot govern agents that make decisions every minute. Quality, lineage, and access control need to be built into the data pipeline itself; otherwise governance becomes a retrospective explanation of risk rather than a constraint on the system before risk materializes.

Decision 4: Models can be bought, but business control cannot be outsourced

Companies do not necessarily need to train their own foundation models, but they must control how agents connect to data and tools. At minimum, four things are required: agents and tools not on an approved list cannot run; every change is versioned and tested; permission checks and audit trails happen before tool calls; and quality gates must be passed before anything moves into production.

There are two distinct categories of risk here, and they cannot be tested in the same way. The first is reliability: can the work actually get done? When models or data change, agent behavior can drift; and when multiple agents are chained together, a single error can amplify as it moves downstream. The second is security: what happens under malicious input? Can the agent be induced to leak data, misuse tools, or exceed its permissions? Passing functional tests only proves the agent works under normal conditions—it says nothing about whether it remains safe when attacked.

The integration and orchestration code written today may need to be rewritten in a few months, but it is still worth writing. Code can be discarded; what remains is a unified data vocabulary, a tool registry, a permission gateway, engineering capability, and project experience. Bain reports that Ramp achieved 99% adoption of its company-wide AI tools, and points to its internal platform Glass as an example of how one person's effective approach became a shared company capability. That figure is a case-study data point from the report and should not be extrapolated as a general adoption benchmark. The transferable lesson is that Ramp did not lock organizational knowledge inside individual tools.

Companies should also continuously reassess their software subscriptions: which vendors remain irreplaceable, which capabilities can now be handled by internal agents and shared tooling, and which platforms are starting to form new bundles. This cannot wait for a three-year renewal negotiation.

Decision 5: Rework the workflow and the workforce together

Launching a new workflow first and training employees afterward is nearly a guarantee that old roles, old incentives, and old habits will re-colonize the new system. Workflow redesign and workforce transformation have to happen in parallel: short delivery cycles, continuous feedback loops, a fixed release cadence, and most decision-making authority delegated to small teams that know the business.

Bain cites self-reported figures from some organizations of 5–10 times software development output with the same headcount. These are not audited metrics under a common standard, and they cannot be converted into a claim of 5–10 times business value. The range does suggest that AI-native software development may be among the first areas to demonstrate new ways of operating. But speed is only an asset when quality, feedback, and reuse improve alongside it; otherwise it just generates technical debt faster.

The end state is not one where only a few technical teams use AI. Bain describes a longer-term goal in which digital literacy and low-code prototyping skills become as common as financial literacy: business users can turn an idea into a working demo in hours and clearly explain where the value comes from. Frontline workers need to be tinkerers who feed edge-case discoveries back to domain teams, and executives must use the tools themselves to develop operational intuition that no briefing document can replace.

But this vision should not be described as present reality. Current tools are still immature: agent registry management, cross-system permissions, discovery of shared methods, and reliable reuse all still require substantial manual effort. The more automated the system becomes, the more it depends on people asking good questions, setting proper constraints, and judging which improvements are worth keeping.

Decision 6: Every mistake should produce at least one new test

Every agent should record both business outcomes and intermediate behavior from day one: what it did, why it did it, whether the result was effective, where failures occurred, and which inputs caused anomalies. Production failures need to feed back into evaluation—they should not simply be patched as one-off bugs.

Companies also need a shared lessons-learned repository with clear rules about what can be added, who curates it, who can read it, and when obsolete content gets removed. Fixing only a single agent keeps lessons trapped inside one team; building a repository without verifying outcomes lets errors propagate faster instead. Only by doing both can the tenth agent benefit from lessons already validated by the first nine.

Decision 7: One mechanism for today, one for tomorrow

Existing governance—responsible for reliable delivery, risk control, and keeping commitments—cannot be eliminated. But quarterly metrics, serial approvals, and fixed roadmaps are designed for stable businesses. They are inherently ill-suited to managing an AI transformation that is still learning quickly.

Companies also need a separate mechanism dedicated to change, one that persistently asks three questions: What did this round teach us? Where do we adjust next? Should budget and talent be reallocated? Day-to-day operations keep today's business running; the transformation mechanism ensures the company has not stopped learning. The two run side by side, each addressing a different set of problems.

Token costs also need to move out of a vague line item in the IT budget and become operational data at the workflow level. When consumption is tracked by domain, workflow, and business outcome, companies can see where to invest more, where to optimize, and where to rebuild entirely. The budget and headcount freed up by agents must be explicitly reinvested in the next round of capability building—not quietly absorbed into existing budgets. Ultimately, accountability for risk must sit with a clearly designated senior leader; it cannot be pushed onto the compliance department or blamed on an opaque system after something goes wrong.

The later you start, the harder experience is to catch up

A company does not fall permanently behind simply because it did not bet on the right model today. Models will keep getting stronger, cheaper, and more replaceable. What is hard to recover is the business experience competitors have already accumulated: where they failed, how they corrected course, and which approaches have been repeatedly validated.

Every time a leading company completes a cycle of effective learning, it adds another layer of unified data standards, operating procedures, test cases, historical experience, and employee judgment. A latecomer can buy the same models, but it cannot buy back the failures, corrections, and reuse that competitors have already lived through. The value of time comes from starting effective learning early—not from being first to deploy.

This remains a strategic judgment, not a law that has been rigorously proven causally. Dirty data, low adoption, faulty memory, market shifts, and competitor reactions can all erode the compounding effect. CEO and board commitment is a necessary condition, not a sufficient one. Whether "proprietary intelligence" ultimately becomes a moat or just the next management slogan depends on whether middle management, engineering, frontline business teams, and governance can jointly operate that learning loop.

So stop asking "how many AI projects did we launch this year?" Instead, ask what each deployment actually leaves behind for the next one. The four questions below do not produce a score. They distill the seven decisions above into the evidence a company genuinely needs to show when a deployment concludes.

If the four questions cannot be answered with verifiable evidence, the company still only has AI activity. When the four questions get consistent affirmative answers, AI begins its shift from a public technology into the company's own proprietary intelligence. And any gap points directly to the decision that needs another round of work—not to another batch of small pilots that accumulate nothing.