Anthropic Studied 400,000 Claude Code Sessions: Expertise Matters More Than Coding Skill for AI Coding Success
- Anthropic analyzed roughly 400,000 Claude Code interaction sessions from October 2025 to April 2026, covering about 235,000 users
- In a typical session, users make about 70% of the "what to do" decisions (planning), while Claude makes about 80% of the "how to do it" decisions (execution)
- The higher a user's expertise on a given task (not their coding background), the higher the session success rate: verified success sits at 28%–33% for experts versus just 15% for novices
- Among sessions that produced code, the success rates across the top 10 profession categories differ by no more than 7 percentage points from one another — all close to software engineers
- Over the 7 months, the share of sessions spent fixing bugs dropped from 33% to 19%, while ops, documentation, and data-analysis sessions doubled, and the average estimated task value rose about 25%–27%
Across 400,000 Conversations, Who's Really Calling the Shots — Human or AI?
Anthropic recently published a research report analyzing roughly 400,000 Claude Code interaction sessions from October 2025 to April 2026, trying to answer one question: can people with no coding background direct an AI agent to complete complex technical work?
This is by far the largest public analysis of real-world Claude Code usage to date — roughly 400,000 interaction sessions, about 235,000 users, spanning 7 months. The research zeroes in on one thing: what actually determines whether someone uses it well.
Why it's worth reading: two findings echo each other. Among sessions that produced code, the success rates of the 10 largest professions are all within 7 percentage points of software engineers'; meanwhile, expert-level users hit "verified success" (28% to 33%) at nearly double the rate of novices (15%). Per this data, what determines success is how much a person understands about the task at hand — not whether they can write code.
How do you analyze that many real conversations without violating privacy? The research uses privacy-preserving analysis: a Claude model automatically reads each de-identified session transcript, labels it, and cross-checks the labels against system-recorded telemetry (like whether the session actually added or removed lines of code). Researchers never read any individual's actual chat log — they only look at aggregate patterns across a large number of users. The two datasets agree closely: among sessions the AI classified as "created or modified code," more than 90% showed actual code changes in the telemetry.
Scope note: this research only covers interactive sessions via the CLI, Claude.ai, and the Claude Code desktop app. It excludes headless mode via single claude -p commands, and excludes batch usage run through third-party IDEs or SDKs.
Users Set the Direction, AI Does the Legwork
The research splits every decision in a session into two types. Planning decisions cover "what to do, which path to take, what counts as done." Execution decisions cover "which file to change, what code to write, which language to use, which command to run." A classifier then judges whether the user or Claude made the call for each decision, producing two numbers per session.
In plain terms: humans set the direction, AI does the legwork. This division shows up consistently, session after session.
Look at it from another angle — swap "decisions" for "actions." A Claude Code session is a back-and-forth between human and Claude: the user writes an instruction, Claude goes and does a string of work, then the user writes the next instruction. A typical session runs about 4 rounds. Historically, each user instruction triggers about 10 Claude actions on average, sometimes more than 100. In each round, Claude reads files, edits code, and runs commands, producing about 2,400 words of output on average.