What Does a Vague Name Really Cost?

How big is the difference between a good and a bad search term? They tested it with three functions.

All three do the exact same thing—create an API client for an external service. Only the names differ:

export function create(apiKey: string) { ... }
export function createClient(apiKey: string) { ... }
export function createStripeClient(apiKey: string) { ... }

Searching each in Modem's ~2,900-file TypeScript repo:

So the problem isn't search speed. grep returns matching lines, not answers. A line like const client = create(config) doesn't tell the AI if this is the client you meant, or just one of hundreds of unrelated creates.

To find out, it opens the file and reads around the hit, often expanding outward or opening other sections. A file can be tens to thousands of lines. At roughly 10 tokens per line (tokens are how AI counts text—they determine buffer limits and cost, roughly one token per English word), eliminating one wrong candidate costs hundreds to thousands of tokens. Eliminate a dozen and you've burned tens of thousands of tokens without touching the task.

Those 1,585 hits are scattered across a lot of noise: test fixtures, database inserts, background task registrations. The 43 hits for createStripeClient are its definition, call sites, and tests—all about the same client.

And the savings go beyond this one task. Context has a limit. Every token spent eliminating a fake create is a token you can't use for the real edit. Worse, when context fills with plausible-but-wrong matches, the model's reliability drops. A name that grep can find in one jump protects both the AI's attention and its budget.

Why Isn't the Import Enough?

At this point, someone will ask: aren't imports like a signpost? Find the definition, go straight there.

That works if you're moving from a call site to a definition. An import is a one-way arrow. But the task here is a rename, which works in reverse: start at the definition and find every call site. Plain-text search has no backlink index. The file containing the definition has no record of who references it.

Modules point one way, they blur at barrels and package names, and they stay silent on methods. The name does the heavy lifting.

What Can Precise Parameter Types Save?

Everything above covers *finding* your code. What happens next depends on types.

Types differ from comments and READMEs in a fundamental way: comments can be ignored, types are enforced by the compiler. A type error during compilation is concrete feedback and the AI often fixes it within one round.

Even before the compiler complains, types are already saving tokens. A signature like this says what goes in and what comes out—often the implementation doesn't need to be opened:

function enrichUser(user: User): EnrichedUser;

TypeScript's any is sneakier: the damage is the same, but it *looks* like a type.

Names for types matter too

Here's a function where all three parameters are strings. Pass them in the wrong order, and the compiler stays silent:

transferOwnership(
  userId: string,
  orgId: string,
  projectId: string
)

transferOwnership(
  userId: UserId,
  orgId: OrgId,
  projectId: ProjectId
)

Research on this self-correction loop—letting a model iterate on compiler errors—shows roughly a 3× success rate increase.

Here's the connection back to the beginning. The OrgId named in that compiler error is likely defined in another file. To fix the error, the AI needs to read that definition, and the only way to find it is the same search it's always used.

Type names and function names obey the same rule: OrgId and UserEnrichmentResult grep directly to their definitions. Names like Result, Data, or Config send the AI back into the haystack. The compiler tells it where to look; the name decides how long the look takes.

This is the real problem with any: the compiler has nothing to say, so the AI has no search term. A bad name, at least, leaves a trail.

Does It Actually Work?

So far, all theory. Names should be specific, types precise. But what's the actual token savings? Modem turned it into an experiment.

They had Fable 5 compress this blog post into a one-page skill file to load into context. Then they gave two authoring models a spec unrelated to the post (a small notification pipeline library with retries, rate limiting, and signing). Each model wrote it twice: once without the skill file, once with. The authors were Haiku 4.5 (in Claude Code) and GPT-5.6 Sol (in Codex CLI). That gave them four versions of the same library.

Haiku without the skill produced exactly the code this post warns about:

Sol's no-skill output was already decent—calculateBackoffDelay in backoff.ts, TokenBucket in token-bucket.ts. With the skill, it went further: backoff.ts became notification-retry-backoff.ts. But the gap between Sol's two versions was visibly narrower than Haiku's.

Next, the reading test. Each of the four versions was dropped into the same 500 files from a real repository. Fresh sessions, only search-and-read tools, and questions like these:

How long is the wait before a retry, and where is it calculated?
At what point is the calculated HMAC signature (a tamper-evident seal) attached to the data being sent?
Given a notification ID, where can I find its full delivery attempt history?

14 model-and-tool combinations each answered 10 questions about all four versions, 3 times each. That's 1,680 runs, tracking tokens consumed and correctness.

With Sol's code, 12 of 14 combinations improved, median 12%; the other two didn't get worse.

The pattern is consistent: rougher original code leaves more room to improve. Sol already wrote decent code, so there was less to save.

No Privilege for the Author

One data point stands out. When Sol read its own no-skill code, switching to the skill version still saved 13% to 38%, depending on the tool driving it.

The model that wrote the code has no memory of where things are. It runs the same search as every other model.

A Type of Failure That Only Appears in the Vague-Named Version

Wrong answers were almost exclusive to Haiku's generic-named code, and they were confidently wrong: asked where this notification library stores its delivery history, it pointed to a Slack delivery ledger for something else in the repo. Asked where the HMAC signature gets attached, it pointed twice to a URL signer in a media proxy service.

The skilled version had zero wrong answers. A file named hmac-payload-signer.ts doesn't look like anything else. These failures barely appeared in Sol's two versions, which were correct 834 out of 840 runs; the few errors were scattered on both sides.

Does It Hold Up on a Real Project? (A 4,943-Line File)

A synthetic codebase only proves so much, so they picked a real challenge: Odysseus, a self-hosted AI workbench that gathered tens of thousands of GitHub stars in its first week.

The Odysseus interface. Image: Modem

Its routes/email_routes.py is 4,943 lines, about 3,900 of which are a single function. All 81 route handlers (each responding to a web request type) are jammed inside it. One function. An entire email client.

Modem refactored only this one file. No behavior change, no route changes, mechanically verified. They split it into concept-named modules:

email_routes.py
  4,943 lines
  81 handlers in one function

email/
  smtp_settings.py
  message_rendering.py
  folder_fallbacks.py
  delivery_history.py

Then they asked four models the same six questions. Same pattern: the models lost in the monolith saved the most (Haiku 4.5 at 34.3%, Grok 4.5 at 31.5%). The ones that weren't lost barely saved anything (GPT-5.6 Sol at 5.9%, Sonnet 5 at 3.9%). Haiku used to burn 400K tokens flipping through the monolith; after the refactor, it stopped.

More important is the worst case. Those catastrophic runs—400K tokens burned, nothing found—happened almost entirely on the un-refactored version. *Almost*, because Haiku had one bad run on the refactored version, and the location explains it: they only refactored one file. Haiku got lost in the untouched 1,800-line email_helpers.py junk drawer next door. The refactor raised the floor, and the new floor is the worst file you didn't touch.

What Do the Saved Tokens Buy You?

The first two experiments measured cost. There's a more important question: Does spending less lead to getting it *right*?

They planted four identical, subtle bugs in both versions of the email code (byte-identical changes, so both versions have the exact same defects). Then four models were given a fixed budget of eight rounds to check the code against the spec. Each model checked each bug twice: 32 reviews per version, 64 total across both.

Grok 4.5 and GPT-5.6 Sol found all of them in both versions. Eight rounds is plenty for a model with a search strategy, even in a 4,943-line file.

Haiku and Sonnet couldn't. In the monolith, they found 4/8 and 3/8 respectively. Most of those failed reviews didn't produce a wrong answer. They burned the whole budget flipping through files and never reached a conclusion. Give them the refactored code, and both go perfect.

Sonnet's row is especially telling. In the earlier retrieval experiment, it barely saved any tokens from this refactor. But add a budget limit, and its accuracy goes from 3/8 to 8/8. Being able to *afford* a search in a monolith and being able to *conclude* something there are two very different things.

Writing AI-Friendly Code (Actionable Advice)

We've covered the mechanism and the data. Now for what you can do today.

The skill file used in the experiment is MIT-licensed. We fetched it, read it, and found several rules that are easier to apply than the blog post suggests.

Almost none of this is new. Descriptive naming, precise types, comments that explain "why" — we've been telling each other this for decades, and forgiving ourselves for not doing it, because a human can always ask a colleague or check git blame. AI can't. It has your words, your types, and a search box. — Modem Engineering Blog

Finally, the edges of these conclusions. The authors list three: the skill file changes naming, file layout, types, and structure all at once, so the token savings can't be credited to naming alone; the Odysseus token numbers are only comparable within a row, not across tools; and the Odysseus test refactored just one file. One more limitation they don't mention, in our view: the skill file was distilled from the blog post itself by Fable 5, and then used to validate the blog—a bit of self-confirmation. They also point to promising data from Cursor showing that embedded retrieval on top of grep does help, so plain-text search might not be the final word. But grep, repository maps, and embedding retrieval all feed on the same input: the words in your source code.

How big is the difference? Modem tested three functions: functionally identical, each creates an API client for Stripe. Only the names differ. All three searches took 40–50 ms. The cost comes after the hit, when the AI opens files one by one to read and judge. Every read burns tokens (AI's unit for reading and writing text—it sets context limits and cost; roughly one token per English word).

And the savings go beyond this one task. Context (how much an AI can hold at once) is finite. Every token spent eliminating a false create is a token you can't use for the actual edit. Fill the context with plausible-but-wrong matches, and the model's reliability drops.

The logic sounds right. But what's the actual saving? Modem had Fable 5 distill the practice into a one-page spec, then had two models write the same requirement (a notification library with retries, rate limiting, and signing) twice: once without the spec, once with. The four versions were dropped into the same 500 real files, then 14 model-and-tool combos each answered 10 questions, 3 times per version.

The wrong answers are worth remembering. Asked where this notification library keeps its delivery log, the model pointed to a Slack delivery ledger for something else entirely—with total confidence. After the rename, code that merely looks like the thing can't shoulder a name that says exactly what it is.

The final set is even more telling. Same four planted bugs, same eight-round budget. The weaker models only found 3 to 4 of them in a real file—4,943 lines, 81 web request handlers packed into a single function. They mostly didn't give wrong answers. They spent all eight rounds flipping through files and never reached a conclusion. Split that one file into concept-named modules, and both go perfect.

All these numbers come from Modem's own design and execution; no third party has replicated them yet. On 2026-07-25, Xiaohu pulled the same branch: that 4,943-line file is already 6,032 lines.

Modem released the spec under the MIT license (free to take and drop into your own project). The most approachable rules don't change any behavior.

One more counterintuitive rule: humans and AI ask questions with spaces ("rate limit"), but RateLimiter won't match a phrase search. So the comment above each export must spell out that phrase in plain words. Modem counted on a ~700K-line repo: a one-word name is globally unique only 61% of the time. Three words: 96%. Three words is the turning point.