Tutorial · Xiaohu's Take

Renaming one function saved up to two-thirds of the tokens for 14 models reading the same code

AI finds code the way grep does: a brute-force text scan. A vague name means hundreds of files to read.
The 1-minute Brief
  • The names you give your functions decide whether an AI flips through 19 files or 459 when it edits one. Modem ran 1,680 comparisons to get this number.
  • Why? Because AI finds code with plain-text search, not a compiler-drawn dependency graph or a language server that resolves symbols. The Claude Code team tried vector databases early on, then dropped them.
  • With search-friendly names, all 14 model-and-tool combos burned fewer tokens reading the same code. The biggest saver dropped from 70K to 24K average tokens. Median savings: 30%. Confidently wrong answers: zero.
  • Saved search budget becomes judgment. Give a weak model an eight-round bug-finding budget and it finds only 3 of 8 bugs in a monolith. Split and rename the code and it catches all 8. Most failed reviews don't even reach a conclusion—they burn the budget flipping through files.
⚑ Source: Modem's engineering blog. They designed and ran the experiments themselves. The post ends with a product pitch. It also notes several test limitations, which we list at the end.
1 A Basic Fact

How AI Actually Finds Your Code

Ask an AI to rename a function and its first move is a search of your repository.

That search is no magic. It scans every line of the repo for the exact string, same as pressing Ctrl+F in your editor. The command-line version is grep, decades old. AI uses its faster cousin ripgrep (evoked as rg).

It won't ask the compiler for a dependency graph, and no language server resolves symbols for it. It only sees the literal text. Search, read the surrounding lines, and if that's not enough, search a different word.

Claude Code searching a repository. It runs ripgrep, reads the context around a hit, and searches again if needed. Video: Modem

That's not a shortcut. The Claude Code team tried vector databases early on, then abandoned them because plain-text search simply worked better. The academic SWE-agent landed on the same approach: give the model a keyword search tool, nothing fancier.

File paths are search terms too. Ask how the session broker works, and the AI will scan filenames for session-broker, sessionBroker, and session_broker. A directory named session-broker/ counts as a hit before a single line of code is read.

So the loop is: search, read context, search again with a different term if needed, and only then start editing.

Search with rg Read the hits Enough? Else new term Start edits Not enough? New search term, next loop The vaguer the name, the more loops. Each loop reads files. The AI navigation loop The AI navigation loop Search with rg Read the hits Enough? Else new term Start edits Next loop The vaguer the name, the more loops. Each loop reads files.
How AI navigates your repository (illustration by Xiaohu)

So one input in this chain is entirely within the developer's control: can the words and filenames in your code serve as good search terms?

Modem did the math. The company decided in early 2025 to let AI write all its code. After a year, models had generated 99.9% of its 360K lines of TypeScript app code and 320K lines of tests. The findings below, and the 1,680 comparison runs, come from that year.

2 The Cost

What Does a Vague Name Really Cost?

How big is the difference between a good and a bad search term? They tested it with three functions.

All three do the exact same thing—create an API client for an external service. Only the names differ:

Three ways to name the same thing
export function create(apiKey: string) { ... }
export function createClient(apiKey: string) { ... }
export function createStripeClient(apiKey: string) { ... }

Searching each in Modem's ~2,900-file TypeScript repo:

Same repo, three names, search hits
create1,585 lines · 459 files
createClient466 lines · 23 files
createStripeClient43 lines · 19 files
All three searches took 40–50 ms; speed wasn't the issue. Bar length is proportional to hit lines. Data: Modem's measurements in their own repo. Chart by Xiaohu.

So the problem isn't search speed. grep returns matching lines, not answers. A line like const client = create(config) doesn't tell the AI if this is the client you meant, or just one of hundreds of unrelated creates.

To find out, it opens the file and reads around the hit, often expanding outward or opening other sections. A file can be tens to thousands of lines. At roughly 10 tokens per line (tokens are how AI counts text—they determine buffer limits and cost, roughly one token per English word), eliminating one wrong candidate costs hundreds to thousands of tokens. Eliminate a dozen and you've burned tens of thousands of tokens without touching the task.

Those 1,585 hits are scattered across a lot of noise: test fixtures, database inserts, background task registrations. The 43 hits for createStripeClient are its definition, call sites, and tests—all about the same client.