Twelve lessons in harness engineering from a Google principal engineer
- Ryan Lopopolo, now a Principal Engineer at Google Cloud, was previously at OpenAI. He distilled a year of work into twelve lessons in a public repo you can hand directly to your agent as reference. Most of the evidence comes from his time at OpenAI.
- The core move is counterintuitive: treat the chosen model and coding agent as an immutable black box. Don't tweak a single parameter. Only change the two things around it: context and tools.
- The lessons split into three layers. Three set the boundaries: lock the worker, deploy the private process-data iceberg, and measure by evidence. Six are levers: give one agent the whole job, route context just in time, make capabilities legible, let the repo teach the agent, keep autonomy inside authority, and prove claims in the real environment. The last three build compounding value: turn feedback into infrastructure, preserve coherence, and run settled work as a loop.
- The evidence: an internal product grew from an empty repo to roughly a million lines of code in five months, with about 1,500 merged PRs — all written by Codex, none by a person.
- What you can take away: a six-step improvement process, three fill-in-the-blank templates, an eleven-row table mapping claims to evidence boundaries, and a nine-item checklist for when your A/B test doesn't count.
Twelve Lessons, and One Word Often Mistranslated
Let's get the term right first. "Harness" here means the gear of a draft horse — the saddle, reins, horseshoes, and traces. The repo's description even starts with a horse emoji. It's easy to misread it as "to restrain," but the direction is the opposite.
The horse can already run fast. What's missing is the saddle, reins, and a track that direct its strength where it counts. That's what harness engineering builds.
Ryan Lopopolo, now a Principal Engineer at Google Cloud working on agentic GCP, wrote this. Before that he was at OpenAI, where he helped build the Seattle office and OpenAI Frontier for enterprise — working on exactly what this guide covers: context, tools, permissions, evals, observability, orchestration, and feedback loops for agents. Most of the evidence in these twelve lessons comes from that stint. Earlier, he led the data marketplace as tech lead at Snowflake, led a 40-person team supporting 350 engineers at Brex, and was tech lead for infrastructure and payments at Stripe as it grew from 600 to 6,000 people. In his spare time he writes Artichoke, a Ruby implementation in Rust.
When he published the twelve lessons, he said they represent a year of his work — practice and technique together.
He distilled that practice into twelve lessons and put them, with evidence, case studies, and executable processes, into a public repo. The repository's own content is licensed under CC BY 4.0. You can use it two ways: read it directly, or hand it to a coding agent along with the system you want to improve, letting the agent use the routing index to find which lesson applies.
Why it's worth your time: Every one of the twelve lessons comes with verifiable evidence. One internal product grew from an empty repo to roughly a million lines of code in five months with about 1,500 merged PRs, all written by Codex with zero handwritten code. A massive 60-hour, 300-million-token refactor needed only an initial prompt plus two follow-ups. After enabling a lint rule, about 600 violations were fixed and tests completed in a single PR.
Here's the full map. The lessons are organized into three layers: top three set the boundaries, middle six are the levers, bottom three determine whether the system compounds over time. We'll go through them one by one.
Lesson 1: Lock the “Worker” Down
Within a single evaluation or deployment cycle, keep three things fixed: the model, the agent's runtime, and the native tool interfaces. If any of these changes — even a generational upgrade — the whole environment needs re-acceptance from scratch.
Locking the black box makes failures attributable. When a capable agent fails, the cause is almost always one of five: missing institutional context, a capability that isn't actually usable, a missing feedback loop, insufficient authority, or no evidence at the exit. All five are properties of the environment; all five can be inspected and changed. Without a locked box, you stay stuck asking "is it the model, or my setup?"
Your own accumulated experience also needs re-acceptance. The habits the old model taught you — how small to slice tasks, how much orchestration to use, how long the inner loop can wait, where to keep a human in the loop — if carried over unchanged, you give away the new model's gains. Conversely, assuming every point release is better in every way is the opposite error.
One observation: GPT-5.3 is more reluctant than GPT-5.2 to tackle hard tasks. A single data point, not a trend.
Two numbers show how much a generational shift matters. When GPT-5.2 shipped, he was on vacation. He came back to find the team producing one to two extra PRs per engineer per day with zero additional environment investment. Later, a separate Symphony environment overhaul reported a 10x increase in PRs per engineer per week. The first came from a model upgrade, the second from environment changes. You have to track these separately — that's what locking the box enables.
During the GPT-5.2 release, the environment didn't change at all.
Result: +1 to +2 PRs per person per day.
During the Symphony intervention, the worker didn't change.
Reported: 10x increase in PRs per person per week.
Inner-loop latency is part of acceptance too. How well an agent reasons depends on how much feedback it gets before its trajectory runs off the rails. In the GPT-5.2 era, the Codex environment had no background shell, so the agent dutifully waited for blocking build scripts. When GPT-5.3 added a background shell, it stopped waiting. The team spent a week swapping custom build scripts for professional tools (trying Bazel, Turbo, and Nx along the way), stopping only when the full build fit in under a minute. That minute is now a hard cap — the moment it breaks, they stop feature work and fix the build graph.
Acceptance also includes subtraction. To let the agent "see" the interface, the team built a custom virtual display plus screen recording. Once the model could see the screen and click the mouse itself (computer use), they deleted the whole stack — without losing end-to-end verification.
Lesson 2: Deploy the Underwater Part of Your Private Process-Data
What's inside a general model's weights is just the tip of your organization's knowledge iceberg. What you actually need to get work done is below the waterline — private and constantly changing.
You need two things clear. One is the data ontology: what entities, metrics, and identifiers exist and how they relate. The other is the work ontology: what results, workflows, roles, tools, exceptions, evidence, and approval relationships exist.
An example helps. An internal data agent with only database access can't resolve the company's internal conflict over what "revenue" and "active user" mean. It also needs product lines, customer segments, teams, pilot customers, current plans, and how shared metric definitions are written. With those, it can give an answer that belongs to *this* company.
Leave current records in the data warehouse, ticketing system, docs, logs, codebase, and customer systems. Fetch an authorized slice via search and connectors. Don't build a second, perishable copy.
There's a pattern you can copy directly. The company Basis has two repositories: Arnold is the production monorepo, and Atlas is a second repository specifically for company context, separate from the production tree. Codex can combine personal notes, operational knowledge, and production code, while each source keeps its own update path. The implementation can be simple — his exact words: "just Codex plus a Git repo full of Markdown."
One more thing belongs to this layer: settled decisions should be promoted to the layer that can enforce them. A team and its security group picked an approved encryption implementation, and the decision stayed in Slack. Later, a new engineer was working with Codex, and Codex installed a different npm crypto package. The team pulled the decision back, wrote it as a guardrail rule in the repo, and re-ran that change.
Lesson 3: Give One Agent the Whole Job
Delegate the outcome. Give one main trajectory full responsibility: breakdown, execution, integration, evidence, and safe landing. Leave the steps to the agent. A person steps in at only three points: direction, judgement, and consequential authorization.
He often gives near-zero-constraint prompts, calling them "lazy prompts" — lazy about method, leaving it to the agent to decide how. Sparse delegation is also a test: it tests whether your environment makes the real requirements findable by the agent itself. When the agent misreads you, what gets exposed is exactly which part of your environment isn't built yet. The accompanying loop is "propose, observe, improve, rerun": review the trajectory, fix the environment, discard that result, run it again.
One request asked for a full red-team analysis, with every finding required to demonstrate impact or exploitability. Codex produced a report, a reproducer, a patch, and regression tests, then handled detailed manual code review and modified the implementation. The author reviewed and approved the implementation, then separately authorized it to prepare a minor release, merge the PR, and submit an advisory to RustSec — the Rust ecosystem's security advisory database.
The prompt only specified the outcome and the acceptance bar. Substantive review and approval at the delivery point stayed in the process.