Give every department its own AI, and it gets harder to use: Sierra collapsed four into one — and it now writes 70% of the company's code
A retrospective: Pinecone has run 75,000+ sessions since launch, serves 600+ employees, and connects to 37 internal systems through an MCP Gateway.
- Sierra's six-person AI acceleration team spent a few months merging four role-specific agents — for support, data analysis, engineering, and sales (PINE, Pinewood, Pinecone, Reggie Jr) — into one unified agent, now called Pinecone.
- Since launching in March, Pinecone has run over 75,000 sessions, serves 600+ employees, and now originates 70% of the company's code pull requests (PRs).
- Pinecone connects to 37 internal systems through a self-built MCP Gateway: it inherits each employee's own permissions, runs a policy check on every call, isolates customer data, and leaves an audit trail.
- The team found more value in having the agent stay with a project and proactively step in when a task is ready, rather than only answering on demand — though most sessions today still start with a human.
- The team admits its current metrics — session counts, PR counts — only measure activity, and it hasn't yet found a good way to measure whether outcomes are actually getting better.
Why everyone wants to clone their best engineer
Sierra's six-person AI acceleration team spent the past six months building internal agents for every department at the AI customer-service company, and wrote up what they learned along the way.
This past March, Sierra merged its four role-specific agents — for support, data analysis, engineering, and sales — into a single unified agent named Pinecone. Four months later, it has run over 75,000 sessions, serves 600+ employees, and now originates 70% of the company's code pull requests.
A 1968 study found that the very best software engineers vastly out-produce average ones. For the next fifty years, companies had exactly one answer to that: try to hire those people.
This past January, Sierra's engineering team came back from break energized by frontier-model progress and started running agents in parallel using git worktree, Claude Code, and Codex. On some tasks, output jumped to 5x.
A built-in Git feature that lets the same repository open several independent working directories at once, so multiple agents can run different tasks in parallel without stepping on each other. Think of it like partitioning one house into several separate rooms — everyone works in their own room, and no one messes up anyone else's stuff.
If agents can 5x an engineer's output within a month, what happens when you bring that acceleration to the whole company? That question is what led to the six-person AI acceleration team — and this piece is their retrospective on what they did and what went wrong along the way.
Give every department its own AI, and it gets harder — not easier
The team's first approach was to split things into four role-specific agents: a support agent (PINE), a data-analysis agent (Pinewood), an engineering agent (Pinecone), and a sales agent (Reggie Jr). It sounded intuitive. In practice, it broke down.
The surface-level problem was that employees had to remember which agent handled what — and the team's fondness for pine-themed names meant PINE, Pinewood, and Pinecone were easy to mix up. The deeper problem was that the work that actually matters at a company mostly crosses departments; it rarely lives inside just one.
A company is essentially a pile of work that needs to get done. Take shipping a product: it touches engineering, but also sales, marketing, legal, operations. Departments exist because no single team or person can do every part of the whole job. AI is changing that — it's increasingly able to carry a piece of work from start to finish on its own.
The team folded all these role-specific agents into Pinecone: one unified agent, one Slack account, one URL, one uninterrupted thread from question to delivery. Pinecone figures out on its own which system to pull data from and what to do — employees no longer have to first figure out which agent to ask.
Sierra had already validated this logic once, in its own product. Agents built on Sierra are end-to-end — one agent handles everything from product questions to account setup, troubleshooting, and billing, not the old IVR-style "press 1 for support, press 2 for sales" routing.
Folding everything into one agent brings the company closer to the actual value: the work itself, getting done. Every improvement benefits the whole company; everyone gets faster together.
A good agent knows when to step in on its own
Most work isn't finished in a single conversation — it spans days, weeks, sometimes months, with learning happening along the way, priorities shifting, new information surfacing. An agent that only shows up when asked, then disappears when the session ends, has limited use.
Pinecone carries context with it continuously, picking up where the last thread left off until the whole piece of work is done — not just the single question that was asked.
Carrying context continuously is also what makes Pinecone proactive. It doesn't wait to be asked — it acts as soon as the next step is ready, say when an artifact triggers a webhook, a task lands in Linear, or a review comes in. It gathers context first, does a first pass, then hands the judgment calls to a human. Meeting notes are ready before the meeting starts; interview write-ups are drafted before the interview ends; when a review lands, it already comes with a summary, key risks, and suggested comments. The goal isn't more notifications — it's less unfinished work landing on someone's plate.
The team hasn't fully gotten here yet — most sessions still start with a human. But flipping that order, so the agent comes looking for a person when needed, is the direction that persistence is pulling the team toward.
What actually blocks AI is that it doesn't know your company
What used to hold AI back was raw model intelligence. Today's frontier models are smart enough. The real bottleneck has moved somewhere else: company-specific context — the actual workflows, the history, the judgment calls that never show up in any training data.
This past January, two people on the team used Claude Code and Opus 4.6, wired up to internal systems through MCP (Model Context Protocol) and command-line tools, to put together a data-analysis agent. With little extra guidance, it could investigate a customer issue across Slack, GitHub, ClickHouse, Salesforce, and PagerDuty in a matter of minutes. Work that used to take an afternoon became the first step of troubleshooting and incident response.
That same pattern extends beyond debugging. An agent with full context can prep for a customer meeting, research an account, review a contract or an RFP, or trace back a product decision — turning scattered work into a finished draft. But giving an agent that much context creates a new security and privacy risk of its own.
Sierra built its own layer to handle this: the MCP Gateway. Pinecone inherits each employee's own permissions, every tool call goes through a policy check, customer data stays isolated, and every call leaves an audit trail.
Pinecone is built on top of Claude Code and Codex. Updates to these two underlying tools are a tailwind, but the frontier keeps shifting — one model might be best at planning, another at writing code, another at prose. By owning the layer above the model, the team can route each task to whichever model fits best, switch away when one model has an outage, control cost, and avoid being locked to a single vendor. The advantage that actually sticks around is owning the context, workflows, and routing layer that makes every model work better.
The agent is the new interface — the old systems are still the old systems
Every piece of work produces something concrete — an artifact. Agents that write code found their artifact first: the pull request (PR). Every other department has its own equivalent: a customer case study, a contract, an RFP questionnaire, a pitch deck, a performance review.
These artifacts are both the input an agent needs to do the work and the output it hands back. Ask Pinecone to help revise a pitch deck, and what comes back is the revised deck itself — not a chat message suggesting how you might revise it.
The team found it works better to plug into the existing systems of record rather than replace them. GitHub still owns the PR, Salesforce still owns the customer account, Linear still owns the issue — the agent is the layer built on top that ties them together.
Replacing these systems would mean rebuilding decades of mature software from scratch. Worse, it would split the company in two: one group working through the agent, another working directly in the original tools, each leaving a different record behind. The team's bet is that these products slowly become more like a backend, with the agent becoming the primary operating interface.
75,000 sessions doesn't mean things actually got better
Since its first commit in March, Pinecone has run over 75,000 sessions and serves 600+ employees. It now originates 70% of the company's PRs, plus hundreds of automated tasks quietly handling work no one explicitly prompted.
Numbers like this are easy to treat as an achievement, and early on that's fair — they prove the tool is actually being used, not sitting on a roadmap slide gathering dust. But session counts and tool-call counts are activity, not outcomes. A team could easily make its token usage look great while nothing downstream actually improved — same error rate, same cycle time, just with AI now in the loop.
So token usage is a reasonable starting point — a team has to build the habit of using the tool before it can even ask whether the tool works. But that's not where the value is, and the team doesn't want the story to end there. What they actually want to know is what the agent changed: did a deal close faster, did a customer's issue get resolved in one pass, did someone get to go home on time instead of grinding through a review until midnight again. The team hasn't found a good way to measure any of that yet — session counts and call counts are just easier to count. Closing that gap is the next thing to figure out.
Still experimenting: getting the AI to reflect on its own day
The team is trying something new with Pinecone: having it reflect each day on the work it completed and suggest improvements to its own skill set.
Long-term, the team wants an agent that keeps learning from Sierra and improving itself — not just one that finishes a day's work for Sierra and stops there.
The team plans more technical posts digging into the system: Allen Chen will cover the iterations Pinecone has gone through and the architecture behind it, Mihai Parparita will cover how the MCP Gateway safely gathers full context, and Rohith Ravi will cover Agency, the underlying infrastructure that supports all of it.
From hunting for talent to giving everyone one
Back to that 1968 study — it found a 10x productivity gap between the best engineers and average ones. For fifty years, the only answer companies had was trying to hire that small handful of people.
A study found top engineers out-produce average ones 10x. For the next fifty years, a company's only answer was trying to hire that small handful of people.
Sierra gives its unified agent, Pinecone, to every single person at the company — so everyone gets that edge, without anyone having to go hire it.
Now there's another answer: give everyone an agent, and everyone gets that edge. This shift isn't just about getting more done. It frees up time for the part only humans can do — judgment, taste, creativity, and keeping a relationship going.
A team could easily make its token usage look great without making anything downstream actually better — same error rate, same cycle time, just with AI now in the loop.Sierra's official blog, "AI-pilling our company: lessons learned"
Give every department its own AI, and it gets harder to use: Sierra collapsed four into one — and it now writes 70% of the company's code
A retrospective from Sierra's six-person team: how to build an internal AI assistant, and what went wrong along the way — one page, with a diagram.
↓ One page, full story · one diagram animates
Sierra is an AI customer-service company. This year it assembled a six-person team and gave every department its own dedicated AI assistant (agent — an AI that can call tools and carry out multi-step work on its own): one for support, one for data analysis, one for engineering, one for sales. Sounded thoughtful. In practice, it was awkward.
✘ But the work that actually matters is mostly cross-department
Employees had to first remember which AI to ask for which task; worse, a single piece of work would bounce between several AIs, with none of them owning it end to end.
Sierra tore down all four dedicated AIs and merged them into one unified assistant, Pinecone. Employees no longer have to think about "who do I ask" — one entry point carries the work from start to finish. They also stopped it from being purely "ask a question, get an answer" — instead it keeps an eye on the task, takes a first pass the moment something's ready, then hands it to a human to sign off.
But handing the keys to that many company systems to a single AI brings a new problem: what makes it safe to touch all that data?
What actually blocks AI isn't a lack of intelligence — it's that it doesn't know your company: the actual processes, the history, the judgment calls only an insider would know. Sierra's fix is a "permission checkpoint" layer called the MCP Gateway: every time Pinecone wants to touch an internal system, it has to pass through this checkpoint first.
XiaoHu asks one question, and Pinecone decides on its own which systems to check. But every call has to pass through the MCP Gateway first: only XiaoHu's own permissions get used, customer data stays isolated, every call is logged. Old systems like GitHub and Salesforce still own their own records — Pinecone is just the unified front door on top. What used to take an afternoon of digging now gets a first draft in minutes.
Four months in, the numbers look great. First, a concrete feel for it: same task, investigating a customer error —
department its own AI.
- × Support → PINE
- × Data → Pinewood
- × Engineering → Pinecone
- × Sales → Reggie Jr
merged into one.
to a human.
a "permission checkpoint" first.
it now originates
600+ people using it.
to giving everyone an AI.
Members can copy or download the full Markdown, and let their AI subscribe and ask.
Found this useful? Leave your email — every new explainer, the moment it's out. Unsubscribe anytime.
