Give every department its own AI, and it gets harder to use: Sierra collapsed four into one — and it now writes 70% of the company's code

A retrospective: Pinecone has run 75,000+ sessions since launch, serves 600+ employees, and connects to 37 internal systems through an MCP Gateway.

By Sierra07-11 · 8 min read⚑ Company PR · read with that in mind
🎧 Listen to the podcast0:00 / --:--
0:00 / --:--
Deep Dive · XiaoHu Explains
Quick Take
  • Sierra's six-person AI acceleration team spent a few months merging four role-specific agents — for support, data analysis, engineering, and sales (PINE, Pinewood, Pinecone, Reggie Jr) — into one unified agent, now called Pinecone.
  • Since launching in March, Pinecone has run over 75,000 sessions, serves 600+ employees, and now originates 70% of the company's code pull requests (PRs).
  • Pinecone connects to 37 internal systems through a self-built MCP Gateway: it inherits each employee's own permissions, runs a policy check on every call, isolates customer data, and leaves an audit trail.
  • The team found more value in having the agent stay with a project and proactively step in when a task is ready, rather than only answering on demand — though most sessions today still start with a human.
  • The team admits its current metrics — session counts, PR counts — only measure activity, and it hasn't yet found a good way to measure whether outcomes are actually getting better.
⚑This piece is adapted from Sierra's official blog, describing the company's own internal AI rollout in its own words. Session counts, user numbers, PR share, and other figures are self-reported by Sierra and have not been independently verified.
01Starting Point

Why everyone wants to clone their best engineer

Sierra's six-person AI acceleration team spent the past six months building internal agents for every department at the AI customer-service company, and wrote up what they learned along the way.

This past March, Sierra merged its four role-specific agents — for support, data analysis, engineering, and sales — into a single unified agent named Pinecone. Four months later, it has run over 75,000 sessions, serves 600+ employees, and now originates 70% of the company's code pull requests.

◆
Through its self-built MCP Gateway, Pinecone connects to 37 internal systems — the widest-reaching, most deeply integrated internal agent the company has run so far.

A 1968 study found that the very best software engineers vastly out-produce average ones. For the next fifty years, companies had exactly one answer to that: try to hire those people.

This past January, Sierra's engineering team came back from break energized by frontier-model progress and started running agents in parallel using git worktree, Claude Code, and Codex. On some tasks, output jumped to 5x.

What is git worktree

A built-in Git feature that lets the same repository open several independent working directories at once, so multiple agents can run different tasks in parallel without stepping on each other. Think of it like partitioning one house into several separate rooms — everyone works in their own room, and no one messes up anyone else's stuff.

If agents can 5x an engineer's output within a month, what happens when you bring that acceleration to the whole company? That question is what led to the six-person AI acceleration team — and this piece is their retrospective on what they did and what went wrong along the way.

5x
Output multiple on some tasks after the engineering team started running agents in parallel with git worktree, Claude Code, and Codex in January
6
Size of the AI acceleration team Sierra assembled
02Reorg

Give every department its own AI, and it gets harder — not easier

The team's first approach was to split things into four role-specific agents: a support agent (PINE), a data-analysis agent (Pinewood), an engineering agent (Pinecone), and a sales agent (Reggie Jr). It sounded intuitive. In practice, it broke down.

The surface-level problem was that employees had to remember which agent handled what — and the team's fondness for pine-themed names meant PINE, Pinewood, and Pinecone were easy to mix up. The deeper problem was that the work that actually matters at a company mostly crosses departments; it rarely lives inside just one.

A company is essentially a pile of work that needs to get done. Take shipping a product: it touches engineering, but also sales, marketing, legal, operations. Departments exist because no single team or person can do every part of the whole job. AI is changing that — it's increasingly able to carry a piece of work from start to finish on its own.

The org-level call

The team folded all these role-specific agents into Pinecone: one unified agent, one Slack account, one URL, one uninterrupted thread from question to delivery. Pinecone figures out on its own which system to pull data from and what to do — employees no longer have to first figure out which agent to ask.

SupportPINE
DataPinewood
EngineeringPinecone
SalesReggie Jr
↓↓↓↓
PineconeOne Slack account · one URL · one uninterrupted thread

Sierra had already validated this logic once, in its own product. Agents built on Sierra are end-to-end — one agent handles everything from product questions to account setup, troubleshooting, and billing, not the old IVR-style "press 1 for support, press 2 for sales" routing.

Folding everything into one agent brings the company closer to the actual value: the work itself, getting done. Every improvement benefits the whole company; everyone gets faster together.

03From Q&A to Proactive

A good agent knows when to step in on its own

Most work isn't finished in a single conversation — it spans days, weeks, sometimes months, with learning happening along the way, priorities shifting, new information surfacing. An agent that only shows up when asked, then disappears when the session ends, has limited use.

Core shift

Pinecone carries context with it continuously, picking up where the last thread left off until the whole piece of work is done — not just the single question that was asked.

Ordinary agent
Only shows up at isolated moments when asked, then disappears when the session ends
Pinecone
Webhook firesTask is readyReview lands
Carries context continuously, proactively takes the first pass at key moments

Carrying context continuously is also what makes Pinecone proactive. It doesn't wait to be asked — it acts as soon as the next step is ready, say when an artifact triggers a webhook, a task lands in Linear, or a review comes in. It gathers context first, does a first pass, then hands the judgment calls to a human. Meeting notes are ready before the meeting starts; interview write-ups are drafted before the interview ends; when a review lands, it already comes with a summary, key risks, and suggested comments. The goal isn't more notifications — it's less unfinished work landing on someone's plate.

The team hasn't fully gotten here yet — most sessions still start with a human. But flipping that order, so the agent comes looking for a person when needed, is the direction that persistence is pulling the team toward.

04Permissions and Context

What actually blocks AI is that it doesn't know your company

What used to hold AI back was raw model intelligence. Today's frontier models are smart enough. The real bottleneck has moved somewhere else: company-specific context — the actual workflows, the history, the judgment calls that never show up in any training data.

This past January, two people on the team used Claude Code and Opus 4.6, wired up to internal systems through MCP (Model Context Protocol) and command-line tools, to put together a data-analysis agent. With little extra guidance, it could investigate a customer issue across Slack, GitHub, ClickHouse, Salesforce, and PagerDuty in a matter of minutes. Work that used to take an afternoon became the first step of troubleshooting and incident response.

That same pattern extends beyond debugging. An agent with full context can prep for a customer meeting, research an account, review a contract or an RFP, or trace back a product decision — turning scattered work into a finished draft. But giving an agent that much context creates a new security and privacy risk of its own.

Sierra built its own layer to handle this: the MCP Gateway. Pinecone inherits each employee's own permissions, every tool call goes through a policy check, customer data stays isolated, and every call leaves an audit trail.

Sierra MCP Gateway
Sierra's MCP Gateway connects to 37 internal systems through a single unified gateway, where every call gets a permission check. Image source: Sierra's official blog.
Inherits employee permissionsPinecone can only do what that specific employee is already permitted to do — never more than their own access level Policy check on every callAccess isn't unconditional once granted — every single tool call runs through a policy check first Customer data isolationDifferent customers' data stays isolated — it never bleeds across into another customer's data within one call Leaves an audit trailEvery call is logged and traceable — if something goes wrong, you can trace who called what and when

Pinecone is built on top of Claude Code and Codex. Updates to these two underlying tools are a tailwind, but the frontier keeps shifting — one model might be best at planning, another at writing code, another at prose. By owning the layer above the model, the team can route each task to whichever model fits best, switch away when one model has an outage, control cost, and avoid being locked to a single vendor. The advantage that actually sticks around is owning the context, workflows, and routing layer that makes every model work better.

05System of Record

The agent is the new interface — the old systems are still the old systems

Every piece of work produces something concrete — an artifact. Agents that write code found their artifact first: the pull request (PR). Every other department has its own equivalent: a customer case study, a contract, an RFP questionnaire, a pitch deck, a performance review.

These artifacts are both the input an agent needs to do the work and the output it hands back. Ask Pinecone to help revise a pitch deck, and what comes back is the revised deck itself — not a chat message suggesting how you might revise it.

The team found it works better to plug into the existing systems of record rather than replace them. GitHub still owns the PR, Salesforce still owns the customer account, Linear still owns the issue — the agent is the layer built on top that ties them together.

Pinecone · Unified operating layer
↓↓↓
GitHubOwns the PR record
SalesforceOwns the customer account
LinearOwns issue progress

Replacing these systems would mean rebuilding decades of mature software from scratch. Worse, it would split the company in two: one group working through the agent, another working directly in the original tools, each leaving a different record behind. The team's bet is that these products slowly become more like a backend, with the agent becoming the primary operating interface.

06Data vs. Truth

75,000 sessions doesn't mean things actually got better

Since its first commit in March, Pinecone has run over 75,000 sessions and serves 600+ employees. It now originates 70% of the company's PRs, plus hundreds of automated tasks quietly handling work no one explicitly prompted.

75,000+
Total sessions Pinecone has run since launching in March
600+
Employees using Pinecone
70%
Share of company PRs now originated by Pinecone

Numbers like this are easy to treat as an achievement, and early on that's fair — they prove the tool is actually being used, not sitting on a roadmap slide gathering dust. But session counts and tool-call counts are activity, not outcomes. A team could easily make its token usage look great while nothing downstream actually improved — same error rate, same cycle time, just with AI now in the loop.

So token usage is a reasonable starting point — a team has to build the habit of using the tool before it can even ask whether the tool works. But that's not where the value is, and the team doesn't want the story to end there. What they actually want to know is what the agent changed: did a deal close faster, did a customer's issue get resolved in one pass, did someone get to go home on time instead of grinding through a review until midnight again. The team hasn't found a good way to measure any of that yet — session counts and call counts are just easier to count. Closing that gap is the next thing to figure out.

07What's Next

Still experimenting: getting the AI to reflect on its own day

The team is trying something new with Pinecone: having it reflect each day on the work it completed and suggest improvements to its own skill set.

Long-term, the team wants an agent that keeps learning from Sierra and improving itself — not just one that finishes a day's work for Sierra and stops there.

The team plans more technical posts digging into the system: Allen Chen will cover the iterations Pinecone has gone through and the architecture behind it, Mihai Parparita will cover how the MCP Gateway safely gathers full context, and Rohith Ravi will cover Agency, the underlying infrastructure that supports all of it.

08Closing

From hunting for talent to giving everyone one

Back to that 1968 study — it found a 10x productivity gap between the best engineers and average ones. For fifty years, the only answer companies had was trying to hire that small handful of people.

1968

A study found top engineers out-produce average ones 10x. For the next fifty years, a company's only answer was trying to hire that small handful of people.

→
2026

Sierra gives its unified agent, Pinecone, to every single person at the company — so everyone gets that edge, without anyone having to go hire it.

Now there's another answer: give everyone an agent, and everyone gets that edge. This shift isn't just about getting more done. It frees up time for the part only humans can do — judgment, taste, creativity, and keeping a relationship going.

A team could easily make its token usage look great without making anything downstream actually better — same error rate, same cycle time, just with AI now in the loop.Sierra's official blog, "AI-pilling our company: lessons learned"
This piece is compiled from Sierra's official blog post "AI-pilling our company: lessons learned," by Neil Rahilly, published July 9, 2026. Session counts, employee usage numbers, PR share, and other figures are self-reported by Sierra.
Source
SierraSierra·Read the source
↑ End of article
✓ This article is free to read in full

Members can copy or download the full Markdown, and let their AI subscribe and ask.

Read the original ↗
📮 New explainers, straight to your inbox

Found this useful? Leave your email — every new explainer, the moment it's out. Unsubscribe anytime.