How DoorDash Built a Great AI Shopping Assistant: It's Not Just the Model — It's Tools and Memory That Make It Work
- DoorDash launched Ask DoorDash, an AI assistant that searches restaurants, plans meals, and builds shopping carts through natural-language conversation, now in gradual rollout on iOS in parts of the US.
- The architecture splits into four layers — an orchestrator agent, specialized domain agents, a shared tool layer, and managed services — with business logic living in the shared tool layer instead of being stuffed into prompts.
- A three-tier memory system (long-term memory via offline batch processing, in-session memory updated in real time, and conversational memory with explicit extraction) powers personalization: 7 days of production data showed memory-backed grocery checkout conversion up roughly 24%, basket size up 17%, conversation turns down 7%, and restaurant recommendation conversion up 15%.
- A homegrown eval framework uses AI to simulate users running against recorded tool data, running 2,000+ automated evals a day, lifting agent quality scores by 8 points and shrinking regression testing from 6 hours to 20 minutes — enabling a model backbone migration that cut latency by 35% without hurting quality.
What DoorDash Just Shipped
Right now only two scenarios are live: restaurant search and grocery shopping — Android and web aren't there yet. Early data shows roughly 70% of messages are discovery-type requests, like "ramen nearby," "what should I eat tonight," or "how do I shop for vegetarian dinners this week." Most users go back and forth several times within the same conversation, tweaking as they go, rather than asking once and leaving. DoorDash also notes that the most common production failure mode right now is "facts not matching up" — recommending a restaurant that's actually closed, quoting a price that doesn't match the shelf, or saying something got added to the cart when it actually didn't. The fix follows the same idea in every case: every factual statement the AI makes must come from an actual tool call to the real system data, not a guess based on context.
What Actually Happens Behind One Order
Turn 2 is the key design choice here: when the user taps around and edits the cart widget, those edits change the cart object directly, without another AI model judgment call — the AI sits idle. That means the most time-consuming step, "call the AI," gets cut out of most of the interaction entirely.
How the Four-Layer Architecture Divides Labor
The orchestrator agent and domain agents talk to each other through a protocol called A2A (agent-to-agent) — like different departments in a company agreeing on a shared memo format: as long as the format matches, the restaurant team and the grocery team can each develop and ship independently, without being locked together for releases.
One detail worth noting is "agent stickiness": if the user's last turn was about groceries and this turn says "add it to the cart," the system defaults to keep routing to the grocery agent instead of re-deciding from scratch, until the user's intent clearly shifts. Each agent runs on Google's Agent Development Kit, with the AI model per role configurable — if something goes wrong, you can swap the model for just that role without shipping a new release.
The Memory System: Getting the Agent to Remember Who You Are
Every conversation an agent has starts from zero — it doesn't know who it's talking to. DoorDash's problem to solve was turning scattered user data into "the right context at the right moment," using a three-tier memory system plus a four-stage memory pipeline.
| Memory Tier | Update Cadence | What It Holds |
|---|---|---|
| Long-term memory | Daily/weekly batch | Diet preferences, dining habits, brand preferences, favorite stores |
| In-session memory | Real-time | Current cart, what's being searched right now, items just viewed then skipped |
| Conversational memory | Generated as the conversation happens | Things the user explicitly says in this conversation that should be remembered long-term |
Having these three memory tiers isn't enough on its own — the AI still has to figure out which categories of memory to check for a given turn. DoorDash breaks that process into four stages:
Take "I want to make chicken tikka masala" as an example: step one, intent resolution, recognizes this as a recipe request, so what needs checking is brand preference and portion-size preference, not nearby restaurants. Step two, query planning, splits it into an "exact-keyword lookup for brand preference" plus a "semantic query for portion history," searching only within grocery-related memory partitions. Step three, retrieval and ranking, runs those in parallel and puts the most recently confirmed "usually buys two portions" at the top. Step four packages everything into a handful of structured facts stuffed into the prompt — say, "two portions, usually buys Oatly oat milk, no meat" — and the agent picks ingredients based on those few lines, instead of dumping hundreds of historical memories on it all at once. Query planning itself is parameterized along three dimensions: whether to search by semantic similarity, exact keyword match, or pull directly from structured fields; which memory partition to search; and boundaries like result count limits, similarity thresholds, and time windows.
To make keyword retrieval actually work, the system also builds a Memory Bank Index at the start of every conversation — a list of the vocabulary that actually exists in this user's current memory, stuffed into the prompt — and the agent searches against that index; searching for terms not in the index just wastes tokens. For especially large memory blocks like order history, the agent first does a lightweight "peek" to get summary info like record count and availability, then decides whether it's worth pulling in the full content, instead of dumping the entire order history into context up front.
When Memory Meets Shelf Reality, Who Gives Way
Memory and current reality are two different things. If a user who "always buys Oatly" happens to hit a store with none in stock, the agent discovers this while checking inventory and switches to an alternative; if a user said "$60 budget" but the cheapest way to cover this week's list actually comes to $72, the AI tells the user honestly rather than faking a $60 total. This reconciliation isn't a separate layer — it happens within the same turn, where the agent first pulls stored preferences, then checks them against the real-time prices, inventory, and hours returned by the search and cart tools; when memory conflicts with reality, the plan adjusts to match reality.
How a Memory Forms — and How It Gets Precisely Forgotten
How a Sentence Becomes a Usable Memory
"We're vegetarian, still buying for two this week."
ContentHousehold is vegetarian, usually buys for two
DurationLong-term, unless the user says otherwise
But if a sentence is just "this one's for a friend, needed tonight," that's a one-off statement and gets skipped — it doesn't go into long-term memory. The bar for "durable" shifts by scenario too: in a restaurant context, "I don't feel like ramen" defaults to being read as a passing mood, unless the user uses explicit durability words like "always" or "never" — otherwise recording it would have the agent avoid noodle dishes forever after; in a grocery context, an offhand "I like Oatly" is exactly the kind of brand signal worth remembering long-term. Price-sensitivity preferences only count if the user actively initiated it — say, they asked "what's cheap at Safeway" themselves — that gets recorded; but if they merely accepted an AI-recommended cheap store passively, that's not treated as a lasting judgment about the person.
"Forgetting" Is Harder Than "Remembering"
If a user says "forget that I like the grilled cheese from Nova's," doing a crude delete by similarity or keyword match would easily wipe out other memories related to "Nova" or "cheese" that the user actually still wants kept.
Deletion works differently from saving: saving is triggered asynchronously and adds no latency, while deletion runs synchronously, so the agent can tell the user in its reply exactly which items were removed. There's also no single expiration rule across all memory: cooking oil bought 80 days ago is probably still in the kitchen; milk bought 12 days ago has probably been finished; a self-stated preference like being vegetarian has no expiration at all, unless the user changes their mind. How long a memory should live depends on what it actually is, not a one-size-fits-all decay formula.
How Do You Know the Agent Is Actually Any Good?
There's no single correct answer for an agent conversation, and manual spot-checking doesn't scale. DoorDash built an automated eval framework that turns "did this conversation actually help the user" into something that can run daily and be measured.
Each standard is written as a binary judgment call — for example, "did the recommended restaurant meet the user's stated delivery time, budget, and dietary restrictions." Each standard first checks whether it even applies to this scenario before scoring, so reorder-specific checks don't accidentally get deducted against a recipe scenario.
Every conversation is instrumented via OpenTelemetry and stored in ClickHouse. Raw logs carry too much information, so a Python script slices them into evidence tailored to each scoring standard — the judge only sees the slice it needs.
An AI plays the user, running through a full conversation against a pre-written scenario script, paired with a pre-recorded set of tool response data (not querying the real database) — guaranteeing every run of the same scenario starts from an identical baseline, so scores stay comparable.
Uses the GEPA algorithm to iteratively refine the scoring prompt until it matches human-labeled judgments; once calibrated, it can automatically review 2,000+ conversations a day.
During evals, calls don't actually hit live inventory or order APIs — they replay a pre-recorded, fixed response instead. That way, no matter how many times the same test scenario runs, the starting conditions are always identical, so any score change can be attributed to the agent itself, not random factors like whether the shelf happens to have stock today.
An algorithm that automatically rewrites the scoring prompt: it keeps generating new versions of the prompt, keeps the one that best matches human-labeled results, and repeats the calibration until the AI judge's judgment gradually converges with a real human reviewer's.
How the Eval System Caught Two Real Incidents
Case One: The AI Sounded Like It Was Reading Code Out Loud
The agent did complete the task, but its replies to the user leaked internal system jargon — words like "reorder skill," "fetch," "in parallel" — sounding like it was reporting to another program, not like a shopping assistant talking to a person.
The agent's communication rules were reorganized, separating internal skill instructions from user-facing language. Offline validation showed this class of issue dropped 11%; a full regression run across all scenarios confirmed no new issues were introduced, and live monitoring after launch confirmed the improvement held.
Case Two: Swapping the Model Tanked the Score
When Gemini 3.5 Flash launched, DoorDash's agent was running on Claude Sonnet 4.6. Flash's benchmark numbers suggested it could be faster, and the team wanted to switch over to cut latency — but swapping the backbone model is a high-risk change, and they needed to confirm quality hadn't dropped before shipping it.
What the evals uncovered was a formatting compatibility problem, not a capability gap: Flash formatted some tool parameters differently than Sonnet did — for example, passing a JSON object somewhere a string was expected; and some conventions baked into the system prompt that Sonnet could infer on its own, Flash couldn't pick up. The fix was equally targeted: on one side, deterministic error-correction logic was added to tool inputs, so malformed parameters get corrected rather than the call simply being dropped; on the other, conventions that were previously left implicit in the prompt were spelled out explicitly — for instance, explicitly requiring the exact store name as given in the data, with no rephrasing allowed. After the fix, a re-run of the evals matched Sonnet's score, within the noise margin of the eval system itself. After launch, tracking live quality and engagement metrics showed latency down 35% with no drop in quality.
What This System Delivers Beyond Ask DoorDash
Business logic living in a shared MCP tool layer, instead of being hardcoded into prompts, means the same search, cart, and inventory tools can be picked up directly by any other team at the company building AI features. Every time the backend service is upgraded, all the agents built on top of it automatically benefit, without each team having to reintegrate separately.
Memory is designed as an independent tool layer, not a feature that belongs to any one agent. If one agent learns during a shopping conversation that "this household is fully vegetarian," other agents — reservations, delivery recommendations, whatever — can immediately use that fact, without every team having to re-solve extraction, dedup, and retrieval on their own.
Automated evals compressing regression testing from 6 hours to 20 minutes means model or prompt changes can be validated in tens of minutes. A high-risk move like "swap the backbone model" can pass through a low-risk offline environment first, before ever being exposed to real users.
Cart edits changing versioned objects directly, without a round trip through the AI model, is an approach to cutting end-to-end latency that doesn't depend on the model itself getting faster — a pattern directly transferable to any "AI plus real-time inventory or pricing" application.
What the Production Data Actually Says
Building a good AI agent is already hard. Knowing whether it's actually good is harder.Raghav Saboo, Head of Recommendations and Search, DoorDash
Buying a Week's Groceries: From Adding Items One by One → One Sentence, AI Builds the Cart in Two Minutes
DoorDash just launched an AI shopping assistant, Ask DoorDash. Its secret to being fast and reliable isn't a smarter model — it's that it remembers who you are, and it gives itself 2,000 practice exams a day. Here's the whole thing, with pictures.
↓ Read the whole page · one figure animates
DoorDash is the largest food delivery and errand platform in the US (think Meituan Waimai). It just launched an AI assistant called Ask DoorDash, currently in gradual rollout on iOS in parts of the US (a small slice of users get to try it first), and it can do two things: search restaurants, and buy groceries.
✘ But it doesn't know which stores are open right now, whether there's avocado in stock, what the real price is, or that your whole family is vegetarian
None of that lives in the AI's "brain" — it changes minute to minute and can barely be scraped off the web. A smart model alone can't answer it.
Buying a week's groceries used to mean searching and adding items one by one in the app. Now it's one sentence.
Search and add items one by one in the app yourself, cobble together a week's worth for two people, and keep an eye on staying under $60.
Say "build a $60 vegetarian list for two," and the AI checks real-time inventory and pricing to build a full cart in two minutes — you just tap to tweak.
There's a time-saving trick here too: when you swap a brand, change a quantity, or remove something already at home on the cart widget, those edits change the data directly and don't go through the AI at all — which is why it's fast. But "one sentence builds the cart" only looks simple. DoorDash had to get two hard things right: getting the AI to remember who you are, and having a way to actually confirm it's any good.
First, the harder of the two problems: how does the AI recall the right handful of facts about you at the right moment?
When XiaoHu tosses off one sentence, the AI doesn't guess from scratch — it first pulls just the relevant few out of hundreds of memories (vegetarian, usually buys Oatly, two portions), leaves out everything else, then checks real-time shelf inventory and pricing before assembling the cart. The same memory system forgets precisely too: say "forget this preference" and it deletes only that one entry, without accidentally wiping out neighboring memories.
The second hard problem: how do you know if the AI is any good? Before, it was just sparse employee feedback (about 1 report a day); now, an AI plays the role of thousands of customers chatting with it, and another AI judge scores the results — automatically "quizzing" it 2,000+ times a day. Combining memory and evals produces this,
never gonna finish
that understands words,
but can't see:
- × which store's still open
- × whether there's avocado
- × the real price
- × that you said your family's vegetarian
than doing it by hand
just picked these 3
AI judge scores it
DoorDash's own measurements,
a 7-day window,
no independent reproduction.
not because the model's smarter
Members can copy or download the full Markdown, and let their AI subscribe and ask.
Found this useful? Leave your email — every new explainer, the moment it's out. Unsubscribe anytime.