Google Research Introduces WikiSkill: How AI Agents Turn Mistakes into Persistent Knowledge and Better Skills

Why Does WikiSkill Matter? (Background and the Core Problem)

Today's AI Agents can already generate skills from their execution records. If an Agent repeatedly writes the wrong spreadsheet formula, for example, the system can extract a rule such as, “After writing a value, read it back and verify it.” That rule can be added to the next prompt so the Agent is less likely to repeat the same mistake.

The problem is that earlier skill-evolution methods mainly optimize the current skill rather than maintaining a separate body of knowledge that keeps growing. EvoSkill stores proposals and their evaluation results, Trace2Skill distills experience from trajectories, and SkillOpt uses rejected edits to guide further optimization. But most of that information remains buried in optimization history or gets compressed directly into the next version of the skill.

That creates three problems:

• Experience is hard to reuse across iterations: When a failure appears in the sixth iteration, the system may not know that it already analyzed a similar root cause in the second.
• The value of failed edits is easily lost: A skill update may lower the score while still revealing a useful error pattern. If the system merely rolls the skill back, it may propose a similar fix later and pay the validation cost again.
• The skill carries too many responsibilities: It must preserve experience while remaining concise enough to guide execution. As experience accumulates, the skill can become bloated; compress it too aggressively, and the rationale and applicability boundaries behind each rule disappear.

WikiSkill introduces the Wiki to close this knowledge gap. It stores “what happened,” “what was learned,” and “what must be followed during execution” separately, so even failed experiments can inform the next optimization round.

WikiSkill's Three-Layer Knowledge Architecture

WikiSkill is more than a new prompt. Its key design is a layered workspace:

• The Raw Layer preserves evidence: It records complete execution trajectories, including reasoning, tool calls, tool outputs, and final answers. Files are append-only, ensuring that later analysis sees what actually happened rather than a retrospective rewrite.
• The Wiki Layer preserves understanding: A Wiki Maintainer turns recurring patterns across trajectories into standalone pages: why a class of failures occurs, which approaches work, and what actionable fixes exist. logs.md records each iteration's findings, while skill-impact.md records every skill edit, validation score, and acceptance or rejection decision.
• The Skills Layer preserves action rules: The executing Agent reads the SKILL.md files in this layer. A companion PURPOSE.md links each skill back to the Wiki patterns it came from, so a concise rule remains traceable to its evidence and rationale.

The layers separately provide traceability, accumulation, and execution. Raw prevents evidence from being overwritten, Wiki prevents experience from disappearing with a version change, and Skills keeps execution instructions focused. A skill can be rolled back while the Wiki is not. The system can therefore undo an ineffective edit without losing the knowledge of why it failed.

SOURCE FIGURE

Original paper figure: three layers and a four-step loop

How Does the System Evolve? (A Four-Step Loop)

One complete iteration has four steps:

1. Inference Agent: The Agent completes training tasks using the current skill, and the full process is written to the Raw Layer. The paper deliberately prevents the Agent from reading the Wiki at this stage. The Wiki is analytical material for the component that improves skills; giving it directly to the executor could let the Agent bypass an incomplete skill and still solve the task, producing misleading training trajectories.
2. Wiki Maintainer: The system samples at most 8 trajectories from the current iteration, including up to 5 failures and 3 successes. Failed trajectories help locate root causes; successful ones reveal effective strategies and prevent the system from breaking behavior that already works. New findings create or revise pages under patterns/ and are added to the evolution log.
3. Skill Proposer: The Proposer first sees the Wiki index, the history of earlier edits, and a summary of which tasks succeeded or failed. It then uses ReAct to choose which pattern pages and raw trajectories to inspect. It creates or modifies only one skill per iteration, avoiding changes with too many variables to diagnose if validation fails.
4. Gating and Rollback: A candidate skill is retained only if it beats the best historical validation score; ties and regressions are rolled back. Whatever the outcome, the diff, score, and decision are recorded in skill-impact.md for the next proposal round.

A concrete example shows why this loop matters. In one iteration, the system notices that an Agent repeatedly picks up an object, inspects it, and returns it to the same place. It proposes a repair rule, but validation does not improve, so the skill is rolled back. The Wiki still preserves the loop pattern and the fact that the repair failed. In the next round, the Proposer uses that record to suggest a more specific rule: do not put the object back where it was. After that passes validation, a newly observed variation can refine the rule further: perform each type of action on a given object only once.

Select “Validation passed” or “Validation failed” to see what happens to the Raw, Wiki, and Skills layers after one iteration.

Main Results and Key Findings

The study spans five very different task types: mathematical reasoning in LiveMath, web search in SealQA, spreadsheet operations in SpreadSheet, long-document question answering in OfficeQA, and embodied sequential action in ALFWorld. The tested models are Qwen 4B, 9B, and 27B, Gemma 31B, and Gemini-3.5-Flash. Baselines include no skill, Trace2Skill, EvoSkill, and SkillOpt. Every reported result is the average of three complete evolution runs, with differences tested using paired Bootstrap resampling.

First, WikiSkill's advantage is not concentrated in a single model. Its average score beats every other skill-evolution method on all five models. Compared with the strongest competing method for each model, WikiSkill adds another 3.3, 5.1, 10.0, 5.8, and 12.0 percentage points, respectively. On LiveMath, Gemini-3.5-Flash rises from 33.0 with no skill to 72.6; on SpreadSheet, it rises from 50.5 to 76.6. On ALFWorld, Qwen-27B improves from 52.8 to 77.6.

SOURCE FIGURE

Original paper table: main results across five models

Second, model scale and skill evolution reinforce each other. The Qwen family gains more from WikiSkill as model size increases: 4B, 9B, and 27B improve by 12.3, 17.5, and 23.9 percentage points, respectively. Stronger models are better both at extracting useful procedural rules and at reliably following complex skills. At the same time, skills can partly compensate for model size: Qwen-9B with WikiSkill averages 47.4, beating Qwen-27B without a skill at 39.4.

Third, skills can transfer across models, but the direction of transfer cannot be assumed. A spreadsheet skill evolved by Qwen-27B lifts Qwen-9B on SpreadSheet from 24.3 to 50.5, well above the 33.6 it achieves with its own evolved skill. An OfficeQA retrieval skill evolved by Qwen-4B does not help its creator—the score falls from 30.2 to 28.5—but when Qwen-27B executes the same skill, its score rises from 42.1 to 52.9. Discovering a rule from experience and executing that rule well are different capabilities.

Transfer can also fail. Qwen-4B's spreadsheet skill causes Gemini-3.5-Flash to fall from 50.5 to 18.1. Trajectory inspection shows why: to protect itself from its own execution errors, the smaller model added low-level patches such as one-line Python commands and string conversions. Those rules can help the smaller model but prevent a stronger model from using complete scripts, create unnecessary tool calls, and exhaust the interaction budget early. General procedures can transfer; workarounds tailored to one model's weaknesses can become constraints.

SOURCE FIGURE

Original paper table: cross-model transfer matrix

Fourth, the ablation study shows that the Wiki should support skill evolution rather than directly solving tasks. When the Inference Agent cannot read the Wiki but the Skill Proposer can, Gemini-3.5-Flash averages 63.7 across four tasks. Removing Wiki access from the Proposer as well drops the average to 48.7. Conversely, giving both the Inference Agent and the Proposer Wiki access yields only 60.9. The paper's explanation is that an executing Agent with direct access to task knowledge in the Wiki may solve training tasks despite an incomplete skill. Its trajectories then stop revealing precisely what the skill lacks.

Toggle Wiki access and see how final skill quality changes

Change Wiki access for the two Agent roles and compare the measured scores of all four configurations.

Inference Agent reads the Wiki Skill Proposer reads the Wiki

Four-task average 63.7

Default: the executor does not read the Wiki; the Proposer does. This produces the highest average score, 63.7.

LiveMath72.6

SealQA44.7

SpreadSheet76.6

OfficeQA60.7

Executor	Proposer	Four-task average
Does not read	Does not read	48.7
Reads	Does not read	45.3
Does not read	Reads	63.7
Reads	Reads	60.9

View the original Table 3

Table 3 lists four Wiki-access configurations for the two types of Agent. The default WikiSkill setting gives access to the Skill Proposer but not the Inference Agent.

The experiments also show that skill evolution does not help every task-model combination equally. On OfficeQA, Qwen-4B falls from 30.2 to 28.5 with WikiSkill because it struggles to follow a multistep retrieval process consistently across extremely long documents. Gemini-3.5-Flash already starts at 100% on the ALFWorld validation set, so the system stops early and every method remains at 85.9 on the test set. WikiSkill corrects errors that can be addressed through procedural experience; it does not guarantee gains in every setting.

Limitations and Future Directions

WikiSkill demonstrates that persistent knowledge can help an Agent evolve its skills, but several clear boundaries remain before an Agent can autonomously grow while it operates:

• Skill retrieval remains unsolved: The experiments inject every active skill directly into the system prompt to remove trigger failures as a variable. As the number of skills grows, selecting the right skill for the current task will require a separate retrieval and routing mechanism.
• The gating rule is overly strict: A candidate skill must immediately beat the best historical validation score to survive. Edits that tie in the short term but could support later improvements are rejected, so evolution may get stuck in a local optimum.
• The Wiki keeps growing: The current system has no automatic mechanism for merging duplicate patterns, resolving conflicting knowledge, or pruning obsolete pages. Over longer runs, the knowledge base itself will need governance.
• The experiments use offline, multi-round evolution: The system completes training tasks first, then maintains the Wiki, edits a skill, and validates it. The paper does not test whether an Agent can learn while executing a single task that lasts for hours and involves hundreds of environment actions, then adjust its strategy immediately.

The paper therefore presents a verifiable mechanism in which experience enters a knowledge base and that knowledge is compiled back into skills. An Agent's failures are no longer disposable logs, and even rolled-back proposals can leave reusable lessons behind. For now, however, this learning happens between task batches, not while the Agent is doing real work and rewriting its capabilities in real time.