WikiSkill separates raw experience, a persistent wiki, and gated skills — then runs a loop where a maintainer distills your agent's traces into knowledge and a proposer turns it into skills that earn their place or get rolled back.
The paper's insight: skill-evolution systems optimize skills, but the insights that guide skill design stay scattered across optimization histories. WikiSkill fixes that with a knowledge layer that never forgets.
Every inference run's full transcript is captured into an immutable raw layer — successes and failures, with the actual tool calls.
A Wiki Maintainer agent samples low-scoring traces and distills them into pattern pages. The wiki is never rolled back — knowledge compounds.
A Skill Proposer writes candidate SKILL.md files. Each is validated on held-out tasks and accepted only if R_val > R_best — otherwise git-rolled-back.
Real numbers from a real run on a free-tier model (gemini-2.5-flash-lite, 8 turns). Click through the stages — this is what actually happened.
One loop, four commands. Evolution runs unattended overnight — the machine learns while you sleep.
$ pip install wikiskill-hermes $ wikiskill init demo --backend claude # pin your agent $ wikiskill evolve demo --iters 3 --model google/gemini-2.5-flash-lite --provider openrouter ✓ baseline validation: 9 val tasks, S₀=∅ → R=0.67 ✓ iter 1: wiki maintenance → skill proposal → gate → R_val=0.44 → REJECTED (rolled back) $ wikiskill compare demo nightly --iters 5 # did the skill actually help?
The framework caught its own bug: a maintainer agent noticed that "dead" runs were being graded against stale files — and the fix (fresh sandboxes per rollout) is now part of the harness. That's the loop policing itself.
Every rollout starts from a re-materialized task sandbox. A dead agent run scores 0.0 — never a stale file from a previous experiment.
Each workspace gets its own profile (HERMES_HOME / CLAUDE_CONFIG_DIR) with fresh sessions and memory — gating sees exactly the candidate skill set.
wikiskill compare runs both sides N times and reports per-task win/loss/tie with a paired exact-binomial p-value.
The wiki, the proposals, the gate outcomes — all in git. Rejected skills stay visible in skill-impact.md so they're never proposed twice.