Definition
Skills get better through real failures, not imagined ones. When your agent does the wrong thing, look at what happened, find the smallest likely cause, and make the smallest change that fixes it. Then rerun the case that failed, plus one that already worked. If an instruction was misread: split vague categories, say what the agent should do, and say who makes the decision.
You saved a small first draft in step 2. Now run it. Your goal is not to imagine every way the skill could fail; it’s to watch what actually happens when an agent uses it. When the agent does the wrong thing:
- Look at what happened.
- Find the smallest likely cause.
- Make the smallest change that fixes it.
- Run the failed case again.
- Make sure something that worked before still works.
That loop is how a one-sentence draft becomes a skill you can trust without turning into a giant instruction manual.
Improve the skill from real failures
Suppose your first skill says:
Pressure-test every assumption, one at a time, until the plan holds up.
You run it and the agent asks six questions in one message. Now you have a real problem to fix. You could add an entire section called “Conversation pacing” — or you could make the smallest change that addresses what actually happened:
Pressure-test every assumption, one question at a time, waiting for my answer before continuing, until the plan holds up.
The failure earned those extra words. That is the basic rule for improving a skill:
Add instructions because something happened, not because something might happen.
If you have a recorded session, use it as evidence. Look at what the agent saw, what it did, and where its behavior stopped matching your intention. You still have to decide why the failure happened, but you are making that decision from a real example instead of guessing. (The safety exception from step 2 still applies: do not wait for a real accident before adding a boundary around destructive or irreversible actions.)
✦Go deeperRevision as a writing workshopopen +close −
This is the oldest editing discipline there is, pointed at a new reader. In a writing workshop, the unit of evidence is the reader’s experience, not the writer’s intention. Here the reader’s experience arrives as a session record, and the running list of what your readers taught you — each failure, each small fix — is sometimes called the misreading journal. Keep it wherever you like; the point is that every important line in the skill can answer the question “what happened that put you here?”
Make the smallest fix that works
When something goes wrong, resist the urge to rewrite the whole skill. Fix the specific problem you observed — the failure picks the fix:
| Asked six questions at once | Add a pacing instruction. |
|---|---|
| Started work before you approved the plan | Add a confirmation step. |
| Made a decision that belongs to the user | Say who makes that decision. |
Small changes are easier to understand and easier to test. They also make it easier to answer an important question later: why is this sentence here? Ideally, every important line in the skill has an answer — it exists because the agent once did something you wanted to prevent, or because it protects an important safety boundary.
If you use version control
If the skill lives in Git, it can help to keep one behavior change per commit.
The second message tells you, months later, which behavior this change earned.
This is useful history, but it is not required for the method. The important part is being able to connect a change in the skill to the behavior that caused it.
Watch how a small skill grows
Imagine the same skill being used over several months. The first version says:
Pressure-test every assumption, one at a time, until the plan holds up.
Then real use teaches you more. The agent asks several questions at once, so you clarify: “Ask one question at a time and wait for my answer.” Later it keeps asking questions without giving any useful opinion of its own, so you tell it to offer its current view. Later still, it starts carrying out the plan while the user is still evaluating it, so you add: “Do not execute the plan until I confirm it.”
Each change answers something that actually happened, and nothing gets added between failures. Here’s that whole history as a timeline — click each failure to reveal its fix:
Five patches, five months
Nothing was added between failures.
That is how a short skill can become more reliable without becoming unnecessarily long.
✦Go deeperOne incident, one lineopen +close −
Cockpit checklists grow exactly this way: an incident happens, investigators read the flight recorder, and the checklist gains a line that answers what actually happened. Nothing gets added between incidents, which is how a document can stay one page long and still carry decades of lessons. Your session records are the flight recorder; the skill is the checklist.
When the agent misreads a line, fix the ambiguity
Sometimes the agent is not missing an instruction — it is following one in a way you did not expect. When that happens, try these three moves.
1. Split vague categories
What counts as "something"? The sentence is doing too much work. Split it, and "facts" and "decisions" get different rules.
2. Say what the agent should do
Negative instructions can leave a hole: the agent still has to decide what to do instead.
Now the replacement behavior is explicit.
3. Say who makes the decision
Resolved by whom — the agent, the user, another tool? Say it.
This matters especially when a skill is eventually used somewhere you did not originally imagine: another session, another teammate’s workflow, or even another skill. Write the rule so it still makes sense when you are not there to explain what you meant.
Press each move to light up the words it touched in a real fix:
Three moves, each reusable on any misread line
If a fact can be found by reading the code, look it up rather than asking me. The decisions, though, are mine. Put each one to me and wait for my answer.
The broken line ran in a context its author never imagined. Write for the furthest context the sentence will ever reach.
Give important rules a short reason
Sometimes a short explanation helps the agent apply a rule correctly in situations you did not explicitly list:
Ask questions one at a time and wait for my answer before continuing. Several questions at once are easy to miss.
The second sentence explains why the rule exists, which can help the agent preserve the intent even when the exact situation changes. Keep the explanation short. If it needs a paragraph or a lot of background, put that in a separate reference file and link to it instead.
Name repeated failure patterns
Sometimes several failures are really the same mistake in different forms. When that happens, it can help to name the pattern and explain why it fails.
Imagine an agent verifying a database migration. It repeatedly compares the newly migrated data against another value produced from the same migrated data. The two sides agree, but the check proves almost nothing, because both values came from the same source. You might describe the pattern like this:
Avoid self-agreeing checks: a test where both sides come from the same transformed data can agree even when the transformation is wrong. Compare against an independent source such as the original data, a fixture, or a hand-computed value.
That instruction covers more than one exact example — it teaches the agent what kind of mistake to recognize. Do this when you see a repeated pattern, not because every possible mistake needs a name.
Always rerun after a fix
Changing the skill does not prove that you fixed the problem. The change is still a hypothesis, so close the loop every time:
- Observe the failure. Look at the run where the agent behaved incorrectly.
- Find the likely cause. Which instruction was missing, vague, or misleading?
- Make the smallest useful change.
- Rerun the failed case. Does the new version fix the problem?
- Rerun something that already worked. Did your fix accidentally break another behavior?
That last step matters: a fix that solves one problem and creates another is not finished. And keep the failed cases you collect. Later, they become regression tests: old problems you rerun to make sure future versions of the skill do not bring them back.
Your turn: run, observe, fix, rerun
Take the draft you made in step 2 and use it on real work at least a few times. When the agent does something you did not want, write down what happened, then ask:
- What is the smallest instruction that would have prevented this?
- Is any word or category too vague?
- Have I said what the agent should do instead?
- Is it clear who makes the decision?
- Would a short reason help the rule make sense in other situations?
Make the smallest useful change, then rerun the case that failed and one case that already worked. Do not try to make the skill complete. Let real use decide what it needs next.
In step 4, you’ll take the version that survived these runs and remove anything that still is not earning its place.