Your Team's Knowledge Is Hiding in Sessions Nobody Reopens

We built the biggest knowledge store our team has ever had, and then kept working as if it wasn’t there.

Over two months, the Paper Compute team ran 766 real agent sessions. Those sessions captured the commands that worked, the approaches that failed, the weird environment fixes, the deployment calls, the debugging trails, and the reasoning that never made it into the final PR.

We used to lose most of that, because nobody opens old sessions. Only 29 pieces of that work were promoted into reusable skills, leaving 737 sessions sitting in the store.

It’s not a capture problem. It’s the problem of what to do with the knowledge once it’s captured. Should the other 737 sessions become skills? Definitely not.

That is the part of agentic development we haven’t adjusted to yet. Work has changed where it leaves its receipts. An agent spins up the environment, chases the bug, ships the fix, and the whole hard-won path lands in a session now, instead of a person’s head or a review thread where the rest of the team would once have caught it.

Our habits haven’t changed though. We’re still hunting for knowledge in the old places, pairing and PRs and the desk next to us, while the real record piles up in a store we filled by accident and rarely open.

So the surprise here isn’t a loss. It’s an asset. Everything the team learned in those two months is still sitting there, findable, waiting for someone to use it.

I read 100 sessions to find out what was worth keeping

I went through the last 100 sessions and graded each on the question:

Does this contain something a teammate would want to use again?

Seventy did. That might sound like an argument for generating more skills, but it’s not. Those 100 sessions covered 13 distinct kinds of work, and the most common work was often the least reusable. Feature implementation was the largest single category, but only 7% of those contained something we’d want to repeat and capture. Debugging produced none.

What did produce reusable work:

• Spinning up a fresh environment: 5 of 6 sessions
• Tooling setup: 64%
• Cutting a release: 67%

I used to start with frequency as my signal for what to add as a skill because it felt like what the team did most should be turned into a repeatable skill.

But feature work happens constantly, and each feature is different. A release happens less often, but the shape stays mostly the same. That led to my finding that repeatable beats frequent.

Make skills from the work you do the same way every time.

One session is usually not a skill

More than half of the sessions worth promoting needed to be combined with other sessions first. One session gives you one pass through a task, including its wrong turns, constraints, and new explorations. When you use one session to create a skill, you’re more likely to capture those accidents.

Across 69 sessions, I found 68 near-overlapping procedures. That was not evidence that we needed 68 skills. It was evidence that the same few patterns were being rediscovered in slightly different forms.

Several sessions show you what stays stable. The repeated steps become the procedure; the incidental details fall away.

A useful skill has two properties:

1. The work has a fixed enough shape to repeat.
2. The moment to use it is obvious.

“Cut the release” has a trigger. “Implement PROJ-1842” does not. A ticket can teach us something, but it shouldn’t become part of the operating system just because it happened once.

Most knowledge should not become a skill

Just because something doesn’t become a skill, doesn’t mean that it shouldn’t be reused. Sometimes the reuse comes from recalling the knowledge:

• How did we fix staging auth last quarter?
• Which command worked around that dependency issue?
• Why did we reject the first deployment approach?

You can’t predict all of those questions, and nobody is going to write up every answer. That knowledge still matters. It just needs to be findable.

Other times you want behavior to repeat:

• Spin up the environment.
• Sync the docs.
• Cut the release.
• Ship the deploy.

Those jobs should not require a fresh investigation every time. They need a reviewed procedure with a clear trigger.

That gives us a much simpler model:

Search is for knowledge you might need again. Skills are for behavior you want repeated.

Search carries the long tail. Skills carry the small set of routines the organization has decided to run consistently.

The distinction matters because promoting everything creates a shelf nobody can navigate. Promoting nothing makes the team reconstruct its most common procedures forever.

The 60-second version

Here’s what this looks like in Paper.

Someone on the team says:

Didn’t somebody already figure out staging auth?

That sentence is already a query.

paperctl search "staging auth?"

The search returns the relevant turns from earlier sessions. The team can see the fix that worked, the approach that failed, and the constraint that made the final version necessary.

If this is a one-off question, we stop there, but let’s say staging auth comes up again and again. By the fourth search, we’re no longer looking at long-tail knowledge. We’re looking at a routine the team has failed to name.

That’s when we pull the related sessions together and generate a skill:

paperctl skill generate <session-ids>

Now the next person doesn’t have to remember the right query or reassemble the answer. The agent loads the reviewed procedure when the task appears.

That’s how search and skills feed each other. Search exposes repetition. Repetition earns promotion. Reuse tells us whether the promotion was right.

A skill people reach for has earned its place. A skill nobody uses was probably long-tail work that belonged in search all along.

What the session store is actually for

I started this analysis thinking we had a knowledge storage problem.

We don’t.

We save plenty. In two months, we saved more than 766 detailed records of how our team worked. Seventy percent of the 100 I read carried knowledge someone could use again.

The problem is deciding what form that knowledge should take.

Most sessions should remain searchable history: available when the strange bug returns, when someone asks why a decision was made, or when an agent is about to re-derive a fix the team already found.

A much smaller set should become skills: the environment setup, the release process, the doc sync, the deploy—work with a stable shape and an obvious trigger.

Capturing agent work gives you a record. It does not automatically give you organizational knowledge. That only happens when you decide what deserves to be recalled, what deserves to be repeated, and what deserves to become part of how the organization works.