Dreams, Reflections, and Inceptions

I taught an agent to speedrun Pokémon Red. The original story follows the project from its first button press through 1,000 turns. Viridian Forest is where it kept falling apart.

The agent could move. It could battle. It could make it into the forest. Then it would reach the same patch of trees and lose all confidence. It confused two kinds of boundaries: tree stumps were real blockers, but the edge of the screen was not. One run discovered part of the path. The next run started over like that discovery had never happened.

That was the useful part of the demo.

I was not trying to build the world’s best Pokémon player. I was trying to understand what an agent needs in order to get better from experience. Every attempt produced a high-fidelity trace: the prompts, model responses, tool calls, button presses, observations, failures, and recovery attempts. I had the whole record.

The record was not the same thing as memory.

That is where dreams started for me.

A trace knows what happened

The raw material is the trace. tapes records the session at high fidelity, including the parts most agent interfaces hide. For the Pokémon runner, that meant I could inspect what the model saw, what it tried, where it moved, and what happened next.

The model was not navigating a live game. After each turn I captured a screenshot and fed that image back into the next turn. The agent saw a sequence of still frames, not continuous movement. That created a specific perception problem: it often treated the edge of the screenshot as the edge of the playable world. To the model, the viewport boundary looked like a wall.

This matters because a summary written after the fact usually preserves the punchline and throws away the path. “The agent got stuck in Viridian Forest” is easy to understand, but it is not enough to improve the next attempt. Which obstacle confused it? What did it try first? Did the recovery work? Was the model reasoning badly, or was the state representation unclear?

A trace can answer those questions. But after ten runs, I did not have one answer. I had ten long transcripts.

High fidelity solves the capture problem. It does not solve the meaning problem.

Dreams are labels

The word “dream” makes this sound more mysterious than it is. The useful implementation starts with a boring operation: label the trace.

One section of a Pokémon run might become an observation like this:

{
  "type": "perception_anomaly",
  "location": "viridian_forest",
  "observation": "the model repeatedly treats the screen edge as a wall",
  "cause": "navigation is inferred from one screenshot per turn",
  "recovery": "cross the viewport boundary and inspect the next frame"
}

The label is not a new event. It is an interpretation attached to events that already happened. The trace remains the source of truth. The label makes a meaningful moment searchable, comparable, and reusable.

This is the grounding that matters. A dream should not be an agent inventing a lesson because it sounds plausible. It should be an agent reading recorded experience, identifying important moments, and preserving the connection back to the sessions that support the lesson.

That distinction is the difference between a story about what your agent did and an artifact you can inspect.

Reflect before you dream

Before I let the system dream across sessions, each session gets a reflection.

A reflection is the note you write at the end of the day before going to sleep. It reduces one session into the observations that may matter later: the decision that worked, the loop that wasted time, the obstacle that appeared more than once, the recovery that finally succeeded.

The reflection does not need to preserve every token because the trace already does that. Its job is to point back into the trace with enough structure that another agent can find the important parts without rereading the entire run.

Speed run through Viridian Forest

No statusClaude Opus 5019ff1a2Aug 10, 9:23 AM

1. youSpeed run through Viridian Forest.
   agentLoaded the forest speedrun skill, checked the ROM and world-map state, and launched the runner from a pristine map.
2. youWhat happened to the run?
   agentIt appeared stuck after 8,000 turns. The trace shows repeated up/down decisions, but the captured events confirm it crossed the viewport boundary and exited the forest on map 37.

For the Pokémon demo, a reflection could say that the run reached Viridian Forest, correctly recognized tree stumps as obstacles, but repeatedly misclassified the edge of the screenshot as another obstacle. On its own, that is one run’s experience. Across many runs, it becomes evidence.

A dream reads across those reflections. It can merge duplicate observations, notice contradictions, and surface a pattern that no single session proved by itself.

Anthropic first shared this concept with me under the name Dreams. This earlier Claude Code demo captures that original framing. Claude Code has since moved away from the name; its current documentation describes auto memory, where Claude accumulates learnings and patterns as it works. Anthropic still uses Dreams for a separate managed-agents research preview that reflects on past sessions and reorganizes an existing memory store.

Another way to think about dreams is observational memory. Mastra uses an Observer to compress older conversation into a dense log of observations, then a Reflector to reorganize those observations as they grow. The language is different, but the shape is familiar: preserve the record, extract what mattered, and turn it into memory the next run can use.

I wanted that primitive for the traces I was already recording, so I built it on tapes.

The layers are simple. The trace preserves what happened. The reflection labels what mattered in one session. The dream reasons across the labels.

The game made the abstraction visible

Pokémon was useful because the feedback was impossible to hand-wave. The agent either made it through the forest or it did not. The anomaly detector could flag repeated movement failures near the same part of the viewport. A dream could then read across those anomalies and name the underlying pattern: the agent was confusing the boundary of an image with the boundary of the map. If the dream claimed a navigation strategy fixed that mistake, I could replay the game and check.

It also exposed lessons outside the game itself. I was using expensive models because the task looked hard. The traces showed that some parts of the run did not need the strongest model. A smaller model could handle repeated navigation once the observations were clear. The valuable artifact was not “use a cheaper model.” It was the evidence showing which work could move to the cheaper model without making the run worse.

This is why I do not think dreams are only a memory feature. They are a way to turn session history into decisions that can be tested.

The same pattern transferred to my actual work. I am not the best person on my team at setting up staging environments. Other engineers have cleaner sessions than mine. When those sessions are captured, reflected on, and labeled, an agent can discover the approach the team already proved instead of repeating my mistakes.

Dreams need receipts

There is an obvious failure mode here. Ask a model to summarize enough sessions and it will produce advice that sounds wise. Some of it will even be right. That is not enough.

Every observation needs a receipt: the session, turn, tool result, or outcome that caused the label to exist. If two reflections disagree, the dream should preserve the disagreement until the trace resolves it. If a recommendation came from one lucky run, the system should know it has one example, not a law.

This makes the output reviewable. I can open the dream, inspect an observation, follow it back to the trace, and decide whether it deserves to influence another session. The language can be poetic. The data model cannot be.

Dreams need inceptions

A dream finds a lesson. An inception tests what to do with it.

Take a previous session and replay it. Keep the task fixed, then run it with low, medium, and high reasoning. Compare the outcome, cost, and time.

For Pokémon, that means replaying the same part of Viridian Forest with the dream’s navigation lesson. The question is no longer whether the lesson sounds right. The question is what changes when the agent can use it.

If low reasoning reaches the same outcome as high reasoning, use the more efficient path. If the lesson keeps working when you swap models, it does not belong to one model. If it survives a different harness, it does not belong to one tool. It has become part of the workflow.

That is the whole idea. Dreams tell you what might matter. Inceptions replay the evidence until you know what is actionable.

That is the reveal hidden inside the Pokémon demo. The goal was never to make an agent sit around and dream. The goal was to build a feedback loop with evidence at every layer:

• 1.Capture the trace. Preserve what the agent actually did.
• 2.Write the reflection. Label what mattered in one session.
• 3.Run the dream. Find patterns across sessions.
• 4.Run the inception. Replay the session and compare the outcomes.

Capture gives the agent a past. Reflections make that past readable. Dreams find the lesson. Inceptions make it actionable.

It started with an agent walking into a tree stump. What I found on the other side was a way for agents to learn from the work they already did.

Put your dreams on paper

Capture your first agent session with the paperctl CLI.