An agent forgets everything between sessions. So I built the memory outside it. Twelve weeks in, the loop has caught 269 mistakes and broken itself twice, both times without saying a word.
The hard part of working with an AI agent day to day is not that it makes mistakes. It is that it makes a mistake, you correct it, and the correction evaporates when the session ends. The next session starts from zero and is perfectly willing to make the same mistake again.
So the memory has to live outside the agent, in something durable it reads on the way in. That is a plumbing problem more than an AI problem, and it has the failure modes of plumbing.
One markdown file. Every entry is dated, states the mistake, and ends with a rule phrased so it can be checked later. There is no reflection in it. The format is closer to a lint rule than a diary.
Three parts. Each one has failed independently.
Two rules about the store turned out to matter more than they look.
Several routines cite the log. Exactly one appends to it. When I checked this I found I had been telling myself three different skills wrote entries, which was wrong, and would have made the write path impossible to reason about the first time something went missing.
The instinct with an append-only file is to archive the old end of it. For this file that is exactly backwards. A lesson from May is not less binding than one from July, and the whole point is that the agent has no sense of which mistakes are recent. Splitting by date would quietly demote two thirds of the rules to somewhere nothing looks.
A feedback loop with a severed read path is a diary. That is not a hypothetical.
Neither failure produced an error. Both were found by going to look.
The first break came from a cleanup. I refactored the agent's always-loaded configuration to be leaner, and the instruction telling it to consult the log lived in the part I removed. Nothing errored. Sessions kept appending faithfully to a file that no session ever opened again.
The second is the one I find genuinely funny. A header had drifted into the middle of the file, and new entries were being appended below it rather than at the top. By the time I noticed, 192 of 261 entries, about three quarters of every lesson recorded, were sitting in a region that a top-down read never reached. The file looked healthy. It had the right number of lines. Opening it showed recent, relevant entries. The missing 74% were just further down than anyone looked.
Both were repaired the same way: merge everything into one list, write down the invariant that there is one list and no second section, and move the read instruction somewhere the refactor could not casually delete. The merge recovered all 260 entries with nothing lost.
Two numbers say more than the entry count does.
Corrections per day appears to rise sharply. Per session it still rises, by 25%. Per unit of work actually done it moves 10% and goes down before it goes up. Same 269 events, three denominators, three different stories.
| month | corrections | sessions | per session | words logged | per 1k words |
|---|---|---|---|---|---|
| May | 88 | 161 | 0.547 | 82,391 | 1.068 |
| June | 72 | 117 | 0.615 | 70,555 | 1.020 |
| July | 109 | 160 | 0.681 | 97,148 | 1.122 |
Neither denominator is right on its own. Counting sessions treats a three minute one-shot task and a five hour sprawl as equal. Counting words fixes that but depends on session notes being written with consistent thoroughness, so a terse note about heavy work shrinks the denominator it should have grown. They are biased in different directions, which is the argument for showing both instead of picking the flattering one.
Before reading anything into a 25% climb it is worth asking whether a constant underlying rate would have produced these numbers anyway. Testing each month's count against what a fixed rate predicts:
| exposure | chi-square | p | verdict |
|---|---|---|---|
| per 1k words | 0.398 | 0.82 | no trend |
| per session | 2.370 | 0.31 | no trend |
Neither is close to significant. With around 90 corrections a month, random variation alone is plus or minus about 11%, so a real trend has to be larger than that to be visible in twelve weeks of data. The 25% climb that looked like a story is inside the noise floor once you account for the fact that counts of rare events are lumpy by nature.
What it does rule out is the least comfortable explanation, that the agent is simply making more mistakes than it used to. That would have shown up here, and it does not.
Comparing every entry against every other one on shared vocabulary, about 36,000 pairs, six came back near-identical. All six were the same entry present twice, confined to a single three day window in late June, with none before it and none in the 126 entries after. That is the fingerprint of the one time repair that reunified the fragmented file, which faithfully kept both copies of anything that had been written into both halves. The write path itself is clean. The duplicates have since been removed, which is why the count here is 269 rather than 275.
That is the finding I would stand behind. A declining rate was only ever plausible if the work stayed inside one domain, and it does not. What the absence of repeats shows is that corrections keep landing somewhere new instead of relitigating old ground. The loop is not making the work error-free. It is making each error survive being made once.
The most common words across all 269 rules are verify and check, which together touch about a sixth of them. The one that says the most is silently, the only adverb near the top. The dominant failure mode is not something breaking. It is something appearing to work.
I cannot measure what the loop prevents. There is no control version of the last twelve weeks without it, and as the denominator problem shows, the headline number moves for reasons that have nothing to do with whether the loop works. The evidence I trust is the absence of repeats, not the count.
The deeper limit is that none of this can see a mistake nobody caught. The log holds errors that were noticed. Sizing the ones that got past both of us would take a fresh audit of finished work, looking for defects that were never logged, which is the only honest way to measure the denominator and is a genuinely expensive thing to do.
There is a selection effect at the entrance, too. The log holds mistakes that got noticed. Some entries are ones the agent caught on its own before I saw them, which is the loop working well. The ones nobody caught are by definition absent, and I have no way to size that group.
The part I would defend without hedging is the shape. A dated mistake bound to a greppable rule beats a page of reflection, because the rule is the only part that ever gets read again. And any loop like this needs someone to periodically open it and confirm the parts are still connected, because when it comes apart it will not tell you.