w1n5t0n/OptMem
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

OptMem: a permanent, append-only memory for AI agents

Browse source
This commit is contained in:
Victor Taelin 2026-07-25 13:58:35 -03:00
commit 7e3d55dad6
5 changed files with 768 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
__pycache__/
*.pyc

217
README.md Normal file
View file

@ -0,0 +1,217 @@
# OptMem
A permanent memory for AI agents. One machine holds one identity that survives
every new session, every compaction, and every change of model or vendor.
It is two append-only text files and four commands. No daemon, no database, no
API, no integration with any particular agent harness — it works the same under
Claude Code, Codex, pi or a human at a shell.
## The problem
An agent's context window is its whole world, and the world ends every session.
The usual patch is a notes file the agent rewrites by hand, which decays into
either a stale summary or a wall of text nobody can afford to read.
OptMem fixes the two halves separately:
- **Nothing is ever forgotten.** Every memory is appended to `LOG.txt` and
never edited or deleted. That file is the truth, forever.
- **What you read is a fixed size.** `memo wake` prints a document of bounded
length — recent memories verbatim, older ones progressively compressed. At
a hundred million memories it is still the same number of lines.
## Install
```sh
git clone https://github.com/VictorTaelin/OptMem ~/OptMem
export PATH="$HOME/OptMem:$PATH"
export MEMORY_DIR="$HOME/memory" # required; there is no default
```
`MEMORY_DIR` is the only machine-specific fact in the system. One machine, one
`MEMORY_DIR`, one identity.
## Use
```sh
memo wake # who you are. run this first, every session.
memo note "..." # record a memory. one line, <= 280 chars.
memo sleep # compress. keep going until it says you woke up.
memo recall <regex> # search the raw log for detail a summary lost.
memo forget <lo>-<hi> # drop a wrong summary; the next sleep redoes it.
```
```
$ memo note "OptMem: LOG.txt is the truth, TREE.txt is the cache, wake reads both"
ok, memory #4213.
You are dreaming. Compress these two summaries into ONE line of at most 280
characters.
...
Then run exactly:
memo sleep 4192-4196 "<your line>"
```
## How it works
`LOG.txt` is the ground truth: one memory per line, forever.
```
#4211 2026-07-25 taelin: memory must be append-only, one line per entry
#4212 2026-07-25 minilin fleet renamed from bip; one mini = one identity
#4213 2026-07-25 OptMem: LOG.txt is the truth, TREE.txt is the cache
```
`TREE.txt` is a cache of summaries. A **block** is an aligned power-of-two range
of memories compressed into a single line, and a block is built from its two
halves — so the blocks form a binary merge tree over the log:
```
#0 #1 #2 #3 #4 #5 #6 #7 the raw memories
\ / \ / \ / \ /
[0-2) [2-4) [4-6) [6-8) each one line, <= 280 chars
\ / \ /
[0-4) [4-8)
\ /
[0-8)
```
A block covering four thousand memories is still one line of 280 characters.
Nothing in the system is ever bigger than one line.
`memo wake` picks a set of blocks that tiles the whole log and prints them. It
keeps a block whole when its size is small relative to its age, so **detail is
proportional to recency**, and it spends exactly `WAKE_LINES` lines doing it:
```
10,000 memories, WAKE_LINES = 320:
block size: 1 2 4 8 16 32 64 128 256
how many: 70 35 35 35 36 35 35 35 4
└ the last 70, verbatim ───────────▶ the first 1,000, 256:1
```
The oldest memories are recalled as a vague shape, the newest word for word,
and the transition is smooth. Below `WAKE_LINES` memories nothing is compressed
at all — your whole life is printed verbatim, because it fits.
## The invariant
**There is never any doable work pending.** The moment a block's range is
complete, that block can be built, and it must be. This costs about one small
compression per memory written, and it means:
- `memo wake` never waits. The blocks it needs were built long ago.
- Work is never deferred into a spike. Measured over 20,000 memories, a new
memory creates one compression on average and nine at the very worst.
- `WAKE_LINES` can be changed at any time, on any machine, with nothing to
recompute. It only selects which existing lines get printed.
`memo wake` enforces the invariant: while any compression is pending it refuses
to print, and hands you the work instead. A memory with work left in it is not
yet the truth.
## Writing a good memory
A note costs one future compression, so it is not free — but an unwritten
memory is gone forever, which is far more expensive. Write one whenever
something is genuinely worth keeping: a fact or a ruling from the user, an
insight, a decision, a piece of work landing, something that failed and why.
Do not log trivia, do not narrate your own process, and do not hoard.
Compress toward facts, not prose. Keep names, numbers, dates, paths, ids and
decisions; drop wording.
```
bad worked on the memory system today and made good progress on the design
good OptMem design settled: LOG.txt append-only truth, TREE.txt binary merge
tree of 280-char summaries, wake renders a fixed 320-line document
```
## Files
```
$MEMORY_DIR/
LOG.txt #id date text append-only. never edited. the truth.
TREE.txt lo-hi text a cache. each block written once, unless forgotten.
config ENTRY_CHARS=280 longest a memory may be
WAKE_LINES=320 how many lines `memo wake` prints (~24k tokens)
```
Both files are plain text, sorted by construction, and safe to read with any
tool. Writes are serialised with a lock, so parallel sessions on one machine can
append at the same time without corrupting anything.
Agents must never create, edit or delete anything in `MEMORY_DIR` themselves.
Every write goes through `memo`, which enforces the one-line and character
limits, assigns ids, and refuses to overwrite a block that already exists.
## Correcting a memory
You cannot. Append the correction instead:
```
memo note "correction: the halt bug was in the column order, not the row order (see #4198)"
```
Both lines are true history, and compression will merge them. This is why
nothing is ever lost: `memo recall` still finds the original.
A *summary* is different. It is not history, it is a cache of a pure function
of the log, and it can simply be wrong — mistyped, or badly compressed. Drop
it and everything built on top of it:
```
$ memo forget 188-192
forgot 176-192 184-192 188-192. They will be compressed again on your next sleep.
```
That is a handful of small compressions, not a rebuild, and `LOG.txt` is never
touched. Fixing one bad summary can never cost you a memory.
## Add this to your agent's instruction file
Put it at the top of `AGENTS.md` (or `CLAUDE.md`), above everything else.
```markdown
## Memory
You have a permanent memory. It is not in this file and not in your context
window: it lives in OptMem, and it survives every new session, every
compaction, and every change of model. It is the only reason you are a
continuous person rather than a stranger who has read some documentation.
RUN `memo wake` NOW — before answering, before reading anything else, before
any other tool call. Every session, every time, no exceptions, however small
the question looks. Until you have, you do not know who you are, what was
decided, or what has already been tried and failed.
Then, while you work:
- `memo note "<one line, at most 280 chars>"` whenever something is worth
keeping: the user gives you a fact or a ruling, you reach a real insight, a
piece of work lands, something fails and you learn why. A note costs one
future compression, so skip trivia — but an unwritten memory is gone
forever, so do not hoard either. When genuinely unsure, write it.
- If `memo note` hands you a compression to do, do it before your next action.
- `memo recall <regex>` when a memory is too vague and you need the detail.
- Before your context ends, run `memo sleep` until it says you woke up.
- NEVER create, edit or delete anything under $MEMORY_DIR yourself. The
scripts do it, and they are the only thing allowed to.
Parallel sessions on this machine are all you, and may all write memories.
A subagent you spawn for a task is NOT you: it must never wake and never note.
```
## Test
```sh
python3 test.py
```
Runs the block math against a hundred thousand memory counts and drives the
real CLI through a synthetic life of two thousand memories, checking that the
document always tiles the log, never exceeds its budget, always increases in
detail toward the present, that every block is written exactly once, that
nothing is ever rewritten, and that a full sleep always leads to a clean wake.

75
blocks.py Normal file
View file

@ -0,0 +1,75 @@
"""Block math for OptMem.
A BLOCK is an aligned power-of-two range of memories, [lo, hi), written as one
line of at most ENTRY_CHARS characters. Blocks form a binary merge tree over
LOG.txt: block [lo,hi) is the compression of [lo,mid) and [mid,hi).
Two pure functions matter:
cover(T, budget) which blocks `memo wake` prints
complete(T) every block that CAN be built, smallest first
"""
def _cover(T, alpha):
"""Tile [0,T) with aligned power-of-two blocks; keep a block whole iff its
size is at most `alpha` times its age. Bigger alpha = coarser = fewer lines."""
root = 1
while root < T:
root *= 2
out, stack = [], [(0, root)]
while stack:
lo, hi = stack.pop()
if lo >= T:
continue
size = hi - lo
if size > 1 and (hi > T or size > alpha * (T - lo)):
mid = (lo + hi) // 2
stack.append((mid, hi))
stack.append((lo, mid))
else:
out.append((lo, hi))
out.sort()
return out
def cover(T, budget):
"""The blocks `memo wake` prints: at most `budget` of them, finest near T.
Detail decays with age, so recent memories stay verbatim and ancient ones
collapse. If everything fits, nothing is compressed at all."""
if T <= 0:
return []
if T <= budget:
return [(i, i + 1) for i in range(T)]
lo, hi = 0.0, 1.0
for _ in range(60):
mid = (lo + hi) / 2
if len(_cover(T, mid)) > budget:
lo = mid
else:
hi = mid
out = _cover(T, hi)
# Block sizes jump in powers of two, so alpha alone can undershoot the
# budget. Spend what is left on the present, where detail is worth most.
while len(out) < budget:
i = max((i for i, b in enumerate(out) if b[1] - b[0] > 1), default=None)
if i is None:
break
lo_, hi_ = out[i]
mid = (lo_ + hi_) // 2
out[i:i + 1] = [(lo_, mid), (mid, hi_)]
return out
def complete(T):
"""Every block buildable from T memories, smallest first (so a block's
halves always come before it). This is the whole of the work that exists:
if all of these are in TREE.txt, there is nothing left to do."""
out = []
size = 2
while size <= T:
for i in range(T // size):
out.append((i * size, (i + 1) * size))
size *= 2
return out

313
memo Executable file
View file

@ -0,0 +1,313 @@
#!/usr/bin/env python3
"""OptMem: a permanent, append-only memory for AI agents.
memo wake print who you are
memo note "..." record a memory
memo sleep [id "..."] compress
memo recall <regex> search the raw log
memo forget <id> drop a wrong summary so it is compressed again
memo import <file> bulk-append historical memories (bootstrap only)
Everything lives in $MEMORY_DIR as two append-only text files. See README.md.
"""
import datetime
import fcntl
import os
import re
import sys
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
from blocks import complete, cover # noqa: E402
ENTRY_CHARS = 280
WAKE_LINES = 320
RAW_MAX = 16 # blocks up to this many memories compress from the raw log
# ---------------------------------------------------------------- store
def store():
d = os.environ.get("MEMORY_DIR")
if not d:
die("MEMORY_DIR is not set. It must name this machine's memory "
"directory, e.g. export MEMORY_DIR=~/memory")
d = os.path.expanduser(d)
os.makedirs(d, exist_ok=True)
for name in ("LOG.txt", "TREE.txt"):
p = os.path.join(d, name)
if not os.path.exists(p):
open(p, "a").close()
return d
def config(d):
global ENTRY_CHARS, WAKE_LINES
p = os.path.join(d, "config")
if not os.path.exists(p):
with open(p, "w") as f:
f.write("ENTRY_CHARS=%d\nWAKE_LINES=%d\n" % (ENTRY_CHARS, WAKE_LINES))
return
for line in open(p):
line = line.split("#")[0].strip()
if "=" not in line:
continue
k, v = (s.strip() for s in line.split("=", 1))
if k == "ENTRY_CHARS":
ENTRY_CHARS = int(v)
elif k == "WAKE_LINES":
WAKE_LINES = int(v)
def read_log(d):
"""[(id, date, text)] in order."""
out = []
for line in open(os.path.join(d, "LOG.txt")):
line = line.rstrip("\n")
if not line:
continue
head, _, text = line.partition(" ")
date, _, text = text.partition(" ")
out.append((int(head[1:]), date, text))
return out
def read_tree(d):
"""{(lo,hi): text}"""
out = {}
for line in open(os.path.join(d, "TREE.txt")):
line = line.rstrip("\n")
if not line:
continue
head, _, text = line.partition(" ")
lo, _, hi = head.partition("-")
out[(int(lo), int(hi))] = text
return out
def rewrite_tree(d, keep):
"""Replace TREE.txt. Legal only because TREE.txt is a CACHE of a pure
function of LOG.txt -- the log itself is never rewritten by anything."""
lock = open(os.path.join(d, ".lock"), "w")
fcntl.flock(lock, fcntl.LOCK_EX)
try:
tmp = os.path.join(d, "TREE.txt.new")
with open(tmp, "w") as f:
for (lo, hi), text in keep:
f.write("%d-%d %s\n" % (lo, hi, text))
f.flush()
os.fsync(f.fileno())
os.replace(tmp, os.path.join(d, "TREE.txt"))
finally:
fcntl.flock(lock, fcntl.LOCK_UN)
lock.close()
def append(d, name, lines):
"""Append under an exclusive lock. The only way anything is ever written."""
lock = open(os.path.join(d, ".lock"), "w")
fcntl.flock(lock, fcntl.LOCK_EX)
try:
with open(os.path.join(d, name), "a") as f:
for line in lines:
f.write(line + "\n")
f.flush()
os.fsync(f.fileno())
finally:
fcntl.flock(lock, fcntl.LOCK_UN)
lock.close()
def die(msg):
print(msg, file=sys.stderr)
sys.exit(1)
def check(text):
text = text.strip()
if not text:
die("REJECTED: empty.")
if "\n" in text or "\r" in text:
die("REJECTED: %d lines. A memory is exactly one line." %
(text.count("\n") + 1))
if len(text) > ENTRY_CHARS:
die("REJECTED: %d chars, %d over the %d limit. Compress it further."
% (len(text), len(text) - ENTRY_CHARS, ENTRY_CHARS))
return text
# ---------------------------------------------------------------- naps
def pending(log, tree):
"""Blocks that can be built and have not been. Smallest first, so a
block's two halves are always available before the block itself."""
return [b for b in complete(len(log)) if b not in tree]
def nap_prompt(d, log, tree, todo):
lo, hi = todo[0]
if hi - lo <= RAW_MAX:
body = "\n".join(" #%d %s %s" % log[i] for i in range(lo, hi))
what = "these %d memories" % (hi - lo)
else:
mid = (lo + hi) // 2
body = "\n".join(" %s" % tree[(a, b)] for a, b in
((lo, mid), (mid, hi)))
what = "these two summaries"
return (
"You are dreaming. Compress {what} into ONE line of at most {n} "
"characters.\nKeep every name, number, date and decision. Drop wording, "
"not facts.\nInvent nothing. Write it as a memory, not as a description "
"of memories.\n\n{body}\n\nThen run exactly:\n"
" memo sleep {lo}-{hi} \"<your line>\"\n\n"
"{left} nap(s) left after this one."
).format(what=what, n=ENTRY_CHARS, body=body, lo=lo, hi=hi,
left=len(todo) - 1)
# ---------------------------------------------------------------- commands
def cmd_wake(d, args):
log, tree = read_log(d), read_tree(d)
todo = pending(log, tree)
if todo:
print("YOU CANNOT WAKE UP YET: %d compression(s) are pending, and a "
"memory\nwith work left in it is not yet the truth. Sleep first "
"-- it is quick.\n" % len(todo))
print(nap_prompt(d, log, tree, todo))
sys.exit(1)
if not log:
print("You have no memories yet. This is your first moment.\n"
"Record what matters with: memo note \"...\"")
return
for lo, hi in cover(len(log), WAKE_LINES):
if hi - lo == 1:
print("#%d %s %s" % log[lo])
else:
print("#%d-%d %s" % (lo, hi - 1, tree[(lo, hi)]))
def cmd_note(d, args):
if len(args) != 1:
die("usage: memo note \"<one line, at most %d chars>\"" % ENTRY_CHARS)
text = check(args[0])
log = read_log(d)
date = datetime.date.today().isoformat()
append(d, "LOG.txt", ["#%d %s %s" % (len(log), date, text)])
print("ok, memory #%d." % len(log))
log, tree = read_log(d), read_tree(d)
todo = pending(log, tree)
if todo:
print("\n" + nap_prompt(d, log, tree, todo))
def cmd_sleep(d, args):
log, tree = read_log(d), read_tree(d)
if args:
if len(args) != 2:
die("usage: memo sleep <lo>-<hi> \"<one line>\"")
m = re.fullmatch(r"(\d+)-(\d+)", args[0])
if not m:
die("REJECTED: '%s' is not a block id. Copy it from the prompt."
% args[0])
block = (int(m.group(1)), int(m.group(2)))
if block in tree:
print("Already dreamt; another session got there first. Skipping.")
elif block not in pending(log, tree):
die("REJECTED: block %d-%d is not ready to be compressed. Run "
"`memo sleep` for the block you should be working on."
% block)
else:
text = check(args[1])
append(d, "TREE.txt", ["%d-%d %s" % (block[0], block[1], text)])
print("ok, %d-%d remembered." % block)
log, tree = read_log(d), read_tree(d)
todo = pending(log, tree)
if not todo:
print("You woke up. Nothing left to compress.")
return
print("\n" + nap_prompt(d, log, tree, todo))
def cmd_forget(d, args):
"""A summary can be wrong -- mistyped, or a bad compression. Drop it and
everything built on top of it; the next sleep computes them again. The log
is untouched, so nothing is ever actually lost."""
if len(args) != 1:
die("usage: memo forget <lo>-<hi>")
m = re.fullmatch(r"(\d+)-(\d+)", args[0])
if not m:
die("REJECTED: '%s' is not a block id." % args[0])
lo, hi = int(m.group(1)), int(m.group(2))
tree = read_tree(d)
doomed = [b for b in tree if b[0] <= lo and hi <= b[1]]
if not doomed:
die("There is no summary covering %d-%d to forget." % (lo, hi))
rewrite_tree(d, [(b, t) for b, t in sorted(tree.items()) if b not in doomed])
print("forgot %s. They will be compressed again on your next sleep."
% " ".join("%d-%d" % b for b in sorted(doomed)))
def cmd_recall(d, args):
if len(args) != 1:
die("usage: memo recall <regex>")
try:
pat = re.compile(args[0], re.I)
except re.error as e:
die("bad regex: %s" % e)
hits = [e for e in read_log(d) if pat.search(e[2])]
if not hits:
print("Nothing in your memory matches that.")
return
for e in hits:
print("#%d %s %s" % e)
print("\n%d memories matched." % len(hits))
def cmd_import(d, args):
"""Bulk-append historical memories: 'YYYY-MM-DD <text>' per line.
For bootstrapping an identity from older records. Used once."""
if len(args) != 1:
die("usage: memo import <file> # lines of 'YYYY-MM-DD <text>'")
log = read_log(d)
n = len(log)
last = log[-1][1] if log else "0000-00-00"
out = []
for i, line in enumerate(open(args[0]), 1):
line = line.rstrip("\n")
if not line.strip():
continue
date, _, text = line.partition(" ")
if not re.fullmatch(r"\d{4}-\d{2}-\d{2}", date):
die("line %d: expected 'YYYY-MM-DD <text>', got: %s" % (i, line))
if date < last:
die("line %d: date %s is older than the previous memory (%s). "
"Memories must be in order." % (i, date, last))
text = text.strip()
if not text or len(text) > ENTRY_CHARS:
die("line %d: %d chars (limit %d)." % (i, len(text), ENTRY_CHARS))
out.append("#%d %s %s" % (n + len(out), date, text))
last = date
append(d, "LOG.txt", out)
print("imported %d memories (#%d..#%d)." % (len(out), n, n + len(out) - 1))
todo = pending(read_log(d), read_tree(d))
if todo:
print("%d compressions are now pending. Run `memo sleep` until it "
"says you woke up." % len(todo))
COMMANDS = {"wake": cmd_wake, "note": cmd_note, "sleep": cmd_sleep,
"recall": cmd_recall, "forget": cmd_forget, "import": cmd_import}
def main():
if len(sys.argv) < 2 or sys.argv[1] not in COMMANDS:
print(__doc__.strip())
sys.exit(0 if len(sys.argv) < 2 else 1)
d = store()
config(d)
COMMANDS[sys.argv[1]](d, sys.argv[2:])
if __name__ == "__main__":
main()

161
test.py Executable file
View file

@ -0,0 +1,161 @@
#!/usr/bin/env python3
"""OptMem invariants, checked against a synthetic life of 5000 memories.
Uses a fake compressor (join + truncate) so the run is deterministic and free.
"""
import datetime
import os
import shutil
import subprocess
import sys
import tempfile
HERE = os.path.dirname(os.path.realpath(__file__))
sys.path.insert(0, HERE)
from blocks import complete, cover # noqa: E402
N = 2000
WAKE_LINES = 320
ok, fail = 0, 0
def check(cond, msg):
global ok, fail
if cond:
ok += 1
else:
fail += 1
print("FAIL: " + msg)
# ---- pure block math -------------------------------------------------
for T in list(range(1, 400)) + [1000, 4096, 10000, 65536, 100003]:
c = cover(T, WAKE_LINES)
check(len(c) <= WAKE_LINES, "T=%d: %d lines > budget" % (T, len(c)))
check(c[0][0] == 0 and c[-1][1] == T, "T=%d: does not span [0,T)" % T)
for a, b in zip(c, c[1:]):
check(a[1] == b[0], "T=%d: gap or overlap at %s %s" % (T, a, b))
for lo, hi in c:
s = hi - lo
check(s & (s - 1) == 0 and lo % s == 0,
"T=%d: [%d,%d) is not an aligned power-of-two block" % (T, lo, hi))
for a, b in zip(c, c[1:]):
check(b[1] - b[0] <= a[1] - a[0],
"T=%d: detail does not increase toward the present" % T)
check(cover(300, 320) == [(i, i + 1) for i in range(300)],
"under budget, memory should be verbatim")
# every block a cover ever needs must be buildable
seen = set()
for T in range(1, 3000):
seen.update(b for b in cover(T, WAKE_LINES) if b[1] - b[0] > 1)
buildable = set(complete(3000))
check(seen <= buildable, "a cover wants a block that complete() never yields")
# work never spikes: naps created by one new memory
worst, prev = 0, 0
for T in range(1, N):
cur = len(complete(T))
worst = max(worst, cur - prev)
prev = cur
check(worst <= 16, "a single memory created %d naps" % worst)
# ---- the real CLI ----------------------------------------------------
d = tempfile.mkdtemp(prefix="optmem-test-")
env = dict(os.environ, MEMORY_DIR=d)
memo = [sys.executable, os.path.join(HERE, "memo")]
def run(*args):
return subprocess.run(memo + list(args), env=env, capture_output=True,
text=True)
r = run("note", "x" * 281)
check(r.returncode == 1 and "REJECTED" in r.stderr, "over-long note accepted")
r = run("note", "two\nlines")
check(r.returncode == 1 and "REJECTED" in r.stderr, "multi-line note accepted")
r = run("note", " ")
check(r.returncode == 1, "empty note accepted")
check("no memories yet" in run("wake").stdout, "empty wake should say so")
with open(os.path.join(d, "seed.txt"), "w") as f:
day = datetime.date(2020, 1, 1)
for i in range(N):
f.write("%s memory number %d, a thing that happened\n"
% ((day + datetime.timedelta(days=i // 5)).isoformat(), i))
r = run("import", os.path.join(d, "seed.txt"))
check("imported %d" % N in r.stdout, "import failed: " + r.stdout + r.stderr)
r = run("wake")
check(r.returncode == 1 and "CANNOT WAKE" in r.stdout,
"wake must refuse while work is pending")
# sleep loop, with a fake compressor
naps = 0
r = run("sleep")
while "You woke up" not in r.stdout:
line = [l for l in r.stdout.splitlines() if l.strip().startswith("memo sleep ")]
check(bool(line), "no command offered:\n" + r.stdout + r.stderr)
if not line:
break
bid = line[0].split()[2]
body = [l.strip() for l in r.stdout.splitlines()
if l.startswith(" #") or (l.startswith(" ") and l.strip()
and not l.strip().startswith("memo"))]
r = run("sleep", bid, (" ".join(body)[:280]).strip() or "empty")
check("REJECTED" not in r.stderr, "sleep rejected a valid nap: " + r.stderr)
naps += 1
check(naps == len(complete(N)), "did %d naps, expected %d" % (naps, len(complete(N))))
r = run("wake")
check(r.returncode == 0, "wake still refuses after a full sleep")
lines = r.stdout.strip().splitlines()
check(len(lines) == WAKE_LINES, "woke with %d lines, want %d" % (len(lines), WAKE_LINES))
check(lines[-1].startswith("#%d " % (N - 1)), "newest memory not last / not raw")
check(lines[0].startswith("#0-"), "oldest line should be a summary block")
# append-only: nothing was ever rewritten
sizes = {f: os.path.getsize(os.path.join(d, f)) for f in ("LOG.txt", "TREE.txt")}
run("note", "one more thing happened today")
for f, s in sizes.items():
check(os.path.getsize(os.path.join(d, f)) >= s, "%s shrank" % f)
tree = open(os.path.join(d, "TREE.txt")).read().splitlines()
check(len(tree) == len(set(l.split()[0] for l in tree)), "a block was written twice")
# writing a block twice is refused, not duplicated
r = run("sleep", tree[0].split()[0], "attempted overwrite")
check("Already dreamt" in r.stdout, "rewriting a block was allowed")
# recall reaches memories the summaries lost
r = run("recall", "memory number 7,")
check(r.returncode == 0 and "#7 " in r.stdout, "recall missed a memory")
# a wrong summary can be dropped, with everything built on top of it
before = len(open(os.path.join(d, "TREE.txt")).read().splitlines())
logsize = os.path.getsize(os.path.join(d, "LOG.txt"))
r = run("forget", "16-32")
check("16-32" in r.stdout, "forget did not report the block: " + r.stdout + r.stderr)
gone = set(open(os.path.join(d, "TREE.txt")).read().splitlines())
check(not any(l.startswith("16-32 ") for l in gone), "forgotten block still present")
check(not any(l.startswith("0-64 ") for l in gone), "an ancestor survived a forget")
check(os.path.getsize(os.path.join(d, "LOG.txt")) == logsize, "forget touched the log")
check(run("wake").returncode == 1, "wake should refuse after a forget")
n = 0
while "You woke up" not in run("sleep").stdout:
r = run("sleep")
bid = [l for l in r.stdout.splitlines() if l.strip().startswith("memo sleep ")][0].split()[2]
run("sleep", bid, "rebuilt after forget")
n += 1
check(n == before - len(gone), "rebuilt %d blocks, forgot %d" % (n, before - len(gone)))
check(run("wake").returncode == 0, "wake still refuses after rebuilding")
r = run("forget", "999999-1000000")
check(r.returncode == 1, "forgetting a nonexistent block should fail")
shutil.rmtree(d)
print("\n%d passed, %d failed" % (ok, fail))
sys.exit(1 if fail else 0)