w1n5t0n/OptMem
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

wake in parts: a single-shot dump is silently truncated by every harness

Browse source 
Measured: Codex CLI cuts tool output at 10 KiB or 256 lines, Claude Code at
30,000 chars, pi at 50 KB -- and each drops a different piece. A 320-line
memory is ~79 KB, so waking was losing memories everywhere, silently. Observed
live on pi: memories #0-#41, the compressed ancient past, vanished.

memo wake now pages the document into parts under the strictest cap and each
part names the exact command for the next. No harness is special-cased.
This commit is contained in:
Victor Taelin 2026-07-25 14:22:29 -03:00
parent 7e3d55dad6
commit e544f6548a
3 changed files with 94 additions and 10 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

24
README.md
View file

@ -35,7 +35,8 @@ export MEMORY_DIR="$HOME/memory" # required; there is no default
## Use
```sh
memo wake # who you are. run this first, every session.
memo wake # who you are. run this first, every session,
# then `memo wake 2`, `3`... until it says so.
memo note "..." # record a memory. one line, <= 280 chars.
memo sleep # compress. keep going until it says you woke up.
memo recall <regex> # search the raw log for detail a summary lost.
@ -129,6 +130,18 @@ good OptMem design settled: LOG.txt append-only truth, TREE.txt binary merge
tree of 280-char summaries, wake renders a fixed 320-line document
```
## Output is delivered in parts
Every harness truncates an over-long command, and each one drops a different
piece: Codex cuts at 10 KiB or 256 lines, Claude Code at 30,000 characters,
pi at 50 KB. A 320-line memory is ~79 KB, so a single-shot `memo wake` gets
mangled everywhere — and silently.
So `memo wake` pages the document into parts that fit the strictest of them
(`PART_CHARS`, `PART_LINES`), and each part ends by naming the exact command
for the next one. Nothing is ever dropped, and no harness is special-cased: if
yours is more generous, raise the two settings and get fewer parts.
## Files
```
@ -137,6 +150,8 @@ $MEMORY_DIR/
TREE.txt lo-hi text a cache. each block written once, unless forgotten.
config ENTRY_CHARS=280 longest a memory may be
WAKE_LINES=320 how many lines `memo wake` prints (~24k tokens)
PART_CHARS=8000 how much of it fits in one command's output
PART_LINES=200 ...and in how many lines
```
Both files are plain text, sorted by construction, and safe to read with any
@ -187,6 +202,13 @@ any other tool call. Every session, every time, no exceptions, however small
the question looks. Until you have, you do not know who you are, what was
decided, or what has already been tried and failed.
Your memory arrives in numbered parts, oldest first, because every agent
harness silently truncates a command that prints too much. `memo wake` gives
you part 1; you must then run `memo wake 2`, `memo wake 3`, and so on, until a
part tells you it was the last one. Stopping early is worse than not waking at
all: you would be holding a confident, detailed picture of your distant past
with no idea what happened recently.
Then, while you work:
- `memo note "<one line, at most 280 chars>"` whenever something is worth

59
memo
View file

@ -1,7 +1,7 @@
#!/usr/bin/env python3
"""OptMem: a permanent, append-only memory for AI agents.
memo wake print who you are
memo wake [part] print who you are
memo note "..." record a memory
memo sleep [id "..."] compress
memo recall <regex> search the raw log
@ -24,6 +24,13 @@ ENTRY_CHARS = 280
WAKE_LINES = 320
RAW_MAX = 16 # blocks up to this many memories compress from the raw log
# Every agent harness silently truncates a command that prints too much --
# Codex at 10 KiB or 256 lines, Claude Code at 30k chars, pi at 50 KB -- and
# each drops a different part. So the memory is handed over in parts that fit
# the strictest of them. These are transport limits, not memory limits.
PART_CHARS = 8000
PART_LINES = 200
# ---------------------------------------------------------------- store
@ -42,11 +49,12 @@ def store():
def config(d):
global ENTRY_CHARS, WAKE_LINES
global ENTRY_CHARS, WAKE_LINES, PART_CHARS, PART_LINES
p = os.path.join(d, "config")
if not os.path.exists(p):
with open(p, "w") as f:
f.write("ENTRY_CHARS=%d\nWAKE_LINES=%d\n" % (ENTRY_CHARS, WAKE_LINES))
f.write("ENTRY_CHARS=%d\nWAKE_LINES=%d\nPART_CHARS=%d\nPART_LINES=%d\n"
% (ENTRY_CHARS, WAKE_LINES, PART_CHARS, PART_LINES))
return
for line in open(p):
line = line.split("#")[0].strip()
@ -57,6 +65,10 @@ def config(d):
ENTRY_CHARS = int(v)
elif k == "WAKE_LINES":
WAKE_LINES = int(v)
elif k == "PART_CHARS":
PART_CHARS = int(v)
elif k == "PART_LINES":
PART_LINES = int(v)
def read_log(d):
@ -167,6 +179,20 @@ def nap_prompt(d, log, tree, todo):
# ---------------------------------------------------------------- commands
def paginate(lines):
"""Split the document into parts that survive any harness's output cap."""
parts, cur, size = [], [], 0
for line in lines:
if cur and (len(cur) >= PART_LINES or size + len(line) > PART_CHARS):
parts.append(cur)
cur, size = [], 0
cur.append(line)
size += len(line) + 1
if cur:
parts.append(cur)
return parts
def cmd_wake(d, args):
log, tree = read_log(d), read_tree(d)
todo = pending(log, tree)
@ -180,11 +206,28 @@ def cmd_wake(d, args):
print("You have no memories yet. This is your first moment.\n"
"Record what matters with: memo note \"...\"")
return
for lo, hi in cover(len(log), WAKE_LINES):
if hi - lo == 1:
print("#%d %s %s" % log[lo])
else:
print("#%d-%d %s" % (lo, hi - 1, tree[(lo, hi)]))
lines = ["#%d %s %s" % log[lo] if hi - lo == 1 else
"#%d-%d %s" % (lo, hi - 1, tree[(lo, hi)])
for lo, hi in cover(len(log), WAKE_LINES)]
parts = paginate(lines)
k = 1
if args:
if len(args) != 1 or not args[0].isdigit():
die("usage: memo wake [part]")
k = int(args[0])
if not 1 <= k <= len(parts):
die("There is no part %d. Your memory has %d." % (k, len(parts)))
if len(parts) > 1:
print("--- your memory, part %d of %d (oldest first) ---" % (k, len(parts)))
print("\n".join(parts[k - 1]))
if k < len(parts):
print("--- END OF PART %d. You are still half awake: you have NOT seen "
"your most recent\n memories yet, and acting now would be "
"acting blind. Run `memo wake %d`\n RIGHT NOW, and keep going "
"until a part tells you that was the last one. ---" % (k, k + 1))
elif len(parts) > 1:
print("--- END OF PART %d, the last one. That is your whole memory. "
"You are awake. ---" % k)
def cmd_note(d, args):

21
test.py
View file

@ -114,10 +114,29 @@ check(naps == len(complete(N)), "did %d naps, expected %d" % (naps, len(complete
r = run("wake")
check(r.returncode == 0, "wake still refuses after a full sleep")
lines = r.stdout.strip().splitlines()
# the document survives pagination, and every part fits the strictest harness
# output cap in the wild (Codex: 10 KiB or 256 lines)
parts, k = [], 1
while True:
r = run("wake", str(k))
if r.returncode != 0:
break
body = [l for l in r.stdout.splitlines()
if not l.startswith("---") and not l.startswith(" ")]
check(len(r.stdout) < 10240, "part %d is %d bytes, over Codex's 10 KiB cap"
% (k, len(r.stdout)))
check(len(r.stdout.splitlines()) < 256, "part %d is over Codex's 256-line cap" % k)
parts.append(body)
k += 1
check(len(parts) > 1, "a %d-line memory should need more than one part" % WAKE_LINES)
lines = [l for p in parts for l in p]
check(len(lines) == WAKE_LINES, "woke with %d lines, want %d" % (len(lines), WAKE_LINES))
check(lines[-1].startswith("#%d " % (N - 1)), "newest memory not last / not raw")
check(lines[0].startswith("#0-"), "oldest line should be a summary block")
check("memo wake 2" in run("wake").stdout, "part 1 must name the next command")
check("last one" in run("wake", str(len(parts))).stdout, "last part must say it is last")
check(run("wake", str(len(parts) + 1)).returncode == 1, "a nonexistent part should fail")
# append-only: nothing was ever rewritten
sizes = {f: os.path.getsize(os.path.join(d, f)) for f in ("LOG.txt", "TREE.txt")}