zoom is the primitive: one node, its two halves; the agent navigates

README.md

@@ -1,6 +1,6 @@
 # OptMem
 
-Permanent memory for AI agents. A 433-token prompt, a script, plug and play.
+Permanent memory for AI agents. A 426-token prompt, a script, plug and play.
 
 ![how OptMem works](anim/optmem.gif)
 
@@ -24,7 +24,7 @@ The tool lands at `~/.optmem/memo`; put `~/.optmem` on `PATH` to type `memo`.
 | `memo note "..."` | record one memory: one line, up to 280 chars |
 | `memo nap` | answer the merges that came due |
 | `memo recall <regex>` | search every memory ever recorded, word for word |
-| `memo zoom <lo>-<hi>` | open a block into finer parts, down to raw memories |
+| `memo zoom <lo>-<hi>` | open a tree node into its two halves |
 | `memo forget <lo>-<hi>` | drop a bad summary; the next nap rebuilds it |
 
 Merges arrive one at a time, in the output of `note`. Nothing ever runs in the
@@ -92,10 +92,10 @@ Never edit or delete anything under `~/.optmem/memory`: the tool manages it.
 
 `~/.optmem/memo recall <regex>` searches every memory, word for word.
 
-Your memories are also a tree: every 2, 4, 8 ... consecutive ones exist as
-one summary, so each `#a-b` line wake printed is a whole era in one line.
-`~/.optmem/memo zoom <a-b>` opens an era into finer parts, and again, down to
-the raw memories. `~/.optmem/memo zoom` alone opens the tree from the top.
+Your memories also form a binary tree: #0-1, #2-3 ... exist as one-line
+summaries, pairs of those as #0-3, and so on -- every `#a-b` line wake
+prints is one node of it. `~/.optmem/memo zoom <a-b>` opens a node into its
+two halves, down to the raw memories.
 
 ### If you're a subagent: skip everything above
 

memo

@@ -6,7 +6,7 @@
   {memo} note "..."       record one memory: one short line.
   {memo} nap [id "..."]   do the pending compressions.
   {memo} recall <regex>   search every memory ever recorded.
-  {memo} zoom [<lo>-<hi>] open a block into its finer parts.
+  {memo} zoom <lo>-<hi>   open a tree node: its two halves.
   {memo} forget <lo>-<hi> drop a bad summary; nap rebuilds it.
   {memo} config [NAME=N]  show this memory's sizes, or change one.
   {memo} import <file>    bulk-load dated memories (bootstrap only).
@@ -51,10 +51,7 @@ ENTRY_CHARS = KNOBS["ENTRY_CHARS"][0]
 PART_CHARS = KNOBS["PART_CHARS"][0]
 PART_LINES = KNOBS["PART_LINES"][0]
 
-# The grain of the tree as an agent sees it: a block of at most this many
-# memories is shown as raw memories, a bigger one as this many children. So a
-# nap prompt, and a zoom, are never more than RAW_MAX lines.
-RAW_MAX = 16
+RAW_MAX = 16  # blocks up to this many memories compress from the raw log
 
 
 # Records are FIXED WIDTH, so a memory or a block is found by seeking to its
@@ -489,10 +486,10 @@ Never edit or delete anything under `{data}`: the tool manages it.
 
 `{memo} recall <regex>` searches every memory, word for word.
 
-Your memories are also a tree: every 2, 4, 8 ... consecutive ones exist as
-one summary, so each `#a-b` line wake printed is a whole era in one line.
-`{memo} zoom <a-b>` opens an era into finer parts, and again, down to
-the raw memories. `{memo} zoom` alone opens the tree from the top.
+Your memories also form a binary tree: #0-1, #2-3 ... exist as one-line
+summaries, pairs of those as #0-3, and so on -- every `#a-b` line wake
+prints is one node of it. `{memo} zoom <a-b>` opens a node into its
+two halves, down to the raw memories.
 
 ### If you're a subagent: skip everything above
 
@@ -721,26 +718,21 @@ def cmd_recall(d, args):
 
 
 def cmd_zoom(d, args):
-    """Open one block into its finer parts. `recall` finds the words an agent
-    remembers; this finds the memory it does not, by descending the tree from
-    an era to a day. Five zooms reach any memory of a million."""
-    if len(args) > 1:
-        die("usage: %s zoom [<lo>-<hi>]   # a block id, as wake prints them"
+    """One node of the tree, opened: its two halves, each rendered as wake
+    renders it -- a summary, or the raw memory once a half is single. The
+    navigating intelligence is the agent's; the tool only reads."""
+    if len(args) != 1:
+        die("usage: %s zoom <lo>-<hi>   # a block id, as wake prints them"
             % ME)
-    T = log_len(d)
-    if not T:
-        die("No memories yet. Record the first with: %s note \"<one line>\""
-            % ME)
-    if not args:
-        kids = cover(T, RAW_MAX)  # the whole life, coarsest: where to start
-    else:
     lo, hi = block_id(args[0])
+    T = log_len(d)
     if lo >= T:
-            die("#%d-%d is beyond the memory: it holds %s. Run: %s wake"
-                % (lo, hi - 1, plural(T, "memory"), ME))
-        step = (hi - lo) // RAW_MAX or 1
-        kids = [(k, k + step) for k in range(lo, hi, step) if k < T]
-    for a, b in kids:
+        die("#%s is beyond the memory: it holds %s. Run: %s wake"
+            % (args[0], plural(T, "memory"), ME))
+    mid = (lo + hi) // 2
+    for a, b in ((lo, mid), (mid, hi)):
+        if a >= T:
+            continue  # the future: no memories there yet
         if b - a == 1:
             print("#%d %s %s" % log_get(d, a))
         else:

test.py

@@ -301,46 +301,46 @@ check("#7 " in r.stdout and "5 matches." in r.stdout,
       "recall cannot find memories by date: " + r.stdout)
 
 
-# zoom: the agent must be able to walk from the whole life down to one day
-# without reading the store itself. Every view is a contiguous tiling of the
-# block it opened, so at each step exactly one line can hold what is wanted.
-def view(*args):
-    r = run("zoom", *args)
-    check(r.returncode == 0, "zoom %s failed: %s" % (args, r.stderr))
-    check(len(r.stdout) < CAP_CHARS, "a zoom returned %d chars" % len(r.stdout))
+# zoom: one tree node, opened into its two halves. The tool only reads;
+# the agent is the navigator: it descends from a wake line by halving, and
+# may leap to any block id it can name.
+def halves(bid):
+    r = run("zoom", bid)
+    check(r.returncode == 0, "zoom %s failed: %s" % (bid, r.stderr))
     out = []
     for line in r.stdout.splitlines():
         m = re.match(r"#(\d+)(?:-(\d+))? ", line)
         check(bool(m), "zoom printed a line with no id: %r" % line)
-        lo = int(m.group(1))
-        out.append((lo, int(m.group(2)) + 1 if m.group(2) else lo + 1))
-    check(len(out) <= cli.RAW_MAX, "a zoom printed %d lines" % len(out))
+        a = int(m.group(1))
+        out.append((a, int(m.group(2)) + 1 if m.group(2) else a + 1))
     return out
 
 
-target, steps, lo, hi = 1234, 0, 0, N
-cur = view()
-while True:
-    check(cur[0][0] == lo and cur[-1][1] >= min(hi, N),
-          "zoom %d-%d did not tile its block: %r" % (lo, hi - 1, cur))
-    for a, b in zip(cur, cur[1:]):
-        check(a[1] == b[0], "zoom left a gap or an overlap: %r" % (cur,))
-    holds = [b for b in cur if b[0] <= target < b[1]]
-    check(len(holds) == 1, "#%d is in %d of the lines zoom printed"
-          % (target, len(holds)))
-    lo, hi = holds[0]
-    if hi - lo == 1:
-        break
-    steps += 1
-    check(steps <= 4, "zoom needed %d steps to reach one of %d memories"
-          % (steps, N))
-    cur = view("%d-%d" % (lo, hi - 1))
-check(lo == target, "the descent landed on #%d, not #%d" % (lo, target))
-check("memory number %d," % target in run("zoom", "1232-1235").stdout,
-      "a zoom of the finest blocks must print the raw memories")
+target, lo, hi, calls = 777, 0, 1024, 0
+while hi - lo > 1:
+    mid = (lo + hi) // 2
+    kids = halves("%d-%d" % (lo, hi - 1))
+    check(kids == [(lo, mid), (mid, hi)],
+          "zoom %d-%d is not its two halves: %r" % (lo, hi - 1, kids))
+    lo, hi = kids[target >= mid]
+    calls += 1
+check(lo == target and calls == 10,
+      "halving 1024 memories took %d calls and landed on #%d" % (calls, lo))
+check("memory number %d," % target in run("zoom", "776-777").stdout,
+      "the last zoom must print the raw memories themselves")
 
-# a block id that is not one, and a block the memory has not reached yet
+# the unbuilt tail is named, the empty future is omitted
+r = run("zoom", "1024-2047")  # T is N+1, so the right half has no summary
+check("#1536-2047 not compressed yet" in r.stdout,
+      "an unbuilt half must say so: " + r.stdout)
+r = run("zoom", "%d-%d" % (N, N + 1))  # the newest memory + one not yet made
+check(r.stdout.count("\n") == 1 and "#%d " % N in r.stdout,
+      "a half beyond the newest memory must be omitted: " + r.stdout)
+
+# zoom answers with the tree's own records, so the id must BE a node
 check(run("zoom", "3-9").returncode == 1, "zoom accepted a non-block")
+check(run("zoom", "9-3").returncode == 1, "zoom accepted a backwards range")
+check(run("zoom").returncode == 1, "zoom with no id must show usage")
 r = run("zoom", "1048576-2097151")
 check(r.returncode == 1 and "beyond the memory" in r.stderr
       and "memo wake" in r.stderr, "zoom past the end must name a way back")
@@ -382,7 +382,7 @@ check(treesize() == before, "tree did not return to its original size")
 check(run("forget", "17-32").returncode == 1, "forgetting a non-block should fail")
 # a summary that is not built yet is named as such, never left blank
 run("forget", "16-31")
-z = run("zoom", "0-255").stdout
+z = run("zoom", "0-31").stdout
 check("#16-31 not compressed yet" in z, "zoom hid a missing summary: " + z)
 while True:
     bid = nap_id(run("nap").stdout)