# Layer 5 — Unlock Cache & Prefilter Optimization **Scope**: Performance layer. Makes FoF post decryption feasible at realistic keyring sizes (target: 400–500 keys × 400–500 wrap slots per post). Three mechanisms: author-direct fast path, winning-`V_x`-per-author cache, and unreadable-posts retry table. This layer is load-bearing, not optional. Without it, a modest keyring × slot matrix makes per-post unlock cost user-visible in feed scroll. --- ## Goal - Trial-decryption cost on any single FoF post approaches O(1) once the reader has successfully read any prior post from that author. - Author-direct posts (author = one of the reader's personas) skip wrap-slot iteration entirely. - Posts that couldn't be decrypted when received get re-attempted automatically when the reader's keyring changes (new `V_x` received). --- ## Lead decisions - **Cache the winning `(persona, V_x)` per author.** First time persona `P` decrypts an FoFClosed post from author `A` using `V_x`, remember the tuple. Next post from `A`: try that `(P, V_x)` first. Author almost always re-wraps under the same set. - **Track unreadable posts.** If no key currently held unwraps a post, insert into `vouch_unreadable_posts` for later retry. Clearing this set is cheap and necessary — a newly-received `V_x` potentially unlocks an arbitrary number of old posts. - **Author-direct fast path.** If `post.author` is one of the reader's persona IDs, the reader is the author and holds `priv_post` implicitly (author-local cache of per-post keypairs). No wrap-slot iteration needed. - **Prefilter tag precompute per-post, not per-feed-fetch.** At ingest time (once per post received), compute the reader's full set of `HMAC(V_x, post_id)[:2B]` tags and note which slots have matching prefilter values. Cache that index. Avoids recomputing HMACs on every re-render. --- ## Data model ### `vouch_unlock_cache` table ``` vouch_unlock_cache( reader_persona_id BLOB, author_id BLOB, winning_v_x_owner BLOB, -- who issued the V_x that unlocked winning_v_x_epoch INTEGER, last_hit_ms INTEGER, hit_count INTEGER, PRIMARY KEY (reader_persona_id, author_id) ) ``` One row per `(reader_persona, author)` pair. Updated on every successful unlock (bump hit_count, touch last_hit_ms). On miss-then-hit with a different `V_x`, overwrite the tuple. ### `vouch_unreadable_posts` table ``` vouch_unreadable_posts( post_id BLOB PRIMARY KEY, author_id BLOB, first_seen_ms INTEGER, last_attempt_ms INTEGER, keyring_sig_at_attempt BLOB -- hash of reader's keyring state at last attempt; skip re-attempt if unchanged ) ``` When a post can't be decrypted, insert here. On keyring change (new `V_x` arrives), sweep this table and re-attempt with the new key only. ### `post_slot_prefilter_index` table (optional, perf-heavy workloads only) ``` post_slot_prefilter_index( post_id BLOB, slot_index INTEGER, prefilter_tag BLOB(2), PRIMARY KEY (post_id, slot_index) ) ``` Cached per-slot tags. On first ingest, parse and insert. Query by tag when trying to decrypt. TBD — OPUS: whether this table is worth it. Alternative: keep `wrap_slots` as a parsed struct in memory on read; iterate linearly. At 500 slots × 20B overhead = 10KB per post header, in-memory iteration is probably faster than SQLite index lookup. Lead leaning: **skip the table; iterate in-memory after parsing**. --- ## Fast path algorithm On incoming FoFClosed post from author `A`: 1. **Author-direct check**: Is `A` in reader's list of personas? If yes → reader authored it; pull `priv_post` from local author cache. Done. 2. **Cache lookup**: Query `vouch_unlock_cache` for `(any_persona, A)`. For each cached winning `V_x`: - Compute `prefilter_tag = HMAC(V_x, post_id)[:2B]`. - Find matching slot(s) in post's `wrap_slots`; attempt AEAD-open. - On success → done. Update `last_hit_ms`, increment `hit_count`. 3. **Full scan fallback**: If cache missed, iterate reader's full keyring × wrap_slots with prefilter. On success → insert/update `vouch_unlock_cache`. Done. 4. **Unreadable**: If full scan fails → insert into `vouch_unreadable_posts`. Step 2 expected cost: 1 HMAC + 1 AEAD attempt (hot path, recurring author). Step 3 cost: `keyring_size × slot_count / 65536` AEAD attempts on average. Step 1 cost: single table lookup. --- ## Keyring-change trigger When a new `V_x` is inserted into `vouch_keys_received` (Layer 1 distribution): 1. Compute the prefilter tag for the new `V_x` against each unreadable post's ID. 2. For every matching slot in `vouch_unreadable_posts` entries → attempt AEAD-open with the new `V_x`. 3. On success → move row out of `vouch_unreadable_posts`, insert into `vouch_unlock_cache`, render post in feed. 4. Emit UI signal: "N new posts from your friends-of-friends are now readable." Cost: proportional to size of `vouch_unreadable_posts`, but only fires when the user receives a new vouch (rare event). --- ## Feed-rendering budget Target: feed scroll renders within existing budget (16ms/frame on desktop, 33ms on mobile). - Post already cached (99% hit rate in steady state): ~microseconds per post. - Cache miss (first post from a new author): tens of microseconds per post. - Cold full-scan: milliseconds; ideally done once per post during ingest, not during render. **Ingest-time decrypt vs render-time decrypt.** Lead leaning: **decrypt at ingest**. Store cleartext body keyed by post_id in a local `post_body_cache` (encrypted at rest by persona key). Feed rendering reads from cache; no per-frame crypto. TBD — OPUS: confirm ingest-time decrypt is acceptable for the privacy model (cleartext at rest is already standard for posts the user can read). --- ## Open questions - **Cache invalidation on persona switch.** If user switches default persona, cache queries filter by reader_persona_id, so this is automatic. Check. - **Multi-persona unlock priority.** If two personas could both decrypt the same post, which wins? Probably doesn't matter — whichever tries first. Consequence: comments on FoF posts default to the persona that decrypted (Layer 2 UX). TBD — OPUS: confirm deterministic iteration order of personas. - **Cache size bounds.** `vouch_unlock_cache` is O(personas × authors_ever_read). Authors-ever-read can grow unbounded. GC old entries past some threshold? Lead leaning: no GC; memory cost is trivial at realistic scales. - **Unreadable retry storm on large keyring arrival.** Receiving an entire bundle of new vouches at once (e.g., account import) could trigger a storm of retries. Throttle / batch. Lead leaning: background task, non-blocking on UI. - **Negative-cache on bad `V_x`.** If a reader holds a `V_x` that has NO overlap with any post they've seen, every unreadable post retry pays the full prefilter scan. Store a "tried and failed" marker per `(post_id, V_x)` to skip. TBD — OPUS: is this worth the storage overhead? Realistically, new `V_x` arrivals are rare enough that the scan cost per arrival is acceptable. --- ## Ship criteria for Layer 5 - `vouch_unlock_cache` and `vouch_unreadable_posts` tables exist. - Author-direct fast path: reader's own posts skip wrap-slot iteration. - `(reader_persona, author)` cache hit → single AEAD attempt on known post from recurring author. - New `V_x` arrival triggers retry sweep on `vouch_unreadable_posts`. - Feed-scroll performance: no user-visible stutter at 500-key keyring × 500-slot posts. - Integration benchmark: 100-post feed from 20 authors, 400-key keyring, 400-slot posts → total decrypt < 100ms cold, < 10ms warm.