diff --git a/IMPLEMENTATION_PLAN_0.6.md b/IMPLEMENTATION_PLAN_0.6.md index da06ecd..56be4c2 100644 --- a/IMPLEMENTATION_PLAN_0.6.md +++ b/IMPLEMENTATION_PLAN_0.6.md @@ -2,13 +2,27 @@ ## Context -0.5.3-beta is graduating to stable. The 0.6.x beta line introduces the Network-ID / Posting-ID split, multi-persona support, ephemeral rotating DM identities, file-holder CDN restructure, and CDN-only DM privacy. +0.5.3 is stable. The 0.6.x beta line introduces the Network-ID / Posting-ID split, multi-persona support, ephemeral rotating DM identities, file-holder CDN restructure, and CDN-only DM privacy. Full architectural plan: `/home/sologretto/.claude/plans/woolly-nibbling-glade.md` Canonical reference: `website/design.html` §28 Memory summary: `reference_identity_architecture.md` -Each phase below is a standalone release. Each phase is backward compatible with peers on earlier versions (they degrade gracefully or stay on the old path). After all phases ship, 0.7.0-beta consolidates and 0.7.x becomes the candidate for the next stable promotion. +### Backward-compatibility stance + +**Beta and stable are separate networks.** A v0.6 node is not expected to interoperate with a v0.5 node. This is an explicit decision to reduce implementation overhead — no dual-writing legacy tables, no "match on either old or new field" handlers, no migration-through-mixed-network testing. + +What we DO preserve: +- **Local upgrade path.** A user upgrading their own install from 0.5 → 0.6 must not lose data. Existing posts, follows, keys, blobs continue to work. Migration is one-way (users don't downgrade) but must be data-safe. +- **Free wire-level compat.** If a protocol field is already `#[serde(default, skip_serializing_if = "Option::is_none")]`, new fields simply follow that pattern. Old serialized data with missing new fields deserializes fine. No special effort. +- **User data interop across versions.** An exported 0.5 identity bundle must be importable into 0.6. + +What we DROP: +- Dual-writing deprecated tables during a phase transition. When a phase swaps out a data structure, the old one is gone. +- "Works with v0.5 peers" testing. Beta is tested against beta. +- Legacy wire-protocol tolerances beyond what serde defaults give for free. + +Each phase is a standalone release. Phases are ordered to satisfy internal code dependencies, not wire-compat boundaries. After all phases ship, 0.7.0-beta consolidates and 0.7.x becomes the candidate for the next stable promotion. --- @@ -17,18 +31,19 @@ Each phase below is a standalone release. Each phase is backward compatible with **Goal:** Eliminate the sender→recipient traffic signal. Encrypted DMs propagate via the existing ManifestPush / CDN tree, indistinguishable on the wire from any other encrypted post. **Scope:** -- Remove or gate `push_post_to_recipients` in `crates/core/src/node.rs` -- Ensure new encrypted posts still trigger a normal header update on neighbor posts (already done by existing CDN logic) so they propagate -- Verify the existing ManifestPush fan-out path reaches recipients who follow the author's posting ID (today they do since follows pull posts) -- Add a "CDN delivery SLA" doc note: expect ~seconds to tens-of-seconds latency for DMs to followers, minutes worst case for offline-then-online recipients +- Remove `push_post_to_recipients` from `crates/core/src/node.rs` entirely +- Remove the `PostPush` message handler from connection.rs and/or leave it as a stub that logs + ignores (avoid needing to invent a new message type ordering) +- Ensure new encrypted posts still trigger a normal header update on neighbor posts (existing CDN logic already does this) so they propagate +- Verify the existing ManifestPush fan-out path reaches recipients who follow the author's posting ID +- Add a "CDN delivery SLA" doc note: expect ~seconds to tens-of-seconds latency for DMs to followers; minutes worst case for offline-then-online recipients **Verification:** - Send an encrypted DM with two test devices. Confirm the recipient receives it without any direct-push message firing (network log inspection) -- Measure p50/p95 delivery latency on a small mesh -- Backward compat: verify a v0.6 client can DM a v0.5 client and vice versa +- Measure p50/p95 delivery latency on a small mesh of 0.6.0 nodes +- Confirm cold-contact DMs (non-follower) don't reach — expected until Phase 3 or comment-intro UX arrives **Risks:** -- Latency regression for non-follower recipients (they won't get it until Phase 3 or comment-intro shipping). For Phase 1, keep the push path behind a compile flag for non-follower recipients if safety is a concern — or explicitly document that "DMs to non-followers don't reach yet; follow or comment on their post" +- Non-follower DMs don't reach until later phases. Ship with UX messaging: "DMs to new contacts require either following them or commenting on their post first." --- @@ -41,7 +56,7 @@ Each phase below is a standalone release. Each phase is backward compatible with - Populate on ManifestPush receive, blob fetch, blob serve, engagement diff exchange - Cap per file_id at 5 holders (LRU on `last_interaction_ms`) - Refactor engagement diff delivery: instead of "send to upstream," send to the file's up-to-5 known holders -- Keep `post_upstream` / `post_downstream` tables for compat reads during the transition +- **Drop `post_upstream` and `post_downstream` tables in this phase's migration.** No dual-write. (Local upgrade safety: before-drop, scan these tables to seed the new `file_holders` table with any existing peer relationships so we don't start empty.) - ManifestPush propagation logic: when header of file A changes, look up A's holders, send diff to them - Receiver behavior: apply header diff, pull any referenced new files @@ -49,10 +64,15 @@ Each phase below is a standalone release. Each phase is backward compatible with - `BlobHeaderDiff` and `ManifestPush` messages stay the same on the wire - What changes is the sender's list of destinations (was: upstreams; now: holders) +**Local migration (user upgrading 0.6.0 → 0.6.1):** +- On first startup with new schema, read `post_upstream` + `post_downstream`, populate `file_holders` with those peers as initial holders for their respective posts +- Drop the old tables after the seed migration +- User's existing posts, follows, blobs all untouched + **Verification:** - Mesh test: create a post, track its propagation; confirm holders accumulate up to 5 diverse peers - Engagement propagation: a reaction on a deeply-nested post still reaches the author (now: via the post's holders) -- Backward compat: a v0.6.1 sending to a v0.6.0 holder should still work (v0.6.0 applies header diff normally) +- Migration test: upgrade a populated 0.6.0 DB to 0.6.1; confirm existing posts still have at least one holder in `file_holders` **Risks:** - Churn in holder sets during network instability (holders going offline trigger LRU replacement). Need soak testing. @@ -77,7 +97,6 @@ Each phase below is a standalone release. Each phase is backward compatible with **Verification:** - Send a DM from A to B where A is not followed by B. Verify B receives it on next pull cycle (via the recipient-match). - Benchmark pull query cost with the new OR-clause. Should be near-zero with the index. -- Backward compat: a v0.6.2 pulling from a v0.6.1 peer still works (peer ignores the "recipient match" intent since it only indexes author) **Risks:** - Query cost grows with posts held × recipient list length. Index is mandatory. @@ -102,16 +121,15 @@ Each phase below is a standalone release. Each phase is backward compatible with **Wire protocol:** - `BlobHeader` already has an `author` field. We just populate it from posting key instead of network key. -- For mixed-version networks: v0.6.3 posts signed by a posting_id that happens to equal the author's network_id are indistinguishable from v0.6.2 posts. Backward compat is automatic. -- First-run migration: existing users have network_id == posting_id. Nothing changes for them until they explicitly create a second persona. +- First-run migration for upgrading users: existing identity becomes the first posting identity. `posting_id == network_id` for pre-split content. Users can create additional posting identities after the upgrade. **Verification:** -- Existing identity still works; no data loss -- Posts from upgraded clients still validate on older clients -- Posts from older clients still decrypt and render on upgraded clients +- Existing identity still works; no data loss after 0.5→0.6 migration chain +- Posts authored before the upgrade still render and validate +- New posts use posting key; confirm signatures verify with the posting key **Risks:** -- Signature verification regression if `author` field handling changes subtly. Need extensive cross-version testing. +- Signature verification regression if `author` field handling changes subtly. Needs unit tests for both "legacy author = network key" and "posting key ≠ network key" cases. - Storage migration needs transaction safety. --- @@ -171,12 +189,14 @@ Each phase below is a standalone release. Each phase is backward compatible with ## Version promotion plan -- **0.5.3-beta → stable** (this work). Last stable of the pre-split architecture. -- 0.6.0 through 0.6.5 are beta releases shipping Phases 1-6. -- **0.7.0-beta** consolidates and cleans up deprecated code paths (post_upstream/downstream tables dropped, old push code removed). -- **0.7.x-beta** gets real-world soak testing as the full new architecture. +- **0.5.3 is stable.** Last stable of the pre-split architecture. Maintenance-only (critical bugs, security). No new features. +- **0.6.0 through 0.6.5** are beta releases shipping Phases 1-6. Beta users form a network distinct from stable users. +- **0.7.0-beta** consolidates the complete new architecture; the legacy code paths are already removed as each phase shipped, so 0.7.0 is mostly polish + integration testing. +- **0.7.x-beta** gets real-world soak testing. - **0.7.N → stable** once the new architecture is proven. +No "migration bridge" version is needed between 0.5 stable and 0.6 beta — users choose a track. Moving between tracks is possible via the existing export/import identity bundle (users can carry their content across, because the bundle format is preserved as a free compat concern). + ## Order-of-operations recommendations - Phases 1 and 2 can overlap (CDN restructure can happen while PostPush is being removed — they touch different code paths). diff --git a/website/design.html b/website/design.html index b961cdf..f52c7c4 100644 --- a/website/design.html +++ b/website/design.html @@ -1756,10 +1756,11 @@ END
  • Phase 1 — Remove direct PostPush for encrypted posts (keeps existing CDN tree). Encrypted DMs propagate via ManifestPush like any other content.
  • Phase 2 — File-holder model + header-diff propagation replaces upstream/downstream. Diverse lateral holder networks per file.
  • Phase 3 — Merged pull + recipient-match search. DM search becomes indistinguishable from follow-pull.
    • -
  • Phase 4 — Posting-key / network-key split. Local-only linked-devices hints. Multi-persona UX.
    • -
  • Phase 5 — Ephemeral rotating IDs for DM threads + local self-archive.
    • +
  • Phase 4 — Posting-key / network-key split. Local-only linked-devices hints. Multi-persona plumbing.
    • +
  • Phase 5 — Multi-persona UX.
    • +
  • Phase 6 — Ephemeral rotating IDs for DM threads + local self-archive.
    • -

    Each phase is backward compatible with peers still on the previous mechanisms.

    +

    Beta and stable are separate networks. The 0.6.x beta line does not interoperate with 0.5.3 stable. This is an explicit simplification — no dual-writing legacy tables, no mixed-version wire handlers, no cross-network testing matrices. Users can cross tracks via the existing export/import identity bundle.