Implementation plan: drop cross-version compat, beta/stable as separate networks

Explicit stance: 0.6.x beta does not interoperate with 0.5 stable.
Removes dual-write migration machinery, mixed-version wire handlers,
and cross-track testing. Users cross tracks via export/import bundle.

Preserves: local upgrade path for user's own data; serde-default wire
field additions; identity bundle format compat.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Scott Reimers 2026-04-21 18:26:55 -04:00
parent 9f2603f981
commit 921a0ec40a
2 changed files with 46 additions and 25 deletions

View file

@ -2,13 +2,27 @@
## Context ## Context
0.5.3-beta is graduating to stable. The 0.6.x beta line introduces the Network-ID / Posting-ID split, multi-persona support, ephemeral rotating DM identities, file-holder CDN restructure, and CDN-only DM privacy. 0.5.3 is stable. The 0.6.x beta line introduces the Network-ID / Posting-ID split, multi-persona support, ephemeral rotating DM identities, file-holder CDN restructure, and CDN-only DM privacy.
Full architectural plan: `/home/sologretto/.claude/plans/woolly-nibbling-glade.md` Full architectural plan: `/home/sologretto/.claude/plans/woolly-nibbling-glade.md`
Canonical reference: `website/design.html` §28 Canonical reference: `website/design.html` §28
Memory summary: `reference_identity_architecture.md` Memory summary: `reference_identity_architecture.md`
Each phase below is a standalone release. Each phase is backward compatible with peers on earlier versions (they degrade gracefully or stay on the old path). After all phases ship, 0.7.0-beta consolidates and 0.7.x becomes the candidate for the next stable promotion. ### Backward-compatibility stance
**Beta and stable are separate networks.** A v0.6 node is not expected to interoperate with a v0.5 node. This is an explicit decision to reduce implementation overhead — no dual-writing legacy tables, no "match on either old or new field" handlers, no migration-through-mixed-network testing.
What we DO preserve:
- **Local upgrade path.** A user upgrading their own install from 0.5 → 0.6 must not lose data. Existing posts, follows, keys, blobs continue to work. Migration is one-way (users don't downgrade) but must be data-safe.
- **Free wire-level compat.** If a protocol field is already `#[serde(default, skip_serializing_if = "Option::is_none")]`, new fields simply follow that pattern. Old serialized data with missing new fields deserializes fine. No special effort.
- **User data interop across versions.** An exported 0.5 identity bundle must be importable into 0.6.
What we DROP:
- Dual-writing deprecated tables during a phase transition. When a phase swaps out a data structure, the old one is gone.
- "Works with v0.5 peers" testing. Beta is tested against beta.
- Legacy wire-protocol tolerances beyond what serde defaults give for free.
Each phase is a standalone release. Phases are ordered to satisfy internal code dependencies, not wire-compat boundaries. After all phases ship, 0.7.0-beta consolidates and 0.7.x becomes the candidate for the next stable promotion.
--- ---
@ -17,18 +31,19 @@ Each phase below is a standalone release. Each phase is backward compatible with
**Goal:** Eliminate the sender→recipient traffic signal. Encrypted DMs propagate via the existing ManifestPush / CDN tree, indistinguishable on the wire from any other encrypted post. **Goal:** Eliminate the sender→recipient traffic signal. Encrypted DMs propagate via the existing ManifestPush / CDN tree, indistinguishable on the wire from any other encrypted post.
**Scope:** **Scope:**
- Remove or gate `push_post_to_recipients` in `crates/core/src/node.rs` - Remove `push_post_to_recipients` from `crates/core/src/node.rs` entirely
- Ensure new encrypted posts still trigger a normal header update on neighbor posts (already done by existing CDN logic) so they propagate - Remove the `PostPush` message handler from connection.rs and/or leave it as a stub that logs + ignores (avoid needing to invent a new message type ordering)
- Verify the existing ManifestPush fan-out path reaches recipients who follow the author's posting ID (today they do since follows pull posts) - Ensure new encrypted posts still trigger a normal header update on neighbor posts (existing CDN logic already does this) so they propagate
- Add a "CDN delivery SLA" doc note: expect ~seconds to tens-of-seconds latency for DMs to followers, minutes worst case for offline-then-online recipients - Verify the existing ManifestPush fan-out path reaches recipients who follow the author's posting ID
- Add a "CDN delivery SLA" doc note: expect ~seconds to tens-of-seconds latency for DMs to followers; minutes worst case for offline-then-online recipients
**Verification:** **Verification:**
- Send an encrypted DM with two test devices. Confirm the recipient receives it without any direct-push message firing (network log inspection) - Send an encrypted DM with two test devices. Confirm the recipient receives it without any direct-push message firing (network log inspection)
- Measure p50/p95 delivery latency on a small mesh - Measure p50/p95 delivery latency on a small mesh of 0.6.0 nodes
- Backward compat: verify a v0.6 client can DM a v0.5 client and vice versa - Confirm cold-contact DMs (non-follower) don't reach — expected until Phase 3 or comment-intro UX arrives
**Risks:** **Risks:**
- Latency regression for non-follower recipients (they won't get it until Phase 3 or comment-intro shipping). For Phase 1, keep the push path behind a compile flag for non-follower recipients if safety is a concern — or explicitly document that "DMs to non-followers don't reach yet; follow or comment on their post" - Non-follower DMs don't reach until later phases. Ship with UX messaging: "DMs to new contacts require either following them or commenting on their post first."
--- ---
@ -41,7 +56,7 @@ Each phase below is a standalone release. Each phase is backward compatible with
- Populate on ManifestPush receive, blob fetch, blob serve, engagement diff exchange - Populate on ManifestPush receive, blob fetch, blob serve, engagement diff exchange
- Cap per file_id at 5 holders (LRU on `last_interaction_ms`) - Cap per file_id at 5 holders (LRU on `last_interaction_ms`)
- Refactor engagement diff delivery: instead of "send to upstream," send to the file's up-to-5 known holders - Refactor engagement diff delivery: instead of "send to upstream," send to the file's up-to-5 known holders
- Keep `post_upstream` / `post_downstream` tables for compat reads during the transition - **Drop `post_upstream` and `post_downstream` tables in this phase's migration.** No dual-write. (Local upgrade safety: before-drop, scan these tables to seed the new `file_holders` table with any existing peer relationships so we don't start empty.)
- ManifestPush propagation logic: when header of file A changes, look up A's holders, send diff to them - ManifestPush propagation logic: when header of file A changes, look up A's holders, send diff to them
- Receiver behavior: apply header diff, pull any referenced new files - Receiver behavior: apply header diff, pull any referenced new files
@ -49,10 +64,15 @@ Each phase below is a standalone release. Each phase is backward compatible with
- `BlobHeaderDiff` and `ManifestPush` messages stay the same on the wire - `BlobHeaderDiff` and `ManifestPush` messages stay the same on the wire
- What changes is the sender's list of destinations (was: upstreams; now: holders) - What changes is the sender's list of destinations (was: upstreams; now: holders)
**Local migration (user upgrading 0.6.0 → 0.6.1):**
- On first startup with new schema, read `post_upstream` + `post_downstream`, populate `file_holders` with those peers as initial holders for their respective posts
- Drop the old tables after the seed migration
- User's existing posts, follows, blobs all untouched
**Verification:** **Verification:**
- Mesh test: create a post, track its propagation; confirm holders accumulate up to 5 diverse peers - Mesh test: create a post, track its propagation; confirm holders accumulate up to 5 diverse peers
- Engagement propagation: a reaction on a deeply-nested post still reaches the author (now: via the post's holders) - Engagement propagation: a reaction on a deeply-nested post still reaches the author (now: via the post's holders)
- Backward compat: a v0.6.1 sending to a v0.6.0 holder should still work (v0.6.0 applies header diff normally) - Migration test: upgrade a populated 0.6.0 DB to 0.6.1; confirm existing posts still have at least one holder in `file_holders`
**Risks:** **Risks:**
- Churn in holder sets during network instability (holders going offline trigger LRU replacement). Need soak testing. - Churn in holder sets during network instability (holders going offline trigger LRU replacement). Need soak testing.
@ -77,7 +97,6 @@ Each phase below is a standalone release. Each phase is backward compatible with
**Verification:** **Verification:**
- Send a DM from A to B where A is not followed by B. Verify B receives it on next pull cycle (via the recipient-match). - Send a DM from A to B where A is not followed by B. Verify B receives it on next pull cycle (via the recipient-match).
- Benchmark pull query cost with the new OR-clause. Should be near-zero with the index. - Benchmark pull query cost with the new OR-clause. Should be near-zero with the index.
- Backward compat: a v0.6.2 pulling from a v0.6.1 peer still works (peer ignores the "recipient match" intent since it only indexes author)
**Risks:** **Risks:**
- Query cost grows with posts held × recipient list length. Index is mandatory. - Query cost grows with posts held × recipient list length. Index is mandatory.
@ -102,16 +121,15 @@ Each phase below is a standalone release. Each phase is backward compatible with
**Wire protocol:** **Wire protocol:**
- `BlobHeader` already has an `author` field. We just populate it from posting key instead of network key. - `BlobHeader` already has an `author` field. We just populate it from posting key instead of network key.
- For mixed-version networks: v0.6.3 posts signed by a posting_id that happens to equal the author's network_id are indistinguishable from v0.6.2 posts. Backward compat is automatic. - First-run migration for upgrading users: existing identity becomes the first posting identity. `posting_id == network_id` for pre-split content. Users can create additional posting identities after the upgrade.
- First-run migration: existing users have network_id == posting_id. Nothing changes for them until they explicitly create a second persona.
**Verification:** **Verification:**
- Existing identity still works; no data loss - Existing identity still works; no data loss after 0.5→0.6 migration chain
- Posts from upgraded clients still validate on older clients - Posts authored before the upgrade still render and validate
- Posts from older clients still decrypt and render on upgraded clients - New posts use posting key; confirm signatures verify with the posting key
**Risks:** **Risks:**
- Signature verification regression if `author` field handling changes subtly. Need extensive cross-version testing. - Signature verification regression if `author` field handling changes subtly. Needs unit tests for both "legacy author = network key" and "posting key ≠ network key" cases.
- Storage migration needs transaction safety. - Storage migration needs transaction safety.
--- ---
@ -171,12 +189,14 @@ Each phase below is a standalone release. Each phase is backward compatible with
## Version promotion plan ## Version promotion plan
- **0.5.3-beta → stable** (this work). Last stable of the pre-split architecture. - **0.5.3 is stable.** Last stable of the pre-split architecture. Maintenance-only (critical bugs, security). No new features.
- 0.6.0 through 0.6.5 are beta releases shipping Phases 1-6. - **0.6.0 through 0.6.5** are beta releases shipping Phases 1-6. Beta users form a network distinct from stable users.
- **0.7.0-beta** consolidates and cleans up deprecated code paths (post_upstream/downstream tables dropped, old push code removed). - **0.7.0-beta** consolidates the complete new architecture; the legacy code paths are already removed as each phase shipped, so 0.7.0 is mostly polish + integration testing.
- **0.7.x-beta** gets real-world soak testing as the full new architecture. - **0.7.x-beta** gets real-world soak testing.
- **0.7.N → stable** once the new architecture is proven. - **0.7.N → stable** once the new architecture is proven.
No "migration bridge" version is needed between 0.5 stable and 0.6 beta — users choose a track. Moving between tracks is possible via the existing export/import identity bundle (users can carry their content across, because the bundle format is preserved as a free compat concern).
## Order-of-operations recommendations ## Order-of-operations recommendations
- Phases 1 and 2 can overlap (CDN restructure can happen while PostPush is being removed — they touch different code paths). - Phases 1 and 2 can overlap (CDN restructure can happen while PostPush is being removed — they touch different code paths).

View file

@ -1756,10 +1756,11 @@ END</code></pre>
<li><strong>Phase 1</strong> — Remove direct <code>PostPush</code> for encrypted posts (keeps existing CDN tree). Encrypted DMs propagate via ManifestPush like any other content.</li> <li><strong>Phase 1</strong> — Remove direct <code>PostPush</code> for encrypted posts (keeps existing CDN tree). Encrypted DMs propagate via ManifestPush like any other content.</li>
<li><strong>Phase 2</strong> — File-holder model + header-diff propagation replaces upstream/downstream. Diverse lateral holder networks per file.</li> <li><strong>Phase 2</strong> — File-holder model + header-diff propagation replaces upstream/downstream. Diverse lateral holder networks per file.</li>
<li><strong>Phase 3</strong> — Merged pull + recipient-match search. DM search becomes indistinguishable from follow-pull.</li> <li><strong>Phase 3</strong> — Merged pull + recipient-match search. DM search becomes indistinguishable from follow-pull.</li>
<li><strong>Phase 4</strong> — Posting-key / network-key split. Local-only linked-devices hints. Multi-persona UX.</li> <li><strong>Phase 4</strong> — Posting-key / network-key split. Local-only linked-devices hints. Multi-persona plumbing.</li>
<li><strong>Phase 5</strong> — Ephemeral rotating IDs for DM threads + local self-archive.</li> <li><strong>Phase 5</strong> — Multi-persona UX.</li>
<li><strong>Phase 6</strong> — Ephemeral rotating IDs for DM threads + local self-archive.</li>
</ol> </ol>
<p>Each phase is backward compatible with peers still on the previous mechanisms.</p> <p><strong>Beta and stable are separate networks.</strong> The 0.6.x beta line does not interoperate with 0.5.3 stable. This is an explicit simplification — no dual-writing legacy tables, no mixed-version wire handlers, no cross-network testing matrices. Users can cross tracks via the existing export/import identity bundle.</p>
</section> </section>
<!-- Appendix A: Timeouts --> <!-- Appendix A: Timeouts -->