Implementation plan: drop cross-version compat, beta/stable as separate networks
Explicit stance: 0.6.x beta does not interoperate with 0.5 stable. Removes dual-write migration machinery, mixed-version wire handlers, and cross-track testing. Users cross tracks via export/import bundle. Preserves: local upgrade path for user's own data; serde-default wire field additions; identity bundle format compat. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
9f2603f981
commit
921a0ec40a
2 changed files with 46 additions and 25 deletions
|
|
@ -2,13 +2,27 @@
|
|||
|
||||
## Context
|
||||
|
||||
0.5.3-beta is graduating to stable. The 0.6.x beta line introduces the Network-ID / Posting-ID split, multi-persona support, ephemeral rotating DM identities, file-holder CDN restructure, and CDN-only DM privacy.
|
||||
0.5.3 is stable. The 0.6.x beta line introduces the Network-ID / Posting-ID split, multi-persona support, ephemeral rotating DM identities, file-holder CDN restructure, and CDN-only DM privacy.
|
||||
|
||||
Full architectural plan: `/home/sologretto/.claude/plans/woolly-nibbling-glade.md`
|
||||
Canonical reference: `website/design.html` §28
|
||||
Memory summary: `reference_identity_architecture.md`
|
||||
|
||||
Each phase below is a standalone release. Each phase is backward compatible with peers on earlier versions (they degrade gracefully or stay on the old path). After all phases ship, 0.7.0-beta consolidates and 0.7.x becomes the candidate for the next stable promotion.
|
||||
### Backward-compatibility stance
|
||||
|
||||
**Beta and stable are separate networks.** A v0.6 node is not expected to interoperate with a v0.5 node. This is an explicit decision to reduce implementation overhead — no dual-writing legacy tables, no "match on either old or new field" handlers, no migration-through-mixed-network testing.
|
||||
|
||||
What we DO preserve:
|
||||
- **Local upgrade path.** A user upgrading their own install from 0.5 → 0.6 must not lose data. Existing posts, follows, keys, blobs continue to work. Migration is one-way (users don't downgrade) but must be data-safe.
|
||||
- **Free wire-level compat.** If a protocol field is already `#[serde(default, skip_serializing_if = "Option::is_none")]`, new fields simply follow that pattern. Old serialized data with missing new fields deserializes fine. No special effort.
|
||||
- **User data interop across versions.** An exported 0.5 identity bundle must be importable into 0.6.
|
||||
|
||||
What we DROP:
|
||||
- Dual-writing deprecated tables during a phase transition. When a phase swaps out a data structure, the old one is gone.
|
||||
- "Works with v0.5 peers" testing. Beta is tested against beta.
|
||||
- Legacy wire-protocol tolerances beyond what serde defaults give for free.
|
||||
|
||||
Each phase is a standalone release. Phases are ordered to satisfy internal code dependencies, not wire-compat boundaries. After all phases ship, 0.7.0-beta consolidates and 0.7.x becomes the candidate for the next stable promotion.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -17,18 +31,19 @@ Each phase below is a standalone release. Each phase is backward compatible with
|
|||
**Goal:** Eliminate the sender→recipient traffic signal. Encrypted DMs propagate via the existing ManifestPush / CDN tree, indistinguishable on the wire from any other encrypted post.
|
||||
|
||||
**Scope:**
|
||||
- Remove or gate `push_post_to_recipients` in `crates/core/src/node.rs`
|
||||
- Ensure new encrypted posts still trigger a normal header update on neighbor posts (already done by existing CDN logic) so they propagate
|
||||
- Verify the existing ManifestPush fan-out path reaches recipients who follow the author's posting ID (today they do since follows pull posts)
|
||||
- Add a "CDN delivery SLA" doc note: expect ~seconds to tens-of-seconds latency for DMs to followers, minutes worst case for offline-then-online recipients
|
||||
- Remove `push_post_to_recipients` from `crates/core/src/node.rs` entirely
|
||||
- Remove the `PostPush` message handler from connection.rs and/or leave it as a stub that logs + ignores (avoid needing to invent a new message type ordering)
|
||||
- Ensure new encrypted posts still trigger a normal header update on neighbor posts (existing CDN logic already does this) so they propagate
|
||||
- Verify the existing ManifestPush fan-out path reaches recipients who follow the author's posting ID
|
||||
- Add a "CDN delivery SLA" doc note: expect ~seconds to tens-of-seconds latency for DMs to followers; minutes worst case for offline-then-online recipients
|
||||
|
||||
**Verification:**
|
||||
- Send an encrypted DM with two test devices. Confirm the recipient receives it without any direct-push message firing (network log inspection)
|
||||
- Measure p50/p95 delivery latency on a small mesh
|
||||
- Backward compat: verify a v0.6 client can DM a v0.5 client and vice versa
|
||||
- Measure p50/p95 delivery latency on a small mesh of 0.6.0 nodes
|
||||
- Confirm cold-contact DMs (non-follower) don't reach — expected until Phase 3 or comment-intro UX arrives
|
||||
|
||||
**Risks:**
|
||||
- Latency regression for non-follower recipients (they won't get it until Phase 3 or comment-intro shipping). For Phase 1, keep the push path behind a compile flag for non-follower recipients if safety is a concern — or explicitly document that "DMs to non-followers don't reach yet; follow or comment on their post"
|
||||
- Non-follower DMs don't reach until later phases. Ship with UX messaging: "DMs to new contacts require either following them or commenting on their post first."
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -41,7 +56,7 @@ Each phase below is a standalone release. Each phase is backward compatible with
|
|||
- Populate on ManifestPush receive, blob fetch, blob serve, engagement diff exchange
|
||||
- Cap per file_id at 5 holders (LRU on `last_interaction_ms`)
|
||||
- Refactor engagement diff delivery: instead of "send to upstream," send to the file's up-to-5 known holders
|
||||
- Keep `post_upstream` / `post_downstream` tables for compat reads during the transition
|
||||
- **Drop `post_upstream` and `post_downstream` tables in this phase's migration.** No dual-write. (Local upgrade safety: before-drop, scan these tables to seed the new `file_holders` table with any existing peer relationships so we don't start empty.)
|
||||
- ManifestPush propagation logic: when header of file A changes, look up A's holders, send diff to them
|
||||
- Receiver behavior: apply header diff, pull any referenced new files
|
||||
|
||||
|
|
@ -49,10 +64,15 @@ Each phase below is a standalone release. Each phase is backward compatible with
|
|||
- `BlobHeaderDiff` and `ManifestPush` messages stay the same on the wire
|
||||
- What changes is the sender's list of destinations (was: upstreams; now: holders)
|
||||
|
||||
**Local migration (user upgrading 0.6.0 → 0.6.1):**
|
||||
- On first startup with new schema, read `post_upstream` + `post_downstream`, populate `file_holders` with those peers as initial holders for their respective posts
|
||||
- Drop the old tables after the seed migration
|
||||
- User's existing posts, follows, blobs all untouched
|
||||
|
||||
**Verification:**
|
||||
- Mesh test: create a post, track its propagation; confirm holders accumulate up to 5 diverse peers
|
||||
- Engagement propagation: a reaction on a deeply-nested post still reaches the author (now: via the post's holders)
|
||||
- Backward compat: a v0.6.1 sending to a v0.6.0 holder should still work (v0.6.0 applies header diff normally)
|
||||
- Migration test: upgrade a populated 0.6.0 DB to 0.6.1; confirm existing posts still have at least one holder in `file_holders`
|
||||
|
||||
**Risks:**
|
||||
- Churn in holder sets during network instability (holders going offline trigger LRU replacement). Need soak testing.
|
||||
|
|
@ -77,7 +97,6 @@ Each phase below is a standalone release. Each phase is backward compatible with
|
|||
**Verification:**
|
||||
- Send a DM from A to B where A is not followed by B. Verify B receives it on next pull cycle (via the recipient-match).
|
||||
- Benchmark pull query cost with the new OR-clause. Should be near-zero with the index.
|
||||
- Backward compat: a v0.6.2 pulling from a v0.6.1 peer still works (peer ignores the "recipient match" intent since it only indexes author)
|
||||
|
||||
**Risks:**
|
||||
- Query cost grows with posts held × recipient list length. Index is mandatory.
|
||||
|
|
@ -102,16 +121,15 @@ Each phase below is a standalone release. Each phase is backward compatible with
|
|||
|
||||
**Wire protocol:**
|
||||
- `BlobHeader` already has an `author` field. We just populate it from posting key instead of network key.
|
||||
- For mixed-version networks: v0.6.3 posts signed by a posting_id that happens to equal the author's network_id are indistinguishable from v0.6.2 posts. Backward compat is automatic.
|
||||
- First-run migration: existing users have network_id == posting_id. Nothing changes for them until they explicitly create a second persona.
|
||||
- First-run migration for upgrading users: existing identity becomes the first posting identity. `posting_id == network_id` for pre-split content. Users can create additional posting identities after the upgrade.
|
||||
|
||||
**Verification:**
|
||||
- Existing identity still works; no data loss
|
||||
- Posts from upgraded clients still validate on older clients
|
||||
- Posts from older clients still decrypt and render on upgraded clients
|
||||
- Existing identity still works; no data loss after 0.5→0.6 migration chain
|
||||
- Posts authored before the upgrade still render and validate
|
||||
- New posts use posting key; confirm signatures verify with the posting key
|
||||
|
||||
**Risks:**
|
||||
- Signature verification regression if `author` field handling changes subtly. Need extensive cross-version testing.
|
||||
- Signature verification regression if `author` field handling changes subtly. Needs unit tests for both "legacy author = network key" and "posting key ≠ network key" cases.
|
||||
- Storage migration needs transaction safety.
|
||||
|
||||
---
|
||||
|
|
@ -171,12 +189,14 @@ Each phase below is a standalone release. Each phase is backward compatible with
|
|||
|
||||
## Version promotion plan
|
||||
|
||||
- **0.5.3-beta → stable** (this work). Last stable of the pre-split architecture.
|
||||
- 0.6.0 through 0.6.5 are beta releases shipping Phases 1-6.
|
||||
- **0.7.0-beta** consolidates and cleans up deprecated code paths (post_upstream/downstream tables dropped, old push code removed).
|
||||
- **0.7.x-beta** gets real-world soak testing as the full new architecture.
|
||||
- **0.5.3 is stable.** Last stable of the pre-split architecture. Maintenance-only (critical bugs, security). No new features.
|
||||
- **0.6.0 through 0.6.5** are beta releases shipping Phases 1-6. Beta users form a network distinct from stable users.
|
||||
- **0.7.0-beta** consolidates the complete new architecture; the legacy code paths are already removed as each phase shipped, so 0.7.0 is mostly polish + integration testing.
|
||||
- **0.7.x-beta** gets real-world soak testing.
|
||||
- **0.7.N → stable** once the new architecture is proven.
|
||||
|
||||
No "migration bridge" version is needed between 0.5 stable and 0.6 beta — users choose a track. Moving between tracks is possible via the existing export/import identity bundle (users can carry their content across, because the bundle format is preserved as a free compat concern).
|
||||
|
||||
## Order-of-operations recommendations
|
||||
|
||||
- Phases 1 and 2 can overlap (CDN restructure can happen while PostPush is being removed — they touch different code paths).
|
||||
|
|
|
|||
|
|
@ -1756,10 +1756,11 @@ END</code></pre>
|
|||
<li><strong>Phase 1</strong> — Remove direct <code>PostPush</code> for encrypted posts (keeps existing CDN tree). Encrypted DMs propagate via ManifestPush like any other content.</li>
|
||||
<li><strong>Phase 2</strong> — File-holder model + header-diff propagation replaces upstream/downstream. Diverse lateral holder networks per file.</li>
|
||||
<li><strong>Phase 3</strong> — Merged pull + recipient-match search. DM search becomes indistinguishable from follow-pull.</li>
|
||||
<li><strong>Phase 4</strong> — Posting-key / network-key split. Local-only linked-devices hints. Multi-persona UX.</li>
|
||||
<li><strong>Phase 5</strong> — Ephemeral rotating IDs for DM threads + local self-archive.</li>
|
||||
<li><strong>Phase 4</strong> — Posting-key / network-key split. Local-only linked-devices hints. Multi-persona plumbing.</li>
|
||||
<li><strong>Phase 5</strong> — Multi-persona UX.</li>
|
||||
<li><strong>Phase 6</strong> — Ephemeral rotating IDs for DM threads + local self-archive.</li>
|
||||
</ol>
|
||||
<p>Each phase is backward compatible with peers still on the previous mechanisms.</p>
|
||||
<p><strong>Beta and stable are separate networks.</strong> The 0.6.x beta line does not interoperate with 0.5.3 stable. This is an explicit simplification — no dual-writing legacy tables, no mixed-version wire handlers, no cross-network testing matrices. Users can cross tracks via the existing export/import identity bundle.</p>
|
||||
</section>
|
||||
|
||||
<!-- Appendix A: Timeouts -->
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue