Design doc: v0.3.2 + v0.3.3 changelog, rate limiting, N2/N3 freshness, bootstrap recovery, schema versioning docs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Scott Reimers
parent
8fad30cf95
commit
ce176a2299
1 changed files with 47 additions and 1 deletions
|
|
@ -43,7 +43,9 @@
|
|||
<p>This is the canonical technical reference for ItsGoin. It describes the vision, the architecture, and the current state of every subsystem — with full implementation detail. This document is versioned; each update records what changed.</p>
|
||||
<div class="card" style="margin-top: 1rem;">
|
||||
<strong style="font-size: 0.85rem; text-transform: uppercase; letter-spacing: 0.05em;">Changelog</strong>
|
||||
<p style="margin-top: 0.5rem;"><strong>v0.3.1</strong> (2026-03-13): Share links + QUIC proxy + content search. Share link format: <code>itsgoin.net/p/<postid_hex>/<author_nodeid_hex></code> — simple, no host encoding needed. itsgoin.net web handler acts as QUIC proxy: receives browser request, searches the network for the post, fetches it on-demand via PostFetch (0xD4/0xD5), renders HTML, serves to browser. No permanent storage of fetched content. Extended worm search — <code>WormQuery</code> now carries optional <code>post_id</code> and <code>blob_id</code> fields for unified node/post/blob search. Each peer checks local storage, CDN downstream tree (up to 100 hosts per post), and blob store. <code>WormResponse</code> gains <code>post_holder</code> and <code>blob_holder</code> fields. Nova fan-out pattern — burst peers include one N2 wide referral; referred peer does its own 101-burst, reaching ~10K nodes with ~202 relay hops. PostFetch (0xD4/0xD5) — lightweight single-post retrieval after worm finds a holder, much lighter than full PullSync. itsgoin.net node deployed as anchor + web handler (<code>--web 8080</code>). “Unavailable” page with honest network model explanation + install CTA. Universal Links / App Links planned for native app interception. | Engagement sync — pull sync now fetches reactions, comments, and policies via BlobHeaderRequest/Response after every sync. Profile push fix — profile updates now sent to all connected mesh peers (not just audience). Auto-sync on follow — following a peer triggers immediate post pull + engagement fetch. Popover UI — notifications settings, network diagnostics, and message threads now open as popovers. Notification settings — per-key settings table in SQLite, configurable message/post/nearby notifications with JS Notification API. Tiered DM polling — smart message refresh based on conversation recency. Reaction display — posts show top 5 most popular emoji + total response count. UI cleanup — removed Suggested Peers and Find Nearby sections, placeholder text changed to “How’s it goin?”, clickable node IDs in activity log.</p>
|
||||
<p style="margin-top: 0.5rem;"><strong>v0.3.3</strong> (2026-03-16): Connection rate limiting — incoming auth failures rate-limited per source IP (3 attempts, exponential backoff to ~256s). Schema versioning — PRAGMA user_version tracks DB version with migration framework. N2/N3 freshness — TTL 7d→5h, full N1/N2 re-broadcast every 4h, startup sweep clears stale entries. Bootstrap isolation recovery — 24h check verifies bootstrap is in N1/N2/N3, reconnects + sticky N1 advertisement if absent. IPv6 HTTP address fix — nodes advertise actual public IPv6 (not 0.0.0.0) for share link redirects. Upstream tracking — post_upstream table records post source for engagement diff routing toward author. Video preload fix — share links and in-app videos use preload=auto. Following Online/Offline split. DM filter from My Posts. Any-type file attachments with download prompt + trust warning. Image lightbox. Audio player.</p>
|
||||
<p><strong>v0.3.2</strong> (2026-03-14): Bidirectional engagement propagation — BlobHeaderDiff flows upstream + downstream through CDN tree. Auto downstream registration on pull sync/push notification. TCP hole punch protocol (TcpPunchRequest/Result 0xD6/0xD7). Tiered web serving (redirect → TCP punch → QUIC proxy). Video playback fix (asset protocol + blob URL fallback). On-demand blob fetch for synced posts missing blob data.</p>
|
||||
<p><strong>v0.3.1</strong> (2026-03-13): Share links + QUIC proxy + content search. Share link format: <code>itsgoin.net/p/<postid_hex>/<author_nodeid_hex></code> — simple, no host encoding needed. itsgoin.net web handler acts as QUIC proxy: receives browser request, searches the network for the post, fetches it on-demand via PostFetch (0xD4/0xD5), renders HTML, serves to browser. No permanent storage of fetched content. Extended worm search — <code>WormQuery</code> now carries optional <code>post_id</code> and <code>blob_id</code> fields for unified node/post/blob search. Each peer checks local storage, CDN downstream tree (up to 100 hosts per post), and blob store. <code>WormResponse</code> gains <code>post_holder</code> and <code>blob_holder</code> fields. Nova fan-out pattern — burst peers include one N2 wide referral; referred peer does its own 101-burst, reaching ~10K nodes with ~202 relay hops. PostFetch (0xD4/0xD5) — lightweight single-post retrieval after worm finds a holder, much lighter than full PullSync. itsgoin.net node deployed as anchor + web handler (<code>--web 8080</code>). “Unavailable” page with honest network model explanation + install CTA. Universal Links / App Links planned for native app interception. | Engagement sync — pull sync now fetches reactions, comments, and policies via BlobHeaderRequest/Response after every sync. Profile push fix — profile updates now sent to all connected mesh peers (not just audience). Auto-sync on follow — following a peer triggers immediate post pull + engagement fetch. Popover UI — notifications settings, network diagnostics, and message threads now open as popovers. Notification settings — per-key settings table in SQLite, configurable message/post/nearby notifications with JS Notification API. Tiered DM polling — smart message refresh based on conversation recency. Reaction display — posts show top 5 most popular emoji + total response count. UI cleanup — removed Suggested Peers and Find Nearby sections, placeholder text changed to “How’s it goin?”, clickable node IDs in activity log.</p>
|
||||
<p><strong>v0.3.0</strong> (2026-03-12): Full rename distsoc → ItsGoin. ALPN, crypto contexts, data paths, Android package ID all changed. Clean break — incompatible with prior versions.</p>
|
||||
<p><strong>v0.2.11</strong> (2026-03-12): Engagement system — reactions (public + private encrypted via X25519 DH + ChaCha20-Poly1305), inline comments with ed25519 signatures, author-controlled comment/react policies (audience-only, public, none), blocklist enforcement. CDN tree for all posts — new <code>post_downstream</code> table (keyed by PostId, max 100 peers) gives every post a propagation tree; <code>PostDownstreamRegister</code> (0xD3) sent when any peer stores a post. 4 new wire messages: BlobHeaderDiff (0xD0) for incremental engagement propagation, BlobHeaderRequest/Response (0xD1/0xD2), PostDownstreamRegister (0xD3). 6 new SQLite tables, 9 new IPC commands. Thread splitting — headers exceeding 16KB auto-split oldest comments into linked thread posts. Frontend: emoji picker, reaction pills, comment threads, policy selects in compose area.</p>
|
||||
<p><strong>v0.2.10</strong> (2026-03-12): Per-family NAT classification — IPv4 and IPv6 public reachability now detected independently. Previously, a public IPv6 address incorrectly set <code>has_public_v4=true</code>, causing nodes behind IPv4 NAT to skip hole punching. STUN now always runs (unless <code>--bind</code>) so IPv6-only anchors correctly classify their IPv4 NAT. Anchor advertised address fallback — anchors without <code>--bind</code> or UPnP now advertise their first public bound address (e.g. IPv6 SLAAC), so peers store them in <code>known_anchors</code> for preferential reconnection. Bootstrap anchor deprioritization — startup connection sequence now tries discovered (non-bootstrap) anchors first, falling back to hardcoded bootstrap anchors only when no discovered anchor is reachable. Reduces load on bootstrap infrastructure as the network grows.</p>
|
||||
|
|
@ -1027,6 +1029,50 @@ FAILURE: C → B → A: AnchorProbeResult { reachable: false }</code></pre
|
|||
<li><strong>Pull (safety net)</strong>: Every 5 minutes, the pull cycle requests <code>BlobHeaderRequest</code> (0xD1) with the local header timestamp. Peers respond with the full header only if theirs is newer. Additive merge — <code>store_reaction</code> upserts, <code>store_comment</code> inserts with ON CONFLICT DO NOTHING.</li>
|
||||
<li><strong>Planned</strong>: Pull engagement from both upstream and downstream peers to catch missed diffs from either direction.</li>
|
||||
</ul>
|
||||
|
||||
<h3>Connection rate limiting</h3>
|
||||
<p>Incoming QUIC connections that fail authentication are rate-limited per source IP to prevent CPU exhaustion from rogue or stale nodes:</p>
|
||||
<ul style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li><strong>First 3 failures</strong>: logged normally, connection attempts proceed</li>
|
||||
<li><strong>4+ failures</strong>: silently dropped with exponential backoff (2<sup>n-3</sup> seconds, capped at ~256s)</li>
|
||||
<li><strong>Successful connection</strong>: clears the failure count for that IP</li>
|
||||
<li><strong>Cleanup</strong>: stale entries removed every 60 seconds (pruned after 5 minutes of inactivity)</li>
|
||||
</ul>
|
||||
|
||||
<h3>N2/N3 freshness</h3>
|
||||
<ul style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li><strong>TTL</strong>: N2/N3 entries expire after <strong>5 hours</strong> (pruned during rebalance cycle)</li>
|
||||
<li><strong>Full state broadcast</strong>: Every <strong>4 hours</strong>, nodes re-broadcast their complete N1/N2 state (not just diffs) to catch any missed incremental updates</li>
|
||||
<li><strong>Disconnect cleanup</strong>: When a mesh peer disconnects, all N2/N3 entries they reported are immediately removed (<code>clear_peer_n2</code>/<code>clear_peer_n3</code>)</li>
|
||||
<li><strong>Startup sweep</strong>: On boot, all N2/N3 and mesh_peers entries are cleared — they're stale from the previous session and will be rebuilt as connections establish</li>
|
||||
</ul>
|
||||
|
||||
<h3>Bootstrap isolation recovery</h3>
|
||||
<p>Prevents network segments from becoming permanently isolated from the main network:</p>
|
||||
<ul style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li><strong>24-hour check</strong>: Starting 24 hours after boot, nodes verify the bootstrap anchor is within their N1/N2/N3 reach</li>
|
||||
<li><strong>If absent</strong>: reconnect to bootstrap, request referrals, connect to referred peers</li>
|
||||
<li><strong>Sticky N1</strong>: The bootstrap is added to a sticky N1 set for 24 hours, so mesh peers learn about it via routing diffs and can independently bridge back to the main network</li>
|
||||
<li><strong>Self-limiting</strong>: Once the bootstrap is in N3, the check passes and no action is taken. Goes silent as the network grows.</li>
|
||||
</ul>
|
||||
|
||||
<h3>Schema versioning</h3>
|
||||
<p>SQLite databases track their schema version via <code>PRAGMA user_version</code>. On startup:</p>
|
||||
<ul style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li>If the database version is older than <code>MIN_MIGRATABLE_VERSION</code>, the database is reset (preserving identity key)</li>
|
||||
<li>If older than the current version, data migrations run once and the version is bumped</li>
|
||||
<li>Schema-level changes (new tables, columns) are handled by <code>init_tables()</code> (<code>CREATE TABLE IF NOT EXISTS</code>) and <code>migrate()</code> (column-level <code>ALTER TABLE</code> checks)</li>
|
||||
</ul>
|
||||
|
||||
<h3>Upstream tracking (<code>post_upstream</code>)</h3>
|
||||
<p>Each post tracks which peer it was received from in the <code>post_upstream</code> table (post_id → peer_node_id). Set during pull sync and push notification. Used for:</p>
|
||||
<ul style="padding-left: 1.25rem; margin: 0.5rem 0; color: var(--text-muted);">
|
||||
<li>Engagement diff propagation toward the post author (hop-by-hop upstream)</li>
|
||||
<li>Future: N+10 identification in blob headers (upstream source's N+10)</li>
|
||||
</ul>
|
||||
|
||||
<h3>IPv6 HTTP address advertisement</h3>
|
||||
<p>Nodes with public IPv6 addresses advertise their actual routable address (from <code>endpoint.addr().ip_addrs()</code>) paired with their bound port, rather than the bind address (<code>0.0.0.0</code>). This enables direct browser-to-node HTTP serving for share links. Unroutable addresses (<code>0.0.0.0</code>, <code>127.x</code>) are filtered out in the tiered web serving redirect path.</p>
|
||||
</section>
|
||||
|
||||
<!-- 20. Encryption -->
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue