From ce176a22994cc6828e64d730a7dd8dc7c164ef3c Mon Sep 17 00:00:00 2001 From: Scott Reimers Date: Mon, 16 Mar 2026 18:47:38 -0400 Subject: [PATCH] Design doc: v0.3.2 + v0.3.3 changelog, rate limiting, N2/N3 freshness, bootstrap recovery, schema versioning docs Co-Authored-By: Claude Opus 4.6 (1M context) --- website/design.html | 48 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 47 insertions(+), 1 deletion(-) diff --git a/website/design.html b/website/design.html index 044ff6b..d673665 100644 --- a/website/design.html +++ b/website/design.html @@ -43,7 +43,9 @@

This is the canonical technical reference for ItsGoin. It describes the vision, the architecture, and the current state of every subsystem — with full implementation detail. This document is versioned; each update records what changed.

Changelog -

v0.3.1 (2026-03-13): Share links + QUIC proxy + content search. Share link format: itsgoin.net/p/<postid_hex>/<author_nodeid_hex> — simple, no host encoding needed. itsgoin.net web handler acts as QUIC proxy: receives browser request, searches the network for the post, fetches it on-demand via PostFetch (0xD4/0xD5), renders HTML, serves to browser. No permanent storage of fetched content. Extended worm search — WormQuery now carries optional post_id and blob_id fields for unified node/post/blob search. Each peer checks local storage, CDN downstream tree (up to 100 hosts per post), and blob store. WormResponse gains post_holder and blob_holder fields. Nova fan-out pattern — burst peers include one N2 wide referral; referred peer does its own 101-burst, reaching ~10K nodes with ~202 relay hops. PostFetch (0xD4/0xD5) — lightweight single-post retrieval after worm finds a holder, much lighter than full PullSync. itsgoin.net node deployed as anchor + web handler (--web 8080). “Unavailable” page with honest network model explanation + install CTA. Universal Links / App Links planned for native app interception. | Engagement sync — pull sync now fetches reactions, comments, and policies via BlobHeaderRequest/Response after every sync. Profile push fix — profile updates now sent to all connected mesh peers (not just audience). Auto-sync on follow — following a peer triggers immediate post pull + engagement fetch. Popover UI — notifications settings, network diagnostics, and message threads now open as popovers. Notification settings — per-key settings table in SQLite, configurable message/post/nearby notifications with JS Notification API. Tiered DM polling — smart message refresh based on conversation recency. Reaction display — posts show top 5 most popular emoji + total response count. UI cleanup — removed Suggested Peers and Find Nearby sections, placeholder text changed to “How’s it goin?”, clickable node IDs in activity log.

+

v0.3.3 (2026-03-16): Connection rate limiting — incoming auth failures rate-limited per source IP (3 attempts, exponential backoff to ~256s). Schema versioning — PRAGMA user_version tracks DB version with migration framework. N2/N3 freshness — TTL 7d→5h, full N1/N2 re-broadcast every 4h, startup sweep clears stale entries. Bootstrap isolation recovery — 24h check verifies bootstrap is in N1/N2/N3, reconnects + sticky N1 advertisement if absent. IPv6 HTTP address fix — nodes advertise actual public IPv6 (not 0.0.0.0) for share link redirects. Upstream tracking — post_upstream table records post source for engagement diff routing toward author. Video preload fix — share links and in-app videos use preload=auto. Following Online/Offline split. DM filter from My Posts. Any-type file attachments with download prompt + trust warning. Image lightbox. Audio player.

+

v0.3.2 (2026-03-14): Bidirectional engagement propagation — BlobHeaderDiff flows upstream + downstream through CDN tree. Auto downstream registration on pull sync/push notification. TCP hole punch protocol (TcpPunchRequest/Result 0xD6/0xD7). Tiered web serving (redirect → TCP punch → QUIC proxy). Video playback fix (asset protocol + blob URL fallback). On-demand blob fetch for synced posts missing blob data.

+

v0.3.1 (2026-03-13): Share links + QUIC proxy + content search. Share link format: itsgoin.net/p/<postid_hex>/<author_nodeid_hex> — simple, no host encoding needed. itsgoin.net web handler acts as QUIC proxy: receives browser request, searches the network for the post, fetches it on-demand via PostFetch (0xD4/0xD5), renders HTML, serves to browser. No permanent storage of fetched content. Extended worm search — WormQuery now carries optional post_id and blob_id fields for unified node/post/blob search. Each peer checks local storage, CDN downstream tree (up to 100 hosts per post), and blob store. WormResponse gains post_holder and blob_holder fields. Nova fan-out pattern — burst peers include one N2 wide referral; referred peer does its own 101-burst, reaching ~10K nodes with ~202 relay hops. PostFetch (0xD4/0xD5) — lightweight single-post retrieval after worm finds a holder, much lighter than full PullSync. itsgoin.net node deployed as anchor + web handler (--web 8080). “Unavailable” page with honest network model explanation + install CTA. Universal Links / App Links planned for native app interception. | Engagement sync — pull sync now fetches reactions, comments, and policies via BlobHeaderRequest/Response after every sync. Profile push fix — profile updates now sent to all connected mesh peers (not just audience). Auto-sync on follow — following a peer triggers immediate post pull + engagement fetch. Popover UI — notifications settings, network diagnostics, and message threads now open as popovers. Notification settings — per-key settings table in SQLite, configurable message/post/nearby notifications with JS Notification API. Tiered DM polling — smart message refresh based on conversation recency. Reaction display — posts show top 5 most popular emoji + total response count. UI cleanup — removed Suggested Peers and Find Nearby sections, placeholder text changed to “How’s it goin?”, clickable node IDs in activity log.

v0.3.0 (2026-03-12): Full rename distsoc → ItsGoin. ALPN, crypto contexts, data paths, Android package ID all changed. Clean break — incompatible with prior versions.

v0.2.11 (2026-03-12): Engagement system — reactions (public + private encrypted via X25519 DH + ChaCha20-Poly1305), inline comments with ed25519 signatures, author-controlled comment/react policies (audience-only, public, none), blocklist enforcement. CDN tree for all posts — new post_downstream table (keyed by PostId, max 100 peers) gives every post a propagation tree; PostDownstreamRegister (0xD3) sent when any peer stores a post. 4 new wire messages: BlobHeaderDiff (0xD0) for incremental engagement propagation, BlobHeaderRequest/Response (0xD1/0xD2), PostDownstreamRegister (0xD3). 6 new SQLite tables, 9 new IPC commands. Thread splitting — headers exceeding 16KB auto-split oldest comments into linked thread posts. Frontend: emoji picker, reaction pills, comment threads, policy selects in compose area.

v0.2.10 (2026-03-12): Per-family NAT classification — IPv4 and IPv6 public reachability now detected independently. Previously, a public IPv6 address incorrectly set has_public_v4=true, causing nodes behind IPv4 NAT to skip hole punching. STUN now always runs (unless --bind) so IPv6-only anchors correctly classify their IPv4 NAT. Anchor advertised address fallback — anchors without --bind or UPnP now advertise their first public bound address (e.g. IPv6 SLAAC), so peers store them in known_anchors for preferential reconnection. Bootstrap anchor deprioritization — startup connection sequence now tries discovered (non-bootstrap) anchors first, falling back to hardcoded bootstrap anchors only when no discovered anchor is reachable. Reduces load on bootstrap infrastructure as the network grows.

@@ -1027,6 +1029,50 @@ FAILURE: C → B → A: AnchorProbeResult { reachable: false }Pull (safety net): Every 5 minutes, the pull cycle requests BlobHeaderRequest (0xD1) with the local header timestamp. Peers respond with the full header only if theirs is newer. Additive merge — store_reaction upserts, store_comment inserts with ON CONFLICT DO NOTHING.
  • Planned: Pull engagement from both upstream and downstream peers to catch missed diffs from either direction.
    • + +

    Connection rate limiting

    +

    Incoming QUIC connections that fail authentication are rate-limited per source IP to prevent CPU exhaustion from rogue or stale nodes:

    +
      +
    • First 3 failures: logged normally, connection attempts proceed
      • +
    • 4+ failures: silently dropped with exponential backoff (2n-3 seconds, capped at ~256s)
      • +
    • Successful connection: clears the failure count for that IP
      • +
    • Cleanup: stale entries removed every 60 seconds (pruned after 5 minutes of inactivity)
      • +
    + +

    N2/N3 freshness

    +
      +
    • TTL: N2/N3 entries expire after 5 hours (pruned during rebalance cycle)
      • +
    • Full state broadcast: Every 4 hours, nodes re-broadcast their complete N1/N2 state (not just diffs) to catch any missed incremental updates
      • +
    • Disconnect cleanup: When a mesh peer disconnects, all N2/N3 entries they reported are immediately removed (clear_peer_n2/clear_peer_n3)
      • +
    • Startup sweep: On boot, all N2/N3 and mesh_peers entries are cleared — they're stale from the previous session and will be rebuilt as connections establish
      • +
    + +

    Bootstrap isolation recovery

    +

    Prevents network segments from becoming permanently isolated from the main network:

    +
      +
    • 24-hour check: Starting 24 hours after boot, nodes verify the bootstrap anchor is within their N1/N2/N3 reach
      • +
    • If absent: reconnect to bootstrap, request referrals, connect to referred peers
      • +
    • Sticky N1: The bootstrap is added to a sticky N1 set for 24 hours, so mesh peers learn about it via routing diffs and can independently bridge back to the main network
      • +
    • Self-limiting: Once the bootstrap is in N3, the check passes and no action is taken. Goes silent as the network grows.
      • +
    + +

    Schema versioning

    +

    SQLite databases track their schema version via PRAGMA user_version. On startup:

    +
      +
    • If the database version is older than MIN_MIGRATABLE_VERSION, the database is reset (preserving identity key)
      • +
    • If older than the current version, data migrations run once and the version is bumped
      • +
    • Schema-level changes (new tables, columns) are handled by init_tables() (CREATE TABLE IF NOT EXISTS) and migrate() (column-level ALTER TABLE checks)
      • +
    + +

    Upstream tracking (post_upstream)

    +

    Each post tracks which peer it was received from in the post_upstream table (post_id → peer_node_id). Set during pull sync and push notification. Used for:

    +
      +
    • Engagement diff propagation toward the post author (hop-by-hop upstream)
      • +
    • Future: N+10 identification in blob headers (upstream source's N+10)
      • +
    + +

    IPv6 HTTP address advertisement

    +

    Nodes with public IPv6 addresses advertise their actual routable address (from endpoint.addr().ip_addrs()) paired with their bound port, rather than the bind address (0.0.0.0). This enables direct browser-to-node HTTP serving for share links. Unroutable addresses (0.0.0.0, 127.x) are filtered out in the tiered web serving redirect path.