v0.4.0: Protocol v4 — header-driven sync, tiered engagement, multi-upstream
Protocol v4 sync overhaul: - Slim PullSyncRequest: per-author timestamps (since_ms) replace full post ID lists Request size O(follows) instead of O(posts). Backward-compatible via serde default. - Tiered pull frequency: 60s ticks, only syncs stale authors (4hr default) Full pull only on first tick (bootstrap). Most ticks skip — no stale authors. - Tiered engagement checks: frequency scales with content age 5min (<72h), 1hr (3-14d), 4hr (14-30d), 24hr (>30d) Single SQL query filters posts due for check. - Header-driven post discovery: ManifestPush triggers PostFetch for missing followed-author posts (capped 10 per manifest). CDN tree = notification system. - Multi-upstream (3 max): composite PK, priority ordering, engagement diffs sent to all upstreams, promote/remove on failure. DB schema: - follows.last_sync_ms — Self Last Encounter per author - posts.last_engagement_ms — last reaction/comment timestamp - posts.last_check_ms — last engagement check timestamp - post_upstream: single-row → 3-row with priority column Lock contention fixes: - get_blob_for_post: 3 locks → 1 - prefetch_blobs_from_peer: lock-free blob checks - fetch_engagement_from_peer: explicit lock release before I/O - serve_post: 4 locks → 2 (eliminated redundant queries) - run_replication_check: 2 locks → 1 - Badge cycle: N+2 IPC calls → 1 (get_badge_counts) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Scott Reimers
parent
1df00eebf8
commit
bbaacf9b6c
10 changed files with 489 additions and 100 deletions
|
|
@ -44,7 +44,7 @@
|
|||
<p>This is the canonical technical reference for ItsGoin. It describes the vision, the architecture, and the current state of every subsystem — with full implementation detail. This document is versioned; each update records what changed.</p>
|
||||
<div class="card" style="margin-top: 1rem;">
|
||||
<strong style="font-size: 0.85rem; text-transform: uppercase; letter-spacing: 0.05em;">Changelog</strong>
|
||||
<p style="margin-top: 0.5rem;"><strong>v0.4.0</strong> (planned): Protocol v4 — header-driven sync. ManifestPush as primary post notification. Slim PullSyncRequest (per-author timestamps, not full post ID list). Tiered engagement checks (5min/1hr/4hr/24hr by content age). Multi-upstream (3 max) with fallback chain. Auto-prefetch followed authors <90d. Self Last Encounter per-author tracking. Encrypted-but-not-for-us CDN caching. Serial engagement polling. ~90% bandwidth reduction for established nodes.</p>
|
||||
<p style="margin-top: 0.5rem;"><strong>v0.4.0</strong> (2026-03-21): Protocol v4 — header-driven sync. ManifestPush as primary post notification. Slim PullSyncRequest (per-author timestamps, not full post ID list). Tiered engagement checks (5min/1hr/4hr/24hr by content age). Multi-upstream (3 max) with fallback chain. Auto-prefetch followed authors <90d. Self Last Encounter per-author tracking. Encrypted-but-not-for-us CDN caching. Serial engagement polling. ~90% bandwidth reduction for established nodes.</p>
|
||||
<p><strong>v0.3.6</strong> (2026-03-20): Active CDN replication — all devices proactively replicate recent posts to peers (desktops > anchors > phones priority). ReplicationRequest/Response (0xE1/0xE2). Device roles (Intermittent/Available/Persistent) advertised in InitialExchange. Bandwidth budgets: replication (pull to cache) + delivery (serve requests), hourly auto-reset, phones 100MB/1GB, desktops 200MB/2GB, anchors 200MB/1GB. Cache management: 1GB default, configurable, eviction cycle activated with share-link priority boost. Engagement distribution fix — BlobHeader JSON rebuilt after diff ops. Tombstone system — deleted reactions/comments tombstoned, propagate via pull sync. Persistent notifications via seen_engagement/seen_messages tables. DOS hardening: fan-out cap (10), prefetch cap (20), downstream registration cap (50), delivery budget enforcement. Pull preference reordered: non-anchors first. Network indicator — header dot (black/red/yellow/green) + capability labels. Tab badges — contextual counts (new posts, engagement, online, unread). Message read tracking on open/close/send. Stats bar removed.</p>
|
||||
<p><strong>v0.3.5</strong> (2026-03-20): Private blob encryption — attachments on encrypted posts (Friends/Circle/Direct) now encrypted with same CEK as post text; public blobs unchanged; CID on ciphertext. Blob prefetch on sync — attachments eagerly fetched after post pull for offline availability. Crypto refactoring — extracted reusable primitives (encrypt/decrypt_bytes_with_cek, unwrap_cek_for_recipient, unwrap_group_cek). Intent-based post filtering — feed/myposts/messages filter on intentKind instead of encryption state. Blob decryption API (get_blob_for_post). Download filename sanitization. Encrypted receipt & comment slots — private posts carry noise-prefilled encrypted slots in BlobHeader for delivery/read/react receipts and private comments; CDN-propagated as opaque bytes; slot key derived from post CEK; 3 new BlobHeaderDiffOps (WriteReceiptSlot, WriteCommentSlot, AddCommentSlots). Message UI — DM delivery indicators (checkmark/double/blue/emoji), auto-seen on view, react button on messages.</p>
|
||||
<p><strong>v0.3.4</strong> (2026-03-18): Comment edit & delete with trust-based propagation. Native notifications via Tauri plugin (messages, posts, reactions, comments). Forward-compatible BlobHeaderDiffOp::Unknown variant. Following Online/Offline lightbox. Comment threading scoping fix. Dropdown text legibility fix. Mobile hamburger nav for website.</p>
|
||||
|
|
@ -1147,7 +1147,7 @@ FAILURE: C → B → A: AnchorProbeResult { reachable: false }</code></pre
|
|||
<p>DM conversations display delivery indicators: single checkmark (sent), double checkmark (delivered/on device), blue double checkmark (seen), emoji (reacted). Opening a conversation auto-marks incoming messages as seen. Messages have a react button for emoji responses.</p>
|
||||
</section>
|
||||
|
||||
<h3>Protocol v4: Header-Driven Sync <span class="badge badge-planned">Planned</span></h3>
|
||||
<h3>Protocol v4: Header-Driven Sync <span class="badge badge-complete">Complete</span></h3>
|
||||
<p>Major sync protocol revision that replaces the current pull-everything-from-everyone model with header-driven discovery, per-author tracking, and tiered engagement polling. Reduces bandwidth by ~90% for established nodes.</p>
|
||||
|
||||
<h4>Core principle: headers as notification</h4>
|
||||
|
|
|
|||
|
|
@ -25,16 +25,16 @@
|
|||
<section>
|
||||
<h1 style="font-size: 2rem; font-weight: 800; letter-spacing: -0.03em; margin-bottom: 0.25rem;">Download ItsGoin</h1>
|
||||
<p>Available for Android and Linux. Free and open source.</p>
|
||||
<p style="color: var(--text-muted); font-size: 0.85rem;">Version 0.3.6 — March 15, 2026</p>
|
||||
<p style="color: var(--text-muted); font-size: 0.85rem;">Version 0.4.0 — March 15, 2026</p>
|
||||
|
||||
<div class="downloads">
|
||||
<a href="itsgoin-0.3.6.apk" class="download-btn btn-android">
|
||||
<a href="itsgoin-0.4.0.apk" class="download-btn btn-android">
|
||||
Android APK
|
||||
<span class="sub">v0.3.6</span>
|
||||
<span class="sub">v0.4.0</span>
|
||||
</a>
|
||||
<a href="itsgoin_0.3.6_amd64.AppImage" class="download-btn btn-linux">
|
||||
<a href="itsgoin_0.4.0_amd64.AppImage" class="download-btn btn-linux">
|
||||
Linux AppImage
|
||||
<span class="sub">v0.3.6</span>
|
||||
<span class="sub">v0.4.0</span>
|
||||
</a>
|
||||
</div>
|
||||
</section>
|
||||
|
|
@ -46,7 +46,7 @@
|
|||
<h3 style="color: var(--accent);">Android</h3>
|
||||
<ol class="steps">
|
||||
<li><strong>Download the APK</strong> — Tap the button above. Your browser may warn that this type of file can be harmful — tap <strong>Download anyway</strong>.</li>
|
||||
<li><strong>Open the file</strong> — When the download finishes, tap the notification or find <code>itsgoin-0.3.6.apk</code> in your Downloads folder and tap it.</li>
|
||||
<li><strong>Open the file</strong> — When the download finishes, tap the notification or find <code>itsgoin-0.4.0.apk</code> in your Downloads folder and tap it.</li>
|
||||
<li><strong>Allow installation</strong> — Android will ask you to allow installs from this source. Tap <strong>Settings</strong>, toggle <strong>"Allow from this source"</strong>, then go back and tap <strong>Install</strong>.</li>
|
||||
<li><strong>Launch the app</strong> — Once installed, tap <strong>Open</strong> or find ItsGoin in your app drawer.</li>
|
||||
</ol>
|
||||
|
|
@ -59,8 +59,8 @@
|
|||
<h3 style="color: var(--green);">Linux (AppImage)</h3>
|
||||
<ol class="steps">
|
||||
<li><strong>Download the AppImage</strong> — Click the button above to download.</li>
|
||||
<li><strong>Make it executable</strong> — Open a terminal and run:<br><code>chmod +x itsgoin_0.3.6_amd64.AppImage</code></li>
|
||||
<li><strong>Run it</strong> — Double-click the file, or from the terminal:<br><code>./itsgoin_0.3.6_amd64.AppImage</code></li>
|
||||
<li><strong>Make it executable</strong> — Open a terminal and run:<br><code>chmod +x itsgoin_0.4.0_amd64.AppImage</code></li>
|
||||
<li><strong>Run it</strong> — Double-click the file, or from the terminal:<br><code>./itsgoin_0.4.0_amd64.AppImage</code></li>
|
||||
</ol>
|
||||
<div class="note">
|
||||
<strong>Note:</strong> If it doesn't launch, you may need to install FUSE:<br><code>sudo apt install libfuse2</code> (Debian/Ubuntu) or <code>sudo dnf install fuse</code> (Fedora).
|
||||
|
|
@ -71,6 +71,17 @@
|
|||
<section>
|
||||
<h2>Changelog</h2>
|
||||
<div class="changelog">
|
||||
<div class="changelog-date">v0.4.0 — March 21, 2026</div>
|
||||
<ul>
|
||||
<li><strong>Protocol v4: Header-driven sync</strong> — Major sync protocol revision. ManifestPush now triggers post discovery from CDN tree headers. Bandwidth reduced ~90% for established nodes.</li>
|
||||
<li><strong>Slim PullSyncRequest</strong> — Per-author timestamps replace full post ID lists. Request size drops from O(posts) to O(follows). Backward-compatible with v3 peers.</li>
|
||||
<li><strong>Tiered pull frequency</strong> — Pull cycle checks every 60s but only syncs stale authors (4-hour default). Full pull only on first tick. Most ticks do nothing.</li>
|
||||
<li><strong>Tiered engagement checks</strong> — Engagement polling frequency scales with content age: 5min (<72h), 1hr (3-14d), 4hr (14-30d), 24hr (>30d). Single SQL query filters due posts.</li>
|
||||
<li><strong>Header-driven post discovery</strong> — ManifestPush triggers PostFetch for missing followed-author posts (capped at 10 per manifest). CDN tree becomes the notification system.</li>
|
||||
<li><strong>Multi-upstream (3 max)</strong> — Posts track up to 3 upstream sources with priority ordering. Engagement diffs sent to all upstreams. Fallback on upstream failure.</li>
|
||||
<li><strong>Lock contention fixes</strong> — 6 hot paths optimized: get_blob_for_post (3→1 locks), prefetch (lock-free blob checks), serve_post (4→2 locks), badge cycle (N+2→1 IPC call).</li>
|
||||
</ul>
|
||||
|
||||
<div class="changelog-date">v0.3.6 — March 20, 2026</div>
|
||||
<ul>
|
||||
<li><strong>Network indicator</strong> — Header shows connection status dot (black/red/yellow/green for 0/1/2-10/11+ connections) with capability labels (Public, Server).</li>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue