Commit graph

15 commits

Author SHA1 Message Date
Scott Reimers
975e7b9bfe Phase 3 (0.6.2-beta): merged pull + recipient-match
A non-follower can now receive DMs addressed to them via a normal pull
cycle, with no distinguishable "searching for DMs" traffic pattern —
the pull query is a uniform list of NodeIds that the server matches
against both authors and wrapped-key recipients.

Schema (migrations on first 0.6.2 launch):
- New post_recipients(post_id, recipient) index table with index on
  recipient column
- Seed migration scans existing encrypted posts, extracts recipients
  and group members from visibility JSON, populates the index

Write path:
- store_post_with_visibility / store_post_with_intent populate
  post_recipients on successful insert
- update_post_visibility rebuilds the index for the updated post
- apply_delete cascade-removes post_recipients entries

Server pull handler (should_send_post):
- Renamed semantic: requester_follows → query_list. Contains every
  NodeId the client wants posts for (follows + own NodeId).
- Encrypted/GroupEncrypted posts match if ANY recipient / group
  member is in query_list (previously only if == requester).
- Wire protocol unchanged — the same PullSyncRequestPayload.follows
  field now carries both follow targets and own NodeId indistinguishably.

Client pull paths (all three call sites in network.rs + connection.rs):
- Always append own NodeId to the query list before sending pull sync.

Storage helper:
- get_post_ids_for_recipients(nids) — bulk IN-match using the
  idx_post_recipients_recipient index, for future SQL-side filtering.

Tests:
- should_send_post's recipient tests updated to pass query_list
  containing requester (matches new contract).
- Added encrypted_post_matches_via_query_list_even_for_third_party_recipient
  proving the server matches on any recipient in the list, not just
  the requester itself.

All 111 core tests pass. Smoke-tested end-to-end: A posts encrypted
DM to B; B connects + syncs; B decrypts and reads DM; both sides'
post_recipients correctly populated on store.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 22:15:55 -04:00
Scott Reimers
5d9ba22427 Phase 2e (0.6.1-beta): drop legacy upstream/downstream tables
The file_holders table is now the only tracker of per-file peer
relationships. post_upstream, post_downstream, blob_upstream, and
blob_downstream are dropped at first launch after the seed migration
copies any existing entries.

Schema:
- DROP TABLE IF EXISTS on all four legacy tables after seeding
- Seed migration guards with sqlite_master table_exists check so fresh
  installs don't crash trying to read non-existent sources
- Remove CREATE TABLE statements for the four tables from init
- Remove Protocol v4 Phase 6 post_upstream priority migration (dead)
- Remove blob_upstream preferred_tree column migration (dead)

Rust:
- Remove add/get/remove post_upstream, post_downstream,
  blob_upstream, blob_downstream methods
- Remove get_blob_upstream_preferred_tree / update variant
- Rewrite get_eviction_candidates's downstream_count subquery to
  count file_holders entries
- Rewrite apply_delete's cascade cleanup to clear file_holders
  instead of post_upstream/post_downstream
- cleanup_cdn_for_blob now clears file_holders for the CID

Callers:
- All dual-write sites in connection.rs and node.rs now do
  touch_file_holder only (legacy writes removed)
- get_stale_manifests replaced with get_stale_manifest_cids; caller
  in node.rs picks a refresh source from file_holders

Tests:
- Remove blob_upstream_crud, blob_downstream_crud_and_limit,
  blob_upstream_preferred_tree, remove_blob_upstream,
  post_downstream_crud
- Add file_holders_lru_cap and file_holders_direction_promotion tests

All 110 core tests passing. Workspace compiles clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 21:42:15 -04:00
Scott Reimers
60463d1817 Phase 2d (0.6.1-beta): route manifest + blob ops through file_holders
Switch ALL propagation-decision reads to the flat holder set.

push_manifest_to_downstream now targets file_holders instead of
blob_downstream. ManifestPush receive-side relay likewise — known
holders fan out to up to 5 most-recent peers instead of a directional
tree.

Blob delete notices: single flat fan-out to file_holders; the legacy
upstream_node tree-healing field is emitted as None (wire-stable via
serde default) and ignored on receive — the post-0.6 flat model
doesn't need sender-role distinction. send_blob_delete_notices keeps
its Option<&Upstream> parameter as an unused placeholder for signature
stability with the call sites in this commit.

Other reads migrated:
- blob fetch cascade: step 2 now tries "known holders" (up to 5)
  instead of a single upstream
- manifest refresh: downstream_count reported from file_holder_count
- web/http post holder enumeration
- Worm search post/blob holder fallback (both connection.rs paths)
- DeleteRecord fan-out rewires to file_holders
- Under-replication replication check: < 2 holders

Storage additions:
- get_file_holder_count(file_id)
- remove_file_holder(file_id, peer_id)

Legacy upstream/downstream writes are still happening from Phase 2b;
those + the tables themselves go in 2e.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 21:09:45 -04:00
Scott Reimers
1658762a68 Phase 2a (0.6.1-beta): add file_holders table + legacy seed migration
New flat per-file holder set replaces the directional upstream/downstream
trees. Keyed by 32-byte content-addressed file_id (works for both PostId
and blob CID). LRU-capped at 5 holders per file on touch.

- HolderDirection enum (Sent/Received/Both) — tracked for potential
  reuse, not load-bearing for propagation
- touch_file_holder / get_file_holders / delete_file_holders
- seed_file_holders_from_legacy: one-time idempotent seed from
  post_upstream, post_downstream, blob_upstream, blob_downstream so
  users upgrading from 0.6.0 don't start with empty holder sets

Table and methods land here; call-site refactor and legacy-table drop
follow in subsequent commits within this phase.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 20:52:30 -04:00
Scott Reimers
288b53ffb1 Feed pagination, duplicate identity detection, pkarr leak fix, Android SAF
Feed pagination:
- Cursor-based pagination: get_feed_page/get_all_posts_page (20 posts/page)
- Batched engagement queries (3 bulk SQL queries instead of 4 per post)
- IntersectionObserver for infinite scroll (sentinel at midpoint)
- Viewport-based media loading (blobs only load when post enters view)
- Pre-fetch next page immediately after current page renders

Duplicate identity detection:
- Anchor detects when a NodeId is already mesh-connected during initial
  exchange and sets duplicate_active flag in response
- Client skips sync tasks when duplicate detected
- Frontend shows red warning banner

Privacy:
- Fixed pkarr leak: clear_address_lookup() removes default dns.iroh.link
  publishing. Only mDNS (local network) discovery enabled.

Android:
- SAF integration via tauri-plugin-android-fs: exports open native "Save As"
  dialog so users can save to Downloads/Drive/etc.
- Download/export paths use app data dir on Android (writable)
- File picker gated behind desktop cfg (blocking_pick not on Android)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 15:35:23 -04:00
Scott Reimers
7b9edc13da Fix blocking_lock panic inside async runtime, reduce pool to 4 connections
spawn_with_arc used blocking_lock() which panics inside tokio runtime.
Changed to async lock().await. Reduced StoragePool from 8 to 4 connections
for Android compatibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 22:10:50 -04:00
Scott Reimers
43adbbdf7d v0.4.3: Lock contention overhaul, StoragePool, mobile bottom nav, text scaling
Eliminate all conn_mgr lock holds during network I/O across 14 actor commands
and bi-stream handlers. PostFetch, TcpPunch, PullFromPeer, FetchEngagement,
ResolveAddress, AnchorProbe use brief locks for data gathering only. WormLookup,
ContentSearch, WormQuery use connection snapshots for lock-free cascade fan-out.
RelayIntroduce extracts forwarding data under brief lock, does I/O outside.
BlobRequest, PostFetchRequest, ManifestRefresh use Arc clones instead of conn_mgr
lock. ConnectionActor hoists shared Arcs (storage, blob_store, endpoint) for
lock-free access. ResolveAddress adds 5s per-query timeout (was unbounded).

Initial exchange failure now aborts mesh upgrade (was silently continuing with
broken connection). connect_to_peer/connect_to_anchor use consistent 15s timeout.
Rebalance connects outside the lock via pending_connects pattern.

StoragePool: 8 concurrent SQLite connections in WAL mode replace single
Mutex<Storage>. Reads run fully parallel; writes serialize at SQLite level only.
PRAGMA busy_timeout=5000 for graceful write contention.

Mobile bottom nav bar (<=768px) with icon tabs. Text sizes: XS/S/M/L/XL
(75%/100%/125%/150%/200%), default M. localStorage persistence for instant
restore. Toast repositioned above mobile nav.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 21:35:38 -04:00
Scott Reimers
6004cae8a8 v0.4.2: Welcome screen, status ticker, notifications, text scaling, networking fixes
Welcome screen with staggered counters while backend bootstraps. Header status
ticker for new posts/messages/reactions/comments/connection changes. Notification
fallback chain (Tauri plugin → Web API → notify-rust). Responsive text scaling
(Small/Normal/Large, persisted). Diagnostics moved to popover with on-demand
connections. Share details lightbox with QR code. Connect string prefers external
address. Stale N1 fix (disconnected routes excluded). Replication handler actively
fetches posts+blobs from requester. Hole punch registers remote address for relay.
Replication semaphore (3 concurrent). Peer labels show truncated node ID.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 14:15:49 -04:00
Scott Reimers
bb6f2b64b0 v0.4.1: Security hardening, lock contention fixes, data cleanup
Security:
- Reaction signatures: ed25519 sign/verify (sign_reaction, verify_reaction_signature)
  Backward-compatible — unsigned reactions from old nodes still accepted
- Comment signature verification: verify_comment_signature now called on receipt
- Reaction removal authorization: only reactor or post author can remove
- BlobHeader author verification: lookup actual author from storage, don't trust payload

Lock contention (4 fixes):
- ManifestPush discovery: cm lock released before PostFetch I/O
- Pull request handler: load under lock, filter without lock, brief re-lock for is_deleted
- Pull sender: split into two brief locks (store posts, then batch upstream+sync)
- Engagement checker: batch all chunk results, single lock for writes

Data cleanup:
- Post deletion cleans post_downstream, post_upstream, seen_engagement tables
- Added TODO-hardening.md documenting remaining DOS/security/lock/data issues

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 19:30:38 -04:00
Scott Reimers
bbaacf9b6c v0.4.0: Protocol v4 — header-driven sync, tiered engagement, multi-upstream
Protocol v4 sync overhaul:
- Slim PullSyncRequest: per-author timestamps (since_ms) replace full post ID lists
  Request size O(follows) instead of O(posts). Backward-compatible via serde default.
- Tiered pull frequency: 60s ticks, only syncs stale authors (4hr default)
  Full pull only on first tick (bootstrap). Most ticks skip — no stale authors.
- Tiered engagement checks: frequency scales with content age
  5min (<72h), 1hr (3-14d), 4hr (14-30d), 24hr (>30d)
  Single SQL query filters posts due for check.
- Header-driven post discovery: ManifestPush triggers PostFetch for missing
  followed-author posts (capped 10 per manifest). CDN tree = notification system.
- Multi-upstream (3 max): composite PK, priority ordering, engagement diffs
  sent to all upstreams, promote/remove on failure.

DB schema:
- follows.last_sync_ms — Self Last Encounter per author
- posts.last_engagement_ms — last reaction/comment timestamp
- posts.last_check_ms — last engagement check timestamp
- post_upstream: single-row → 3-row with priority column

Lock contention fixes:
- get_blob_for_post: 3 locks → 1
- prefetch_blobs_from_peer: lock-free blob checks
- fetch_engagement_from_peer: explicit lock release before I/O
- serve_post: 4 locks → 2 (eliminated redundant queries)
- run_replication_check: 2 locks → 1
- Badge cycle: N+2 IPC calls → 1 (get_badge_counts)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 16:13:45 -04:00
Scott Reimers
a7e632de88 v0.3.6: Active CDN replication, device roles, budgets, tombstones, engagement fix, DOS hardening
Active CDN replication:
- All devices proactively replicate recent posts (<72h, <2 replicas) to peers
- Target priority: desktops (300) > anchors (200) > phones (100) + cache_pressure
- ReplicationRequest/Response (0xE1/0xE2) wire messages
- 10-min cycle, 2-min initial delay, cap 20 posts per request
- Graceful with small networks (1 peer = 1 replica, 0 peers = silent skip)

Device roles & budgets:
- Intermittent (phone), Available (desktop), Persistent (anchor)
- Advertised in InitialExchange, stored per-peer
- Replication budget: phones 100MB/hr, desktops/anchors 200MB/hr
- Delivery budget: phones 1GB/hr, desktops 2GB/hr, anchors 1GB/hr
- Hourly auto-reset, enforcement on blob serving

Cache management:
- 1GB default cache limit, configurable in settings UI
- Eviction cycle activated (was implemented but never started)
- Share-link priority boost (+100 for 3+ downstream)
- Cache pressure score (0-255) for replication targeting

Engagement distribution fix:
- BlobHeader JSON rebuilt after BlobHeaderDiff ops
- Previously reactions/comments stored in tables but header stayed stale

Tombstone system:
- deleted_at column on reactions and comments
- Tombstones propagate through pull sync (additive merge respects timestamps)
- UI queries filter WHERE deleted_at IS NULL

Persistent notifications:
- seen_engagement and seen_messages tables replace in-memory Sets
- Only notify on genuinely unseen content, survives restarts

DOS hardening:
- BlobHeaderDiff fan-out: single batched task, max 10 concurrent via JoinSet
- Blob prefetch: cap 20 per cycle, newest first
- PostDownstreamRegister: cap 50 per sync
- Delivery budget enforcement on BlobRequest handler
- Pull preference: non-anchors first to preserve anchor delivery budget

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 21:00:28 -04:00
Scott Reimers
a41b11c0b8 v0.3.5: Private blob encryption, blob prefetch, intent-based filtering, crypto refactoring
Private blob encryption:
- Encrypted posts (Friends/Circle/Direct) now encrypt attachment blobs with same CEK
- Public blobs unchanged, CID computed on ciphertext for private
- decrypt_blob_for_post/get_blob_for_post for transparent decryption on retrieval

Blob prefetch:
- Pull cycle and sync_with eagerly fetch missing blobs after post sync
- prefetch_blobs_from_peer scans for missing attachments, fetches via fallback chain
- Runs outside conn_mgr lock at Node level

Crypto refactoring:
- Extracted: encrypt/decrypt_bytes_with_cek, wrap/unwrap_cek_for_recipients
- unwrap_cek_for_recipient, unwrap_group_cek, random_cek
- encrypt_post_with_cek, encrypt_post_for_group_with_cek variants
- All existing functions refactored to delegate, 19 crypto tests pass

Intent-based filtering:
- intent_kind field on PostDto ("public"/"friends"/"circle"/"direct"/"unknown")
- Feed/MyPosts filter on intentKind !== 'direct' instead of visibility
- Messages filter with backward-compatible fallback for pre-intent posts
- get_post_intent storage method

IPC updates:
- resolve_blob_data helper using get_blob_for_post with network fallback
- sanitize_download_filename prevents path traversal
- get_blob_path accepts optional post_id_hex

Website:
- Mobile hamburger nav on all pages
- Mesh/Non-mesh N1 labels in network diagnostics

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 12:44:07 -04:00
Scott Reimers
0abc244ee9 v0.3.4: Comment edit/delete, native notifications, forward-compatible protocol, UI fixes
Comment edit & delete:
- EditComment/DeleteComment BlobHeaderDiffOps with upstream+downstream propagation
- Trust-based: comment author can edit/delete, post author can delete
- Storage: edit_comment(), delete_comment() methods
- Frontend: inline edit (Enter/Escape), delete with confirm

Native notifications:
- tauri-plugin-notification for system notifications on all platforms
- Triggers for messages, new posts, reactions, and comments
- notif_reacts setting added, button-group toggles replace dropdowns
- _notifReady flag prevents startup spam

Protocol hardening:
- BlobHeaderDiffOp::Unknown variant with #[serde(other)] for forward compatibility
- Old nodes silently skip unknown ops instead of crashing

UI fixes:
- Self removed from Following list
- Offline follows in lightbox popup (auto-show if no one online)
- Sent DMs filtered from My Posts
- Comment threading scoped to closest .post (fixes duplicate ID issue)
- Select dropdown text legible in WebKitGTK (black on white options)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 00:47:53 -04:00
Scott Reimers
8fad30cf95 v0.3.3: Rate limiting, IPv6 fix, schema versioning, video preload, engagement propagation
Security & stability:
- Incoming auth-fail rate limiting per source IP (3 attempts, then exponential backoff)
- Schema versioning via PRAGMA user_version with migration framework

Networking:
- IPv6 http_addr fix: advertise actual public IPv6 instead of 0.0.0.0
- N2/N3 TTL reduced from 7 days to 5 hours
- Full N1/N2 state re-broadcast every 4 hours
- Bootstrap isolation recovery: 24h check with sticky N1 advertising
- Bidirectional engagement propagation (upstream + downstream)
- Auto downstream registration on pull sync and push notification
- post_upstream table for CDN tree traversal

Media & UI:
- Video preload="auto" for share links and in-app blob URLs
- Following: Online/Offline split with last-seen timestamps
- DMs filtered from My Posts tab
- Image lightbox, audio player, file attachments with download prompt
- Share link unroutable address filtering

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 18:37:24 -04:00
Scott Reimers
800388cda4 ItsGoin v0.3.2 — Decentralized social media network
No central server, user-owned data, reverse-chronological feed.
Rust core + Tauri desktop + Android app + plain HTML/CSS/JS frontend.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 20:23:09 -04:00