Skip to content

Add daily Must Read notifications, reusable push audiences and analytics - #115

Draft
abhi1693 wants to merge 4 commits into
masterfrom
feat/daily-must-read-web-push
Draft

abhi1693 wants to merge 4 commits into
masterfrom
feat/daily-must-read-web-push

Conversation

@abhi1693

@abhi1693 abhi1693 commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

What changes and why?

Signed-in readers can enable browser notifications in Settings → Notifications to receive one personalized unread Must Read article at 9 AM in their local timezone. Permission is requested only after clicking Enable. The article comes from the reader's stable ranked selection; days without an eligible unread pick are skipped. All enabled browsers receive the same article using the earliest active registration's delivery timezone.

Browser delivery now has a reusable typed publisher, separate from daily article selection:

  • enqueue_web_push publishes an immutable message, audience, unique event key and expiry in the producer's database transaction.
  • PushAudience supports one account, an account list, all consented accounts, or followers of any listed active topic/approved source.
  • Bounded UUID pages commit browser jobs and their cursor together. Concurrent expansion and replay cannot duplicate jobs; later enrollment cannot join an earlier publication.
  • Type policies own content and recipient validation. The generic sender, retries and lease recovery depend on generic events rather than daily article tables. Daily claims remain durable and are published atomically with their events.

daily_must_read remains the only enabled producer, type, browser handler and consent option. Custom messages cannot reuse daily consent. Another type can use the same publisher/audiences after adding its policy, browser handler and explicit consent option; no custom composer or public send endpoint is introduced.

Admin → Push analytics adds 7/30/90-day publication cohorts, daily trends and per-type totals for published events, distinct recipient accounts, browser jobs, relay acceptance, reported displays/clicks/opens, failures, skipped jobs, pending jobs and retry attempts. Click rate uses notifications with a reported display, and missing browser feedback remains unconfirmed. This aggregate admin view supports future types without exposing account IDs, endpoints or notification content.

The service worker reports successful display and validated click/navigation outcomes to the user API with its current cookie session and CSRF token. Receipts are first-write-only and bound to the original account, browser session and consent epoch. Passive authentication checks expiry and the current policy without renewing sessions or recording reader activity. A notification open never marks an article as read. Bounded retries/timeouts run outside the worker’s consent queue so telemetry cannot delay revocation.

Before displaying any queued push, the website worker verifies live server-side authorization for its original account, session, consent and attempted event. This prevents an already relay-accepted notification from revealing a previous account's personalized title after sign-out from either extension, whose origin cannot directly clear website worker storage. Failed or unavailable authorization suppresses that delivery without deleting consent.

Notification enrollment is cancelled when the account/session changes, expires or its settings component unmounts. Late responses and stale cleanup cannot restore or remove a newer binding. Temporary delivery pauses preserve existing enabled consent, allowing delivery to resume without revisiting settings. Compose supplies the same public feature configuration to admin analytics while keeping signing credentials in delivery workers only.

Standard Web Push/VAPID uses existing PostgreSQL, Redis, scheduler and RQ workers without a notification SaaS account or provider fee. Chrome and Edge extensions open website notification settings without adding extension permissions. Session/account/consent checks, per-type consent, local deduplication, safe article actions and serialized click/revocation prevent stale-account delivery. Workers validate the current authentication-policy fingerprint without receiving OIDC credentials. Relay TTL, payload expiry and retries stop at the earliest event or live authorization deadline.

Validation

  • Full final commit checks passed: backend unit suite with coverage, Ruff, mypy, frontend formatting/lint/types/unit tests, and Chrome/Edge extension types/unit tests/builds.
  • 413 backend regressions passed in the review-fix validation, covering passive display authorization, receipt authentication/idempotency/privacy, analytics, audience expansion, daily limits, encrypted delivery, session/policy changes and Compose configuration. Disposable PostgreSQL/Redis were used for integration cases. The earlier feature validation also covered scheduler integration and container bootstrap.
  • Production website, admin, Chrome and Edge builds passed. Notification browser checks passed on the website and both built extensions; additional actual built Chrome/Edge sign-out regressions prove queued website-worker titles are suppressed. Worker regressions cover duplicate/failed displays, clicks versus successful navigation, consent/account changes, relay-response races, bounded telemetry failures and revocation while telemetry is in flight.
  • The admin dashboard passed production browser checks at 1440/768/390/320px, including date filters, loading/empty/disabled states, polling errors and retained data. Desktop/mobile screenshots were inspected; no page errors or CSP violations occurred.
  • Populated upgrades from current release schema 0022 and legacy 0018 passed, preserving existing records. The updated 0023 → 0022 → 0023 round trip, nullable receipt timestamps, event index and event/consent uniqueness checks passed.
  • Admin OpenAPI/client regeneration completed and generated imports compiled. Cryptographic and SSRF regressions remain covered, including actual AES128GCM decryption and VAPID verification.
  • Added 24 focused reader-settings regressions for expired/changed sessions, pending enrollment, stale cleanup, unmount, permission rejection and temporary pauses.
  • Addressed the reported static-analysis findings with secure random private browser-artifact directories, positive denominators, bounded helper ranges, empty audience rejection, concrete IndexedDB errors and a single awaited permission request.
  • Fixed the previous head’s scheduler assertion mismatch and duplicate container --no-editable flag. A new canary executes the production Docker dependency-stage arguments and proves first-party code is neither built nor imported in that stage.

Gaps: relay delivery and system notification display are simulated in browser tests; live vendor-relay delivery and real Safari/iPhone/iPad devices are unverified. Hosted CI, CodeQL and AMD64/ARM64 runtime checks must pass on the updated head before merge. The previous head’s six-sample performance comparison passed. Its Sonar gate reported 75.3% new coverage against 80%, dominated by missing service-worker coverage reporting despite browser execution; the updated head must clear that gate before merge.

Native GitHub screenshot attachment upload is unavailable in this session, so UI screenshots are not attached. No review-only images are committed.

Rollout notes

  • Apply migration 0023 with the coordinated application update; readiness expects this revision even while push is disabled. This refactors the same unreleased draft migration and adds nullable display/click/open timestamps plus an event-cohort index. Disposable development databases that applied the earlier draft 0023 need a coordinated reset/downgrade before testing the updated schema; published migrations are unchanged.
  • Browser analytics is first party and uses the same database/outbox, with no paid analytics service. Browser receipts are accepted for the last 30 days only with current session/consent authorization. Missing receipt reports remain unknown; enabled-registration counts reflect stored consent and absolute expiry, not a live Redis-session census.
  • Push remains disabled by default. Generate stable VAPID keys outside the checkout with scripts/generate_web_push_keys.py. Public configuration goes to user-api, scheduler and delivery workers; only workers receive private signing credentials. See the setup and publisher guide.
  • Local testing supports http://localhost:3000 (and HTTP loopback addresses); other reader hosts still require HTTPS. Use matching reader/site origins, HTTP development cookie mode and the registered local OIDC callback, then restart all affected services. Signing-key changes remain local and ignored; no credentials are committed.
  • With push enabled, user-api startup publishes the current session-policy fingerprint in Redis. Missing policy state postpones delivery. Start the updated user-api before expecting workers to deliver alerts.
  • Enrollment begins at the next local 9 AM. Sign-out, account changes or expired sessions require fresh enrollment. Rotating VAPID keys also requires enrollment again; iPhone/iPad requires the Home Screen web app. Browser/OS settings and connectivity can delay or prevent display.
  • Keep dependency builds wheel-only except the explicitly reviewed http-ece==1.2.1 source archive, guarded by registry/version/SHA-256. CI and container dependency layers share this policy; unreviewed upgrades fail closed.
  • Roll back the feature by disabling DEVFEED_WEB_PUSH_ENABLED while retaining the updated application/schema. Schema downgrade drops push subscriptions, events, claims and deliveries and must be coordinated with the earlier application version.
  • No deployment, release or browser-store submission has been performed. Auto-merge remains disabled.

Ready for review

  • The description explains the change and validation, including gaps.
  • Relevant documentation and regression tests are updated.

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Backend unit coverage

PASS · minimum combined coverage 67.60%.

Tests: 3642 passed, 0 failed, 0 skipped. Source files: 283.

Workspace Lines Branches Combined
All backend code 71.66% 53.55% 67.78%
apps/admin-api 65.29% 36.20% 60.50%
apps/aggregator 74.03% 57.28% 70.23%
apps/api 72.77% 58.45% 70.13%
apps/article-enrichment-worker 80.00% 50.00% 71.43%
apps/cli 68.76% 42.93% 64.81%
apps/images-worker 80.00% 50.00% 71.43%
apps/mcp 82.99% 73.88% 81.28%
apps/notifications 62.80% 51.28% 60.38%
apps/search-indexer 85.48% 85.71% 85.53%
apps/source-discovery-worker 80.00% 50.00% 71.43%
apps/user-api 58.95% 32.68% 54.16%
packages/core 73.57% 56.14% 69.42%
packages/http 94.94% 87.77% 93.29%

Unit coverage includes every backend workspace. PostgreSQL/Redis integration tests run separately.

Commit: 1f5ba00 · HTML, JSON, XML reports and job logs · attempt 1

Comment thread apps/web/public/web-push-sw.js Fixed
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

PR reader API performance

Result: INCONCLUSIVE

5 inconclusive · 3 within threshold. Zero request failures across all six runs.

Base b302fc8849b2ccdc5b29efe9bb82718bb28e2380 → PR 1f5ba00832c91ec31da959b06762e869e1598f78

Three independent runs per revision, each on a fresh GitHub-hosted ARM64 runner. 16 users, 60 seconds/run, 1,000 synthetic articles, cache off, five PgBouncer slots.

Latency comparison

All latency values are in milliseconds; lower is better. Values are medians of three runs.

Route Base p95 PR p95 Change Result
All requests 100 110 +10 (+10.0%) Within threshold
Latest feed 120 130 +10 (+8.3%) Inconclusive
Feed pagination 86 98 +12 (+14.0%) Within threshold
Article detail 76 83 +7 (+9.2%) Inconclusive
Source feed 99 92 -7 (-7.1%) Within threshold
Topic feed 92 88 -4 (-4.3%) Inconclusive
Feed filters 73 68 -5 (-6.8%) Within threshold
Sources 44 41 -3 (-6.8%) Inconclusive
Topics 44 42 -2 (-4.5%) Inconclusive
Additional latency and throughput metrics

Each cell shows base → PR. Latencies are in milliseconds.

Endpoint p50 p99 Requests/s
Aggregated 30 → 29 140 → 140 30.14 → 29.73
/v1/feed [latest] 46 → 46 170 → 170 8.74 → 8.71
/v1/feed [page] 27 → 26 110 → 120 3.88 → 3.77
/v1/articles/[id] 19 → 18 120 → 120 6.11 → 6.26
/v1/feed [source] 28 → 27 130 → 130 2.05 → 1.84
/v1/feed [topic] 27 → 28 120 → 120 1.94 → 2.05
/v1/feed/options 23 → 22 100 → 89 2.43 → 2.33
/v1/sources 12 → 11 96 → 78 2.43 → 2.33
/v1/topics 12 → 12 70 → 84 2.43 → 2.33
Individual runs
Run Requests Failures Exit code
Base 1 1705 0 0
Base 2 1675 0 0
Base 3 1646 0 0
PR 1 1693 0 0
PR 2 1670 0 0
PR 3 1641 0 0
How to interpret this report

Regressed / Improved: a p95 regression needs >20% and >20 ms increase in the medians and at least two head samples versus the base median. Improvements use the inverse threshold. Within threshold means the change did not meet both thresholds.

Inconclusive: a within-revision p95 range >35% of its median and >20 ms is too noisy for a confident latency verdict.

Inconclusive latency comparisons are non-blocking: the check passes without claiming an improvement. Confirmed regressions and test/report errors fail the PR gate. Throughput is descriptive: paced users do not measure maximum capacity. This uncached public API test does not establish production health, browser performance or worker capacity. Separate runners can differ in hardware or host load; repetitions and noise checks reduce but cannot eliminate that uncertainty.

Full HTML/CSV reports and logs · attempt 1

@abhi1693 abhi1693 changed the title Add opt-in daily Must Read browser notifications Add daily Must Read notifications with reusable Web Push audiences Oct 6, 2026
@abhi1693 abhi1693 changed the title Add daily Must Read notifications with reusable Web Push audiences Add daily Must Read notifications, reusable push audiences and analytics Oct 6, 2026
Comment thread apps/admin/tests/browser/push-analytics.mjs Fixed
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Lighthouse · ⏸️ Not measured

Tests did not pass for this commit.
Full reports and logs · 1f5ba00 · Attempt 1

@sonarqubecloud

sonarqubecloud Bot commented Oct 6, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
75.9% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants