filiksyos · Oct 11, 2026 · Oct 9, 2026 · Oct 11, 2026
diff --git a/docs/sponsorship-analytics.md b/docs/sponsorship-analytics.md
@@ -31,7 +31,7 @@ Deployment SQL: `docs/sql/sponsorship-audience-rpc.sql`. It has no arguments; ca
 
 `SECURITY INVOKER` preserves the caller's privileges. The existing service_role already has SELECT on the source (verified read-only). Execute permission is revoked from PUBLIC, anon and authenticated and granted only to service_role in the same transaction. It adds no table grants, raw-data access, credentials, new RLS policies or SECURITY DEFINER bypass. Review and explicitly verify deployment before relying on it.
 
-The query scans only two bounded indexed date ranges (30 days and the previous calendar month), using the existing `(event_type, event_timestamp DESC)` index. It does not transfer raw rows to the app. A bounded paginated REST alternative would fetch roughly 69 thousand rows over roughly 69 requests at a 1,000-row cap per refresh at current traffic, which is needlessly expensive.
+The optimized query scans the union of the two bounded date ranges once (30 days and the previous calendar month), using the existing `(event_type, event_timestamp DESC)` index. It materializes only timestamp/device pairs, then groups by UTC date and device within each window; this preserves the original distinct visitor semantics while avoiding two raw-table scans and expensive composite DISTINCT sorting. It does not transfer raw rows to the app. A bounded paginated REST alternative would fetch roughly 69 thousand rows over roughly 69 requests at a 1,000-row cap per refresh at current traffic, which is needlessly expensive.
 
 Call only from a server-only module using the existing admin client; never import that module into client code. Cache successful results for one hour, coalesce concurrent refreshes, and abort the HTTP RPC request after a bounded timeout (e.g. 15 seconds). An HTTP abort bounds app waiting, not necessarily database execution: retain the deployment's existing database statement timeout and do not claim cancellation proves the backend query stopped. Do not launch unbounded retries.
 
@@ -48,3 +48,24 @@ If credentials/RPC/data are unavailable, omit the live metric. Never substitute
 ## Deployment status
 
 The server-only aggregate function was installed on 2026-10-08 as migration `20261008080556_server_only_sponsorship_audience`. A service_role call returned 35,087 September visits; actual anon and authenticated calls were both denied. SECURITY INVOKER and fixed pg_catalog search path / UTC were verified. Security advisors reported no finding against this new function; unrelated existing findings were left unchanged. Prepared frontend code caches the aggregate RPC for one hour, returns a labeled owner snapshot only when its historical month matches the previous calendar month, and otherwise shows monthly traffic unavailable. The RPC is installed and verified; end-to-end UI refresh still needs verification on an accessible app deployment.
+
+## Production timeout diagnosis (2026-10-09)
+
+The first production verification used the labeled snapshot fallback. Supabase logs confirmed that the app authenticated as `service_role`, but `/rest/v1/rpc/get_sponsorship_audience` returned HTTP 500. The matching Postgres log reported a statement timeout. The API's existing `authenticator` role has an 8-second statement timeout; a direct `EXPLAIN ANALYZE` of the original function took 13.05 seconds. A direct service-role SQL call succeeded and returned the expected 35,087 September visits because the diagnostic SQL connection has a longer timeout. That direct success did not establish that the REST path worked.
+
+The app's 15-second HTTP abort cannot override the shorter database limit. Preserve the existing timeout and access controls; optimize the aggregate instead. The hourly Next.js cache also caches failures, so a successful database fix may not become visible until the cached failure expires or the cache key changes in a subsequent approved deployment.
+
+The equivalent single-scan rewrite measured 2.165 seconds, then 1.928 seconds under `service_role` with a transaction-local 8-second timeout. A fixed-time comparison at `2026-10-09T11:00:00Z` compared every output column in both directions with `EXCEPT ALL` and found zero differences. Both implementations returned September 35,087 visits / 74,693 raw events, and rolling 32,137 visits / 67,269 raw events. These timing measurements describe the observed database workload, not a permanent performance guarantee.
+
+The body rewrite was installed as migration `20261009112433_optimize_sponsorship_audience_aggregate`. Post-installation testing exposed a material limitation: the actual SQL function took 6.774 seconds on one run and exceeded the production-equivalent 8-second limit on another. Inline benchmarks alone were insufficient; most work remains scanning/filtering the wide raw heap, with runtime varying under load. This body-only result required the covering index verified below.
+
+A targeted covering partial index is prepared in `docs/sql/sponsorship-audience-index.sql`, using the exact unchanged filter and covering timestamp/device ID. It must be built with `CONCURRENTLY` outside a transaction so analytics ingestion remains available. Existing indexes were inspected and none covers this predicate/device pair. The index was approved and installed on 2026-10-11. The migration tool rejected concurrent creation because it wraps SQL in a transaction (Postgres 25001); the same approved standalone SQL then completed through execute_sql. No blocking index build was used. Catalog checks confirmed indisready=true and indisvalid=true, size 9,984 kB. There is no migration-history row for the standalone index; its exact reproducible DDL is tracked in this repository. After approval, check for any existing index, inspect `indisready`/`indisvalid` after the build, and test the actual service-role function repeatedly under the existing 8-second limit before checking the live page after its hourly cache expires.
+
+Read-only synthetic regression SQL is provided in `docs/sql/sponsorship-audience-regression.sql`. It checks all-field equivalence, explicit expected counts, null-only/empty windows, UTC day boundaries, duplicate device visits, half-open date bounds and canonical/scope exclusions. All three result booleans passed. The Node regression suite checks that its candidate aggregate matches the production SQL body and that access-control and concurrent-index safeguards are retained.
+
+
+## Live verification (2026-10-11)
+
+After the concurrent index build, the actual service-role RPC completed under the unchanged 8-second timeout, returning September 35,087 visits and 74,693 raw events with no missing device IDs. The first measured function execution was 7.491 seconds with cold heap reads; three subsequent actual-function runs under service_role with the same 8-second limit completed in 5.461, 3.497 and 1.718 seconds; an expanded plan confirmed use of `Index Only Scan` on `vercel_analytics_sponsorship_audience_idx`. The current dataset still needs heap visibility checks for recently ingested pages, so cold performance varies. This change does not raise any database or application timeout.
+
+Supabase REST logs independently confirmed HTTP 200 for the production app's RPC. A public cloud-browser check of `https://www.gitreverse.com/sponsor` showed “35K visitors last month” and “Monthly traffic updates hourly”, replacing the snapshot fallback. SECURITY INVOKER, pg_catalog search path, UTC, service_role execution and anon/authenticated denial remain unchanged. The lifetime figure remains owner supplied; only the last-month figure refreshes from the drain.
diff --git a/docs/sql/sponsorship-audience-index.sql b/docs/sql/sponsorship-audience-index.sql
@@ -0,0 +1,9 @@
+-- Run as a standalone statement outside a transaction to keep ingestion available.
+-- Narrow covering index for the existing server-only audience RPC.
+-- Does not change role grants, RLS, credentials, or statement timeouts.
+CREATE INDEX CONCURRENTLY vercel_analytics_sponsorship_audience_idx
+ON public.vercel_analytics_events_raw (event_timestamp DESC) INCLUDE (device_id)
+WHERE project_id = 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF'
+  AND vercel_environment = 'production'
+  AND event_type = 'pageview'
+  AND origin ~ '^https://(www\.)?gitreverse\.com([/?#]|$)';
diff --git a/docs/sql/sponsorship-audience-regression.sql b/docs/sql/sponsorship-audience-regression.sql
@@ -0,0 +1,125 @@
+-- Read-only regression: synthetic data only; no function/schema changes.
+-- Covers UTC dates, same-device duplicates, missing devices, empty windows,
+-- partial-day and month boundaries, canonical domains and fixed data scope.
+WITH fixtures(event_timestamp, device_id, project_id, vercel_environment, event_type, origin) AS (
+  VALUES
+    (timestamptz '2026-09-01 00:00:00+00', 1::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-01 01:00:00+00', 1::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-02 00:00:00+00', 1::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-09 10:59:59+00', 2::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-09 11:00:00+00', 2::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-09 12:00:00+00', 2::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-09 13:00:00+00', NULL::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-09 14:00:00+00', NULL::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-10 01:00:00+02', 3::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-10 02:00:00+02', 3::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-30 23:59:59+00', 4::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-10-01 00:00:00+00', 4::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-10-09 10:59:59+00', 5::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-10-09 11:00:00+00', 6::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-08-31 23:59:59+00', 7::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-15 12:00:00+00', 100::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://www.gitreverse.com/path'),
+    (timestamptz '2026-09-15 12:00:00+00', 101::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com?x=1'),
+    (timestamptz '2026-09-15 12:00:00+00', 102::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com#anchor'),
+    (timestamptz '2026-09-15 12:00:00+00', 103::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse.com.evil.example'),
+    (timestamptz '2026-09-15 12:00:00+00', 104::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'http://gitreverse.com'),
+    (timestamptz '2026-09-15 12:00:00+00', 105::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://gitreverse-com.example'),
+    (timestamptz '2026-09-15 12:00:00+00', 106::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'pageview', 'https://preview.gitreverse.com'),
+    (timestamptz '2026-09-15 12:00:00+00', 107::bigint, 'other_project', 'production', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-15 12:00:00+00', 108::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'preview', 'pageview', 'https://gitreverse.com'),
+    (timestamptz '2026-09-15 12:00:00+00', 109::bigint, 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF', 'production', 'custom', 'https://gitreverse.com')
+), original AS (
+WITH windows(key, start_at, end_at, measured) AS (
+    VALUES
+      ('previous_calendar_month', timestamptz '2026-09-01 00:00:00+00', timestamptz '2026-10-01 00:00:00+00', timestamptz '2026-10-09 11:00:00+00'),
+      ('rolling_30_days', timestamptz '2026-09-09 11:00:00+00', timestamptz '2026-10-09 11:00:00+00', timestamptz '2026-10-09 11:00:00+00'),
+      ('null_devices_only', timestamptz '2026-09-09 13:00:00+00', timestamptz '2026-09-09 15:00:00+00', timestamptz '2026-10-09 11:00:00+00'),
+      ('empty', timestamptz '2027-01-01 00:00:00+00', timestamptz '2027-01-02 00:00:00+00', timestamptz '2026-10-09 11:00:00+00')
+  )
+  SELECT w.key, w.start_at, w.end_at,
+    a.daily_unique_visits, a.raw_pageview_events,
+    a.first_event_at, a.latest_event_at, a.active_utc_dates,
+    a.missing_device_events, w.measured
+  FROM windows AS w
+  CROSS JOIN LATERAL (
+    SELECT
+      count(DISTINCT ((e.event_timestamp AT TIME ZONE 'UTC')::date, e.device_id))
+        FILTER (WHERE e.device_id IS NOT NULL) AS daily_unique_visits,
+      count(*) AS raw_pageview_events,
+      min(e.event_timestamp) AS first_event_at,
+      max(e.event_timestamp) AS latest_event_at,
+      count(DISTINCT (e.event_timestamp AT TIME ZONE 'UTC')::date) AS active_utc_dates,
+      count(*) FILTER (WHERE e.device_id IS NULL) AS missing_device_events
+    FROM fixtures AS e
+    WHERE e.project_id = 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF'
+      AND e.vercel_environment = 'production'
+      AND e.event_type = 'pageview'
+      AND e.origin ~ '^https://(www\.)?gitreverse\.com([/?#]|$)'
+      AND e.event_timestamp >= w.start_at
+      AND e.event_timestamp < w.end_at
+  ) AS a
+), candidate AS (
+WITH windows(key, start_at, end_at, measured) AS (
+    VALUES
+      ('previous_calendar_month', timestamptz '2026-09-01 00:00:00+00', timestamptz '2026-10-01 00:00:00+00', timestamptz '2026-10-09 11:00:00+00'),
+      ('rolling_30_days', timestamptz '2026-09-09 11:00:00+00', timestamptz '2026-10-09 11:00:00+00', timestamptz '2026-10-09 11:00:00+00'),
+      ('null_devices_only', timestamptz '2026-09-09 13:00:00+00', timestamptz '2026-09-09 15:00:00+00', timestamptz '2026-10-09 11:00:00+00'),
+      ('empty', timestamptz '2027-01-01 00:00:00+00', timestamptz '2027-01-02 00:00:00+00', timestamptz '2026-10-09 11:00:00+00')
+  ), filtered AS MATERIALIZED (
+    -- Read and canonical-filter the union of both windows only once.
+    SELECT e.event_timestamp,
+      (e.event_timestamp AT TIME ZONE 'UTC')::date AS event_date,
+      e.device_id
+    FROM fixtures AS e
+    WHERE e.project_id = 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF'
+      AND e.vercel_environment = 'production'
+      AND e.event_type = 'pageview'
+      AND e.origin ~ '^https://(www\.)?gitreverse\.com([/?#]|$)'
+      AND e.event_timestamp >= (SELECT min(start_at) FROM windows)
+      AND e.event_timestamp < (SELECT max(end_at) FROM windows)
+  )
+  SELECT w.key, w.start_at, w.end_at,
+    a.daily_unique_visits, a.raw_pageview_events,
+    a.first_event_at, a.latest_event_at, a.active_utc_dates,
+    a.missing_device_events, w.measured
+  FROM windows AS w
+  CROSS JOIN LATERAL (
+    SELECT
+      count(*) FILTER (WHERE device_id IS NOT NULL) AS daily_unique_visits,
+      coalesce(sum(event_count), 0)::bigint AS raw_pageview_events,
+      min(first_at) AS first_event_at,
+      max(latest_at) AS latest_event_at,
+      count(DISTINCT event_date) AS active_utc_dates,
+      coalesce(sum(event_count) FILTER (WHERE device_id IS NULL), 0)::bigint
+        AS missing_device_events
+    FROM (
+      -- Scalar grouping avoids expensive composite COUNT(DISTINCT) comparison.
+      -- Keep null-device groups for raw-event and missing-device totals.
+      SELECT e.event_date, e.device_id, count(*) AS event_count,
+        min(e.event_timestamp) AS first_at, max(e.event_timestamp) AS latest_at
+      FROM filtered AS e
+      WHERE e.event_timestamp >= w.start_at AND e.event_timestamp < w.end_at
+      GROUP BY e.event_date, e.device_id
+    ) AS daily_devices
+  ) AS a
+), differences AS (
+  (SELECT * FROM original EXCEPT ALL SELECT * FROM candidate)
+  UNION ALL
+  (SELECT * FROM candidate EXCEPT ALL SELECT * FROM original)
+), expected(key, daily_unique_visits, raw_pageview_events, active_utc_dates, missing_device_events) AS (
+  VALUES
+    ('previous_calendar_month', 9::bigint, 14::bigint, 6::bigint, 2::bigint),
+    ('rolling_30_days', 9::bigint, 12::bigint, 6::bigint, 2::bigint),
+    ('null_devices_only', 0::bigint, 2::bigint, 1::bigint, 2::bigint),
+    ('empty', 0::bigint, 0::bigint, 0::bigint, 0::bigint)
+)
+SELECT
+  NOT EXISTS (SELECT 1 FROM differences) AS equivalent_all_fields,
+  NOT EXISTS (
+    SELECT c.key FROM candidate c FULL JOIN expected e USING (key)
+    WHERE c.key IS NULL OR e.key IS NULL OR (c.daily_unique_visits, c.raw_pageview_events, c.active_utc_dates, c.missing_device_events)
+      IS DISTINCT FROM
+      (e.daily_unique_visits, e.raw_pageview_events, e.active_utc_dates, e.missing_device_events)
+  ) AS expected_counts_match,
+  (SELECT first_event_at IS NULL AND latest_event_at IS NULL FROM candidate WHERE key = 'empty') AS empty_dates_are_null,
+  (SELECT jsonb_agg(c ORDER BY key) FROM candidate c) AS actual;
diff --git a/docs/sql/sponsorship-audience-rpc.sql b/docs/sql/sponsorship-audience-rpc.sql
@@ -33,6 +33,18 @@ AS $function$
     SELECT 'previous_calendar_month'::text,
       month_start - interval '1 month', month_start, measured
     FROM bounds
+  ), filtered AS MATERIALIZED (
+    -- Read and canonical-filter the union of both windows only once.
+    SELECT e.event_timestamp,
+      (e.event_timestamp AT TIME ZONE 'UTC')::date AS event_date,
+      e.device_id
+    FROM public.vercel_analytics_events_raw AS e
+    WHERE e.project_id = 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF'
+      AND e.vercel_environment = 'production'
+      AND e.event_type = 'pageview'
+      AND e.origin ~ '^https://(www\.)?gitreverse\.com([/?#]|$)'
+      AND e.event_timestamp >= (SELECT min(start_at) FROM windows)
+      AND e.event_timestamp < (SELECT max(end_at) FROM windows)
   )
   SELECT w.key, w.start_at, w.end_at,
     a.daily_unique_visits, a.raw_pageview_events,
@@ -41,20 +53,22 @@ AS $function$
   FROM windows AS w
   CROSS JOIN LATERAL (
     SELECT
-      count(DISTINCT ((e.event_timestamp AT TIME ZONE 'UTC')::date, e.device_id))
-        FILTER (WHERE e.device_id IS NOT NULL) AS daily_unique_visits,
-      count(*) AS raw_pageview_events,
-      min(e.event_timestamp) AS first_event_at,
-      max(e.event_timestamp) AS latest_event_at,
-      count(DISTINCT (e.event_timestamp AT TIME ZONE 'UTC')::date) AS active_utc_dates,
-      count(*) FILTER (WHERE e.device_id IS NULL) AS missing_device_events
-    FROM public.vercel_analytics_events_raw AS e
-    WHERE e.project_id = 'prj_z7TUohej5S9f0TFdK6BKZX0FkDaF'
-      AND e.vercel_environment = 'production'
-      AND e.event_type = 'pageview'
-      AND e.origin ~ '^https://(www\.)?gitreverse\.com([/?#]|$)'
-      AND e.event_timestamp >= w.start_at
-      AND e.event_timestamp < w.end_at
+      count(*) FILTER (WHERE device_id IS NOT NULL) AS daily_unique_visits,
+      coalesce(sum(event_count), 0)::bigint AS raw_pageview_events,
+      min(first_at) AS first_event_at,
+      max(latest_at) AS latest_event_at,
+      count(DISTINCT event_date) AS active_utc_dates,
+      coalesce(sum(event_count) FILTER (WHERE device_id IS NULL), 0)::bigint
+        AS missing_device_events
+    FROM (
+      -- Scalar grouping avoids expensive composite COUNT(DISTINCT) comparison.
+      -- Keep null-device groups for raw-event and missing-device totals.
+      SELECT e.event_date, e.device_id, count(*) AS event_count,
+        min(e.event_timestamp) AS first_at, max(e.event_timestamp) AS latest_at
+      FROM filtered AS e
+      WHERE e.event_timestamp >= w.start_at AND e.event_timestamp < w.end_at
+      GROUP BY e.event_date, e.device_id
+    ) AS daily_devices
   ) AS a;
 $function$;