Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -169,15 +169,16 @@ BLOG_API_SECRET=
# AGENT_SANDBOX_MAX_ENDPOINTS=200
# EPHEMERAL_TTL_HOURS=12

# Agent auth (auth.md) - optional
# RESEND_API_KEY=
# SMTP transport for agent OTP email (production only; SMTP -> Resend -> dev fallback)
# Outbound email over SMTP (required in production, where it is Amazon SES's
# SMTP endpoint; outside production mail is only logged, never sent)
# STARTTLS: SMTP_PORT=587 + SMTP_SECURE=false; implicit TLS: SMTP_PORT=465 + SMTP_SECURE=true
# SMTP_HOST=
# SMTP_HOST=email-smtp.eu-central-1.amazonaws.com
# SMTP_PORT=587
# SMTP_SECURE=false
# SMTP_USER=
# SMTP_PASS=
# SES configuration set named on every message, with its kind as a message tag
# SES_CONFIGURATION_SET=webhooks-cc-app
# Shared from-address for all outbound email (falls back to AGENT_EMAIL_FROM)
# EMAIL_FROM=webhooks.cc <noreply@webhooks.cc>
# AGENT_EMAIL_FROM=webhooks.cc <noreply@webhooks.cc>
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ Decisions worth knowing, with the reasons:
- **Analytics: consent in the browser, accounts on the server.** posthog-js runs in cookieless mode until the visitor accepts in the cookie banner (`cookieless_mode: "on_reject"`, opted out by default, Do Not Track and GPC count as a decline), so nothing is stored in the browser without consent and a signed-in user is identified only after it; each site keeps its own choice, and the site switcher carries it over sealed in its grant and ticket (an accept only answers an unanswered banner, a decline always applies); a signed-in browser also saves it on the account (`users.analytics_consent`, migration 00082, `lib/analytics-account-consent.ts`) and is answered from it by the same rules, the banner waiting for that answer, an accept made before signing in is saved only when made in the same tab, since a remembered one may be someone else's on a shared computer, a decline that only Do Not Track or GPC made is never saved, and a choice is kept in the browser from the click until the account has it, then sent again on a later load unless the account was told something since (`analytics_consent_at` moves on every save, compared on the server's clock); the PostHog project's cookieless tracking setting must stay on or those events are dropped. `before_send` (`lib/analytics-redact.ts`) strips share tokens, fragments and every query parameter not on its list from every URL, because slugs, Polar session tokens and auth codes ride in them. Account analytics come from the server (`lib/analytics-server.ts`, on in production builds): every audit event is mirrored, so the API, CLI, SDKs, MCP and agents are counted without call sites of their own; applied Polar webhooks become `order_paid` (`revenue` in cents, `subscription_id`) and subscription events; and `lib/analytics-reporter.ts` sends hourly signups and a daily usage snapshot per account from `endpoint_daily_stats` (migration 00080). Never send an event per capture, and never put contents, URLs, email addresses or secrets in properties; accounts are named by id only. Dashboard components that show captured content or endpoint names carry `ph-no-capture`, so add it to new ones. /privacy promises the data stays with us and is never sold or shared; keep it true.
- **Outbound deliveries hide the origin IP** by going through the Cloudflare Worker when `NOTIFY_PROXY_URL` is set; otherwise the web app delivers directly with SSRF-safe DNS pinning.
- **Email/password auth is GoTrue configuration, not app code.** SMTP, minimum password length 8, and the template URLs live in the Supabase instance's `docker-compose.override.yml`; see `infra/supabase/gotrue-email-auth.md`. Email links carry a `token_hash` to `/auth/confirm`, which verifies server-side. If GoTrue could not fetch the templates at startup it keeps serving its defaults until the auth container restarts.
- **Our own mail goes straight to Amazon SES over SMTP.** The web app (`sendEmail()` in `lib/email/mailer.ts`, the only way it sends) and GoTrue both send from `noreply@webhooks.cc` through SES's SMTP endpoint with one IAM user, not through the API: GoTrue can only speak SMTP, and at our volume the API's extras are all reachable over SMTP, so one credential and one path serve both. Every message lands in the `webhooks-cc-app` configuration set, the app's by header and GoTrue's through the address identity's default (`infra/supabase/gotrue-email-auth.md`), and each app message carries its `kind` as a message tag. There is no queue and no fallback transport: a failed send throws, and the caller retries (the pollers), shows the error, or reports it; the transport never retries, because the welcome email must not go out twice. The SES side and its event destinations are not in this repository.
- **Agent registration follows auth.md v0.6.** An agent registers, gets a signed identity assertion, and exchanges it for short-lived access tokens that are ordinary `whcc_` rows in `api_keys`, so the bearer path and every route accept them unchanged; each exchange reads the registration again, so revoking it ends everything at once. Pre-claim tokens have no user: every consumer of an API key must tolerate a null user, and the routes that need one refuse them, which confines them to the sandbox. Sandbox endpoints belong to the registration, never to a user, so guest reads and the guest claim must keep skipping them; they have their own pool, budget and billing key in `capture_webhook()`. A claim (the human signs in and types the agent's code; the email the agent named decides who may) adopts them into the human's account in one transaction, deletes the pre-claim tokens, and lets the agent collect account tokens once with its claim token; claimed agent tokens are capped as connected agents, not counted as API keys, and never reach session-only routes. The proof of work on anonymous registration only raises the cost of scripted abuse; the rate limits, the live cap and the pool are the real bounds. `AGENT_IDJAG_PROVIDERS` stays empty in production until a real ID-JAG issuer exists. An ID-JAG identity `(iss, sub)` never binds silently to an existing account: without a delegation on file, an account with its verified email gets `interaction_required` and the same claim ceremony (`link_agent_idjag_identity()`, migration 00057), only an account created for the email in that request links at once, `auth_time` must be recent on every presentation, and a provider's Security Event Token at `/api/agent/event/notify` revokes the delegation with its tokens. The protocol itself is documented in `/auth.md` (`lib/agent/metadata.ts`).
- **The pytest fixtures have one home, the Python SDK.** `webhooks_cc` and `webhooks_cc_endpoint` are defined in `packages/sdk-python` (`webhooks_cc/pytest_plugin.py`) and registered by the SDK's own `pytest11` entry point, named `webhooks_cc`, because projects that install only `webhooks-cc` rely on them. `pytest-webhooks-cc` (`packages/pytest-webhooks-cc`) exists so the plugin is found where pytest plugins are looked for (the plugin list on docs.pytest.org is generated from PyPI projects named `pytest-*`, and reads the summary, the status classifier and the `pytest` requirement): it depends on the SDK, defines no fixtures, and has an entry point of its own under another name (`pytest_webhooks_cc`). Never point a second entry point at the SDK's module or list it in `pytest_plugins`: pluggy refuses one module under two names, and every run would fail. The plugin registers the SDK's module itself only when pytest did not (`PYTEST_DISABLE_PLUGIN_AUTOLOAD`), under the SDK's name, so `-p no:webhooks_cc` still turns the fixtures off. Add fixtures to the SDK, and to the plugin only what is pytest's alone (report lines, options).
- **Agents keep their registrations in one shared file.** The MCP server, the TypeScript, Python and Go SDKs and the CLI read and write `~/.config/whk/agent.json` (XDG, macOS and Windows rules, `WHK_AGENT_FILE` override; the CLI's `token.json` lives beside it), so a connection a human approved once survives restarts and works from every tool. The format and the rules (live slots, the cached access token, atomic 0600 writes, what `invalid_grant` removes) are in `packages/sdk/conformance/agent-credentials.md`, and every implementation runs `agent-credentials.json`; change them together. The claim token is never written, as `/auth.md` asks, so only the process that registered a sandbox can move its endpoints into an account; a later one connects the human through a service_auth registration. An explicit API key always wins over the file.
Expand Down
6 changes: 6 additions & 0 deletions apps/web/changelog/web/direct-ses-mail.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"title": "Account email sent directly through Amazon SES",
"items": [
"Sign-up, password, verification code and notification emails leave through Amazon SES directly instead of a relay, with bounces and complaints reported back"
]
}
1 change: 1 addition & 0 deletions apps/web/lib/agent/agent-auth.ts
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,7 @@ export async function issueVerifiedEmailClaim(args: {
try {
await sendEmail({
to: email,
kind: "agent_code",
subject: "Your webhooks.cc verification code",
text: `Your webhooks.cc verification code is ${otp}. It expires in 10 minutes.`,
});
Expand Down
1 change: 1 addition & 0 deletions apps/web/lib/agent/notify.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ export async function notifyAgentConnected(input: {
try {
await sendEmail({
to: input.email,
kind: "agent_connected",
subject: "An agent was connected to your webhooks.cc account",
text: lines.join("\n"),
});
Expand Down
1 change: 1 addition & 0 deletions apps/web/lib/alerts/notify.ts
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,7 @@ async function notifyOne(event: AlertEventRow): Promise<boolean> {
try {
await sendEmail({
to: context.ownerEmail,
kind: "alert",
subject: `${event.resolved_at ? "Resolved" : "Alert"}: ${summary}`,
text: `${text}\n\nYou get this because alerts are on for this endpoint. Change them in its settings.\n`,
});
Expand Down
4 changes: 2 additions & 2 deletions apps/web/lib/email/dev-transport.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ import type { EmailMessage } from "./mailer";
*
* It logs the message (including any OTP) for local development and records the
* last message per recipient so the integration test can read the OTP via
* `getLastOtpForEmail` without DNS/SPF/Resend. Production never imports the
* test hooks (it uses the Resend transport instead).
* `getLastOtpForEmail` without sending anything. Production never imports the
* test hooks (it sends over SMTP instead).
*/

/** Last delivered message keyed by lower-cased recipient email. */
Expand Down
67 changes: 23 additions & 44 deletions apps/web/lib/email/mailer.ts
Original file line number Diff line number Diff line change
@@ -1,25 +1,18 @@
import { serverEnv } from "@/lib/env";

/**
* Minimal email abstraction for the verified_email OTP flow.
* The one way the web app sends email.
*
* Transport is selected at call time, and real delivery only happens in
* production. Production fails closed — it never falls back to the dev/test
* transport. The selection order in production is:
* - SMTP (real delivery) when `SMTP_HOST` is set. The `nodemailer` dependency
* is dynamic-imported so it only loads when used. If an SMTP send fails and
* `RESEND_API_KEY` is set, it falls back to Resend; otherwise it throws.
* - Resend (real delivery) when `RESEND_API_KEY` is set (and SMTP_HOST is
* unset). The `resend` dependency is dynamic-imported so it only loads when
* used.
* - When neither is configured (or both fail), it throws.
* In production every message goes over SMTP (`SMTP_HOST`, in production
* Amazon SES's SMTP endpoint) and a failure throws: there is no fallback and
* no queue, so each caller decides whether to retry (see the pollers), show
* the error, or report it. Production never falls back to the dev transport.
* The `nodemailer` dependency is dynamic-imported so it only loads there.
*
* In non-production (NODE_ENV !== "production", i.e. tests and local dev) the
* dev/test transport is ALWAYS used even when SMTP_HOST / RESEND_API_KEY are
* set in .env.local: it logs the message (incl. OTP) and records it for the
* integration test to read via `getLastOtpForEmail` — never sends. Use the
* directly-importable `sendViaSmtp` / `verifySmtp` for real SMTP delivery in
* any environment (e.g. the deliverability test).
* Everywhere else (NODE_ENV !== "production", i.e. tests and local dev) the
* dev transport is used even when SMTP_HOST is set in .env.local: it logs the
* message (incl. any code) and records it for the integration tests to read
* via `getLastOtpForEmail`, and never sends.
*/

export interface EmailMessage {
Expand All @@ -29,38 +22,24 @@ export interface EmailMessage {
html?: string;
/** Extra headers, such as Auto-Submitted on automatic notices. */
headers?: Record<string, string>;
/**
* Which mail this is, as a short tag (letters, digits, `_`, `-`). It goes
* to SES as a message tag, so a bounce or complaint event names the kind of
* mail it was about, and into the log line of the send. Never a value that
* identifies a person.
*/
kind: string;
}

export async function sendEmail(message: EmailMessage): Promise<void> {
const { SMTP_HOST, RESEND_API_KEY } = serverEnv();

if (process.env.NODE_ENV === "production") {
// Production never falls back to the dev transport: it fails closed.
if (SMTP_HOST) {
const { sendViaSmtp } = await import("./smtp-transport");
try {
await sendViaSmtp(message);
return;
} catch (error) {
// SMTP is primary; fall back to Resend if available, otherwise rethrow.
if (!RESEND_API_KEY) {
throw error;
}
const { sendViaResend } = await import("./resend-transport");
await sendViaResend(message, RESEND_API_KEY);
return;
}
}

if (RESEND_API_KEY) {
const { sendViaResend } = await import("./resend-transport");
await sendViaResend(message, RESEND_API_KEY);
return;
if (!serverEnv().SMTP_HOST) {
throw new Error("Email delivery is not configured: set SMTP_HOST in production");
}

throw new Error(
"Email delivery is not configured: set SMTP_HOST or RESEND_API_KEY in production"
);
const { sendViaSmtp } = await import("./smtp-transport");
const { messageId } = await sendViaSmtp(message);
console.info(`[email] sent kind=${message.kind} id=${messageId}`);
return;
}

const { sendViaDevTransport } = await import("./dev-transport");
Expand Down
1 change: 1 addition & 0 deletions apps/web/lib/email/quota-exhausted-email.ts
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,7 @@ export function buildQuotaExhaustedEmail(params: {

return {
to,
kind: "quota_exhausted",
subject: "Your webhooks.cc endpoints are rejecting webhooks",
text: [
...intro,
Expand Down
26 changes: 0 additions & 26 deletions apps/web/lib/email/resend-transport.ts

This file was deleted.

36 changes: 36 additions & 0 deletions apps/web/lib/email/smtp-transport.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import { describe, expect, it } from "vitest";
import { sesHeaders, sesMessageId } from "./smtp-transport";

describe("sesHeaders", () => {
it("sends nothing without a configuration set", () => {
expect(sesHeaders({ kind: "welcome" }, undefined)).toEqual({});
});

it("names the configuration set and tags the kind", () => {
expect(sesHeaders({ kind: "agent_code" }, "webhooks-cc-app")).toEqual({
"X-SES-CONFIGURATION-SET": "webhooks-cc-app",
"X-SES-MESSAGE-TAGS": "kind=agent_code",
});
});

it("drops characters SES refuses in a tag value", () => {
expect(sesHeaders({ kind: "a b,c=d;e" }, "set")["X-SES-MESSAGE-TAGS"]).toBe("kind=abcde");
expect(sesHeaders({ kind: " " }, "set")["X-SES-MESSAGE-TAGS"]).toBe("kind=unknown");
});
});

describe("sesMessageId", () => {
it("reads the id from SES's reply to the message", () => {
expect(
sesMessageId(
"250 Ok 0107019a6d1f2b3c-4d5e6f70-1234-4abc-9def-0123456789ab-000000",
"<x@local>"
)
).toBe("0107019a6d1f2b3c-4d5e6f70-1234-4abc-9def-0123456789ab-000000");
});

it("falls back to the generated Message-ID for other servers", () => {
expect(sesMessageId("250 2.0.0 Ok: queued as 4ABC", "<x@local>")).toBe("<x@local>");
expect(sesMessageId(undefined, "<x@local>")).toBe("<x@local>");
});
});
Loading
Loading