Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
8eae563
docs: plan file uploads on the submit page
coa-chrisc Oct 16, 2026
4e22de3
feat(uploads): add the upload: reference contract
coa-chrisc Oct 16, 2026
450e047
feat(uploads): add the stateless upload service
coa-chrisc Oct 16, 2026
be45344
feat(uploads): mint the scaffolder's read token and clean up uploads
coa-chrisc Oct 16, 2026
521133d
feat(uploads): resolve upload: references in the scaffolder
coa-chrisc Oct 16, 2026
35a8f0e
feat(uploads): attach files on the submit page
coa-chrisc Oct 16, 2026
9b51b9f
fix(uploads): clean up when a draft pull request closes
coa-chrisc Oct 16, 2026
8bcfb61
docs(uploads): document file uploads on the submit page
coa-chrisc Oct 16, 2026
f32bcf7
feat(submit): generate agent instructions and an answers schema
coa-chrisc Oct 16, 2026
c52c19e
feat(submit): import an agent's answers on the submit page
coa-chrisc Oct 16, 2026
164b471
chore(uploads): say why a failed stream cancel is ignored
coa-chrisc Oct 16, 2026
40a2452
fix(uploads): harden cleanup, token minting and per-user caps
coa-chrisc Oct 16, 2026
435ef32
fix(submit): remove the right upload and keep files on fallback
coa-chrisc Oct 16, 2026
6927243
test: match .gitattributes lines exactly instead of building a regex
coa-chrisc Oct 16, 2026
a86cc8c
style: format agent_files test
coa-chrisc Oct 16, 2026
697de15
fix(submit): say an expired sign-in leaves uploaded files behind
coa-chrisc Oct 17, 2026
e09229d
fix(uploads): request blobs with GitHub's documented raw media type
coa-chrisc Oct 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,5 @@ docs/bchc/** merge=ours
.github/ISSUE_TEMPLATE/new-entry.yml merge=ours
.github/ISSUE_TEMPLATE/config.yml merge=ours
assets/js/configurator/defaults.generated.js merge=ours
submit/answers.schema.json merge=ours
submit/agent-instructions.txt merge=ours
45 changes: 45 additions & 0 deletions .github/workflows/new-entry.yml
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,45 @@ jobs:
const notify = await import(pathToFileURL(`${process.env.GITHUB_WORKSPACE}/scripts/lib/notify_github.mjs`).href);
await notify.notifyIssue({ github, context, core, status: notify.STATUS.received });

# Optional: files attached on /submit/ live in a PRIVATE uploads repository
# (docs/submit-uploads-plan.md), and the issue body names them with
# `upload:` lines. Reading them needs a credential for that repository, so
# this mints one — and only when the repository variable UPLOADS_REPO
# (`owner/name`) is set; unset, neither step runs and nothing changes.
# The token is a GitHub App installation token that can READ CONTENTS of
# that one repository and nothing else (`permission-contents: read`), it
# expires within the hour, and it is handed to the scaffold step below and
# to no other step. Deleting uploads is a separate, write-capable token
# minted in uploads-cleanup.yml, never here. The variable is split in a
# step of its own because the action takes the owner and the repository
# name separately and expressions cannot split a string.
#
# Both steps are non-fatal: a misconfigured or unreachable App must not
# cost a submitter their draft. A failed mint leaves the token empty, and
# the scaffolder drafts the entry anyway with one warning per `upload:`
# answer it could not collect (scripts/lib/uploads.mjs).
- name: Split the uploads repository name
id: uploads-repo
if: vars.UPLOADS_REPO != ''
continue-on-error: true
env:
UPLOADS_REPO: ${{ vars.UPLOADS_REPO }}
run: |
echo "owner=${UPLOADS_REPO%%/*}" >> "$GITHUB_OUTPUT"
echo "name=${UPLOADS_REPO#*/}" >> "$GITHUB_OUTPUT"

- name: Mint a read-only token for the uploads repository
id: uploads-token
if: vars.UPLOADS_REPO != ''
continue-on-error: true
uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
with:
app-id: ${{ secrets.UPLOADS_APP_ID }}
private-key: ${{ secrets.UPLOADS_APP_PRIVATE_KEY }}
owner: ${{ steps.uploads-repo.outputs.owner }}
repositories: ${{ steps.uploads-repo.outputs.name }}
permission-contents: read

- name: Scaffold entry files
id: scaffold
continue-on-error: true
Expand All @@ -166,6 +205,12 @@ jobs:
ISSUE_NUMBER: ${{ github.event.issue.number }}
# Lets the scaffolder download images attached to a private issue.
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Empty when UPLOADS_REPO is unset. The scaffolder resolves `upload:`
# lines against the ref built from the ISSUE AUTHOR's id, so a
# reference copied into someone else's issue finds nothing.
UPLOADS_TOKEN: ${{ steps.uploads-token.outputs.token }}
UPLOADS_REPO: ${{ vars.UPLOADS_REPO }}
ISSUE_AUTHOR_ID: ${{ github.event.issue.user.id }}

- name: Report the failure on the issue
if: steps.scaffold.outcome == 'failure'
Expand Down
134 changes: 134 additions & 0 deletions .github/workflows/uploads-cleanup.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
name: Uploads cleanup

# Files attached on /submit/ are stored by the optional upload service as one
# branch each (`u/<user-id>/<set>/<sha256>`) in a PRIVATE uploads repository
# (docs/submit-uploads-plan.md). This deletes them once they are not needed:
#
# * when a `content:new-entry` issue closes, or its draft pull request
# (branch `entry/<slug>-<issue>`) closes merged or declined, the author's
# sets named in the issue body. The pull request trigger exists because a
# close made with the built-in GITHUB_TOKEN (a decline, a "Closes #N" on
# merge) starts no `issues: closed` run;
# * every day, any set older than 14 days that no open `content:new-entry`
# issue by the same author still references.
#
# Deletion targets always come from the uploads repository's own ref listing;
# issue text can only cause a set to be kept (scripts/uploads_cleanup.mjs).
#
# Off until the repository variable UPLOADS_REPO (`owner/name`) is set, with the
# secrets UPLOADS_APP_ID and UPLOADS_APP_PRIVATE_KEY of a GitHub App installed
# on that repository. A manual run from the Actions tab sweeps; its "Preview
# only" box lists what would go without deleting anything.

on:
issues:
types:
- closed
# Same-repository drafts only (the job's `if`); the branch name is read from
# the event and used through `env:` only. The checkout below names the default
# branch because a declined, conflicting draft has no merge ref to check out.
pull_request:
types:
- closed
schedule:
# 04:47 UTC daily, off the hour-zero peak where scheduled runs queue for ages.
- cron: "47 4 * * *"
workflow_dispatch:
inputs:
dry_run:
description: "Preview only — list what would be deleted, delete nothing"
type: boolean
default: false

permissions: {}

# GitHub keeps only ONE pending run per concurrency group and cancels any
# older pending one, so a single shared group would drop queued close runs.
# Each closed issue or pull request gets a group of its own (a re-close of the
# same one waits behind the first, and a dropped pending duplicate would have
# done the same work); the daily and manual sweeps share the `sweep` group.
# Runs in different groups may overlap: every deletion target comes from a
# fresh ref listing and a ref that is already gone (404/422) counts as "already
# gone", not a failure, so two runs deleting the same files is harmless.
concurrency:
group: uploads-cleanup-${{ github.event.issue.number || github.event.pull_request.number || 'sweep' }}
cancel-in-progress: false

jobs:
# A job whose `if` is false shows up as a bare grey "skipped" with no reason
# (see metrics.yml). This one runs only when the feature is off and says so.
turned-off:
if: >-
vars.UPLOADS_REPO == '' && (github.event_name != 'pull_request' || (contains(github.event.pull_request.labels.*.name, 'content:new-entry') && github.event.pull_request.head.repo.full_name == github.repository && startsWith(github.event.pull_request.head.ref, 'entry/')))
&& (github.event_name != 'issues' || contains(github.event.issue.labels.*.name, 'content:new-entry'))
runs-on: ubuntu-latest
steps:
- name: Say why nothing ran
run: |
echo "Uploads cleanup is turned off: the repository variable UPLOADS_REPO is not set, so there is no uploads repository to clean. Set it in Settings -> Secrets and variables -> Actions -> Variables to turn uploads on." >> "$GITHUB_STEP_SUMMARY"

cleanup:
if: >-
vars.UPLOADS_REPO != '' && (github.event_name != 'pull_request' || (contains(github.event.pull_request.labels.*.name, 'content:new-entry') && github.event.pull_request.head.repo.full_name == github.repository && startsWith(github.event.pull_request.head.ref, 'entry/')))
&& (github.event_name != 'issues' || contains(github.event.issue.labels.*.name, 'content:new-entry'))
runs-on: ubuntu-latest
permissions:
# Checkout, and listing this repository's open issues for the sweep.
contents: read
issues: read

steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.repository.default_branch }}
persist-credentials: false

- name: Setup Node
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version-file: ".node-version"
cache: npm

# The script uses Node built-ins and scripts/lib only; no npm install.
- name: Select exact npm
run: node scripts/install_exact_npm.mjs

- name: Split the uploads repository name
id: uploads-repo
env:
UPLOADS_REPO: ${{ vars.UPLOADS_REPO }}
run: |
echo "owner=${UPLOADS_REPO%%/*}" >> "$GITHUB_OUTPUT"
echo "name=${UPLOADS_REPO#*/}" >> "$GITHUB_OUTPUT"

# The only place a write-capable token for the uploads repository is
# minted: contents write on that one repository (deleting a branch is a
# contents write), valid for under an hour. new-entry.yml mints a
# read-only one for scaffolding.
- name: Mint a token for the uploads repository
id: uploads-token
uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
with:
app-id: ${{ secrets.UPLOADS_APP_ID }}
private-key: ${{ secrets.UPLOADS_APP_PRIVATE_KEY }}
owner: ${{ steps.uploads-repo.outputs.owner }}
repositories: ${{ steps.uploads-repo.outputs.name }}
permission-contents: write

# Issue text reaches the script through `env:` only, never a `run:` line.
- name: Delete unneeded uploads
run: node scripts/uploads_cleanup.mjs
env:
MODE: ${{ github.event_name == 'issues' && 'closed' || github.event_name == 'pull_request' && 'pr-closed' || 'sweep' }}
# pr-closed: the issue is found from the draft branch name, which the
# script re-checks; the fork check is in the job's `if`.
PR_BRANCH: ${{ github.event.pull_request.head.ref }}
ISSUE_NUMBER: ${{ github.event.issue.number }}
UPLOADS_TOKEN: ${{ steps.uploads-token.outputs.token }}
UPLOADS_REPO: ${{ vars.UPLOADS_REPO }}
CATALOG_REPO: ${{ github.repository }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ISSUE_BODY: ${{ github.event.issue.body }}
ISSUE_AUTHOR_ID: ${{ github.event.issue.user.id }}
DRY_RUN: ${{ inputs.dry_run }}
2 changes: 2 additions & 0 deletions .phct/ownership.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ ownership:
- .github/ISSUE_TEMPLATE/new-entry.yml
- .github/ISSUE_TEMPLATE/config.yml
- assets/js/configurator/defaults.generated.js
- submit/answers.schema.json
- submit/agent-instructions.txt

# Protected from the parent merge, then deliberately updated by the updater.
# It is excluded from byte-for-byte deployment content checks.
Expand Down
4 changes: 4 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,9 @@ assets/css/
# Liquid template that emits JSON — it is not parseable JSON at rest.
entries.json

# Generated by `npm run generate`: Liquid inside JSON, and plain text; neither is formatted.
submit/answers.schema.json
submit/agent-instructions.txt

# Written by `vercel link` when the sign-in service is deployed; gitignored there.
.vercel/
39 changes: 39 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,45 @@ major version, and each entry says so when it happens.

## [Unreleased]

### Added

- Optional **attach files on the submit page**. Set `submit.uploads_url` in
`_data/site.yml` (it needs `submit.signin_url`; `npm run validate` checks
both) and a signed-in submitter gets a file picker on the screenshots question,
with alt text beside each picture, and on each file question (the slide deck).
Each file uploads as soon as it is picked, pictures are re-encoded in the browser
first (EXIF and location data stripped, longest side 2,400 px), the draft remembers
the files, and the issue carries `upload:` references instead of asking the
submitter to drag files onto GitHub afterwards. Off by default; blank keeps the page
exactly as it was. Files are limited to 4 MB each, 3 drafts and 9 files per draft per
person.
- `services/uploads/`, the **upload service** behind it: a stateless Vercel project with
no runtime dependencies. It checks the submitter's GitHub token, the origin, the
file's real type and size and the per-person caps, then writes each file as its own
parentless commit and branch in a private uploads repository, with a second GitHub App
of its own. Setup is in `services/uploads/README.md`.
- The scaffolder resolves `upload:` references. `new-entry.yml` mints a read-only token for
the uploads repository (repository variable `UPLOADS_REPO`, secrets `UPLOADS_APP_ID` and
`UPLOADS_APP_PRIVATE_KEY`) and hands it to the scaffold step only; the files are read as
the issue's author, re-hashed, checked like any attachment and committed into the entry.
An unset or broken setup, or a file that has expired, becomes a warning on the draft,
and dragging files onto the issue keeps working.
- **Uploads cleanup** workflow (`uploads-cleanup.yml`): deletes a submission's files when
its draft pull request is merged or closed (or its issue is closed by hand), and
daily sweeps any file older than 14 days that no open submission names. A manual run with
**Preview only** lists what would go. Admin guide: *Files attached on the submit page*,
including how to purge something sensitive.
- "Using an AI agent?" on `/submit/`: an instructions file and a JSON Schema for answers,
both generated from `_data/schema.yml` by `npm run generate`, and an **Import answers**
box that loads an agent's JSON into the form and runs the form's own checks.

### Security

- `SECURITY.md` and the sign-in README now say that, on a deployment that also runs the
upload service, the submitter's sign-in token is sent to that service as well as to
`api.github.com` (one `GET /user` call), and that the private uploads repository does
not keep screenshots private once a submission is scaffolded.

## [1.11.0-rc.10] — 2026-10-13

Tenth candidate. rc.9 failed closed in BCHC on three new tests that read the
Expand Down
36 changes: 36 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,42 @@ See [docs/admin-guide.md](docs/admin-guide.md#checks-on-a-generated-pull-request
Every `uses:` in `.github/workflows/` names a full commit SHA with the release
tag in a trailing comment. Dependabot proposes updates weekly.

## File uploads on the submit page (optional)

Off unless a deployment sets `submit.uploads_url`. Then a signed-in submitter can attach screenshots
and a PDF deck on `/submit/` through a small service the deployment runs
([`services/uploads/`](services/uploads/README.md)). What that changes:

- **The service stores nothing.** No database, no file storage, nothing on disk. It checks each file
(origin, a real PNG, JPEG, WebP or PDF judged by its bytes, at most 4 MB, per-person caps) and writes
it to a **private** GitHub repository in the same request. It has no runtime dependencies.
- **A private repository does not keep screenshots private.** It protects drafts that were never
submitted and stops the service being an anonymous file host. Once a submission is scaffolded, its
files are committed to a public `entry/*` draft branch and shown in the draft pull request before
anyone has reviewed them, just as files dragged into an issue are public the moment they are attached.
The page tells submitters not to show personal information; reviewers still open every picture at
full size ([docs/incidents.md](docs/incidents.md)).
- **The submitter's sign-in token goes to the upload service too.** Without uploads it is sent only to
`api.github.com`. With uploads the page also sends it to the service the deployment runs, which uses
it for one `GET /user` call to learn who is uploading and never logs, stores or forwards it. It can
do only what the sign-in App allows, and cannot touch the uploads repository.
- **A reference cannot name someone else's files.** The scaffolder builds the branch name itself from
the issue author's numeric id, so a reference copied into another person's issue finds nothing, and
no reference can name another repository. It re-hashes every file and rejects a mismatch.
- **The uploads token is read-only and reaches one step.** The scaffold job mints a token that can
read contents of the uploads repository only, hands it to the scaffold step alone, and the generic
image and attachment fetchers (which send their token to any GitHub host a submitter names) never
see it. Deleting uses a separate write token minted only in `uploads-cleanup.yml`, whose deletion
targets always come from the repository's own branch listing; issue text can only cause a set to be
kept.
- **Deleted is not erased.** One parentless commit per file means deleting a branch leaves the file
unreachable, but git keeps it until GitHub garbage-collects. To purge something sensitive at once,
contact GitHub Support. The procedure is in
[docs/admin-guide.md](docs/admin-guide.md#files-attached-on-the-submit-page).

Report a problem with the upload service the same way as any other: see
[Reporting a vulnerability](#reporting-a-vulnerability) below.

## Who can submit

Open by default: the catalog is meant to collect work from people without
Expand Down
6 changes: 6 additions & 0 deletions _data/site.yml
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,12 @@ submit:
# answers as an issue straight from /submit/, however long. Blank keeps the
# link hand-off only. See docs/configuration.md.
signin_url: ""
# Optional file uploads on /submit/: the address of the upload service you
# deploy from services/uploads/ (e.g. https://catalog-uploads.vercel.app).
# Signed-in submitters can then attach screenshots and a PDF deck right on
# the page. Needs signin_url, because uploads work only for signed-in
# submitters. Blank turns it off. See docs/configuration.md.
uploads_url: ""

# "Check your submission" page (module: status). A submitter types the number
# GitHub gave their submission and sees its title, date and stage, read from
Expand Down
3 changes: 2 additions & 1 deletion assets/js/configurator/defaults.generated.js
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,8 @@ export const SITE = {
"turnaround": "Intake checks it within about five business days and the Governance Committee reviews it within about ten more; you keep ownership of anything you share.",
"review_note": "Please do not include protected health information, credentials or non-public data. Link out to repositories and documents rather than pasting sensitive content.",
"fallback_email": "catalog@example.org",
"signin_url": ""
"signin_url": "",
"uploads_url": ""
},
"status": {
"heading": "Check your submission",
Expand Down
Loading
Loading