Skip to content

Share the explanation page on the PR - #26

Merged
adamw merged 19 commits into
VirtusLab:masterfrom
stivens:feat/share-preview-on-pr
Nov 17, 2026
Merged

adamw merged 19 commits into
VirtusLab:masterfrom
stivens:feat/share-preview-on-pr

Conversation

@stivens

@stivens stivens commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Adds a way to share Tulip's explanation page on the PR itself, so other reviewers can read it without running Tulip (and paying for the LLM passes) themselves.

Important

This is a nice-to-have feature, and hosting the page is the troublesome part because of GitHub's limitations, so maybe we want some other approach. GitHub has no API for attaching a file to a comment (and .html isn't an accepted attachment type anyway), so this PR stores the page in the repo on hidden refs and links it from a comment. That works, but it has real downsides, listed under Limitations.

What it does

The page is already a single self-contained index.html (ADR 0030). Each run now also signs it: <meta> tags in its <head> record the PR URL, the head commit, the render time and an HMAC signature. There are three ways to share it:

tulip <PR URL> --attach-preview-as-comment   # as soon as the run ends
tulip share <path to index.html>             # after reading the page, or to retry
tulip <PR URL> --serve                       # click "Share on PR" on the page

Sharing does three things:

  1. Stores the page. It becomes an orphan commit under a hidden ref, refs/tulip/previews/pr-<n>/<commit>. The ref goes in the PR's repo if you can push to it, otherwise in your fork, which is created if needed.
  2. Comments on the PR. The comment links the file, says which commit it explains, and gives a gh api command that downloads it.
  3. Keeps the comment current. Sharing again for the same commit edits your comment. Sharing for a new commit posts a new comment, so reviewers get notified, and hides the old one as outdated (or, without permission to hide it, edits it into a pointer to the new one).

Example comment:

🌷 Tulip explanation of this PR (as of a6ff060): tulip-pr-3-a6ff060.html (2026-10-19 14:36 UTC)

The changes grouped by concern, most important first, with diagrams. Download the file (from its page, or with this command) and open it in a browser:

gh api 'repos/acme/widgets/contents/tulip-pr-3-a6ff060.html?ref=<commit>' -H 'Accept: application/vnd.github.raw' > tulip-pr-3-a6ff060.html

Limitations of this approach

  • No in-browser viewing. GitHub doesn't render HTML from a repo, so reviewers download the file and open it locally. It doesn't work on mobile.
  • A warning on the file page. GitHub shows "This commit does not belong to any branch" there. The comment explains why, but it still looks odd.
  • Writes to the repo. Writing needs push access, or a fork, which some orgs don't allow. Push rulesets, SAML SSO or token restrictions only show up when the write happens. A just-created fork may not accept writes yet; sharing then fails and says to retry in a minute.
  • Fork access. A private repo's fork is never more visible than the repo, but it gets only team permissions: people with individual access to the repo can read the PR but not a preview stored in a fork.
  • Previews pile up. They are kept on the hidden refs, which clones don't fetch, and nothing prunes them. ADR 0032 explains how to delete them by hand.
  • GitHub only. GitLab could use its native uploads API, but that isn't implemented.

Other approaches we could take instead:

  • Explanation in the comment. Post the explanation itself as markdown (GitHub renders mermaid diagrams), and maybe keep the HTML as the "full version".
  • A CI action. A GitHub Action could upload the page as a workflow artifact. This needs LLM credentials in CI.
  • Real hosting. Host the page somewhere that renders it, for example as part of Visdom. GitHub Pages is public for private repos outside Enterprise, so it doesn't fit.

Security notes

  • Only pages from your own runs. tulip share only publishes a page signed with your key: HMAC-SHA256, per user, created atomically next to Tulip's settings with permissions 0600. The signature sits in the page itself; it is computed with a placeholder in its place, and only the leading <head> Tulip writes is read. A page someone hands you, or one edited since, can't be posted under your name. If the page can't be signed (e.g. a read-only home), sharing is turned off for that run and the reason is logged.
  • Page CSP. The page's deny-by-default CSP (ADR 0030) allows its inlined scripts by hash, so model-written markup still can't run scripts or fetch anything.
  • Comment state. A comment counts as Tulip's only if it starts with Tulip's header and ends with the hidden marker, so a comment quoting the marker is never edited or hidden. Anyone with write access can edit the marker, so it's treated as untrusted:
    • an earlier link is carried over only if it is exactly the URL Tulip would build for that preview;
    • times are re-serialized;
    • < and > are escaped.
  • Serve endpoint. POST /api/share keeps serve mode's Host and Origin guards and also requires a JSON content type.
  • Stale previews. Sharing refuses a closed PR, and a PR that has moved past the previewed commit.

Testing

  • pnpm lint, tsc and pnpm test all pass (1122 tests).
  • Live checks on a private test repo (VirtusLab/tulip-test-repo):
    • hidden refs, created through the API or git push, start no unfiltered on: push workflow and aren't fetched by a default clone;
    • a run with the flag; the comment's download command returns a file byte-identical to the local page;
    • tulip share for the same commit edits the comment;
    • a new commit posts a new comment and hides the old one as outdated;
    • a tampered page and a page for an outdated commit are refused, and leave no ref behind;
    • the serve button, and its Host, Origin and content-type guards; Ctrl+C exits cleanly.
  • The fork path, which needs an account without push access -> Share the explanation page on the PR #26 (comment)

Reviewing

  • The design and the alternatives that were considered are in docs/adr/0032-preview-comment.md.
  • Existing code changed only where the feature needs it:
    • the template gets the tulip-* meta tags and the Share button;
    • render.ts signs index.html, and renders serve mode's page separately; the server now serves that page from memory, so index.html is always the static page that gets shared;
    • run.ts wires sharing in;
    • server.ts gets the new route, with one Origin check for all POSTs;
    • the "gh CLI not found" message no longer says "run without --serve".

🤖 Generated with Claude Code

@stivens

stivens commented Oct 19, 2026

Copy link
Copy Markdown
Member Author

🌷 Tulip explanation of this PR (as of 2af6393): tulip-pr-26-2af6393.html (2026-10-19 16:07 UTC)

Download the raw file and open it in a browser: the changes grouped by concern, most important first, with diagrams.

GitHub's "not on any branch" warning is expected: previews live on hidden refs.

@stivens
stivens marked this pull request as ready for review October 19, 2026 16:08
stivens and others added 9 commits November 17, 2026 09:28
Every run writes a signed run manifest next to the single-file page
(ADR 0030). Three ways to share the page:

- --attach-preview-as-comment: share as soon as the run ends
- tulip share <run dir>: share an earlier run, after reading it or to retry
- a "Share on PR" button in --serve mode

The run dir's index.html is always the static page; --serve serves its
own page, with the review boxes and Share button, from memory.

Sharing stores the page as an orphan commit under a hidden ref
(refs/tulip/previews/pr-<n>/<commit>) in the PR's repo, or the user's
fork without push access, and posts or updates a PR comment linking it.
GitHub only. Design and trade-offs: docs/adr/0029-preview-comment.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a fork created in the last ten minutes isn't ready for writes (404/409),
the share now fails at once asking to retry in a minute, instead of retrying
for up to five minutes.
Closing the server waits for an in-flight POST /api/share, so stopping the
share only after serveForReview resolved let it keep running after Ctrl+C.
The PR URL, head commit, render time and HMAC now live in meta tags at the
start of index.html's head, so the page alone can be shared with tulip share.
Write the key to a temp file and link it into place, so no run reads a
partial key and an existing key is never replaced.
An earlier-preview link must equal the URL storePreviewFile builds for this PR and the
entry's commit, in the PR or store repo. Replaces the looser prefix and URL-shape checks.
One Origin check for all POSTs, one errorMessage helper for the new code, and
shared helpers for the page's POSTs, CLI argument errors and the commit SHA pattern.
GitHub serves the page as text and may not show a multi-MB file, so the comment
now gives a command that downloads the current preview via the contents API.
Private forks get only the repo's team permissions, so people with individual access can't open the preview.
@adamw
adamw force-pushed the feat/share-preview-on-pr branch from 2af6393 to bb77a55 Compare November 17, 2026 09:56
adamw added 10 commits November 17, 2026 10:17
Checked on github.com that refs/tulip/* refs start no on: push workflow and aren't cloned.
0029 was already taken by the context-dirs ADR on master.
A user's comment that only quotes the marker was edited or minimized. Now the comment must start with Tulip's header and end with the marker, whose state is read from the end.
No Share button, no tulip share hint, and --attach-preview-as-comment skips the share and exits 1.
Define the signature length once, reuse errorMessage in run.ts, and check the preview file name for shell safety instead of re-encoding its format.
Within a run, build the preview from the signed page render returns instead of reading it back from disk; tulip share still verifies the file.
Drop the fork-age check: it only picked between two messages. Keep GitHub's error and add the retry hint.
Tests for others' comments, a PR closed before publish, fork-stored links, the latest comment, the history cap and a failed Share request.
The Share button, serve shutdown and retry tests now wait for the state they need.
@adamw
adamw merged commit a908d3a into VirtusLab:master Nov 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants