Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
8c672c9
fix(ci): ci-full reads the suite title renames pin by the name the fl…
jdogresorg Sep 17, 2026
250a9a7
test(pins): record AT1 suite titles, test wall times and the identity…
jdogresorg Sep 17, 2026
90cd827
ci: sibling checkouts follow the pull request's base branch, keeping …
jdogresorg Sep 17, 2026
51d4a13
test: split encoder observability registrations
jdogresorg Sep 17, 2026
3292ea4
fix(ci): read the identity pin from bin/pins/at1-identity.json
jdogresorg Sep 17, 2026
9381dae
merge: level develop with master for the v0.20.1 patch train
jdogresorg Sep 19, 2026
0978c82
docs: restore encoder comment rationale
jdogresorg Sep 21, 2026
ee71444
Merge: docs: restore encoder comment rationale
jdogresorg Sep 21, 2026
716156b
fix test unit setup loading
jdogresorg Sep 21, 2026
7046790
Merge: fix test unit setup loading
jdogresorg Sep 21, 2026
9a53ce1
test: capture reveal prefund warnings at logger
jdogresorg Sep 21, 2026
e433c28
encoder: mark leg-headroom capability and log the undersized P2SH leg…
jdogresorg Sep 21, 2026
4dd04d4
Merge: test: capture reveal prefund warnings at logger
jdogresorg Sep 21, 2026
cc95837
Merge: encoder: mark leg-headroom capability and log the undersized P…
jdogresorg Sep 21, 2026
27e5269
fix(docker): reap encoder container zombies
jdogresorg Sep 22, 2026
6daf8ff
chore(observability): vendor the hub canonical module, unmatched-rout…
jdogresorg Sep 22, 2026
4b155d8
chore(pins): re-pin the vendored observability module after the hub sync
jdogresorg Sep 22, 2026
0f5e863
build(ci): grade the fast tier on a push, defer the heavy tiers to th…
jdogresorg Sep 22, 2026
d81dac4
Merge: fix(docker): reap encoder container zombies
jdogresorg Sep 22, 2026
58d96a0
Run API under tini
jdogresorg Sep 22, 2026
9149e0c
docs(encoder): repoint guard and limiter comments at the reorganized …
jdogresorg Sep 23, 2026
7a86549
docs(encoder): repoint guard and limiter comments at the reorganized …
jdogresorg Sep 23, 2026
7455f97
refactor(encoder): drop named exports nothing imports
jdogresorg Sep 23, 2026
3d08713
fix(encoder): gate before parsing large bodies, keep node error bodie…
jdogresorg Sep 23, 2026
5a8b7e5
test(encoder): resolve the manifest byte-identity guard through the s…
jdogresorg Sep 23, 2026
5eff25b
Merge: Run API under tini
jdogresorg Sep 23, 2026
afdec82
Merge: docs(encoder): repoint guard and limiter comments at the reorg…
jdogresorg Sep 23, 2026
d6784a1
chore(release): v0.20.1
jdogresorg Sep 23, 2026
336a10d
Pin the Node base image by digest to the consensus runtime
jdogresorg Sep 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 16 additions & 15 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
# touch neither the guard nor the file it reads. Empty everywhere
# else, which keeps the existing behaviour. Same expression as
# xchain-indexer and xchain-explorer.
siblings-ref: ${{ (startsWith(github.head_ref, 'release/') || startsWith(github.head_ref, 'hotfix/')) && github.head_ref || '' }}
siblings-ref: ${{ (startsWith(github.head_ref, 'release/') || startsWith(github.head_ref, 'hotfix/')) && github.head_ref || github.base_ref || '' }}
# Override the Node version for a repo if ever needed:
# with:
# node-version: "20"
Expand All @@ -52,15 +52,15 @@ jobs:
uses: actions/checkout@v4
with:
repository: XChain-Platform/xchain-hub
ref: ${{ github.ref == 'refs/heads/master' && 'master' || 'develop' }}
ref: ${{ github.base_ref || (github.ref == 'refs/heads/master' && 'master' || 'develop') }}
ssh-key: ${{ secrets.XCHAIN_HUB_DEPLOY_KEY }}
path: xchain-hub

- name: Check out the sibling xchain-decoder (envelope gate twin)
uses: actions/checkout@v4
with:
repository: XChain-Platform/xchain-decoder
ref: ${{ github.ref == 'refs/heads/master' && 'master' || 'develop' }}
ref: ${{ github.base_ref || (github.ref == 'refs/heads/master' && 'master' || 'develop') }}
path: xchain-decoder

- name: Use Node.js 22
Expand Down Expand Up @@ -93,22 +93,23 @@ jobs:
console.log("consensus pin conformance OK (testnet, regtest)");
'

# Identity pin: bin/pins/identity.json records the sha256 of the vendored
# coin files and the roundtrip conformance fixture. The tool compares only
# the entries the pin names, so an emptied pin would read as holding: first
# refuse a pin with no coin or conformance entries, then fail on any moved
# or missing file, so a stale pin cannot reach develop or master green
# through a direct push or a release pull request. bin/ci-full.sh runs the
# same commands before a push; both use only node builtins, so no install.
- name: Identity pin (vendored coins, conformance fixture)
# Identity pin: bin/pins/at1-identity.json records the sha256 of the
# vendored coin files and every whole-file vendored twin. The tool
# compares only the entries the pin names, so an emptied pin would read
# as holding: first refuse a pin with no coin or vendored-twin entries,
# then fail on any moved or missing file, so a stale pin cannot reach
# develop or master green through a direct push or a release pull
# request. bin/ci-full.sh runs the same commands before a push; both
# use only node builtins, so no install.
- name: Identity pin (vendored coins, vendored twin files)
working-directory: xchain-encoder
run: |
node -e '
const pin = require("./bin/pins/identity.json");
for (const group of ["coins", "conformance"]) {
const pin = require("./bin/pins/at1-identity.json");
for (const group of ["coins", "vendoredTwins"]) {
if (!Object.keys(pin[group] || {}).length) throw new Error("identity pin names no " + group + " files");
}
' && node bin/pin-identity.js --compare bin/pins/identity.json
' && node bin/pin-identity.js --compare bin/pins/at1-identity.json

# Coverage ratchet: re-run the unit suite under c8 and fail if line or
# branch coverage drops below this repo's floor in bin/coverage-thresholds.json
Expand All @@ -134,7 +135,7 @@ jobs:
# whose cross-repo guards disagree with the gate for a reason that exists
# nowhere but in CI.
with:
ref: ${{ (startsWith(github.head_ref, 'release/') || startsWith(github.head_ref, 'hotfix/')) && github.head_ref || '' }}
ref: ${{ (startsWith(github.head_ref, 'release/') || startsWith(github.head_ref, 'hotfix/')) && github.head_ref || github.base_ref || '' }}

- name: Use Node.js 22
uses: actions/setup-node@v4
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.20.1] - 2026-09-23

### Fixed
- Rejected oversized request bodies before parsing, preserved node error responses, and ran the API under an init process that reaps child processes.


## [0.20.0] - 2026-09-17

### Fixed
Expand Down
23 changes: 10 additions & 13 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
# Pinned to node:22-bookworm, the tag .nvmrc and package.json engines already
# declare and the sibling service images already build on. `node:latest` floats:
# xchain-node rebuilds this image on every update (ModuleService.buildAndUp), so
# a routine rolling upgrade silently moves the runtime off the declared Node 22
# with no signal anywhere.
FROM node:22-bookworm
# Pinned by digest to the node:22.23.2-bookworm image whose V8/ICU build
# matches xchain-vm's consensus runtime pin: the floating node:22-bookworm
# tag can advance to a Node patch that fails that check.
FROM node:22.23.2-bookworm@sha256:dd5847a04b0deee391fa145f1f4c6d214196668b6bcc7988ebed67249f226844

RUN apt-get update \
&& apt-get install -y --no-install-recommends tini \
&& rm -rf /var/lib/apt/lists/*

RUN mkdir /XChainEncoder/
COPY ./package.json /XChainEncoder/package.json
Expand All @@ -17,10 +19,5 @@ COPY ./docs /XChainEncoder/docs
# (xchain-node at `docker run`, docker-compose.yml via env_file). An optional
# `COPY ./.en[v]` glob here builds only under BuildKit.

# Run node directly rather than through `npm run api` (which is this exact
# command). npm builds a three-process tree, npm -> sh -c -> node, and neither
# wrapper forwards signals: measured on the regtest encoder, `docker stop` kills
# npm, node is never told anything and dies with the container, so its SIGTERM
# handler never runs and the instance lockfile survives into the next boot.
# Exec form, no shell, so node is PID 1 and gets the signal itself.
CMD ["node", "./src/api.js"]
ENTRYPOINT ["tini", "--"]
CMD ["node", "./src/api.js"]
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# XChain Platform Encoder

<p align="center">
<img src="https://img.shields.io/badge/version-0.20.0-blue" alt="Version">
<img src="https://img.shields.io/badge/version-0.20.1-blue" alt="Version">
<img src="https://img.shields.io/badge/tests-1%2C787%2B%20passing-brightgreen" alt="Tests">
<img src="https://img.shields.io/badge/node-%3E%3D22-green" alt="Node">
<img src="https://img.shields.io/badge/license-AGPL--3.0--or--later-blue" alt="License">
Expand All @@ -31,7 +31,7 @@ PSBT encoding service for the XChain Platform. Takes an ACTION string, a set of
- **Token-gated content support**: encodes [FILE v1](https://github.com/XChain-Platform/xchain-documentation/blob/master/protocol/actions/file.md) gated files and `BATCH(FILE, MESSAGE)` issuer-publish flows; ciphertext travels as `rawData` via P2WSH alongside the action string
- **JSON-RPC API**: Express server with Helmet security headers, optional API key auth, configurable rate limiting, CORS
- **Browser bundle**: Browserify build for client-side PSBT generation without a server
- **Single-instance guard**: refuses to boot when `ENCODER_REPLICAS` declares more than one replica, and takes an exclusive PID lockfile against a second local process; the UTXO reservation guard, the recent-build duplicate refusal and the rate limiter are in-process only until a shared store exists
- **Single-instance guard**: refuses to boot when `ENCODER_REPLICAS` declares more than one replica, and takes an exclusive PID lockfile against a second local process; the UTXO reservation guard, the recent-build duplicate refusal, the envelope-cancel owner set, the `release_inputs` reservation tickets, the rate limiter and the concurrency-gate counters are in-process only until a shared store exists
- **1330+ tests**: unit, integration, e2e, boundary, security, fuzz, chaos, mutation, regression, performance, smoke

## Documentation
Expand Down Expand Up @@ -88,11 +88,12 @@ npm run api
| `FEE_NO_ESTIMATE_RELAY_MULTIPLIER` | No | `10` | Multiple of the node's relay floor charged on a non-mainnet chain when `estimatesmartfee` has no data. Raise it where miners ignore the documented rate (`100` gives 0.1 DOGE/kB). Mainnet is unaffected |
| `DUST_AMOUNT` | No | Coin default | Floor in base units on every value output the encoder authors (funding legs, data outputs, change). Only raises the floor: the coin's consensus dust threshold and its relay-policy soft-dust floor (Dogecoin: 0.01 DOGE, below which each output adds the whole limit to the required relay fee) already apply |
| `XCHAIN_COMPRESSION_DEFAULT` | No | Enabled | Deployment default for transparent FILE compression; set `0`, `false`, or `off` to disable |
| `ENCODER_REPLICAS` | No | `1` | Deploy-manifest declared replica count; boot refuses above `1` until the in-process reservation, recent-build and rate-limit stores are shared |
| `ENCODER_REPLICAS` | No | `1` | Deploy-manifest declared replica count; boot refuses above `1` until the in-process reservation, recent-build, envelope-cancel owner, reservation-ticket, rate-limit and concurrency-gate state is shared |
| `ENCODER_INSTANCE_LOCK_FILE` | No | `<os tmpdir>/xchain-encoder-<ENCODER_API_PORT>.lock` (`default` in place of the port when `ENCODER_API_PORT` is unset) | Same-host PID lockfile taken exclusively at boot, so a second encoder process started on one host fails fast instead of racing UTXO selections. Point intentionally separate deployments (different coins or networks) on one host at different files. It cannot see replicas on other hosts or containers; `ENCODER_REPLICAS` is that declaration |
| `API_KEY` | No | Disabled | API key for `x-api-key` header authentication |
| `ENCODER_RATE_LIMIT_RPM` | No | `60` | Maximum requests per minute per IP |
| `ENCODER_MAX_RPC_BATCH` | No | `20` | Maximum JSON-RPC batch array length per request |
| `ENCODER_MAX_CONCURRENT_REQUESTS` | No | `50` | Global cap on requests served at once across all client IPs; excess gets an immediate 429 + `Retry-After` instead of queueing. `GET /status` and `GET /openrpc.json` are exempt; `0` disables |
| `ENCODER_MAX_CONCURRENT_REQUESTS` | No | `50` | Global cap on requests served at once across all client IPs; excess gets an immediate 429 + `Retry-After` instead of queueing. `/status` and `/openrpc.json` are exempt, over `GET` or `HEAD`, in any letter case and with or without a trailing slash; `0` disables |
| `ENCODER_MAX_CONCURRENT_PROBES` | No | `16` | Private concurrency reserve for the two exempt probe routes, so healthchecks stay answerable while the cap above sheds without becoming an uncapped bypass; `0` disables |
| `ENCODER_TRUST_PROXY` | No | `loopback, uniquelocal` | Express `trust proxy` setting; controls which hop the per-IP rate limiter keys the client IP on. `false`, a hop count, or an address/CIDR list per the Express docs |
| `ENCODER_MAINTENANCE_FILE` | No | `/tmp/xchain-encoder-maintenance.json` | Where the encoder looks for an operator-declared scheduled-maintenance window. `health` and `GET /status` report it as `maintenance` beside the readiness fields, so a status board can tell a planned outage from a fault; it never changes a readiness field or the 503. See [Scheduled maintenance](#scheduled-maintenance) |
Expand Down
58 changes: 49 additions & 9 deletions bin/ci-full.sh
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,35 @@ SELF="$(pwd)"
SIB="$(cd .. && pwd)"

FAILED=""
# >>> ci-tier (generated block; re-run the tier wirer to update) >>>
# Tier classes. A push grades the FAST tier only: the unit job, the pin and
# drift guards, and the structure and hygiene checks the hook runs before it
# dispatches. The tiers named below (coverage re-runs, perf scenarios) are
# skipped when the gate sets CI_TIER=fast, and each skip is recorded so the
# closing verdict can never claim a green it did not earn. Nothing stops
# being graded: a scheduled sweep re-runs this same script with CI_TIER=full
# on every repo every three hours and before any release or deploy, and a
# red there is tracked down and fixed first. CI_TIER is unset for a hand
# run, so a bare `npm run ci:full` still runs every tier as it always did.
CI_TIER_FULL_ONLY=(
"coverage ratchet (coverage:check)"
)
DEFERRED=""
ci_tier_deferred() {
[ "${CI_TIER:-full}" = "fast" ] || return 1
local t
for t in ${CI_TIER_FULL_ONLY[@]+"${CI_TIER_FULL_ONLY[@]}"}; do
if [ "$t" = "$1" ]; then
DEFERRED="$DEFERRED [$1]"
echo; echo "ci:full ===== $1 DEFERRED (CI_TIER=fast, runs in the full sweep) ====="
return 0
fi
done
return 1
}
# <<< ci-tier <<<
run_tier() {
ci_tier_deferred "$1" && return 0 # ci-tier guard (generated)
local name="$1"; shift
echo; echo "ci:full ===== $name ====="
if "$@"; then
Expand Down Expand Up @@ -93,26 +121,26 @@ run_tier "drift: coin consensus-pin conformance" node -e '
'

# --- identity pin (this gate only; no ci.yml job runs it) --------------
# bin/pins/identity.json holds the sha256 of the vendored coin files and the
# roundtrip conformance fixture. The tool compares only the entries the pin
# bin/pins/at1-identity.json holds the sha256 of the vendored coin files and
# every whole-file vendored twin. The tool compares only the entries the pin
# names, so an emptied pin would read as holding: the tier first refuses a pin
# with no coin or conformance entries, then fails on any moved or missing file.
# with no coin or vendored-twin entries, then fails on any moved or missing file.
identity_pin_check() {
node -e '
const pin = require("./bin/pins/identity.json");
for (const group of ["coins", "conformance"]) {
const pin = require("./bin/pins/at1-identity.json");
for (const group of ["coins", "vendoredTwins"]) {
if (!Object.keys(pin[group] || {}).length) throw new Error("identity pin names no " + group + " files");
}
' && node bin/pin-identity.js --compare bin/pins/identity.json
' && node bin/pin-identity.js --compare bin/pins/at1-identity.json
}
run_tier "identity pin (vendored coins, conformance fixture)" identity_pin_check
run_tier "identity pin (vendored coins, vendored twin files)" identity_pin_check

# --- suite-title pin (this gate only; no ci.yml job runs it) -----------
# Guards that every npm test script still collects the same test titles it
# did at the pin, through the declared rename and split maps.
run_tier "suite-title pin (at1)" node bin/suite-title-map.js \
--compare bin/pins/at1-suite-titles.json \
--rename-map bin/pins/test-rename-map.json \
--rename-map bin/pins/suite-title-renames.json \
--split-map bin/pins/suite-title-splits.json

# --- job: coverage (needs: ci) ------------------------------------------
Expand All @@ -122,8 +150,20 @@ run_tier "suite-title pin (at1)" node bin/suite-title-map.js \
run_tier "coverage ratchet (coverage:check)" npm run coverage:check

echo
# >>> ci-tier summary (generated) >>>
echo "ci:full: tier class ${CI_TIER:-full}"
if [ -n "${DEFERRED:-}" ]; then
echo "ci:full: DEFERRED to the full sweep:$DEFERRED"
fi
# <<< ci-tier summary <<<
if [ -n "$FAILED" ]; then
echo "ci:full: RED tiers:$FAILED"
exit 1
fi
echo "ci:full: all tiers green (same set GitHub CI runs)"
# >>> ci-tier verdict (generated) >>>
if [ "${CI_TIER:-full}" = "fast" ]; then
echo "ci:full: all FAST tiers green; the DEFERRED tiers above were NOT graded here"
else
echo "ci:full: all tiers green (same set GitHub CI runs)"
fi
# <<< ci-tier verdict <<<
Loading
Loading