Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .github/workflows/inspection-log.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Inspection log
on:
push:
paths: ['applications/inspection-log/**', '.github/workflows/inspection-log.yml']
pull_request:
paths: ['applications/inspection-log/**', '.github/workflows/inspection-log.yml']
permissions:
contents: read
jobs:
ruby:
runs-on: ubuntu-24.04-arm
defaults:
run:
working-directory: applications/inspection-log
steps:
- uses: actions/checkout@v7.0.1
- name: Build verified Ruby 4.0.7
run: |
sudo apt-get update
sudo apt-get install -y build-essential autoconf bison pkg-config libssl-dev libyaml-dev libreadline-dev zlib1g-dev libffi-dev libgmp-dev libpq-dev libncurses-dev libgdbm-dev
curl -fsSLo "$RUNNER_TEMP/ruby.tar.gz" https://cache.ruby-lang.org/pub/ruby/4.0/ruby-4.0.7.tar.gz
echo "911ace20f90d068ca0e4dda6d0e4f0f81e52e52f2dd4f4004c721e253412e82d $RUNNER_TEMP/ruby.tar.gz" | sha256sum --check
tar -xzf "$RUNNER_TEMP/ruby.tar.gz" -C "$RUNNER_TEMP"
cd "$RUNNER_TEMP/ruby-4.0.7"
./configure --prefix="$RUNNER_TEMP/ruby" --disable-install-doc --disable-yjit --disable-zjit
make -j2
make install
echo "$RUNNER_TEMP/ruby/bin" >> "$GITHUB_PATH"
- name: Frozen native bundle and input/form/CSV checks without database credentials
env:
BUNDLE_PATH: ${{ runner.temp }}/inspection-bundle
BUNDLE_FROZEN: 'true'
run: |
gem install bundler --version 4.0.22 --no-document
bundle _4.0.22_ install
bin/check
8 changes: 8 additions & 0 deletions applications/inspection-log/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Runtime-only private file. Use KEY=value lines; launch.py does not run shell syntax.
PGHOST=YOUR_MANAGED_POSTGRES_HOST
PGPORT=5432
PGDATABASE=postgres
PGSSLROOTCERT=/absolute/path/to/managed-postgres-ca.pem
PGPASSWORD=YOUR_INSPECTION_APP_PASSWORD
SESSION_SECRET=YOUR_PRIVATE_128_CHARACTER_LOWERCASE_HEX_SECRET
TZ=Pacific/Honolulu
4 changes: 4 additions & 0 deletions applications/inspection-log/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
/.bundle/
/vendor/
/.env
/__pycache__/
1 change: 1 addition & 0 deletions applications/inspection-log/.ruby-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
4.0.7
17 changes: 17 additions & 0 deletions applications/inspection-log/Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
source 'https://rubygems.org'
ruby '4.0.7'

gem 'sinatra', '4.2.1'
gem 'sequel', '5.109.0'
gem 'pg', '1.7.0', force_ruby_platform: true
gem 'puma', '8.0.2'
gem 'rack', '3.2.7'
gem 'rack-session', '2.1.2'
gem 'rack-protection', '4.2.1'
gem 'erb', '6.0.7'
gem 'csv', '3.3.6'

group :test do
gem 'minitest', '6.0.6'
gem 'rack-test', '2.2.0'
end
88 changes: 88 additions & 0 deletions applications/inspection-log/Gemfile.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
GEM
remote: https://rubygems.org/
specs:
base64 (0.3.0)
bigdecimal (4.1.3)
csv (3.3.6)
drb (2.2.3)
erb (6.0.7)
logger (1.7.0)
minitest (6.0.6)
drb (~> 2.0)
prism (~> 1.5)
mustermann (3.1.1)
nio4r (2.7.5)
pg (1.7.0)
prism (1.9.0)
puma (8.0.2)
nio4r (~> 2.0)
rack (3.2.7)
rack-protection (4.2.1)
base64 (>= 0.1.0)
logger (>= 1.6.0)
rack (>= 3.0.0, < 4)
rack-session (2.1.2)
base64 (>= 0.1.0)
rack (>= 3.0.0)
rack-test (2.2.0)
rack (>= 1.3)
sequel (5.109.0)
bigdecimal
sinatra (4.2.1)
logger (>= 1.6.0)
mustermann (~> 3.0)
rack (>= 3.0.0, < 4)
rack-protection (= 4.2.1)
rack-session (>= 2.0.0, < 3)
tilt (~> 2.0)
tilt (2.9.0)

PLATFORMS
aarch64-linux
aarch64-linux-musl
arm64-darwin
ruby
x86_64-darwin
x86_64-linux
x86_64-linux-musl

DEPENDENCIES
csv (= 3.3.6)
erb (= 6.0.7)
minitest (= 6.0.6)
pg (= 1.7.0)
puma (= 8.0.2)
rack (= 3.2.7)
rack-protection (= 4.2.1)
rack-session (= 2.1.2)
rack-test (= 2.2.0)
sequel (= 5.109.0)
sinatra (= 4.2.1)

CHECKSUMS
base64 (0.3.0) sha256=27337aeabad6ffae05c265c450490628ef3ebd4b67be58257393227588f5a97b
bigdecimal (4.1.3) sha256=61ebe1e5e559bdc3cc6f2c0ee7f427321fc838f59611c294356eb04d6e21cf66
bundler (4.0.22) sha256=d8d5ec84c8555e0af71db63ed7aee4d1a8fb839ec46d84212d61979242a5d75a
csv (3.3.6) sha256=aba61e7e507a66f03d45cb1f3c4b6359861c3504038b422962875dce099e4456
drb (2.2.3) sha256=0b00d6fdb50995fe4a45dea13663493c841112e4068656854646f418fda13373
erb (6.0.7) sha256=c5ca6dc25b0ef974a44dc8f59fe847577122483b1968a38dec305c60bf91ee92
logger (1.7.0) sha256=196edec7cc44b66cfb40f9755ce11b392f21f7967696af15d274dde7edff0203
minitest (6.0.6) sha256=153ea36d1d987a62942382b61075745042a2b3123b1cd48f4c3675af9cc7d6f1
mustermann (3.1.1) sha256=4c6170c7234d5499c345562ba7c7dfe73e1754286dcc1abb053064d66a127198
nio4r (2.7.5) sha256=6c90168e48fb5f8e768419c93abb94ba2b892a1d0602cb06eef16d8b7df1dca1
pg (1.7.0) sha256=f7b536501284e883d4c9ef86dbde0313063e4aaafc23575b0017b0d572a26afb
prism (1.9.0) sha256=7b530c6a9f92c24300014919c9dcbc055bf4cdf51ec30aed099b06cd6674ef85
puma (8.0.2) sha256=c8ed871dfbbe66448ea9ffd46692342d9804d4071522b52b5331b7b6e7b686fb
rack (3.2.7) sha256=93e13e1c24f93556671d85d2d79fa228c3485815c50d7e2f265b5330c6528fb7
rack-protection (4.2.1) sha256=cf6e2842df8c55f5e4d1a4be015e603e19e9bc3a7178bae58949ccbb58558bac
rack-session (2.1.2) sha256=595434f8c0c3473ae7d7ac56ecda6cc6dfd9d37c0b2b5255330aa1576967ffe8
rack-test (2.2.0) sha256=005a36692c306ac0b4a9350355ee080fd09ddef1148a5f8b2ac636c720f5c463
sequel (5.109.0) sha256=e0d1d474442d620c8497d7f4851c0cc3a277920519051b8014b2877aa55efffe
sinatra (4.2.1) sha256=b7aeb9b11d046b552972ade834f1f9be98b185fa8444480688e3627625377080
tilt (2.9.0) sha256=da5735d0280bba96e9a91041bb14aee435ccad5c17b0fa519249ae543d9aa3a5

RUBY VERSION
ruby 4.0.7

BUNDLED WITH
4.0.22
188 changes: 188 additions & 0 deletions applications/inspection-log/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
# Inspection log

A small Sinatra workbench for recording complete inspections of three synthetic assets in [ClickHouse Managed Postgres](https://clickhouse.com/docs/products/managed-postgres/overview). Choose an asset, calendar day, outcome, three checklist results and a short note. Save once, view immutable records, filter asset history, and download a bounded CSV.

This is a trusted **loopback-only** sample. There is no operator identity, authentication or tenant isolation. Cookie sessions contain CSRF state. The shared database role covers every fixture asset; it is trusted server configuration. The app does not integrate with live equipment.

## Transaction and retry behavior

[`Store#save`](lib/store.rb) uses one explicit Sequel transaction. It locks the single seeded fixture-budget row, looks up the retained request UUID, then increments the budget and inserts the inspection header plus its three named checklist results. An exact canonical retry returns the original inspection. Reusing the UUID with different content returns 409 and preserves the submitted fields. Replay lookup precedes the 200-record capacity check, so a saved form can still replay at capacity.

The canonical digest includes asset, Date, outcome, ordered housing/cable/label results and note. UUID spelling is normalized to lowercase. CRLF and CR normalize to LF in the stored note and digest; other note whitespace is retained. CSRF tokens are excluded from the digest. A lost acknowledgment may be retried with the same UUID and fields. Keep that UUID; changing it can create a second record. Validation 422, conflict 409, capacity 409 and database 503 pages retain safe submitted fields. Invalid UTF-8/control text gets 400 without echoing it.

The budget lock intentionally serializes saves across all assets. It avoids a racing pre-count and is suitable for this small fixture. Header IDs are bounded 1–200 by a database check, request UUIDs are unique, and budget usage cannot exceed 200. A transaction failure restores the budget, header and children together. There is no edit/delete route or expiry of retry keys.

Calendar days remain Ruby `Date` / PostgreSQL `DATE` and serialize with `iso8601`; they never pass through a midnight timestamp. Supported days are 2000-01-01 through 2100-12-31. History shows at most 25 rows ordered by day descending, then numeric ID descending. CSV includes at most 100 rows ordered by day ascending, then numeric ID ascending, across at most 31 inclusive calendar days. An oversized CSV gets 422 asking for narrower filters, rather than a truncated download. Rows are loaded before CSV generation so a slow download holds no pool checkout.

Ruby CSV handles commas, quotes and newlines. Text in asset/note cells that starts with a formula-looking prefix (including leading whitespace) or a tab/newline receives a leading apostrophe. This **changes exported text** and reduces a defined formula risk; it is not a guarantee across every spreadsheet/import workflow. Review exports for the intended importer. See [OWASP's CSV injection discussion](https://community.owasp.org/attacks/CSV_Injection).

## Native Linux setup

Verified on Ubuntu 24.04 ARM 64 on 2 October 2026: Ruby 4.0.7, Bundler 4.0.22, Sinatra 4.2.1, Sequel 5.109.0, pg 1.7.0 / native libpq 16.15, Puma 8.0.2, Rack 3.2.7, rack-session 2.1.2, rack-protection 4.2.1, ERB 6.0.7 and CSV 3.3.6. The owned Cloud fixture ran PostgreSQL 18.6. `Gemfile.lock` pins transitive releases and gem checksums; only Linux ARM 64 was tested. The pg gem is deliberately compiled from source against libpq.

Build in a native Linux directory, not a shared host mount:

```bash
sudo apt-get update
sudo apt-get install -y build-essential autoconf bison pkg-config libssl-dev \
libyaml-dev libreadline-dev zlib1g-dev libffi-dev libgmp-dev libpq-dev \
libncurses-dev libgdbm-dev curl ca-certificates postgresql-client python3 jq
mkdir -p "$HOME/toolchains"
cd "$HOME/toolchains"
curl -fsSLo ruby-4.0.7.tar.gz https://cache.ruby-lang.org/pub/ruby/4.0/ruby-4.0.7.tar.gz
echo '911ace20f90d068ca0e4dda6d0e4f0f81e52e52f2dd4f4004c721e253412e82d ruby-4.0.7.tar.gz' | sha256sum --check
tar -xzf ruby-4.0.7.tar.gz
cd ruby-4.0.7
./configure --prefix="$HOME/.local/ruby-4.0.7" --disable-install-doc --disable-yjit --disable-zjit
make -j2
make install
export PATH="$HOME/.local/ruby-4.0.7/bin:$PATH"
gem install bundler --version 4.0.22 --no-document
cd /absolute/native/path/to/examples/applications/inspection-log
bundle _4.0.22_ config set --local path "$HOME/.local/inspection-bundle"
bundle _4.0.22_ config set --local frozen true
bundle _4.0.22_ install
bin/check
```

The archive checksum comes from the [official Ruby downloads page](https://www.ruby-lang.org/en/downloads/). `bin/check` needs no database credentials: it checks Ruby syntax, compiles ERB in a rendering method context, and runs 5 tests / 50 assertions for validation, raw form boundaries and parsed CSV. CI performs the same native build/frozen install on Ubuntu ARM 64.

## Create the owned Cloud fixture

Install and authenticate [clickhousectl](https://clickhouse.com/blog/getting-started-clickhousectl). These commands use its 0.5.0 syntax and an explicit organization. Save create output privately: it returns the administrator password once; subsequent `get` does not.

```bash
mkdir -p "$HOME/inspection-private"
chmod 700 "$HOME/inspection-private"
export INSPECTION_ORG_ID='YOUR_ORG_ID'
clickhousectl cloud postgres create --json --org-id "$INSPECTION_ORG_ID" \
--name inspection-log-example --provider aws --region us-east-1 \
--size c6gd.large --pg-version 18 --ha-type none \
> "$HOME/inspection-private/create.json"
chmod 600 "$HOME/inspection-private/create.json"
export INSPECTION_PG_ID=$(jq -r '.id' "$HOME/inspection-private/create.json")
clickhousectl cloud postgres get "$INSPECTION_PG_ID" --org-id "$INSPECTION_ORG_ID"
```

Repeat `get` until `state` is `running`, then retrieve the Cloud CA:

```bash
clickhousectl cloud postgres certs get "$INSPECTION_PG_ID" \
--org-id "$INSPECTION_ORG_ID" --output "$HOME/inspection-private/ca.pem"
```

This is a billable no-HA fixture. Delete it after the exercise; stopping the local app does not delete its database.

## Bootstrap, migrate and seed

In a setup terminal, export administrator connection fields from the private receipt and generate independent role/session secrets. Do not print those values:

```bash
export PGHOST=$(jq -r '.hostname' "$HOME/inspection-private/create.json")
export PGPORT=5432 PGDATABASE=postgres
export PGUSER=$(jq -r '.username' "$HOME/inspection-private/create.json")
export PGPASSWORD=$(jq -r '.password' "$HOME/inspection-private/create.json")
export PGSSLMODE=verify-full PGCONNECT_TIMEOUT=5
export PGSSLROOTCERT="$HOME/inspection-private/ca.pem"
python3 - <<'PY'
from pathlib import Path
import os, secrets
path = Path.home() / 'inspection-private/passwords.env'
path.write_text('MIGRATOR_PASSWORD=Aa1' + secrets.token_hex(24) + '\n'
'APP_PASSWORD=Aa1' + secrets.token_hex(24) + '\n'
'SESSION_SECRET=' + secrets.token_hex(64) + '\n')
path.chmod(0o600)
PY
set -a
source "$HOME/inspection-private/passwords.env"
set +a
psql -X -v ON_ERROR_STOP=1 -v migrator_password="$MIGRATOR_PASSWORD" \
-v app_password="$APP_PASSWORD" -f sql/bootstrap.sql
bin/migrate
psql -X -v ON_ERROR_STOP=1 -f sql/seed.sql
```

`bin/migrate` uses the separate `inspection_migrator` login and assumes the non-login `inspection_owner` role on its single connection. It uses the same verified TLS configuration as runtime. Repeat migration stays at version 1; repeat seed creates no extra assets/budget. `bin/migrate 0` drops this app's tables and data; use it only on a disposable fixture before migrating and seeding again.

The runtime has select on assets/budget/header/children, insert on header/children, and update only on the budget's `used` column. It cannot alter/delete history, edit assets, create schema objects/TEMP tables, or assume the owner role. The three-child completeness invariant belongs to the application transaction; a trusted owner or leaked runtime credential can bypass application validation. There is no row-level tenant boundary.

Write a runtime-only file from the setup terminal:

```bash
python3 - <<'PY'
from pathlib import Path
import os
values = {key: os.environ[key] for key in ('PGHOST', 'PGPORT', 'PGDATABASE', 'PGSSLROOTCERT')}
values.update(PGPASSWORD=os.environ['APP_PASSWORD'], SESSION_SECRET=os.environ['SESSION_SECRET'], TZ='Pacific/Honolulu')
path = Path.home() / 'inspection-private/app.env'
path.write_text(''.join(key + '=' + value + '\n' for key, value in values.items()))
path.chmod(0o600)
PY
```

## Run the protected form

Open a new terminal that has not sourced setup credentials. Set the Ruby path, change to the app directory and start with the runtime file:

```bash
export PATH="$HOME/.local/ruby-4.0.7/bin:$PATH"
cd /absolute/native/path/to/examples/applications/inspection-log
python3 checks/launch.py "$HOME/inspection-private/app.env"
```

The launcher passes only runtime fields and basic process settings to Puma; inherited administrator/migrator variables are excluded. Visit `http://127.0.0.1:9292/`, choose an asset/day/checklist, save, then follow **View asset history** and **Download CSV**.

Puma binds 127.0.0.1 with 4 threads and no worker processes. Sequel permits 4 connections, a 3-second pool-acquisition wait and 5-second connection timeout. Each connection sets 8-second statement,3-second lock and 10-second idle-transaction timeouts. These are layer limits, not an end-to-end HTTP deadline. Sequel's [pg_auto_parameterize extension](https://sequel.jeremyevans.net/rdoc-plugins/files/lib/sequel/extensions/pg_auto_parameterize_rb.html) binds values in runtime dataset SQL. Fixed setup SQL contains no untrusted fragments.

Sequel's pg adapter uses `sslmode: verify-full` and the downloaded `sslrootcert`; there is no insecure fallback. [Sequel documents adapter/pool configuration](https://sequel.jeremyevans.net/rdoc/files/doc/opening_databases_rdoc.html) and [transaction behavior](https://sequel.jeremyevans.net/rdoc/files/doc/transactions_rdoc.html).

The app keeps Sinatra protection enabled and uses Rack's masked authenticity tokens with a private cookie-session secret. Exact allowed Host values are localhost/127.0.0.1; POST requires the matching HTTP loopback Origin. Cookies are HttpOnly/SameSite Strict and deliberately not Secure on local HTTP. This configuration is not a public deployment recipe. URL-encoded saves are capped at 32 KiB before parsing; duplicate decoded field names return 400. Empty `&` separators are ignored. Unknown fields/enums, dates, checklist results and note bounds are validated before SQL. ERB escapes displayed asset/note/field text.

## Reproduce Cloud acceptance

The `checks/` helpers use a disposable Cloud fixture and create synthetic data. They do not need CI credentials. Install browser tooling separately from the app bundle in the VM:

```bash
python3 -m venv "$HOME/inspection-private/browser"
"$HOME/inspection-private/browser/bin/pip" install playwright==1.63.0
"$HOME/inspection-private/browser/bin/python" -m playwright install --with-deps chromium
export EVIDENCE_DIR="$HOME/inspection-private"
"$HOME/inspection-private/browser/bin/python" checks/browser.py
```

Run the browser helper exactly once on the empty seeded fixture: it creates record 1, checks escaping/non-UTC DATE and parses CSV. Then, in a separate **test setup** terminal, explicitly load administrator PG* fields as above and export `APP_PASSWORD` from `passwords.env`. With the app still running:

```bash
set -a
source "$HOME/inspection-private/passwords.env"
set +a
python3 checks/cloud.py
```

The helper checks native middleware, two independent session retries, conflicting fields, an owner-installed second-child fault with cleanup in `finally`, runtime grant/check failures and last-slot contention. It deliberately fills the fixture to 200 records. Its final CSV/history assertions can also run read-only as `python3 checks/cloud.py --reports-only`; this does not reset or rerun mutations.

For the genuine process restart, record the **actual Puma PID** in a private file (a background `python3 checks/launch.py ...` retains its PID through exec). Keep the admin/test variables in the test terminal to verify the launcher excludes them:

```bash
export RUNTIME_ENV="$HOME/inspection-private/app.env"
export SERVER_PID_FILE="$HOME/inspection-private/server.pid"
export EVIDENCE_DIR="$HOME/inspection-private"
python3 checks/security_restart.py
"$HOME/inspection-private/browser/bin/python" checks/mobile.py
```

`security_restart.py` checks the actual `Database.connect` path against the correct endpoint, an untrusted CA and an IP/name mismatch. It signals only the recorded Puma PID, asserts that process has disappeared before starting the replacement, then compares persisted asset/note/day/time/request fields and exact CSV. It also replays record 1's uppercase UUID at capacity. The helpers expect the stated browser fixture; keep failed attempts distinct from passing results.

Observed on 2 October 2026: fresh frozen bundle;5 tests / 50 assertions; migration 1/repeat 1/down 0/up 1 and repeated seed; actual browser desktop/mobile; protected HTTP boundaries; concurrent retained requests with one debit; post-header/first-child rollback; cap 200 with 600 children; CSV overflow 422 / exact 100 rows; history 25 numeric ordering; actual runtime 42501/23514/23505; CA/name negatives through Sequel; real Puma 18797→19032 with original gone, exact persisted content/CSV and retained replay. These are bounded correctness checks, not a load benchmark.

## Cleanup

Stop the recorded Puma process, then delete only the service you created:

```bash
kill "$(cat "$HOME/inspection-private/server.pid")"
clickhousectl cloud postgres delete "$INSPECTION_PG_ID" --org-id "$INSPECTION_ORG_ID"
clickhousectl cloud postgres list --org-id "$INSPECTION_ORG_ID"
```

Confirm its ID is absent, allowing for asynchronous deletion. Postgres deletion accepts a running service and has no `--force` flag. It removes the fixture permanently. Remove private credential/receipt files when no longer needed. The review run's owned service was deleted after acceptance.
Loading