Skip to content

feat(server): Harden production document import boundaries - #457

Merged
shenyankm merged 5 commits into
mainfrom
sheny/import-production-hardening
Oct 29, 2026
Merged

shenyankm merged 5 commits into
mainfrom
sheny/import-production-hardening

Conversation

@shenyankm

@shenyankm shenyankm commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

Summary

Imports now reserve optional estimated provider costs durably before dispatch and apply required OS restrictions to the existing native workers. Missing independent quality or operational evidence produces a blocked preflight instead of a production-readiness claim.

  • Reuse the PostgreSQL Store/admission transaction for task/grading and UTC-day cost reservations. Retain reservations across uncertainty, deletion and restarts; preserve the conservative unknown-call recovery from fix(import): preserve recovery across timeouts and page reloads #455.
  • Apply Landlock/seccomp on Linux and the macOS sandbox profile before parsing untrusted bytes. Keep converter HOME/XDG/IPC private, deny unrelated file access and network connections, and support one bounded native LibreOffice profile-initialization restart.
  • Keep cost-budget rejections out of completed durable model-attempt results. Explicit checkpoint resume rechecks task/daily limits and the UTC day while retaining rejected call records and completed Tencent jobs. Cost admission occurs before spending model-call slots, preserving correction allowance through repeated rejections.
  • Bind quality evidence to the deployed provider/model, hashed endpoint and recorded evaluation settings through the authenticated build metric. Keep the quality audit hash separate from the runtime service-code hash, and reject historical evidence missing these bindings.
  • Patch the existing langgraph-sdk transitive dependency to 0.4.6 for CVE-2026-118667, which blocked the locked-dependency audit on Python 3.14 and 3.15.
  • Select up to eight source-linked context pages from explicit references or unambiguous printed question numbers without crossing failed pages. Preserve review flags and immutable figure bounds.
  • Require frozen source/label hashes and separate human-review declarations for production holdouts. Add matched quality/load/restore/monitoring preflight, private restore evidence, cost/sandbox metrics and alert tests. Document that model confidence is not measured accuracy.

AI assistance: Codex implemented the changes, inspected the complete diff and exercised local checks with substitute providers. Native container checks use real PDFium and LibreOffice. No paid model evaluation or deployment was performed.

Linked issue

Closes #456

Affected areas

AI service, native preprocessing workers, deployment configuration, evaluation/operations tooling and English documentation. No practice App or import Web UI changes; service schema 5, App schema 11 and shared ZIP/API contracts remain unchanged.

Validation

Relevant checks follow CONTRIBUTING.md. Native container, Web and alert rows retain the original 731c3fe2 evidence; the review follow-up reruns the complete service checks and connected regression cases.

Check or manual flow Platform / environment Result and evidence
make verify AI_PYTHON=/Users/sheny/.local/share/uv/python/cpython-3.14-macos-aarch64-none/bin/python3.14 macOS ARM64, Python 3.14.7, disposable PostgreSQL/RabbitMQ, substitute providers Passed on 478d2348: 3,164 tests, 93% total coverage; lock consistency, Ruff, Pyright, 30 evaluation fixtures, recovery probes, source/wheel build and package integrity.
make install-locked; make audit AI_PYTHON=/Users/sheny/.local/share/uv/python/cpython-3.14-macos-aarch64-none/bin/python3.14 macOS ARM64, locked SDK 0.4.6, CI-pinned uv 0.12.13 Passed: runtime/development and build-requirement scans both report no known vulnerabilities. Only the SDK package record changed; generated contracts and 51 shared fixtures also passed.
Cost-admission rejection followed by explicit checkpoint resume macOS ARM64, actual parser graph, disposable PostgreSQL, substitute providers Passed: task-limit increase, daily-limit increase and next UTC day, each after five consecutive rejected attempts. No model-call slots spent on rejection; one slot and one provider call after recovery. All rejected receipts retained and no repeated Tencent requests.
Production configuration binding and connected configuration/auth/evaluation/load tests macOS ARM64, actual ASGI metrics and provider/HTTP substitutes Passed: 260 focused cases. Reject changed model/provider/endpoint/token limits, forged or missing quality identity, and missing/malformed/conflicting deployed fingerprints before load submissions; credential rotation preserves identity and no secrets/raw endpoints are exported.
Focused execution identity, cost admission, terminal rejection, stability and structured-output regressions Same local environment Passed: 123 tests against the final code, including atomic concurrent/day/restart cost boundaries.
make image-check Linux ARM64 Docker, non-root/read-only/capabilities dropped, Python 3.15, LibreOffice 7.4.7 Passed: denied unrelated reads/deletion, denied IP/external Unix connections, real PDF rendering/PPTX conversion, cleanup and mounted storage.
make web-check macOS, browser HTTP substitutes Passed: 225 unit tests and 56 Playwright tests.
make alerts-check Pinned Prometheus container Passed: alert rule tests and configuration syntax.
python app/scripts/export-contracts.py --check; git diff --check macOS, selected Python 3.14.7 Passed. No generated contract drift.
Native sandbox regressions macOS ARM64 child processes Passed: unrelated file reads/deletion and IP/external Unix connections denied; descendants inherit restrictions. Real macOS LibreOffice conversion not run because no engine is installed.
Paid real-document evaluation, production HTTPS/load, off-host disaster recovery and receiver delivery External target deployment Not run: independent reviewed documents and a target deployment were not supplied. The preflight explicitly retains these evidence/manual requirements.
GitHub CI PR head Pending for 478d2348; the prior-head audit failure was CVE-2026-118667 in SDK 0.4.4. Local audit and complete service verification pass with SDK 0.4.6.
App build/native practice acceptance macOS App N/A: App implementation and contracts are unchanged.

Risk and rollout

  • Data, contracts, and backup compatibility: No schema migration or contract changes. Cost reservations use a separate retained Store namespace included in complete database backups. Quality/capacity evidence without runtime-code and model-configuration bindings must be regenerated; historical reports remain readable. Restoring an older ledger requires reconciling provider activity since its snapshot before enabling paid work. The restore timer measures data restoration, not verified usable full RTO.
  • Configuration, dependencies, and security: Cost limits default off and require operator-supplied currency/prices when enabled; estimates are conservative admission accounting, not guaranteed provider invoice caps. Production Compose requires the worker sandbox. Linux requires Landlock ABI 3+ and seccomp on x86-64/aarch64. Configure native /usr/lib/libreoffice/program/soffice.bin with its exact version in the supplied image; the shell launcher is incompatible with private IPC. No new dependency, credential destination or client-side model work is introduced.
  • Main risk and rollout or recovery steps: Verify enforcement on the actual host/kernel before accepting untrusted files; unavailable enforcement fails closed. Retain the original deployment for existing checkpoint-compatible tasks, or explicitly parse again. Keep unknown paid-call receipts and explicit recovery. Scorer 6.7.0 requires a fresh comparable baseline. The automated preflight does not attest human annotation independence, image identity, notification delivery or successful off-host recovery; finish the documented real acceptance campaign before claiming production readiness. Loopback binding is preserved; external HTTPS must be verified on the selected deployment.

Checklist

  • I reviewed the full diff and kept it focused, including any required generated files and documentation.
  • I recorded relevant validation results and explained remaining gaps.
  • I checked code, fixtures, logs, and attachments for secrets and personal data.

Reserve estimated provider costs durably before dispatch, restrict native workers, and bound source-linked page context. Require matched independent quality and operational evidence for production preflight while preserving unknown call receipts and existing contracts.

Co-authored-by: Codex <codex@openai.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-29T03:27:20.668524Z 478d234 New commits
🔒 Security Review ⚠️ Failed 2026-10-29T03:26:13.692958Z
We couldn’t complete this review. Comment @codex security review to try again.
478d234 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 731c3fe26e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread server/src/practiq_ai/llm.py
Keep budget rejections out of completed durable task results so explicit
resume can recheck changed limits or a new UTC day without replaying Tencent.

Co-authored-by: Codex <codex@openai.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5a6e0f740e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread server/src/practiq_ai/llm.py
Comment thread server/scripts/production_acceptance.py Outdated
shenyankm and others added 3 commits October 29, 2026 10:32
Keep repeated recoverable cost rejections from consuming correction slots.
Retain rejection receipts and prior reservations across explicit recovery.

Co-authored-by: Codex <codex@openai.com>
Share a non-secret configuration fingerprint across evaluation and metrics,
record it from the deployed service before load submissions, and reject
mismatched quality evidence. Keep audit and runtime code hashes distinct.

Co-authored-by: Codex <codex@openai.com>
Resolve CVE-2026-118667 reported by the locked-dependency audit.
Regenerate the lock with the CI-pinned uv without changing other packages.

Co-authored-by: Codex <codex@openai.com>
@shenyankm
shenyankm merged commit abb7f91 into main Oct 29, 2026
13 checks passed
@shenyankm
shenyankm deleted the sheny/import-production-hardening branch October 29, 2026 03:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Harden production document import boundaries

1 participant