pwa(storage): engineer a durable IDB+OPFS storage tier with persistence, quota & recovery UX

[qnbs]

Parent

Part of #478 — PWA first-class excellence/hardening.

Why this can exceed native behavior

Browser storage is often treated as a limitation. Properly engineered, the combination of IndexedDB + OPFS + StorageManager can instead give WorldScript a highly portable, sandboxed, zero-install storage subsystem with browser-managed origin isolation and efficient worker-local file access.

The objective is not “move everything to OPFS.” The objective is to build a deliberate storage hierarchy based on data shape, durability, regeneration cost, security and performance.

Proposed storage tiers

Tier 1 — IndexedDB: structured authoritative state

Best suited to:

• project/domain records;
• settings;
• metadata/index manifests;
• transactional structured records;
• encryption metadata where existing architecture requires it.

Keep schema/version/migration discipline explicit.

Tier 2 — OPFS: large/binary/random-access origin-private artifacts

Evaluate for:

• local-AI model shards/artifacts;
• DuckDB/SQLite-like WASM databases where supported architecture benefits;
• large RAG/index artifacts;
• generated thumbnails/previews;
• large binary project assets;
• regenerable caches whose IndexedDB representation is inefficient.

OPFS should not automatically become authority for ordinary project records.

Tier 3 — Cache Storage: network-derived/re-fetchable resources

Use only for resources whose caching policy is safe and explicit. Do not use Cache Storage as a generic database.

Tier 4 — user-controlled export/backup

External escape hatch through download and, where supported, File System Access (#481).

Persistent storage

Use navigator.storage.persisted() / navigator.storage.persist() where supported and appropriate.

UX must be truthful:

• show whether durable/persistent treatment is currently granted;
• explain that browser policy ultimately controls eviction;
• do not repeatedly nag if persistence is denied;
• provide a backup/export recommendation when risk is elevated.

Storage health dashboard

Add a privacy-safe storage panel showing approximate categories, not manuscript content:

Projects / structured data
Images/assets
Snapshots/backups
RAG/indexes
Local AI models
Regenerable caches
Service-worker assets
Estimated total / quota where browser exposes it
Persistence granted: yes/no/unknown

Actions:

• export backup;
• remove selected downloaded local-AI models;
• clear regenerable caches/indexes;
• rebuild indexes;
• inspect storage-health warnings;
• factory reset only as an explicit destructive last resort.

Never offer “clear all site data” as the normal first troubleshooting step.

Quota-pressure strategy

Define deterministic behavior at thresholds rather than waiting for random QuotaExceededError.

Suggested policy:

1. estimate storage/quota where available;
2. warn before risky large model/index download;
3. reserve/estimate required space where practical;
4. evict regenerable data first;
5. never silently evict authoritative project data;
6. surface backup/export before destructive cleanup;
7. recover gracefully when the browser estimate is unavailable or inaccurate.

Atomicity and recovery

For large OPFS writes:

• write to staging/versioned target;
• validate integrity/checksum/schema where appropriate;
• atomically promote where available/appropriate or use manifest indirection;
• retain previous valid generation until new generation is committed;
• clean orphan staging files deterministically;
• interruption/reload must produce old-valid or new-valid state, not half-authoritative data.

For structured IDB data, preserve transaction boundaries and migration journals already present in the repository.

Worker performance

Evaluate OPFS synchronous access handles inside workers only where they materially improve random I/O for WASM databases/model artifacts.

Never block the UI thread for large storage operations.

Measure:

• project open/save;
• model cold/warm load;
• RAG/index open/query/update;
• large asset import;
• migration/recovery;
• memory overhead vs IndexedDB blob paths.

Adopt OPFS only where measured benefit justifies complexity.

Encryption/security boundary

Do not assume OPFS is encrypted merely because it is origin-private.

For protected data:

• apply the product's explicit at-rest encryption contract where required;
• keep credential encryption distinct from project encryption;
• avoid plaintext staging leaks for protected authoritative content;
• ensure lock/migration semantics cover data moved between IDB and OPFS;
• document which classes are protected vs regenerable/non-sensitive.

Coordinate with #445 for renderer-neutral security truth where semantics overlap, while keeping browser implementation details PWA-specific.

Migration strategy

Any IDB→OPFS movement must be:

• versioned;
• resumable or safely restartable;
• idempotent;
• non-destructive until target validation succeeds;
• tested from representative historical versions;
• reversible where practical or explicitly one-way with export safety.

Do not migrate existing healthy data merely to increase use of a newer API.

Cross-browser behavior

Feature-detect OPFS and persistence APIs. Unsupported browsers continue using the current proven storage adapter/fallback.

No core writing feature may fail solely because OPFS is absent unless the support matrix explicitly classifies that feature as capability-limited.

Tests

• persistence granted/denied/unknown;
• quota low/full;
• large model download interrupted;
• OPFS write interrupted;
• migration reload/crash;
• orphan staging recovery;
• cache eviction vs authoritative data;
• encryption locked/unlocked;
• storage estimate unavailable;
• unsupported OPFS fallback;
• multiple tabs contending (pwa(concurrency): make multi-tab editing safe with Web Locks, BroadcastChannel & lifecycle ownership #480);
• service-worker update/version skew.

Acceptance criteria

• Authoritative vs regenerable storage classes are inventoried.
• IDB/OPFS/Cache Storage responsibilities are explicit.
• OPFS adoption is measurement-driven.
• Persistent-storage request/status UX is truthful.
• Quota pressure cannot silently delete authoritative project data.
• Large writes/migrations are interruption-safe.
• Storage health/cleanup UI distinguishes safe cache cleanup from destructive data removal.
• Protected-data encryption boundaries remain truthful.
• Worker-based large I/O does not block editor interaction.
• Cross-browser fallback is tested.

Non-goals

• replacing IndexedDB wholesale;
• claiming browser persistence is equivalent to an OS backup;
• storing sensitive plaintext because OPFS is origin-private;
• using Cache Storage for authoritative project records;
• making OPFS a requirement for basic PWA writing.

[qnbs]

Architecture refinement / OPFS seed reconciliation — 2026-08-24

OPFS is not greenfield in the current repository. workers/v2/duckdb.worker.ts already probes navigator.storage.getDirectory(), opens an OPFS-backed DuckDB file handle, and falls back when OPFS setup fails. Treat that path as the first production seed and audit/reuse its lessons before creating new OPFS infrastructure.

Preserve current high-level storage authority

services/storageService.ts and services/storageBackend.ts already define the current browser/native storage abstraction. Do not add a whole-application OPFS backend that competes with IndexedDB for ordinary project truth merely because OPFS exists.

Preferred shape:

storageService
  │
  ├─ authoritative structured PWA state
  │    └─ existing IndexedDB path
  │
  └─ specialized large-artifact repositories
       ├─ DuckDB/analytics OPFS
       ├─ local-AI artifacts when measured/justified
       ├─ RAG/index artifacts when measured/justified
       └─ fallback implementation

Adopt OPFS per data class after measurement.

Security/encryption

Origin-private is not equivalent to product encryption. Do not invent an unrelated OPFS crypto format when protected-data semantics overlap current secure-record/encryption invariants. Reuse common envelope/record-identity/migration concepts where applicable while keeping browser implementation details PWA-specific.

Migration

No healthy IDB data should be moved to OPFS merely to increase use of a newer API. Every IDB→OPFS transition requires a real performance/durability benefit, an explicit authority classification, interruption-safe staging/commit, and fallback semantics.

Additional implementation checks

• Audit whether the current DuckDB OPFS support probe leaves a __duckdb_opfs_test__ artifact and clean it deterministically if appropriate.
• Keep synchronous OPFS access handles worker-only when used.
• Reuse pwa(concurrency): make multi-tab editing safe with Web Locks, BroadcastChannel & lifecycle ownership #480 ownership/admission primitives for conflicting artifact writes rather than creating storage-private ad-hoc locks without inventory.
• Feed capability/persistence/quota state through pwa(capabilities): establish progressive-enhancement registry, browser support tiers & runtime diagnostics #489 rather than duplicating generic capability detection.

Acceptance additions

• Current DuckDB OPFS implementation is documented as an existing seed.
• OPFS is not introduced as a second live-project authority.
• Each OPFS-admitted data class has measured rationale and fallback.
• Protected staging cannot silently downgrade to plaintext.
• Artifact cleanup/orphan recovery is deterministic.
• Existing IDB transaction/migration authorities remain intact for structured project state.