Skip to content

feat: monitor RAM/VRAM and require consent for disk offload - #146

Open
sunxfancy wants to merge 3 commits into
devfrom
codex/resource-monitor-offload
Open

sunxfancy wants to merge 3 commits into
devfrom
codex/resource-monitor-offload

Conversation

@sunxfancy

Copy link
Copy Markdown
Owner

Behavior

Show server RAM and per-CUDA-device VRAM usage in Functional UI, and select model placement with reserved runtime headroom. Low VRAM enables CPU offload; host-memory pressure creates a one-shot disk-offload confirmation. Rejection, expiry, unavailable confirmation service, or insufficient disk space stops loading.

Integrates image ModelCache, TRELLIS, DiffSynth video loaders (Wan/CogVideo/Hunyuan/AnimateDiff), and the LTX/H3 runners. Preserves current dev's LoRA, native INT8, MPS, and pipeline behavior by staging checkpoints on CPU and finalizing residency after weight transforms. Includes deterministic API schema/types and resource-policy documentation.

Validation

  • 70 focused Python tests passed: resource monitor (11), runtime offload (15), model memory (15), video memory (4), LoRA adapters (11), workflow quantization (14).
  • yarn api:check, yarn build:components, and yarn build:functional_ui passed.
  • Small real CPU tensors verify output preservation, disk mapping, exception recovery, and native INT8 compatibility; adapter contracts exercise policy routing.

Limits

Real pretrained video/3D GPU inference and output quality are not validated by these checks. Initial deserialization, conversion, activations, and LoRA updates can still exceed memory estimates. FP8 retains native casting/offload; automatic FP8 disk offload fails explicitly instead of changing precision. Voice/LLM and additional independent runtimes are outside this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant