You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Good First Issue] Add ONNX Runtime inference path with PyTorch fallback #12
scripts/export_model.py exports PyTorch models to ONNX, but the API still loads .pth checkpoints directly. ONNX Runtime would provide faster inference and lower memory usage.
Scope
Update _load_model() in src/climatevision/inference/pipeline.py to prefer .onnx files when present
Fall back to PyTorch .pth if ONNX file is missing
Verify ONNX input/output tensor signatures match per analysis type:
It makes ONNX Runtime the preferred backend when an exported .onnx model is present (falling back to the .pth checkpoint, then untrained demo weights), adds per-request latency logging — the /api/predict response now carries backend and inference_ms — and a benchmark_inference() helper that times both backends side by side. scripts/export_model.py gains --all to export every enabled analysis type to models/unet_<type>.onnx (reading each type'''s num_classes from config.yaml), and validate_onnx now asserts the (N, n_classes, H, W) output signature. Covered by a new tests/test_onnx_inference.py.
Happy to adjust the load-priority default or the response fields if you'''d prefer them shaped differently.
Activity
[-][Help Wanted] Add ONNX Runtime inference path with PyTorch fallback[/-][+][Good First Issue] Add ONNX Runtime inference path with PyTorch fallback[/+]UTKARSH698 commentedon Jul 22, 2026
Opened #126 for this.
It makes ONNX Runtime the preferred backend when an exported
.onnxmodel is present (falling back to the.pthcheckpoint, then untrained demo weights), adds per-request latency logging — the/api/predictresponse now carriesbackendandinference_ms— and abenchmark_inference()helper that times both backends side by side.scripts/export_model.pygains--allto export every enabled analysis type tomodels/unet_<type>.onnx(reading each type'''snum_classesfromconfig.yaml), andvalidate_onnxnow asserts the(N, n_classes, H, W)output signature. Covered by a newtests/test_onnx_inference.py.Happy to adjust the load-priority default or the response fields if you'''d prefer them shaped differently.