Repository navigation
Sg/backend improvements - #32
Open
saurabhinfocusp wants to merge 15 commits into
Open
saurabhinfocusp wants to merge 15 commits into
saurabhinfocusp wants to merge 15 commits into
Conversation
CI's yapf check failed because the new P0 files did not match the project's .style.yapf (google style, 80 cols). Reformatted with `yapf -ir --style .style.yapf` and `isort --settings-path .isort.cfg`; no logic changes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CI's yapf check was failing because add_argument calls exceeded the 80-col limit. Reformatted with `yapf -ir --style .style.yapf`; no logic changes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Explainability: scalr/explain.py implements Grad x Input saliency attribution, wired into AnnotationModel.explain()/explain_class() and a new `scalr explain` CLI subcommand, to report which genes supported or contradicted a prediction (per cell or per class). Streaming predict: AnnotationModel.predict() gains streaming/chunk_size params (implied by passing a file path); gene alignment and scoring now happen chunk-by-chunk instead of materializing one aligned matrix for the whole dataset, per the low-resource design goal. Forwarded through scalr.annotate() and the `scalr annotate` CLI. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add Python-API and CLI usage examples for model.explain()/explain_class() and streaming/chunked model.predict(), added in a6d45c0. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- scalr.models.download()/push() pull/publish model artifacts to the Hugging Face Hub (optional `hub` extra); scalr.load_model auto-downloads a "<namespace>/<name>"-shaped name not found locally. Adds `scalr models push|pull` CLI subcommands. - scalr.detect_drift(adata, model) compares a query dataset's per-gene and library-size/detected-gene-count distributions against training-time reference stats recorded by scalr.train, to catch whole-cohort drift that per-cell is_unknown/is_possible_doublet flags can miss. Adds `scalr drift` CLI subcommand. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- build_loss_fn resolves a CustomLossBase subclass registered under scalr.nn.loss by name (same name/params pattern as models/preprocessors/ etc.), falling back to torch.nn built-ins. - FeatureExtractionPipeline.feature_scoring() now scores chunked models in parallel (joblib, threading backend) using the existing num_workers setting, instead of serially. - FeatureExtractionPipeline.set_data_and_targets() was silently dropping its sample_chunksize argument, breaking feature_subsetted_model_training() whenever data was set this way instead of via load_data_and_targets_from_config(). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- align_genes kept the LAST occurrence of a duplicated gene name (a dict comprehension), contradicting scalr.validation's documented "only the first occurrence will be used" contract; query cells could silently score against the wrong column's expression values. - FeatureSubsetting.train_chunked_models() resolved model/train configs by mutating `self` inside a joblib.Parallel(n_jobs>1) worker closure, which never propagates back to the parent process under the process-based backend; the pipeline's written-back config ended up stale/unresolved whenever num_workers > 1. Now sourced from each worker's return value. - Fixed scalr.hierarchy.aggregate_to_broad's docstring: is_unknown is carried over unchanged from the fine-level result, not recomputed, per its own inline comment and existing test. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
scalr.analysis (DgeLMEM et al.) and scalr.feature.scoring.ShapScorer pulled in scanpy/statsmodels/shap eagerly on `import scalr`, adding ~1.5s and noisy scanpy-internal deprecation FutureWarnings to every `scalr` CLI invocation, including commands unrelated to analysis/SHAP (e.g. `scalr models list`). Deferred via PEP 562 package __getattr__; getattr(module, name) (the name/params builder pattern used by build_analyser/build_scorer) triggers the lazy import transparently, same as an eager import would. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…trix bug anndata==0.11.x (the whole previously-allowed >=0.11.2 floor) raises AttributeError: 'backed_csr_matrix' object has no attribute '_validate_indices' when constructing an AnnCollection over a chunked .h5ad directory (used whenever num_workers > 1), breaking that path end-to-end. Confirmed fixed in anndata>=0.12; bumped the floor accordingly (still within the existing <0.13 ceiling). That surfaced a second, pre-existing bug: DgePseudoBulk._make_design_matrix built each pseudobulk sample's AnnData.X from a scipy sparse .sum(axis=0), which returns np.matrix, not ndarray; anndata 0.11.x tolerated this silently, but anndata>=0.12 no longer does, which broke pydeseq2's internal genewise-dispersion masking. Fixed by converting to a plain ndarray. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
AnnCollection's default per-chunk dtype probing (harmonize_dtypes=True) crashes on backed sparse chunks under some anndata versions/backends (AttributeError: 'backed_csr_matrix' object has no attribute '_validate_indices'). Chunks read here are always written by scalr's own write_chunkwise_data from a single source, so their dtypes are already consistent by construction — the probing is redundant work scaLR doesn't need, on top of being unreliable across anndata versions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
*Other updates