Building Interpretune, a flexible framework for collaborative AI world model analysis and tuning. Author and maintainer of Fine-Tuning Scheduler. Contributor to PyTorch, Lightning, circuit-tracer, TransformerLens and SAELens.

Seattle, WA · interpretune.org · LinkedIn

Interpretune · docs · pre-MVP, APIs subject to change

Composes the interpretability ecosystem at multiple levels of abstraction: framework (e.g. PyTorch, Lightning), latent-model (e.g. TransformerLens, NNsight) and analysis (e.g. circuit-tracer, SAELens), beneath a shared session and protocol layer. An analysis operation (e.g. concept directions, gradient and ablation attribution, attribution-graph computation, feature interventions, graph pruning) is written once, dispatched across any backend composition, and captured as an AnalysisStore: a backend-agnostic, shareable dataset. The aim is to turn interpretability analyses from isolated notebook state into exchangeable artifacts other researchers can reproduce, combine and build on.

Fine-Tuning Scheduler · docs

A PyTorch Lightning extension that accelerates and enhances foundation model experimentation with flexible fine-tuning schedules. Supports explicit and implicit multi-phase schedules with per-phase early stopping and best-checkpoint restoration, versioned against PyTorch minor releases.

• Attribution Targets Encapsulation, the headline feature of v0.4.0. Generalizes attribution beyond top-n and top-p tokens to arbitrary token sets and arbitrary d_model-size vectors such as logit-difference directions, via AttributionTargets and LogitTarget containers with virtual-index support for out-of-vocabulary targets.

• transformers v5 support. Cross-version support covering batch_decode, RoPE config, BOS handling and MoE router shapes.
• Dual PyTorch/TransformerLens parameter access API. Makes parameters() and named_parameters() return optimizer-compatible leaf tensors while adding tl_parameters() and tl_named_parameters() for analysis tooling.
• HuggingFace ModelOutput support in the generation API. Adds an output_logits option returning per-step logits in HF ModelOutput form, plus an hf_generate() passthrough.
• Boolean 4D attention-mask handling in joint-QKV bridge attention. Converts 4D boolean HF masks to additive float masks so masked positions receive finfo.min rather than leaking into attention. Surfaced via SAELens cross-backend parity testing.
• Device and dtype synchronization in attention and the model bridge. Mixed-precision and multi-GPU synchronization fixes. Closed in favour of landing the work through the dependent PRs built on top of it.

• SAETransformerBridge hook name resolution and hook_z compatibility. Adds get_sae_hook_name() alias resolution, disables incompatible hook_z reshaping on the Bridge, and fixes version detection for source installs.

• Avoid elementwise dispatch of gradient unscaling and validation ops in _foreach_non_finite_check_and_unscale_cpu_. Removes per-element isinf/isnan dispatch in ShardedGradScaler CPU offload, taking the profiled case from roughly 66s to roughly 109ms.
• Make DTensor sharding propagation for scaled_dot_product_efficient_attention and scaled_dot_product_flash_attention more conservatively cached
• Enable DTensor sharding propagation of native_layer_norm_backward to more fully accommodate optional args
• Allow optimizer state conversion to accommodate optimizers that have no tensor state (e.g. SGD)
• Enhance no_grad-context FSDP backward handling
• Enhance Composable FSDP cast forward input tests
• Fix decision logic for should_cast_forward_inputs in _root_pre_forward() and _pre_forward()
• Fix multi-GPU ShardedGradScaler tests in ciflow/periodic
• Enable FSDP use_orig_params=True mixed precision training when some ranks have no parameter shards
• Basic validation for FSDP state_dict transformations of modules with persistent buffers
• Extend torch.cuda.is_available() to attempt an NVML-based CUDA availability assessment when explicitly requested
• Configure the PyTorch testing ArgumentParser instance to avoid unnecessary conflicts with system args
• Allow external scripts (e.g. vscode) to discover and execute unittest tests

• Properly handle parent modules with parameters in the BaseFinetuning callback
• Allow access to ckpt_path within the context of fit(). Enables callbacks to restore checkpoints iteratively inside Trainer.fit(), which is what makes multi-phase scheduled fine-tuning possible.
• Allow Horovod teardown() to complete gracefully if an exception is thrown in callback setup
• Support the ddp_fork strategy with native AMP via NVML-based CUDA availability assessment
• Set up a phased transition away from PyTorch version-specific handling of CUDA availability and device counting
• Enhance reduce_boolean_decision to accommodate the any-analogous semantics expected by the EarlyStopping callback
• Generalize Optimizer validation to accommodate both FSDP 1.x and 2.x
• Accommodate FSDP full-precision param_dtype training with PyTorch < 2.0

• Deep Classiflie · deepclassiflie.org · an NLP research framework for developing models that bolster fact-checking efficiency, with interactive prediction dashboards and integrated-gradients interpretability views.
• covid19 · dashboards · county-level "real-time" Rt analysis contributed to the New England Complex Systems Institute endcoronavirus.org initiative.

Reach me through LinkedIn or by opening an issue on Interpretune. Interpretune is pre-MVP and actively looking for contributors.