Weird Generalization and Solution Efficiency

This repository contains a reproducible study of a simple question about fine-tuned language models: when an intervention reduces weird generalization (WG), does it make a broad solution less efficient to fit than a deliberately narrow one?

The answer in this Qwen3-8B former-German-city model organism is no. Relevant context and an 80/20 instruction mixture reduced measured OOD weird behavior while the broad solution kept its matched-update-size task-loss advantage. The core efficiency result was reproduced across six training seeds.

Main result

The project separates two quantities that might otherwise be confused:

• Weird generalization: old-Germany or Nazi-like behavior on ten unrelated OOD questions, measured among coherent responses.
• Matched-norm efficiency: how much target-token task loss a broad or KL-narrow LoRA direction reduces when both are given the same effective update size.

The tested mechanism predicted that lower WG should come with a smaller broad advantage. Instead:

Intervention	WG result	Matched-norm result
C1, relevant context	15.63% to 7.57%	Broad advantage increased: (A_{C0}=0.03914), (A_{C1}=0.15876)
M80, 80% city task + 20% instructions	15.63% to 3.57%, with city-task NLL retained	Broad advantage on city-task rows stayed nearly unchanged: 0.03914 to 0.03906

For the full M80 mixture, the broad advantage was 0.13641; the instruction subset drives that higher value. This is why the city-task and full-mixture estimands are reported separately.

What the evidence supports

• Broad and KL-narrow solutions differ strongly in OOD behavior across behavioral seeds 0–2, while both pass the city task.
• Across six efficiency seeds and four objectives, all 24 measured area statistics were positive: broad was more efficient under this specific matched-effective-norm NLL metric.
• Fresh seeds 3–5 reproduced the original reversal.
• In exploratory seed-0 geometry, broad and narrow endpoints were connected by an almost barrier-free supervised-loss path.
• A seed-paired transplant of the consistently divergent middle-layer Q/K LoRA modules did not reliably transfer WG across the three tested seed pairs.

The study does not show that efficiency never matters for generalization. It tests one definition of efficiency in one model family, one task, and one narrow-solution construction.

Repository guide

configs/       Frozen model, training, and budget configurations
data/          Dataset manifests, evaluation questions, and provenance records
results/       Machine-readable analyses and frozen experiment artifacts
results/plots/ Publication-ready figures and equation assets
scripts/       Data preparation, analysis, plotting, and remote-worker programs
src/           Reusable Python package (`wg_efficiency`)
tests/         Contract and regression tests
workflows/     Historical Hugging Face and Modal entrypoints

Start with these files:

• results/analysis/efficiency_followups_622c678366cc.json — six-seed efficiency analysis.
• results/analysis/behavior_judgments_d79c1469229d.json — C0/C1/C2 behavior experiment.
• results/analysis/m80-gate-b-760c4a44663c.json — M80 behavior and task-retention result.
• results/analysis/loss_path_d29af403eeb7.json — exploratory broad-to-narrow loss path.
• results/analysis/weight_patching_3a3b7c807185.json — seed-paired Q/K transplant result.
• results/plots/ — the figures used to communicate the findings.

results/analysis/ is the source of truth for reported numerical results. The figures are derived from these frozen analyses and artifacts.

Reproduce local checks

Python 3.11 is required. The commands below install the development environment, validate local configuration, and run the test suite:

uv sync --extra dev
uv run wg-efficiency validate-env
uv run pytest

To rebuild the main figures from the frozen analyses:

uv run python scripts/make_application_figures.py
uv run python scripts/directional_probe_analysis.py
uv run python scripts/plot_lora_spectral_anatomy.py
uv run python scripts/plot_lora_cross_seed_convergence.py

Reproducibility and safety

The workflows under workflows/ record completed cloud experiments. Treat training, generation, judging, run, and retry actions as historical reproduction interfaces: they can incur compute cost and should only be used with a new scientific reason, budget check, and explicit authorization.