LoRA fine-tuning hyperparameter sensitivity analysis and best-practice guide. Based on LLaMA-Factory, covering 45 parameter combinations across rank, alpha, and target_modules on Chinese and English datasets.
- rank=16 achieves 97.8% of rank=64 performance with 75% fewer trainable params
- alpha/rank ratio below 1.0 causes unstable training; above 4.0 gives diminishing returns
- Recommended: r=16, alpha=32, target all linear + attention layers
pip install -e . # install LLaMA-Factory first
python run_experiments.py --config configs/experiment_config.yaml
python analyze_results.py --results_dir ./outputs/
See analysis_report.md for the full writeup.
MIT