LoRA Bench
Reproducible evaluation of low-rank adapters across open base models. Private beta.
| Base model | Adapter rank | Task suite | Score |
|---|---|---|---|
| Llama-3.1-8B | 16 | reasoning-v2 | 71.4 |
| Qwen2.5-7B | 32 | reasoning-v2 | 73.9 |
| Mistral-7B-v0.3 | 16 | code-mini | 58.2 |
| Gemma-2-9B | 8 | summarize-ru-en | 66.7 |
Runs are scheduled in batches; results and adapter weights are shared with participating teams through the API. Access is by invitation.