Add two under 50m models
#230
by FlameF0X - opened
anwgpt/anwgpt4.1-base
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| arc_challenge | 1 | none | 0 | acc | ↑ | 0.2065 | ± | 0.0118 |
| none | 0 | acc_norm | ↑ | 0.2543 | ± | 0.0127 | ||
| arc_easy | 1 | none | 0 | acc | ↑ | 0.2652 | ± | 0.0091 |
| none | 0 | acc_norm | ↑ | 0.2769 | ± | 0.0092 | ||
| hellaswag | 1 | none | 0 | acc | ↑ | 0.2542 | ± | 0.0043 |
| none | 0 | acc_norm | ↑ | 0.2539 | ± | 0.0043 | ||
| piqa | 1 | none | 0 | acc | ↑ | 0.5180 | ± | 0.0117 |
| none | 0 | acc_norm | ↑ | 0.4853 | ± | 0.0117 |
anwgpt/anwgpt4-base
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| arc_challenge | 1 | none | 0 | acc | ↑ | 0.1886 | ± | 0.0114 |
| none | 0 | acc_norm | ↑ | 0.2389 | ± | 0.0125 | ||
| arc_easy | 1 | none | 0 | acc | ↑ | 0.2736 | ± | 0.0091 |
| none | 0 | acc_norm | ↑ | 0.2689 | ± | 0.0091 | ||
| hellaswag | 1 | none | 0 | acc | ↑ | 0.2546 | ± | 0.0043 |
| none | 0 | acc_norm | ↑ | 0.2591 | ± | 0.0044 | ||
| piqa | 1 | none | 0 | acc | ↑ | 0.5212 | ± | 0.0117 |
| none | 0 | acc_norm | ↑ | 0.5082 | ± | 0.0117 |
both using
lm_eval --model hf \
--model_args pretrained=model \
--tasks hellaswag,arc_easy,arc_challenge,piqa \
--batch_size auto