Add two under 50m models

#230
by FlameF0X - opened

anwgpt/anwgpt4.1-base

Tasks Version Filter n-shot Metric Value Stderr
arc_challenge 1 none 0 acc ↑ 0.2065 ± 0.0118
none 0 acc_norm ↑ 0.2543 ± 0.0127
arc_easy 1 none 0 acc ↑ 0.2652 ± 0.0091
none 0 acc_norm ↑ 0.2769 ± 0.0092
hellaswag 1 none 0 acc ↑ 0.2542 ± 0.0043
none 0 acc_norm ↑ 0.2539 ± 0.0043
piqa 1 none 0 acc ↑ 0.5180 ± 0.0117
none 0 acc_norm ↑ 0.4853 ± 0.0117

anwgpt/anwgpt4-base

Tasks Version Filter n-shot Metric Value Stderr
arc_challenge 1 none 0 acc ↑ 0.1886 ± 0.0114
none 0 acc_norm ↑ 0.2389 ± 0.0125
arc_easy 1 none 0 acc ↑ 0.2736 ± 0.0091
none 0 acc_norm ↑ 0.2689 ± 0.0091
hellaswag 1 none 0 acc ↑ 0.2546 ± 0.0043
none 0 acc_norm ↑ 0.2591 ± 0.0044
piqa 1 none 0 acc ↑ 0.5212 ± 0.0117
none 0 acc_norm ↑ 0.5082 ± 0.0117

both using

lm_eval --model hf \
    --model_args pretrained=model \
    --tasks hellaswag,arc_easy,arc_challenge,piqa \
    --batch_size auto

Sign up or log in to comment