Benchmarks

Each measured size at its default quant, the one ardana pull <family>:<size> fetches, on all 231 items, best first. Sizes whose intervals overlap are not reliably different.

Ordered by accuracy, highest first.

A stopped run covers fewer, easier items, so it follows the complete ones whatever the order.

Accuracy and calibration

Each measured size at its default quant: how often it is right against how far its confidence is from its accuracy. Up and to the left is better. The shaded quadrant is above the median accuracy and below the median ECE. The line, or a ring when one size beats all the others, marks the sizes no other size beats on both.

Accuracy and size on disk

Every measured quant: accuracy against the size of its GGUF, the download and roughly the memory it needs. Up and to the left is better. The shaded quadrant is above the median accuracy and below the median size of the quants shown; the line joins the quants no smaller quant beats. Each quant’s figures are in its family’s tables.

By tier

Each measured size at its default quant, best first: its accuracy on the 48 easy, 72 standard and 111 hard items. A long line means the hard items cost it most. Tier figures carry no interval; each family’s tables give their counts.

By model

• decider

  Typed decisions with calibrated probabilities in one forward pass.

  • 0.8b
  • 2b
  • 4b
  3 of 3 sizes measured

• apertus

  Apertus, a stock instruct model read zero-shot in the chat layout.

  • 8b
  0 of 1 size measured

• gemma-3

  Gemma 3, a stock instruct model read zero-shot in the chat layout.

  • 1b
  • 4b
  • 12b
  3 of 3 sizes measured

• gemma-4

  Gemma 4, a stock instruct model read zero-shot in the chat layout.

  • e2b
  • e4b
  • 12b
  • 26b-a4b
  • 31b
  5 of 5 sizes measured

• granite-4.2

  Granite 4.2, a stock instruct model read zero-shot in the chat layout.

  • 3b
  • 8b
  0 of 2 sizes measured

• jevk5

  JevK5, an open Jev alternative fine-tuned from Qwen3.5 for typed decisions, read in the chat layout.

  • 2b
  • 4b
  • 9b
  3 of 3 sizes measured

• lfm2.5

  LFM2.5, a stock instruct model read zero-shot in the chat layout.

  • 1.2b
  • 2.6b
  • 8b-a1b
  3 of 3 sizes measured

• ling-3.0

  Ling 3.0, a stock instruct model read zero-shot in the chat layout.

  • tiny
  1 of 1 size measured

• llama-3.1

  Llama 3.1, a stock instruct model read zero-shot in the chat layout.

  • 8b
  1 of 1 size measured

• llama-3.2

  Llama 3.2, a stock instruct model read zero-shot in the chat layout.

  • 1b
  • 3b
  2 of 2 sizes measured

• minicpm5

  MiniCPM5, a stock instruct model read zero-shot in the chat layout.

  • 1b
  • 2b
  2 of 2 sizes measured

• nemotron-3-nano

  Nemotron 3 Nano, a stock instruct model read zero-shot in the chat layout.

  • 4b
  0 of 1 size measured

• olmo-3

  Olmo 3, a stock instruct model read zero-shot in the chat layout.

  • 7b
  1 of 1 size measured

• ornith-1.5

  Ornith 1.5, a stock instruct model read zero-shot in the chat layout.

  • 9b
  1 of 1 size measured

• phi-4

  Phi-4, a stock instruct model read zero-shot in the chat layout.

  • mini
  • 14b
  2 of 2 sizes measured

• qwen3

  Qwen3, a stock instruct model read zero-shot in the chat layout.

  • 0.6b
  • 1.7b
  • 4b
  • 8b
  • 14b
  5 of 5 sizes measured

• qwen3.5

  Qwen3.5, a stock instruct model read zero-shot in the chat layout.

  • 0.8b
  • 2b
  • 4b
  • 9b
  4 of 4 sizes measured

• qwen3.6

  Qwen3.6, a stock instruct model read zero-shot in the chat layout.

  • 35b-a3b
  1 of 1 size measured

• qwen3.8

  Qwen3.8, a stock instruct model read zero-shot in the chat layout.

  • 27b
  1 of 1 size measured

• smollm3

  SmolLM3, a stock instruct model read zero-shot in the chat layout.

  • 3b
  1 of 1 size measured

• spark-x2.5

  Spark X2.5, a stock instruct model read zero-shot in the chat layout.

  • 1.7b
  • 4b
  0 of 2 sizes measured