ModelBenchkuhn-labs

Historical benchmark · different suite. View current ModelProbe scorecards →

Hardware tier · GPU

GPU leaderboard

Ranked by capability composite. Everything above the entry cards — consumer Ampere–Ada through Blackwell / Hopper datacenter silicon, any count; the exact recorded device is on every row, and every row's serving worker is verified against BOSH logs.

How to read this board
  • The number. Capability composite — the equal-weight mean of 18 category means over the 146-item suite.
  • Ships. Ships = mean of per-PRD mean scores over the non-saturated, mechanically graded PRDs, run under the opencode on-prem harness on placement-attested workers. Means, not medians: PRD scores are coarse and bimodal, so a median can hide a 40% failure rate behind a 1.0. Saturated PRDs (every model mean ≥ 0.95) are excluded, not hidden. Provisional until every model × PRD cell has at least two runs in one era. It is the site's headline; this board is the diagnostic behind it — on this panel the capability composite correlates negatively (ρ ≈ −0.6, n = 6) with real-work outcome; it is a diagnostic, not a ranking.
  • Shared ranks. Rows whose gap to the tier anchor is below the paired minimum detectable effect (power 0.80) are statistically indistinguishable at this suite size, so they share a rank and a T chip instead of being ordered by noise.
  • The chips. ⚠ capped — the run hit its wall-clock cap; not attempted: the whole run hit its wall-clock cap before these items were sent to the model. They still score 0 (runner rule) but are not a verdict on the model. partial — some categories produced no gradable rows, so the composite is not comparable and the row is excluded from statistical tiering. ◆ verified — the serving worker for this row was verified against BOSH logs.

1 of 18 category carries no information in this panel: debugging (all 0.96).