Eval Landscape (Capabilities)

150+ model releases from 20+ labs. This compilation focuses on foundation models intended for general-purpose use. Notable omissions: image generation, video generation, and speech models.

More context

A model release is a big moment. I was interested to track what labs self-report in release cards as a snapshot of important capabilities that they are currently tracking. Benchmarks naturally have a finite lifespan. When it saturates or stops being useful, it gets retired. Early on, it was all MMLU and math. More recently (as of April 2026), now there's maybe one math benchmark per card, and a lot more focus on agentic and tool-use evaluations. The landscape shifts quickly.

Much of this data was compiled with the help of Claude and other AI coding tools, scraping model cards, extracting scores from PDFs and blog post images, and cross-referencing across sources. The data is not perfect, but the intention is to get a general sense of things. If you spot a major error, let me know.

1. Benchmark Lifecycles

When benchmarks first appeared and last appeared in official model cards. Row color indicates the benchmark's category — toggle "By lab" to color each report dot by the lab that reported it instead.

Sort:

Range:

Dot color: By category By lab

2. Saturation Curves

Best-known scores over time. Each point is a new record. Lines and points are colored by benchmark category — toggle "By lab" to color each data point by the lab that produced the score. The "Frontier" preset is a hand-picked shortlist of benchmarks still being actively reported in recent model cards.

Range:

Dot color: By category By lab

3. Release Cadence

Model releases per lab over time. Hollow dots indicate releases with no public benchmarks. The "Other" row includes smaller labs, startups, and organizations that are no longer active.

Range:

4. Model Card Complexity

Number of distinct benchmarks reported per model release (Y axis). Circle size is the geometric mean size of those benchmarks — the natural average when sizes span several orders of magnitude. Releases with no size-known benchmarks fall back to a neutral small circle.

Range:

5. Typical Benchmark Size by Lab

Geometric mean size (# of items) of benchmarks reported on each model card, for the major frontier labs. Log-scale Y axis. Each dot is one release; dashed line is a least-squares fit per lab. See tools/eval-landscape/NOTES.md for why we use geometric mean.

Range: