Research · Updated 16 Oct, 10:38 pm IST

Study finds AI benchmarks cover only a small share of US work tasks

Why it matters for readers: It helps you understand that popular AI tests cover only a small slice of real work.

The authors estimate 90% of American working hours are spent on tasks with little or no coverage from existing AI benchmarks.1
The Benchmark Map maps 923 O*NET occupations to AI benchmark prompts to measure coverage of real-world tasks.1
Only 208 of 923 occupations have a task represented by at least five benchmark prompts, indicating sparse coverage.1
Coverage is uneven across job families: six of 22 job families have no occupations meeting that five-prompt threshold.1
The analysis finds benchmark coverage is concentrated in larger wage-bill labor markets rather than being randomly distributed.1

Get a brief like this every morning

Uzha reads hundreds of sources and gives you the stories that matter for your work, with every source linked. Free.