07 · Measuring Models
HumanEval
Coding ability benchmark
A benchmark of coding problems measuring programming ability — does the generated code pass the tests?
Concrete example
The model writes a function; the test suite runs it to check it actually works.