Research data
Every table behind every figure on the research hub, downloadable as CSV and JSON. Each table links back to the study and page it belongs to.
| Table | Study | Rows | Evidence class | Download |
|---|---|---|---|---|
| Optimization leaderboard, SPEC-027 | Optimization leaderboard | 13 | measured-local | |
| Every drawn skill, with its outcome | Does the skill beat the base model? | 60 | measured-local | |
| Beating the base model by shape, size and installs | Does the skill beat the base model? | 11 | measured-local | |
| Rule checks, lane K, three repetitions per arm | Rule checks | 15 | measured-local | |
| Prepared facts, R9, three repetitions per arm | Prepared facts | 31 | measured-local | |
| Prepared facts by selection cohort | Prepared facts | 3 | measured-local | |
| Cheaper on a smaller model, per skill | Cheaper on a smaller model | 34 | measured-local | |
| Cheaper on a smaller model, per task | Cheaper on a smaller model | 68 | measured-local | |
| Rule checks from absolute prohibitions | Rule checks from absolute prohibitions | 20 | measured-local | |
| Tasks graded without the skill under the stricter prompt | Tasks the assistant already solves | 18 | measured-local | |
| Rule checks on file paths | Rule checks on file paths | 10 | measured-local | |
| Rule checks from conditional prohibitions, the rest of the band | Rule checks from conditional prohibitions, the rest of the band | 5 | measured-local | |
| The mission rule on the unseen bands | The mission rule, out of sample | 35 | measured-local | |
| Helper extraction candidates, lane K | Helpers built from documented steps | 15 | measured-local | |
| Random-draw loop, outcome per skill drawn | Improvement on a random draw | 8 | measured-local | |
| Random-draw loop, per round | Improvement on a random draw | 8 | measured-local | |
| Random-draw loop, the comparisons | Improvement on a random draw | 3 | measured-local | |
| Delegation pilot, F5, three repetitions per arm | Handing a step to a cheaper model or to code | 6 | measured-local | |
| Rule-check bands, seen and unseen | Rule checks out of sample | 5 | measured-local |
19 tables across 14 studies.