Week 1 · homework
A frontier agent vs. your tasks
waiting
Each task you submitted is attempted by a frontier AI agent, alone, in a sealed container, and graded by your own verifier. The agent sees your instruction, your data and the container, with internet access; it never sees your solution or tests.
- solved
- not solved
- error (rerun)
- running
- queued
bars share one scale per column
| Outcome | Tasks | Avg agent time | Avg share of time limit | Avg model calls | Avg cost | Avg partial score |
Partial score is our own regrade, which gives credit for being close; the official reward is all-or-nothing. Each task page explains its method.
| Task | Result | Reward | Agent time | Model calls | Cost |
Reading the board
What the results mean
- Solved: your verifier gave reward 1, every test passed. Not solved: at least one test failed, so your task held this time.
- Error is an infrastructure failure, not the agent's: an outage or a crashed container before grading. Those runs are repeated.
- Agent time is measured against your task's own time limit. Cost is the model's token cost at list price.
- A failure only shows your task is hard if the agent had everything it needed. Next week you read its full attempt and decide: was it the science, or a gap in your task?