Which instances count
Every submission covers all of the open round's private Self-hosted instances. They are separate from the Main track's instances and are never published. Public instances are for development and don't count.
What can happen to an instance
After the submission closes, the checker verifies each uploaded solution and recomputes its objective.
| Result | Score | Message |
|---|---|---|
| Feasible | Scored | – |
| No solution uploaded | 0 | No solution submitted |
| Infeasible | 0 | Infeasible: followed by the constraints the solution violates |
| The checker could not read the solution | 0 | Checker exited with code 1 |
| A problem on our side, 3 times | 0 | Evaluation failed on our side |
The checker recomputes the objective from your decisions, so a claimed objective_value that does not match them is rejected. A problem on our side, such as a checker that fails to start, is retried a minute later; only after 3 failed tries does the instance score 0.
Instance score
A feasible solution scores the ratio of its objective to the instance's reference objective, a frozen best-known value. Matching the reference scores 1, a worse solution less, and a better one more, up to 2.
| Objective and reference | Minimize | Maximize |
|---|---|---|
| Both positive | reference ÷ objective | objective ÷ reference |
| Both negative | objective ÷ reference | reference ÷ objective |
| On opposite sides of 0, or the reference is 0 | 2 if better, 0 if worse | 2 if better, 0 if worse |
Equal values score 1, including both 0. The objective is the one the checker recomputes from your decisions. Infeasible solutions and solutions the checker rejects score 0.
From instances to a submission score
Instance scores are averaged within each problem, then across problems, so every problem weighs the same however many instances it has.
Leaderboard
- The leaderboard is per stage and per track. Self-hosted and Main are ranked separately and never compared.
- A team's stage score comes from one Completed submission per round: the one with the highest score, then the least time used, then the earliest. Its problem scores are averaged over every problem released so far. A round the team skipped counts 0 on its problems, and so does the open round until the team submits to it.
- Teams are ranked by that score, highest first. Equal scores are ranked by time used, lowest first: each private instance's time from download to upload as a share of its time limit plus the 60-second grace, or 1 without a feasible solution, averaged like the score. A skipped round's problems count 1, so leaving work undone never helps.
- Next to the score, Avg passed is the mean over only the problems your round-best submissions passed (a feasible solution for every private instance, none failed), and Avg submitted the mean over those they uploaded at least one solution for. Under each, how many of the released problems that is ("5 of 6"). Neither is used for ranking.
- The menu and the sub-track cards rank teams on one category or one formulation instead: the average of its released problems' scores in each team's round-best submissions, with ties broken by time used on the same problems.
What you see
For every instance: its score, or the message it scored 0 with.