Skip to main content

Delivery checks for Harbor tasks

Catch broken graders before your lab does.

We replay every task in your delivery: your reference, an empty submission and up to two candidate solutions, each twice. You get a receipt your lab can verify.

Fixed scope and fee, quoted within one business day.

What the check catches

Full marks for code that does nothing

A module that skips at import scored 60 of 60 tests, the same as the reference.

vf2-speedup-networkx 3.0.3, historical revision

Same output as the reference, zero marks

A refactor matching the reference's output scored 0 of 27, because its new file never reached the grader.

cargo-flight-dispatch 2.0.4

A one-slot store passed every test

A store that keeps only the last value passed 7 of 7 tests.

kv-store-grpc

Public Terminal-Bench tasks, each control run twice on Harbor 0.24.0.

How it works

  1. Send the tasks

    The task revisions, your Harbor setup, and any candidate solutions with the verdict you expect.

  2. We replay each one

    Reference, empty submission and candidates, each run twice. Every file is hashed and every file that reaches the grader is recorded.

  3. You get the receipt

    For each task: rewards, test counts, every mismatch with what you expected, and what to fix.

A receipt your lab can check

Reproducible

Every run and file is checksummed, so your lab can replay it.

Plain verdicts

Each mismatch is named with the evidence beside it.

Yours to share

Forward it to the lab that receives your delivery.

Delivery check receipt

Harbor 0.24.0

Task
cargo-flight-dispatch 2.0.4
Reference
reward 1 · 27/27 · twice
Empty submission
reward 0 · 9/27 · twice
Refactor, expected to pass
reward 0 · 0/27 · twice

Outcome

FINDING

Excerpt from the October 24, 2026 sample. Read the full sample

Check your task.toml

Runs in your browser. Nothing is uploaded.

Choose a task.toml to see what reaches the grader.

Order without a call

Fixed scope and fee, quoted within one business day. Payment is by invoice, and the fee is the same whatever the result.

  1. Which tasks, at which revision?
  2. Which Harbor setup do you run?
  3. What decision waits on the result, and by when?

Ship tasks your lab accepts the first time.