Skip to content

Task structure

my-task/
├── instruction.md # what the agent is told
├── task.toml # timeouts, resources, verifier config
├── environment/
│ ├── Dockerfile # docker-compose.yaml and Apptainer.def also supported
│ └── ...
├── solution/
│ └── solve.sh # reference solution, used for oracle runs
└── tests/
├── test.sh # verifier entrypoint
└── ...

Only what the agent should see. Never the grading logic. From the upstream hello-world example:

Create a file called hello.txt with "Hello, world!" as the content.

Real fields (schema_version = "1.4"):

schema_version = "1.4"
[task]
name = "harbor/hello-world"
version = "1.0.0"
authors = []
keywords = []
[metadata]
difficulty = "easy"
category = "programming"
tags = ["trivial"]
[verifier]
timeout_sec = 120.0
[agent]
timeout_sec = 120.0
[environment]
build_timeout_sec = 600.0
cpus = 1
memory_mb = 2048
storage_mb = 10240
gpus = 0
mcp_servers = []
[verifier.env]
[solution.env]
  • The instruction only relies on what exists in the environment
  • tests/ can decide success without a human
  • solution/solve.sh scores full marks (oracle baseline)
  • There is no shortcut around the tests (reward hacking)
  • Resources and timeouts are pinned in task.toml