# Build and check a task package


A task package gives an agent work to do in a terminal and gives a verifier a repeatable way to check the result. Use the Harbor task format to build the files. Use Humanbased's inventory fields to describe the exact revision you checked. [Harbor task format](https://docs.harborframework.com/tasks/overview)

### Separate the task files by audience

| Files | What to put there | Who needs it |
| --- | --- | --- |
| `instruction.md` | The work, available inputs, output paths and success criteria | The solver |
| `task.toml` and `environment/` | Run limits, resource requirements, input files and a reproducible environment. Pin the image by digest. | The solver and runner |
| `tests/` | The verifier, its fixtures and its grading dependencies | The verifier |
| `solution/` | A reference implementation that proves the task is solvable | An authorized verifier or reviewer |

In separate-verifier mode, list each output the verifier needs under `artifacts` in `task.toml`. Treat those outputs as untrusted input. Keep hidden tests, expected answers and the reference implementation out of the agent image and the Marketplace preview. Record whether a reference exists. Harbor itself makes the reference optional. [Separate verifier](https://docs.harborframework.com/tasks/separate-verifier) · [Artifact transfer](https://docs.harborframework.com/tasks/artifacts) · [Reference solution](https://docs.harborframework.com/tasks/solution)

### Record the evidence for one revision

| Inventory field | What to record |
| --- | --- |
| Task family and package version | The family, exact package revision, manifest digest and pinned environment digest. Count distinct tasks in the listed snapshot. |
| Verifier tier and revision | Record each method and its version separately. Tier 1 is a programmatic check, Tier 2 is comparison to a reference oracle, and Tier 3 is rubric-based judgment. A tier describes the method of checking. |
| Reference-solution presence | Record whether the package has a reference, with a revision or digest available to authorized readers. |
| Calibration | For a frontier-agent calibration claim, pin three agent configurations and run eight trials each on the same package and verifier revision. Record models, settings, budgets, trial counts and a batch reference. Compare only runs with compatible configurations. |
| Run evidence | Record execution state (`completed`, `budget_exhausted`, or `infra_error`), check result, and the reward-validity decision as separate fields. Use the verifier report for the check result. Count infrastructure errors under execution state and give the numerator and denominator for every reported rate. |

When checking a listing, match its reported metrics to the selected package and verifier revisions. Treat missing fields as unavailable. Distinguish seller-reported evidence from results the platform ran. Share only approved summaries and references with buyers. Raw verification logs stay internal.

### Run a small example

This synthetic firewall audit uses four invented input lines. It illustrates the package layout and verifier checks without exposing a real task's answer. Use Harbor 0.25.0 and a running local Docker engine. Run the following in a new working directory.

```bash
mkdir -p firewall-port-audit/{environment,solution,tests}

cat > firewall-port-audit/instruction.md <<'EOF'
Read /app/rules.txt. Each row contains a protocol and port separated by a
space. Write /app/ports.txt with the distinct TCP port numbers, in numeric
ascending order, one per line, ending with a newline.
EOF

cat > firewall-port-audit/task.toml <<'EOF'
artifacts = ["/app/ports.txt"]

[agent]
timeout_sec = 30

[environment]
build_timeout_sec = 120

[verifier]
environment_mode = "separate"
timeout_sec = 30
EOF

cat > firewall-port-audit/environment/Dockerfile <<'EOF'
FROM python:3.12-slim@sha256:4840d514c583a4d3d8cad3940e68dee6f0f6f3635c12fcd0a3d74c21a5b93a54
WORKDIR /app
RUN printf '%s\n' 'tcp 443' 'udp 53' 'tcp 22' 'tcp 443' > /app/rules.txt
EOF

cat > firewall-port-audit/solution/solve.sh <<'EOF'
#!/bin/sh
set -eu
awk '$1 == "tcp" { print $2 }' /app/rules.txt | sort -nu > /app/ports.txt
EOF

cat > firewall-port-audit/tests/test.sh <<'EOF'
#!/bin/sh
set -eu
mkdir -p /logs/verifier
if printf '22\n443\n' | cmp -s - /app/ports.txt; then
  printf '1\n' > /logs/verifier/reward.txt
else
  printf '0\n' > /logs/verifier/reward.txt
fi
EOF

chmod +x firewall-port-audit/{solution/solve.sh,tests/test.sh}
harbor run -p firewall-port-audit -a oracle -e docker -o jobs --job-name oracle
harbor run -p firewall-port-audit -a nop -e docker -o jobs --job-name nop
```

The reference run should report reward `1` and the no-op run should report `0`, both with zero exceptions. Check each job's `result.json`. In a calibration record, retain the exact package and verifier revision for every run and report infrastructure errors separately. These two smoke checks establish solvability and no-op rejection for the example. A frontier-agent calibration claim requires its own model trial batch. [Harbor authoring tutorial](https://docs.harborframework.com/tutorials/create-a-task)

### Choose a generation recipe

Choose the recipe that matches the evidence your item needs.

- [Verify a software task](/data/task-package-software)
- [Verify an ML reproduction task](/data/task-package-ml)
- [Verify a scientific calculation task](/data/task-package-science)
- [Verify a synthetic reconciliation task](/data/task-package-finance)
- [Verify a tool-recovery trajectory](/data/task-package-fundamentals)
- [Verify a formal proof task](/data/task-package-math)

For the inventory card, record the item kind, exact version, author and verifier roles, source and permitted uses, and which evidence is available. Count distinct included tasks for a task package. Count individually identified and accessible trajectories for a trajectory set. Record who corrected an item, the reason, its review state and the before-and-after check results. If a person or model scored an explanation, keep its rubric, calibration and review separate from the program result.
