# Verify a software task


Use the [task-package specification](/data/task-packages) to choose the package layout and record evidence for the exact revision.

Start with a narrow terminal behavior that a solver can reproduce. This recipe follows the Terminal-Bench task pattern with an original port-audit fixture. [Terminal-Bench repository](https://github.com/harbor-framework/terminal-bench)

## Author and verify

Write the input format, output path, ordering and newline requirements in the instruction. In the solver image, include the input and required utilities. Put the expected outputs and case runner in the separate verifier. This is a Tier 1 programmatic check. A judge adds no evidence when the output has an exact contract.

## Calibrate and correct

Run the reference, a distinct correct implementation and a no-op. Keep at least one input that exposes hardcoded answers. In model trials, record the package and verifier versions, the outcome and whether the reward was valid. An image start failure is an infrastructure outcome. If a hardcoded answer passes, save the failing case and rerun both verifier revisions.

## Worked example

Use Python 3.12 in an empty working directory. This local smoke check uses two synthetic inputs. It should print that the reference passed both cases and the hardcoded output failed the second.

```bash
python3.12 - <<'PY'
def reference(lines):
    ports = {int(port) for protocol, port in map(str.split, lines) if protocol == "tcp"}
    return "".join(f"{port}\n" for port in sorted(ports))

cases = [
    (["tcp 443", "udp 53", "tcp 22", "tcp 443"], "22\n443\n"),
    (["tcp 80", "tcp 8080", "udp 22"], "80\n8080\n"),
]
assert [reference(lines) == expected for lines, expected in cases] == [True, True]
assert ["22\n443\n" == expected for _, expected in cases] == [True, False]
assert ["" == expected for _, expected in cases] == [False, False]
print("reference 2/2, hardcoded 1/2, no-op 0/2")
PY
```

To make a Harbor package, turn the case list into verifier-only fixtures and use the reference run and no-op commands in the task-package spec. The smoke check proves the cases distinguish outputs on these inputs.
