# Verify an ML reproduction task


Use the [task-package specification](/data/task-packages) to choose the package layout and record evidence for the exact revision.

Start from an analysis whose result you can reproduce on licensed or original inputs. This example uses the analysis-reproduction pattern and original synthetic observations.

## Author and verify

Give the solver the training observations and the feature values and IDs it must predict. Keep the targets for the held-out split with the verifier. Specify the prediction schema, mean absolute error and threshold before running. Tier 1 checks reject a missing ID, a non-finite prediction or a score above the threshold. The metric supplies the verdict without a judge.

## Calibrate and correct

Use a known-good reproduction and a constant predictor as controls. Pin both split digests, metric code and the task revision. Mark missing or malformed predictions as failed checks. Record runner failures separately. If a constant predictor passes, revise the split or threshold with a new verifier revision and rerun the controls.

## Worked example

Use Python 3.12. The script fits a slope from training observations and compares predictions with two held-out targets. The reference should pass and both the constant and missing-ID controls should fail.

```bash
python3.12 - <<'PY'
from math import isfinite

train = [(1, 2), (2, 4), (3, 6)]
features = {"a": 4, "b": 5}
targets = {"a": 8, "b": 10}
slope = sum(x * y for x, y in train) / sum(x * x for x, _ in train)

def passes(predictions):
    if predictions.keys() != targets.keys():
        return False
    if not all(type(value) in (int, float) and isfinite(value) for value in predictions.values()):
        return False
    error = sum(abs(predictions[key] - value) for key, value in targets.items())
    return error / len(targets) <= 0.10

assert passes({key: slope * x for key, x in features.items()})
assert not passes({"a": 4, "b": 4})
assert not passes({"a": 8})
assert not passes({"a": 8, "b": "10"})
print("reference pass, constant fail, missing ID fail")
PY
```

When packaging the task, read predictions from the declared output artifact and score them against targets stored with the verifier. Preserve the actual error and threshold in the private evidence.
