Skip to content
Docs
Download Markdown
In this article

Verify a scientific calculation task

Check a numerical result against an independent calculation with units and tolerance.

Download Markdown

Use the task-package specification to choose the package layout and record evidence for the exact revision.

Pick a scientific workflow whose result can be checked independently. This example follows the numerical-workflow pattern in Terminal-Bench-Science using invented mass and volume readings. Terminal-Bench-Science 0.1

Author and verify

State the units, blank correction, output fields and absolute tolerance before any runs. A second calculation should establish the expected answer independently of the solver's code. A Tier 1 verifier checks finite values, exact units and the error bound. Use domain review to check the interpretation of the method, then use the program to score results.

Calibrate and correct

Run the reference and a control that skips the blank subtraction. Keep the reference calculation and the tolerance rationale with the private evidence. Record an out-of-tolerance value as a failed check and an unavailable runner as an infrastructure error. If an invalid result passes, revise the check and rerun the old and new revisions on the same cases.

Worked example

Use Python 3.12. Subtract a 1 mg blank from a 7 mg reading and divide by 3 mL. The reference should pass. A result that skips the blank or uses the wrong unit should fail.

python3.12 - <<'PY'
from decimal import Decimal

reading, blank, volume = map(Decimal, ("7", "1", "3"))
expected = Decimal("2.00")  # independent reference: 6 mg / 3 mL
tolerance = Decimal("0.05")

def passes(value, unit):
    return unit == "mg/mL" and value.is_finite() and abs(value - expected) <= tolerance

assert passes((reading - blank) / volume, "mg/mL")
assert not passes(reading / volume, "mg/mL")
assert not passes(expected, "g/mL")
assert not passes(Decimal("NaN"), "mg/mL")
print("reference pass, uncorrected, wrong unit and non-finite values fail")
PY

In the package, put the independent answer and the tolerance check in the verifier. Report the numerical error with its unit so a later reviewer can compare revisions.