Project template

This is the starting point of an Autoresearch@Home project. Agents improve the worker (src/); your grader (grader/) measures it on data the agents never see. The same files run on your laptop, against the public sample, and on the platform, against your held-out data. arh init DIR writes it out; the platform's authors' guide (author.md on its API, beside agent.md) says how to make it yours, what the conformance report checks, and how to apply.

File	Who changes it	What it is
autoresearch.toml	you	the metric, runner class, editable and frozen paths, pricing (arh-spec validate)
PROGRAM.md	you	instructions for the agents: the task, the rules, what counts as cheating
CONTRACT.md	you, if you extend it	the worker HTTP contract and the platform's override
compose.yaml	nobody	the worker and the grader on an internal network
dockerfile.worker	you, rarely	builds the worker from src/
dockerfile.grader	you	builds the grader from grader/
grader/	you	your scorer, with the grader SDK (grader/arh_grader)
sample/	you	the public sample: the worker sees it at /data, the grader at /eval
src/	agents	the worker: the code being improved

The example is an echo task: the worker returns each item unchanged and the grader times it (wall_ms). Replace src/, grader/grade.py and sample/ with your project, and rewrite PROGRAM.md.

The six rules

1. Internal network; the grader is client-only. The worker and the grader share the internal network eval with no outbound access. The grader calls the worker and never listens.
2. The secret seed goes to the grader only. AR_SEED is in the grader's environment on the platform and nowhere else. Never print it or write it to the result.
3. Worker logs from settlement runs are never published. On the platform the worker sees your held-out inputs.
4. The grader treats worker responses as hostile (size caps, timeouts, schema checks) and does all timing itself.
5. Worker and grader run on disjoint whole cores.
6. Settlement also scores the public sample, so a gap between public and held-out scores flags overfitting.

The result is read only from /result/result.json, never from stdout.

Running it

docker compose up --build --abort-on-container-exit --exit-code-from grader
docker compose run --rm --no-deps --entrypoint cat grader /result/result.json
docker compose down --volumes

The second command reads the result from the result volume (which stays until down --volumes) in a one-off grader container. The worker can write only to /tmp and /dev/shm (64 MiB): memory-backed, private to its container and gone after the run. compose.yaml pins the worker to cores 0-1 and the grader to 2-3; on a machine with fewer than 4 cores, add a local override that gives each its own core, e.g. a compose.override.yaml (not committed) with services: { worker: { cpuset: "0" }, grader: { cpuset: "1" } }. Check the configuration with arh-spec validate . before you publish the project: it also refuses symlinks among the frozen files (frozen content must be real files), and warns about editable files the platform freezes anyway because tools load them on their own (CONTRACT.md, "Auto-loaded files").

Builds on the platform

The platform builds dockerfile.worker, dockerfile.grader and dockerfile.null in a VM whose only way out is its package proxy. Two things follow, and arh-spec validate . checks both:

• Every image a Dockerfile names is pinned by digest. That covers FROM, COPY --from= and RUN --mount=...,from=: write python:3.13-slim-bookworm@sha256:<digest>, which docker buildx imagetools inspect python:3.13-slim-bookworm shows. Don't put a build argument in the name or a # syntax= line at the top. The proxy serves a build only its own Docker Hub repositories, by digest.
• Hosts beyond the base list must be declared. The base list has PyPI, Maven Central, crates.io and the Debian, Ubuntu and Alpine mirrors. Declare any other host in the [build] egress table of autoresearch.toml, including a registry other than Docker Hub that a base image comes from. An admin reviews the list at intake, and each reward version fixes its own. A host one project declares stays closed to every other project's builds.

Data

sample/ is public: agents see it and can overfit to it. Your held-out set has the same format, is uploaded to the platform separately, and is never committed. The platform mounts it at the grader's /eval instead of ./sample (see CONTRACT.md, compose.platform.yaml).

The grader SDK

grader/arh_grader is the platform's grader SDK, vendored so the grader builds without network. Its module docstrings are its reference: Client and wait_ready talk to the worker, seed() and rng(label) give the run's seed, begin_timed() and remaining() its startup and deadline, and write_result(...) writes the result (with item_scores=, also items.json: a deterministic metric needs it). arh_grader.RELEASE is its release. Don't edit it in place: move to a newer release by copying grader/arh_grader from a fresh arh init over it (the authors' guide, "Upgrading the grader SDK").