Overview
The core component is a Rust library that implements the majority of The Test Cabinet’s functionality. Every other component links against this library and exposes its functionality through its own interface rather than re-implementing any of it, so a run behaves identically whether it is launched from a script, a remote request, or a window. See Architecture.
Responsibilities
Section titled “Responsibilities”The core owns everything that happens during a run, and defines the data contracts the rest of the system is built around.
- Test cases — resolving a test case
version and its selected variant,
and reading the
test-case.tomlmanifest that says what gets seeded, rendered, and checked. - Execution — seeding a fresh git repository, running the harness inside an isolated container, and collecting the produced working tree.
- Agent harnesses — a single abstraction for invoking any supported coding harness, absorbing each one’s quirks.
- Orchestrators — the harness-agnostic strategy that decides how a run’s harness sessions are conducted.
- Engines — the runtime a produced game is built on, selected independently of the test case.
- Test case groups — the repo-defined sets of related cases the home page renders one leaderboard per.
- Harness events — translating each harness’s raw output into one normalized, live event stream.
- Metrics — recording the run time, token, and cost data every run produces.
- Validation — the automated first pass that builds, loads, and optionally screenshot-compares an implementation.
- Run records — the fixed data contract a run emits.
- Results — getting a finished run onto the gallery through review and publish.
Wrapping the core
Section titled “Wrapping the core”The wrapping components are thin, adding only the surface their own interface requires. The behavior of a run lives in the core.