Skip to content
Sources: AGENTS.md ai/test-matrix.yml Makefile tests/AGENTS.md commit 3053bf160578 build-config

Testing strategy

CCB testing seeks the smallest reproducible evidence that covers the risk; it does not make every pull request run every platform. Commands come from ai/test-matrix.yml, Makefile, CI, and test source.

Narrow to broad

  1. Run the target file's formatter, schema check, or static contract check.
  2. Run the nearest unit or regression test, with a focused Catch2 filter.
  3. Run subsystem loading or integration checks such as full JSON loading.
  4. Expand to matrix builds for public contracts, platforms, or releases.
Change Required evidence Typical reason to expand
C++ implementation make astyle-check, focused Catch2 Shared core, serialization, or performance hotspot
Test framework/public header make -j2 tests Compiler or feature-combination differences
JSON/EOC/mod Formatter and make -j2 json-check Loader, schema, or cross-mod interaction
Public Lua contract Declaration, coverage, and focused unit checks Native registration or Platform contract change
Agent/docs metadata Metadata checker and tools/agent tests Generated inventory or CI routing changes
Android Gradle unit/build target Java/native boundary or resource packaging

Platform v1 contract acceptance commands

Finish a coherent Lua domain batch, including declarations and test source, before running these acceptance checks. These are current commands, not a claim that this documentation change rebuilt the game or verified every API behavior.

# validation: agent-context
python3 tools/agent/check_project_metadata.py
python3 -m unittest discover -s tools/agent -p 'test_*.py'

# validation: lua-contract
python3 tools/lua_api/check_luals_declarations.py
python3 tools/lua_api/check_platform_native_inventory.py
python3 tools/lua_api/check_platform_contract.py
python3 tools/lua_api/check_platform_coverage.py
python3 tools/lua_api/check_cmake_contract.py
python3 -m unittest discover -s tools/lua_api -p 'test_*.py'

Write regression tests

  • Name observable behaviour, not an implementation detail.
  • Use a minimal fixture without test-order, current-time, or unfixed-randomness dependencies.
  • On failure, preserve the filter, assertion context, log, and RNG seed.
  • For a bug, first show the test catches the original problem, then show it passes after the fix.
  • Do not hide a deadlock, infinite loop, or regression by increasing a timeout.

Report truthfully

Separate Passed, Failed, and Not run in the pull request. If Windows, MSVC, Android, or an expensive build did not run, say so; Linux configuration is not their substitute. Record the first root cause, correction, and rerun result. For an intermittent failure, report its seed and reproduction count.

Use the validation quickstart for commands and debugging for failure analysis.