.. _howto-test-mode:

Run a project's tests
=====================

Goal: run a tree's own test suite on a runner. Same pipeline as a
build, no artifact, an honest verdict.

Basic usage
-----------

.. code-block:: console

   $ saggar submit ./my-project --test
   $ saggar submit ./my-project --test-cmd "make check"
   $ saggar submit ./my-project --test --arch arm64 --env trixie

``--test`` runs the tree's own test suite instead of building: same
pipeline, same target overrides (``--arch``, ``--env``), same live
logs, same failure metadata. The suite's exit code is the verdict.

``--test-cmd`` overrides detection with an explicit command (API:
``"test_cmd"``, which implies ``"mode": "test"``).

What gets autodetected
----------------------

Per build system: ``cargo test``, ``go test ./...``, ``ctest`` and
``meson test`` (after the build those runners need), a ``make -n``
probe for a ``check`` or ``test`` target, ``npm run test`` / ``pnpm``
/ ``bun`` when the manifest declares the script, ``deno task test``,
``mix test`` (with a ``test/`` directory), pytest through the
project's tool, ``swift test`` (with a ``Tests/`` directory), gradle
and maven (with a real suite), and the rake convention through
bundler.

Honesty rules
-------------

- A tree with no detectable suite fails the job, never a green
  no-op, naming what was probed; the answer is an explicit
  ``--test-cmd``.
- Green means success with nothing collected: a test job produces no
  artifact and is never registry material.
- Red is a ``real`` failure in the ``test`` phase. Infra noise still
  retries silently; a red test never does.

Structured results
------------------

When the suite's runner writes report files, saggar parses them and
serves what failed; no log scraping:

.. code-block:: console

   $ curl -H "X-Auth-Token: $SAGGAR_TOKEN" \
       https://saggar.dev/api/v1/jobs/j-…/results

The ``results`` field on the job object carries per-file suite/case
counts and the failing case names with their failure messages. Each
build system declares where its own runner drops reports: surefire XML
for maven, the Test task's XML for gradle, nextest's conventional
``target/nextest/*/junit.xml`` for cargo (plain ``cargo test`` writes
no report; the project's nextest profile opts in), and meson's
``testlog.json``.

Results are collected green or red (a failing suite's report is the
most valuable one), but they are a bonus layer: a malformed report is
skipped with one log line, absence is normal, and the verdict stays
the exit code.

Tests on every push
-------------------

By default a watch builds the tree and also runs its tests: a
triggered push produces the formats×arches group plus a test member
sharing the group id. See :doc:`ci`. A watch pinned to a formats list
cannot also be tests-only; the group composition is how both coexist.