Test intelligence

Ingest a coverage report and repowise can answer two questions your CI cannot: which files are risky and untested, and which tests a given diff actually exercises. That second one turns a 4,000-test suite into the 40 tests that guard the change you just made.

Everything here is an index lookup: no LLM, no network. The same report also gates a pull request on its patch coverage, with no index at all: see Gates in CI.

Quick start#

# 1. Produce a report. Any of these work.
pytest --cov --cov-report=lcov:coverage.lcov
pytest --cov=<package> --cov-context=test   # also builds the per-test map (needs pytest-cov)

# 2. Ingest it.
repowise coverage add coverage.lcov
repowise coverage add .coverage             # per-file coverage + per-test map
repowise coverage status

# 3. Use it.
repowise health                             # untested hotspots light up
repowise impacted-tests main...HEAD         # the tests guarding this branch
repowise coverage check origin/main...HEAD --report coverage.lcov --fail-under 80

repowise coverage status

  Coverage (lcov)
    Files:  412
    Lines:  71.4%
    Branch: 63.9%

  Test-to-code map (coverage.py)
    Tests:   1,204
    Files:   388
    Records: 19,551

Two dimensions, one command#

repowise coverage add stores two different things, and the difference matters for everything below.

The aggregate always gets stored. The map is only built when the report carries per-test contexts. A report without contexts still ingests fine, it just skips the map.

The rows are point-in-time: each ingest replaces them. Ingest at the same commit you intend to query, so line numbers line up. Each ingest is also recorded with its repo-wide figure and the commit it measured (the newest 50 are kept), which draws the coverage trend on the dashboard once there are three reports, and in get_health(include=["coverage"]).

Measured and inferred#

Without a coverage report, repowise still answers from the dependency graph, which records which tests reach which files. Those answers are inferred, always labelled as such, and never turned into a percentage.

The two kinds are shown side by side and never averaged. A call edge says control can reach a file, not that a run did, so inference is safe as a floor ("something reaches this, do not call it untested") and unsafe as a quantity. The graph also links tests to wiring no import states: runner setup files, helpers a test starts by path, conftest.py files and the fixtures a test requests.

Recall reads low because coverage counts a file as run when it was merely imported; among files where over three quarters of what ran was inside function bodies, recall is 77%.

Supported formats#

Force a parser with --format lcov|cobertura|clover|repowise-json|go-coverprofile|jacoco. The normalized JSON shape lets you feed any runner once you map it:

{ "format": "repowise-coverage-v1",
  "files": { "src/foo.py": { "line_coverage_pct": 87.5,
                             "total_coverable_lines": 40 } } }

With no path argument, add auto-discovers coverage/lcov.info, lcov.info, coverage.lcov, coverage.xml, **/cobertura.xml, **/clover.xml, target/llvm-cov/**/*.lcov, coverage.out, cover.out, JaCoCo XML under target/site/ or build/reports/jacoco/, and a repo-root .coverage. Multiple reports merge: a line any report ran counts as run.

Report paths are matched to indexed files by exact key first, then basename, then the longest trailing-path overlap. A tie refuses to guess and is reported as ambiguous rather than mapped to the wrong file. If a whole report comes back unmatched, set coverage.strip_prefix in .repowise/config.yaml.

Building a per-test map#

The map needs a report that records which test covered each line. Two paths build it.

coverage.py dynamic contexts (the main one):

pytest --cov=<package> --cov-context=test   # needs pytest-cov
repowise coverage add .coverage

The .coverage file is read directly as read-only SQLite. Repowise decodes the numbits line bitmaps itself, so it has no runtime dependency on coverage.py, and falls back to the arc table when line bits are absent. Contexts look like tests/test_auth.py::TestLogin::test_ok|run; the leading path becomes the test's own file when it is resolvable.

Per-test LCOV: a report where each end_of_record block carries a distinct TN: name. Blocks with a blank TN: are skipped. A bare suite label with no path resolves to a test id with no test file, which is still usable for "run these" but not for staleness reasoning.

If you ran without contexts, coverage add says so explicitly rather than silently producing an empty map. Rows land in a test_coverage table indexed both ways (repo plus source file for the reverse lookup, repo plus test id for the forward one), capped at 250,000 rows, and the CLI reports how many were dropped if you hit the cap.

Impacted tests#

repowise impacted-tests diffs a change, looks up the changed lines in the map, and returns the tests whose recorded coverage intersects them.

repowise impacted-tests                        # staged changes (the default locally)
repowise impacted-tests main...HEAD            # a branch or PR, from the merge-base
repowise impacted-tests abc123                 # a single commit
repowise impacted-tests main...HEAD --format list | xargs pytest
repowise impacted-tests main...HEAD --format args --runner pytest   # for CI
repowise impacted-tests main...HEAD --format args --prioritize      # whole suite, likeliest to fail first
repowise impacted-tests main...HEAD --explain tests/unit/test_api.py

--prioritize emits the whole suite with the selection at the head, ordered likeliest to fail first, so nothing is skipped. --explain says why a test was or was not selected (the changed file, the evidence and the import route, or the rule that runs it) and its place in the run order. Manifests, lockfiles, production __init__.py files and assets select the tests their reach implies: over 83 replayed failed PR runs, 9 needed the full suite. When the index lags the change's base, files changed in between are compared and only the routes they rewired are added.

Flags are on repowise impacted-tests. It always says which path fired, and it never lets a guess pass for evidence:

With --format list the caveats go to stderr so the pipe into pytest stays clean. For CI, --format args prints runner arguments, or :all whenever any part of the answer is not known. The rules for when it runs everything are on Gates in CI.

Untested hotspots#

Coverage feeds the code health layer's test-coverage markers. The sharpest of them is untested_hotspot, the textbook "write tests before you refactor" case. It fires only when a file is all three of:

1. A hotspot. Flagged as one by the git layer (top-quartile churn with activity floors under it).
2. Centrally depended on. At least 4 dependents. Below that, a churning file is usually a leaf one author is iterating on, and flagging it is noise.
3. Under-tested. Line coverage below 40%. With no coverage ingested at all, it fires only when no paired test file exists and no test reaches the file through the graph.

The file must also be code with at least one function, and not a test file itself: a config file or a module of constants has nothing a test could exercise.

Severity is CRITICAL at 15% coverage or less with 10+ dependents, HIGH at one of those two, MEDIUM otherwise. Its sibling coverage_gap handles the has-coverage-but-thin case, and coverage_gradient applies a continuous deduction proportional to the uncovered fraction, so a file is penalised in proportion to how much of it is untested rather than only at a cliff.

From an agent#

get_risk(changed_files=[...]) leads with a directive whose tests_to_run is the run list repowise impacted-tests makes, with tests_to_run_basis (measured when coverage decided a changed file, else inferred), tests_to_run_why, and tests_run_all when every test must run. tests_to_update names up to three test files the change will probably need edited. include=["tests"] adds test_recommendations, each row keeping its basis:

get_change_risk(revspec=...) returns impacted_tests, the same selection over the change's diff. In its line_coverage block, untested_changes is the strong signal (the file is in the map but nothing covers the lines you touched), stale_test_candidates flags covered lines whose guarding test is absent from the diff, and no_coverage_data means the file is not in the map.

get_change_risk also carries patch_coverage: the share of changed lines the stored coverage ran, with the tests to extend for each uncovered range. With no revspec it covers everything a push would bring, so an agent can self-check before pushing. See get_change_risk.

Empty means unknown, not "no tests"#

Every surface carries a discriminator beside the list. For get_change_risk, impacted_tests.status is one of:

Every status but selected carries run_all: true: nothing vouches for a subset. get_risk keeps coverage availability, freshness and map presence explicit in test_impact.coverage, so an empty list under unavailable analysis reads as unknown, never "no tests needed". The CLI prints "unknown, run the full suite", and per-file lookups report a file with no data as no_coverage_data, not as uncovered.

Configuration#

The coverage: block in .repowise/config.yaml sets discovery:

coverage:
  auto_discover: true
  artifacts:                     # override the discovery globs
    - "coverage/lcov.info"
  format: lcov                   # skip format sniffing
  strip_prefix: "/build/src/"    # trim an absolute prefix from report paths
  reingest_on_update: false

Report paths, ignores and the CI thresholds are on Configuration. With hooks.coverage_reingest: true, an agent's full test run in Claude Code refreshes the stored coverage in the background.

Coverage is also auto-discovered and ingested during init and update, and repowise init --coverage-report <path> takes explicit reports (repeatable). Note that --coverage-report is test coverage, while --coverage controls documentation breadth. Two different things, similarly named.

See also#

• Code health, the coverage markers and how they deduct from the score.
• get_risk, where tests_to_run lives.
• Coverage and CI gates, every coverage flag.