Test Coverage Gap Localizer: a % tells you nothing about this PR

A test coverage gap is a line changed in a pull request that no test executes. The Test Coverage Gap Localizer in bugweg looks for exactly that, because a repo-wide coverage percentage cannot say whether the code in this PR is tested. A project can sit at 92% coverage and still ship a PR where the one new conditional handling refund amounts is not touched by a single assertion.

The Test Coverage Gap Localizer is included on the Team and Enterprise plans and during the free trial. It is not part of the Developer plan.

Why the percentage is the wrong unit

A coverage percentage is an aggregate over the whole codebase, so it is dominated by whatever was already covered last quarter. It can stay flat, or even improve, while a PR adds an unexercised branch in the part of the code most likely to break. The two measurements answer different questions:

Say a PR adds a branch to an order-processing function: if a customer’s loyalty tier changed mid-cart, recalculate the discount before applying tax. The recalculation is new, it is in the diff, and nothing in the existing suite runs it. The repo-wide number barely moves because the file was already well covered elsewhere, so a reviewer who only sees a healthy percentage moves on.

What the localizer does

The localizer looks only at lines added in this PR. It runs on PRs that bugweg’s triage rates MEDIUM risk or higher. Where it can, it runs the repository’s own jest, vitest or pytest suite with coverage on the PR head, in an isolated sandbox, and checks which of the added lines actually executed. It then reports each new or changed function with added lines that never ran. The report appears per function in the bugweg check run summary, not as an annotation on each diff line.

When the measured run is not possible, the check run says so and falls back to a clearly labelled static scan that asks whether a nearby test file calls the changed function. That happens, for example, when dependencies fail to install, the PR changes package.json or test configuration, the setup is unsupported (pnpm workspaces, Go, Java or Ruby), or the dependency install or the test run takes longer than 120 seconds.

A suggested test, run in the sandbox

For each gap, bugweg generates a suggested test and runs up to three of them per PR. The check run lists each one by file name, marks it passed, failed or not run, and shows how many of the flagged lines it actually executed. It runs in your repo’s environment when time allows, otherwise in an isolated harness of the source file plus the test, and the check run says which. The test code itself is not shown in the check run or dashboard today.

Nothing is committed to your repository.

Where it stops

The features page lists the disclosed limits: npm installs skip postinstall scripts, git and private-registry dependencies are unreachable, one project is measured per PR in a monorepo, and results for one-line functions are approximate. It checks changed .ts, .py, .go, .java and .rb files, up to eight changed functions per PR; .js, .jsx and .tsx files are not checked yet. Suggested tests are only run with jest, vitest or pytest, so suggested tests for .go, .java and .rb files are listed as not run, with no passed or failed result. A passing suggested test shows that those lines ran, not that the behavior is correct, so it does not replace review.

The same per-PR thinking runs through the rest of bugweg. Dependency Blast Radius asks which call sites a changed signature breaks, and the features page lists what each check does today. Plan details are on the pricing page.

A checklist for any PR

• Which functions did this PR add or change?
• Does at least one test execute each added line, and is that measured rather than assumed?
• Is the riskiest added line among those that run?
• For any gap, is there a test that would exercise it, and does it pass?