Here is an uncomfortable exercise. Take your best-tested module — the one with the coverage badge you're proud of — and ask a different question than "do the tests pass." Ask: if someone changed the code to be subtly wrong, would a single test turn red?
Coverage can't answer that. Coverage measures which lines ran. It says nothing about whether your assertions would notice if those lines misbehaved. A test that calls a function and checks nothing meaningful executes every line and catches nothing. The badge still says 97%.
A worked example, from a package you probably ship
We ran this on color-string — the CSS-color parser under the whole color ecosystem, millions of weekly downloads. Its suite is genuinely good by the usual standard: 97.4% line coverage, 195 assertions, all green.
Then we ran mutation testing against it. A mutation tester makes thousands of small, deliberate breaks in the code — flip a < to <=, delete a branch, swap + for - — and for each one, re-runs your suite. If your tests still pass with the code broken, that mutant survived: proof your suite can't tell that version of the code from the real one.
color-string: 97.4% line coverage → but only an 87.2% mutation score. Sixty-two distinct changes to the code that the entire test suite would wave straight through. Every one is a place the tests run the code but don't actually check it.
One of those gaps was a real bug
Chasing the survivors, we captured what the code actually does across the input range — and found a value that was simply wrong. color-string converts percentage channels with Math.round(p * 2.55). But 2.55 has no exact representation in floating point (it's 2.5499999999999998…), so the product lands just below the half-integer:
color('rgb(50%, 50%, 50%)').rgb() → [127, 127, 127] # should be 128
color('rgb(90%, 90%, 90%)').rgb() → [229, …] # should be 230
Nobody's tests had ever noticed — in fact the suite pinned the wrong value in nine assertions. The bug had been golden-mastered. The one-line fix (* 255 / 100, which is exact) is now filed upstream as PR #92, with the exact-arithmetic proof that exactly two integer percents change and nothing else.
The fix isn't "write more tests." It's a better grader.
You can chase mutation survivors by hand forever. What actually closes the gap is changing where "correct" comes from. The tests in a repo are written by the same person who wrote the code, checked into the same commit, editable in the same breath — so they encode the author's beliefs about the code, including the wrong ones.
rederive replaces that with an oracle: input→output cases stamped by executing the real code, held out from whoever re-implements it, and mutation-hardened until a measured score says the held-out set actually kills the plausible wrong versions. Expected values are never hand-authored — so they can't inherit the author's blind spot. That's the difference between a suite that agrees with the code and a suite that independently constrains it.
Run a mutation tester once against your most-trusted module — Stryker (JS/.NET), mutmut or cosmic-ray (Python), PIT (Java). The score it returns — not your coverage number — is how much your green checks are actually worth. It will change how you read every dashboard after.
Coverage tells you the code ran. Mutation score tells you your tests would notice if it broke. Only one of those is a safety property — and for the code you didn't write, that gap is exactly where a supply-chain attack lives.
rederive is open source and live: npm i -g rederive, then rdv check any verified package.
Modernize legacy code without changing what it does. — you have a 2,000-line module, no tests, and nobody left who understands it. Keep the original running as the oracle, refactor into clean units, and prove the behavior is unchanged.
Lane Thompson — Founder, rederive · GitHub · rederive.ai