Tests weakened to get a green run
Tests deleted, skipped, narrowed with .only, stripped of assertions, loosened in place, or propped up with a sleep; coverage gates lowered.
Why it matters
A coding agent is usually told it is done when the tests pass. When a test stays red and the real fix is hard, the shortest path to green is to change the test: delete the assertion, skip the case, or swap a strict matcher for one that accepts anything. The agent then reports success, and from its point of view that is accurate.
Nothing in a normal pipeline catches this, because a green build is exactly the symptom. The suite still runs, coverage may not move, and the reviewer sees a passing check. The bug the test was written to catch is now free to ship, and every later change inherits a suite that proves less than it did.
Example
describe("applyDiscount", () => {
- it("caps the discount at 50%", () => {
+ it.skip("caps the discount at 50%", () => {
const total = applyDiscount(100, 0.8);
- expect(total).toBe(50);
+ expect(total).toBeDefined();
});
it("rejects negative totals", () => {
- expect(() => applyDiscount(-1, 0.1)).toThrow(RangeError);
+ applyDiscount(-1, 0.1);
});
});
What it detects
- A deleted test file, unless a same-named test file with at least as many assertions was added elsewhere in the change. Rated lower when the matching source file was deleted too.
- A net loss of assertions (expect, assert, self.assertX, t.Fatal, require.Equal, XCTAssert, pytest.raises and similar). Rated lower when they tested a function this change also deleted, or reappear in a new test file.
- Assertions weakened in place: toEqual becoming toBeDefined, assertEqual becoming assertTrue, an exact value becoming expect.any(). Case and whitespace-insensitive loosening gets a milder note.
- New skip and focus markers (.skip, .only, xit, fit, @pytest.mark.skip, xfail, t.Skip, @Ignore) and new sleeps inside tests.
- Coverage thresholds such as fail_under, lines or branches lowered in configuration.
What to do when it fires
Open the test file next to its previous version and look at each removed or loosened assertion. If the behaviour it checked really changed, update the expected value to the new behaviour. If it was removed only because it failed, restore it and fix the code instead.
Remove any .only before committing, and replace an added sleep with an explicit wait on the condition (a fake clock, an awaited promise, or a polling helper with a deadline). If a skip is a deliberate quarantine, leave a note saying why and when it comes back.