Files
learn-rust/learning-records/0006-green-suite-blind-to-the-cli-half.md
T

3.2 KiB

A green suite says nothing about the surface it cannot reach

The 0008 drill landed 46/46 with the shipped spec unedited. The library half is genuinely produced: tally<T, K, F> written from its signature and used at three T/K pairs, Store::load as one collect::<Result<Vec<Task>, TaskError>>()?, remove_completed on Vec::retain, titles_with as a filter/map/collect chain, count_by_priority a one-line delegate. Nothing in src/store.rs or src/stats.rs needed correcting beyond a commented-out loop left behind.

The CLI half of the same drill missed three of its requirements, and every one of them was invisible to those 46 tests:

  • stats printed high / low / medium, because main.rs looped over [Priority::High, Low, Medium]. The spec asked for high, medium, low.
  • clear printed nothing, discarding the usize that remove_completed returns. Expected cleared 1 completed.
  • Untested-because-unreachable, so also unfixed: the in-progress arm of Status::parse, the Display line format, and Command::parse's case folding.

Evidence

cargo test in tasks/: 17 + 7 + 8 + 14 = 46 passed, 0 failed. Every one of those tests lives in tests/ and therefore imports the library; run lives in src/main.rs, which a binary crate does not export, so no test in the workspace can call it. The three defects sit entirely inside run.

Measured rather than argued: six one-line mutations planted in a copy of the crate (lessons/0009-mutants.sh) and the user's suite run against each. Result before 0009: 0 killed, 3 survived, 3 skipped. The same six against a reference implementation with 13 more tests: 6 killed, 0 survived. The suite's blindness is not a matter of degree — it is a whole surface.

Implications

  • This is the third repeat of LR-0003's finding, and the first time the cause is structural rather than a missing check. 0003 lost the stderr/exit-1 contract, 0006 lost the ?/From collapse, 0008 lost the CLI output shape. A drill step whose result no test can observe does not land, however clearly the prose states it.
  • The fix is architectural, so it is the drill. 0009 moves run into src/cli.rs and gives it out: &mut impl Write. That is not a lesson about tests bolted onto a refactor; the refactor is the only way the tests can exist, which is exactly the book's argument for a thin main.rs.
  • Grade a test suite by planted bugs, not by test count. The user has now run 46 tests and would reasonably infer the crate is well covered. Six mutations refute that in four seconds and give a finishing condition (6 killed, 0 survived) that counting cannot.
  • Ship no new spec file for 0009. Every earlier lesson handed over assert_eq!s to satisfy; the skill being built here is writing them, so the only deliverable is the mutation script. The drill names the behaviour to pin in prose — per the rule in NOTES line 62 — and leaves the assertions to the user.
  • Watch for on the next read: whether the assertions are exact (assert_eq! on the whole printed string) or hedged (assert!(out.contains("cleared"))). The hedged form passes the mutants that matter least and is the likeliest way this drill goes green while staying blind.