👋 Hey {{first_name|there}},
You shipped the AI feature. It demoed well, the room nodded, out it went. Here's the question nobody in that room could answer: Is it any good? Not "did the handful of examples we tried look fine." Good across the thousands of inputs you didn't pick, and still good on Tuesday, three weeks from now, when someone edits the prompt to fix one complaint.
Why this matters
Last issue's data leak (Lesson #61) was a kind of silent failure. Here's the general version: an AI feature can be wrong in a way that looks exactly like being right. Same confident tone, same clean formatting, no error thrown. A bad summary and a good summary are visually identical until you read both closely, and nobody reads thousands of them closely.
Regular software you test by asserting: input X gives output Y. A probabilistic feature won't cooperate. The same input can give different outputs, "correct" is a range instead of a value, and the thing you shipped on, a good demo, is the least trustworthy test there is. You chose those inputs. You watched a few of them. Of course, they looked fine.
So most teams ship on vibes and then maintain on vibes. Someone tweaks the prompt to fix a customer complaint, the complaint goes away, and five other cases quietly get worse. Nobody sees it, because nothing is watching. The feature doesn't so much improve or degrade as wander.
🧭 The shift
From: "We tried a bunch of inputs, and the outputs looked good."
To: "We have a graded set of cases that tells us, on every change, whether quality went up or down."
Trying things is not testing them. A test you can rerun, that grades itself, that tells you when a change made things worse, that's the line between having an opinion about your feature and having evidence.