Process integrity / note 001
The output is right. The system can still be wrong.
Accuracy is a result. Reliability is a property of the process that keeps producing results.
If a model lands on the correct answer because of a lucky guess, an accidental cue, contaminated context, or a broken rubric that happens to reward the output, you do not have a reliable system. You have a successful sample.
The evaluation question is therefore larger than “Was this answer correct?” It is: what evidence did the model use, what constraints shaped the answer, what would happen on the nearby hard case, and would another evaluator reach the same judgment under the same rule?
That is the difference between scoring outputs and testing a system.
Verification economics / note 002
AI made generation cheap. It did not make truth cheap.
The bottleneck moved.
A model can produce ten drafts, fifty images, or a thousand classifications faster than a human can inspect them. That is economically useful — but it creates a second-order problem: plausible output arrives faster than validated output.
The scarce resource becomes verification. Evidence gathering. Edge-case review. Cross-checking. Ground truth. Calibration. Knowing when the system is uncertain and when the reviewer should intervene.
If a workflow celebrates generation speed without pricing the verification burden, the “productivity gain” may simply be deferred QA debt.
Failure taxonomy / note 003
Stop putting every defect in the “model” bucket.
A bad result can be produced by several layers that look identical from the outside.
The model may lack the capability. The prompt may be ambiguous. The rubric may encode the wrong standard. The annotation may be noisy. The reviewer may be inconsistent. The workflow may be asking a human to validate something the interface never actually shows.
Those are different failure modes with different fixes.
Calling all of them “the AI got it wrong” is convenient, but operationally useless. If you cannot identify the layer that failed, you cannot know whether your intervention improved anything.
Systems / note 004
“AI-capable” hardware is not the same as an intelligent system.
The intelligence may be in the workflow surrounding the device.
A camera, processor, robot, vehicle, or edge device can contain impressive hardware and still produce mediocre outcomes if sensing, orchestration, fallback logic, data quality, and validation are weak.
The inverse is also true: ordinary hardware can become powerful when the surrounding workflow is designed well. The system is not the chip. The system is the full loop from observation to decision to action to verification.
That is the layer I find most interesting.