Engineer’s brain.
Evaluator’s job.
I moved from physical-system reliability into AI evaluation without changing the core habit: separate symptoms from causes, test the rule, and verify what actually works.
My earlier career was built around physical systems: non-destructive testing, reliability work, vibration analysis, HVAC and chiller systems, VFDs, data, and technical reporting. That work trains a particular reflex. When something looks wrong, do not immediately reach for a story. Find the signal. Identify the failure path. Verify the diagnosis.
I now apply that same reflex to AI systems. A model answer is only one observable. The real system includes the prompt, the rubric, the data, the annotation, the evaluator, the interface, the workflow, and whatever action happens downstream.
That is why I am drawn to multimodal and robotics evaluation. The work forces you to decide what the evidence actually supports, distinguish outcome quality from execution quality, and preserve uncertainty instead of laundering it into a confident label.
How I work
I tend to compress messy systems into a small number of failure buckets, then test which bucket survives contact with the evidence. I care about acceptance criteria, ownership, ground truth, false positives and false negatives, and whether a metric is measuring the thing people claim it measures.
The short version: I like systems that can explain why they should be trusted.