Evaluating AI: Why One Output Is an Example, Not a Verdict

A single output from a generative AI system cannot establish how well that system performs. The same prompt, submitted to the same model, can return a policy-compliant answer, omit an exception, or confidently promise a refund the customer was never owed. Each of those responses is real. None, taken alone, represents the system.

That framing comes from a recent Nielsen Norman Group article on AI evaluation, which addresses designers and researchers who specify or review generative features. The article draws a line between deterministic software, in which one successful execution signals reliable behavior, and nondeterministic systems, which can return different outputs for identical inputs.

Teams often evaluate AI by running it once, reading the result, and concluding what the technology can do. The article attributes this habit to decades of conventional engineering practice: a feature that works once will work every time. Generative systems offer no such guarantee.

"One good output demonstrates that a system can perform a task," the article states. "It does not show how often or how reliably the system will do so."

What does the change require?

Practitioners must treat each output as one sample drawn from a distribution, not as evidence of capability. The article recommends three movements in any evaluation of a generative feature:

• Run multiple representative inputs that cover the real range of user cases, including edge conditions and adversarial prompts.
• Repeat each run several times so variance surfaces rather than a single answer.
• Summarize results with confidence intervals rather than a binary pass or fail.

A confidence interval matters because it quantifies the spread of outputs a system produces. A model that returns the correct policy answer on most runs and the wrong answer on others carries a different risk profile than a model that returns the correct answer on every run.

Teams that report a single successful interaction will record identical results for both models, and will therefore misjudge the risk each one carries.

The article's central warning extends beyond product teams. A researcher who publishes a single impressive AI example invites the same misreading as a developer who ships after a single test. Any group communicating AI capability, whether to executives, regulators, or end users, faces an obligation to publish the evaluation protocol alongside the headline result.

Showing the prompt set, the run count, and the interval matters as much as showing the answer.

As generative features move from prototype to production across customer service, search, and content tooling, evaluation discipline will increasingly separate teams that can defend their systems from teams that cannot. The article closes on one demand: stop treating one output as an answer and start treating it as one data point.