Nielsen Norman Group Reworks Usability 101 for the AI Era

Usability redesigns improved targeted UX metrics by 75% on average across projects analyzed by the Nielsen Norman Group in 2020 — a figure the firm re-centers in its fully rewritten introduction to usability, which replaces Jakob Nielsen's original 2012 article. The 2020 number is not a direct ROI estimate, and results varied widely across projects, the firm notes. Still, it grounds the guide's central claim: measured improvements in usability translate into measurable gains in sales and signups.

What defines usability in the new guide?

The guide defines usability as how easily people complete a task using an interface, broken into five measurable components:

• Learnability — how fast people learn basic tasks.
• Efficiency — how quickly they perform tasks after learning the design.
• Memorability — how easily they regain proficiency after a break.
• Errors — how many slips and mistakes occur, and how easily users recover.
• Satisfaction — how pleasant the design is to use.

The weight of each component depends on the product. For a tax-filing website used once a year, learnability dominates. For a CAD application an architect uses for hours daily, efficiency matters most, because the architect learns the system once and then runs thousands of sessions through it.

Usability is never absolute, the guide argues. A 10-column comparison table works on a large monitor but fails on a phone. A page legible to a 20-year-old may defeat a 60-year-old. Every usability claim is therefore relative: usable for whom, for which task, under which conditions.

How does usability relate to utility and UX?

A product needs both utility and usability to be useful. An easy-to-use product that does not do what users want has no value, and neither does a capable product too hard to operate. User experience is broader still, covering every interaction with a company — customer support, pricing, post-purchase service. Usability is one quality of good UX, alongside utility, accessibility and trustworthiness.

Poor usability carries direct costs. When people cannot figure out an offer, find a product or complete a form, they leave for a competitor. When they cannot leave — government services or employer-mandated tools — they spend more time, make more errors and need more training and support. The organization pays those costs.

What does the testing method look like?

Usability testing remains the core method: recruit representative users, assign representative tasks, observe behavior. For qualitative testing aimed at finding and fixing problems, five users often surface most major issues. The firm recommends iterative design — test, fix, test again — starting with a simple prototype before implementation.

Quantitative testing measures usability through four metrics: success rate, time on task, error count, and satisfaction ratings collected with instruments such as the Single Ease Question (SEQ) or the System Usability Scale (SUS). The firm recommends approximately 40 participants for reasonably precise estimates. The metrics map loosely to the five components; time on task can reflect efficiency, learnability or memorability depending on when it is measured.

No laboratory is required. Most studies today run remotely, moderated over video or unmoderated through platforms recording screen and voice. In-person testing still matters for kiosks, physical products and gesture-driven mobile apps. A quiet room with a door suffices. Anyone can run a test, the guide cautions, but leading questions or an over-helpful facilitator produce misleading results.

What changes with AI?

The rewrite's sharpest departure from 2012 concerns AI. First, AI may accelerate evaluation — preparing studies, analyzing data, or automatically locating usability problems — and experimental approaches include synthetic users, digital twins and AI interviewers. Their reliability remains an open research question, and synthetic users should not replace real ones.

Second, generative AI breaks a core assumption: the same request can return equally accurate answers, one coherent and well formatted, another poorly explained and hard to scan. Usability can now vary between interactions for the same user and task, and generative UIs that build layouts dynamically will extend that variability to controls. Evaluation methods must adapt.

Third, AI agents create new questions: how people delegate tasks, understand agent behavior, and regain control. Finally, the guide separates human usability from machine operability — an agent can extract a phone number from low-contrast HTML a human can barely read, while a human-friendly site may baffle an agent.

The underlying principle survives every change: evidence of how people actually interact with a design must drive its improvement.