What Is Usability Testing?
Usability testing evaluates how easy and intuitive a product is to use by observing real users as they attempt to complete tasks. Unlike functional testing (which asks “does it work?”), usability testing asks “can real people figure out how to use it?”
A feature that works perfectly from a technical perspective can still be a disaster if users cannot find it, do not understand it, or make constant errors while using it. Usability testing catches these problems before they reach production and frustrate your users.
As a QA engineer, you may not design the UX, but you are often the last line of defense before users encounter the product. Understanding usability principles makes you a far more effective tester.
Nielsen’s 10 Usability Heuristics
Jakob Nielsen’s 10 heuristics are the foundation of usability evaluation. Published in 1994, they remain the gold standard for heuristic evaluation today.
1. Visibility of System Status
The system should keep users informed about what is happening through appropriate feedback within reasonable time.
Examples: Loading spinners, progress bars, “Your order has been placed” confirmations, “Saving…” indicators.
What to test: Does the system provide feedback after every user action? Are loading states visible? Do error messages appear promptly?
2. Match Between System and the Real World
The system should speak the user’s language, using familiar words, phrases, and concepts rather than technical jargon.
Examples: Using “Shopping Cart” instead of “Item Queue,” using calendar date pickers instead of requiring YYYY-MM-DD format.
What to test: Is the language user-friendly? Are icons intuitive? Does the information flow follow a natural, logical order?
3. User Control and Freedom
Users often perform actions by mistake. They need a clearly marked “emergency exit” to leave the unwanted state.
Examples: Undo/redo functionality, “Cancel” buttons, “Go back” navigation, “Are you sure?” confirmation dialogs for destructive actions.
What to test: Can users undo actions? Can they navigate back? Are destructive actions reversible or at least confirmed?
4. Consistency and Standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform and industry conventions.
Examples: The save icon is always a floppy disk, links are always underlined or colored, the logo always navigates to the homepage.
What to test: Are UI elements consistent across pages? Do buttons with the same function look the same? Does the app follow platform conventions?
5. Error Prevention
Even better than good error messages is a careful design that prevents errors from occurring in the first place.
Examples: Disabling the “Submit” button until all required fields are filled, calendar pickers that prevent selecting invalid dates, auto-complete suggestions.
What to test: Can users enter invalid data? Are there safeguards against common mistakes? Does the form validate input before submission?
6. Recognition Rather Than Recall
Minimize the user’s memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialog to another.
Examples: Recently viewed items, auto-complete in search, visible navigation breadcrumbs, tooltips on icons.
What to test: Does the user need to memorize anything? Are instructions visible when needed? Are options visible rather than hidden?
7. Flexibility and Efficiency of Use
Accelerators — unseen by the novice user — may speed up interaction for expert users. The system should cater to both inexperienced and experienced users.
Examples: Keyboard shortcuts, customizable dashboards, saved filters, “Quick actions” menus, drag-and-drop alongside form-based input.
What to test: Can power users accomplish tasks efficiently? Are there shortcuts? Can the interface be customized?
8. Aesthetic and Minimalist Design
Dialogs should not contain information that is irrelevant or rarely needed. Every extra unit of information competes with relevant information and diminishes its visibility.
What to test: Is the interface cluttered? Is important information easy to find? Are secondary actions hidden appropriately?
9. Help Users Recognize, Diagnose, and Recover from Errors
Error messages should be expressed in plain language (no error codes), precisely indicate the problem, and constructively suggest a solution.
Examples: “The email address you entered is not valid. Please use the format name@domain.com” instead of “Error 422: Validation failed.”
What to test: Are error messages helpful? Do they explain what went wrong and how to fix it? Are they in plain language?
10. Help and Documentation
Even though it is better if the system can be used without documentation, it may be necessary to provide help. Any such information should be easy to search, focused on the user’s task, and list concrete steps.
What to test: Is there contextual help? Is the documentation searchable? Are common tasks covered with step-by-step guides?
Usability Testing Methods
Moderated Testing
A facilitator guides participants through tasks while observing and asking follow-up questions.
Best for: Discovering the “why” behind usability issues, testing complex workflows, gathering qualitative insights.
Process: Recruit 5-8 participants, prepare a task script, observe in person or via video call, take notes on struggles, ask follow-up questions.
Unmoderated Remote Testing
Participants complete tasks independently using a testing platform that records their screen and voice.
Best for: Testing with more participants (20-100+), geographic diversity, quick turnaround. Less depth but more scale.
Tools: Maze, UserTesting, Lookback, UsabilityHub.
A/B Testing
Compare two versions of a design by showing each to a different group of users and measuring which performs better.
Best for: Optimizing specific elements (button color, layout, copy), data-driven decisions with statistical significance.
Card Sorting
Participants organize topics into categories that make sense to them, revealing how users expect information to be structured.
Best for: Designing navigation, information architecture, menu structure.
Tree Testing
The reverse of card sorting — give users a task and see if they can find it in your proposed navigation structure (without the actual UI).
Best for: Validating navigation structure before building the UI.
Usability Metrics
| Metric | What It Measures | How to Collect |
|---|---|---|
| Task Success Rate | % of participants who complete a task successfully | Observation during testing |
| Time on Task | How long it takes to complete a task | Timer during testing |
| Error Rate | Number of errors per task | Observation counting mistakes |
| SUS Score | Overall perceived usability (0-100) | Post-test 10-question survey |
| Net Promoter Score (NPS) | Likelihood to recommend (0-10) | Post-test single question |
| Task-Level Satisfaction | Satisfaction per task (1-7 scale) | After each task in testing |
The System Usability Scale (SUS)
SUS is a 10-question survey scored from 0-100. It is the most widely used standardized usability questionnaire.
Scoring interpretation:
- 80-100: Excellent (users love it)
- 68-79: Good (above average)
- 50-67: OK (room for improvement)
- Below 50: Poor (significant usability issues)
Tools for Usability Testing
| Tool | Type | Best For |
|---|---|---|
| Maze | Unmoderated remote | Rapid prototype testing |
| UserTesting | Moderated + unmoderated | Comprehensive studies |
| Hotjar | Heatmaps + recordings | Understanding user behavior on live sites |
| Lookback | Moderated remote | Live interview + screen recording |
| Optimal Workshop | Card sorting + tree testing | Information architecture |
| Figma | Prototype testing | Testing designs before development |
Exercise: Heuristic Evaluation of a Website
Perform a heuristic evaluation of a website (choose any public website or use a project you are working on).
Task
Evaluate the website against all 10 of Nielsen’s heuristics. For each heuristic, document:
- A specific example where the site follows the heuristic well
- A specific example where the site violates the heuristic (if applicable)
- A severity rating (1=cosmetic, 2=minor, 3=major, 4=critical)
- A recommendation for improvement
Hint: Evaluation Template
Use this format for each heuristic:
| # | Heuristic | Good Example | Violation | Severity | Recommendation |
|---|---|---|---|---|---|
| 1 | Visibility of system status | … | … | 1-4 | … |
| 2 | Match with real world | … | … | 1-4 | … |
Focus on real user scenarios. For example, try completing a purchase, signing up, searching for something, or recovering from an error.
Solution: Example Evaluation of an E-commerce Site
| # | Heuristic | Good Example | Violation | Severity | Recommendation |
|---|---|---|---|---|---|
| 1 | Visibility of system status | Progress bar during checkout shows step 2/4 | No feedback after clicking “Add to Cart” — users click multiple times | 3 | Add a visual confirmation (toast notification or cart icon animation) when an item is added |
| 2 | Match with real world | Product categories use familiar terms (Shoes, Electronics) | Shipping option labeled “Expedited Fulfillment” instead of “Fast Shipping” | 2 | Use plain language: “Fast Shipping (2-3 days)” |
| 3 | User control and freedom | “Continue Shopping” button on cart page | No undo after removing an item from cart | 3 | Add “Undo” option for 10 seconds after item removal |
| 4 | Consistency and standards | All primary buttons are blue, secondary are gray | “Buy Now” is green on product pages but blue in the cart | 2 | Standardize CTA button colors across the site |
| 5 | Error prevention | Date picker prevents selecting past dates for delivery | Email field accepts “test@test” without proper validation | 3 | Add real-time email format validation |
| 6 | Recognition rather than recall | Recently viewed products shown on homepage | Checkout requires re-entering shipping address even for logged-in users | 3 | Auto-fill saved addresses for returning users |
| 7 | Flexibility and efficiency | Keyboard shortcut (/) for search | No way to reorder a previous order | 2 | Add “Buy Again” feature to order history |
| 8 | Aesthetic and minimalist design | Clean product pages with prominent images | Homepage has 3 banners, 4 promotional sections, and a popup — overwhelming | 2 | Reduce homepage to one hero banner and one featured section |
| 9 | Error messages | “This email is already registered. Log in instead?” is helpful | “Payment failed” with no explanation or retry option | 4 | Show specific reason (expired card, insufficient funds) and offer retry |
| 10 | Help and documentation | FAQ section covers common shipping questions | No contextual help during checkout (e.g., “What is CVV?”) | 2 | Add tooltip icons next to form fields explaining unfamiliar terms |
Summary: 2 critical issues (no add-to-cart feedback, unhelpful payment error), 3 major issues, 3 minor issues, and 2 cosmetic issues. Priority: Fix the payment error message and add-to-cart feedback first.
Pro Tips
- 5 Users Find 85% of Issues: Research by Nielsen shows that testing with just 5 users uncovers approximately 85% of usability problems. You do not need large sample sizes for qualitative usability testing.
- Think-Aloud Protocol: Ask participants to verbalize their thoughts as they complete tasks. “I’m looking for the search bar… I expected it to be in the top right… Oh, it’s hidden behind this hamburger menu.” This reveals thought processes you cannot observe visually.
- Test Early with Prototypes: You do not need a working product to test usability. Paper prototypes, wireframes, and Figma prototypes can reveal major usability issues before a single line of code is written.
- QA as Usability Advocate: As a QA engineer, you interact with the product more than anyone. Document usability issues you encounter during functional testing, even if they are not “bugs” in the traditional sense.
- Competitive Benchmarking: Perform heuristic evaluations on competitor products to identify usability patterns that users already expect. This is especially useful when testing a new product in an established market.