Evaluation of UIs

Passive Evaluation

• customer reports
• logs
• surveys

Active Evaluation

• observations(video)
• scientific experiments
• usability testing

Predictive Evaluation

• heuristic evaluation

Heuristic Evaluation

Purpose:

• Find usability problems in a design

How:

• provide a set of usability principles (heuristics)
• a small group of evaluators (3-5) review the UI
  • independently verify compliance with heuristics

When:

• can be done on a fully developed user interface or paper sketches.

10 Usability heuristics

• Visibility
• Mapping
• Freedom
• Consistency
• Error prevention
• Recognition
• Flexibility
• Minimalism
• Error Recovery
• Help

Methodology

Evaluation Steps:

1. Pre-Assessment Training
   • If the system is for everyone ("walk-up-and-use") or the evaluators are experts in the domain, no assistance is required.
   • Otherwise, the evaluators should be provided with scenarios and a minimum of information about the domain (organization of knowledge) and anticipated processes.
2. Assessment
   • at least two passes for each evaluator
     • first to get an idea of the scope of the system
     • second to focus on specific items
   • each evaluator produces a list of issues
     • explain why by referring to the heuristics
     • be specific and list each problem separately
     • indicate where the problem is
       • unique location in the user interface
       • two or more places in the UI to compare
       • overall structure of the UI
       • something is missing
3. Prioritization
   • severity combines:
     • frequency (one or more places)
     • impact (aesthetic or functional)
     • persistence of the problem
4. Debriefing
   • meeting between the evaluators and the design team
   • discussion about:
     • list of problems found
     • severity perceived by each of the evaluators
     • possibility of improvements
     • estimation of resource allocation to solve problems