Heuristic Evaluation of User Interfaces

This system was called “Mantel” as an abbreviation of our hypothetical telephone many,. Manhattan Telephone (neither the company nor the system has any single screen and a :few system messages that the specification could be contained on a single page. The design document used for this experiment is reprinted as an appendix to [Moloch and Nielsen 19901 which also gives a complete list and in-depth explanation of the 30 known usability problems in the Mantel design.

The evaluators were readers of the Danish Computerized magazine where our design was printed as an exercise in a contest. 77 solutions were mailed in, mostly written by industrial computer professionals. Our main reason for conducting this experiment was to ensure that we had data from real computer professionals and not Just from students. We should note that these evaluators did not have the (potential) benefit of having attended our lecture on the usability heuristics. Experiments 3 and 4: Two Voice Response Systems: “Savings” and “Transport” Tab 2.

Summary outpour experiments. Experiment 1: Delta Experiments 3 and 4 were conducted to get data from heuristic evaluations of “live” systems (as opposed to the specification-only designs in experiments 1 and 2). Both experiments were done with the same. Roof of 34 computer science students as evaluators. Again, the students were taking a course in user interface design and were given a lecture on our usability heuristics, but there was no overlap between the group of evaluators in these experiments and the group from experiment 1 .

Both interfaces were “voice response” systems where users would dial up an information system from a touch tone telephone and interact with the system by pushing buttons on the 12-key keypad. The first seem was run by a large Savings Union to give their customers information about their account balance, current foreign currency exchange rates, etc. This interface is refer& to as the “Savings” De- Experiment 1 tested the user interlace to the Danish videotaped system, Delta.

The evaluators were given a set of ten screen dumps from the general search system and from the Scandinavian Airlines (ASS) subsystem. This means that the evaluators did not have accessed a “live” system, but in 250 CHI mending’s sign in this article and it contained a total of 48 known usability problems. The second system was used by the municipal public transportation company in Copenhagen to provide commuters with information about bus routes. This interface is referred to as the “Transport” design and had a total of 34 known usability problems.

Them were four usability problems which were related to inconsistency across the two voice response systems, Since the two systems are aimed at the same user population in the form of the average citizen and since they are accessed through the same terminal equipment, it would improve their collective usability if they both used the same conventions. Unfortunately there are differences, such as the use of the squarer key. In the Savings system, it is an end-of-command nomad in the Transport system which does not use an end-of-command key at ah.

The four shared inconsistency problems have been included in the count of usability problems for both systems. Since the same evaluators were used for both voice response experiments, we can compare the performance of the individual evaluators. In this comparison, we have excluded the four consistency problems discussed above which are shared among the two systems. A regression analysis of the two sets of evaluations is shown in Figure.