Heuristic Review

Number of Participants	Facilitators	Category	Duration	Level of Difficulty
Groups of 4-5	1 Moderator	Test Phase	2 Hours	Moderate

Description

The heuristic review is a Heuristic review is an analytical method. A group of experts examines user interfaces for possible usability problems using a list of proven principles, also called heuristics. After a short introduction to the knowledge domain, the future user group and their future tasks, a small group of evaluators examine the system. They collect potential problems that future users might have, assign them to specific heuristics and prioritize the problems according to their severity. Optionally, solutions for the problems found can be searched for, but this is not a mandatory part of the method. The goal of the method is:

• Detect as many usability problems as possible with as little effort as possible
• Prioritization of problems to determine an order of correction them
• Detect & fix as many “obvious” problems as possible before user tests are conducted
• This way less real users are “consumed” in total

For more information about the method, please refer to the slides of the PowerPoint presentation. Here you will find more detailed information and a complete insight into the method.

Materials

Preparation

Before the implementation some things should be prepared. These preparations are differentiated in the following table for a digital execution and a presence event.

Digital/Remote

• Before the workshop, the workshop leader should review all materials and talk through the presentation several times.
• It may also be advisable for untrained people to test the entire process in advance.
• The moderator should ensure a solid internet connection.
• The moderator should choose a quiet room without noise.
• The selected video conferencing tool should be tested before the implementation. It is best to try out all functions between 2 different internet networks. This way you can familiarize yourself with the tool and reduce the likelihood of technical problems.
• The access link for the participants should be tested before.
• The moderator should check his/her microphone and video before the event so that it can be clearly understood by the participants.
• The moderator should consider beforehand how the participants should use microphone and video. If the number of participants is large, he or she should recommend that they only switch on their microphone when they are speaking. Otherwise there will probably be some background noise.
• Although mandatory use of the video function can be a dissuasive factor, it increases the willingness to participate, especially in a group that is unfamiliar with each other, and creates a slightly less anonymous environment.

Physical/Presence

• Before the workshop, the workshop leader should review all materials and talk through the presentation several times.
• It may also be advisable for untrained people to test the entire process in advance.
• The moderator must find a room for the workshop and inform the participants about it in time, so that they can plan their arrival.
• Date and time for the workshop have to be fixed.
• Technical components such as beamer and laptop should be tested before the workshop and set up before the participants arrive.
• The room should be prepared for group setting. It is best to set up group tables and equip them with the workshop materials.
• Before the presentation, the moderator should consider how he or she wants to put the groups together. If the participants already know each other, it may be advisable to form random groups and distribute name cards. This can increase the diversity of the individual groups.
• The required material should be printed out in sufficient quantity and distributed on the tables.
• It is best to make 2 or 3 more prints than necessary of everything. That way you are well prepared for eventualities.

Step-by-Step Instructions

1. First, the workshop leader should welcome the participants and briefly introduce himself and the topic of the workshop.
2. He should then present the schedule of the following workshop to the participants so that they know what to expect. It is recommendable to present the agenda visibly during the whole workshop so that the participants know where they are at all times. In addition, the agenda also provides a brief summary of the workshop.
3. You can then begin with the PowerPoint presentation. First, the theoretical background of the heuristic review is explained on slides 3 to 18.
   1. After it, open questions should be inquired about. All participants have the opportunity to fully understand the theory of the method.
4. Then the practical part is introduced by explaining the exercise. This is done with the slides 19 to 21.
   1. Before starting the exercise, the moderator can explicitly highlight the Handout. This allows the participants to eliminate any uncertainties during the practice.
   2. Here too, the workshop leader should clarify any open questions
5. The moderator presents phase 1 and introduces the participants to the scenario and the target group. He thus completes the “pre-evaluation” training of Slide 23 of the PowerPoint presentation.
6. Then the group work will begin. The moderator leaves the field to the participants, but is ready for questions. Open slide 24.
   1. In a digital workshop the participants are sent to the different breakout sessions
7. After 25 minutes, the workshop leader can briefly inform the groups that they should be finished with Phase 2 and go to slide 25.
8. After 10 minutes, the workshop leader can briefly inform the groups that they should be finished with Phase 3 and git to slide 26.
9. After 20 minutes, the workshop leader can briefly inform the groups that they should be finished with Phase 4 and go to slide 27.
10. After the 60 minutes, the moderator should bring the participants back from the group work. It is useful not to break off exactly after 60 minutes, but to give the participants 1 or 2 minutes to complete the work, if necessary.
11. The moderator then initiates the presentation of the results and discussion. Each group has the opportunity to present their results and verbalize their experiences with the method. Since the latter can be difficult for non-routine workshop participants, it is helpful to prepare specific questions in advance that can initiate a discussion.
    1. These could be, for example:
    2. How did you feel about the method?
    3. When could you imagine using this method?
    4. What were the biggest challenges?
    5. What surprised you?
12. Finally, the moderator has the opportunity to get feedback for his workshop. He can do this either as an open discussion or by using more anonymous questionnaires/sheets. The first carries the risk that the participants do not dare or do not want to take the time for it. Anonymous variants have the advantage that everyone can express themselves, but on the other hand, the moderator cannot ask questions if anything is unclear. The workshop leader must decide which method promises the most advantages for him and this workshop.

Remarks, Tips, Limitations

It is advisable to use a website as example for the exercise, where the workshop participants belong approximately to the potential user group. This make it easier for them to act from the perspective of future users and to familiarize themselves with the knowledge domain.
Especially the assignment of problems to heuristics can be difficult in the beginning. Therefore the moderator should inform himself well about them, so that he can react to questions.
Even if it seems unnecessary in advance, it is recommendable for less experienced workshop leaders to test the entire procedure before the actual workshop. It is best to do this with another person who is not familiar with the method. This person will be better able to identify possible problems.
You should always plan a time buffer for the workshop. This way you don’t have to stop the workshop if the schedule is delayed. Especially the discussion at the end can quickly cost more time than planned. However, the exchange is very supportive for all participants and should therefore be brought to a satisfactory conclusion.

Strengths: Low cost and time saving with less equipment needed. Can uncover problems users may not even notice. Does not require much experience.
Weaknesses: Tends to only identify minor problems. Quality of results depends on participant’s expertise. Reviewers aren’t real users! So there still might be problems uncovered.

References

e-pixler NEW MEDIA GmbH. (n.d.). Was ist Heuristische Evaluation?. Retrieved from https://www.e-pixler.de/glossar/usability/heuristische-evaluation.html
Heuristische Evaluation. Retrieved from https://de.ryte.com/wiki/Heuristische_Evaluation
Moser, C. (2012). User Experience Design. Mit erlebniszentrierter Softwareentwicklung zu Produkten, die begeistern. Heidelberg: Springer Vieweg
msg systems ag. (n.d.). Heuristische Evaluation. Retrieved from https://www.user-experience-methods.com/evaluation/heuristic-evaluation.html
Sarodnick, F. & Brau, H. (2016). Methoden der Usability Evaluation: Wissenschaftliche Grundlagen und praktische Anwendung. 3., unveränderte Auflage. Bern: Verlag Hans Huber, Hogrefe AG
Schneider, W. (2018). Grundsätze der Dialoggestaltung nach DIN EN ISO 9241-110. Retrieved from https://www.ergo-online.de/ergonomie-und-gesundheit/software/dialoggestaltung/artikel/grundsaetze-der-diashylogshygestaltung-nach-din-en-iso-9241-110/grundsaetze-der-dialoggestaltung-nach-din-en-iso-9241-110/

Contributed by Johanna Langendorf.