🔗 Paper on which the article is based

written by Mohammed Bilal Ahmed and Nicolas Perez

✏️ Introduction

The authors of the paper focused on the heuristic analysis of user interfaces. They wanted to test and see what could be the best way to identify problems in a user interface. Although heuristic analysis is not the only way. It is by far the most practical when compared to other options.

What did the authors assess and find?

To test its effectiveness the authors of the paper conducted 4 experiments with different parameters. Each experiment had either a different set of evaluators or a different set of software that the team of evaluators had to assess. The first and the second experiment did not have live working software to test. Furthermore, these two experiments had different sets of evaluators. The first experiment had a team of university students with some educational background in heuristics and the second experiment was done by working industry professionals. However, the last two experiments had the same sets of evaluators but different live working software. Through these experiments, the author was able to study the effectiveness of a heuristic analysis of a user interface.

Limitations of the paper

The authors found that after a certain point. Adding more evaluators did not have a significant impact. Although this is dependent on the complexity of the software as well. In any case, none of the evaluators were able to find 100% of the problem. In Fact, most problems found by the team of evaluators were in experiment one and that too was only 51%. Another interesting point that was highlighted in the experiment was the fact that heuristic analysis works best when one aggregates a team’s results. This also ties in with the idea of what one defines as a “usability” problem to begin with. Different evaluators may have different ideas about what constitutes a usability problem. Aggregating results in such cases helps one see issues in a more holistic light. This then leads to the limitations of this paper. The paper does have some limitations in the demographics that it reaches. The two demographics that are covered are college students who recently had a lecture on evaluation heuristics and industry professionals. The paper itself points to this limitation by making a note that usability experts would be better at identifying problems through heuristic analysis. In addition, three of the four tests in the paper were done on only one of the covered demographics: college students. However, the data from the industry professionals fits the same trends as the data from students, which points to there being little correlation between the demographics of the evaluators and the conclusions on aggregate analysis. More data should be collected to be able to state that for sure.

The contributions of the paper to the scientific community

While the ideas the paper covers may seem rather simple, the paper presents a large positive contribution to both the academic community and the industry as a whole. Everything in life is built off other things, and this paper functions as a foundation to work off of. It uses experimental data to show just how many resources should be allocated to test the functionality of user interfaces without wasting both time and money. It also provides recommendations on how best to analyze user interfaces and find problems.