In the last months, I conducted a few usability studies and upon reflecting on these I decided to share my experience as it might be helpful to anyone starting on usability. This article attemps at summarizing my experience and thoughts on usability experiments.

When trying to start a usability study or experiment, the practitioner or researcher must answer some initial questions about their future work.

Regarding your research, in general, the most important question to answer  is “What is my motivation or why I am doing it?”. In a few words, as a  researcher, you must not only formulate your research question but also, its answer.

Research methods are here to help you create and solve a new question on usability, user experience and also, on human-computer interaction.

Motivation

Every time I start a usability experiment or study, I ask myself  the following questions:

• Am I asking the right questions? 
• Am I understanding the problem?
• Have I identified the data that can answer the questions?
• Have I selected the right research methods for getting that data?

These questions help me when trying to define the usability study. Each question is a key point when determining the contents of the study.

In my opinion, the last points are valid to any usability study, and also, for HCI in general.

Purposes

Kjeldskov [1] and Kan [2] write about the purposes when starting on HCI Mobile research. In my opinion, those are purposes on general HCI research. In this article, I want to focus on the business side of the Usabilities studies, based on my practical experience.

In general, the practitioners and researchers ask themselves the following questions when evaluating a product:

• How difficult is to use this product?
• Do people use efficiently the product?
• Is the terminology easy to understand?
• Are the people capable of finding the information or functionality there want?

Methods

Usability studies is a technique used in the user-centered interaction design. Its purpose is to test a product in users in order to  identify the issues of a product. A complete list of usability methods can be found on the Nielsen-Normal Group [8], but for now, I only want to list those methods I have used successfully and, in my experience, every researcher must know. The most popular techniques are:

• Expert review
• Automated Usability Evaluation
• Remote usability testing
• Questionnaires and Interviews:
• Hallway testing
• Paper Prototype Testing

Expert review is the cheapest and quickest evaluation method. This method consists of an evaluation done by one (or a few) person  who is an expert on the field evaluation the product. More information on this type of evaluation can be found on the Nielsen Normal Group website [9]. 

Automated Usability Evaluation is similar to Expert review. In that scenario, automated programs or applications test the product in order to locate the more common issues. I have successfully used Feng-GUI but it is not free and only works for websites.

Remote usability testing is related to a technique that involves the evaluation of the product by participants and researchers that are not located in the same place; they are working separately and sometimes in an asynchronous way. The tests are carried out not in the laboratory but in a normal context and environment for the participants, so the participant feels comfortable. I have used TryMyUI for testing in a project. This application records (through your browser) your interview and plays it later. It works perfectly but there is not available a free account. 

Questionnaires and interviews are another popular way of conduct usability studies. It involves a more flexible interview or a rigid questionnaire (like SUS[3] or  QUIS[4,7]). Both techniques provide quantitative and qualitative data to the researcher.

Hallway testing is a technique that uses random participants on the testing rather than skilled participants. That technique is very effective testing products for the first time in the development state. I have used TryMyUI for testing in a project. This application allows to record (through your browser) your interview and play it later. It works perfectly but there is not available a free account. 

Paper Prototype Testing is a method usually used in an early development stage, ideally when the product is not yet coded.  It is based on prototypes drawn in paper; the participants must use that prototypes like a real application. It is extremely cheap and quick to prepare.

On the participants side, you will ask yourself how many people you may need. Indeed, the HCI literature solves it for you; Nielsen [5]  and Virzi [6] say that only 5 participants are needed in order to find the 80% of the issues.

Tools

In order to conduct a successful study, I have used the following tools. There are other tools available, but I use the following because the free account they offer and the easy way I can setup my tests. In some way, these tools offer functionalities that apply any research method. In general, the best tools are not free and the free ones do not cover all the test cases you may need. Tools are here to help do the job and are not mandatory for a successful study.

chalkmark

The researcher can perform some quality test with the free account the product has. I find especially useful the ‘first click testing’. You only need some wireframes or screenshots of your product, define some tasks and find a few participants. This software will do the rest, showing you the selected results for the analysis.

iperception

Let you create a free account with limited functionalities but enough for basic testing. Evaluation is done through questionnaires that are building upon the existing questions on the platform or adding new ones. Access to the questionnaire on the free version is done through a web browser.

Benefits

The main benefit will be obtained if the usability study is done when the product is still in the development stage. During that stage, it helps to identify problems that could be fixed before the product is on sale or coded. This is important because every usability problem has an impact on the user experience and a negative feeling must be fixed asap.

Internet references to dig deeper

1.  http://usabilitygeek.com/user-experience-ux-tools/
2. https://www.nngroup.com/articles/which-ux-research-methods/
3. https://www.usability.gov/how-to-and-tools/methods/usability-testing.html

References

1. Kjeldskov, J., & Paay, J. (2000). A Longitudinal Review of Mobile HCI Research Methods, 69–78.
2. Kan, M., & Sadler, K. (2005). Emerging Research Methods for Understanding Mobile Technology Use, 1–10.
3. Brooke. (1996). Brooke, J.: SUS: A “quick and dirty” usability scale. In: Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, I.L, dustrypp, 189–194.
4. Schneiderman, B. (1998). Designing the User Interface. 3rd Edition. Addison Wesley Inc., California. [6] Rubin, J. (1994). Handbook of Usability Testing. Wiley. 
5. Nielsen, J. (2000). Why you only need to test with 5 users. Alertbox. Retrieved from http://www.useit.com/alertbox/20000319.html
6. Virzi, R. A. (1992). Refining the test phase of usability evaluation: how many subjects is enough? Human Factors
7. John P. Chin, Virginia A. Diehl, and Kent L. Norman. 1988. Development of an instrument measuring user satisfaction of the human-computer interface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’88), J. J. O’Hare (Ed.). ACM, New York, NY, USA, 213-218
8. https://www.nngroup.com/articles/which-ux-research-methods/
9. https://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/