Usability testing is a method where researchers give participants tasks on one or more user interfaces. While the participants are performing tasks, the researchers observe them and listen to feedback.
There are two types of data that can be collected in a usability testing study:
Quantitative data allows us to determine the usability of our design indirectly. Values such as the number of tasks completed, number of errors, etc., can give an idea of the interface’s usability. However, quantitative data is not absolute. It needs a reference point. For example, say 40% of participants commit the same particular error. We cannot comment if this is good or bad in the absence of a reference point. Therefore, usability studies are more useful when comparing results to a previously known standard rather than directly describing the design’s usability.
Quantitative data can protect against randomness by providing statistically significant data. However, it does not inform of the exact cause of the issue and the probable solutions.
Quantitative research and data:
Qualitative data offers detailed insights into the usability of the system. Researchers can see the users struggle with particular UI elements and task flows and ask them follow-up questions. This informs the researchers of the exact cause of the issue and thus allows them to craft solutions.
Qualitative research and data:
Lastly, both methods must follow the basic rules of good experimental design:
Internal validity - the setup of the experiment must not favor one condition over another.
External validity - the study group must represent the target audience, and study conditions must reflect how the study will be conducted in a real-life setting.
Both of the criteria above are important to ensure unbiased results that are fair.