Yesterday I wrote about the importance of selecting the most appropriate research method for the problem or opportunity at hand. A significant step in that process is deciding whether the research questions are best answered by qualitative research or by quantitative research.
Some people assume that there are two categories of research methods, some of them qualitative and the others quantitative. However, it isn’t the methods themselves that are qualitative or quantitative. It is the way in which the data is captured and analyzed that is either qualitative or quantitative. For example, if a survey’s question structure or sample designation is not statistically sound, then it is actually qualitative research, not quantitative, despite the table of percentages that is reported at the end of the project. Interviews, on the other hand, could be conducted in a way that would be quantitative and statistically relevant; I haven’t found a client yet who was willing to pay for quantitative interview data, though…
Quantitative research is different from qualitative research because it has a known confidence interval, and it is relatively small. The confidence interval is typically reported as +/- percentage points of error, so, for example, a result with a +/- 5% margin of error has a confidence interval of 10 percentage points. In qualitative research, the confidence interval is either unknown or it is relatively large. This is usually due to the reliance on rich data, or non-numeric data, which leads to less precision in the resulting data. Another reason that the confidence level is either unknown or very large in qualitative research is because the sample size is typically smaller than in quantitative research. Increasing the sample size would lead to a smaller confidence interval, indicating greater precision in the results. Sample size in qualitative research is usually limited by budget considerations or because of the time it would take to capture and analyze rich data from a large number of participants.
Researchers whom I’ve met in corporate settings seem to favor quantitative research over qualitative research, because they can express their results with confidence. However, qualitative research, if conducted properly, can provide insights that quantitative research will not provide, or cannot provide in a cost effective manner. Qualitative research can help the design team:
- Discover unseen factors that impact whether users will find a given system easy or effective.
- Understand unfamiliar phenomena, or phenomena that cannot easily be measured.
- Model interactive behavior, as opposed to determining the statistical prevalence of a given attitude or preference.
Examples of research methods that are typically associated with qualitative data collection include:
- Depth Interviews
- Intercept interviews
- Ethnographic research
- Focus groups
- User testing (usability testing)
- Paper prototyping
