Quality in quantitative studies are regularly discussed and taught in psychology and there are numerous frameworks consisting of criteria that determines what is and (what is not) a high quality study. This is often informed by the type of research design used, for example a randomized controlled trial is considered the ‘gold standard’ in terms of quality due to the blinding of participants and personnel to reduce bias, whereas a before and after study with no control group may produce biased results.
Qualitative research, on the other hand, is based on an interpretation of results, and numerous methods can be used based on the epistemological stance of the researcher. Quality in qualitative studies differs to the rigid quality checklists in quantitative studies due to the range of methodologies used. Instead quality appraisals are based on general criteria that can be applied to any qualitative methodology or study (Salter et al., 2008). Lincoln and Guba (1985) instead, propose four broad categories of quality as alternatives for quantitatively-oriented criteria; Credibility, Transferability, Dependability and Confirmability. To assess the quality of a study, you will need to have an understanding of qualitative methods. I have provided some questions you might want to ask as you read a study to help you assess this.
1) Credibility (as an alternative to internal validity), the degree to which research techniques and analysis are sound to ensure the believability of the interpretation. Do the research methods used help to answer the research question being asked, or would a different methodology allow the researchers to make more accurate claims about the research? Was the data collected in a methodologically sound way? Were interpretations checked by other members of the team? Were results presented to participants to check understanding? Is the author trained in qualitative methods? To improve credibility a researcher can become engaged in the phenomenon (prolonged engagement) and observe the phenomenon (persistent observation) to increase their understanding before analysis. The results and analysis can be checked by other researchers and by presentation to participants, to improve the credibility of the researcher’s findings.
2) Transferability (as an alternative to external validity), the degree to which the results of the study can be generalized or transferred to other contexts or settings. Is there a good description of the participants, the method, and the analysis? Lincoln and Guba (1985), posited that a ‘thick description’ of the research process will enable researchers to assess whether these results can be transferred to other settings or contexts.
3) Dependability (as an alternative to reliability), the degree to which the research has the ability to account for the changing context within which research occurs. Can the findings of this research be repeated? To improve the dependability of qualitative studies, it often helps to have a researcher, who is not part of the research team, code and analyze data to see if you come to the same conclusions.
4) Confirmability (as an alternative to objectivity), the degree to which results of the study can be confirmed or corroborated by others. Are the results of this study due to the respondents or the researchers? Are quotes used? To improve the dependability of qualitative research, it helps to have an audit trail of the analysis, so the conclusions drawn from the analysis can be shown to be a product of the data. Confirmability can also be improved by triangulation; triangulation refers to the same results being found in different studies or by different methods, in this way, each study supports the other, increasing the confirmability of your results.
Whilst I recommend awareness and knowledge of qualitative methods before you begin to assess the quality of qualitative studies, hopefully, this quick tutorial will act as a primer and provide some starter questions to ask when reading and assessing qualitative research.
References
LINCOLN, Y. & GUBA, E. (1985) Naturalistic Inquiry, Thousand Oaks, California, Sage.
SALTER, K., HELLINGS, C., FOLEY, N. & TEASELL, R. (2008) The experience of living with stroke: a qualitative meta-synthesis. Jounral of Rehabilitative Medicine, 40, 595-602.
