PRO development rigorous qualitative research as

Authors: Kathryn Eilene Lasch Patrick Marquis Marc Vigneux Linda Abetz Benoit Arnould Martha Bayliss Bruce Crawford Kathleen Rosa

Publish Date: 2010/05/30

Volume: 19, Issue: 8, Pages: 1087-1096

PDF Link

Abstract

Recently published articles have described criteria to assess qualitative research in the health field in general but very few articles have delineated qualitative methods to be used in the development of PatientReported Outcomes PROs In fact how PROs are developed with subject input through focus groups and interviews has been given relatively short shrift in the PRO literature when compared to the plethora of quantitative articles on the psychometric properties of PROs If documented at all most PRO validation articles give little for the reader to evaluate the content validity of the measures and the credibility and trustworthiness of the methods used to develop them Increasingly however scientists and authorities want to be assured that PRO items and scales have meaning and relevance to subjects This article was developed by an international interdisciplinary group of psychologists psychometricians regulatory experts a physician and a sociologist It presents rigorous and appropriate qualitative research methods for developing PROs with content validity The approach described combines an overarching phenomenological theoretical framework with grounded theory data collection and analysis methods to yield PRO items and scales that have content validityPatientreported outcomes PROs in clinical trials effectiveness studies and public health research have been defined as “any report coming directly from subjects without interpretation of the physician or others about how they function overall or feel in relation to a condition and its therapy” 1 p 125 The value of qualitative research in the development of PRO measures has been recognized for many years Witness the growing acceptance of such research by a new edition of a book that devoted a brief chapter to qualitative research in an otherwise comprehensive volume on quantitative methods that are used to measure quality of life in clinical trials 2 A more recent focus has been placed on the concepts being measured and their meaning—not in terms of correlation coefficients or factorial structure but their authenticity for subjects ie their content validity The emergence of content validity as a construct was to guard against strictly numerical evaluation of tests and other measures that overlooked serious threats to the validity of inferences derived from their scores 3 This article presents an approach incorporating an overarching phenomenological approach into grounded theory data collection and analysis methods to most accurately include the subject’s voice in PRO developmentThe quest for authenticity in instrument development evolved from a pragmatic approach ranging from literature review clinician expertise and the psychometric performance of items from large samples and batteries eg Medical Outcomes Study Short Form SF36 to direct involvement by subjects in item generation 4 When subjects have been included to date however the systematic analysis of their words and the link from their words to concepts underlying items is usually neither documented nor transparent Transparency and systematization however are considered hallmarks of good qualitative research 5 Their absence in qualitative research in the PRO field makes it difficult to communicate and compare results Other essential issues in the conduct of rigorous qualitative research for PRO development include who does one interview how does one analyze the data systematically and transparently how does one develop a conceptual framework to undergird a questionnaire from participants’ responses and above all what overarching theoretical framework a guide as to which concepts and which relationships between those concepts should be the focus of a research study if any would best serve PRO development A conceptual framework as defined by the Food and Drug Administration FDA one of the major constituents for PROs represents the demonstrated relationships between and among items on a questionnaire and domains multidimensional concept in which items are grouped together1 6The FDA issued a PRO draft guidance document in 2006 and a final Guidance to Industry in 2009 that when followed makes it critical for instrument developers or reviewers to use and understand stateoftheart methods in qualitative research 6 7 8 Adherence to this guideline is required if the questionnaire is intended as an endpoint to evaluate treatment benefit assessing clear concepts that might support a labeling and/or advertising claim Recently members of the Study Endpoints Label Development SEALD division in the FDA gave presentations at the 45th Annual Meeting of the Drug Information Association in which they emphasized the importance of content validity as an important and qualitative measure Content validity in general means that a measure captures what it intended to measure In these presentations the FDA more specifically defined content validity of a PRO as 1 evidence that the items and domains measure the intended concepts as depicted in the conceptual framework and desired claim 2 evidence that the items domains and concepts were developed with subject input and are appropriate comprehensive and interpretable by subjects and 3 that the study sample is representative of the target populationBoth the collection of qualitative data and its analysis have become more systematized and rigorous in the past 30 years as health researchers have increasingly incorporated them into their work The most informative ways to interview participants have been refined Even when provided with discussion guides and training to conduct focus groups or indepth interviews however interviewers untrained in qualitative research methods use these guides as though they were conducting a structured interview They often ask questions that put words in the subjects’ mouths and do not dig deeper than what is directly asked or rarely go beyond the scope of questioning Probes that ask a study participant to describe more fully the meaning of a concept that is spontaneously offered are rarely used Using guides as rigid scripts limits the collection of data that is ideal for capturing subjects’ meaning of the experience of a condition and its treatment In addition the PRO field generally has not taken full advantage of the decades of knowledge in the field of survey research psychology to construct items and responses that most clearly depict the experience of a symptom or an impact of a treatment or a condition 9 10 11Researchers have published or presented criteria on how to evaluate qualitative research in health literature in general and in the development of PROs in particular 12 13 14 15 16 However very little information is available in the PRO field on how to collect and analyze qualitative data compared to the plethora of literature on psychometric methods to support the validity of PROs Only one article to our knowledge published in 2008 specifically discusses qualitative research methods to assure clarity and content validity in PROs 17We present an approach to develop a PRO instrument with content validity This approach was developed by an international interdisciplinary team of psychologists psychometricians regulatory experts a physician and a sociologist with over 25 years of experience conducting qualitative research We describe how qualitative research and the psychology of survey response may best be applied to capture both the meaning of medical conditions to subjects and treatment impactSimilar to its quantitative equivalent qualitative research is an umbrella term for various theoretical models and data collection methods 18 19 Anthropologists sociologists nursing researchers and recently psychologists have applied various methods and theories to the health arena 7 20 21 22 23 There is also extensive literature on the psychology of survey research that addresses how respondents answer items on a questionnaire 9 10 11 24 25 26 27 28 The most commonly used cognitive model is the question/answer model proposed by Tourrangeau in 1984 29 This model identifies the cognitive stages in answering a survey question including comprehension retrieval judgment response selection and response reporting 25 This literature takes into account the interactive aspects of the interview context and the cognitive processes that are involved in answering items Its focus has been on the improvement of questionnaire design rather than the blank slate involvement of subjects to capture important concepts

Keywords: