A content validity and cognitive interview process to evaluate an Illness Perception Questionnaire for African Americans with type 2 diabetes
BMC Research Notes volume 12, Article number: 308 (2019)
The Illness Perception Questionnaire (IPQ) is the only available and empirically validated tool used to gain insight into patient illness beliefs. However, the IPQ has reliability and validity problems when used with African Americans (AAs) and needs to be culturally-adapted and validated for use with this group. This study aimed to utilize findings from focus groups to culturally adapt the IPQ for use in AAs with diabetes. Ten cognitive interviews among AAs with type 2 diabetes explored patients’ interpretation and understanding of the adapted IPQ.
Forty-four new survey items were added to the IPQ. Twenty-nine of the forty-four items were determined as the appropriate number of questions to be tested because of time, and to reduce respondent burden. After the first round of interviews, an item-by-item review of the new items identified problems related to AA comprehension of certain items, their applicability, and wording/tone. Five items identified as problematic were related to AAs understanding of a cure for diabetes, their perception of how food influences their diabetes, how their identity as AAs influence diabetes control, and the dialogue about diabetes within their families and/or community. Findings support the newly developed illness perception questions as culturally specific to AAs with diabetes after being tested for content validity and participant understanding using cognitive interviews.
Type 2 diabetes, a highly burdensome chronic illness affects 3.7 African Americans (AAs) in the United States and leads to a greater risk of death and disability compared to non-Hispanic whites . AAs are also likely to develop diabetes-related complications such as heart disease, stroke, and amputations that eventually lead to death, compared to non-Hispanic whites . A prominent contributor to these diabetes health disparities is poor medication adherence . Medication adherence is defined by the World Health Organization as the extent to which patients take their medication in correspondence with the agreed recommendations from a health care provider . Poor medication adherence causes 50% of treatment failures, leading to 125,000 deaths and 69% of medication-related admissions [2, 5,6,7,8]. While there are many reasons for poor medication adherence, compared to patient sociodemographic and clinical factors, patient beliefs about their illness has one of the largest influences on medication adherence [9, 10].
Poor medication adherence among AAs may be partly due to the lack of personalized and culturally-adapted interventions that account for their unique illness beliefs which may be different compared to other racial groups [3, 11]. Also, the differences in diabetes complications between AAs and whites may result from these overlooked illness beliefs. For example, compared to whites, some AAs have religious beliefs about the control of diabetes including a belief in God’s ability to cure diabetes . One of the important steps in improving medication adherence among AAs with diabetes is to assess their beliefs about diabetes in a valid and reliable way. The only validated and reliable measure available to assess illness beliefs is the Illness Perception Questionnaire (IPQ). The components of the survey provide a framework for patients to make sense of their symptoms, assess their health risk, and direct their actions and coping skills. However, this survey has only been validated in a Western European population [12, 13] and has significant reliability and validity measurement problems when used with AAs [14,15,16,17]. For example, in a study by Ward et al. , five of the revised Illness Perception Questionnaire sub-scales had consistency/accuracy issues when used among AAs with mental illness in relation to their health-seeking behavior . The measure did not address unique AA cultural beliefs that defined their coherent view of their mental illness. The challenges in measuring beliefs about mental illness might be similar when measuring beliefs about diabetes. For example, AA diabetes-related cultural beliefs such as familial relationships, their racial identity and/or religious beliefs are not accounted for in the IPQ. Thus, there is an urgent need for a valid and reliable culturally-adapted survey to assess illness beliefs for AAs. Importantly, to develop and implement culturally appropriate diabetes medication adherence interventions which account for AAs illness beliefs, a measure that appropriately assesses their culturally informed beliefs is needed.
In evaluating and making changes to a survey tool, it is important to understand if added or revised items measure what the researchers intend and respondents understand and correctly interpret the items . For this study, subsequent to the adaptation of the Illness Perception Questionnaire for AAs with diabetes, it was important to ensure the adapted questionnaire had content (survey items) that represented illness perceptions and was understandable to AAs. Hence, cognitive interviewing (a psychologically oriented method for empirically studying how individuals mentally process and respond to survey questionnaires)  was used as a means to pre-test the culturally adapted IPQ to explore if the new survey items measured what was intended and whether respondents understood and could respond to the items.
The study aimed to utilize qualitative findings from prior focus groups [20, 21] to adapt the established Illness Perception Questionnaire [13, 22] for use in AAs with diabetes, and to use cognitive interviews to explore patients’ interpretation and understanding of the survey. This study describes the cognitive interviewing process used to pretest the survey. The cognitive interviews were an important pre-survey evaluation step in the questionnaire development process because it identifies response errors, and can inform early identification of questionnaire problems . We discuss the methods that went into refining and validating the questionnaire in this early development process. Cognitive interviewing identifies potential response problems that may compromise the response rate and the quality of the data and are used in creating culturally appropriate questionnaires for diverse populations. The items selected for evaluation were significantly different in context and topic compared to the original items.
Given the focus of this study, a qualitative approach with the use of cognitive interviewing seemed the best fit. Cognitive interviewing identifies sources of response error in survey questionnaires by focusing on the cognitive process respondents use to answer questions [18, 19, 24, 25].
To prepare for the cognitive interviews, we conducted the following:
Based on the revised IPQ measure , an adapted survey with culturally-embedded characterizations of illness beliefs was developed using themes generated from six focus groups of 40 AAs with diabetes [20, 21].
Forty-four new items were added to the 39-item revised IPQ, resulting in a culturally adapted tool with 83 survey items (Additional file 1: Appendix).
Two experts on illness perceptions (EW and CB), as well as an expert on survey design and psychometric methods (DB), examined the survey for content areas needed to measure the illness beliefs important to AAs medication adherence, as well as survey item structure and wording. Then, the cognitive interviews were completed.
Six women and four men English-speaking AA age 45–60 years were recruited using purposive sampling. The inclusion criteria were self-identified AA/Blacks who spoke English, self-reported diagnosis of type 2 diabetes by a health care provider at least 1 year prior to the time of the study and reported taking at least one diabetes medication by mouth. The cognitive interviews were conducted 2 weeks apart in March 2018. The study was approved by the University of Wisconsin-Madison Health Sciences Institutional Review Board (IRB number: 2018-0281).
Study participants were recruited from different AA community locations in a Midwestern City in the United States. Flyers were distributed at AA churches, non-profit organizations, public libraries, and community centers. Interested participants then contacted the study recruiter about their intention to participate. Informed consent was obtained prior to the interviews.
Measures and data collection procedures
Interviews were conducted in private locations at the convenience of participants. Two iterative interview processes were conducted 2 weeks apart, with 5 respondents in each phase. All interviews were audio-recorded. The survey was administered by an AA Ph.D. trained health-equity researcher and an AA project assistant trained in facilitating cognitive interviews. At the start of the interview, standardized instructions were read aloud to each participant. Respondents were told that the essence of the interview was not to test respondents for right or wrong answers. Rather, it was to understand their thought process and understanding of questions and to explore any suggestions or comments they had about the questions.
Verbal probing (a form of data collection in which the cognitive interviewer administers a series of probe questions specifically designed to elicit detailed information beyond that normally provided by respondents) was used . The interviewers guided the interaction more proactively by asking additional, direct questions about the basis of respondents’ responses. Respondents were offered a copy of the Likert-scale and asked to identify their response choice following each question by the interviewer. Sample questions included, “There is no cure for your diabetes. Do you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree?” The interviewer further probed the respondents of their answer choices. Probing questions were tailored to the respondents’ replies. For example, “Tell me more about why you answered [ANSWER] for this question.” If a participant chose “neither agree nor disagree” they were further probed to decide if they had to choose between “agree” or “disagree,” which option would they have selected and the reason for that choice. Salient words in the questions were selected and further probed. For example, the interviewer asked, “What did the word “cure” mean to you in this question?” For each question, respondents were encouraged to share any suggestions or concerns regarding the questions. To ensure that respondents only focused on each separate survey question per time, only the pertinent question was visible to the respondents at each time.
Think aloud methods (which requests that the survey respondents actively verbalize their thoughts as they attempt to answer the survey questions) were also used as a cognitive process for exploring participants’ understanding . The respondents used this think-aloud technique to talk through the process of their answer selection. Basic demographic characteristics were collected prior to the interviews. Participants were given $50 cash upon completion of the interviews.
All interviews were transcribed by a certified transcriptionist. Thematic analysis was used to analyze the interviews by combining the notes pertaining to each evaluated survey item, exploring and aggregating common themes across interviews, and identifying key findings that may indicate differences between the intended interpretation and that of respondents . The analysis was completed by two individuals trained in qualitative data analysis. Data analysis was conducted using Nvivo 11 (QSR International-Melbourne) and continued until the researchers could not identify newer dimensions within the data (data saturation). After all analyses were completed independently, the coders met to identify similarities and differences between themes and discuss any divergences. Data analysis was completed upon data saturation.
Ten AA men and women participated. Most respondents were female (n = 6, 60.0%) with a mean age of 57.5 (± 4.5) years. Each participant evaluated each survey item and response option presented to them. The experts on illness perceptions, survey design, and psychometric methods determined that each new survey item was important in the measurement of AA beliefs about diabetes. Using face value, they evaluated each new item for how important it was in the conceptualization of AAs’ illness beliefs. If the survey item did not seem to represent AAs beliefs, they indicated this in their evaluation. Hence, no items were deleted.
After the interviews, an item-by-item review of the 29 new survey items identified problems with five items related to AA’s comprehension, reference/applicability, and wording/tone. These items were related to AA’s understanding of a cure for diabetes, perception of how food influences their diabetes, how their identity as AA influences diabetes control, and the dialogue about diabetes within their families and/or community (Table 1). For the problem types, comprehension of question was related to their understanding of what the question meant, unclear reference focused on their ability to connect the question to their life and answer appropriately, wording or tone referred to their sensitivity to the language/word used. Five of the items were revised to make them clearer and no questions were deleted (Table 2).
Our findings suggest that to create a culturally adapted survey that is usable by AAs, focusing on the target population perception of the meaning of the questions prior to survey administration is fundamental in the design of the survey. Listening to and probing respondents as they verbalized their thought processes allowed for the refinement of the survey. Out of 29 newly developed survey items that were culturally specific to AAs with diabetes and tested using cognitive interviews, five questions were identified as problematic and revised accordingly.
The study results show that by utilizing cognitive interviewing, the comprehension of questions for a culturally adapted illness perception questionnaire can be improved and made culturally appropriate for underserved populations, and questions can be easily interpreted when they are more applicable to the daily life of respondents. By better addressing respondents’ concern about the new survey items in the culturally adapted IPQ, more reliable and valid data on AAs illness beliefs about type 2 diabetes can be collected and utilized for clinical and research purposes. For example, the adapted questionnaire can be offered to AA patients with poor control of diabetes and/or medication nonadherence to understand which illness beliefs should be addressed in an intervention.
Although the Illness Perception Questionnaire has been pretested in different populations, no intensive cognitive interviewing process had been conducted before prior use in an AA population. This study’s results further support that cognitive interviews are an effective technique for improving health surveys and modifying them to be culturally relevant to diverse populations [19, 26]. The finalized version of the culturally adapted Illness Perception Questionnaire for AAs with diabetes will be further tested for reliability and validity among a larger group of AAs with type 2 diabetes.
The study findings support the newly developed illness perception questions as culturally specific to AAs with diabetes after being tested for content validity and participants’ understanding using cognitive interviews.
The study had some limitations. General probes were administered before specific probes for most questions. It is possible that the response to the specific probes was enhanced because participants had been primed from the general probes. The respondents were selected from a single city in a Midwestern, United States town. AAs from more diverse locations may have a different interpretation of the survey questions. Future studies should conduct further cognitive testing among AAs from different parts of the United States. Due to the time constraints of the cognitive interview process, only 29 survey items out of the 44 new items were tested.
Availability of data and materials
The dataset used and/or analyzed for the current study are available from the corresponding author on reasonable request.
Centers for Disease Control and Prevention. National Diabetes Statistics Report. Atlanta: Centers for Disease Control and Prevention, U.S. Dept of Health and Human Services; 2017.
Overcoming barriers to treatment adherence in minorities and persons living in poverty. https://grants.nih.gov/grants/guide/rfa-files/rfa-hl-01-005.html. Accessed 24 May 2019.
Hu D, Juarez DT, Yeboah M, Castillo TP. Interventions to increase medication adherence in African-American and Latino populations: a literature review. Hawaii J Med Public Health. 2014;73:11.
Dobbels F, Van Damme-Lombaert R, Vanhaecke J, De Geest S. Growing pains: non-adherence with the immunosuppressive regimen in adolescent transplant recipients. Pediatr Transplant. 2005;9:381–90.
Haynes RB, McKibbon KA, Kanani R. Systematic review of randomised trials of interventions to assist patients to follow prescriptions for medications. Lancet. 1996;348:383–6.
Kelly JA, Kalichman SC. Behavioral research in HIV/AIDS primary and secondary prevention: recent advances and future directions. J Consult Clin Psychol. 2002;70:626.
Leventhal H, Nerenz DR, Steele DJ. Illness representations and coping with health threats. In: Baum A, Taylor SE, Singer JE, editors. Handbook of psychology and health. Hillsdale, NJ: Erlbaum; 1984. p. 219–52.
Haynes RB, Sackett DL. Compliance in health care. Baltimore: Johns Hopkins University Press; 1979.
Horne R, Weinman J. Self-regulation and self-management in asthma: exploring the role of illness perceptions and treatment beliefs in explaining non-adherence to preventer medication. Psychol Health. 2002;17:17–32.
Barnes L, Moss-Morris R, Kaufusi M. Illness beliefs and adherence in diabetes mellitus: a comparison between Tongan and European patients. N Z Med J. 2004;117(1188):U743.
Skelly AH, Dougherty M, Gesler WM, Soward AC, Burns D, Arcury TA. African American beliefs about diabetes. West J Nurs Res. 2006;28:9–29.
Diefenbach MA, Leventhal H. The common-sense model of illness representation: theoretical and practical considerations. J Soc Distress Homeless. 1996;5:11–38.
Weinman J, Petrie KJ, Moss-Morris R, Horne R. The illness perception questionnaire: a new method for assessing the cognitive representation of illness. Psychol Health. 1996;11:431–45.
Ward E, Wiltshire JC, Detry MA, Brown RL. African American men and women’s attitude toward mental illness, perceptions of stigma, and preferred coping behaviors. Nurs Res. 2013;62:185.
Ward EC, le Clark O, Heidrich S. African American Women’s beliefs, coping behaviors, and barriers to seeking mental health services. Qual Health Res. 2009;19:1589–601.
Ward EC, Heidrich SM. African American women’s beliefs about mental illness, stigma, and preferred coping behaviors. Res Nurs Health. 2009;32:480–92.
Abubakari A-R, Jones MC, Lauder W, Kirk A, Devendra D, Anderson J. Psychometric properties of the Revised Illness Perception Questionnaire: factor structure and reliability among African-origin populations with type 2 diabetes. Int J Nurs Stud. 2012;49:672–81.
Collins D. Pretesting survey instruments: an overview of cognitive methods. Qual Life Res. 2003;12:229–38.
Willis GB. Cognitive interviewing: a tool for improving questionnaire design. Thousand Oaks: Sage Publications; 2004.
Shiyanbola OO, Ward EC, Brown CM. Utilizing the common sense model to explore African Americans’ perception of type 2 diabetes: a qualitative study. PLoS ONE. 2018;13:e0207692.
Shiyanbola OO, Ward E, Brown C. Sociocultural influences on African Americans’ representations of type 2 diabetes: a qualitative study. Ethn Dis. 2018;28:25–32.
Moss-Morris R, Weinman J, Petrie K, Horne R, Cameron L, Buick D. The revised illness perception questionnaire (IPQ-R). Psychol Health. 2002;17:1–16.
Buers C, Triemstra M, Bloemendal E, Zwijnenberg NC, Hendriks M, Delnoij DMJ. The value of cognitive interviewing for optimizing a patient experience survey. Int J Soc Res Methodol. 2014;17:325–40.
Jobe JB. Cognitive psychology and self-reports: models and methods. Qual Life Res. 2003;12:219–27.
Daugherty S, Harris-Kojetin L, Squire C, Jaël E, Harris-Kojetin L. Maximizing the quality of cognitive interviewing data: an exploration of three approaches and their informational contributions. In: Proceedings of the annual meeting of the American Statistical Association; 2001.
Willis GB, Artino AR Jr. What do our respondents think we’re asking? using cognitive interviewing to improve medical education surveys. J Grad Med Educ. 2013;5:353–6.
Blair J, Conrad FG. Sample size for cognitive interview pretesting. Public Opin Q. 2011;75:636–58.
Nowell L, Norris J, White D, Moules N. Thematic analysis: striving to meet the trustworthiness criteria. Int J Qual Methods. 2017;16:1–13.
We would like to acknowledge Vic Bankston who helped with data collection and Ariana Song who helped with the data analysis.
This project is supported by the Clinical and Translational Science Award (CTSA) program, through the National Institutes of Health (NIH) National Center for Advancing Translational Sciences, grants UL1TR000427 and 5KL2TR000428. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The funding body had no role in the design of the study, data collection, analysis, interpretation of data and in writing the manuscript.
Ethics approval and consent to participate
Ethics approval was obtained from the University of Wisconsin-Madison Health Sciences Institutional Review Board, (Reference Number: 2018-0281). An informed consent form and study information sheet was given to participants. Study participants gave written consent before completing the interviews.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
A preliminary version of the Culturally Adapted Illness Perception Questionnaire for African Americans with Diabetes. This is the preliminary version of the Culturally Adapted Illness Perception Questionnaire for African Americans with Diabetes.
About this article
Cite this article
Shiyanbola, O.O., Bolt, D., Tarfa, A. et al. A content validity and cognitive interview process to evaluate an Illness Perception Questionnaire for African Americans with type 2 diabetes. BMC Res Notes 12, 308 (2019). https://doi.org/10.1186/s13104-019-4342-9