- Data note
- Open Access
Changes in COVID-19 risk perceptions: methods of an internet survey conducted in six countries
BMC Research Notes volume 14, Article number: 428 (2021)
This study assessed changes in behaviors/attitudes related to the COVID-19. With the understanding that behaviors and vaccine decision-making could contribute to global spread of infectious diseases, this study collected several waves of internet-based surveys from individuals in the United States, mainland China, Taiwan, Malaysia, Indonesia, and India. The aims of this study were to (1) characterize the relationship between the epidemiology of disease and changes over time in risk perceptions, knowledge, and attitudes towards hygienic behaviors; (2) examine if risk perceptions affect acceptance of less-than-ideal vaccines; and (3) contrast adherence to public health recommendations across countries which have had different governmental responses to the outbreak.
We conducted cross-sectional online surveys in six countries from March 2020 to April 2021. By the end of June 2021, there will be six waves of surveys for the United States and China, and four waves for the rest of countries. There are common sets of questions for all countries, however, some questions were adapted to reflect local situations and some questions were designed intentionally for specific countries to capture different COVID-19 mitigation actions. Participants were asked about their adherence towards countermeasures, risk perceptions, and acceptance of a hypothetical vaccine for COVID-19.
The goal of this project is to understand the international dimensions of behavioral changes and vaccine decisions that members of the general population make regarding COVID-19. The six countries studied in this project have had diverse mitigation responses to COVID-19. As of March 22, 2020, at the start of this project, China had 81,397 cases; US 32,057; Malaysia 1306; Indonesia 514; India 376; and Taiwan 169 . Their on-going responses reflected the epidemiological circumstances and have changed over time . The on-going roll-out of the COVID-19 vaccine has also differed widely across the countries. For example, the COVID-19 vaccination rollout started January 13, 2021 in Indonesia, February 24, 2021, in Malaysia, and March 22, 2021, in Taiwan, with the rate of vaccination uptake varying widely across countries [3, 4].
The aims of this study were to characterize the relationship between the epidemiology of disease and the changes over time in risk perceptions, knowledge, and attitudes towards hygienic behaviors; examine if risk perceptions affect acceptance of less-than-ideal vaccines; and contrast adherence to public health recommendations across countries which have had different governmental responses to the outbreak.
We used an opt-in internet-based sample. A survey research firm obtained potential participants from social media and online advertisements and collected contact information and basic demographic data. Basic information about these panelists is available online . At each wave, the research firm sent a link to the survey to a subset of these panelists. The full sets of questionnaires by wave, along with Table A, which describes the sample size and dates of data collection, are publicly available at: https://doi.org/10.6084/m9.figshare.14792058.
The required sample size is around 800 for most countries for each wave. With an alpha of 0.05, a power of 80%, and a proportion of 50% (a statistically conservative estimate of what proportion of the population supports a given public health action), the margin of error will be 4%. This margin of error will allow us to assess substantial trends over time. We sampled more individuals in the US across six waves to do sub-analyses by race/ethnicity and to track changes in behaviors at a finer scale.
We used quota-based sampling, such that the distribution of individuals by age and gender in this population reflected that of the adult population in each country . Subsequently, we created weights for each country using a raking procedure , which reflected the distribution of age, gender, and region within a country (and race/ethnicity in the US). These weights do not account for differential access to internet. The age ranges sampled was similar across countries. All adults ≥ 18 years were eligible, except for Taiwan, where adults ≥ 20 years were eligible.
Outcomes of interest include mask-wearing behavior, measured by the number of days an individual went outside the house in the past week and how often they wore a mask. Similar with how it was asked for previous pandemics , we asked individuals their perceived likelihood of acquiring SARS-CoV-2 in the next month and their perceived risk of death if infected. Vaccination intent was measured differently for waves 1–5 versus after. In waves 1–5, we embedded vaccination intent in an experiment, where we first randomized to receive different information about a vaccine. A COVID-19 vaccine was said to be either 95% or 50% effective and to have a 5% or 20% risk of side effects like fever. These numbers were based off the range of plausible values when the survey was first created March 2020, with the effectiveness bounds based on influenza and measles vaccines [9, 10]. In addition to the international samples, this survey experiment was also embedded within a study in Detroit, MI, and in another study in Shanghai, China [11, 12].
Starting in wave 6, we asked individuals vaccination questions based off draft questions from the National Institutes of Health (NIH) Community Engagement Alliance Against COVID-19 Disparities (CEAL).
Vaccine hesitancy was measured through a 10-item scale, which was developed by the World Health Organization (WHO) Strategic Advisory Group on Immunization (SAGE) Vaccine Hesitancy Working Group for parental attitudes towards pediatric vaccines . We found our adaptation, the adult Vaccine Hesitancy Scale (aVHS), to be highly related to influenza and COVID-19 vaccination behaviors . Table 1 provides an overview of the datasets.
This survey used Internet-based samples, which allowed us to rapidly collect information and to avoid person-to-person contact. However, internet samples may have inherent biases. There is sampling bias in that individuals who participate need to have access to the internet, so individuals of lower socioeconomic status will be less likely to participate. With the exception of the vaccine questions, we tried not to change questions across surveys to maintain comparability. An education variable was only asked starting in wave 6. We pre-tested the questionnaires in all countries to make sure the language used is at a 6th grade level and is clear without ambiguity or confusion. Additionally, individuals may answer rapidly with little thought. We eliminated individuals who took less than 180 s on the survey, and we required individuals to answer each question for each survey, except for wave 1 and for all waves in Taiwan due to the requirements by the IRB.
Adult Vaccine Hesitancy Scale
Community Engagement Alliance Against COVID-19 Disparities
National Institutes of Health
Strategic Advisory Group on Immunization
World Health Organization
Center for Systems Science and Engineering. Coronavirus COVID-19 global cases. 2020. https://gisanddata.maps.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6. Accessed 18 Mar 2020.
Hale T, Angrist N, Goldszmidt R, Kira B, Petherick A, Phillips T, et al. A global panel database of pandemic policies (Oxford COVID-19 government response tracker). Nat Hum Behav. 2021;5:529–38.
The special committee for ensuring access to covid-19 vaccine supply (JKJAV). National covid-19 immunisation programme. 2021. https://www.vaksincovid.gov.my/pdf/National_COVID-19_Immunisation_Programme_English.pdf. Accessed 16 June 2021.
Mathieu E, Ritchie H, Ortiz-Ospina E, Roser M, Hasell J, Appel C, et al. A global database of COVID-19 vaccinations. Nat Hum Behav. 2021;5:947–53.
Dynata. Data & insights platform. 2021. http://info.dynata.com/panel-book. Accessed 16 June 2021.
US Census. International data base. 2021. https://www.census.gov/data-tools/demo/idb/#/country?YR_ANIM=2021. Accessed 16 June 2021.
Izrael D, Hoaglin DC, Battaglia MP. A SAS macro for balancing a weighted sample. Statistics and data analysis. 2000. https://support.sas.com/resources/papers/proceedings/proceedings/sugi25/25/st/25p258.pdf. Accessed 16 June 2021.
Gidengil CA, Parker AM, Zikmund-Fisher BJ. Trends in risk perceptions and vaccination intentions: a longitudinal study of the first year of the H1N1 pandemic. Am J Public Health. 2012;102:672–9.
Uzicanin A, Zimmerman L. Field effectiveness of live attenuated measles-containing vaccines: a review of published literature. J Infect Dis. 2011;204(Suppl 1):S133–48.
Centers for Disease Control and Prevention. Seasonal influenza vaccine effectiveness, 2004–2018. Influenza (Flu). 2019. https://www.cdc.gov/flu/professionals/vaccination/effectiveness-studies.htm. Accessed 19 Mar 2019.
Wagner AL, Sheinfeld Gorin S, Boulton ML, Glover BA, Morenoff JD. Effect of vaccine effectiveness and safety on COVID-19 vaccine acceptance in Detroit, Michigan, July 2020. Hum Vaccines Immunother. 2021;17:2940–5.
Lu J, Wen X, Guo Q, Ji M, Zhang F, Wagner AL, et al. Sensitivity to COVID-19 vaccine effectiveness and safety in Shanghai, China. Vaccines. 2021;9:472.
Larson HJ, Jarrett C, Schulz W, Chaudhuri M, Zhou Y, Dube E, et al. Measuring vaccine hesitancy: the development of a survey tool. Vaccine. 2015;33:4165–75.
Akel KB, Masters NB, Shih S-F, Lu Y, Wagner AL. Modification of a vaccine hesitancy scale for use in adult vaccinations in the United States and China. Hum Vaccines Immunother. 2021;17:2639–46.
Wagner AL. COVID-19 vaccine hesitancy surveys. Inter-university consortium for political and social research. 2021. https://doi.org/10.3886/E130422V2. Accessed 31 Aug 2021.
We appreciate the contributions of Mengdi Ji and Kaitlyn B. Akel in the survey design and data cleaning.
Waves 1 in the United States and in China were supported by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health (Award Number K01AI137123). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The other waves were supported by an award from the National Science Foundation, Division of Social and Economic Sciences (Award Number: #2027836).
Ethics approval and consent to participate
The protocol was reviewed and approved by ethical review committees in each country, including the University of Michigan Health Sciences and Behavioral Sciences Institutional Review Board (#HUM00180096), the Fudan University School of Public Health ethical review committee (#IRB00002408), the National Taiwan University Hospital Research Ethics Committee (#202007102RINB), the Universiti Tunku Abdul Rahman (#U/SERC/107/2020), the Institutional Review Board at Universitas Syiah Kuala/Dr. Zaionel Abidin Hospital, Indonesia (#041/EA/FK-RSUDZA/2020), and the Sigma-IRB in New Delhi, India (#10003/IRB/20-21). Participants read an informed consent form and clicked “I agree to participate in the study” prior to any data collection occurring.
Consent for publication
We declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Zhang, F., Shih, SF., Harapan, H. et al. Changes in COVID-19 risk perceptions: methods of an internet survey conducted in six countries. BMC Res Notes 14, 428 (2021). https://doi.org/10.1186/s13104-021-05846-8
- Vaccine hesitancy
- Risk perceptions
- Behavioral/attitudinal changes