Lower objectively and subjectively assessed numeracy are both associated with poorer self-rated health
BMC Research Notes volume 14, Article number: 321 (2021)
To compare an objective with a subjective numeracy assessment for association with self-reported health status, where numeracy refers to “the degree to which individuals have the capacity to access, process, interpret, communicate, and act on numerical, quantitative, graphical, biostatistical, and probabilistic health information needed to make effective health decisions”
We completed a secondary analysis of two population-based surveys, the Empire State Poll (n = 763) and the Program for the International Assessment of Adult Competencies (PIAAC; n = 2609). The first survey assessed numeracy with a 3-item subjective instrument. The second assessed numeracy with more than 20 math problems. Both used the same measure for self-reported health status. Lower numeracy, whether subjectively or objectively assessed, was associated with worse self-reported health, even after controlling for education and other sociodemographic confounders. The odds ratios for the association were very similar (0.91 and 0.90 respectively). A lengthy objective numeracy assessment and a brief self-report assessment had similar associations with health status. A brief self-report measure of numeracy has similar properties to a lengthy objective assessment and is likely to be more feasible to use to screen patients in practice.
The primary objective of this work was to assess the association between self-rated health status and numeracy using two different approaches for assessing numeracy—a short subjective scale and a longer objective numeracy instrument.
Many individuals throughout the world have low health numeracy, defined as “the degree to which individuals have the capacity to access, process, interpret, communicate, and act on numerical, quantitative, graphical, biostatistical, and probabilistic health information needed to make effective health decisions” . People with lower numeracy have demonstrably poorer health knowledge, self-care practices, and health outcomes [2,3,4].
Identifying patients with low health numeracy could allow communicators to adapt their information to the needs of the patient by simplifying or providing additional explanations for quantitative concepts [5,6,7]. Instruments for measuring numeracy and health numeracy range from long, objective examinations to short subjective scales [8,9,10,11,12,13,14,15,16]. Although short subjective assessments are likely to be more acceptable to patients, are more feasible to apply in clinical practice, and are correlated with objective measures [8, 9], it is necessary to show that they are in fact associated with health status. Previous studies have compared objective (e.g. Lipkus et al., 2001) and subjective measures of numeracy (e.g. Fagerlin et al.’s short numeracy scale known as the SNS-8), though they only included single items from the objective assessments, other than the entire, psychometrically evaluated battery [17,18,19]. Dolan and colleagues also assessed the relationship between the SNS-8 and various other objective measures of health numeracy, but did not correlate these with health-related constructs such as self-rated health .
In this study, we present results of two numeracy assessments of two population-based samples and compare the relationship between numeracy, self-rated health, and sociodemographic variables. We hypothesized that numeracy would be positively associated with self-rated health status using both a short, subjective survey and the longer objective survey.
One data set is from the Empire State Poll (ESP), a study conducted annually to understand community, economic, and social science issues among New York State residents, with questions submitted by academic researchers. The Survey Research Institute (SRI) of Cornell University conducted all surveys. Our team submitted a validated, three-question, subjective numeracy assessment, known as the SNS-3, and a question on self-rated health status (see Table 1) . The SNS-3 was derived from the previously validated Subjective Numeracy Scale 8 (SNS-8) . The SNS-8 was validated by demonstrating that higher scores on the SNS-8 predicted better performance on multiple, objective quantitative problems  The validation study of the SNS-3 demonstrated that it had high internal validity (median Cronbach’s α = 0.78), a high correlation with the SNS-8 (median ρ = 0.91), and statistically significant correlations with multiple other objective measures of quantitative skills and literacy .
The second data set is the Program for the International Assessment of Adult Competencies (PIAAC), an international assessment conducted by the intergovernmental Organization for Economic Co-Operation and Development. The PIAAC is conducted to assess basic competencies in a number of skills, including numeracy (our focus), literacy, and problem solving.
ESP participants included New York State residents aged 18 and over who spoke English or Spanish proficiently. Responses were balanced between upstate and downstate residents (New York City area, or “downstate,” and the remainder of the state, or “upstate”). Cornell SRI recruited 800 participants via a random-digit-dial telephone survey between February and April in 2019. The ESP is approved by the Cornell University Institutional Review Board. Data are included in Additional files 1 and 2.
The PIAAC survey included United States residents aged 16–74 years living in households or group quarters, which does not account for persons living in shelters, the incarcerated, military personnel who live in barracks or bases, or persons who live in institutionalized group quarters, such as hospitals or nursing homes. Those aged under 18 years were excluded from our analysis. The PIAAC U.S. 2017 involves a stratified random sample intended to allow for nation and state-level estimates of basic skills. Data were collected between March and November of 2017. Participants were first administered a screener via phone to determine eligibility. Eligible consenting participants completed an in-person background questionnaire then completed competency assessments using a computer or paper-and-pencil (reserved for those who would not or could not use a computer). PIAAC data are publicly available at http://www.oecd.org.
Our primary dependent variable was self-rated health, measured with the same SF-36 question in ESP and PIAAC: “In general, would you say your health is excellent, very good, good, fair, or poor?” Participants could also respond “Do not know” or decline to answer. Our primary independent variable was numeracy. In the ESP, numeracy was assessed with the SNS-3 questions (Table 1), answered on a six-point Likert scale. Likert scale responses are converted to numbers and summed to produce a single score . SNS-3 scores can range from 3 to 18, with higher scores indicating higher subjective numeracy.
The PIAAC measures numeracy using 24 (paper-based) to 52 (computer-based) items covering broad contexts, cognitive processes, and mathematical content (examples in Table 1). Scores are computed as plausible values, a method that has been demonstrated to provide more accurate estimates of group proficiency than single point estimation methods [22, 23]. The PIAAC numeracy score (i.e., average of the 10 plausible values for the sample) can range from 0 to 500, with higher scores indicating higher objective numeracy.
All analyses were conducted using R software version 4.0 (R Core Team, 2018). We first assessed differences in sample demographics between the ESP and PIAAC data sets using Fisher’s exact tests. For both samples, we then created logistic models of the relationship between self-rated health and numeracy, with sociodemographic covariates. We dichotomized self-rated health into not good (poor, fair) and good (good, very good, and excellent) after a preliminary analysis suggested this split improved the AUC and predictive power of the model. For ESP data, weights were added for education, income, race, and ethnicity variables based on a comparison between demographics of the sample and the state using Fisher’s exact tests. The PIAAC was already stratified by gender, age, education, income, race, and ethnicity to reflect the demographics of the United States, so weighting was not necessary .
Because the ESP score could range from 3 to 18 (i.e., it had 15 score increments), we also binned the PIAAC numeracy score into 15 increments by creating score categories of 0 to 32, 33 to 65, 66 to 98, etc., up to 500. This improved comparability of the analyses by ensuring that a single increment in numeracy score represented 1/15 of the total possible range in both cases.
Variable selection was conducted on all data based on the model with the lowest Akaike information criterion using stepwise selection in both forward and backward directions using the glm functions from the R stats packages. We randomly split the data into training and test datasets in ratio of 7:3, used the selected variables to build the model on the training data, and applied it on the test data to obtain the AUC and the prediction error (misclassification) rate.
For the ESP and PIAAC data, 763 and 2609 participants had complete demographic and numeracy data, respectively (Table 2). There were significant differences in the demographic distribution across all categories analyzed, except for race. PIAAC participants with incomplete data for problem solving competencies were also excluded (analysis not presented here). Low numeracy was prevalent in both samples, with 24.8% of ESP participants in the lowest category of SNS-3 scores, and 22.8% of the PIAAC participants in the lowest two categories of numeracy (Table 2).
Table 3 presents the results of the final logistic regression models, including their respective power and AUC. As shown, in both ESP and PIAAC, self-rated health was significantly associated with numeracy, even in models that controlled for age, education, and income.
As hypothesized, lower objective/subjective numeracy was associated with lower self-rated health in both models. The odds ratios show that a single point decrease in the SNS-3 score was associated with 9% lower odds of having good health; similarly, a single category decrease in PIAAC numeracy score was associated with a 10% lower odds of good health. Other variables associated with lower self-rated health in both models included: higher age, lower education, and lower income. In the ESP model only, male gender and Hispanic ethnicity were associated with lower self-rated health.
Numeracy assessed subjectively with a brief 3-item instrument is associated with self-rated health, even after controlling for education and other sociodemographic variables. The association is similar to the association found when assessing numeracy with a complex objective assessment, which would be burdensome to administer in clinical practice. With the subjective and the objective assessments, a single increment in numeracy score was associated with an odds ratio of approximately 0.90, indicating about 10% reduced odds of good self-reported health. The similarity in results suggest that it is not always necessary to evaluate numeracy with a burdensome objective scale, but that a short subjective scale could instead be used in clinical practice. In addition, the brief subjective questions are simple to score and interpret.
These two analyses also replicated previous findings confirming that younger age, higher income, and higher education were all associated with both higher numeracy and better self-rated health. All of these relationships were in the direction that would be expected from previous research [28,29,30]. However, the association between low numeracy and poor health is not fully attributable to these other variables as it remained significant even after controlling for the demographic variables. There were significant differences in the demographic makeup of the two samples. While observation of statistically significant differences may be expected in samples as large those in the PIAAC data, it may be beneficial to have more balanced demographic samples, perhaps representing the same target population (instead of New York state vs. the entire United States), in future studies.
It could be helpful to begin including questions related to literacy/numeracy in initial visits to help health professionals better tailor their communication. Once patients with low numeracy are identified, a growing body of research demonstrates how rewriting patient information can help . For example, Davis et al. demonstrated that rewording medication instructions by phrasing instructions in terms of time of day rather than pills per day disproportionately improves comprehension among low numeracy patients , and Yin and colleagues have shown that low-literacy patients benefit from pictogram-enhanced medication instructions (33).
The study has several limitations. The ESP and PIAAC were conducted for different purposes in different populations. The ESP sample was also found to be significantly different in demographic distribution from the NYS population, which was handled using survey weights.
Availability of data and materials
All data generated or analysed during this study are included in this published article [and its additional files]. Empire State Poll data is included as additional material, and the PIAAC data is publicly available at http://www.oecd.org.
Subjective Numeracy Scale
Empire State Poll
Survey Research Institute
Program for the International Assessment of Adult Competencies
Area Under the Curve
Golbeck AL, Ahlers-Schmidt CR, Paschal AM, Dismuke SE. A definition and operational framework for health numeracy. Am J Prev Med. 2005;29(4):375–6.
Reyna VF, Nelson WL, Han PK, Dieckmann NF. How numeracy influences risk comprehension and medical decision making. Psychol Bull. 2009;135(6):943–73.
Peters E, Västfjäll D, Slovic P, Mertz C, Mazzocco K, Dickert S. Numeracy and decision making. Psychol Sci. 2006;17(5):407–13.
Yin HS, Mendelsohn AL, Wolf MS, Parker RM, Fierman A, Van Schaick L, et al. Parents’ medication administration errors: role of dosing instruments and health literacy. Arch Pediatr Adolesc Med. 2010;164(2):181–6.
Gigerenzer G, Gaissmaier W, Kurz-Milcke E, Schwartz LM, Woloshin S. Helping doctors and patients make sense of health statistics: toward an evidence-based society. Psychol Sci Public Interest. 2007;8:53–96.
Rothman RL, Housam R, Weiss H, Davis D, Gregory R, Gebretsadik T, et al. Patient understanding of food labels: the role of literacy and numeracy. Am J Prev Med. 2006;31(5):391–8.
Tait AR, Voepel-Lewis T, Zikmund-Fisher BJ, Fagerlin A. The effect of format on parents’ understanding of the risks and benefits of clinical research: a comparison between text, tables, and graphics. J Health Commun. 2010;15:487–501.
Zikmund-Fisher BJ, Smith DM, Ubel PA, Fagerlin A. Validation of the Subjective Numeracy Scale: effects of low numeracy on comprehension of risk communications and utility elicitations. Med Decis Making. 2007;27(5):663–71.
Fagerlin A, Zikmund-Fisher BJ, Ubel PA, Jankovic A, Derry HA, Smith DM. Measuring numeracy without a math test: development of the Subjective Numeracy Scale. Med Decis Making. 2007;27(5):672–80.
Schwartz LM, Woloshin S, Black WC, Welch HG. The role of numeracy in understanding the benefit of screening mammography. Ann Intern Med. 1997;127(11):966–72.
Lipkus IM, Samsa G, Rimer BK. General performance on a numeracy scale among highly educated samples. Med Decis Making. 2001;21(1):37–44.
Schapira MM, Walker CM, Cappaert KJ, Ganschow PS, Fletcher KE, McGinley EL, et al. The numeracy understanding in medicine instrument: a measure of health numeracy developed using item response theory. Med Decis Making. 2012;32(6):851–65.
Schapira MM, Walker CM, Miller T, Fletcher KE, Ganschow PS, Jacobs EA, et al. Development and validation of the numeracy understanding in Medicine Instrument short form. J Health Commun. 2014;19(sup2):240–53.
Weller JA, Dieckmann NF, Tusler M, Mertz CK, Burns WJ, Peters E. Development and testing of an abbreviated numeracy scale: a rasch analysis approach. J Behav Decis Mak. 2013;26(2):198–212.
Cokely ET, Galesic M, Schulz E, Ghazal S, Garcia-Retamero R. Measuring risk literacy: The Berlin numeracy test. Judgment and Decision making. 2012.
McNaughton CD, Cavanaugh KL, Kripalani S, Rothman RL, Wallston KA. Validation of a short, 3-item version of the subjective numeracy scale. Med Decis Making. 2015;35(8):932–6.
Nelson WL, Moser RP, Han PK. Exploring objective and subjective numeracy at a population level: findings from the 2007 Health Information National Trends Survey (HINTS). J Health Commun. 2013;18(2):192–205.
Kiechle ES, Bailey SC, Hedlund LA, Viera AJ, Sheridan SL. Different measures, different outcomes? a systematic review of performance-based versus self-reported measures of health literacy and numeracy. J Gen Intern Med. 2015;30(10):1538–46.
Ciampa PJ, Osborn CY, Peterson NB, Rothman RL. Patient numeracy, perceptions of provider communication, and colorectal cancer screening utilization. J Health Commun. 2010;15(sup3):157–68.
Dolan JG, Cherkasky OA, Li Q, Chin N, Veazie PJ. Should Health Numeracy Be Assessed Objectively or Subjectively? Med Decis Making. 2015.
Ware JE Jr, Gandek B. Overview of the SF-36 health survey and the international quality of life assessment (IQOLA) project. J Clin Epidemiol. 1998;51(11):903–12.
Von Davier M, Gonzalez E, Mislevy R. What are plausible values and why are they useful. IERI Monograph Series. 2009;2(1):9–36.
Yamamoto K, Khorramdel L, Von Davier M. Scaling PIAAC cognitive data. Technical report of the survey of adult skills (PIAAC). 2013:408–40.
Program for the International Assessment of Adult Competencies. PIAAC U.S. 2017 Sample Public Use File Codebook. 2017.
Cornell Survey Resarch Institute. Questionnaire Empire State Poll (ESP). 2019.
Krenzke T, VanDeKerckhove W, Thornton N, et al. U.S. Program for the International Assessment of Adult Competencies (PIAAC) 2012/2014/2017: Main Study, National Supplement, and PIAAC 2017 Technical Report. 2019. Contract No.: NCES 2020–224.
Program for the International Assessment of Adult Competencies (PIAAC). PIAAC Proficiency Levels for Numeracy 2020. https://nces.ed.gov/surveys/piaac/numproficiencylevel.asp.
Gonzales P, Guzmán JC, Partelow L, Pahlke E, Jocelyn L, Kastberg D, et al. Highlights from the Trends in International Mathematics and Science Study (TIMSS), 2003. NCES 2005–005. US Department of Education. 2004.
Lemke M, Sen A, Pahlke E, Partelow L, Miller D, Williams T, et al. International Outcomes of Learning in Mathematics Literacy and Problem Solving: PISA 2003 Results From the US Perspective. Highlights. NCES 2005–003. US Department of Education. 2004.
Kutner M, Greenburg E, Jin Y, Paulsen C. The Health Literacy of America's Adults: Results from the 2003 National Assessment of Adult Literacy. NCES 2006–483. National Center for Education Statistics. 2006.
Ancker JS, Senathirajah Y, Kukafka R, Starren JB. Design features of graphs in health risk communication: a systematic review. J Am Med Inform Assoc. 2006;13(6):608–18.
Davis TC, Federman AD, Bass PF, Jackson RH, Middlebrooks M, Parker RM, et al. Improving patient understanding of prescription drug label instructions. J Gen Intern Med. 2009;24(1):57–62.
Yin HS, Dreyer BP, van Schaick L, Foltin GL, Dinglas C, Mendelsohn AL. Randomized controlled trial of a pictogram-based intervention to reduce liquid medication dosing errors and improve adherence among caregivers of young children. Arch Pediatr Adolesc Med. 2008;162(9):814–22.
The authors gratefully acknowledge discussions and debates in the Health Numeracy Lab Group.
This work (including Drs Ancker and Benda) were supported by National Library of Medicine R01 LM012964.
Ethics approval and consent to participate
This research was approved by the Cornell University Institutional Review Board. Details of the PIAAC ethics and confidentiality procedures are available here: https://www.oecd.org/skills/piaac/publicdataandanalysis/.
Consent for publication
The authors have no relevant competing interests to disclose.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Benda, N.C., Yang, Z., Li, H. et al. Lower objectively and subjectively assessed numeracy are both associated with poorer self-rated health. BMC Res Notes 14, 321 (2021). https://doi.org/10.1186/s13104-021-05737-y