- Research note
- Open Access
Acceptability of smartphone text- and voice-based ecological momentary assessment (EMA) methods among low income housing residents in New York City
BMC Research Notes volume 10, Article number: 517 (2017)
This study aimed to evaluate the acceptability of smartphone-based text message- and voice-based ecological momentary assessment (EMA) methods among a sample of low-income housing residents in New York City. Using data from the community-based NYC Low Income Housing, Neighborhoods and Health Study (n = 112), the acceptability of text message- and voice-based EMA methods were assessed via survey.
Overall, 88.4% of participants reported that they would participate in a study that utilized text message-based EMA. These analyses showed no appreciable differences by sub-groups (p > .05). Overall, 80.2% of participants reported that they would participate in a study that used voice-based EMA. This voice-based method was least acceptable among participants younger than 25 years old compared to participants of all other ages, χ2(2) = 10.107, p = .006 (among the younger participants 60.7% reported “yes” regarding the anticipated acceptability of voice-based EMA and 39.3% reported “no”). Overall, this work suggests that text message- and voice-based EMA methods are acceptable for use among low-income housing residents. However, the association between age and the acceptability of voice-based EMA suggests that these methods may be less suited for younger populations.
Ecological momentary assessment (EMA) includes a range of methods aimed at capturing data on health behaviors and outcomes in real-time from participants as they experience their daily lives . EMA methods were developed in part in response to the limitations of retrospective recall, as self-reported recall data can frequently be unreliable and are often systematically biased . EMA methods recognize that many behaviors and experiences can be affected by context, so data must be collected on a given experience or behavior in its real-life settings for it to be representative . Studies utilizing EMA often involve many repeated measures over varying durations, affording the temporal resolution needed to assess the dynamics of within-subject changes in behavior and experience over time and across context .
These methods have employed various modalities to collect data on behavior in real-time in real world environments, including traditional paper and pencil diaries , palm-top computers , telephones , smartphone applications , and collection of medication compliance data by instrumented pill bottles . EMA methods have been employed in a variety of populations to study a variety of behaviors and contexts, including mood and affect , tobacco, alcohol, and drug use [6, 11, 12], physical activity and sedentary behavior , and eating behaviors . With few exceptions [15,16,17], most studies utilizing these approaches have been conducted among advantaged populations. As an example of an exception, one EMA study was conducted among a sample of Black men who have sex with men . To our knowledge, no studies have assessed the acceptability of these methods among low-income and racial/ethnic minority populations in an intersectional manner.
Low-income populations are disproportionately impacted by adverse health outcomes, including obesity, diabetes, and hypertension . These health outcomes are affected at the individual-level by diet and exercise, which are in turn, can be affected by the characteristics of the built and food environment at the structural level . Given the limitations of self-reported measures utilized in previous studies of neighborhood contexts and cardiovascular health behaviors in this population [20,21,22], a more nuanced understanding of these behaviors and contexts may be gained by employing EMA methods. However, previous research has noted that low-income adults are less likely to participate in research studies and community programs [23, 24], underscoring the need to assess the acceptability and feasibility of various EMA methods prior to implementing them. As such, this study aims to evaluate the acceptability of text message- and voice-based EMA methods among a sample of low-income housing residents in New York City (NYC) who were predominantly racial/ethnic minorities. This study additionally aims to assess differences in the acceptability of these methods by demographic sub-groups (e.g., race/ethnicity, education level).
Data come from the NYC Low Income Housing, Neighborhoods, and Health Study, a community-based study of neighborhood environments and cardiovascular health among low income housing residents, which has been described elsewhere in detail [25, 26]. Briefly, the overall study included 120 low-income residents, most of whom reported living in public housing. This was a convenience sample, as participants were recruited through community-based approaches, which included distributing flyers outside four selected public housing developments in the Manhattan and Queens boroughs of NYC. In addition, we recruited through flyers posted and circulated by community-based organizations that work with low-income individuals, flyers posted in community locations, and through word of mouth. Inclusion criteria included reporting living in low-income (e.g., public) housing in NYC, being 18 years old or older, being able to speak English, self-reporting not being pregnant, self-reporting no restrictions to usual physical activity, and being willing to wear a global positioning system (GPS) device for 1 week.
Self-administered survey measures
Acceptability of smartphone-based ecological momentary assessment (EMA) methods
The acceptability of two different EMA methods were assessed with two items: “Would you participate in a study that sent you texts via a smartphone asking you questions about your current mood, surroundings, and feelings?” (text message-based EMA) and “Would you participate in a study that called you to ask questions about your current mood, surroundings, and feelings?” (voice-based EMA). Response options for these two items were “Yes” and “No.”
Cell phone ownership and use
Cell phone use was assessed with one item reading, “Have you previously used a cell phone?” with two response options (yes, no). Cell phone ownership was assessed with one item reading, “Do you have a cell phone?” with two response options (yes, no). If an individual reported cell phone ownership, they were asked, “Do you own a smartphone?” with two response options (yes, no). If an individual reported smartphone ownership, they were asked “What is the operating system?” with four response options (Apple, Android, Blackberry, Other).
Blood pressure and body mass index (BMI)
The blood pressure and BMI protocols have been described in detail elsewhere and were collected at our research office the day the survey was administered [27, 28]. In brief, participant height and weight were measured to the nearest tenth of a centimeter and to the nearest tenth of a kilogram. These measurements were then used to calculate BMI using standard formulas. BMI under 18.5 were classified as underweight, between 18.5 and 24.9 were classified as normal weight, between 25.0 and 29.9 were classified as overweight, and 30.0 and over were classified as obese. Blood pressure was measured a single time in the seated position with the participants’ legs uncrossed and arms outstretched after the participants had been seated for 15–30 s, using a Welch Allyn Vital Signs 300 monitor. Measured hypertension was classified as a systolic pressure ≥ 140 mmHg or a diastolic pressure ≥ 90 mmHg. Pre-hypertension was classified as a systolic pressure between 120 and 139 mmHg or a diastolic pressure between 80 and 89 mmHg. Normal blood pressure was classified as a systolic pressure below 120 mmHg and a diastolic pressure below 80 mmHg .
Participants reported age (years), gender (male, female), race/ethnicity (White, Black, Hispanic, Asian, Other), household income (less than $25,000; $25,000 to $49,999; $50,000 to $74,999; $75,000 or greater), educational attainment (less than 12th grade, high school or GED, some college, bachelor’s degree, graduate degree), employment status (working full-time, working part-time, not working, retired, in school), and health insurance status (yes, no). This information was collected via survey.
The analytical sample was restricted to participants who answered both EMA acceptability items (n = 112), representing 94.2% of the overall sample. Descriptive statistics (e.g. frequencies) were calculated for all variables. Differences in acceptability of each of the EMA methods by socio-demographic characteristics and health status were assessed using Chi square tests. Statistical significance was set at p < .05.
The demographics of the sample (n = 112) are reported in Table 1. The average age of the sample was 38.7 years old (SD = 14.3). More than half (55.4%) of the participants were female. Most participants (71.5%) identified their race/ethnicity as Black/African American or Hispanic/Latino. In addition, most (71.4%) reported an income lower than $25,000. A majority (71.4%) completed high school or some equivalent designation. Few reported working full or part time (34.8%), where 51.8% reported not currently working. Over two-thirds of the sample (70.5%) had a BMI classified as overweight or obese, where 42.9% were obese. Two-thirds (67.0%) had a blood pressure classified as hypertensive or pre-hypertensive, where 31.3% were hypertensive.
Cell phone use and ownership
Almost all participants (96.4%) reported previously using a cell phone. Most participants (92.0%) owned a cell phone. Among those who reported cell phone ownership (n = 103), 56.3% owned a smartphone. Among those who reported owning smartphone (n = 58), 63.8% reported owning Android-based smartphones, 25.9% reported owning Apple-based smartphones, 1.7% reported owning Blackberry-based smartphones, and 8.6% reported owning a smartphone that used another operating system.
Acceptability of ecological momentary assessment methods
Overall, 88.4% of participants reported that they would participate in a study that sent them text messages via smartphone asking them questions about their current mood, surroundings, and feelings. Differences in acceptability of text message-based EMA methods by socio-demographic characteristics, health status, and cell phone ownership and use are displayed in Table 2. These analyses showed no appreciable differences by sub-groups (p > .05).
Overall, 80.2% of participants reported that they would participate in a study that called them to asked them questions about their current mood, surroundings, and feelings. Differences in acceptability of voice-based EMA methods by socio-demographic characteristics, health status, and cell phone ownership and use are shown in Table 3. These methods were less acceptable among individuals younger than 25 years old (60.7%) compared to individuals of other ages, χ2(2) = 10.107, p = .006 (among the younger participants 60.7% reported “yes” regarding the anticipated acceptability of voice-based EMA and 39.3% reported “no”). Additionally, these methods were less acceptable among individuals who reported currently being in school compared all others, χ2(4) = 15.202, p = .004 (28.6% versus 71.4%). These were the only observed differences in acceptability and we note that some cells have very small sample sizes.
Overall, this work suggests that smartphone-based text message- and voice-based EMA methods are acceptable for use among low-income housing residents who were predominantly racial/ethnic minorities. Specifically, the lack of significant variations in acceptability of text message-based EMA suggests that these methods can be implemented in diverse populations of low income housing residents regardless of socio-demographic characteristics and cellphone ownership and use, which has tremendous implications for EMA research in this unique population. However, the association between age and the acceptability of voice-based EMA suggests that these methods may be better suited for older populations of low-income housing residents. To our knowledge, this is the first study to examine the acceptability of various EMA methods among low-income housing residents. As discussed previously, overall, the majority of studies using EMA approaches have been conducted among advantaged populations.
Future research would benefit from the utilization of these highly innovative methods among this population to overcome the limitations of traditional survey-based assessments of poor health behaviors, which are highly prevent among low incoming housing residents [20,21,22, 28, 30]. For example, EMA via palmtop computers was recently used to collect real-time information about participants’ environment and eating patterns to predict overeating that could lead to weight gain among a sample of 39 undergraduate women . EMA methods administered via smartphones have been paired with accelerometer use among a sample of 110 adults to examine physical activity and sedentary behaviors . Beyond addressing traditional limitations of research, EMA methods offer new avenues for engagement and inquiry for collecting data using mobile technologies real-time and tailoring behavioral health interventions to specific spatio-temporal contexts.
This study provides evidence for the acceptability of EMA methods among low-incoming housing residents. The lack of significant variations in acceptability of text message-based EMA suggests that these methods can be implemented in diverse populations of low-income housing residents regardless of socio-demographic characteristics and cellphone ownership and use. However, the association between age and the acceptability of voice-based EMA suggests that these methods may be less suited for younger populations.
While our study has several important strengths (including a sample of predominantly racial/ethnic minority low-income housing residents), our study also has limitations. First, we note that this was a relatively small convenience sample of low-income housing residents in NYC who perhaps were motivated to engage in research. Consequently, our findings might not be generalizable to all low-income populations, including those in other geographic regions especially those in rural geographies. However, our sample included a diverse sample of low-income adults across different NYC neighborhoods . Also, given that 20% of Americans now live in the 100 largest cities, and more than 70% of Americans living in urban areas with urbanization still on the rise, the relevance is large and growing . Second, our study was limited to English speaking low-income housing residents. As such, our results may not be generalizable to non-English speaking low-income housing populations. In addition, social desirability bias might be a concern. These data are 3 years old, so there may be have been changes overtime in the acceptability of EMA methods (perhaps a wider acceptance).
body mass index
ecological momentary assessment
New York City
Stone AA, Shiffman S. Ecological momentary assessment (EMA) in behavorial medicine. Ann Behav Med. 1994;16(3):199–202.
Shiffman S, Hufford M, Hickcox M, Paty JA, Gnys M, Kassel JD. Remember that? A comparison of real-time versus retrospective recall of smoking lapses. J Consult Clin Psychol. 1997;65(2):292–300. https://doi.org/10.1037/0022-006X.65.2.292.a.
Shiffman S, Stone AA. Introduction to the special section: ecological momentary assessment in health psychology. Health Psychol. 1998;17(1):3–5. https://doi.org/10.1037/h0092706.
Schwartz JE, Stone AA. Strategies for analyzing ecological momentary assessment data. Health Psychol. 1998;17(1):6–16. https://doi.org/10.1037/0278-6126.96.36.199.
Green AS, Rafaeli E, Bolger N, Shrout PE, Reis HT. Paper or plastic? Data equivalence in paper and electronic diaries. Psychol Methods. 2006;11(1):87–105. https://doi.org/10.1037/1082-989X.11.1.87.
Shiffman S, Balabanis MH, Gwaltney CJ, Paty JA, Gnys M, Kassel JD, et al. Prediction of lapse from associations between smoking and situational antecedents assessed by ecological momentary assessment. Drug Alcohol Depend. 2007;91(2):159–68. https://doi.org/10.1016/j.drugalcdep.2007.05.017.
Collins RL, Kashdan TB, Gollnisch G. The feasibility of using cellular phones to collect ecological momentary assessment data: application to alcohol consumption. Exp Clin Psychopharmacol. 2003;11(1):73–8. https://doi.org/10.1037/1064-12188.8.131.52.
Runyan JD, Steenbergh TA, Bainbridge C, Daugherty DA, Oke L, Fry BN. A smartphone ecological momentary assessment/intervention “app” for collecting real-time data and promoting self-awareness. PLoS ONE. 2013;8(8):e71325. https://doi.org/10.1371/journal.pone.0071325.
Haberer JE, Kahane J, Kigozi I, Emenyonu N, Hunt P, Martin J, et al. Real-time adherence monitoring for HIV antiretroviral therapy. AIDS Behav. 2010;14(6):1340–6. https://doi.org/10.1007/s10461-010-9799-4.
Ebner-Priemer UW, Trull TJ. Ecological momentary assessment of mood disorders and mood dysregulation. Psychol Assess. 2009;21(4):463–75. https://doi.org/10.1037/a0017075.
Litt MD, Cooney NL, Morse P. Ecological momentary assessment (EMA) with treated alcoholics: methodological problems and potential solutions. Health Psychol. 1998;17(1):48–52. https://doi.org/10.1037/0278-6184.108.40.206.
Hopper JW, Su Z, Looby AR, Ryan ET, Penetar DM, Palmer CM, et al. Incidence and patterns of polydrug use and craving for ecstasy in regular ecstasy users: an ecological momentary assessment study. Drug Alcohol Depend. 2006;85(3):221–35. https://doi.org/10.1016/j.drugalcdep.2006.04.012.
Biddle SJ, Gorely T, Marshall SJ, Cameron N. The prevalence of sedentary behavior and physical activity in leisure time: a study of Scottish adolescents using ecological momentary assessment. Prev Med. 2009;48(2):151–5. https://doi.org/10.1016/j.ypmed.2008.10.025.
Haedt-Matt AA, Keel PK. Revisiting the affect regulation model of binge eating: a meta-analysis of studies using ecological momentary assessment. Psychol Bull. 2011;137(4):660–81. https://doi.org/10.1037/a0023660.
Yang C, Linas B, Kirk G, Bollinger R, Chang L, Chander G, et al. Feasibility and acceptability of smartphone-based ecological momentary assessment of alcohol use among African American men who have sex with men in Baltimore. JMIR mHealth uHealth. 2015;3(2):e67. https://doi.org/10.2196/mhealth.4344.
Freedman MJ, Lester KM, McNamara C, Milby JB, Schumacher JE. Cell phones for ecological momentary assessment with cocaine-addicted homeless patients in treatment. J Subst Abuse Treat. 2006;30(2):105–11. https://doi.org/10.1016/j.jsat.2005.10.005.
Soong A, Chen JC, Borzekowski DL. Using ecological momentary assessment to study tobacco behavior in urban india: there’s an app for that. JMIR Res Protoc. 2015;4(2):e76. https://doi.org/10.2196/resprot.4408.
Winkleby MA, Jatulis DE, Frank E, Fortmann SP. Socioeconomic status and health: how education, income, and occupation contribute to risk factors for cardiovascular disease. Am J Public Health. 1992;82(6):816–20. https://doi.org/10.2105/ajph.82.6.816.
Macintyre S, Ellaway A. Ecological approaches: Rediscovering the role of the physical and social environment. In: Berkman LF, Kawachi I, editors. Social epidemiology. New York: Oxford University Press; 2000. p. 332–48.
Bennett GG, McNeill LH, Wolin KY, Duncan DT, Puleo E, Emmons KM. Safe to walk? Neighborhood safety and physical activity among public housing residents. PLoS Med. 2007;4(10):e306. https://doi.org/10.1371/journal.pmed.0040306.
Heinrich KM, Lee RE, Regan GR, Reese-Smith JY, Howard HH, Keith Haddock C, et al. How does the built environment relate to body mass index and obesity prevalence among public housing residents? Am J Health Promot. 2008;22(3):187–94. https://doi.org/10.4278/ajhp.22.3.187.
Lovasi GS, Hutson MA, Guerra M, Neckerman KM. Built environments and obesity in disadvantaged populations. Epidemiol Rev. 2009;31:7–20. https://doi.org/10.1093/epirev/mxp005.
Heinrichs N, Bertram H, Kuschel A, Hahlweg K. Parent recruitment and retention in a universal prevention program for child behavior and emotional problems: barriers to research and program participation. Prev Sci. 2005;6(4):275–86. https://doi.org/10.1007/s11121-005-0006-1.
Jin-Hyung L, Scott D, Floyd MF. Structural inequalities in outdoor recreation participation: a multiple hierarchy stratification perspective. J Leisure Res. 2001;33(4):427.
Duncan DT, Regan SD, Shelley D, Day K, Ruff RR, Al-Bayan M, et al. Application of global positioning system methods for the study of obesity and hypertension risk among low-income housing residents in New York City: a spatial feasibility study. Geospatial Health. 2014;9(1):57–70. https://doi.org/10.4081/gh.2014.6.
Duncan DT, Regan SD. Mapping multi-day GPS data: a cartographic study in NYC. J Maps. 2016;12(4):668–70. https://doi.org/10.1080/17445647.2015.1060180.
Duncan DT, Ruff RR, Chaix B, Regan SD, Williams JH, Ravenell J, et al. Perceived spatial stigma, body mass index and blood pressure: a global positioning system study among low-income housing residents in New York City. Geospatial Health. 2016;11(2):164–73. https://doi.org/10.4081/gh.2016.399.
Williams JH, Duncan DT, Cantor J, Elbel B, Ogedegbe G, Ravenell J. A comparison of self-reported and measured blood pressure status among low-income housing residents in New York City. J Health Disparities Res Pract. 2016;9(4):153–65.
Giles TD, Berk BC, Black HR, Cohn JN, Kostis JB, Izzo JL, et al. Expanding the definition and classification of hypertension. J Clin Hypertens. 2005;7(9):505–12. https://doi.org/10.1111/j.1524-6175.2005.04806.x.
Digenis-Bury EC, Brooks DR, Chen L, Ostrem M, Horsburgh CR. Use of a population-based survey to describe the health of Boston public housing residents. Am J Public Health. 2008;98(1):85–91. https://doi.org/10.2105/ajph.2006.094912.
Thomas JG, Doshi S, Crosby RD, Lowe MR. Ecological momentary assessment of obesogenic eating behavior: combining person-specific and environmental predictors. Obesity (Silver Spring). 2011;19(8):1574–9. https://doi.org/10.1038/oby.2010.335.
Dunton GF, Liao Y, Kawabata K, Intille S. Momentary assessment of adults’ physical activity and sedentary behavior: feasibility and validity. Front Psychol. 2012;3(260):260. https://doi.org/10.3389/fpsyg.2012.00260.
United States Census Bureau. Annual estimates of the resident population for incorporated places over 50,000, ranked by July 1, 2011 population. Washington, DC: United States Census Bureau; 2012.
DTD conceived the study, drafted the manuscript, interpreted the results and was principal investigator on the parent study; WCG conducted the statistical analysis, drafted the manuscript, participated in the data collection and interpreted the results; JHW participated in the data collection and critically revised the manuscript regarding important intellectual content. BE critically revised the manuscript regarding important intellectual content. All authors have given final approval of the version to be published and are publicly responsible for its contents. All authors read and approved the final manuscript.
We thank the participants for engaging in this research. Finally, thanks to all research assistants, study staff and participants involved in the NYC Low-Income Housing, Neighborhoods and Health Study.
The authors declare that they have no competing interests.
Availability of data and materials
The datasets generated and/or analyzed during the current study are available from the corresponding author upon reasonable request.
Consent for publication
All participants provided an informed written consent and no personal details or images were collected.
Ethics approval and consent to participate
All protocols were reviewed and approved by the Institutional Review Board at the New York University School of Medicine. Written informed consent was obtained from all participants prior to data collection.
The NYC Low-Income Housing, Neighborhoods and Health Study was supported by the New York University-Health and Hospitals Corporation Clinical and Translational Science Institute (NYU-HHC CTSI) Pilot Project Awards program (Principal Investigator: Dr. Dustin Duncan). The NYU-HHC CTSI is support in part by the National Center for Advancing Translation Sciences of the National Institutes of Health (Grant Number: UL1TR000038; Dr. Bruce Cronstein, Principal Investigator and Dr. Judith Hochman, co-Principal Investigator).
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.