- Research note
- Open Access
Validity and reliability of a dish-based semi-quantitative food frequency questionnaire for assessment of energy and nutrient intake among Iranian adults
BMC Research Notes volume 13, Article number: 95 (2020)
This study aimed to assess the validity and reliability of a dish-based, semi-quantitative food frequency questionnaire (DFFQ) for epidemiological studies in Iran. The DFFQ included 142 items (84 foods and 58 mixed dishes) which was filled in by 230 adults (110 men). All participants completed two separate DFFQs with a 6 months interval as well as six 24-h recalls, each month. Dietary biomarkers and anthropometric measurements were made. The validity was evaluated by comparing the DFFQ against 24-h dietary recalls and dietary biomarkers, including serum retinol and beta-carotene. Reliability was evaluated using intra-class correlation coefficient (ICC) and validity was determined by unadjusted and energy adjusted correlation coefficients (CC), de-attenuated CC, and cross-classification analyses.
ICC for reliability ranged between 0.42 and 0.76. De-attenuated CC for the FFQ and the 24-h recalls ranged between 0.13 and 0.54 (Mean = 0.38). The de-attenuated CC between the DFFQ and plasma levels of retinol and beta-carotene were 0.58 (P = 0.0001) and 0.40 (P = 0.0001), respectively. Cross-classification analysis revealed that on average 73% were correctly classified into same or adjacent quartiles and 5% were classified in opposite quartiles.
Dietary factors play an important role in development of non-communicable diseases (NCDs) . To have a better understanding of any relationship between diet and NCDs, evaluation of habitual intake over a long period of time is needed . Assessment of dietary habits is crucial in large, prospective epidemiological studies which provide a clear understanding of health status. Furthermore, having reliable dietary data allows researchers to examine the relationship between food and nutrients consumption and the susceptibility to diseases . However, since eating behavior is a cultural issue [4, 5] a dietary assessment tool developed for one population may not be appropriate for the other.
Food frequency questionnaires (FFQs) are widely used to assess dietary patterns in different societies. FFQs, especially food-based FFQs (FFFQs), are reliable tools which can be easily used in countries whose dietary patterns are based on single foods rather than mixed dishes [6,7,8]. Dish-based FFQs (DFFQ) may have several advantages over FFFQs in the societies whose dietary practices are different from western countries. It is documented that compared to FFFQs a DFFQ calculated antioxidant vitamins, phytochemicals and fatty acids more accurately .
Furthermore, a DFFQ can, facilitate evaluation of dietary intake over a long period, since it is much easier to recall the frequency intake of mixed dishes rather than individual ingredients included in them .
A number of questionnaires, mainly FFFQ, have been developed in both Western countries [6,7,8, 10,11,12] and Asian countries [13,14,15,16], including Iran [4, 17,18,19]. However, Iranian dietary pattern is typically characterized by a plenty of mixed dishes. The dishes contain several ingredients which are cooked by different methods.
Therefore, it is very challenging, if not impossible, to estimate usual intake of single ingredients through an FFFQ. Several international studies have developed DFFQs for epidemiological studies [3, 16, 20]. However, there were some limitations namely not considering seasonal variations of foods , the generalizability [16, 20], and the gold standards affairs which have limited their usage as a valid tool. To the best of our knowledge, no earlier study in Iran has developed a validated DFFQ. Despite presence of two formerly developed DFFQs in Iran their applicability are limited due to validity issues or the specific target groups for which the FFQs were designed [21, 22].
The purpose of the current study was to evaluate the validity and reliability of a novel dish-based, semi-quantitative FFQ  to be used for epidemiological studies in Iran
Study design and participants
A total of 230 male and female adults whose ages ranged between 18 and 65 years, were non-smokers (due to probable effect of smoking on biomarkers level), non-pregnant and/or nursing, living in Tehran, and were not on a specific diet were recruited in the study. Samples size was calculated based on Willett’s recommendation. According to him 100 to 200 participants seems to be a reasonable sample size for such a validation study . We supposed five age subgroups including 18 to 30, 31 to 40, and 41 to 50 years old with 50 participants in each of them and 51-years old to higher with 75 participants. Details of sampling process is presented in Additional file 1: Figure S1. The convenience sample consisted of volunteers of neighboring banks, staff of NNFTRI, Taxi Organization of Tehran, a Local Community in district 21 of Tehran and ordinary people as well as university students who were invited directly in their workplaces or were informed by the participants of the study. All participants were paid cash donations for their contribution to the study.
Food frequency questionnaire (FFQ)
To evaluate validity and reliability of the FFQ several steps were taken. Summary of steps taken for the validation of the DFFQ are presented in Fig. 1.
I-Development of the DFFQ
A previously developed dish-based semi-quantitative FFQ was used to assess monthly subjects’ habitual dietary intake . The 142 items questionnaire included 84 food items and 58 mixed dishes and was completed by trained interviewers through face to face interview with the participants. Visual aids including locally used utensils were provided to help the participants to understand the concept of the portion sizes. All subjects were supposed to complete 2 individual FFQs with a 6 month interval, one in winter and another in summer of 2016.
In this section, data from the FFQ was evaluated against six 24-h recalls and dietary biomarkers.
II-I-24-h dietary recall
Six 24-h recalls were collected as a reference method almost every month. Multiple probing questions were used to complete each 24-h dietary recall including diverse portion size descriptions, food models and detailed food preparation and cooking methods by interviewing the person responsible for cooking.
Five milliliter of fasting antecubital venous blood was taken from all participants. After keeping at room temperature (RT) for 20–30 min at dark, blood samples were centrifuged at 800g for 20 min. Then the recovered sera were aliquoted in fresh microtubes and kept at − 80 °C until the day of analysis. Serum concentrations of β-carotene and retinol were measured in two occasions (Fig. 1) using high performance liquid chromatography with the method described elsewhere .
Height and weight were measured using standard procedures . Weight was measured with minimal clothing and without shoes using a digital scale (Soehnle brand) ± 100 g. Height was measured using a non-elastic measuring tape (STABILA brand) while standing against a wall, bare-footed with the scapula in normal circumstance to the nearest 1 cm. Body mass index (BMI) was calculated through dividing weight (kg) by height in meters squared (m2).
Outcome measures included mean energy and nutrient intake values obtained from the DFFQs and 24-h-recalls.
Demographic characteristics of the study population were tabulated. Normality of distribution was evaluated using the Kolmogorov–Smirnov test. Firstly, de-attenuated correlation coefficients was calculated using the approach described by Rosner , to take into account within-person variations caused by day-to-day fluctuations, through following formula: Rtrue = r_observed√ (1 + λx/nx) (1 + λy/ny) , in which λx represents the ratio of the within-and between-person variances for x, and nx represents the number of replicates for the x variable. For this study, nx = 2. Within-and between-person variations (λx) were obtained from its respective intra-class correlations for nutrient intakes estimated by the two FFQs and, ny = 6. Within-and between-person variations (λy) were obtained from its respective intra-class correlations for nutrient intakes estimated by the six 24-h recalls. Reproducibility was measured by Intra-class Correlation Coefficient (ICC).
In the second step, since total energy intake can introduce extraneous variation in recorded food intake, intake estimates were adjusted for total energy intake using the residual method .
Finally the degree of agreement based on quartile categorization of nutrients intakes according to the DFFQs and the 24-h recalls was evaluated by examining the proportion of subjects classified by the reference method and DFFQs that fell into the same, adjacent, or extreme quartiles.
Characteristics of study participants are presented in Additional file 2.
Findings of correlation coefficient between DFFQs and 24-h recalls indicate that, compared with recalls, the DFFQs significantly overestimated energy and nutrients intakes (Table 1).
The unadjusted Spearman’s correlations for nutrient intakes ranged from 0.091 for folate to 0.51 for protein and thiamin. Energy adjustments slightly improved the correlations for some nutrients including dietary fiber and thiamin.
The de-attenuation correction improved the correlation coefficients for energy, macronutrients and most of micronutrients, statistically significant (P < 0.001).
The mean correlation coefficients between the DFFQs and serum concentrations of retinol and β-carotene were 0.26 (P = 0.0001) and 0.13 (P = 0.047), respectively which was improved by de-attenuation to 0.58 and 0.40, respectively (P < 0.001) (Table 1).
Cross classification analysis of two methods into the same and adjacent quartiles of nutrients intake ranged between 67.7% for dietary fiber to 82.6% for thiamin. On average 73% were correctly classified into same or adjacent quartiles and a mean of 5% were classified in opposite quartile.
In order to evaluate the reproducibility of the DFFQ, energy and nutrients intakes obtained from DFFQ-1 were compared with those from DFFQ-2 ranged between 0.23 (carbohydrate) to 0.76 (energy and thiamin) (Table 2).
In the present study, validation and reliability of a dish-based semi-quantitative FFQ was evaluated. The inclusion of dishes in the list of FFQ food items may improve accuracy and precision of the collected data in two ways [3, 22]. First, NCDs are linked to culture-specific cooking methods and ingredients [3, 30,31,32]. Second, in the FFQs in which dishes are not included people may not report invisible parts of a mixed dish because they are neither engaged in their cooking process nor can see the ingredients of different recipes. Consequently, they cannot remember the consumption of mentioned foods .
Generally, FFQs are used for ranking individuals according to food or nutrient intake rather than for estimating absolute amounts of intake. In this regard, we used cross-classification analysis whose results for the majority of nutrients were promising, in which approximately more than 60% of participants were classified in the correct quartiles, which is consistent with other studies [12, 20, 26].
In the current study we found that the DFFQ overestimated the consumption of energy and nutrients compared to the 24-h recalls. The issue of overestimation of nutrient intake by FFQs has been reported in other studies . However, if this overestimation is due to a systematic error it can be modified using a correcting factor.
Unadjusted Spearman’s correlation ranged between 0.23 and 0.52, which indicates a fair to medium correlation [12, 34, 35]. The reported correlations are similar to those obtained in other similar validation studies. According to a study carried out in Chile the values ranged between 0.26 and 0.47 . In another study in Colombia reported correlations ranged between 0.18 and 0.38 in urban areas and between 0.00 and 0.31 in rural areas . In the current study, energy adjustment increased the correlation coefficients for some nutrients but decreased them for the majority of other nutrients. It is documented when the source of variability of nutrient consumption is related to energy intake, energy adjustment increases values of correlation coefficients, however, it will be decreased when it is due to systematic errors (overestimation and underestimation) . In our study, the lower correlation values found in some nutrients may indicate that the DFFQ, to some extent, systematically overestimated intake of those nutrients. However, error in overestimation is expected in the FFQ. Similar to other studies [12, 15, 37] energy adjustment did not improve the crude correlations in our study. Compared with other studies in Denmark , Mexico , Canada , France , Ecuador , Valencia , and Scotland , our DFFQ had stronger correlation for total energy, protein, fat, thiamin, carbohydrate, calcium, iron, cholesterol, riboflavin, dietary fiber, zinc, vitamin C, and vitamin E, after the de-attenuation correction.
Compared to other studies reproducibility of the current DFFQ was reasonably acceptable. In other studies the reported correlation coefficients for reliability ranged from 0.06 (for γ-Tocopherol) to 0.31 (for vitamins A and C) , and 0.62 (for protein) to 0.88 (for calcium) . In an article the correlation coefficients for FFQs ranged from 0.4 to 0.8 (2009) .
To the best of our knowledge this questionnaire is the first valid and reliable DFFQ in Iran which can be used in various epidemiological studies and settings. It ranks people based on their dietary intake and whereby at risk groups can be screened. Effectiveness of nutritional interventions can be evaluated by it, as well. Further studies will reveal the weaknesses of the DFFQ which in turn help in improving its validity.
The proposed DFFQ in this study showed a relatively acceptable reproducibility and validity in ranking the participants according to energy and nutrients intakes. Therefore it can be used as a reliable tool in epidemiological studies. The DFFQ can evaluate dietary intake among adults in different settings. It can screen out nutritional risk factors and evaluate the effectiveness of interventions, as well.
Since the source of error between the two instruments of food frequency questionnaire and 24-h recall is common and both of them are prone to recall bias the recommended method for such studies is dietary record . However, as the interviewers needed training and we had the time limit, we skipped doing it.
The generalizability of the DFFQ may be limited due to the fact that the study participants were a convenience sample residing in Tehran. The DFFQ needs to be modified and validated according to various climates and food cultures in the country. Therefore the current DFFQ can be used only in climates and food cultures similar to Tehran. Therefore, for studies undertaken in other climates the DFFQ should be modified accordingly.
Availability of data and materials
The dataset supporting the conclusions of this article can be made available upon request after approval by the authors.
24-h dietary recall
Food frequency questionnaire
Dish-based food frequency questionnaire
Intraclass correlation coefficient
Body mass index
WHO. Global action plan for the prevention and control of noncommunicable diseases 2013–2020. Geneva: World Health Organization; 2013.
Ebrahim S. Chronic diseases and calls to action. Int J Epidemiol. 2008;37(2):225–30.
Kim YO, et al. A study testing the usefulness of a dish-based food-frequency questionnaire developed for epidemiological studies in Korea. Br J Nutr. 2008;101(8):1218–27.
Esfahani FH, et al. Reproducibility and relative validity of food group intake in a food frequency questionnaire developed for the Tehran Lipid and Glucose Study. J Epidemiol. 2010;20(2):150–8.
Andersen LF, Bere E, Kolbiornsen. Validity and reproducibility of self-reported intake of fruit and vegetable among 6th graders. Eur J Clin Nutr. 2004;58:771–7.
Palacios C, et al. Validation and reproducibility of a semi-quantitative FFQ as a measure of dietary intake in adults from Puerto Rico. Public Health Nutr. 2015;18(14):2550–8.
Korkalo L, et al. Parents’ reports of preschoolers’ diets: relative validity of a food frequency questionnaire and dietary patterns. Nutrients. 2019;11(1):159.
Vilela S, et al. Evaluation of a short food frequency questionnaire for dietary intake assessment among children. Eur J Clin Nutr. 2019;73(5):679.
Al Jawaldeh A, Osman D, Tawfik A, World Health Organization. Food and nutrition surveillance systems: a manual for policy-makers and programme managers. 2014.
Fangupo LJ, et al. Relative validity and reproducibility of a food frequency questionnaire to assess energy intake from minimally processed and ultra-processed foods in young children. Nutrients. 2019;11(6):1290.
Cade J, et al. Development, validation and utilisation of food-frequency questionnaires—a review. Public Health Nutr. 2002;5(4):567–87.
Dehghan M, et al. Validation of a semi-quantitative food frequency questionnaire for Argentinean adults. PLoS ONE. 2012;7(5):e37958.
Almajwal A, et al. Development of food frequency questionnaire (FFQ) for the assessment of dietary intake among overweight and obese Saudi young children. Nutrire. 2018;43(1):29.
Kobayashi T, et al. Development of a food frequency questionnaire to estimate habitual dietary intake in Japanese children. Nutr J. 2010;9(1):17.
Xia W, et al. Reproducibility and relative validity of a food frequency questionnaire developed for female adolescents in Suihua, North China. PLoS ONE. 2011;6(5):e19656.
Yum J, Lee S. Development and evaluation of a dish-based semiquantitative food frequency questionnaire for Korean adolescents. Nutr Res Pract. 2016;10(4):433–41.
Malekahmadi M, et al. Development, validity, and reliability of a food frequency questionnaire for antioxidants in elderly Iranian people. J Res Med Sci. 2016;21:14.
Malekshah AF, et al. Validity and reliability of a new food frequency questionnaire compared to 24 h recalls and biochemical measurements: pilot phase of Golestan cohort study of esophageal cancer. Eur J Clin Nutr. 2006;60(8):971–7.
Mohammadifard N, et al. Validity and reproducibility of a food frequency questionnaire for assessment of fruit and vegetable intake in Iranian adults. J Res Med Sci. 2011;16(10):1286.
Lin P-I, et al. Validation of a dish-based semiquantitative food questionnaire in rural Bangladesh. Nutrients. 2017;9(1):49.
Nematy M, et al. Validity and reproducibility of Iranian food frequency questionnaire. Switz Res Park J. 2014;102(12):2137–46.
Keshteli A, et al. A dish-based semi-quantitative food frequency questionnaire for assessment of dietary intakes in epidemiologic studies in Iran: design and development. Int J Prev Med. 2014;5(1):29–36.
Mills VC, et al. Relative validity and reproducibility of a food frequency questionnaire for identifying the dietary patterns of toddlers in New Zealand. J Acad Nutr Diet. 2015;115(4):551–8.
Willett W. Nutritional epidemiology. Oxford: Oxford University Press; 2012.
Amini M, et al. Development of a dish-based semi-quantitative food frequency questionnaire for Iranian population. Med J Islam Repub Iran. in press (accepted 18 Aug 2019).
Rodriguez C, et al. Development and validation of a food frequency questionnaire to estimate intake among children and adolescents in Urban Peru. Nutrients. 2017;9(10):1121.
WHO Multicentre Growth Reference Study Group. WHO child growth standards based on length/height, weight and age. Acta Paediatr. 2006;Suppl 450:76–85. http://www.who.int/childgrowth/training/module_b_measuring_growth.pdf. Accessed 23 Mar 2019.
Willett WC, et al. Reproducibility and validity of a semiquantitative food frequency questionnaire. Am J Epidemiol. 1985;122(1):51–65.
Ahmad Esmaillzadeh LA. Food intake patterns may explain the high prevalence of cardiovascular risk factors among iranian women. J Nutr. 2008;138:1469–75.
Aung W, et al. Fatty acid profiles of various vegetable oils and the association between the use of palm oil vs. peanut oil and risk factors for non-communicable diseases in Yangon region, Myanmar. Nutrients. 2018;10(9):1193.
Li D. Omega-3 polyunsaturated fatty acids and non-communicable diseases: meta-analysis based systematic review. Asia Pac J Clin Nutr. 2015;24(1):10.
Onyango EM, Onyango BM. The rise of noncommunicable diseases in Kenya: an examination of the time trends and contribution of the changes in diet and physical inactivity. J Epidemiol Glob Health. 2018;8(1):1–7.
Moreno-Rojas R, Romero-Saldaña M, Molina-Recio G. Validation of a food frequency questionnaire for the indigenous Épera-Siapidara people in Ecuador. Nutr Hosp. 2017;34(6):1368–75.
Mateos LG, et al. Validación de una encuesta para evaluar el estado nutricional y los estilos de vida en las etapas preconcepcional, embarazo y lactancia. Rev Esp Nutr Comunitaria. 2013;19(2):105–13.
Elorriaga N, et al. Validation of a self-administered FFQ in adults in Argentina, Chile and Uruguay. Public Health Nutr. 2015;18(1):59–67.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;159–174.
Liu L, et al. Assessing the validity of a self-administered food-frequency questionnaire (FFQ) in the adult population of Newfoundland and Labrador, Canada. Nutr J. 2013;12:49.
Bjerregaard AA, et al. Relative validity of a web-based food frequency questionnaire for Danish adolescents. Nutr J. 2018;17(1):9.
Denova-Gutiérrez E, et al. Validity of a food frequency questionnaire to assess food intake in Mexican adolescent and adult population. Salud Pública de México. 2016;58:617–28.
Deschamps V, et al. Reproducibility and relative validity of a food-frequency questionnaire among French adults and adolescents. Eur J Clin Nutr. 2009;63(2):282.
Villena-Esponera MP, et al. Validation of a Food Frequency Questionnaire for the indigenous Epera-Siapidara people in Ecuador. Nutr Hosp. 2017;34(5):1368–75.
Vioque J, et al. Reproducibility and validity of a food frequency questionnaire designed to assess diet in children aged 4-5 years. PLoS ONE. 2016;11(11):e0167338.
Hollis JL, et al. Assessing the relative validity of the Scottish Collaborative Group FFQ for measuring dietary intake in adults. Public Health Nutr. 2017;20(3):449–55.
Silva-Jaramillo KM, Neutzling MB, Drehmer M. FFQ for the adult population of the capital of Ecuador (FFQ-Quito): development, reliability and validity. Public Health Nutr. 2015;18(14):2540–9.
The authors hereby express their appreciation to Shahid Beheshti University of Medical Sciences, NNFTRI for funding the study. All coordinators and interviewers, and all people who participated in this study are appreciated. We would like to express our gratitude to Pooneh Alipoor, Fariba Bagheri, Saeed Esfandiari, Dr. Hasan Ein-Zinab, Omokolsoom Hashemkhani, Homa Heidari, Ali Kalayi, Iman Khosroshahi, Neda Lotfollahi, Bita Mohammadali-Zangeneh, Nooshin Nemati, Mahdi Noormohammadi-Zarnagh, Dr. Azadeh Nikoosaleh, Farzaneh Omid, Shahin Rajab, Dr. Hamid Rasekhi, Nastaran Shariatzadeh, Maliheh Zahehedi Rad, Fatemeh Zakeri, Telma Zowghi, for their contribution in data collection, data management and/or data entry. We would also like to extend our special thanks to Head and staff of Local Community of Tehransar (Naft-e-Shomali) and Taxi Organization of Tehran for their sincere help during the course of the study.
This work was supported by National Nutrition and Food Technology Research Institute (NNFTRI), Islamic Republic of Iran (Grant No. 450/1901-28/09/2016). NNFTRI led all aspects of the study, including designing, data collection, analysis, and interpretation of data and writing the manuscript.
Ethics approval and consent to participate
The study protocol was approved by NNFTRI’s ethics committee. The ethics committee code was IR.SBMU.nnftri.Rec.1395.43. Written informed consent was obtained from participants.
Consent to publish
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original version of this article was revised to incorporate the authors’ final corrections, which unfortunately only reached us following the article’s publication.
About this article
Cite this article
Doustmohammadian, A., Amini, M., Esmaillzadeh, A. et al. Validity and reliability of a dish-based semi-quantitative food frequency questionnaire for assessment of energy and nutrient intake among Iranian adults. BMC Res Notes 13, 95 (2020). https://doi.org/10.1186/s13104-020-04944-3
- Food frequency questionnaire