Validity and reliability of a dish-based semi-quantitative food frequency questionnaire for assessment of energy and nutrient intake among Iranian adults

Objective This study aimed to assess the validity and reliability of a dish-based, semi-quantitative food frequency questionnaire (DFFQ) for epidemiological studies in Iran. The DFFQ included 142 items (84 foods and 58 mixed dishes) which was filled in by 230 adults (110 men). All participants completed two separate DFFQs with a 6 months interval as well as six 24-h recalls, each month. Dietary biomarkers and anthropometric measurements were made. The validity was evaluated by comparing the DFFQ against 24-h dietary recalls and dietary biomarkers, including serum retinol and beta-carotene. Reliability was evaluated using intra-class correlation coefficient (ICC) and validity was determined by unadjusted and energy adjusted correlation coefficients (CC), de-attenuated CC, and cross-classification analyses. Results ICC for reliability ranged between 0.42 and 0.76. De-attenuated CC for the FFQ and the 24-h recalls ranged between 0.13 and 0.54 (Mean = 0.38). The de-attenuated CC between the DFFQ and plasma levels of retinol and beta-carotene were 0.58 (P = 0.0001) and 0.40 (P = 0.0001), respectively. Cross-classification analysis revealed that on average 73% were correctly classified into same or adjacent quartiles and 5% were classified in opposite quartiles.


Introduction
Dietary factors play an important role in development of non-communicable diseases (NCDs) [1]. To have a better understanding of any relationship between diet and NCDs, evaluation of habitual intake over a long period of time is needed [2]. Assessment of dietary habits is crucial in large, prospective epidemiological studies which provide a clear understanding of health status. Furthermore, having reliable dietary data allows researchers to examine the relationship between food and nutrients consumption and the susceptibility to diseases [3]. However, since eating behavior is a cultural issue [4,5] and a dietary assessment tool developed for one population may not be appropriate for the other.
Food frequency questionnaires (FFQs) are widely used to assess dietary patterns in different societies. FFQs, especially food-based FFQs (FFFQs), are reliable tools which can be easily used in countries whose dietary patterns are based on single foods rather than mixed dishes [6][7][8]. Dish-based FFQs (DFFQ) may have several advantages over FFFQs in the societies whose dietary practices are different from western countries. It is documented that compared to FFFQs a DFFQ calculated antioxidant vitamins, phytochemicals and fatty acids more accurately [3].
Furthermore, a DFFQ can, facilitate evaluation of dietary intake over a long period, since it is much easier to recall the frequency intake of mixed dishes rather than individual ingredients included in them [9].
Therefore, it is very challenging, if not impossible, to estimate usual intake of single ingredients through an FFFQ. Several international studies have developed DFFQs for epidemiological studies [3,16,20]. However, there were some limitations namely not considering seasonal variations of foods [3], the generalizability [16,20], and the gold standards affairs which have limited their usage as a valid tool. To the best of our knowledge, no earlier study in Iran has developed a validated DFFQ. Despite presence of two formerly developed DFFQs in Iran their applicability are limited due to validity issues or the specific target groups for which the FFQs were designed [21,22].
The purpose of the current study was to evaluate the validity and reliability of a novel dish-based, semi-quantitative FFQ [23] to be used for epidemiological studies in Iran

Study design and participants
A total of 230 male and female adults whose ages ranged between 18 and 65 years, were non-smokers (due to probable effect of smoking on biomarkers level), nonpregnant and/or nursing, living in Tehran, and were not on a specific diet were recruited in the study. Samples size was calculated based on Willett's recommendation. According to him 100 to 200 participants seems to be a reasonable sample size for such a validation study [24]. We supposed five age subgroups including 18 to 30, 31 to 40, and 41 to 50 years old with 50 participants in each of them and 51-years old to higher with 75 participants. Details of sampling process is presented in Additional file 1: Figure S1. The convenience sample consisted of volunteers of neighboring banks, staff of NNFTRI, Taxi Organization of Tehran, a Local Community in district 21 of Tehran and ordinary people as well as university students who were invited directly in their workplaces or were informed by the participants of the study. All participants were paid cash donations for their contribution to the study.

Food frequency questionnaire (FFQ)
To evaluate validity and reliability of the FFQ several steps were taken. Summary of steps taken for the validation of the DFFQ are presented in Fig. 1.
I-Development of the DFFQ A previously developed dish-based semi-quantitative FFQ was used to assess monthly subjects' habitual dietary intake [25]. The 142 items questionnaire included 84 food items and 58 mixed dishes and was completed by trained interviewers through face to face interview with the participants. Visual aids including locally used utensils were provided to help the participants to understand the concept of the portion sizes. All subjects were supposed to complete 2 individual FFQs with a 6 month interval, one in winter and another in summer of 2016.
II-Criterion validation In this section, data from the FFQ was evaluated against six 24-h recalls and dietary biomarkers.
II-I-24-h dietary recall Six 24-h recalls were collected as a reference method almost every month. Multiple probing questions were used to complete each 24-h dietary recall including diverse portion size descriptions, food models and detailed food preparation and cooking methods by interviewing the person responsible for cooking.
II-II-Dietary biomarkers Five milliliter of fasting antecubital venous blood was taken from all participants. After keeping at room temperature (RT) for 20-30 min at dark, blood samples were centrifuged at 800g for 20 min. Then the recovered sera were aliquoted in fresh microtubes and kept at − 80 °C until the day of analysis. Serum concentrations of β-carotene and retinol were measured in two occasions ( Fig. 1) using high performance liquid chromatography with the method described elsewhere [26].

Anthropometry
Height and weight were measured using standard procedures [27]. Weight was measured with minimal clothing and without shoes using a digital scale (Soehnle brand) ± 100 g. Height was measured using a non-elastic measuring tape (STABILA brand) while standing against a wall, bare-footed with the scapula in normal circumstance to the nearest 1 cm. Body mass index (BMI) was calculated through dividing weight (kg) by height in meters squared (m 2 ).

Main outcomes
Outcome measures included mean energy and nutrient intake values obtained from the DFFQs and 24-h-recalls.

Statistical analysis
Demographic characteristics of the study population were tabulated. Normality of distribution was evaluated using the Kolmogorov-Smirnov test. Firstly, de-attenuated correlation coefficients was calculated using the approach described by Rosner [28], to take into account within-person variations caused by day-to-day fluctuations, through following formula: R true = r_observed√ (1 + λ x /n x ) (1 + λ y /n y ) [28], in which λx represents the ratio of the within-and between-person variances for x, and n x represents the number of replicates for the x variable. For this study, n x = 2. Within-and between-person variations (λ x ) were obtained from its respective intraclass correlations for nutrient intakes estimated by the two FFQs and, n y = 6. Within-and between-person variations (λ y ) were obtained from its respective intra-class correlations for nutrient intakes estimated by the six 24-h recalls. Reproducibility was measured by Intra-class Correlation Coefficient (ICC).
In the second step, since total energy intake can introduce extraneous variation in recorded food intake, intake estimates were adjusted for total energy intake using the residual method [29].
Finally the degree of agreement based on quartile categorization of nutrients intakes according to the FFQs and the 24-h recalls was evaluated by examining the proportion of subjects classified by the reference method and DFFQs that fell into the same, adjacent, or extreme quartiles.

Results
Characteristics of study participants are presented in Additional file 2.
Findings of correlation coefficient between DFFQs and 24-h recalls indicate that, compared with recalls, the DFFQs significantly overestimated energy and nutrients intakes ( Table 1).
The unadjusted Spearman's correlations for nutrient intakes ranged from 0.091 for folate to 0.51 for protein and thiamin. Energy adjustments slightly improved the correlations for some nutrients including dietary fiber and thiamin.
The de-attenuation correction improved the correlation coefficients for energy, macronutrients and most of micronutrients, statistically significant (P < 0.001).
Cross classification analysis of two methods into the same and adjacent quartiles of nutrients intake ranged between 67.7% for dietary fiber to 82.6% for thiamin. On average were correctl classified into same or adjacent quartiles and a mean of 5% were classified in opposite quartile.
In order to evaluate the reproducibility of the DFFQ, energy and nutrients intakes obtained from DFFQ-1 were compared with those from DFFQ-2 ranged between 0.23 (carbohydrate) to 0.76 (energy and thiamin) ( Table 2).

Discussion
In the present study, validation and reliability of a dishbased semi-quantitative FFQ was evaluated. The inclusion of dishes in the list of FFQ food items may improve accuracy and precision of the collected data in two ways in two ways [3,22]. First, NCDs are linked to culture-specific cooking methods and ingredients [3,[30][31][32]. Second, in the FFQs in which dishes are not included people may not report invisible parts of a mixed dish because they are neither engaged in their cooking process nor can see the ingredients of different recipes. Consequently, they cannot remember the consumption of mentioned foods [22].
Generally, FFQs are used for ranking individuals according to food or nutrient intake rather than for estimating absolute amounts of intake. In this regard, we used cross-classification analysis whose results for the majority of nutrients were promising, in which approximately more than 60% of participants were classified in the correct quartiles, which is consistent with other studies [12,20,26].
In the current study we found that the DFFQ overestimated the consumption of energy and nutrients compared to the 24-h recalls. The issue of overestimation of nutrient intake by FFQs has been reported in other studies [33]. However, if this overestimation is due to a systematic error it can be modified using a correcting factor.
Unadjusted Spearman's correlation ranged between 0.23 and 0.52, which indicates a fair to medium correlation [12,34,35]. The reported correlations are similar to those obtained in other similar validation studies: According to a study carried out in Chile the values between 0.26 and 0.47 [35]. Another study in Colombia the reported correlations between 0.18 and 0.38 in urban areas and between 0.00 and 0.31 in rural areas [12]. In the current study, energy adjustment increased the correlation coefficients for some nutrients but decreased them for the majority of other nutrients. It is documented when the source of variability of nutrient consumption is related to energy intake, energy adjustment increases values of correlation coefficients, however, it will be decreased when it is due to systematic errors (overestimation and underestimation) [36]. In our study, the lower correlation values found in some nutrients may indicate that the DFFQ, to some extent, systematically overestimated intake of those nutrients. However, error in overestimation is expected in the FFQ. Similar to other studies [12,15,37] energy adjustment did not improve the crude correlations in our study. Compared with other studies in Denmark [38], Mexico [39], Canada [37], France [40], Ecuador [41], Valencia [42], and Scotland [43], our DFFQ had stronger correlation for total energy, protein, fat, thiamin, carbohydrate, calcium, iron, Compared to other studies reproducibility of the current DFFQ was reasonably acceptable. In other studies the reported correlation coefficients for reliability ranged from 0.06 (for γ-Tocopherol) to 0.31 (for vitamins A and C) [26], and 0.62 (for protein) to 0.88 (for calcium) [44]. In an article the correlation coefficients for FFQs ranged from 0.4 to 0. 8 (2009) [40].
To the best of our knowledge this questionnaire is the first valid and reliable DFFQ in Iran which can be used in various epidemiological studies and settings. It ranks people based on their dietary intake and whereby at risk groups can be screened. Effectiveness of nutritional interventions can be evaluated by it, as well. Further studies will reveal the weaknesses of the DFFQ which in turn help in improving its validity.

Conclusion
The proposed DFFQ in this study showed a relatively acceptable reproducibility and validity in ranking the participants according to energy and nutrients intakes. Therefore it can be used as a reliable tool in epidemiological studies. The DFFQ can evaluate dietary intake among adults in different settings. It can screen out nutritional risk factors and evaluate the effectiveness of interventions, as well.

Limitation
Since the source of error between the two instruments of food frequency questionnaire and 24-h recall is common and both of them are prone to recall bias the recommended method for such studies is dietary record [24]. However, as the interviewers needed training and we had the time limit, we skipped doing it.
The generalizability of the DFFQ may be limited due to the fact that the study participants were a convenience sample residing in Tehran. The DFFQ needs to be modified and validated according to various climates and food cultures in the country. Therefore the current DFFQ can be used only in climates and food cultures similar to Tehran. Therefore, for studies undertaken in other climates the DFFQ should be modified accordingly.