Skip to main content

Translation, and validation of Dysphagia Outcome and Severity Scale (DOSS): Swedish version



Swallowing dysfunction (dysphagia) significantly impacts patient and medical outcomes. In Sweden, there is no comprehensive outcome measure for dysphagia that incorporates holistic assessment and dysphagia impact on a person’s impairment, function and participation. The Dysphagia Outcome and Severity Scale (DOSS) was developed and validated (in English) and incorporates the World Health Organisation’s (WHO) aforementioned, International Classification of Functioning (ICF) aspects. This study translated then evaluated the validity and reliability of the Swedish version, DOSS-S.


Translation occurred based on WHO recommendations. The Content Validity Index (CVI) of the translated version (DOSS-S) was assessed twice by 11 (multi-professional) dysphagia experts. Criterion validity and rater reliability was calculated using 18 Speech Pathologists assessing patient cases from International Dysphagia Diet Standardization Initiative—Functional Diet Scale (IDDS-FDS) research.


Very high CVI values (0.96–0.99) for the linguistic correlation, and high CVI values (0.84–0.94) for applicability correlation were achieved. High criterion validity of DOSS-S with IDDSI-FDS was demonstrated (rs = 0.89, p < 0.01), with very high inter and intra rater reliabilities (ICC > 0.90).


The DOSS-S demonstrated very high validity values, and very high inter and intra rater reliability. This research contributes to improved dysphagia management by providing interprofessional dysphagia clinicians with a validated scale to identify patient progression, communicate dysphagia status between regions and countries, and document patient outcomes using an ICF framework.

Peer Review reports


Oropharyngeal dysphagia is the term used to describe swallowing difficulties pertaining to the oral cavity and/or pharynx [1]. Within the adult population, oropharyngeal dysphagia is a symptom of many different medical diagnoses (e.g., stroke, traumatic brain injury, neurological diseases, dementia, and head and neck cancer) [2, 3] and is also associated with intensive care and tracheostomy [4]. Oropharyngeal dysphagia may result in complications such as aspiration pneumonia, malnutrition, dehydration [1]. To appropriately assess and treat dysphagia and to reduce dysphagia related consequences, valid and reliable assessments scales are essential.

Instrumental assessments, such as Flexible Endoscopic Evaluation of Swallowing (FEES) and Modified Barium Swallow (MBS), are considered the gold standard of dysphagia assessments. Several valid and reliable tools and rating scales are available to assist the clinician in determining dysphagia impairment and severity when using these instrumental assessments. This includes, but is not limited to, the Penetration-Aspiration Scale [5], and the Yale Pharyngeal Residue Severity Rating Scale [6], and Modified Barium Swallow-Impairment, MBS-Imp scale [7]. Such impairment-based scales assist the clinician in rating the swallow safety (penetration-aspiration), and also the efficiency of the swallow (pharyngeal residue), across different bolus consistencies and amounts. Valid and reliable scales are essential for standardising the assessment process and the rating of different dysphagia characteristics. This, in turn, allows for evidenced-based and impairment-specific interventions, such as (i) modifying foods/fluids or trialing swallow manoeuvres (compensatory techniques), and/or (ii) providing rehabilitation/training strategies to target the specific underlying pathophysiology (impairment) [8].

However, when assessing dysphagia, it is also important to consider, not only the impairment or disorder, however the greater range of dysphagia management influences. For optimal dysphagia management, the World Health Organisation’s (WHO) International Classification of Functioning, Disability, and Health (ICF) advocates for the medical diagnosis and impairment level to be supplemented with a measurement of functional activity and the patient’s ability to participate while considering environmental and personal factors [9].

In 1999, O ‘Neil and colleagues [10], noted that many scales assessing dysphagia predominantly investigated the impairment aspect to dysphagia and did not incorporate the many other influencing factors which impact a person’s functional activity or participation—which became more clearly articulated by the ICF framework in 2002 [9]. O’Neil and colleagues [10], identified the need to address the lack of functional and participation measures in dysphagia assessment and subsequently developed and validated the Dysphagia Outcome and Severity Scale (DOSS). The DOSS was established and evaluated using 135 MBS studies and 4 DOSS-trained SLPs. It is a 7-point rating scale, with a score of 1 representing a severe swallow dysfunction and oral intake level, while a score of 7 indicates a normal swallow function and oral intake. Each of the seven DOSS levels comprehensively describe: (a) the impairment, considering the oral and pharyngeal stage physiology such as safety (penetration/aspiration) and efficiency (pharyngeal retention); (b) functional aspects such as diet recommendations and need for swallowing strategies; (c) level of nutrition (alternative feeding aspects); and (d) level of supervision and independence with oral intake. Development of the DOSS required raters to not only consider (i) the MBS assessment results, however also (ii) the environmental and personal factors such as patient’s premorbid nutrition, cognition, acuity of dysphagia, current medical status and environment in terms of supervision/support available for the patient [10]. The DOSS is reported to therefore provide a holistic dysphagia outcome and severity scale for objective and patient-centered dysphagia management [10].

The DOSS is both valid and reliable [10] and furthermore has consistently been used internationally within research to validate/develop other dysphagia screening instruments, dysphagia questionnaires, and rating scales [11,12,13,14,15]. Results from these researchers indicate strong construct validity [11], criterion validity [12, 13], and fair [13] to very high reliability [10, 15].

Currently, in Sweden, there is no holistic dysphagia rating scale that considers structural/physiological impairment, along with functional activity and participation measurements, as recommended by O’Neill et al., [10] and the ICF model [9]. Therefore, this research investigated the translation and cultural adaptation of DOSS into Swedish (DOSS-S) and further studied the validity and reliability of DOSS-S.


Study design

This multistep translation and validation study was conducted in Sweden. Ethical approval was granted by the Swedish Ethical Review Authority (Dnr 2020–05246).

Participants and procedures

Translation was based on the WHOs guidelines for translating and adapting instruments with a focus on meaning and content rather than literal translation [16]. As per previous literature, the process was enhanced using a strong multi-step translation methodology. [17, 18]. Translation, validation, and rater reliability was conducted with different participants in three phases: (1) multistep translation and validation via an Expert Panel using the Content Validity Index (CVI) [19], (2) criterion validity analysis, and (3) rater reliability testing, see Additional file 2: Figure S1, Flow Diagram, with process further described below.

Part 1: Multi-step translation and content validity

  • Step 1. Forward translation (from English to Swedish) was conducted by authors KM and TLP with Swedish as first language, strong English language proficiency. Individual translation by each author was conducted first, with a subsequent comparison of the two translations, to then create a synthesised version (DOSS-S translation 1.0).

  • Step 2. DOSS-S translation 1.0 was reviewed by an Expert Panel (n = 11) using CVI [19]. The Expert Panel consisted of nine speech and language pathologists (SLPs), one otolaryngologist and one radiologist, from six different regions within Sweden. Recruitment occurred via convenience sampling [20]. Inclusion criteria included (a) experience with dysphagia instrumental assessment, (b) ≥ five years clinical experience in dysphagia management, and (c) fluency in Swedish and with English proficiency. The Expert Panel participants had 6–31 years of experience in dysphagia management, 3–31 years of instrumental assessment experience.

    Validity, as per CVI [19], used the Expert Panel to review the Swedish DOSS-S version compared to the English version. Each translation of the seven DOSS-S levels were rated by the Expert Panel using CVI, considering (a) linguistic equivalence between the English and Swedish versions, and (b) the understandability and applicability of the Swedish translation considering the health care context and dysphagia management in Sweden [18, 19], see Table 1.

    Table 1 Rating Scale for Content Validity Index (CVI)
  • Step 3. Review of DOSS-S 1.0. The authorship team reviewed the Expert Panel’s review. (Authorship team = two SLP Honours students, and two research supervisors with 9–23 years of dysphagia management experience and 6–22 years of experience in instrumental dysphagia evaluations, both VFS and FEES). Author PH has Swedish as a first language with strong English proficiency; LB has English as a first language with strong Swedish proficiency. Review and synthesis of the Expert Panel’s CVI ratings and feedback resulted in DOSS-S translation 2.0.

  • Step 4. Back translation (Swedish translation 2.0 to English) was conducted by an independent translator (Registered Dietitian, 30 years’ experience, with English as a first language and strong Swedish proficiency). Back translation focused on content, meaning and cultural equivalence, not linguistic equivalence.

  • Step 5. Review of Translation 2.0 occurred with all four authors considering the DOSS-S translation 2.0, the forward translation’s differences, weaknesses, and areas for improvement. The new version, DOSS-S 3.0, was subsequently formulated.

  • Step 6. The second Expert Panel review used the same 11 experts to rate (CVI) and review the latest DOSS-S, as per review 1 and CVI protocol (Table 1).

  • Step 7. Revision and final version of DOSS-S was established after the authorship team synthesised the comments, feedback and CVI from the Expert Panel’s second review. Lastly, each of the seven DOSS-S levels were reviewed to ensure consistency with descriptive information presentation for each dysphagia level, that is, (i) functional impact was described first, followed by (ii) oral dysfunction symptoms, then (iii) pharyngeal phase symptoms, See Additional file 1: Appendix 1, DOSS—Swedish (DOSS-S).

Part 2: Criterion validity

Both criterion validity and rater reliability used published patient cases from the development of the International Dysphagia Diet Standardization Initiative (IDDSI) Functional Diet Scale [21]. Eighteen SLPs, from eight different regions, with 1–15 years of dysphagia experience, were recruited to assess the 10 patient cases using the finalised DOSS-S version. The 18 SLPs represented a range of different healthcare regions in Sweden (Skåne, Västra Götaland, Östergötland, Örebro, Stockholm, Västernorrland, Västerbotten) with one Swedish SLP recruited from Norway. The SLPs represented different clinical workplaces, including neurology, acute stroke, head and neck cancer, otolaryngology, rehabilitation centre, intensive care, pediatric dysphagia, and home rehabilitation. Fourteen SLPs had experience regarding instrumental assessment of swallowing, two had experience using DOSS. Recruitment occurred via convenience sampling [20], using the National Dysphagia network, other professional forums/social media. For criterion validity, results for the patient cases, as rated by the 18 SLPs using the DOSS-S, were compared with the published IDDSI Functional Diet Scale for the same 10 patients.

Part 3: Rater reliability

Inter rater reliability was calculated using the above 18 SLPs rating the 10 patient cases with DOSS-S. For intra rater reliability, the same 10 patient cases (re-randomised) were sent to the 18 SLPs to re-rate using the DOSS-S two weeks after the initial DOSS-S rating. Data was missing for four SLPs, leaving a total of 14 SLP ratings available for intra rater reliability analysis.

Statistical analyses

Statistical analysis was performed in the IBM SPSS Statistics version 26 with significance set at p < 0.05.

Content validity

  1. (1)

    The Item-CVI uses the Expert Panels ratings of the DOSS-S seven items. Item-CVI is calculated using acceptable ratings (score of 3 or 4) whereby the sum of expert panelists who gave the same item an acceptable rating is divided by the total number of expert panelists (n = 11) [18, 19].

  2. (2)

    Scale-CVI /Average, describes the scale's content validity in its entirety, and is calculated by dividing the sum of all Item-CVIs by the number of items on the scale (n = 7) [18, 19, 22]. Statistical analyses interpretation [19, 22,23,24] is provided in Table 2.

  3. (3)

    The difference between the Expert Panels’ first and second ratings using the Scale-CVI/Average measure (using scores 1, 2, 3, or 4) was calculated using Wilcoxon signed-rank test.

Table 2 Statistical analyses interpretation

Criterion validity

The correlation between the DOSS-S scores and the IDDSI-FDS patient scores used Spearman's correlation analysis. Table 2 provides statistical interpretation for Spearman’s correlation analysis.

Inter and intra rater reliability

Intraclass Correlation Coefficient (ICC), with a confidence interval (CI) of 95%, was used using’two-way mixed’ model,’absolute agreement’, based on the mean value of the 18 assessments. For intra rater reliability, the weighted Kappa analysis was performed, CI = 95%. For inter rater reliability, the mean of all weighted Kappas across patient cases was calculated. To interpret ICC and Kappa values, see Table 2.


Part 1: Translation and content validity

Results for (a) linguistic equivalence, and (b) understandability and applicability in a Swedish health care context, are reported in Table 3.

Table 3 Expert panel’s Content Validity Index (CVI) ratings

Part 2: Criterion validity

The SLPs' dysphagia ratings using DOSS-S showed a high correlation with the previously published IDDSI-FDS results (rs = 0.89, p < 0.01).

Part 3: reliability

Rater reliability demonstrated very high agreement, see Table 4.

Table 4 Rater reliability


This research presents data for the validity and reliability of the Dysphagia Outcome and Severity Scale – Swedish version. Outcomes from the current study, demonstrate a high-quality translation with excellent content validity (CVI > 90%), strong criterion validity (rs = 0.89), and good–excellent agreement (inter rater reliability K = 0.77, ICC = 0.995; intra rater reliability K = 0.85, ICC = 0.97). These results, in terms of DOSS-S content validity, criterion validity and rater reliability are discussed further below.

Content validity

Although results demonstrate excellent content validity for the DOSS -S, with both literal translation (linguistic correlation) and cultural adaptation (understandability and applicability), this is likely influenced by the multi-step translation process with several revisions ensuring improvements at each step. Similar results are reported by previous translation research demonstrating high validity using a multistep translation process [17, 18]. Of interest, is that, for the linguistic correlation, the improvement from Expert Rating 1 to 2, was not significant (p = 0.62), which is likely due to the already high CVIs prior to the second Expert Panel rating. For the cultural adaptation aspects (understandability and applicability), improvements from the first to second translation were, in fact, significant (p = 0.015) with all Item-CVIs reaching the acceptable cut-off value (> 0.79) by the final expert panel review.

Criterion validity

The DOSS-S demonstrated strong correlation (rs = 0.89) with the IDDSI-FDS scale, a functional diet scale also with established high validity and reliability [21]. Such results are congruent with previous research [11,12,13, 25] where the original DOSS (in English) has shown high – very high correlations/criterion validity when compared to other dysphagia measures.


In the current study, the DOSS-S when used by Swedish SLPs to rate previously published patient cases, demonstrated excellent inter and intra rater reliability. The high reliability results are similar to reliability testing from the original DOSS [10] and other research [15]. Furthermore, reliability results from the present study, are likely to be representative of the larger Swedish SLP population since the 18 SLPs recruited were heterogenic in years of experience (1–15 years) and from seven different healthcare regions in Sweden. The high inter rater reliability from the DOSS-S results in this study are congruent with reliability results from the original DOSS study [10], and research by Kidney et al. [15], however, diverge from reliability findings by Zarkada and Regan [14]. Upon further investigation, this difference is likely due to the variable methodology used within these aforementioned studies.

Limitations and future directions

As with all research, the current study has its limitations. In terms of criterion validity, results were based on the ratings by 18 SLPs rating the 10 published cases from the validated IDDSI-FDS [21], methodology which may have influenced these results. It is possible, and expected, that a different correlation would be achieved if the DOSS-S was compared with another dysphagia rating or functional diet scale, a recommendation for future research. Rater reliability was also calculated using these 10 published IDDSI-FDS cases. Future research should investigate DOSS-S validity and reliability with (a) a greater number of patient cases, and (b) using instrumental assessments. Rasch analysis is also warranted. Additionally, DOSS-S training and calibration may be recommended prior to use.


Results demonstrate a high-quality DOSS-S translation with high - very high content, criterion validity and rater-reliability. This research presents a valid, reliable and comprehensive dysphagia assessment scale which incorporates the WHOs ICF aspects such as impairment at the structural/physiological level, functional activity, and participation measures – not previously available for Swedish clinicians.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.


  1. Sasegbon A, Hamdy S. The anatomy and physiology of normal and abnormal swallowing in oropharyngeal dysphagia. J Neurogastroenterol Motility. 2017;29(11):1–15.

    Article  Google Scholar 

  2. Baijens LW, Clavé P, Ekberg O, Forster A, Kolb GF, Leners JC, Masiero S, Mateos-Nozal J, Ortega O, Smithard DG, Speyer R, Walshe M. European Society for Swallowing Disorders—European Union Geriatric Medicine Society white paper: oropharyngeal dysphagia as a geriatric syndrome. Clin Interv Aging. 2016;11:1403–28.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Hutcheson KA, Nurgalieva Z, Zhao H, et al. Two-year prevalence of dysphagia and related outcomes in head and neck cancer survivors: an updated SEER-Medicare analysis. Head Neck. 2019;41(2):479–87.

    Article  PubMed  Google Scholar 

  4. Zuercher P, Moret CS, Dziewas R, et al. Dysphagia in the intensive care unit: epidemiology, mechanisms, and clinical management. Crit Care. 2019;23:103–14.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Rosenbek JC, Robbins JA, Roecker EB, Coyle JL, Wood JL. A penetration-aspiration scale. Dysphagia. 1996;11:93–8.

    Article  CAS  PubMed  Google Scholar 

  6. Neubauer PD, Rademaker AW, Leder SB. The Yale pharyngeal residue severity rating scale: an anatomically defined and imagebased tool. Dysphagia. 2015;30:521–8.

    Article  PubMed  Google Scholar 

  7. Martin-Harris B, Brodsky MB, Michel Y, Castell DO, Schleicher M, Sandidge J, Maxwell R, Blair J. MBS measurement tool for swallow impairment – MBSImp: establishing a standard. Dysphagia. 2008;23(4):392–405.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Langmore SE, Pisegna JM. Efficacy of exercises to rehabilitate dysphagia: a critique of the literature. Int J Speech Lang Pathol. 2015;17(3):222–9.

    Article  PubMed  Google Scholar 

  9. World Health Organization. Towards a Common Language for Functioning, Disability and Health, ICF. 2002.

  10. O’Neil KH, Purdy M, Falk J, Gallo L. The dysphagia outcome and severity scale. Dysphagia. 1999;14(3):139–45.

    Article  CAS  PubMed  Google Scholar 

  11. Audag N, Goubau C, Danse E, et al. Validation and reliability of the french version of the Sydney Swallow Questionnaire. Dysphagia. 2019;34:556–66.

    Article  PubMed  Google Scholar 

  12. Järvenpää P, Kuuskoski J, Pietarinen P, et al. Finnish version of the eating assessment tool (F-EAT-10): a valid and reliable patient-reported outcome measure for dysphagia evaluation. Dysphagia. 2021;29(11):995–1007.

    Article  Google Scholar 

  13. Pacheco-Castilho AC, de Martini VG, Reichardt B, et al. Translation and validation of the TOR-BSST© into Brazilian Portuguese for Adults with Stroke. Dysphagia. 2021;36:533–40.

    Article  PubMed  Google Scholar 

  14. Zarkada A, Regan J. Inter-rater reliability of the dysphagia outcome and severity scale (DOSS): effects of clinical experience. Audio-Recording Training Dysphagia. 2018;33(3):329–36.

    Article  PubMed  Google Scholar 

  15. Kidney D, Alexander M, Corr B, O’Toole O, Hardiman O. Oropharyngeal dysphagia in amyotrophic lateral sclerosis: neurological and dysphagia specific rating scales. Amyotroph Lateral Scler Other Motor Neuron Disord. 2004;5(3):150–3.

    Article  PubMed  Google Scholar 

  16. World Health Organization. Process of translation and adaptation of instruments. 2020.

  17. Acquadro C, Conway K, Hareendran A, Aaronson N. Literature review of methods to translate health-related quality of life questionnaires for use in multinational clinical trials. Value Health. 2008;11(3):509–21.

    Article  PubMed  Google Scholar 

  18. McGreevy J, Orrevall Y, Belqaid K, Bernhardson BM. Reflections on the process of translation and cultural adaptation of an instrument to investigate taste and smell changes in adults with cancer. Scand J Caring Sci. 2013;28(1):204–11.

    Article  PubMed  Google Scholar 

  19. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30:459–67.

    Article  PubMed  Google Scholar 

  20. Borg E, Westerlund J. Statistik för beteendevetare - faktabok (3 uppl.). Liber. 2006. ISBN: 9789147097371.

  21. Steele MC, Namasivayam-MacDonald AM, Guida BT, Cichero AJ, Duivestein J, Hanson B, Lam P, Riquelme L. Creation and initial validation of the international dysphagia diet standardisation initiative functional diet scale. Arch Phys Med Rehabil. 2018;99(5):934–44.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Rodrigues IB, Adachi JD, Beattie KA, MacDermid JC. Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis. BMC Musculoskelet Disord. 2017;18(1):540–9.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–63.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Altman DG. Practical statistics for medical research. London: Chapman and Hall; 1991.

    Google Scholar 

  25. Arenaz Búa B, Bülow M. Validation in Swedish of Sydney swallow questionnaire. BMC Res Notes. 2014;7(1):742–8.

    Article  PubMed  PubMed Central  Google Scholar 

Download references


Not applicable.


Open access funding provided by Karolinska Institute. This research was part of the Honours students’ (KM, TLP) speech pathology course work – no funding was received. The primary supervisor (LB) received no payment, apart from salary from Gothenburg University. Secondary supervisor (PH) was an external supervisor from Umeå University and received standard reimbursement, equivalent to 15 h supervision, according to the University of Gothenburg’s external supervisor payment schedule. No other external funding was granted to any of the authors. Gothenburg University as an organisation did not influence/had no role in the design of the study, data collection, analysis, interpretation of data nor in the writing of the manuscript.

Author information

Authors and Affiliations



Study and manuscript components. Klara Movander Study design (5%) Data collection (45%) Statistical analysis (40%) Interpretation of data (25%) Wrote the paper (45%) Substantial revision (5%) Final edits and approval (25%) Tove Larsson Palmquist. Study design (5%) Data collection (45%) Statistical analysis (40%) Interpretation of data (25%) Wrote the paper (45%) Substantial revision (5%) Final edits and approval (25%). Patricia Hägglund. Study design (40%) Data collection (5%) Statistical analysis (10%) Interpretation of data (25%) Wrote the paper (5%) Substantial revision (40%) Final edits and approval (25%) Liza Bergström. Study design (50%) Data collection (5%) Statistical analysis (10%) Interpretation of data (25%) Wrote the paper (5%) Substantial revision (50%) Final edits and approval (25%).

Corresponding author

Correspondence to Liza Bergström.

Ethics declarations

Ethics approval and consent to participate

This research was conducted according to the Declaration of Helsinki. All participants provided written informed consent. Ethical approval was granted by the Swedish Ethical Review Authority (Dnr 2020-05246).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. Appendix 1, DOSS - Swedish (DOSS-S).

The translated (Swedish) version of the Dysphagia Outcome and Severity Scale (DOSS).

Additional file 2: Figure S1. Flow Diagram.

Flow diagram depicting the process of DOSS translation from English to Swedish, and the validation and reliability testing of the DOSS-S.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Movander, K., Larsson Palmquist, T., Hägglund, P. et al. Translation, and validation of Dysphagia Outcome and Severity Scale (DOSS): Swedish version. BMC Res Notes 16, 369 (2023).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: