Preliminary validity and reliability of a Thai Berlin questionnaire in stroke patients
© Saengsuwan et al.; licensee BioMed Central Ltd. 2014
Received: 3 November 2013
Accepted: 2 June 2014
Published: 9 June 2014
Obstructive sleep apnea (OSA) is a major risk factor for stroke. The Berlin Questionnaire (BQ) has been shown to be a valid tool to screen for OSA. The literature has limited data on using the BQ in stroke patients; particularly in Thailand and other developing countries. Here, we aimed to develop a Thai-language Berlin Questionnaire (Thai BQ) and to preliminarily assess construct validity, test-retest reliability and the agreement of the Thai BQ with the Thai Epworth Sleepiness Scale (Thai ESS), another screening tool for OSA.
A hospital-based cross-sectional study was performed from January to July, 2011. One hundred first-ever stroke patients, including acute and chronic cases, and their caregivers were enrolled. The Thai BQ was developed using the forward-backward translation method. Evaluation of construct validity was done by factor analysis. Internal consistency of the Thai BQ and the Thai ESS were evaluated using Cronbach’s alpha coefficient. Test-retest reliability and the agreement of the Thai BQ and the Thai ESS were evaluated using Cohen’s kappa coefficient.
Factor analysis identified 4 main factors: Factor 1–Snoring behaviour; Factor 2–Sleepiness during driving; Factor 3–Daytime fatigue; and Factor 4–Hypertension or obesity. Cronbach’s alpha coefficient was 0.77 (95% confidence interval (CI) = 0.69-0.83) and Cohen’s kappa coefficient was 0.86 (95% CI = 0.74-0.98) in the Thai BQ. Cronbach’s alpha coefficient was 0.59 (95% CI = 0.45-0.70) and the Cohen’s kappa coefficient was 0.81 (95% CI = 0.60-1.00) in the Thai ESS. The agreement between the Thai ESS and the Thai BQ was fair.
The Thai BQ is a valid and reliable tool to screen for OSA in stroke patients. As factor analysis revealed 4 factors in contrast to the 3 factors in the original BQ, further modification of the Thai BQ is required.
KeywordsStroke Obstructive sleep apnea Berlin questionnaire Epworth sleepiness scale Validity
The prevalence of obstructive sleep apnea (OSA) in stroke patients is higher than in the general population (55-71% vs. 12–18.5%) [1–4]. OSA is not only an independent risk factor for stroke: stroke patients with OSA also have worse outcomes, higher rates of recurrent stroke and higher mortality rates [5–7]. Moreover, recent evidence has shown that continuous positive airway pressure (CPAP) treatment in acute stroke improved neurological outcome compared to control treatment if OSA was detected . Thus, it is important to identify OSA in stroke patients. OSA in stroke patients, however, has not been widely evaluated, partly because of the lack of awareness of the impact of OSA in stroke and the limited availability of polysomnography.
In an attempt to address this need, many questionnaires have been developed to identify people at risk of OSA. Two of the most widely used self-administered screening tools are the Berlin Questionnaire (BQ) and the Epworth Sleepiness Scale (ESS) [9, 10]. The ESS contains 8 subjective and self-administered items relating to daytime sleepiness. In Thailand, the Thai ESS has already been translated and tested in both healthy volunteers and able-bodied participants with OSA. It has high internal consistency demonstrated by Cronbach’s alpha coefficient of 0.87 and high test-retest reliability as shown by an intra-class correlation coefficient of 0.79 . For ESS, however, Onen et al. found that almost 60% of people aged more than 65 years were not able to answer at least one question in the ESS. Further, age and cognitive status were related to differences in the ESS score . Many stroke patients may also have these limitations.
The BQ has a total of 11 questions in 3 categories: snoring behaviour, waketime fatigue or sleepiness, and the diagnosis of hypertension or obesity. It has good internal consistency as demonstrated by a Cronbach’s alpha value of 0.86-0.92 . In addition, Ramachandran and Josephs published findings from a meta-analysis of clinical screening tests for OSA and concluded that the BQ was the most accurate questionnaire for predicting diagnosis of OSA .
The BQ and ESS as screening tools for OSA in stroke patients are limited in clinical practice particularly in Thailand and other developing countries (Additional file 1). Both tools are helpful for clinicians to select appropriate cases for referral to sleep facilities which are lacking in the developing world. The aims of this study were to 1) develop a Thai-language version of the BQ, 2) study the preliminary validity and reliability of the Thai BQ, and 3) compare the agreement of the caregiver-administered Thai BQ with the patient-administered Thai ESS.
The cross-sectional study was conducted in both the in and out-patient units in Srinagarind Hospital, Thailand, from January to July 2011. The study was reviewed and approved by the Khon Kaen University Ethics Committee for Human Research (HE531422). All participants signed a consent form prior to participation.
The eligible population was comprised of patients older than 18 years with first-ever haemorrhagic or ischemic stroke, with radiographic evidence of cerebrovascular disease. Exclusion criteria were: 1) Mini-Mental State Examination of less than 24, aphasia or psychiatric disorder; 2) concurrent cardiopulmonary disease affecting breathing or tiredness, e.g., congestive heart failure or asthma; 3) coexisting neurological conditions such as myasthenia gravis and 4) taking medication affecting sleep.
Translation of the original BQ into the Thai language
Translation of the BQ was kindly permitted by the original developer . The translation involved converting the questions and rating criteria of the original BQ (Additional files 2 and 3). The Thai version was then translated back to English (forward-backward translation method) using two bilingual translators (one physician of rehabilitation medicine and one of sleep medicine).
The translated version was reviewed by an expert panel of four to evaluate the equivalence to the original version and the content validity. The expert panel consisted of one physician of rehabilitation medicine, one otolaryngologist, one sleep medicine physician and one sleep lab nurse. The content validity of the Thai BQ was judged by item-objective congruence (IOC). Each expert independently rated the relevance of each item on the Thai BQ and the original BQ to determine whether the translated items fell within the content domain as specified by the original BQ, using a scale of +1 (clear agreement), 0 (unclear) or −1 (no clear agreement). The IOC for each item was the summation of scores given by the experts divided by the number of experts. The average IOC score was 0.86. Every item in the Thai BQ scored more than 0.75 which is considered acceptable .
The Thai BQ version was then used to test structural validity, test-retest reliability and the agreement of the Thai BQ and the Thai ESS by interviewing the caregiver of stroke patients for the Thai BQ and the stroke patients themselves for the Thai ESS. All patients and caregivers completed the questionnaire twice within 48 hours for the test-retest reliability analysis. High risk patients were defined as those positive in more than 2 categories of the BQ or having a Thai ESS score equal to or more than 10.
Baseline characteristics of patients who participated were recorded: age, gender, weight, height, side of stroke and type of stroke together with underlying disease profile.
Construct validity was tested using exploratory factor analysis to identify the latent factors underlying the 11 questions of the BQ. The internal consistency of each category was calculated using Cronbach’s alpha coefficient. The test-retest reliability of the Thai BQ and agreement between the Thai BQ and the Thai ESS were examined using Cohen’s kappa coefficient. The statistical analysis was carried out using SPSS version 19.
General characteristics of stroke patients (n = 100) who participated in the study
Age (Years): Mean (SD)
Type of stroke (n)
Duration after stroke (days)
Underlying disease (n)
Numbers of responses for each item of the Thai BQ (n = 100)
1. Do you snore?
2. Your snoring is
Louder than talking
Very loud–can be heard in adjacent room
3. How often do you snore?
Nearly every day
3-4 times a week
4. Has your snoring ever bother other people?
5. Has anyone noticed that you quit breathing during your sleep?
Nearly every day
3-4 times a week
6. How often do you feel tired or fatigued after your sleep?
Nearly every day
3-4 times a week
7. During your waking time, do you feel tired, fatigue or not up to par?
Nearly every day
3-4 times a week
8. Have you ever nodded off or fallen asleep while driving a vehicle?
9. How often does this occur?
Nearly every day
10. Do you have high blood pressure?
11. Do you have BMI ≥ 30 kg/m2?
Factor structure of the Thai BQ
Factor 1: Snoring behaviour
Do you snore?
Your snoring is?
How often do you snore?
Has your snoring ever bothered other people?
Has anyone noticed that you quit breathing during your sleep? If yes, how frequently?
Factor 2: Sleepiness during driving
Have you ever nodded off or fallen asleep while driving a vehicle?
How often does this occur?
Factor 3: Daytime fatigue
How often do you feel tired or fatigued after your sleep?
During your waking time, do you feel tired, fatigue or not up to par?
Factor 4: Hypertension or obesity
Do you have high blood pressure?
Do you have BMI ≥ 30 kg/m2?
Reliability of the Thai BQ and the Thai ESS
Evaluation of internal consistency and test-retest reliability of the factors of the Thai BQ
Number of questions
Cronbach’s alpha (95% CI)
Kappa (95% CI)
1. Snoring behaviour
2. Sleepiness during driving
3. Daytime fatigue
4. Hypertension or obesity
Agreement between the Thai BQ and the Thai ESS
The Thai BQ was developed by translation and adaptation of the original BQ, which allows assessment of the risk of OSA in general populations . The preliminary validity and reliability of the Thai BQ was examined. The results of the study showed that the Thai BQ had good internal consistency in factor 1 (snoring behaviour) but not in factor 3 (daytime fatigue). Overall, the Thai BQ had good internal consistency and good test-retest reliability. The BQ allows the stroke caregiver to respond to objective questions relating to factors including snoring behaviour, i.e., the questionnaire rates the loudness and frequency of snoring. This was also shown in the study of Sagaspe et al., where the results of self-reported and bed-partner-reported BQ were compared: it was found that the bed-partner-reported BQ showed slightly better results. The authors concluded that this may be due to better identification of snoring or excessive daytime sleepiness by bed partners .
The prevalence of high risk for OSA identified from this study was very low (24%), although the study was conducted in a higher mean age group (mean age of 64 years) compared to other studies [1–4]. Twenty one patients (87.5%) were diagnosed as being at a high risk for OSA mainly by categories 1 and 3, 1 patient (4.2%) by categories 1 and 2, 1 patient by categories 2 and 3, and 1 patient by all 3 categories. This result was similar to other cross-sectional studies and a meta-analysis which showed that category 1 and category 3 are significant elements in the OSA population [13, 18, 19].
Due to the fact that stroke mostly occurs in middle-aged and elderly people, stroke patients usually have a higher energy expenditure while walking or performing daily life activities [20, 21]. This can lead to tiredness and there is a high prevalence of fatigue in stroke patients, ranging from 16.1–58.1% [22, 23]. In addition, stroke patients were found to have a high prevalence of sleep disorders (78%) such as insomnia or OSA . Stroke patients should have both a high percentage of positive in category 2 and high risk for OSA identified from the ESS. Surprisingly, this study did not demonstrate this and the positive result in category 2 is low. There are some possible explanations for the low positive results in category 2 such as: 1) the questions in category 2 (factor 3) may not directly reflect daytime fatigue and the patients’ caregivers may not report fatigue because they considered fatigue as normal for stroke patients; 2) Thai elderly normally take a nap during the day so they feel less sleepy or fatigued during their waking time-the questions in factor 3 mainly focus on daytime fatigue or tiredness but do not involve sleepiness; lastly, 3) another possibility may relate to the question regarding sleepiness while driving, since stroke patients do not normally drive. Thus questions 8 and 9 in category 2 will be reported as 0: this was found in 93% of the answers. This was also observed in the study of Banhiran et al. which studied the validity and reliability of the Thai ESS in able-bodied participants. They found that participants responded to the question related to sleepiness in a car while stopped for a few minutes in the traffic as “no chance of dozing” because most participants do not drive .
The low reporting of sleepiness while driving may explain why 2 factors from factor analysis (daytime fatigue and sleepiness during driving) from category 2 are yielded instead of only 1. As the original BQ mainly aimed to assess the risk of OSA in the general population, the questions need to be modified before use in stroke patients to conform better to their daily lives.
There was only fair agreement between high risk of OSA diagnosed from the Thai BQ and the Thai ESS. This may be a result of the difference between the questionnaires, as a high risk of OSA was diagnosed using different criteria. The high risk of OSA according to the ESS mainly focusses on daytime sleepiness while for the BQ in this study the diagnosis relied on snoring behaviour and the diagnosis of hypertension or obesity. The internal consistency and test-retest reliability were lower in the present study compared to a previous study in Thailand. This may be due to differences between the respective study populations: the study by Banhiran et al., included able-bodied subjects with a lower average age .
As there was a discrepancy between the results of the Thai BQ and the Thai ESS for identifying the high risk group for OSA, 10 patients in the discrepancy group agreed to be investigated by polysomnography. Of the 10 patients, 1 was diagnosed as high risk by the Thai ESS and 9 were diagnosed as high risk by the Thai BQ. All 10 patients were diagnosed as OSA by polysomnography. Although this study shows a trend towards the Thai BQ in diagnosing OSA in stroke patients, further examination of the screening properties of the Thai BQ is required.
There are some limitations in the study. Patients with cardiopulmonary disease, aphasia or cognitive perceptual deficit were excluded. This exclusion may account for the lower prevalence of high risk OSA identified in this study population. Furthermore, mainly because of resource limitations, all participants were unable to be studied using overnight polysomnography so the predictive validity and the test properties of the Thai BQ could not be demonstrated. This study may provide baseline knowledge for further development of a modified version of the Thai BQ for the diagnosis of high risk OSA for stroke patients.
The next step is to focus on modification of the Thai BQ in category 2. The question regarding sleepiness while driving may need to be changed to a more appropriate question involving sleepiness during daily activities that are relevant to stroke patients. The appropriate cut off point for BMI should also be evaluated since the diagnosis of obesity in Asia and in western countries is different [25, 26]. Finally, a study of the discriminative and predictive validity of the BQ in comparison with the gold standard, which is polysomnography, should be carried out.
In conclusion, the Thai BQ has been developed. The study showed that the Thai BQ has an acceptable preliminary internal consistency and test-retest reliability. As factor analysis revealed 4 factors, in contrast to the 3 factors in the original study, modification of the Thai BQ is required. Comparison with the gold standard for the predictive validity of the Thai BQ is desirable before administration of the questionnaire in Thai stroke patients.
We gratefully acknowledge the contributions of the staff in the expert panel who shared their opinions on the Thai BQ and the staff in the polysomnography laboratory. This study was supported by TRF grants from Senior Research Scholar Grant, Thailand Research Fund grant number RTA5580004 and the Higher Education Research Promotion and National Research University Project of Thailand, Office of the Higher Education Commission, Thailand through the Health Cluster (SHeP-GMS), Khon Kaen University. The authors would like to thank Prof. James A. Will (University of Wisconsin) for his kind review of the manuscript.
- Chan W, Coutts SB, Hanly P: Sleep apnea in patients with transient ischemic attack and minor stroke: opportunity for risk reduction of recurrent stroke?. Stroke. 2010, 41: 2973-2975. 10.1161/STROKEAHA.110.596759.PubMedView ArticleGoogle Scholar
- Srijithesh PR, Shukla G, Srivastav A, Goyal V, Singh S, Behari M: Validity of the Berlin Questionnaire in identifying obstructive sleep apnea syndrome when administered to the informants of stroke patients. J Clin Neurosci. 2011, 18: 340-343. 10.1016/j.jocn.2010.04.048.PubMedView ArticleGoogle Scholar
- Dyken ME, Somers VK, Yamada T, Ren ZY, Zimmerman B: Investigating the relationship between stroke and obstructive sleep apnea. Stroke. 1996, 27: 1668-1677.View ArticleGoogle Scholar
- Bassetti C, Aldrich MS: Sleep apnea in acute cerebrovascular diseases: Final report on 128 patients. Sleep. 1999, 22: 217-223.PubMedGoogle Scholar
- Yaggi HK, Concato J, Kernan WN, Lichtman JH, Brass LM, Mohsenin V: Obstructive sleep apnea as a risk factor for stroke and death. N Eng J Med. 2005, 353: 2034-2041. 10.1056/NEJMoa043104.View ArticleGoogle Scholar
- Mansukhani MP, Bellolio MF, Kolla BP, Enduri S, Somers VK, Stead LG: Worse outcome after stroke in patients with obstructive sleep apnea: an observational cohort study. J Stroke Cerebrovasc Dis. 2011, 20: 401-405. 10.1016/j.jstrokecerebrovasdis.2010.02.011.PubMedView ArticleGoogle Scholar
- Good DC, Henkle JQ, Gelber D, Welsh J, Verhulst S: Sleep-disordered breathing and poor functional outcome after stroke. Stroke. 1996, 27: 252-259. 10.1161/01.STR.27.2.252.PubMedView ArticleGoogle Scholar
- Parra O, Sanchez-Armengol A, Bonnin M, Arboix A, Campos-Rodriquez F, Pèrez-Ronchel J, Duran-Cantolla J, de la Torre G, Gonzalez M, de la Pena M, Carmen Jiménez M, Masa F, Casado I, Luz Alonso M, Macarron JL: Early treatment of obstructive apnoea and stroke outcome: a randomised controlled trial. Eur Respir J. 2011, 37: 1128-36. 10.1183/09031936.00034410.PubMedView ArticleGoogle Scholar
- Netzer NC, Stoohs RA, Netzer CM, Clark K, Strohl KP: Using the Berlin questionnaire to identify patients at risk for the sleep apnea syndrome. Ann Intern Med. 1999, 131: 485-491.PubMedView ArticleGoogle Scholar
- Johns MW: A new method for measuring daytime sleepiness - the epworth sleepiness scale. Sleep. 1991, 14: 540-545.PubMedGoogle Scholar
- Banhiran W, Assanasen P, Nopmaneejumruslers C, Metheetrairut C: Epworth sleepiness scale in obstructive sleep disordered breathing: the reliability and validity of the Thai version. Sleep Breath. 2011, 15: 571-577. 10.1007/s11325-010-0405-9.PubMedView ArticleGoogle Scholar
- Onen F, Moreau T, Gooneratne SN, Petit C, Falissard B, Hakki OS: Limits of the Epworth Sleepiness Scale in older adults. Sleep Breath. 2012, 17: 343-350.PubMedView ArticleGoogle Scholar
- Ramachandran SK, Josephs LA: A Meta-analysis of Clinical Screening Tests for Obstructive Sleep Apnea. Anesthesiology. 2009, 110: 928-939. 10.1097/ALN.0b013e31819c47b6.PubMedView ArticleGoogle Scholar
- Waltz CF, Strickland OL, Lenz ER: Measurements in Nursing and Health Research. 2010, New York: Springer publishing company, 4Google Scholar
- Field A: Discovering Statistics using SPSS. 2009, London: Sage, 3Google Scholar
- Landis JR, Koch GG: Measurement of observer agreement for categorical data. Biometrics. 1977, 33: 159-174. 10.2307/2529310.PubMedView ArticleGoogle Scholar
- Sagaspe P, Leger D, Taillard J, Bayon V, Chaumet G, Philip P: Might the Berlin Sleep Questionnaire applied to bed partners be used to screen sleep apneic patients?. Sleep Med. 2010, 11: 479-483. 10.1016/j.sleep.2010.01.007.PubMedView ArticleGoogle Scholar
- Sharma SK, Vasudev C, Sinha S, Banga A, Pandey RM, Handa KK: Validation of the modified Berlin questionnaire to identify patients at risk for the obstructive sleep apnoea syndrome. Indian J Med Res. 2006, 124: 281-290.PubMedGoogle Scholar
- Vaz AP, Drummond M, Mota PC, Severo M, Almeida J, Winck JC: Translation of Berlin questionnaire to Portuguese language and its application in OSA identification in a sleep disordered breathing clinic. Rev Port Pneumol. 2011, 17: 59-65. 10.1016/S0873-2159(11)70015-0.PubMedView ArticleGoogle Scholar
- Singh A, Stewart A, Franzen D: Obstructive sleep apnea as a risk factor for stroke and death. Int J Ther Rehabil. 2011, 18: 683-693. 10.12968/ijtr.2011.18.12.683.View ArticleGoogle Scholar
- Zamparo P, Francescato MP, De Luca G, Lovati L, di Prampero PE: The energy cost of level walking in patients with hemiplegia. Scand J Med Sci Sports. 1995, 5: 348-352.PubMedView ArticleGoogle Scholar
- Appelros P: Prevalence and predictors of pain and fatigue after stroke: a population-based study. Int J Rehabil Res. 2006, 29: 329-333. 10.1097/MRR.0b013e328010c7b8.PubMedView ArticleGoogle Scholar
- Hubacher M, Calabrese P, Bassetti C, Carota A, Stoecklin M, Penner I-K: Assessment of Post-Stroke Fatigue: The fatigue scale for motor and cognitive functions. Eur Neurol. 2012, 67: 377-384. 10.1159/000336736.PubMedView ArticleGoogle Scholar
- Pasic Z, Smajlovic D, Dostovic Z, Kojic B, Selmanovic S: Incidence and types of sleep disorders in patients with stroke. Med Arh. 2011, 65: 225-227.PubMedView ArticleGoogle Scholar
- Barba C, Cavalli-Sforza T, Cutter J, Darnton-Hill I, Deurenberg P, Deurenberg-Yap M, Gill T, James P, Ko G, Miu AH, Kosulwat V, Kumanyika S, Kurpad A, Mascie-Taylor N, Moon HK, Nakadomo F, Nishida C, Noor MI, Reddy KS, Rush E, Schultz JT, Seidell J, Stevens J, Swinburn B, Tan K, Weisell R, Zhao-su W, Yajnik CS, Yoshiike N, Zimmet P: Appropriate body-mass index for Asian populations and its implications for policy and intervention strategies. Lancet. 2004, 363: 157-163.View ArticleGoogle Scholar
- Temcharoen P, Kaewboonruang P, Pradipasen M, Srisorachart S: The optimal cut-off points of body mass index which reflect the risk factors of cardiovascular disease in the urban Thai male population. J Med Assoc Thai. 2009, 92 (Suppl 7): S68-74.PubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.