Skip to main content

Preliminary validity and reliability of a Thai Berlin questionnaire in stroke patients



Obstructive sleep apnea (OSA) is a major risk factor for stroke. The Berlin Questionnaire (BQ) has been shown to be a valid tool to screen for OSA. The literature has limited data on using the BQ in stroke patients; particularly in Thailand and other developing countries. Here, we aimed to develop a Thai-language Berlin Questionnaire (Thai BQ) and to preliminarily assess construct validity, test-retest reliability and the agreement of the Thai BQ with the Thai Epworth Sleepiness Scale (Thai ESS), another screening tool for OSA.


A hospital-based cross-sectional study was performed from January to July, 2011. One hundred first-ever stroke patients, including acute and chronic cases, and their caregivers were enrolled. The Thai BQ was developed using the forward-backward translation method. Evaluation of construct validity was done by factor analysis. Internal consistency of the Thai BQ and the Thai ESS were evaluated using Cronbach’s alpha coefficient. Test-retest reliability and the agreement of the Thai BQ and the Thai ESS were evaluated using Cohen’s kappa coefficient.


Factor analysis identified 4 main factors: Factor 1–Snoring behaviour; Factor 2–Sleepiness during driving; Factor 3–Daytime fatigue; and Factor 4–Hypertension or obesity. Cronbach’s alpha coefficient was 0.77 (95% confidence interval (CI) = 0.69-0.83) and Cohen’s kappa coefficient was 0.86 (95% CI = 0.74-0.98) in the Thai BQ. Cronbach’s alpha coefficient was 0.59 (95% CI = 0.45-0.70) and the Cohen’s kappa coefficient was 0.81 (95% CI = 0.60-1.00) in the Thai ESS. The agreement between the Thai ESS and the Thai BQ was fair.


The Thai BQ is a valid and reliable tool to screen for OSA in stroke patients. As factor analysis revealed 4 factors in contrast to the 3 factors in the original BQ, further modification of the Thai BQ is required.


The prevalence of obstructive sleep apnea (OSA) in stroke patients is higher than in the general population (55-71% vs. 12–18.5%) [14]. OSA is not only an independent risk factor for stroke: stroke patients with OSA also have worse outcomes, higher rates of recurrent stroke and higher mortality rates [57]. Moreover, recent evidence has shown that continuous positive airway pressure (CPAP) treatment in acute stroke improved neurological outcome compared to control treatment if OSA was detected [8]. Thus, it is important to identify OSA in stroke patients. OSA in stroke patients, however, has not been widely evaluated, partly because of the lack of awareness of the impact of OSA in stroke and the limited availability of polysomnography.

In an attempt to address this need, many questionnaires have been developed to identify people at risk of OSA. Two of the most widely used self-administered screening tools are the Berlin Questionnaire (BQ) and the Epworth Sleepiness Scale (ESS) [9, 10]. The ESS contains 8 subjective and self-administered items relating to daytime sleepiness. In Thailand, the Thai ESS has already been translated and tested in both healthy volunteers and able-bodied participants with OSA. It has high internal consistency demonstrated by Cronbach’s alpha coefficient of 0.87 and high test-retest reliability as shown by an intra-class correlation coefficient of 0.79 [11]. For ESS, however, Onen et al. found that almost 60% of people aged more than 65 years were not able to answer at least one question in the ESS. Further, age and cognitive status were related to differences in the ESS score [12]. Many stroke patients may also have these limitations.

The BQ has a total of 11 questions in 3 categories: snoring behaviour, waketime fatigue or sleepiness, and the diagnosis of hypertension or obesity. It has good internal consistency as demonstrated by a Cronbach’s alpha value of 0.86-0.92 [9]. In addition, Ramachandran and Josephs published findings from a meta-analysis of clinical screening tests for OSA and concluded that the BQ was the most accurate questionnaire for predicting diagnosis of OSA [13].

The BQ and ESS as screening tools for OSA in stroke patients are limited in clinical practice particularly in Thailand and other developing countries (Additional file 1). Both tools are helpful for clinicians to select appropriate cases for referral to sleep facilities which are lacking in the developing world. The aims of this study were to 1) develop a Thai-language version of the BQ, 2) study the preliminary validity and reliability of the Thai BQ, and 3) compare the agreement of the caregiver-administered Thai BQ with the patient-administered Thai ESS.


The cross-sectional study was conducted in both the in and out-patient units in Srinagarind Hospital, Thailand, from January to July 2011. The study was reviewed and approved by the Khon Kaen University Ethics Committee for Human Research (HE531422). All participants signed a consent form prior to participation.


The eligible population was comprised of patients older than 18 years with first-ever haemorrhagic or ischemic stroke, with radiographic evidence of cerebrovascular disease. Exclusion criteria were: 1) Mini-Mental State Examination of less than 24, aphasia or psychiatric disorder; 2) concurrent cardiopulmonary disease affecting breathing or tiredness, e.g., congestive heart failure or asthma; 3) coexisting neurological conditions such as myasthenia gravis and 4) taking medication affecting sleep.

Translation of the original BQ into the Thai language

Translation of the BQ was kindly permitted by the original developer [9]. The translation involved converting the questions and rating criteria of the original BQ (Additional files 2 and 3). The Thai version was then translated back to English (forward-backward translation method) using two bilingual translators (one physician of rehabilitation medicine and one of sleep medicine).

The translated version was reviewed by an expert panel of four to evaluate the equivalence to the original version and the content validity. The expert panel consisted of one physician of rehabilitation medicine, one otolaryngologist, one sleep medicine physician and one sleep lab nurse. The content validity of the Thai BQ was judged by item-objective congruence (IOC). Each expert independently rated the relevance of each item on the Thai BQ and the original BQ to determine whether the translated items fell within the content domain as specified by the original BQ, using a scale of +1 (clear agreement), 0 (unclear) or −1 (no clear agreement). The IOC for each item was the summation of scores given by the experts divided by the number of experts. The average IOC score was 0.86. Every item in the Thai BQ scored more than 0.75 which is considered acceptable [14].

The Thai BQ version was then used to test structural validity, test-retest reliability and the agreement of the Thai BQ and the Thai ESS by interviewing the caregiver of stroke patients for the Thai BQ and the stroke patients themselves for the Thai ESS. All patients and caregivers completed the questionnaire twice within 48 hours for the test-retest reliability analysis. High risk patients were defined as those positive in more than 2 categories of the BQ or having a Thai ESS score equal to or more than 10.

Baseline characteristics of patients who participated were recorded: age, gender, weight, height, side of stroke and type of stroke together with underlying disease profile.

Statistical analysis

Construct validity was tested using exploratory factor analysis to identify the latent factors underlying the 11 questions of the BQ. The internal consistency of each category was calculated using Cronbach’s alpha coefficient. The test-retest reliability of the Thai BQ and agreement between the Thai BQ and the Thai ESS were examined using Cohen’s kappa coefficient. The statistical analysis was carried out using SPSS version 19.


A total of 100 first-ever stroke patients were included in the study. Of those, fifty-one patients were male (51%). The mean age and BMI of all patients were 64 years (range 37–95 years) and 24 kg/m2 (Table 1). Regarding stroke types, 70 patients had ischemic strokes, while 25 patients had hemorrhagic stroke and 5 patients had both ischemic and haemorrhagic stroke. The median time after stroke was 14 days (1st and 3rd quartile: 3–150 days). The main comorbidities were hypertension (42%) and diabetes mellitus (34%).

Table 1 General characteristics of stroke patients (n = 100) who participated in the study

Snoring and witnessed apnea was found in 67% and 3% of patients. 45% and 4% of patients were positive for categories 1 and 2 (Table 2). Twelve patients had daytime fatigue and seven patients had fallen asleep while driving. The percentage of patients with hypertension or obesity was 47%. Twenty-four patients had a high risk for OSA as identified by the Thai BQ. The average Thai ESS score of all patients was 4.6. Eight patients were at risk for OSA as identified by the Thai ESS.

Table 2 Numbers of responses for each item of the Thai BQ (n = 100)

Construct validity

Principal component analysis (PCA) was conducted on the 11 items of the BQ with orthogonal rotation (varimax). The Kaiser-Meyer-Olkin measure verified the sampling adequacy for the analysis, KMO = 0.724 (‘good’ sample size). Barlett’s test of sphericity x2 (55) = 742.3, p <; 0.001, indicated that correlations between items were sufficiently large for PCA [15]. An initial analysis was performed to obtain eigenvalues for each component in the data. Four factors had eigenvalues above Kaiser’s criterion of 1 and in combination explained 76.2% of the variance (Table 3). Our results revealed 4 main factors instead of 3 main factors (categories) as classified in the original BQ. These can be described as: factor 1-snoring behaviour; factor 2-daytime fatigue; factor 3-sleepiness during driving; and factor 4-hypertension or obesity. Factors 1 and 4 in the Thai BQ corresponded to categories 1 and 3 in the original BQ. Factors 2 and 3 in combination represented category 2 in the original BQ.

Table 3 Factor structure of the Thai BQ

Reliability of the Thai BQ and the Thai ESS

The internal consistency of factor 1 of the Thai BQ was high (Cronbach’s alpha = 0.89). However, factor 3 which depicted daytime fatigue, the internal consistency was low (Cronbach’s alpha = 0.42) [15]. Since the fourth factor was summed into only 1 in total when calculating the risk for OSA, this factor was excluded from the calculations of internal consistency. The overall internal consistency of the Thai BQ was acceptable (Cronbach’s alpha = 0.77). The test-retest reliability (Cohen’s kappa coefficient) of the Thai BQ ranged from 0.66 to 0.98, (substantial to almost perfect agreement) [16]. Factor 3 had the lowest kappa value (Table 4). The internal consistency of the Thai ESS as evaluated by Cronbach’s alpha was 0.59 (95% CI = 0.45-0.70) and the test-retest reliability calculated using Cohen’s kappa coefficient was 0.81 (95% CI = 0.60-1.00).

Table 4 Evaluation of internal consistency and test-retest reliability of the factors of the Thai BQ

The agreement of the diagnosis of the high risk of OSA from the Thai BQ and the Thai ESS was also demonstrated by Cohen’s kappa coefficient. The results showed kappa values of 0.29 (95% CI = 0.10-0.48), which means that the agreement of the Thai BQ and the Thai ESS was fair, (Table 5) [16].

Table 5 Agreement between the Thai BQ and the Thai ESS


The Thai BQ was developed by translation and adaptation of the original BQ, which allows assessment of the risk of OSA in general populations [9]. The preliminary validity and reliability of the Thai BQ was examined. The results of the study showed that the Thai BQ had good internal consistency in factor 1 (snoring behaviour) but not in factor 3 (daytime fatigue). Overall, the Thai BQ had good internal consistency and good test-retest reliability. The BQ allows the stroke caregiver to respond to objective questions relating to factors including snoring behaviour, i.e., the questionnaire rates the loudness and frequency of snoring. This was also shown in the study of Sagaspe et al., where the results of self-reported and bed-partner-reported BQ were compared: it was found that the bed-partner-reported BQ showed slightly better results. The authors concluded that this may be due to better identification of snoring or excessive daytime sleepiness by bed partners [17].

The prevalence of high risk for OSA identified from this study was very low (24%), although the study was conducted in a higher mean age group (mean age of 64 years) compared to other studies [14]. Twenty one patients (87.5%) were diagnosed as being at a high risk for OSA mainly by categories 1 and 3, 1 patient (4.2%) by categories 1 and 2, 1 patient by categories 2 and 3, and 1 patient by all 3 categories. This result was similar to other cross-sectional studies and a meta-analysis which showed that category 1 and category 3 are significant elements in the OSA population [13, 18, 19].

Due to the fact that stroke mostly occurs in middle-aged and elderly people, stroke patients usually have a higher energy expenditure while walking or performing daily life activities [20, 21]. This can lead to tiredness and there is a high prevalence of fatigue in stroke patients, ranging from 16.1–58.1% [22, 23]. In addition, stroke patients were found to have a high prevalence of sleep disorders (78%) such as insomnia or OSA [24]. Stroke patients should have both a high percentage of positive in category 2 and high risk for OSA identified from the ESS. Surprisingly, this study did not demonstrate this and the positive result in category 2 is low. There are some possible explanations for the low positive results in category 2 such as: 1) the questions in category 2 (factor 3) may not directly reflect daytime fatigue and the patients’ caregivers may not report fatigue because they considered fatigue as normal for stroke patients; 2) Thai elderly normally take a nap during the day so they feel less sleepy or fatigued during their waking time-the questions in factor 3 mainly focus on daytime fatigue or tiredness but do not involve sleepiness; lastly, 3) another possibility may relate to the question regarding sleepiness while driving, since stroke patients do not normally drive. Thus questions 8 and 9 in category 2 will be reported as 0: this was found in 93% of the answers. This was also observed in the study of Banhiran et al. which studied the validity and reliability of the Thai ESS in able-bodied participants. They found that participants responded to the question related to sleepiness in a car while stopped for a few minutes in the traffic as “no chance of dozing” because most participants do not drive [11].

The low reporting of sleepiness while driving may explain why 2 factors from factor analysis (daytime fatigue and sleepiness during driving) from category 2 are yielded instead of only 1. As the original BQ mainly aimed to assess the risk of OSA in the general population, the questions need to be modified before use in stroke patients to conform better to their daily lives.

There was only fair agreement between high risk of OSA diagnosed from the Thai BQ and the Thai ESS. This may be a result of the difference between the questionnaires, as a high risk of OSA was diagnosed using different criteria. The high risk of OSA according to the ESS mainly focusses on daytime sleepiness while for the BQ in this study the diagnosis relied on snoring behaviour and the diagnosis of hypertension or obesity. The internal consistency and test-retest reliability were lower in the present study compared to a previous study in Thailand. This may be due to differences between the respective study populations: the study by Banhiran et al., included able-bodied subjects with a lower average age [11].

As there was a discrepancy between the results of the Thai BQ and the Thai ESS for identifying the high risk group for OSA, 10 patients in the discrepancy group agreed to be investigated by polysomnography. Of the 10 patients, 1 was diagnosed as high risk by the Thai ESS and 9 were diagnosed as high risk by the Thai BQ. All 10 patients were diagnosed as OSA by polysomnography. Although this study shows a trend towards the Thai BQ in diagnosing OSA in stroke patients, further examination of the screening properties of the Thai BQ is required.

There are some limitations in the study. Patients with cardiopulmonary disease, aphasia or cognitive perceptual deficit were excluded. This exclusion may account for the lower prevalence of high risk OSA identified in this study population. Furthermore, mainly because of resource limitations, all participants were unable to be studied using overnight polysomnography so the predictive validity and the test properties of the Thai BQ could not be demonstrated. This study may provide baseline knowledge for further development of a modified version of the Thai BQ for the diagnosis of high risk OSA for stroke patients.

The next step is to focus on modification of the Thai BQ in category 2. The question regarding sleepiness while driving may need to be changed to a more appropriate question involving sleepiness during daily activities that are relevant to stroke patients. The appropriate cut off point for BMI should also be evaluated since the diagnosis of obesity in Asia and in western countries is different [25, 26]. Finally, a study of the discriminative and predictive validity of the BQ in comparison with the gold standard, which is polysomnography, should be carried out.


In conclusion, the Thai BQ has been developed. The study showed that the Thai BQ has an acceptable preliminary internal consistency and test-retest reliability. As factor analysis revealed 4 factors, in contrast to the 3 factors in the original study, modification of the Thai BQ is required. Comparison with the gold standard for the predictive validity of the Thai BQ is desirable before administration of the questionnaire in Thai stroke patients.


  1. Chan W, Coutts SB, Hanly P: Sleep apnea in patients with transient ischemic attack and minor stroke: opportunity for risk reduction of recurrent stroke?. Stroke. 2010, 41: 2973-2975. 10.1161/STROKEAHA.110.596759.

    Article  PubMed  Google Scholar 

  2. Srijithesh PR, Shukla G, Srivastav A, Goyal V, Singh S, Behari M: Validity of the Berlin Questionnaire in identifying obstructive sleep apnea syndrome when administered to the informants of stroke patients. J Clin Neurosci. 2011, 18: 340-343. 10.1016/j.jocn.2010.04.048.

    Article  PubMed  CAS  Google Scholar 

  3. Dyken ME, Somers VK, Yamada T, Ren ZY, Zimmerman B: Investigating the relationship between stroke and obstructive sleep apnea. Stroke. 1996, 27: 1668-1677.

    Article  Google Scholar 

  4. Bassetti C, Aldrich MS: Sleep apnea in acute cerebrovascular diseases: Final report on 128 patients. Sleep. 1999, 22: 217-223.

    PubMed  CAS  Google Scholar 

  5. Yaggi HK, Concato J, Kernan WN, Lichtman JH, Brass LM, Mohsenin V: Obstructive sleep apnea as a risk factor for stroke and death. N Eng J Med. 2005, 353: 2034-2041. 10.1056/NEJMoa043104.

    Article  CAS  Google Scholar 

  6. Mansukhani MP, Bellolio MF, Kolla BP, Enduri S, Somers VK, Stead LG: Worse outcome after stroke in patients with obstructive sleep apnea: an observational cohort study. J Stroke Cerebrovasc Dis. 2011, 20: 401-405. 10.1016/j.jstrokecerebrovasdis.2010.02.011.

    Article  PubMed  Google Scholar 

  7. Good DC, Henkle JQ, Gelber D, Welsh J, Verhulst S: Sleep-disordered breathing and poor functional outcome after stroke. Stroke. 1996, 27: 252-259. 10.1161/01.STR.27.2.252.

    Article  PubMed  CAS  Google Scholar 

  8. Parra O, Sanchez-Armengol A, Bonnin M, Arboix A, Campos-Rodriquez F, Pèrez-Ronchel J, Duran-Cantolla J, de la Torre G, Gonzalez M, de la Pena M, Carmen Jiménez M, Masa F, Casado I, Luz Alonso M, Macarron JL: Early treatment of obstructive apnoea and stroke outcome: a randomised controlled trial. Eur Respir J. 2011, 37: 1128-36. 10.1183/09031936.00034410.

    Article  PubMed  CAS  Google Scholar 

  9. Netzer NC, Stoohs RA, Netzer CM, Clark K, Strohl KP: Using the Berlin questionnaire to identify patients at risk for the sleep apnea syndrome. Ann Intern Med. 1999, 131: 485-491.

    Article  PubMed  CAS  Google Scholar 

  10. Johns MW: A new method for measuring daytime sleepiness - the epworth sleepiness scale. Sleep. 1991, 14: 540-545.

    PubMed  CAS  Google Scholar 

  11. Banhiran W, Assanasen P, Nopmaneejumruslers C, Metheetrairut C: Epworth sleepiness scale in obstructive sleep disordered breathing: the reliability and validity of the Thai version. Sleep Breath. 2011, 15: 571-577. 10.1007/s11325-010-0405-9.

    Article  PubMed  Google Scholar 

  12. Onen F, Moreau T, Gooneratne SN, Petit C, Falissard B, Hakki OS: Limits of the Epworth Sleepiness Scale in older adults. Sleep Breath. 2012, 17: 343-350.

    Article  PubMed  Google Scholar 

  13. Ramachandran SK, Josephs LA: A Meta-analysis of Clinical Screening Tests for Obstructive Sleep Apnea. Anesthesiology. 2009, 110: 928-939. 10.1097/ALN.0b013e31819c47b6.

    Article  PubMed  Google Scholar 

  14. Waltz CF, Strickland OL, Lenz ER: Measurements in Nursing and Health Research. 2010, New York: Springer publishing company, 4

    Google Scholar 

  15. Field A: Discovering Statistics using SPSS. 2009, London: Sage, 3

    Google Scholar 

  16. Landis JR, Koch GG: Measurement of observer agreement for categorical data. Biometrics. 1977, 33: 159-174. 10.2307/2529310.

    Article  PubMed  CAS  Google Scholar 

  17. Sagaspe P, Leger D, Taillard J, Bayon V, Chaumet G, Philip P: Might the Berlin Sleep Questionnaire applied to bed partners be used to screen sleep apneic patients?. Sleep Med. 2010, 11: 479-483. 10.1016/j.sleep.2010.01.007.

    Article  PubMed  Google Scholar 

  18. Sharma SK, Vasudev C, Sinha S, Banga A, Pandey RM, Handa KK: Validation of the modified Berlin questionnaire to identify patients at risk for the obstructive sleep apnoea syndrome. Indian J Med Res. 2006, 124: 281-290.

    PubMed  CAS  Google Scholar 

  19. Vaz AP, Drummond M, Mota PC, Severo M, Almeida J, Winck JC: Translation of Berlin questionnaire to Portuguese language and its application in OSA identification in a sleep disordered breathing clinic. Rev Port Pneumol. 2011, 17: 59-65. 10.1016/S0873-2159(11)70015-0.

    Article  PubMed  CAS  Google Scholar 

  20. Singh A, Stewart A, Franzen D: Obstructive sleep apnea as a risk factor for stroke and death. Int J Ther Rehabil. 2011, 18: 683-693. 10.12968/ijtr.2011.18.12.683.

    Article  Google Scholar 

  21. Zamparo P, Francescato MP, De Luca G, Lovati L, di Prampero PE: The energy cost of level walking in patients with hemiplegia. Scand J Med Sci Sports. 1995, 5: 348-352.

    Article  PubMed  CAS  Google Scholar 

  22. Appelros P: Prevalence and predictors of pain and fatigue after stroke: a population-based study. Int J Rehabil Res. 2006, 29: 329-333. 10.1097/MRR.0b013e328010c7b8.

    Article  PubMed  Google Scholar 

  23. Hubacher M, Calabrese P, Bassetti C, Carota A, Stoecklin M, Penner I-K: Assessment of Post-Stroke Fatigue: The fatigue scale for motor and cognitive functions. Eur Neurol. 2012, 67: 377-384. 10.1159/000336736.

    Article  PubMed  Google Scholar 

  24. Pasic Z, Smajlovic D, Dostovic Z, Kojic B, Selmanovic S: Incidence and types of sleep disorders in patients with stroke. Med Arh. 2011, 65: 225-227.

    Article  PubMed  Google Scholar 

  25. Barba C, Cavalli-Sforza T, Cutter J, Darnton-Hill I, Deurenberg P, Deurenberg-Yap M, Gill T, James P, Ko G, Miu AH, Kosulwat V, Kumanyika S, Kurpad A, Mascie-Taylor N, Moon HK, Nakadomo F, Nishida C, Noor MI, Reddy KS, Rush E, Schultz JT, Seidell J, Stevens J, Swinburn B, Tan K, Weisell R, Zhao-su W, Yajnik CS, Yoshiike N, Zimmet P: Appropriate body-mass index for Asian populations and its implications for policy and intervention strategies. Lancet. 2004, 363: 157-163.

    Article  Google Scholar 

  26. Temcharoen P, Kaewboonruang P, Pradipasen M, Srisorachart S: The optimal cut-off points of body mass index which reflect the risk factors of cardiovascular disease in the urban Thai male population. J Med Assoc Thai. 2009, 92 (Suppl 7): S68-74.

    PubMed  Google Scholar 

Download references


We gratefully acknowledge the contributions of the staff in the expert panel who shared their opinions on the Thai BQ and the staff in the polysomnography laboratory. This study was supported by TRF grants from Senior Research Scholar Grant, Thailand Research Fund grant number RTA5580004 and the Higher Education Research Promotion and National Research University Project of Thailand, Office of the Higher Education Commission, Thailand through the Health Cluster (SHeP-GMS), Khon Kaen University. The authors would like to thank Prof. James A. Will (University of Wisconsin) for his kind review of the manuscript.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Kittisak Sawanyawisuth.

Additional information

Competing interests

The authors declare no conflict of interest.

Authors’ contributions

JitS and NU contributed the study design, data collection, analysis, interpretation, manuscript preparation and revision. KS and JiaS helped with study design, analysis, interpretation, manuscript preparation and revision. All authors read and approved the final version of the manuscript.

Electronic supplementary material

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver ( ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saengsuwan, J., Ungtrakul, N., Saengsuwan, J. et al. Preliminary validity and reliability of a Thai Berlin questionnaire in stroke patients. BMC Res Notes 7, 348 (2014).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: