Research article | Open | Published:
Development and validation of the Japanese version of cognitive flexibility scale
BMC Research Notesvolume 9, Article number: 275 (2016)
Various instruments have been developed to assess cognitive flexibility, which is an important construct in psychology. Among these, the self-report cognitive flexibility scale (CFS) is particularly popular for use with English speakers; however, there is not yet a Japanese version of this scale. This study reports on the development of a Japanese version of the cognitive flexibility scale (CFS-J), and the assessment of its internal consistency, test–retest reliability, and validities.
We used the standard translation–back-translation process to develop the Japanese wording of the items and tested these using a sample of 335 eligible participants who did not have a mental illness, were aged 18 years or older, and lived in the suburbs of Tokyo. Participants included office workers, public servants, and college students; 71.6 % were women and 64.8 % were students. The translated scale’s internal consistency reliability was assessed by calculating Cronbach’s alpha and McDonald’s omega, and test–retest reliability was assessed with 107 eligible participants via intra-class correlation coefficient (ICC) and Spearman’s correlation of coefficient. Exploratory factory analysis (EFA) and correlations with other scales were used to examine the factor-based and concurrent validities of the CFS-J.
Results indicated that the CFS-J has good internal consistency (Cronbach’s alpha = 0.847, McDonald’s omega = 0.871) and acceptable test–retest reliability (Spearman’s = 0.687, ICC = 0.689). EFA provided evidence that the CFS-J has a one-factor structure and factor loadings were generally appropriate. The total CFS-J score was significantly and positively correlated with the cognitive flexibility inventory-Japanese version and its two subscales, along with the cognitive control scale and the positive subscale of the short Japanese version of the automatic thought questionnaire–revised (ATQ-R); further, it had a significantly negative correlation with the negative subscale of the ATQ-R (ps < 0.001). This study developed a Japanese version of the cognitive flexibility scale and confirmed its reliability and validity among a sample of people with no current mental illness, who were living in the suburbs of Tokyo.
Cognitive flexibility is one aspect of executive functioning that encompasses the ability to produce diverse ideas, consider response alternatives, and modify behaviors to manage changing circumstances . The most popular definition appears to be “the readiness with which the person’s concept system changes selectively in response to appropriate environmental stimuli” , and Spiro and Jehng  defined cognitive flexibility, in their cognitive flexibility theory, as “the ability to adaptively re-assemble diverse elements of knowledge to fit the particular needs of a given understanding or problem-solving situation,” and Johnco  stated that “cognitive flexibility is likely to be an important mental ability to facilitate the learning of cognitive restructuring as a skill to increase adaptive functioning and the ability to adjust to changes in life circumstances.”
Many instruments have been developed to assess cognitive flexibility, including performance-based measures, such as the Wisconsin card sorting test  and the Stroop color and word test . These “task-oriented” instruments generally require ponderous processing by administrators, and are difficult for applying to large groups or for continuous implementation in clinical situations; thus, self-reported measures are more practical owing to their brevity and easy administration. Two self-reported questionnaires have been developed to assess cognitive flexibility: the cognitive flexibility scale (CFS)  and the cognitive flexibility inventory (CFI) . Because the CFS was developed earlier than the CFI, a number of subsequent studies have utilized the former instrument for assessing cognitive flexibility, and reported the usefulness of the CFS for assessing the severity of various psychiatric disorders in clinical situations, e.g., generalized anxiety disorder , depression and anxiety disorders [3, 9], eating disorders , and posttraumatic stress disorder [11, 12].
The above evidence suggests that cognitive flexibility could also be a useful measure of the outcomes of psychiatric treatment. However, there is no existing Japanese language version of the CFS, which has been used as the popular instrument for measuring cognitive flexibility especially in the English-speaking world; therefore, in this study we translated this scale into Japanese. We also conducted an initial exploration of the internal consistency reliability, test–retest reliability, and concurrent validity (including factor-based validity) of the Japanese version of the CFS (CFS-J) among people with no existing mental illness and who were living in the suburbs of Tokyo.
We translated the CFS into Japanese with the permission of the original author , then conducted a back-translation into English to preserve its correspondence with the original scale, other than allowing for linguistic and cultural differences. Six experts with backgrounds in psychiatry (two), psychiatric nursing (one), and clinical psychology (three), who were native Japanese speakers with a good command of English, translated the 12 CFS items into Japanese. A translation company performed the Japanese to English back-translation, and a bilingual English literature professor who was not given any information about the CFS checked for correspondence of meaning between the translated and original versions of the scale. Then, the back-translated version was sent to the author of the CFS to check for agreement of meaning with the original scale, with comments provided on problematic items. We repeated the translation and back-translation procedures until the author approved the correspondence between all the items in the English and Japanese versions, and this version was examined in the study described below. These translation and back-translation processes follow the usual guidelines for cross-cultural scale adaptations .
The sample size of 300 was targeted according to the recommended procedure  for structural equation modeling sampling sufficiency (n:q = 10:1 ratio of number of subjects to number of test items) . Participants were volunteers recruited from October 2014 to February 2015 by the authors and their assistants, and included office workers at private companies, public school teachers, and junior college and university students living in the suburbs of Tokyo. An explanation of the aims and implications of the study and data collection procedures, including information that non-participants would not be disadvantaged, was provided on the cover page of the questionnaire, and 389 participants that consented to take part in the study submitted informed consent and completed the questionnaires. None of the participants were undergoing regular treatment for a mental illness at a hospital. Of the 389 eligible participants, we excluded 54 because they gave one or more missing responses or apparently unreliable responses (e.g., filling the same response number for all items). We used data from the remaining 335 (86.1 % of the initial sample) participants for further analyses. Furthermore, a subsample (n = 168) of the eligible participants was asked to complete the same version of the CFS-J one to 2 weeks later to examine the scale’s test–retest reliability, and valid data from both data collections were obtained from 107 respondents. This study was approved by the ethics committee of the Graduate School of Medicine at Chiba University (receipt number 1894).
Cognitive flexibility scale-Japanese version (CFS-J)
The CFS-J is 12-item self-report questionnaire ("Appendix"). The original English-language version of the CFS was developed by Martin and Rubin  to assess three aspects of cognitive flexibility: (a) the awareness that options and alternatives are available in any given situation, (b) the willingness to be flexible and adapt to the situation, and (c) self-efficacy in being flexible. Responses are made on a 6-point Likert scale: 6 (strongly agree), 5 (agree), 4 (slightly agree), 3 (slightly disagree), 2 (disagree), and 1 (strongly disagree). Items 2, 3, 5, and 10 of the scale are reverse-coded. Total scores range from 12 to 72, with higher scores indicating better cognitive flexibility (see Table 2 for the content of the original questionnaire). The original authors  and successive study  have shown that it has good concurrent, construct, and criterion-related validity when used with college student samples. The factor structure of the original CFS was reported in neither the original study , nor successive study , but Dennis and Vander Wal  reported that the scale has a one-factor structure, although no evidence for this was provided. The CFS has also been translated into Turkish and confirmed to be valid and reliable .
Cognitive flexibility inventory-Japanese version (CFI-J)
The CFI  is a 20-item, two-subscale self-report questionnaire, to which responses are made on a 7-point Likert scale ranged from 1 = “strongly disagree” to 7 = “strongly agree”. The CFI is designed to assess cognitive flexibility in relation to thinking adaptively, rather than maladaptively, when encountering stressful life events. The “alternative” subscale (10 items; α = 0.91) reflects a person’s ability to generate multiple solutions to difficult situations and perceive multiple alternative explanations of events; the “control” subscale (six items; α = 0.84–0.86) reflects a person’s tendency to perceive difficult situations as being controllable. Higher scores on the CFI are indicative of higher cognitive flexibility. The reliability and validity of the CFI-J’s alternative (α = 0.88) and control (α = 0.77) subscales have been confirmed by Tokuyoshi and Iwasaki .
Cognitive control scale (CCS)
Cognitive control refers to processes that allow information processing and behavior to vary adaptively from moment to moment depending on current goals, rather than remaining rigid and inflexible . The CCS  is an 11-item Japanese language self-report questionnaire designed to assess the ability to voluntarily use cognitive behavioral therapy-like skills in daily life, and is based on Freeman’s cognitive skills . Responses to the CCS are scored on a 4-point Likert scale ranging from 1 = “never” to 4 = “most surely”. The CCS (α = 0.81) comprises two subscales: “logical analysis” (six items; α = 0.79), which measures active and objective problem-solving skills, and “refraining from catastrophic thinking” (five items; α = 0.72), which measures the ability to be detached from negative thinking to alleviate catastrophic cognitions.
Automatic thought questionnaire-revised-short version in Japanese (ATQ-R-JS)
The ATQ-R  is a 40-item self-report questionnaire comprising 10 positive and 30 negative items that appear in the original ATQ, which was developed by Hollon and Kendall  to assess the extent of an individual’s experience of negative and positive automatic thoughts. Responses are made on a 5-point Likert scale ranging from 1 = “not at all” to 5 = “all the time”. Because the 40-item ATQ-R is not considered an ideal instrument for use in clinical situations, Sakamoto et al.  developed a short version in Japanese (ATQ-R-JS), which comprises two subscales of negative (six items; α = 0.88) and positive (six items; α = 0.82) automatic thoughts.
Cronbach’s alpha and McDonald’s omega coefficients were used to examine the internal consistency of the CFS-J, and Spearman’s correlation coefficient and intra-class correlation coefficients (ICC) were used to establish the test–retest reliability over an interval of 1–2 weeks. The scale’s concurrent validity was assessed through calculating correlations with related measures, comprising the CFI-J, CCS, and ATQ-R-JS. We performed an exploratory factor analysis (EFA) to examine the factor structure of the CFS, using the maximum likelihood method with promax rotation and Kaiser’s normalization. We used the following criteria to determine the optimal number of factors: Kaiser’s eigenvalues , parallel analysis , and minimum average partials (MAP) . Factor loadings of 0.4 or greater were considered to be acceptable . All data were analyzed using SPSS for Windows (version 15) and R (version 3.1.3).
Respondents’ demographic characteristics and total score distributions
The demographic characteristics of the eligible respondents (n = 335) and their scale score means and standard deviations are listed in Table 1. About 72 % were female. We do not have the exact distribution of participants’ ages because we collected this information as stratified data; however, all were aged 18 years or older and the median age stratum was 20 s. The mean CFS-J total score was 46.3, with a standard deviation of 7.7, minimum of 24, maximum of 71, kurtosis of 0.26, and skewness of 0.27, and the hypothesis of normal distribution was not supported (p = 0.047) by the Shapiro–Wilk test of normality.
Cronbach’s alpha coefficient for the CFS-J was 0.847 and McDonald’s omega coefficient was 0.871, indicating the acceptable range for internal consistency . When the alpha coefficient for each of the 12 items was calculated by successively eliminating each item, results ranged from 0.819 to 0.852. Simultaneously eliminating items 2, 5, and 11 improved the internal consistency substantially to 0.863 (omega = 0.888), and the means (SD) of the nine-item CFS-J were 34.7 (6.4), 37.0 (6.6), and 33.9 (6.0) for total scores, men, and women, respectively. The correlation between scores on the 12- and nine-item versions was significantly high at 0.969.
The test–retest reliability assessment was conducted with 107 eligible responders (33 men and 74 women). Mean (SD) scores on the CFS-J were 47.1 (7.4) for the total sample, 48.9 (7.4) for men, and 46.3 (6.9) for women. The test–retest correlation had a moderate to strong range (Spearman’s = 0.687, ICC = 0.689). When items 2, 5, and 11 were removed, the correlation for the nine-item CFS-J substantially improved (Spearman’s = 0.692, ICC = 0.709).
The Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy  for the 12 item scale was 0.880, indicating meritorious sampling adequacy , and Bartlett’s test of sphericity was significant (p < 0.001). The KMO increased to 0.882 in the nine-item CFS-J after items 2, 5, and 11 were eliminated. The eigenvalues of the first five factors in the 12-item CFS-J were 4.83, 1.09, 0.99, 0.88, and 0.79, in descending order, accounting for 40.3, 9.1, 8.3, 7.4, and 6.6 % of the variance, respectively. Kaiser’s criterion with eigenvalues over 1.0 supported a two-factor solution. The MAP values for the first three principal components of the CFS-J item correlation matrix were 0.018, 0.029, and 0.044, respectively, thus indicating a one-factor solution. The first three eigenvalues of simulated components by the parallel analysis were 1.32, 1.24, and 1.17, thus indicating a one-factor solution. In the nine-item CFS-J, the eigenvalues of the first three factors were 4.41, 0.84, and 0.76, accounting for 49.0, 9.3, and 8.2 % of the variance, respectively, thus Kaiser’s criterion with eigenvalues over 1.0 supported a one-factor solution, and the MAP and the parallel analysis also supported a one-factor solution.
EFA was used to examine the factor structure because a two-factor solution was possible, however, exhaustive analyses, including popular oblique rotations other than the promax method, could not adequately resolve the two-factor model. Thus, the factor loadings for the one-factor model only are shown in Table 2. It can be seen from the table that items 2, 5, and 11 had relatively smaller loadings on the factor, whereas all others had an acceptable level of 0.4 or more. The results of the one-factor model for the nine-item version of the scale, after eliminating these three items, are also shown in Table 2. The factor loadings of all items were 0.4 or more. The two right-hand columns show the Pearson correlation coefficients between each item and the rest of the total CFS-J score. All items indicated significant results based on the null hypothesis test for correlation coefficients (p < 0.001).
Table 3 shows the Pearson correlation coefficients between the CFS-J (12- and nine-item versions) and the CFI-J, CCS, and ATQ-R-JS scales, as well as the results of the null hypothesis test. The CFS-J had a significantly positive correlation with the CFI-J and its alternative and control subscales, with the CCS, and with the positive subscale of the ATQ-R-JS. Further, it had a significant negative correlation with the negative subscale of the ATQ-R-JS.
Reliability and validity
Cronbach’s alpha and McDonald’s omega coefficients in the total sample of the 12- and nine-item versions of the CFS-J exceeded 0.8, which indicates that the scale has good internal consistency. The test–retest reliability, as measured by Spearman’s coefficient of correlation and ICC, which were moderate for the 12-item and good for the nine-item scale versions, was also satisfactory.
EFA provided evidence that the 12- and nine-item versions of the CFS-J have a one-factor structure, although Kaiser’s criterion with eigenvalues over 1.0 suggested the possibility of a two-factor solution. Even with a one-factor solution, the factor accounted for 35.5 % of the variance for the 12-item and 42.9 % for the nine-item versions of the scale, and all factor loadings were appropriate at 0.4 or more, except items 2 and 5 of the 12-item version. Therefore, a one-factor solution adequately explained the factor structure of the CFS-J.
The concurrent validity of the CFS-J was confirmed by correlations between scores on the CFS-J and those on other related measures. CFS-J scores were significantly and positively related with those on the CFI-J, indicating that a person with a higher CFS-J score is likely to also have a higher CFI-J score. Further, cognitive flexibility, as assessed by the CFS-J, was significantly and positively associated with cognitive control skills, as assessed by the CCS. Correlations with the positive and negative subscales of the ATQ-R-JS indicated that greater cognitive flexibility, as measured by the CFS-J, was significantly associated with increased positive and decreased negative automatic thoughts, respectively. These results are consistent with those of Johnco , who stated that “these processes seem important for the successful implementation of cognitive restructuring, where the individual is required to identify a negative automatic thought, generate evidence that contradicts that thought, and subsequently generate a more adaptive or helpful way of interpreting the situation.” In this way, we established the concurrent validity of the CFS-J.
Appropriate version of the CFS-J for use in future studies
The internal consistency (Cronbach’s alpha and McDonald’s omega) results suggested that the nine-item version of the CFS-J was superior to the 12-item version; however, the internal consistency (alpha = 0.847, omega = 0.871) and test–retest reliability (0.687) of the 12-item version were also more than acceptable, which suggests that both versions are adequate for use in future psychological assessment. Most factor loadings were more than adequate at 0.4. The correlation coefficients between the 12-item version and the other assessed scales were generally higher than those for the nine-item version, and the correlation between scores on the 12- and nine-item versions was significantly high. Moreover, there is no substantial difference in burden of testing time between the two versions, and the result of 12-item version is internationally comparable with other language versions. On the basis of these considerations, we recommend using the 12-item version of the CFS-J for conducting research and assessments in future studies.
Limitations and future research directions
The unique contributions of this paper are that we translated the CFS, which is in common use in the English language as a tool to assess cognitive flexibility, into Japanese and then confirmed its reliability and validity. However, the results of this study have some potential limitations. First, generalization of the results should be done with caution because there were more women than men participants, they were recruited from one specific area in Japan and aged 18 years or older, and most were aged in their 20 s. In future studies, it will be necessary to examine whether the developed measure is suitable for use with samples of different demographic types. Second, the participants comprised a nonclinical sample. Future studies should assess the reliability and validity of the CFS-J using clinical samples, including patients with depression, an anxiety disorder, posttraumatic stress disorder, or an eating disorder, given that the English version of CFS has been applied and demonstrated to have potential effectiveness in assessing the outcomes of treatment for this population.
We developed the Japanese version of the CFS and confirmed its internal consistency, test–retest reliability, and concurrent validity using nonclinical samples. The translation processes followed the standard practice of translation and back-translation, including approval of correspondence of the translated items by the original scale’s author. Results indicated that the CFS-J has good internal consistency and acceptable test–retest reliability over the space of –2 weeks, and that it has a one-factor structure with adequate factor loadings. The CFS-J also showed significant correlations with other concurrent measures, including CFI-J, CSS, and ATQ-R-JS. Thus, the CFS-J has good reliability and validity as an instrument for assessing cognitive flexibility in nonclinical samples.
automatic thought questionnaire-revised
automatic thought questionnaire-revised-short version in Japanese
cognitive control scale
cognitive flexibility inventory-Japanese
cognitive flexibility scale
cognitive flexibility scale-Japanese
exploratory factor analysis
intra-class correlation coefficients
Kaiser-Meyer-Olkin measure of sampling adequacy
minimum average partials
Johnco C. The influence of cognitive flexibility on treatment outcome and cognitive restructuring skill acquisition during cognitive behavioral treatment for anxiety and depression in older adults: results of a pilot study. Behav Res Ther. 2014;57:55–64.
Scott W. Cognitive complexity and cognitive flexibility. Sociometry. 1962;25(4):405–14.
Spiro R, Jehng J. Cognitive flexibility and hypertext: theory and technology for the nonlinear and multidimensional traversal of complex subject matter. In: Nix D, Spiro R, editors. Cognition, education and multimedia: Exploring ideas in high technology. Hillsdale: Lawrence Erlbaum Associates Inc; 1990. p. 163–205.
Berg EA. A simple objective test for measuring flexibility in thinking. J Gen Psychol. 1948;39:15–22.
Golden CJ. Stroop color and word test: a manual for clinical and experimental uses. Wood Dale: Stoelting Company; 1978.
Martin M, Rubin R. A new measure of cognitive flexibility. Psychol Rep. 1995;76:623–6.
Dennis J, Vander Wal J. The cognitive flexibility inventory: instrument development and estimates of reliability and validity. Cognitive Ther Res. 2011;34:241–53.
Lee JK, Orsillo SM. Investigating cognitive flexibility as a potential mechanism of mindfulness in generalized anxiety disorder. J Behav Ther Exp Psychiatry. 2014;45(1):208–16.
Gunduz B. The contributions of attachment styles, irrational beliefs and psychological symptoms to the prediction of cognitive flexibility. Educ Sci Theory Prac. 2013;13(4):2079–85.
Lounes N, Khan G, Tchanturia K. Assessment of cognitive flexibility in anorexia nervosa–self-report or experimental measure? a brief report. J Int Neuropsychol Soc. 2011;17:925–8.
Palm KM, Follette VM. The roles of cognitive flexibility and experiential avoidance in explaining psychological distress in survivors of interpersonal victimization. J Psychopathol Behav Assessment. 2011;33:79–86.
Freeman D, Thompson C, Vorontsova G, Dunn L-A, Carter A, Garety P, Kuipers E, Slater M, Glucksman E, Antley A, Ehlers A. Paranoia and post-traumatic stress disorder in the months after a physical assault: a longitudinal study examining shared and differential predictors. Psychol Med. 2013;43:2673–84.
Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46:1417–32.
Rouquette A, Falissard B. Sample size requirements for the internal validation of psychiatric scales. Int J Methods Psychiatr Res. 2011;20(4):235–49.
Kline R. Principles and practice of structural equation modeling. 3rd ed. New york: The Guilford press; 2011.
Martin M, Anderson C. The cognitive flexibility scale: three validity studies. Commun Rep. 1998;11:1–9.
Çelikkaleli Ö. The validity and reliability of the cognitive flexibility scale. Edu Sci. 2014;39(176):339–46.
Tokuyoshi Y, Iwsaki S. Development and validation of cognitive flexibility inventory—Japanese (in Japanese). Proceedings of the 76th annual convention of the Japanese psychological association; 2012, p 672.
UC Davis. Cognitive Control. Translational cognitive and affective neuroscience Cater Lab, University of California, Davis. 2015. http://carterlab.ucdavis.edu/research/control.php.
Sugiura T, Umaoka K. Cognitive control and depression in female university students. Jpn J Health Psychol. 2003;16:31–42.
Freeman A, Pretzer J, Fleming B, Simon KM. Clinical applications of cognitive therapy. New York: Plenum Press; 1990.
Kendalle PC, Howard BL, Hays RC. Self-referent speech and psychopathology: the balance of positive and negative thinking. Cognitive Ther Res. 1989;13:383–95.
Hollon S, Kendall P. Cognitive self-statements in depression: development of an automatic thoughts questionnaire. Cognitive Ther Res. 1980;4:383–95.
Sakamoto S, Tanaka E, Tanno Y, Ono Y. Testing Beck’s model of depression: using the DAS and the ATQ (in Japanese). Psychol Res, Nihon Univ. 2004;25:14–23.
Guttman L. Some necessary conditions for common-factor analysis. Psychometrika. 1954;19(2):149–61.
Horn J. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30(2):179–85.
Velicer W. Determining the number of components from the matrix of partial correlations. Psychometrika. 1976;41(3):321–7.
Stevens J. Applied multivariate statistics for the social sciences. 2nd ed. Abingdon: Lawrence Erlbaus Associates; 1992.
Kline P. The handbook of psychological testing. 2nd ed. London: Routledge; 2000.
Kaiser HF. A second generation little jiffy. Psychometrika. 1970;35(4):401–15.
Hair JF, Anderson RE, Tatham RL, Black WC. Multivariate data analysis. 5th ed. Upper Saddle River: Prentice-Hall; 1998.
KO, designed the current project, translation, data collection, data analyze and drafted the manuscript. SN, helped data collection and translation of original questionnaires. ES, supervised whole processes of the project. All authors read and approved the final manuscript.
The Authors would like to thank Dr. Naoto Oshiro, Financial Technology Research Institute, Japan for support to conduct data processing, and anonymous reviewers whose constructive comments improved the quality of this article.
Availability of data and materials
Data are available upon request from the first author.
The authors declare that they have no competing interests.