Development and validation of a short version of the Partnership Self-Assessment Tool (PSAT) among professionals in Dutch disease-management partnerships
© Cramm et al; licensee BioMed Central Ltd. 2011
Received: 18 March 2011
Accepted: 30 June 2011
Published: 30 June 2011
The extent to which partnership synergy is created within quality improvement programmes in the Netherlands is unknown. In this article, we describe the psychometric testing of the Partnership Self-Assessment Tool (PSAT) among professionals in twenty-two disease-management partnerships participating in quality improvement projects focused on chronic care in the Netherlands. Our objectives are to validate the PSAT in the Netherlands and to reduce the number of items of the original PSAT while maintaining validity and reliability.
The Dutch version of the PSAT was tested in twenty-two disease-management partnerships with 218 professionals. We tested the instrument by means of structural equation modelling, and examined its validity and reliability.
After eliminating 14 items, the confirmatory factor analyses revealed good indices of fit with the resulting 15-item PSAT-Short version (PSAT-S). Internal consistency as represented by Cronbach's alpha ranged from acceptable (0.75) for the 'efficiency' subscale to excellent for the 'leadership' subscale (0.87). Convergent validity was provided with high correlations of the partnership dimensions and partnership synergy (ranged from 0.512 to 0.609) and high correlations with chronic illness care (ranged from 0.447 to 0.329).
The psychometric properties and convergent validity of the PSAT-S were satisfactory rendering it a valid and reliable instrument for assessing partnership synergy and its dimensions of partnership functioning.
Keywordschronic care measurement quality chronic illness health care partnership synergy isease management
The prevalence of individuals with chronic illness is growing fast because of the rapid aging of the population and the greater longevity of individuals. The multiple and often complex needs of populations affected by the epidemic of chronic illnesses requires approaches that include collaboration among health care professionals of various organizations and extend beyond traditional acute episodic health care and the services of any single organization [1–5]. Partnerships are increasingly used to enhance health service delivery in response to this explosion in chronic disease prevalence. While interprofessional health partnerships are internationally acknowledged as integral for comprehensive chronic illness care, the evidence for effectiveness of such partnerships is lacking [6, 7]. Partnership collaboration requires relationships, procedures, and structures that are quite different from the ways many people and organizations have worked in the past and research indicates that these partnerships are also generating a good deal of frustration . In addition, building effective partnerships is time consuming, resource intensive, and often very difficult [9–12]. A number of special challenges are involved in the management of interprofessional health partnerships [12, 13]. One of the main critical tasks of partnership management is to enhance the capacity of partnerships to achieve high levels of synergy. Partnership's ability to create synergy resulting in the redesign of patient care is an essential factor for successful disease-management [7, 8]. Synergy is the degree to which the partnership combines the complementary strengths, perspectives, values and resources of all partners in the search for better solutions [14: p. 5] and is generally regarded as the product of a partnership . The synergy that a partnership can achieve is more than simply an exchange of resources among its partners. Theoretically, when partners effectively merge their perspectives, knowledge, and skills to create synergy, they create something new and valuable: a whole that is greater than the sum of its parts. Lasker and colleagues  developed a framework that supports the people responsible for managing partnerships in realizing high levels of synergy. The Partnership Self-Assessment Tool (PSAT) was developed based on this framework by public health specialists for practical use by groups working to promote health and well-being in their communities . It measures partnership synergy and other related dimensions of the partnership process . Establishment of construct validity during development of the PSAT items was rigorous [7, 9]. In addition, it included data from qualitative interviews with members of community health promotion partnerships, an extensive review of relevant literature and measures, as well as input from a panel of experts. Furthermore, the PSAT was tested in 63 health-related partnerships in operation at least 18 months in urban, suburban or rural areas in the US . A recent study conducted by Butt and colleagues  showed that the PSAT is a valid partnership process measurement tool.
The PSAT has not been validated in the Netherlands to date. In this article, we describe the psychometric testing of the PSAT among professionals in twenty-two disease-management partnerships participating in quality improvement projects focused on chronic care in the Netherlands. Our objectives are to validate the PSAT in the Netherlands and to reduce the number of items of the original PSAT while maintaining validity and reliability.
The study is in the context of a national programme on "disease management of chronic diseases" carried out by ZonMw (Netherlands Organisation for Health Research and Development) and commissioned by the Dutch Ministry of Health. The study was approved by the ethics committee of the Erasmus University Medical Centre of Rotterdam (September 2009).
This study is focused on the evaluation of the implementation of twenty-two disease-management partnerships to enhance knowledge on disease-management experiments in chronic disease care, and stimulate implementation of knowledge and insights of successful programs. These disease-management partnerships are located in various Dutch regions and consist of a variety of collaborations between organizations and/or professionals, e.g. collaborations between general practices and hospitals, primary care collaborations (including physiotherapists and dieticians) or primary and community settings. A questionnaire was sent to all 393 professionals participating within the 22 disease-management partnership (consisting of 153 organizations). Either a package of questionnaires was sent to the contact person of each participating organization (which were distributed to potential respondents through their mail boxes or delivered personally at team meetings) or questionnaires were sent directly to the potential respondents. Two weeks later the same procedure was used to send a reminder to non-respondents. A total of 218 respondents filled in the questionnaire (55% response rate; range 35-100%).
The disease-management partnerships are targeted at different patient populations; including cardiovascular diseases (9 disease-management partnerships), chronic obstructive pulmonary disease (COPD) (5), diabetes (3), heart failure (1), stroke (1), depression (1), psychotic diseases (1) and eating disorders (1). The intervention concerns the implementation of disease-management partnerships aiming to improve quality of chronic care based on the Chronic Care Models (CCM). Each disease-management partnership consists of a combination of patient-related, professional-directed and organizational interventions. The exact programme components for each region may vary. The core of a DMP is described below; for detailed programme information, see our study protocol .
All 22 disease-management partnerships implemented self-care interventions e.g. patient education on lifestyle, regulatory skills, and proactive coping.
Care standards, guidelines, and protocols are essential parts of the 22 disease-management partnerships. They are integrated through timely reminders, feedback, and other methods that increase their visibility at the time that clinical decisions are made. The disease-management partnerships are built on these (multidisciplinary) guidelines. The implementation strategies for professional interventions may, however, vary. All disease-management partnerships provide training for their professionals and implementation of these guideline is supported in almost all disease-management partnerships by ICT tools such as integrated information systems.
Many forms of organisational changes are applied in the 22 disease-management partnerships e.g. new collaborations of care providers, allocating tasks differently, transferring information and scheduling appointments more effectively, case management, using new types of health professionals, redefining professionals' roles and redistributing their tasks, planned interaction between professionals, and regular follow-up meetings by the care team.
The PSAT was used to measure partnership synergy and dimensions of partnership functioning [7, 8]. This instrument contains partnership synergy (nine items e.g. by working together, how well are these partners able to identify new and creative ways to solve problems) and four dimensions of partnership functioning: leadership (11 items e.g. fostering respect, trust, inclusiveness, and openness in the partnership), efficiency (three items e.g. how well your partnership uses the partners' financial resources), administration and management (nine items e.g. evaluating the progress and impact of the partnership), and resources (six items e.g. data and information: statistical data, information about community perceptions, values, resources, and politics). Responses to all items are structured by a five-point Likert scale.
Partnership synergy and dimensions of partnership functioning scores were derived by calculating the mean of responses within each section. The partnership synergy score reflects the extent to which the participants in the disease-management partnership are accomplishing more together than they can on their own .
Since the disease-management partnerships aim to improve quality of chronic care based on the CCM, we tested convergent validity with the Assessment of Chronic Illness Care (ACIC). The ACIC consists of 34 items covering the seven areas of the CCM: health care organization (6 items); community linkages (3); self-management support (4); delivery system design (6); decision support (4); clinical information systems (5). The ACIC also covers integrating the six components, such as linking patients' self-management goals to information systems (6 items) . Subscale scores for the seven sections of the ACIC were derived by calculating the mean of responses within each section, and overall scores were derived by calculating the mean of all subscale scores. We used overall scores to assess the effectiveness of disease-management partnership in delivering chronic-illness care. The highest possible score (11) for any individual item, subscale, or overall score was taken to indicate optimal chronic-illness care delivery, and the lowest score (0) was considered to correspond with limited chronic illness care delivery.
The sample characteristics were analysed using descriptive statistics.
We data-screened the items by examining number of missing, number of 'don't know' answers, mean and standard deviation of each item.
To verify the factor structure of the questionnaire and to test whether the relationship between observed variables and their underlying latent constructs exists, confirmatory factor analysis was executed using the LISREL program . In line with the theoretical assumptions of the measurement model, no correlation errors either within or across sets of items were allowed in the model.
Item reduction analysis was performed to develop a short version of the questionnaire that can be used in case the original version is considered to be too long. Items were removed from the original pool following several criteria: 1) Items were excluded one by one, first by eliminating items with factor loadings below .40 and second by following the modification indices, 2) eliminating items was stopped when reliability (Cronbach's alpha) of each subscale drops below .70 and 3) there should be as few items as possible with a minimum of three, without loss of content and psychometric quality. Listwise deletion of cases with missing data on these 29 items resulted in N = 68. Imputation was done by replacing missing values by the mean of each disease-management partnership as scored by the other professionals of the same disease-management partnership, resulting in N = 218 which was the total sample. To test the measurement models four indices of model fit were used. The cut-off criteria for these four indices were those proposed by Hu and Bentler . First, the overall test of goodness-of-fit assesses the discrepancy between the model implied and the sample covariance matrix by means of a normal-theory weighted least squares test. A plausible model has low, preferably non-significant χ2 values. However, Chi-square is overly sensitive when the sample size is large (anything over 200) , leading to difficulty in obtaining the desired non-significant level . Secondly, the Root Means Square Error of Approximation (RMSEA) reflects the estimation error divided by the degrees of freedom as a penalty function. Values on RMSEA below cut-off value 0.06 indicate small differences between the estimated and observed model. Thirdly, we used the Standardized Root Means square Residual (SRMR), which is a scale invariant index for global fit that ranges between 0 and 1. Values on SRMR lower than cut-off value 0.08 indicate a good fit. As a fourth index of model fit the Incremental Fit Index (IFI) was calculated. This index compares the independence model (i.e. observed variables are unrelated) to the estimated model. Preferably, values on IFI should be larger than cut-off value 0.95.
Internal consistency of the subscales was assessed by calculating Cronbach's alphas, inter-item correlations within each subscale and correlations between subscales.
We then investigated convergent validity of the instrument. Validity reflects the degree to which a scale measures what it is intended to measure. We evaluated convergent validity by analyzing the associations between the dimensions of partnership (original instrument and short version) with partnership synergy and the ACIC.
Sample characteristics professionals (n = 218)
- more than 3 years
Working hours per week
- more than 29 hours
- General Practitioner
- practice nurses
- policy and management
- para-/perimedical professionals
- medical/social specialists
Item characteristics and factor loadings of the first full model
Taking responsibility for the partnership
Inspiring or motivating people involved in the partnership
Empowering people involved in the partnership
Communicating the vision of the partnership
Working to develop a common language within the partnership
Fostering respect, trust, inclusiveness, and openness in the partnership
Creating an environment where differences of opinion can be voiced
Resolving conflict among partners
Combining the perspectives, resources, and skills of partners
Helping the partnership be creative and look at things differently
Recruiting diverse people and organizations into the partnership
How well your partnership uses the partners' financial resources.
How well your partnership uses the partners' in-kind resources
How well your partnership uses the partners' time.
Administration and Management
Coordinating communication among partners
Coordinating communication with people and organizations outside the partnership
Organizing partnership activities, including meetings and projects
Applying for and managing grants and funds
Preparing materials that inform partners and help them make timely decisions
Performing secretarial duties
Providing orientation to new partners as they join the partnership
Evaluating the progress and impact of the partnership
Minimizing the barriers to participation in the partnership's meetings and activities
Non financial resources
Skills and expertise
Data and information
Connections to target populations
Connections to political decision-makers, government agencies, other organizations/groups
Legitimacy and credibility
Influence and ability to bring people together for meetings and activities
Confirmatory Factor analysis with 29 items
Model fit of the full and short models
On imputed data (n = 218)
Model 1: 29 items on imputed data
Model 2: final short version 15 items
Listwise deletion 29 items (n = 68)
Model 1: 29 items
Model 2: final short version
Listwise deletion 15 items (n = 111)
Model 2: final short version
Item reduction analysis
Following the factor loadings, modification indices and checking the internal consistency of each subscale, the stepwise procedure resulted in elimination of items in the following order: 27, 16, 18, 4, 20, 5, 8, 19, 21, 7, 6, 9, 10 en 28. The final short version consisted of 15 items with three or four items for each subscale. Item reduction could be done without loss of model fit. The overall fit of this final model was comparable with the model fit of the full version (Table 3 model 2). Due to a decrease in the number of estimated parameters, the Normal Theory Weighted Least Square χ2 significantly decreased to 432.634, RMSEA was around cut-off and still acceptable. The value of IFI remained 0.99 indicating that the specified relations between variables are well supported by the data. The SRMR index decreased to 0.0638, which is still below the cut-off point of 0.08 and indicates that the global fit of the overall model is good. The final short model on imputed data resulted in comparable factor loadings.
Rerun of the full model and item reduction analysis on the non-imputed dataset with N = 68 resulted in good fit indices and exactly the same order of items that were eliminated.
Internal consistency and inter-correlations
Scale characteristics and inter-correlations of the shortened subscales (n = 218)
Items short version
original full scale
scale mean (sd)
inter-item correlations range
1, 2, 3, 11
12, 13, 14
3. Administration and Management
15, 17, 22, 23
4. Non financial resources
24, 25, 26, 29
Correlation analyses of the partnership dimensions (original instrument and short version) with partnership synergy and assessment of chronic illness care (n = 218)
Assessment of Chronic Illness Care (ACIC)
Assessment of Chronic Illness Care (ACIC)
This study aimed to validate the PSAT in various disease-management partnerships. Our results showed that the PSAT is a valid and reliable instrument to measure partnership synergy and its dimensions. The advantages achieved by partnerships with high levels of synergy are likely to enhance partnership effectiveness in prevention and health promotion that have been identified by other investigators . The PSAT scores assess partnership's strengths and weaknesses in areas that are known to be related to synergy: (1) effectiveness of the partnership's leadership; (2) efficiency of the partnership; (3) effectiveness of the partnership's administration and management; and (4) sufficiency of the partnership's resources. This information can help partnerships identify in which areas they are doing well and in which areas the partnership needs to improve the collaborative process [7, 8].
We investigated individual assessment of each professional participating in the disease-management partnerships. Analyses could also be performed at the partnership level taking into account the hierarchical structure of the data for individuals nested within partnerships. As there is the potential for considerable variation within partnerships and since the main purpose of our study was to compare the psychometric properties of the PSAT in disease-management partnerships, we performed confirmatory factor analyses on the individual level. Ignoring the hierarchical structure of the data may lead to a worse fit of the model. The factor loadings found with the two methods (individual versus team level) will however be similar in value [25, 26].
The cumbersome length of the PSAT led us to perform an item reduction analysis and develop a short version (PSAT-S), which we later showed was reliable and valid. The results of the confirmatory factor analyses revealed good indices of fit with the PSAT-S. As indicated by the high reliability coefficient, the scale showed good internal consistency. In case the original PSAT is considered too lengthy, the PSAT-S is thus a good alternative. We found support for convergent validity of the original PSAT and PSAT-S through high correlations between the partnership dimensions, partnership synergy and chronic illness care.
Several psychometric properties could not be evaluated in this study and thus remain undefined, such as its predictive value. The instrument's sensitivity to change requires further investigation. As does its sensitivity to identify changes in the domains of partnership functioning and synergy for different partnerships. We also recommend testing the English version of the PSAT-S in other countries to ensure international validity. In addition to further research on the predictive value of the PSAT and PSAT-S, the presence of a control group (or control sites) would have also strengthened our conclusions. While it is possible that completing the PSAT or PSAT-S could act as an intervention based on the incidental education awarded by the survey itself, we do not think it likely given the difficulty in producing partnership synergy. Response rates varied widely (35% to 100%). We do not know how the results might have varied with a better response. Finally, we did not include patients as partners. Future research is necessary to investigate patients' perspectives of partnership functioning and use of the PSAT or PSAT-S among patients as well as professionals.
With these shortcomings in mind, we conclude that the psychometric properties of the PSAT and the PSAT-S are good and the PSAT-S is a promising alternate instrument to evaluate partnership synergy and its dimensions.
Partnership Self-Assessment Tool
Partnership Self-Assessment Tool Short version
Netherlands Organisation for Health Research and Development
chronic obstructive pulmonary disease
Chronic Care Model
Assessment of Chronic Illness Care
Root Means Square Error of Approximation
Root Means square Residual
Incremental Fit Index
The research was supported by a grant provided by the Netherlands Organisation for Health Research and Development (ZonMw, project number 300030201). The views expressed in the paper are those of the authors.
- Conrad DA: Coordinating patient care services in regional health systems: the challenge of clinical integration. Hosp Health Serv Adm. 1993, 38 (4): 491-508.PubMedGoogle Scholar
- Wagner EH, Austin BT, vonKorff M: Organizing care for patients with chronic illness. Milbank Q. 1996, 74 (4): 511-44. 10.2307/3350391.PubMedView ArticleGoogle Scholar
- Fleury M-J, Mercier C: Integrated local networks as a model for organizing mental health services. Adm Policy Ment Health. 2002, 30 (1): 55-73. 10.1023/A:1021227600823.PubMedView ArticleGoogle Scholar
- Barr VJ, Robinson S, Marin-Link B, Underhill L, Dotts A, Ravensdale D, et al: The expanded chronic care model: an integration of concepts and strategies from population health promotion and the Chronic Care Model. Hosp Q. 2003, 7 (1): 73-82.PubMedGoogle Scholar
- Nasmith L, Cote B, Cox J, Inkell D, Rubenstein H, Jimenez V, et al: The challenge of promoting integration: conceptualization, implementation, and assessment of a pilot care delivery model for patients with type 2 diabetes. Fam Med. 2004, 36 (1): 40-5.PubMedGoogle Scholar
- Polivka BJ, Dresbach SH, Heimlich JE, Elliott M: Interagency relationships among rural early intervention collaboratives. Public Health Nurs. 2001, 18 (5): 340-9. 10.1046/j.1525-1446.2001.00340.x.PubMedView ArticleGoogle Scholar
- Weiss E, Anderson R, Lasker R: Making the most of collaboration: exploring the relationship between partnership synergy and partnership functioning. Health Educ Behav. 2002, 29 (6): 683-98. 10.1177/109019802237938.PubMedView ArticleGoogle Scholar
- Lasker R, Weiss ES, Miller R: Partnership Synergy: A practical framework for studying and strengthening the collaborative advantage. Milbank Q. 2001, 79 (2): 179-205. 10.1111/1468-0009.00203.PubMedPubMed CentralView ArticleGoogle Scholar
- Cheadle A, Beery W, Wagner E, et al: Conference report: community-based health promotion-state of the art and recommendations for the future. Am J Prev Med. 1997, 13: 240-3.PubMedGoogle Scholar
- Wandersman A, Goodman RM, Butterfoss FD: Understanding coalitions and how they operate. Community organizing and community building for health. Edited by: Minkler M. 1997, New Brunswick, N.J.: Rutgers University Press, 261-77.Google Scholar
- Kreuter MW, Lezin NA, Young LA: Evaluating community-based collaborative mechanisms: implications for practitioners. Health Promot Pract. 2000, 1: 49-63. 10.1177/152483990000100109.View ArticleGoogle Scholar
- Mitchell SM, Shortell SM: The governance and management of effective community health partnerships: a typology for research, policy and practice. Milbank Q. 2000, 78 (2): 241-89. 10.1111/1468-0009.00170.PubMedPubMed CentralView ArticleGoogle Scholar
- Zuckerman HS, Kaluzny AD, Ricketts TC: Alliances in health care: what we know, what we think we know, and what we should know. Health Care Manage Rev. 1995, 20: 54-64.PubMedView ArticleGoogle Scholar
- Gray B: Collaborating Finding Common Ground for Multiparty Problems. 1989, Jossey-Bass, San FranciscoGoogle Scholar
- Lasker RD, Weiss ES: Creating partnership synergy: the critical role of community stakeholders. J Health Hum Serv Adm. 2003, 26: 119-139.PubMedGoogle Scholar
- D'Amour D, Ferrada-Videla M, San Martin Rodriguez L, Beaulieu M-D: The conceptual basis for interprofessional collaboration: core concepts and theoretical frameworks. J Interprof Care. 2005, 19: 116-31. 10.1080/13561820500082529.PubMedView ArticleGoogle Scholar
- Butt G, Markle-Reid M, Browne G: Interprofessional partnerships in chronic illness care: a conceptual model for measuring partnership effectiveness. Int J Integr Care. 2008, 8 (15): 1-14.Google Scholar
- Lemmens KM, Rutten-Van Mölken MP, Cramm JM, Huijsman R, Bal RA, Nieboer AP: Evaluation of a large scale implementation of disease management programmes in various Dutch regions: a study protocol. BMC Health Serv Res. 2011, 11 (1): 6.-10.1186/1472-6963-11-6.PubMedPubMed CentralView ArticleGoogle Scholar
- Bonomi AE, Wagner EH, Glasgow RE, VonKorff M: Assessment of chronic illness care (ACIC): a practical tool to measure quality improvement. Health Serv Res. 2002, 37 (3): 791-820. 10.1111/1475-6773.00049.PubMedPubMed CentralView ArticleGoogle Scholar
- Jöreskog K, Sörbom D: User's reference guide. 1996, Chicago: Scientific Software InternationalGoogle Scholar
- Hu L, Bentler PM: Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling. 1999, 6: 1-55.View ArticleGoogle Scholar
- Hayduk LA: Structural equation modeling with LISREL: essentials and advances. 1987, Baltimore: Johns Hopkins University PressGoogle Scholar
- Bagozzi RP, Yi Y, Phillips LW: Assessing construct validity in organizational research. Adm Sci Q. 1991, 36: 421-458. 10.2307/2393203.View ArticleGoogle Scholar
- Jones J, Barry M: Exploring the relationship between synergy and partnership functioning factors in health promotion partnerships. Health Promot Int. 2011Google Scholar
- Muthe'n BO: Multilevel covariance structure analysis. Sociol Methods Res. 1994, 376-98. 22
- Dyer NG, Hanges PJ, Hall RJ: Applying multilevel confirmatory factor analysis techniques to the study of leadership. Leadership Q. 2005, 16: 149-67. 10.1016/j.leaqua.2004.09.009.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.