Final Report Summary - COFI (Comparing policy framework, structure, effectiveness and cost-effectiveness of functional and integrated systems of mental health care)
Executive Summary:
EXECUTIVE SUMMARY
Background
Mental disorders are a leading cause of disability in Europe and determine high health-care costs and loss of productivity. Effective service organisation is required to maximise the effects of care and reduce this burden. A highly debated question in mental health care is whether to privilege personal continuity of care or specialisation of services. In COFI, we addressed this question by comparing two alternative systems of mental health care:
1) Functional systems: different primary clinicians are responsible for the treatment of a patient, depending on whether the patient is in inpatient or outpatient care. In these systems, the focus is on specialisation of care.
2) Integrated systems: the same primary clinician is responsible for both the inpatient and outpatient care of a patient. These systems privilege personal continuity of care.
Methods
COFI was carried out in five countries - Belgium, Italy, Germany, Poland and the UK, where the two systems are both available in routine mental health care. This meant that we did not change or allocate patients to a care system but studied these systems in a natural experiment, so that results are immediately transferable to policy and practice. Patients with a diagnosis of psychotic, affective or neurotic / somatoform disorders (F2-F4) admitted to a psychiatric hospital during a 14-month period were included and followed-up for 12 months following admission. Overall, 57 hospitals and 7,302 patients participated. The primary outcome was rehospitalisation over one year. We also assessed other clinical and social outcomes (inpatient bed-days, compulsory rehospitalisations, adverse events, social functioning), quality of care, and experience and costs of care.
Results:
We followed-up 6369 out of the 7302 patients (87.1%). Over 35% had been readmitted to hospital. We did not find any statistically significant differences in re-admissions between functional and integrated systems. We also did not find any differences in inpatient-bed days, compulsory readmission, adverse events and social functioning of patients. Moreover, no difference was found in the individual countries or in subgroups defined by age, gender, socio-economic status, migrant status, and whether or not patients had been previously hospitalised. Costs and quality of care did not appear to differ between the two systems either, despite the high variety in funding and governance arrangements that we described in the different countries. The only difference found between the two systems was in the experience of hospital care (but not the overall experience of care) which was significantly better in integrated systems.
Impact and dissemination:
The equivalence of the two systems in terms of clinical and cost outcomes is a major contribution to scientific knowledge and policymaking decisions in Europe and across the world. Changing mental health care organisation on a system level is expensive. Our findings show that re-organisations of care from integrated to functional systems, or vice-versa, are unlikely to have a significant impact on clinical outcomes justifying the expense. This knowledge has the potential to save substantial amount of resources for European Union Member States, which can instead be used for more influential changes in mental health care provision. We summarised these findings and recommendations in guidelines for policy-makers and commissioners of mental health services and circulated them widely. Dissemination activities included national workshops in each participating country and international workshops with world-renowned experts and policy-makers. Scientific dissemination included presentations at the most attended conferences on mental health care and publications in major peer-reviewed journals.
Project Context and Objectives:
A summary description of project context and objectives (not exceeding 4 pages).
Background
Across Europe, mental disorders are a leading cause of disability that result in high health-care costs and loss of productivity through unemployment and absenteeism from work. Within a given year, estimates suggest that approx. 40% of people with the EU – that is 165 million people, suffer from a mental disorder. In order to reduce the burden of mental disorders on individuals, communities and society as a whole, optimal service organisation is required. Throughout Europe, countries are seeking to improve the organisation of mental health care. This is often achieved through radical reforms with far reaching changes for the national health care systems, the staff who work in them and the patients who receive care. What’s more, such sweeping changes consume a large amount of resources both financial and human. However, the reforms happening within and across European countries have been inconsistent, with policies regarding the optimal organisation of mental health care currently being made in the absence of any scientific evidence.
A central question related to ongoing mental health care reforms relates to the organisation between inpatient (hospital) and outpatient (community) services. In particular, there is a debate as to whether the same responsible clinician – usually the consultant psychiatrist, should see the patient when they are an inpatient and as an outpatient within the community, or whether there should be separate in and outpatient consultant psychiatrists. The two dominant models of care related to this debate have been summarised as follows:
1. Functional: care is provided by separate psychiatrists in distinct services, with the transition between services coordinated through a network of regulated referrals;
2. Integrated: care is provided by the same consultant psychiatrist across different services, the responsible clinician co-ordinates all interventions.
The debate regarding which of the two systems is most effective has led to the on-going and inconsistent reorganisations of mental health care, despite being driven by only poor and partial research evidence.
Previous evidence - Systematic review and findings
An initial review of the existing evidence for studies comparing integrated and functional systems of mental health care identified 17 studies using a range of quantitative and qualitative methods. In eleven of the studies, integrated systems were associated with a shorter length of stay, better recovery after a first episode of psychosis and greater satisfaction – from both the clinician and patient perspective. However, two studies highlighted that functional care was associated with fewer hospital admissions and fewer serious incidents on the ward, whilst the remaining studies provided mixed results, favouring integrated systems for some outcomes, and functional systems for others.
Qualitative interviews with clinicians working in each system and with patients who have received mental health care suggest that each system has certain advantages. The advantages of each system have been outlined in Table 1.
Limitations of the existing research
Although previous studies have assessed the two types of mental health care, these studies had various limitations that made it difficult to draw any firm conclusions. Firstly, many of the previous studies used before/after designs or conducted cross-sectional surveys at a single time point. One of the main limitations with using a before and after design, is that the new system (regardless of whether it was functional or integrated) produced better outcomes. Secondly, studies often failed to control for confounding variables that may influence the result. Thirdly, many of the studies were small. This meant that they lacked the statistical power to test for small differences. Finally, most studies only focused on one or two outcomes, which varied across the review. In order to evaluate properly the two systems of mental health care, comprehensive data was needed on a range of clinical, social, economic and patient-reported outcomes.
This meant that prior to the COFI study, no high-quality research evidence was available from large-scale studies to help inform the debate and guide policies on the best way to organise mental health care services.
COFI was designed to overcome the limitations identified with the previous research to specifically:
a. Provide enough statistical power to properly investigate small differences between integrated and functional care as even small differences in outcomes may be important at a public health level. For example, a 5% difference in the use of hospital services has cost implications at a national level.
b. Overcome the “novelty effect” by studying the two systems in routine operation without the need to implement a new system or change the services being studied.
c. Investigate the benefits of the two systems of mental health care for different subgroups of patients
d. Obtain reliable findings that can be applied across contexts.
Context for the COFI study
The COFI study was conducted in five European countries (Belgium, Germany, Italy, Poland, and United Kingdom). Importantly, within each country, functional and integrated systems of mental health care already co-existed in routine practice. This enabled the COFI study to compare the two mental health systems in day-to-day mental health services, without the need to set up new services or change the organisation of care. We selected countries to include in the COFI study that all have different funding arrangements, traditions of mental health care and mental health services that look and operate differently. The countries included represent a range of mental health services across Europe. Thus, by comparing the results of integrated and functional care within and across these five countries, the results of the study are generalisable to other countries within Europe and beyond.
How are functional and integrated mental health services organised in the countries of the COFI study?
The frameworks which guide mental health policies and practice in different European countries traditionally tend to favour one approach over the other. For example, in Germany, Belgium and Poland mental health services were traditionally functional with separate in and outpatient psychiatrists. In contrast, within the UK and in Italy, mental health services tended to be integrated.
In the United Kingdom, mental health services are provided at a regional level by metal health National Health Service (NHS) Trusts. Over the last decade, Trusts have gradually shifted from an integrated system of mental health care to a functional way of working. The aim of this shift was to improve the quality of inpatient care; however, the shift has been criticised for reducing continuity of care and having a negative impact on the relationship between the psychiatrist and the patient. Likewise, in Italy, which was also traditionally integrated, local differences in how services are funded and arranged has resulted in both systems being available, often within neighbouring areas.
In contrast, within Germany, the traditional model of mental health care was functional. This was characterised by different services providing different activities, with the link between services organised through referrals. However, recent initiatives and projects have aimed to strengthen the coordination between services, making one service responsible for both in- and outpatient care. This has led to an increase in the number of integrated services within the country.
Finally, within both Belgium and Poland a number of substantial reforms of mental health care are currently ongoing. This has resulted in integrated and functional mental health care co-existing within the same region.
The differences and similarities of the mental health reforms within the five COFI countries highlight the importance of providing good quality evidence for policy-makers regarding this key question. The design and timing of COFI therefore provided an ideal opportunity for study.
Overall aim of COFI
The overall aim of COFI was to compare the effectiveness and cost-effectiveness of functional and integrated systems of mental health care across Europe. To overcome problems identified with existing research, COFI utilised countries where both systems of mental health care were in routine operation, and employed a “natural experiment” as the design. This would enable the results to be readily applied to mental health care systems across Europe, as the investigators did not change the care they were studying. To fully meet this aim, the comparison considered clinical and social outcomes, as well as patients’ quality of life, health and social needs, safety and cost-effectiveness of care. Different patient subgroups were defined by diagnosis, age, gender, socio-economic and migrant status to understand whether the different systems of care are more suited to particular patient groups.
Research questions addressed
To achieve the overall aim of the COFI project, the research questions addressed throughout were:
1) Primary research question:
a) Do rates of rehospitalisation differ between the functional and integrated approaches?
2) Clinical and social outcomes
a) Is there a difference in number of voluntary and compulsory rehospitalisations per patient and yearly inpatient bed days between the two approaches?
b) Are untoward incidents (deaths, suicides, physical violence committed or experienced by patients) more frequent in either approach?
c) What are the social outcomes, in terms of employment, accommodation, living situation, subjective quality of life and social contacts of patients treated by either approach?
3) Costs of care:
a) What are the direct costs of care (i.e. related to use of services) within each approach?
b) Which approach is most cost-effective?
4) Quality of care
a) Does adherence to national and international treatment guidelines, and clinical decision making vary between the two systems?
5) Experience of care
a) What are treatment satisfaction and the experience of care provided by the two approaches and do they differ?
6) National policies and legislations
a) What national policies and legislations on the organization of mental health care support the implementation of functional and integrated systems
Project Results:
The COFI project brought together a multi disciplinary consortium of experts based in six institutions across five European countries. Members of the team included psychiatrists, psychologists, social scientists, health economists and experts in policy analysis (Figure 1).
The study was designed and implemented to provide high-quality evidence on a range of clinical, social and patient-reported outcomes for integrated and functional care. The comparisons focused on individuals with a diagnosis of psychotic (F2), affective (F3) neurotic, stress-related or somatoform (F4) disorders, who are the main users of psychiatric care, and provided information on the effectiveness of the two systems for different patient sub-groups. Overall, the aim was to provide outcomes and comparisons what would help clinicians, patients, service managers and policy makers to optimise the organisation of mental health care - particularly the transition between in- and outpatient services.
To achieve the project objectives and to address systematically the research questions, the project was organised into a number of interlinking work packages, with the different consortium members responsible for different work streams:
WP1: Project management (QMUL) – including overall management of the project and consortium.
WP2: Study preparation (QMUL) – Preparation of all the materials required for the natural experiment.
WP3: Assessment of existing national policies and legislations (UCL) – identifying and assessing existing national policies regarding mental health care to explore how these policies are implemented in practice to support continuity of care (Research question 6).
WP4: Data collection and data quality assurance (QMUL) - Central to the COFI project was the natural experimental comparing functional and integrated mental health care. This WP outlined the procedures for data collection and quality assurance (Research questions 1-4).
WP5: Comparison of the effectiveness of functional and integrated systems on clinical and social outcomes (QMUL) - comparing the clinical and social outcomes to test comparatively the effectiveness of the two systems (Research question 1-2).
WP6: Comparative analysis of the effectiveness of the two systems on patient reported outcomes (UNIVR) – providing evidence for the effectiveness of the two systems on patient reported outcomes, including quality of life and satisfaction with services (Research question 5).
WP7: Comparison of the quality of care provided by functional and integrated systems (TUD) –assessing care quality, therapeutic relationships and adherence to guidelines (Research 4).
WP8: Comparison of the cost structure and cost-effectiveness of the functional and integrated systems (KCL) – Costs associated with both systems of care were collated during this WP and cost-effectiveness determined by combing cost data with quality of life data collected in the previous WPs (Research question 3).
WP9: Qualitative study (IPiN) – exploration of the experience of patients and clinicians who receive or deliver each system of care through in-depth qualitative interviews (Research question 5).
WP10: Integration and dissemination of findings (QMUL) – integration of the findings to produce guidelines and recommendations for policy-makers and wide dissemination of the project.
Figures 2 provides a schematic diagram of the overall COFI project design.
The project successfully achieved the aims of each work package, presented here is a summary of the main findings across the different work packages.
Outcome 1) Existing policy to support continuity
WP3 identified and assessed existing policies and legislation documents relating to the organisation of mental health care within the five countries. A literature review of policies was supplemented by interviews with key informants such as service managers, clinicians, and health service researchers. The interviews provided more detail on the actual implementation of each policy within the healthcare context. The review and interviews highlighted the complexity of mental health care within and across countries, and how different organisational characteristics may support the implementation of integrated or functional approaches to care.
Two main organisations of mental health services that typified care within the countries of the COFI study were identified. Within the UK and Italy, mental health services are predominantly public organisations funded from taxes paid by the population; these are termed National Health Systems. In contrast, within the three remaining countries, private not-for-profit and for-profit organisations and practitioners provide care. These private organisations and clinicians receive funds from public authorities and income-related contribution. These were termed Regulated Market (RM) Systems. Despite falling into these two categories, there was significant variation between the organisation of care within the COFI study.
Features of the mental health systems that impact on the implementation of functional and integrated care were then identified. These system features were related to i) financing mechanisms including out-of-pocket payments for patients, and the mode of payment of doctors; ii) provision of health services such as the public-private mix and freedom of patient choice; and iii) policy regulation. Each of these features of the health care system and how they were related to NHS and RM systems was described in detail within the deliverable for WP3 - Care system conditions facilitating or hindering integrated personal continuity of care for SMI patients: A comparison of five European systems.
Briefly, these features of the system differed according to whether a country operated a NHS or RM system. Even within countries that broadly operated using the same system, there was variation in practice. Table 2 highlights how each of the system features operates within each country.
Table 2 – Features of mental health systems across the five countries
Overall, it was found that NHS systems (UK and Italy) favoured integrated approaches. This was due to the organisational mechanisms focusing on long-term community care, with fewer out of pocket payments for patients, more national and local regulation on care pathways and less choice for patients regarding their individual clinicians. In contrast, in countries where RMS were dominant, whether integrated or functional systems were favoured was dependent on the choice of clinicians and individual patients – choices that were strongly determined by financing mechanisms and the different funding available. When the focus of care was on acute episodes, with patients having more out-of-pocket payments and having more freedom to choose services and clinicians, systems tended to favour functional care, although integrated care was also possible.
The findings of WP3 were important in highlighting the context for the study. The matrix summarised above was submitted as a publication to add to the scientific knowledge in this area.
Outcome 2) State-of-the art research methodology
The focus of the project was to overcome the limitations identified with previous research (as highlighted by the systematic review and summarised in the background section) to produce reliable evidence for policy-makers regarding optimal organisation of care. In order to overcome the limitations, the study adopted a state-of-the-art research design conducting a large-scale multi-site natural experiment. A natural experiment means the investigators do not change anything about the system being studied, nor do they allocate patients to different conditions. By studying systems that were already in routine operation within the five countries, we therefore avoided the “novelty effect” described earlier.
The deliverable for WP5 – Report on the assessment of the effectiveness of functional and integrated care in the total sample and in patient subgroups, provides full details of the study methods. Briefly, over a 14 month period (from October 2014 – December 15), the study aimed to recruit all eligible consecutive admissions to general adult inpatient psychiatric wards. Patients were followed up for one-year following the date of their admission (termed the index admission). We aimed to include patients who were representative of the individuals who use psychiatric hospital care. The inclusion criteria balanced this aim whilst ensuring comparable populations across the five countries.
The inclusion criteria for patients were:
i. Aged 18 years of age or older;
ii. A clinical diagnosis of psychotic disorder (F20–29), affective disorder (F30–39) or anxiety/somatisation disorder (F40–49) according to the International Classification of Diseases—ICD—10;
iii. Hospitalised in a general adult psychiatric inpatient unit – defined as at least one overnight stay
iv. Sufficient command of the language of the host country to provide written informed consent and understand the questions in the research interviews; and
v. Capacity to provide informed consent.
Patients were excluded if they had an organic brain disorder or severe cognitive impairment.
Where possible patients were approached within 48 hours of admission. This was to reduce the chance of bias within the sample and to include as many inpatients as possible. A range of variables including clinical, social, cost, patient-rated and safety outcomes were collected to address the different research questions (the variables will be described in detail within the relevant sections of this report).
Allocation to the two approaches differed across the countries. In the UK and Italy, the locality of the patient determined whether they received functional or integrated care. All patients within the same area and hence the same hospital received the same type of care. In contrast, in the three remaining countries (Belgium, Germany and Poland), whether the patient receives functional or integrated care is dependent on a number of patient, clinician, service and organisational factors. Different patients within the same area and within the same hospital receive either functional or integrated care based on clinical decisions, insurance arrangements, local practices and service arrangements or the choice of the patient.
Fifty-seven hospitals were included across the five countries. The sample size calculation indicated 5123 patients would be needed at one-year follow up to achieve our aims. We recruited 7302 eligible patients and were able to follow up 6369 participants (87.2%). Of those followed up, 2313 received personal continuity, 4051 specialisation and five were not allocated to either system.
Given the number of recruited and followed up patients, one of the main achievements is that COFI is one of the largest longitudinal studies of psychiatric inpatients ever conducted within Europe.
The consort diagram for the study is shown in Figure 1 – which was adapted from the main results paper (Giacco, et al. (under review) The same of different psychiatrists for in- and outpatient treatment? One year outcomes in a natural experiment in five countries. British Journal of Psychiatry).
The full method for the study was published in the protocol paper – Giacco et al. (2015) Specialised teams or personal continuity across inpatient and outpatient mental healthcare? Study protocol for a natural experiment. BMJ Open 25;5
Outcome 3) Increased understanding of inpatient services across Europe
One direct result of the project, and specifically the large multi-site natural experiment, is an increased understanding of inpatient psychiatric care including a) the characteristics of people who use inpatient services, b) satisfaction with inpatient services including the initial experience of hospital admission and c) length of stay across the five countries. Such information is crucial for service managers and policy-makers to help optimise inpatient care to not only improve outcomes for all patients, but also improve the experience.
a) Characteristics of individuals using inpatient services:
COFI was unique in that it aimed to include individuals within 48 hours of admission, and included both voluntary and involuntarily admitted patients, with few restrictions on inclusion. The characteristics of the patients included in the COFI study are shown in Table 3.
b) Satisfaction with inpatient treatment
Within the UK, allocation to either functional or integrated care is geographically based. Given the large UK sample size, we were able to assess satisfaction with the initial index admission, to test whether there were any significant differences between the two types of care. The results indicated that individuals in integrated systems were significantly more satisfied with their inpatient care compared to those treated in functional systems. Furthermore, the results were still significant, even when controlling for the hospital, and potential confounding factors such as age, gender, admission status, severity of illness and diagnosis. Full results are reported in Bird, et al. (2018). In-patient treatment in functional and sectorised care: patient satisfaction and length of stay. British Journal of Psychiatry, 212, 81-7.
In addition to the impact of the system of mental health care, we also wanted to assess which patient characteristics may be associated with satisfaction with inpatient care. Data was included from all study sites. Based on the 7302 participants who were included in the COFI baseline, we established that the following factors were associated with lower satisfaction with inpatient care:
- Being younger
- Unemployed
- Living alone
- Not having seen a close friend within the last week
- More severe illness
- Repeat admissions to hospital
- Higher education levels
- Comorbid personality disorder
- Being involuntarily admitted
- Recruited from the UK.
Full results are reported in Bird et al. (under review) Factors associated with satisfaction of inpatient psychiatric care: A cross-country comparison. Australian and New Zealand Journal of Psychiatry.
To explore in-depth the reason behind both positive and negative appraisals, a small qualitative study was conducted with 61 patients admitted to three different hospitals within the UK. The brief interviews (maximum of 15 minutes) explored the experience of the first few days in hospital, and linked the themes to both positive and negative appraisals. One of the main results of the analysis indicated that although there was a lot of overlap, with individuals describing both positive and negative experiences, there were distinct themes linked to positive and negative appraisal of inpatient care (Shown in Table 4). The full findings, including illustrative examples of each theme are available in the full publication, Chevalier et al. (in press) Exploring the initial experience of hospitalisation to an acute psychiatric ward. PLoSOne.
) Length of stay across Europe
The large sample in COFI allowed us to explore what patient characteristics were associated with increased length of stay within the hospital, and to explore whether these were the same or different across the countries. Full details of the analysis of length of stay are available in the published paper
Dimitri, et al. (2018). Predictors of length of stay in psychiatric inpatient units: Does their effect vary across countries? European Psychiatry, 48:6-12.
To summarise, data indicated large variation across countries, with the average length of stay under 18 days for Italy (17.9) compared to nearly 55 days for Belgium (54.9). Across all countries, having psychosis, greater symptom severity, being homeless, living alone, receiving benefits, having a medical comorbidity, increased social isolation, a history of previous admission and being involuntarily admitted all predicted longer stays. However, there were interactions with the country for some of the variables. This suggested that although there are some generalisability across countries, the same patients have different lengths of stay depending on the country. For example, homelessness predicted a longer length of stay in most countries, apart from Germany where it was associated with a shorter length of stay. For most other factors, the direction of the association was the same, but the predictive values were significantly different. Furthermore the differences in length of stay between the countries remained significant, even when the different predictor variables were adjusted for, suggesting that factors aside from patient characteristics determine differences in the length of stay across countries.
Outcome 4) Comparison of the effectiveness of the two systems on clinical and social outcomes
Work packages 4 and 5 concentrated on the main aim of the COFI study – namely to compare the two systems of mental health care with regards to patient clinical and social outcomes. Of the 7302 participants included in the study, 6369 (87.2%) were followed up after one year and provided data on rehospitalisation.
One finding of the study related to the percentage of individuals who are rehospitalised within one year. COFI was able to provide reliable data on rehospitalisation rates for a range of diagnostic groups and within different counties. Across the whole sample 2259 (35.5%) of patients were readmitted to hospital within one year. Within functional systems, 1405 out of 4051 (34.7%) were rehospitalised compared to 854/2313 (37%) in integrated systems. The difference between the two systems of care was not significant, and remained not significant when controlling for the hospital and all other potential confounding variables. This indicated that for the primary outcome of the study there was no difference in effectiveness between integrated and functional care.
The difference in rehospitalisation rates between functional and integrated care was also assessed within difference patient sub-groups. The following sub-groups were included in the analysis; country, diagnostic group, gender, age, socio-economic status, migrant status and type of admission. As with the main sample, the differences between functional and integrated care for each of the sub-groups investigated was not significant. Therefore corroborating and strengthening the main finding.
One interesting outcome and products of the study is the dataset available for rehospitalisation rates for different sub-groups of patients. These are presented in Table 5.
We also assessed other clinical and social outcomes that could be influenced by the system of care. Total length of stay per year (bed days) and the number of involuntary admissions were important clinical outcomes. As with rehospitalisation we found no differences between functional and integrated care for either the main sample or in any of the sub-groups studied. Across the study, data was collected on a number of untoward events, including death (any reason), completed suicide, suicide attempt, side-effects of medication requiring somatic hospitalisation, victim of physical violence or perpetrator of physical violence. The total number of untoward events and a dichotomous variable of whether or not a person had experienced an untoward event were assessed. In both case there were no differences between functional and integrated care for the main sample or any of the sub-groups.
Finally, as a measure of objective social situation, the SIX index was calculated. This includes items on Employment, Housing, Living situation and Friendships. Consistent with the clinical outcomes, no significant differences were found between the two systems of care for either the main or sub-group samples.
Taken together, the results present a consistent narrative indicating that there is no difference in effectiveness between the two systems of mental health care. The full results of the study and in-depth analysis is available in the COFI main paper: Giacco, et al. (under review) The same of different psychiatrists for in- and outpatient treatment? One year outcomes in a natural experiment in five countries. British Journal of Psychiatry), and within the deliverable for WP5: Report on the assessment of the effective of integrated and functional care in the total sample and in patient subgroups.
Outcome 5) Comparison of the effectiveness of the two systems on Patient-Reported Outcomes
Patient-satisfaction is a key patient-reported outcome when optimising mental health services. It has been shown to impact on treatment adherence, and predicts clinical outcomes within both the hospital and the community. One of the main contributions of the COFI study to the scientific knowledge regarding the assessment of patient satisfaction was the submission of a thesis on the topic (Patient's appraisal of mental health care in five European countries: findings from the COFI study). Part of the thesis included a published systematic review detailing existing ways to measure satisfaction – which informed the choice of tools within the COFI study. Full details of the review can be found in the published paper Miglietta et al. (2018) Scales for assessing patient satisfaction with mental health care: A systematic review. Journal of Psychiatric Research, 100: 33-46.
A total of twenty-eight scales were identified in the review, each scale has been used at least twice in the literature. The scales were diverse, targeting different mental health services (inpatient, outpatients, supported accommodation), and varied in the number of items they included (3-45) and on the likert scales used (4-10 points). However there were some commonalities, with the contents of the scale covering “relationship with staff”, “staff competence”, “overall satisfaction”. The four most frequently used scales were reported. This helped inform the decision regarding the best way to measure satisfaction. Within the COFI study, two of the four most frequently used measures, the -Client Assessment of Treatment scale (CAT) and Verona Service satisfaction scale (VSSS) were used to measure satisfaction with inpatient and overall care respectively.
Unlike, inpatient satisfaction (presented above), there was no difference between the two types of care relating to overall satisfaction with services as indicated on the VSSS-32, although some sub-scales indicated a trend towards favouring integrated care.
Alongside patient satisfaction, quality of life and the experience of discrimination were also measured. For quality of life, the MANSA was used, whereas for discrimination, a modified version of the DISC-12 was used to separate actual and anticipated discrimination. Key findings of these analyses indicated that there were no differences between the two systems of care for either outcome. However, there was significant variation between the countries for anticipated and experienced discrimination, with patients in Belgium reporting higher levels of experienced discrimination, and individuals in Poland anticipating more discrimination. The findings also indicated that patients anticipated discrimination in all areas of their lives including finding a job, developing a close relationship and feeling the need to conceal their diagnosis. Finally, contrary to previous campaigns, which often focus on schizophrenia and psychosis, individuals with all conditions were likely to experience and anticipate discrimination. This was particularly the case for individuals with anxiety / somatoform disorders. This suggests that national stigma campaigns should not just focus on those with a diagnosis of psychosis; rather they should address the discrimination experienced and anticipated by all individuals with mental health conditions, including those with anxiety disorders.
Outcome 6) Adherence to guidelines and quality of care
To compare the systems of care with reference to adherence to treatment guidelines, we selected a very specific patient group, namely individuals with bipolar disorder hospitalised for a depressive episode. The review of existing guidelines highlighted the wealth of potentially relevant guidelines that could be followed within services. It also highlighted how different countries follow and adhere to different policies and practices even when considering the same diagnostic group. Full details of the guideline review can be found in the deliverable D7.1 Report on the comparison of adherence to guidelines of the interventions provided by the two systems
Based on the extraction of data from the different guidelines, a new measure of guideline adherence was created. This included the following aspects (which could be adapted and tailored for any other major psychiatric Axis I condition)
− Appropriateness of pharmacological treatment to patients’ diagnosis
− Adherence to standards of dosage and length of the treatment
− Appropriate provision of psychological and psychosocial interventions
− Short interval between discharge from the hospital and ambulatory mental health encounter
− Patient involvement in clinical decision-making
The grade of fulfilling the criterion was rated between 0 and 2 for each item, with a sum score calculated over the five aspects (max 10).
Consistent with the main findings, there were no significant differences in the scores between the two types of care regarding adherence to guidelines. For all items, and for the total sum score, the two types of care were equivalent, as shown in Table 6. What the analysis did however highlight, was the lack of availability for psychological therapies. Unlike all other items, where both systems scored 1.16 and 1.43 (out of a maximum of 2), appropriate provision of psychological therapies scored only 0.55 for functional care and 0.42 for integrated. This highlighted the need for better access to psychological therapies for patients across Europe.
Within the larger sub-sample (2181 patients), two further aspects of the quality of care were comparatively investigated between the two systems, namely therapeutic relationships and involvement in clinical decision-making. Therapeutic relationships were rated using the Helping-Alliance Scale (HAS). On average, a HAS score of 7.41 (SD 2.15) was calculated across all five countries. Consistent with the majority of outcomes reported, no significant difference in therapeutic relationship was found between the two care systems. A paper draft has been prepared: Schon et al. (in preparation) Predictors on therapeutic relationship: A study on patient’s view of health alliance in psychiatric care in five different/European countries.
A new tool was developed to assess clinical decision-making (shown in Box 1).
1. desire for information regarding your pharmacological treatment choices in a given situation.
a. I should be informed about all the different treatment strategies/alternatives.
• Strongly disagree, Slightly disagree, Neither agree nor disagree,
Slightly agree, Strongly agree
2. attitude towards decision-making regarding your pharmacological treatment.
a. I want to make the final decision.
b. I want to make the final decision after seriously considering my clinicians opinion.
c. My clinician and I should share responsibility for making the best decision for me.
d. My clinician should make the final decision, but seriously consider my opinion.
e. My clinician should make the final decision.
3. recall the last decision regarding your pharmacological treatment that was made.
a. The last decision affected
• beginning of pharmacological treatment
• discontinuation of pharmacological treatment
• dosage alterations
• addition of another drug
• switch to another medication
b. The last decision was made
• less than 1 month ago
• 1-2 months ago
• 3-6 months ago
• more than 6 months ago
4. indicate how much you agree with the statement.
a. I have been informed about different treatment strategies/alternatives
b. I had the chance to ask for as much information as I needed about the different treatment choices
c. The clinician took my expectations and concerns into consideration
d. I know the advantages and disadvantages of the different treatment options
• Strongly disagree, Slightly disagree, Neither agree nor disagree,
Slightly agree, Strongly agree
5. indicate which statement is true for you for this decision
• I made the final decision.
• I made the final decision after seriously considering my clinicians opinion.
• My clinician and I shared responsibility for making the best decision for me.
• My clinician made the final decision, but seriously considered my opinion.
• My clinician made the final decision.
Box 1 – Clinical decision-making tool
The measure assesses the concordance between the wishes of the patient and what they actually received. Although the study did not find any differences in decision-making between the two types of care, it did highlight a wider issue with patient involvement within mental health services. Although over 60% of patients strongly asserted their wish to be informed about all the different pharmacological treatment strategies and alternatives, less than 25% (22.8%) felt adequately informed.
Deliverable 7.2 Report on the comparison of patients' satisfaction with therapeutic relationship within the two systems provided full details of the analysis.
Outcome 7) Economic costs and unit costs of integrated and functional care across Europe
One outcome of the project with the potential for lasting impact related to the economic evaluation conducted as part of the project. Through the use of existing literature (e.g. Personal Social Services Research Unit annual compendium and NHS Reference costs or the Agency of Health Technology Assessment and Pricing) and a new data collection tool completed by the hospital and community services, unit costs for different mental health services have been calculated for each of the countries included in the study.
The resulting unit costs are shown in Table 7, which highlights the cost of each service in the local currency. These costs were then transformed into Purchasing Power Parity (PPP) adjusted costs (in Euros) for the purposes of the analysis.
Unit costs were combined with data on patient quality of life (collected as part of the sub-group) to determine the cost effectiveness of each system. Overall, the results indicated that there was no difference between the two types of care. This was in part due to the variation in both unit costs and resource use across the difference countries, and due to differences in quality of life measures. Table 8 summarises the main findings for each country.
The full detailed economic analysis is currently being prepared for submission, with full details included in the deliverables 8.1 Report on the assessment of mental health care costs within the two systems and 8.2 Report on the comparison of the cost effectiveness of the two systems in the total sample and patients' subgroups and presentation of different models of cost analysis.
Outcome 8) Understanding the experience of care from the perspective of patients and clinicians
To understand the experiences of each system of mental health care, both patients and clinicians were included in in-depth interviews. For the patients, the sample included individuals with a range of diagnoses and different experiences of hospital care, whilst the clinicians varied in their professional background and experience within services. Details of the interview procedure, topic guides and details of the thematic analysis conducted are included in the deliverable for WP9 – Report on qualitative analysis of in-depth interviews through thematic analysis.
Thematic analysis was used to analyse the data, with codebooks developed separately for clinician and patients interviews. The interviews generated a wealth of data, with the codebooks containing 266 codes (grouped into ten families) for patients and 245 codes (grouped into nine families) for clinicians. Within the deliverable and papers submitted for publication, full details of the codebook with example quotations are available. Here we have focused on the presentation of the main advantages and disadvantages reported for each system (presented in Table 9). Across the 188 patient interviews and 63 clinician interviews, one of the most striking results was the consistency of themes. Although interviewed separately with slightly modified versions of the topic guide, the themes present in the data were, in most cases apparent for both clinicians and patients, and across the five different countries, despite the differences in care organisation.
Potential Impact:
To maximise the impact of the project, one of the main outputs was the development of the COFI guidelines. The guidelines included a summary of the main findings and presented the recommendations from the study. These recommendations have the potential to impact on both the delivery of mental health services and on research into mental health care. The summary findings and the recommendations are presented below:
Summary of main findings and conclusions:
In summary, the main findings of the study are:
• Across the study, 35-40% of people were readmitted to hospital within a one-year period, although there was some variation within and across countries.
• The system of care e.g. functional or integrated care, had no significant impact on any clinical and social outcomes for patients at one-year following admission to a psychiatric hospital. This result was consistently found for the main sample and for different subgroups, including subgroups by country, diagnosis, age, and whether or not the person had previously been admitted.
• Patients were more satisfied with their inpatient treatment within integrated care systems; however, there was no difference in overall satisfaction (including both in and outpatient care) following one year.
• Patient and clinician experience indicated a mirroring of the advantages and disadvantages of each system, such that the advantages of one system mirrored the disadvantages of the other. The views on the advantages and disadvantages were consistent between patients and clinicians and across the different countries.
• Anticipated and experienced discrimination was apparent for all patients regardless of diagnostic group. Experienced and anticipated discrimination was particularly pronounced for individuals with a primary diagnosis of F2 (psychosis) and F4 (anxiety / somatoform).
• Overall, despite being a hotly contested topic, whether patients are treated by the same or different psychiatrists across in- and out-patient care had no significant impact on any of their outcomes as measured at the one-year follow-up.
Recommendations arising from the study
A. If the aim is to improve long-term outcomes for patients, costly reorganisations of mental health care systems should not focus on changing from integrated to functional care or vice versa.
B. If the aim is to improve patients’ experience of in-patient treatment, integrated care may be seen as preferable.
C. Both approaches have strengths and weaknesses – organisational considerations in the local context and preferences of patients and clinicians may favour one of the two approaches.
D. Patient preferences are particularly important when there is a choice between the two systems in the same service. Potential advantages and disadvantages of each system should be considered when making the choice.
E. Mental health services should consider ways to address the discrimination that psychiatric patients anticipate and experience, including patients with anxiety and somatoform disorders.
F. Research and policy may focus less on the organisation of care at a system level, but instead aim to improve the actual treatment within any given organisational system.
The guidelines and recommendations, and the project more widely will have the following anticipated impact:
Impact for services and policy makers
The recommendations of the COFI project suggest that changing the organisation of care by moving from functional to integrated systems or vice versa will not greatly benefit patients in terms of clinical, social and economic outcomes. Having high quality evidence highlighting the equivalent effectiveness in terms of the main outcomes has the potential to save resources for mental health services across Europe. Mental health care reforms and changes in the organisation of care consumes both human and financial resources. Such changes often occur in the absence of any evidence. The COFI study is the largest study to assess the impact of different service models in routine operations, and the results can be easily used by policy-makers and commissioners considering such a reform.
Recommendations B, C and D are particularly important when planning services, as they point to the advantages and disadvantages of both systems, and to the consideration of patient choice when planning care. At a local and national level, policy makers may wish to consider the features of mental health services identified in WP3, which were linked to either implementing functional or integrated care. The local context including the wishes of patients and clinicians should be taken into account when reorganising services. Recommendations A and F suggest that both policy and research should move away from studying change at the system level, but should instead focus on the content of care and the interventions delivered. For example, findings of the COFI project indicated that less than 25% of patients received sufficient information to enable informed and/or joint clinical decisions, whereas the assessment of adherence to guidelines highlighted issues with the provision of psychological treatments.
The guidelines were developed based on feedback from international experts and have been presented to Ministry of Health members as well as international experts at a number of COFI dissemination events. The dissemination plan focused on the presentation of the findings to service providers and policy-makers through the use of workshops and travelling roadshows. These events have been held in each country and well attended by a range of stakeholders.
Impact on the scientific community
The natural experiment conducted as part of COFI is one of the largest prospective studies conducted with psychiatric inpatients. The large multi-site nature of the study and the use of a natural experimental design ensured that the shortcomings of previous research were overcome. In particular, the COFI study provided new scientific knowledge regarding different outcomes of care for different patient sub-groups. This included a large population of individuals with a diagnosis of anxiety / somatoform disorder, who are often underrepresented within research studies. The large sample, choice of different European countries and wide inclusion criteria ensure the findings are applicable across settings.
Unlike previous studies conducted within the area, the COFI study was adequately powered to detect even small differences in outcomes. Therefore, the main finding that there was no difference between the two systems of care cannot be attributed to methodological limitations. The consistency of the lack of significant differences between any of the outcomes and for any of the patient sub-groups provides clear evidence in this area.
It is often common within research to recommend that further studies be undertaken to replicate or provide more evidence. However, given the size, scope and methodological strengths of the COFI study, we have recommended that further research focuses less on organisation at the system level (concerning this particular question) as presented in Recommendation F. Instead the findings could suggest that a) research should focus on the content of care – including effective interventions and practices to further reduce the burden of mental health disorders, b) on how systems may support the advantages and disadvantages of both models to provide continuity for patients c) overcoming the limitations identified within both systems such as lack of access to psychological treatments or the perceived lack of information regarding treatment choices and d) on other areas such as experienced or anticipated discrimination (Recommendations E, F).
The findings of the COFI study have been widely disseminated within the scientific community with dissemination events including workshops where we have discussed the methodology, and conferences to discuss the results. Five papers have now been published, with a further 15 close to submission. The dataset generated from COFI is unique, such that the range of outcomes and patients provide avenues for further investigation and hypothesis generation for the foreseeable future.
Impact of user involvement
The COFI study was regularly supported by the service user and carer organisation SUGAR (Service User and carer Group Advising on Research). SUGAR is an award winning public and patient involvement group part-funded by East London NHS Foundation Trust and City University London. SUGAR have been involved in the COFI study right from the design of the research proposal and research questions, through to dissemination. The lasting impact of this partnership is evidenced by the lay summary of the COFI project. The lay summary presents the methods, findings and recommendations of the project to a lay audience, by removing technical language and ensuring information is in bite-sized chunks. The lay summary was drafted in collaboration with SUGAR to ensure that only essential information was included.
The COFI findings, particularly regarding patient-reported outcomes and satisfaction with care were highlighted by SUGAR. The lack of access to psychological treatment, the discrimination experienced and anticipated by all patient groups and requirements for more information are tangible findings which highlight the need for future improvements within mental health services, regardless of their organisation (i.e. whether functional or integrated).
The patient voice was also captured in the wealth of qualitative interviews conducted with patients – both as inpatients (61 interviews) to understand their experience of inpatient care, and within the community (188 patients). The findings highlight the key advantages and disadvantages of both systems from the patient perspective, and should be used by service providers and policy-makers when planning services (Recommendations C and D).
To celebrate and widely disseminate the lay summary, we held an interactive workshop event at the Whitechapel Gallery. The gallery was chosen as an accessible and creative space in order to engage patients, carers, researchers, clinicians and members of the public.
Impact globally
Finally, the recommendations from the COFI study extend beyond the five countries that were included in the project. By including countries with a range of different care systems, traditions of mental health care and service provision, the results are more widely applicable to other countries within Europe and beyond.
The findings of the study have been presented internationally at seminars and workshops in Australia and Far-East Asia. We have presented to international guests (from Africa, South America, the Balkan countries and representative from the WHO) within a global health workshop held in London.
Aside from the main transferable results of the project, the project also provides valuable learning material for future international multi-site trials, including the need to understand the context of each country. Our analyses highlighted the variations that occur both within and across countries, and how these would need to be considered when drawing conclusions. The use of COFI as an example of international research was presented at a keynote lecture within the European conference on Integrated Care and Assertive Outreach in Hamburg.
Main dissemination activities:
The COFI project included a comprehensive dissemination plan in order to ensure maximum impact of the project findings in optimising mental health service delivery to ultimately reduce the burden of mental disorders. Dissemination activities focused on four main activities, i) peer-reviewed scientific publications, ii) presentations to a scientific audience, iii) presentations aimed at a wider audience including clinicians and patients and iv) larger-scale conferences and workshops.
Publications
Five peer-reviewed publications have so far been published as part of the study. The publications cover the background, protocol, methodology and baseline results of the study:
Omer S, Priebe S, Giacco D. Continuity across inpatient and outpatient mental health care or specialisation of teams? A systematic review. European psychiatry : the journal of the Association of European Psychiatrists. 2015;30(2):258-70.
Giacco D, Bird VJ, McCrone P, Lorant V, Nicaise P, Pfennig A, et al. Specialised teams or personal continuity across inpatient and outpatient mental healthcare? Study protocol for a natural experiment. BMJ open. 2015;5(11):e008996.
Miglietta E, Belessiotis-Richards C, Ruggeri M, Priebe S. Scales for assessing patient satisfaction with mental health care: A systematic review. Journal of psychiatric research. 2018;100:33
Dimitri G, Giacco D, Bauer M, Bird VJ, Greenberg L, Lasalvia A, et al. Predictors of length of stay in psychiatric inpatient units: Does their effect vary across countries? European psychiatry : the journal of the Association of European Psychiatrists. 2018;48:6-12.
Bird VJ, Giacco D, Nicaise P, Pfennig A, Lasalvia A, Welbel M, et al. In-patient treatment in functional and sectorised care: patient satisfaction and length of stay. The British journal of psychiatry : the journal of mental science. 2018;212(2):81-7.
A further 15 publications have either been submitted for publication or will be submitted within the next two months. The papers cover the main findings, with at least two papers per work package associated with data collection (WP4 – 9).
Presentations to scientific audiences
The COFI project has been widely presented at academic conferences and events. This has included hosting symposia, seminars and workshops at international and national conferences, including the World Psychiatric Association Congress, and European specific conferences such as the 4th European Conference on Integrated and Assertive Outreach. Seminar presentations focused on presenting the findings from each of the work packages, whilst workshops were hosted to discuss the implications of the project with different stakeholders including policy-makers and service managers.
Presentations to a wider audience
To maximise the impact of the project in influencing service delivery and improving mental health care, national workshops were held within each of the countries. The workshops or roadshows included a basic slide set providing information about the study methods and overall results. The contents were then tailored to the setting of the workshop, to ensure that the results presented were applicable to the audience. The workshops varied in size, and were conducted within the hospitals and community mental health services included in the study, and also at local universities and academic institutions.
International workshops
Five larger scale events/ international workshops were held in the final six months of the project to develop the COFI guidelines and recommendations, and to disseminate these widely to ensure their influence on research and practice. The five events were:
1) Brussels Research Day (6th and 7th December 2017) – Hosted by partners UCL in Brussels, this event brought together academics from different disciplines, clinicians, hospital managers and policy makers to discuss the main findings, including the features of the mental health systems which support continuity (within both integrated and functional systems).
2) International global scenario planning workshop (10th April 2018) – This workshop was hosted by QMUL and involved representatives from around the world in collaboration with the NIHR Global Health Research Group on the Effectiveness of Psychosocial Interventions for Community Mental Health Care. This included researchers and clinicians from institutions in Bosnia-Herzegovina, Colombia, Egypt and across the five study countries. Representatives from different NHS Trusts were included alongside members of the World Health Organisation (WHO) and people with lived experience of using mental health services. COFI was used as a starting point for the discussion surrounding the future of global mental health care. During the workshop, arts based methods were used to engage the workshop members, and potential futures for mental health discussed.
3) Poland International Dissemination Conference (9th and 10th May 2018) – Hosted by partners IPiN within Warsaw, this event presented the key findings of the project to a wide audience of experts, academics, clinicians, managers, policy makers and representatives from the Ministry of Health including the former Minister of Health and current Deputy-Minister. The interactive programme, which was simultaneously translated into English and Polish, provided a space for discussion of the results in the context of ongoing health care reforms within Poland. The conference ended with a panel discussion including the Ministry members who outlined their vision for improvements to mental health services in Poland.
4) European workshop on the COFI Guidelines and Recommendations (14th May 2018) - hosted by QMUL in Brussels, this workshop involved the discussion and development of the COFI Guidelines and Recommendations. The meeting was attended by all teams of the COFI project, and included international experts in functional and integrated care. The meeting was chaired by members of EUCOM who represented further European countries. Following this event, the COFI guidelines and recommendations were finalised.
5) Lay summary launch and celebration (21st May 2018) – the final COFI event was hosted by QMUL at the Whitechapel Gallery. It acted as a celebration of the COFI project and focused on disseminating the results and recommendations to a lay audience. The lay summary, which was produced in collaboration with SUGAR were launched at the event, which included researchers, clinicians, patients, carers and members of the public.
Slide set for future dissemination targeted to different stakeholder groups.
To add wider dissemination of the project and to ensure dissemination continues beyond the end of the project, a slide set has been created summarising the main findings and recommendations. The slide set aims to condense the information included in this complex programme into a digestible format for use at future events. The slide set outlines the main findings and recommendations of the project and is available for a scientific audience and for a lay audience as well.
List of Websites:
http://cofi.qmul.ac.uk/
Contact details:
Study Manager: Dr Victoria Bird, Queen Mary University of London, v.j.bird@qmul.ac.uk
PI: Professor Stefan Priebe, Queen Mary University of London, s.priebe@qmul.ac.uk
EXECUTIVE SUMMARY
Background
Mental disorders are a leading cause of disability in Europe and determine high health-care costs and loss of productivity. Effective service organisation is required to maximise the effects of care and reduce this burden. A highly debated question in mental health care is whether to privilege personal continuity of care or specialisation of services. In COFI, we addressed this question by comparing two alternative systems of mental health care:
1) Functional systems: different primary clinicians are responsible for the treatment of a patient, depending on whether the patient is in inpatient or outpatient care. In these systems, the focus is on specialisation of care.
2) Integrated systems: the same primary clinician is responsible for both the inpatient and outpatient care of a patient. These systems privilege personal continuity of care.
Methods
COFI was carried out in five countries - Belgium, Italy, Germany, Poland and the UK, where the two systems are both available in routine mental health care. This meant that we did not change or allocate patients to a care system but studied these systems in a natural experiment, so that results are immediately transferable to policy and practice. Patients with a diagnosis of psychotic, affective or neurotic / somatoform disorders (F2-F4) admitted to a psychiatric hospital during a 14-month period were included and followed-up for 12 months following admission. Overall, 57 hospitals and 7,302 patients participated. The primary outcome was rehospitalisation over one year. We also assessed other clinical and social outcomes (inpatient bed-days, compulsory rehospitalisations, adverse events, social functioning), quality of care, and experience and costs of care.
Results:
We followed-up 6369 out of the 7302 patients (87.1%). Over 35% had been readmitted to hospital. We did not find any statistically significant differences in re-admissions between functional and integrated systems. We also did not find any differences in inpatient-bed days, compulsory readmission, adverse events and social functioning of patients. Moreover, no difference was found in the individual countries or in subgroups defined by age, gender, socio-economic status, migrant status, and whether or not patients had been previously hospitalised. Costs and quality of care did not appear to differ between the two systems either, despite the high variety in funding and governance arrangements that we described in the different countries. The only difference found between the two systems was in the experience of hospital care (but not the overall experience of care) which was significantly better in integrated systems.
Impact and dissemination:
The equivalence of the two systems in terms of clinical and cost outcomes is a major contribution to scientific knowledge and policymaking decisions in Europe and across the world. Changing mental health care organisation on a system level is expensive. Our findings show that re-organisations of care from integrated to functional systems, or vice-versa, are unlikely to have a significant impact on clinical outcomes justifying the expense. This knowledge has the potential to save substantial amount of resources for European Union Member States, which can instead be used for more influential changes in mental health care provision. We summarised these findings and recommendations in guidelines for policy-makers and commissioners of mental health services and circulated them widely. Dissemination activities included national workshops in each participating country and international workshops with world-renowned experts and policy-makers. Scientific dissemination included presentations at the most attended conferences on mental health care and publications in major peer-reviewed journals.
Project Context and Objectives:
A summary description of project context and objectives (not exceeding 4 pages).
Background
Across Europe, mental disorders are a leading cause of disability that result in high health-care costs and loss of productivity through unemployment and absenteeism from work. Within a given year, estimates suggest that approx. 40% of people with the EU – that is 165 million people, suffer from a mental disorder. In order to reduce the burden of mental disorders on individuals, communities and society as a whole, optimal service organisation is required. Throughout Europe, countries are seeking to improve the organisation of mental health care. This is often achieved through radical reforms with far reaching changes for the national health care systems, the staff who work in them and the patients who receive care. What’s more, such sweeping changes consume a large amount of resources both financial and human. However, the reforms happening within and across European countries have been inconsistent, with policies regarding the optimal organisation of mental health care currently being made in the absence of any scientific evidence.
A central question related to ongoing mental health care reforms relates to the organisation between inpatient (hospital) and outpatient (community) services. In particular, there is a debate as to whether the same responsible clinician – usually the consultant psychiatrist, should see the patient when they are an inpatient and as an outpatient within the community, or whether there should be separate in and outpatient consultant psychiatrists. The two dominant models of care related to this debate have been summarised as follows:
1. Functional: care is provided by separate psychiatrists in distinct services, with the transition between services coordinated through a network of regulated referrals;
2. Integrated: care is provided by the same consultant psychiatrist across different services, the responsible clinician co-ordinates all interventions.
The debate regarding which of the two systems is most effective has led to the on-going and inconsistent reorganisations of mental health care, despite being driven by only poor and partial research evidence.
Previous evidence - Systematic review and findings
An initial review of the existing evidence for studies comparing integrated and functional systems of mental health care identified 17 studies using a range of quantitative and qualitative methods. In eleven of the studies, integrated systems were associated with a shorter length of stay, better recovery after a first episode of psychosis and greater satisfaction – from both the clinician and patient perspective. However, two studies highlighted that functional care was associated with fewer hospital admissions and fewer serious incidents on the ward, whilst the remaining studies provided mixed results, favouring integrated systems for some outcomes, and functional systems for others.
Qualitative interviews with clinicians working in each system and with patients who have received mental health care suggest that each system has certain advantages. The advantages of each system have been outlined in Table 1.
Limitations of the existing research
Although previous studies have assessed the two types of mental health care, these studies had various limitations that made it difficult to draw any firm conclusions. Firstly, many of the previous studies used before/after designs or conducted cross-sectional surveys at a single time point. One of the main limitations with using a before and after design, is that the new system (regardless of whether it was functional or integrated) produced better outcomes. Secondly, studies often failed to control for confounding variables that may influence the result. Thirdly, many of the studies were small. This meant that they lacked the statistical power to test for small differences. Finally, most studies only focused on one or two outcomes, which varied across the review. In order to evaluate properly the two systems of mental health care, comprehensive data was needed on a range of clinical, social, economic and patient-reported outcomes.
This meant that prior to the COFI study, no high-quality research evidence was available from large-scale studies to help inform the debate and guide policies on the best way to organise mental health care services.
COFI was designed to overcome the limitations identified with the previous research to specifically:
a. Provide enough statistical power to properly investigate small differences between integrated and functional care as even small differences in outcomes may be important at a public health level. For example, a 5% difference in the use of hospital services has cost implications at a national level.
b. Overcome the “novelty effect” by studying the two systems in routine operation without the need to implement a new system or change the services being studied.
c. Investigate the benefits of the two systems of mental health care for different subgroups of patients
d. Obtain reliable findings that can be applied across contexts.
Context for the COFI study
The COFI study was conducted in five European countries (Belgium, Germany, Italy, Poland, and United Kingdom). Importantly, within each country, functional and integrated systems of mental health care already co-existed in routine practice. This enabled the COFI study to compare the two mental health systems in day-to-day mental health services, without the need to set up new services or change the organisation of care. We selected countries to include in the COFI study that all have different funding arrangements, traditions of mental health care and mental health services that look and operate differently. The countries included represent a range of mental health services across Europe. Thus, by comparing the results of integrated and functional care within and across these five countries, the results of the study are generalisable to other countries within Europe and beyond.
How are functional and integrated mental health services organised in the countries of the COFI study?
The frameworks which guide mental health policies and practice in different European countries traditionally tend to favour one approach over the other. For example, in Germany, Belgium and Poland mental health services were traditionally functional with separate in and outpatient psychiatrists. In contrast, within the UK and in Italy, mental health services tended to be integrated.
In the United Kingdom, mental health services are provided at a regional level by metal health National Health Service (NHS) Trusts. Over the last decade, Trusts have gradually shifted from an integrated system of mental health care to a functional way of working. The aim of this shift was to improve the quality of inpatient care; however, the shift has been criticised for reducing continuity of care and having a negative impact on the relationship between the psychiatrist and the patient. Likewise, in Italy, which was also traditionally integrated, local differences in how services are funded and arranged has resulted in both systems being available, often within neighbouring areas.
In contrast, within Germany, the traditional model of mental health care was functional. This was characterised by different services providing different activities, with the link between services organised through referrals. However, recent initiatives and projects have aimed to strengthen the coordination between services, making one service responsible for both in- and outpatient care. This has led to an increase in the number of integrated services within the country.
Finally, within both Belgium and Poland a number of substantial reforms of mental health care are currently ongoing. This has resulted in integrated and functional mental health care co-existing within the same region.
The differences and similarities of the mental health reforms within the five COFI countries highlight the importance of providing good quality evidence for policy-makers regarding this key question. The design and timing of COFI therefore provided an ideal opportunity for study.
Overall aim of COFI
The overall aim of COFI was to compare the effectiveness and cost-effectiveness of functional and integrated systems of mental health care across Europe. To overcome problems identified with existing research, COFI utilised countries where both systems of mental health care were in routine operation, and employed a “natural experiment” as the design. This would enable the results to be readily applied to mental health care systems across Europe, as the investigators did not change the care they were studying. To fully meet this aim, the comparison considered clinical and social outcomes, as well as patients’ quality of life, health and social needs, safety and cost-effectiveness of care. Different patient subgroups were defined by diagnosis, age, gender, socio-economic and migrant status to understand whether the different systems of care are more suited to particular patient groups.
Research questions addressed
To achieve the overall aim of the COFI project, the research questions addressed throughout were:
1) Primary research question:
a) Do rates of rehospitalisation differ between the functional and integrated approaches?
2) Clinical and social outcomes
a) Is there a difference in number of voluntary and compulsory rehospitalisations per patient and yearly inpatient bed days between the two approaches?
b) Are untoward incidents (deaths, suicides, physical violence committed or experienced by patients) more frequent in either approach?
c) What are the social outcomes, in terms of employment, accommodation, living situation, subjective quality of life and social contacts of patients treated by either approach?
3) Costs of care:
a) What are the direct costs of care (i.e. related to use of services) within each approach?
b) Which approach is most cost-effective?
4) Quality of care
a) Does adherence to national and international treatment guidelines, and clinical decision making vary between the two systems?
5) Experience of care
a) What are treatment satisfaction and the experience of care provided by the two approaches and do they differ?
6) National policies and legislations
a) What national policies and legislations on the organization of mental health care support the implementation of functional and integrated systems
Project Results:
The COFI project brought together a multi disciplinary consortium of experts based in six institutions across five European countries. Members of the team included psychiatrists, psychologists, social scientists, health economists and experts in policy analysis (Figure 1).
The study was designed and implemented to provide high-quality evidence on a range of clinical, social and patient-reported outcomes for integrated and functional care. The comparisons focused on individuals with a diagnosis of psychotic (F2), affective (F3) neurotic, stress-related or somatoform (F4) disorders, who are the main users of psychiatric care, and provided information on the effectiveness of the two systems for different patient sub-groups. Overall, the aim was to provide outcomes and comparisons what would help clinicians, patients, service managers and policy makers to optimise the organisation of mental health care - particularly the transition between in- and outpatient services.
To achieve the project objectives and to address systematically the research questions, the project was organised into a number of interlinking work packages, with the different consortium members responsible for different work streams:
WP1: Project management (QMUL) – including overall management of the project and consortium.
WP2: Study preparation (QMUL) – Preparation of all the materials required for the natural experiment.
WP3: Assessment of existing national policies and legislations (UCL) – identifying and assessing existing national policies regarding mental health care to explore how these policies are implemented in practice to support continuity of care (Research question 6).
WP4: Data collection and data quality assurance (QMUL) - Central to the COFI project was the natural experimental comparing functional and integrated mental health care. This WP outlined the procedures for data collection and quality assurance (Research questions 1-4).
WP5: Comparison of the effectiveness of functional and integrated systems on clinical and social outcomes (QMUL) - comparing the clinical and social outcomes to test comparatively the effectiveness of the two systems (Research question 1-2).
WP6: Comparative analysis of the effectiveness of the two systems on patient reported outcomes (UNIVR) – providing evidence for the effectiveness of the two systems on patient reported outcomes, including quality of life and satisfaction with services (Research question 5).
WP7: Comparison of the quality of care provided by functional and integrated systems (TUD) –assessing care quality, therapeutic relationships and adherence to guidelines (Research 4).
WP8: Comparison of the cost structure and cost-effectiveness of the functional and integrated systems (KCL) – Costs associated with both systems of care were collated during this WP and cost-effectiveness determined by combing cost data with quality of life data collected in the previous WPs (Research question 3).
WP9: Qualitative study (IPiN) – exploration of the experience of patients and clinicians who receive or deliver each system of care through in-depth qualitative interviews (Research question 5).
WP10: Integration and dissemination of findings (QMUL) – integration of the findings to produce guidelines and recommendations for policy-makers and wide dissemination of the project.
Figures 2 provides a schematic diagram of the overall COFI project design.
The project successfully achieved the aims of each work package, presented here is a summary of the main findings across the different work packages.
Outcome 1) Existing policy to support continuity
WP3 identified and assessed existing policies and legislation documents relating to the organisation of mental health care within the five countries. A literature review of policies was supplemented by interviews with key informants such as service managers, clinicians, and health service researchers. The interviews provided more detail on the actual implementation of each policy within the healthcare context. The review and interviews highlighted the complexity of mental health care within and across countries, and how different organisational characteristics may support the implementation of integrated or functional approaches to care.
Two main organisations of mental health services that typified care within the countries of the COFI study were identified. Within the UK and Italy, mental health services are predominantly public organisations funded from taxes paid by the population; these are termed National Health Systems. In contrast, within the three remaining countries, private not-for-profit and for-profit organisations and practitioners provide care. These private organisations and clinicians receive funds from public authorities and income-related contribution. These were termed Regulated Market (RM) Systems. Despite falling into these two categories, there was significant variation between the organisation of care within the COFI study.
Features of the mental health systems that impact on the implementation of functional and integrated care were then identified. These system features were related to i) financing mechanisms including out-of-pocket payments for patients, and the mode of payment of doctors; ii) provision of health services such as the public-private mix and freedom of patient choice; and iii) policy regulation. Each of these features of the health care system and how they were related to NHS and RM systems was described in detail within the deliverable for WP3 - Care system conditions facilitating or hindering integrated personal continuity of care for SMI patients: A comparison of five European systems.
Briefly, these features of the system differed according to whether a country operated a NHS or RM system. Even within countries that broadly operated using the same system, there was variation in practice. Table 2 highlights how each of the system features operates within each country.
Table 2 – Features of mental health systems across the five countries
Overall, it was found that NHS systems (UK and Italy) favoured integrated approaches. This was due to the organisational mechanisms focusing on long-term community care, with fewer out of pocket payments for patients, more national and local regulation on care pathways and less choice for patients regarding their individual clinicians. In contrast, in countries where RMS were dominant, whether integrated or functional systems were favoured was dependent on the choice of clinicians and individual patients – choices that were strongly determined by financing mechanisms and the different funding available. When the focus of care was on acute episodes, with patients having more out-of-pocket payments and having more freedom to choose services and clinicians, systems tended to favour functional care, although integrated care was also possible.
The findings of WP3 were important in highlighting the context for the study. The matrix summarised above was submitted as a publication to add to the scientific knowledge in this area.
Outcome 2) State-of-the art research methodology
The focus of the project was to overcome the limitations identified with previous research (as highlighted by the systematic review and summarised in the background section) to produce reliable evidence for policy-makers regarding optimal organisation of care. In order to overcome the limitations, the study adopted a state-of-the-art research design conducting a large-scale multi-site natural experiment. A natural experiment means the investigators do not change anything about the system being studied, nor do they allocate patients to different conditions. By studying systems that were already in routine operation within the five countries, we therefore avoided the “novelty effect” described earlier.
The deliverable for WP5 – Report on the assessment of the effectiveness of functional and integrated care in the total sample and in patient subgroups, provides full details of the study methods. Briefly, over a 14 month period (from October 2014 – December 15), the study aimed to recruit all eligible consecutive admissions to general adult inpatient psychiatric wards. Patients were followed up for one-year following the date of their admission (termed the index admission). We aimed to include patients who were representative of the individuals who use psychiatric hospital care. The inclusion criteria balanced this aim whilst ensuring comparable populations across the five countries.
The inclusion criteria for patients were:
i. Aged 18 years of age or older;
ii. A clinical diagnosis of psychotic disorder (F20–29), affective disorder (F30–39) or anxiety/somatisation disorder (F40–49) according to the International Classification of Diseases—ICD—10;
iii. Hospitalised in a general adult psychiatric inpatient unit – defined as at least one overnight stay
iv. Sufficient command of the language of the host country to provide written informed consent and understand the questions in the research interviews; and
v. Capacity to provide informed consent.
Patients were excluded if they had an organic brain disorder or severe cognitive impairment.
Where possible patients were approached within 48 hours of admission. This was to reduce the chance of bias within the sample and to include as many inpatients as possible. A range of variables including clinical, social, cost, patient-rated and safety outcomes were collected to address the different research questions (the variables will be described in detail within the relevant sections of this report).
Allocation to the two approaches differed across the countries. In the UK and Italy, the locality of the patient determined whether they received functional or integrated care. All patients within the same area and hence the same hospital received the same type of care. In contrast, in the three remaining countries (Belgium, Germany and Poland), whether the patient receives functional or integrated care is dependent on a number of patient, clinician, service and organisational factors. Different patients within the same area and within the same hospital receive either functional or integrated care based on clinical decisions, insurance arrangements, local practices and service arrangements or the choice of the patient.
Fifty-seven hospitals were included across the five countries. The sample size calculation indicated 5123 patients would be needed at one-year follow up to achieve our aims. We recruited 7302 eligible patients and were able to follow up 6369 participants (87.2%). Of those followed up, 2313 received personal continuity, 4051 specialisation and five were not allocated to either system.
Given the number of recruited and followed up patients, one of the main achievements is that COFI is one of the largest longitudinal studies of psychiatric inpatients ever conducted within Europe.
The consort diagram for the study is shown in Figure 1 – which was adapted from the main results paper (Giacco, et al. (under review) The same of different psychiatrists for in- and outpatient treatment? One year outcomes in a natural experiment in five countries. British Journal of Psychiatry).
The full method for the study was published in the protocol paper – Giacco et al. (2015) Specialised teams or personal continuity across inpatient and outpatient mental healthcare? Study protocol for a natural experiment. BMJ Open 25;5
Outcome 3) Increased understanding of inpatient services across Europe
One direct result of the project, and specifically the large multi-site natural experiment, is an increased understanding of inpatient psychiatric care including a) the characteristics of people who use inpatient services, b) satisfaction with inpatient services including the initial experience of hospital admission and c) length of stay across the five countries. Such information is crucial for service managers and policy-makers to help optimise inpatient care to not only improve outcomes for all patients, but also improve the experience.
a) Characteristics of individuals using inpatient services:
COFI was unique in that it aimed to include individuals within 48 hours of admission, and included both voluntary and involuntarily admitted patients, with few restrictions on inclusion. The characteristics of the patients included in the COFI study are shown in Table 3.
b) Satisfaction with inpatient treatment
Within the UK, allocation to either functional or integrated care is geographically based. Given the large UK sample size, we were able to assess satisfaction with the initial index admission, to test whether there were any significant differences between the two types of care. The results indicated that individuals in integrated systems were significantly more satisfied with their inpatient care compared to those treated in functional systems. Furthermore, the results were still significant, even when controlling for the hospital, and potential confounding factors such as age, gender, admission status, severity of illness and diagnosis. Full results are reported in Bird, et al. (2018). In-patient treatment in functional and sectorised care: patient satisfaction and length of stay. British Journal of Psychiatry, 212, 81-7.
In addition to the impact of the system of mental health care, we also wanted to assess which patient characteristics may be associated with satisfaction with inpatient care. Data was included from all study sites. Based on the 7302 participants who were included in the COFI baseline, we established that the following factors were associated with lower satisfaction with inpatient care:
- Being younger
- Unemployed
- Living alone
- Not having seen a close friend within the last week
- More severe illness
- Repeat admissions to hospital
- Higher education levels
- Comorbid personality disorder
- Being involuntarily admitted
- Recruited from the UK.
Full results are reported in Bird et al. (under review) Factors associated with satisfaction of inpatient psychiatric care: A cross-country comparison. Australian and New Zealand Journal of Psychiatry.
To explore in-depth the reason behind both positive and negative appraisals, a small qualitative study was conducted with 61 patients admitted to three different hospitals within the UK. The brief interviews (maximum of 15 minutes) explored the experience of the first few days in hospital, and linked the themes to both positive and negative appraisals. One of the main results of the analysis indicated that although there was a lot of overlap, with individuals describing both positive and negative experiences, there were distinct themes linked to positive and negative appraisal of inpatient care (Shown in Table 4). The full findings, including illustrative examples of each theme are available in the full publication, Chevalier et al. (in press) Exploring the initial experience of hospitalisation to an acute psychiatric ward. PLoSOne.
) Length of stay across Europe
The large sample in COFI allowed us to explore what patient characteristics were associated with increased length of stay within the hospital, and to explore whether these were the same or different across the countries. Full details of the analysis of length of stay are available in the published paper
Dimitri, et al. (2018). Predictors of length of stay in psychiatric inpatient units: Does their effect vary across countries? European Psychiatry, 48:6-12.
To summarise, data indicated large variation across countries, with the average length of stay under 18 days for Italy (17.9) compared to nearly 55 days for Belgium (54.9). Across all countries, having psychosis, greater symptom severity, being homeless, living alone, receiving benefits, having a medical comorbidity, increased social isolation, a history of previous admission and being involuntarily admitted all predicted longer stays. However, there were interactions with the country for some of the variables. This suggested that although there are some generalisability across countries, the same patients have different lengths of stay depending on the country. For example, homelessness predicted a longer length of stay in most countries, apart from Germany where it was associated with a shorter length of stay. For most other factors, the direction of the association was the same, but the predictive values were significantly different. Furthermore the differences in length of stay between the countries remained significant, even when the different predictor variables were adjusted for, suggesting that factors aside from patient characteristics determine differences in the length of stay across countries.
Outcome 4) Comparison of the effectiveness of the two systems on clinical and social outcomes
Work packages 4 and 5 concentrated on the main aim of the COFI study – namely to compare the two systems of mental health care with regards to patient clinical and social outcomes. Of the 7302 participants included in the study, 6369 (87.2%) were followed up after one year and provided data on rehospitalisation.
One finding of the study related to the percentage of individuals who are rehospitalised within one year. COFI was able to provide reliable data on rehospitalisation rates for a range of diagnostic groups and within different counties. Across the whole sample 2259 (35.5%) of patients were readmitted to hospital within one year. Within functional systems, 1405 out of 4051 (34.7%) were rehospitalised compared to 854/2313 (37%) in integrated systems. The difference between the two systems of care was not significant, and remained not significant when controlling for the hospital and all other potential confounding variables. This indicated that for the primary outcome of the study there was no difference in effectiveness between integrated and functional care.
The difference in rehospitalisation rates between functional and integrated care was also assessed within difference patient sub-groups. The following sub-groups were included in the analysis; country, diagnostic group, gender, age, socio-economic status, migrant status and type of admission. As with the main sample, the differences between functional and integrated care for each of the sub-groups investigated was not significant. Therefore corroborating and strengthening the main finding.
One interesting outcome and products of the study is the dataset available for rehospitalisation rates for different sub-groups of patients. These are presented in Table 5.
We also assessed other clinical and social outcomes that could be influenced by the system of care. Total length of stay per year (bed days) and the number of involuntary admissions were important clinical outcomes. As with rehospitalisation we found no differences between functional and integrated care for either the main sample or in any of the sub-groups studied. Across the study, data was collected on a number of untoward events, including death (any reason), completed suicide, suicide attempt, side-effects of medication requiring somatic hospitalisation, victim of physical violence or perpetrator of physical violence. The total number of untoward events and a dichotomous variable of whether or not a person had experienced an untoward event were assessed. In both case there were no differences between functional and integrated care for the main sample or any of the sub-groups.
Finally, as a measure of objective social situation, the SIX index was calculated. This includes items on Employment, Housing, Living situation and Friendships. Consistent with the clinical outcomes, no significant differences were found between the two systems of care for either the main or sub-group samples.
Taken together, the results present a consistent narrative indicating that there is no difference in effectiveness between the two systems of mental health care. The full results of the study and in-depth analysis is available in the COFI main paper: Giacco, et al. (under review) The same of different psychiatrists for in- and outpatient treatment? One year outcomes in a natural experiment in five countries. British Journal of Psychiatry), and within the deliverable for WP5: Report on the assessment of the effective of integrated and functional care in the total sample and in patient subgroups.
Outcome 5) Comparison of the effectiveness of the two systems on Patient-Reported Outcomes
Patient-satisfaction is a key patient-reported outcome when optimising mental health services. It has been shown to impact on treatment adherence, and predicts clinical outcomes within both the hospital and the community. One of the main contributions of the COFI study to the scientific knowledge regarding the assessment of patient satisfaction was the submission of a thesis on the topic (Patient's appraisal of mental health care in five European countries: findings from the COFI study). Part of the thesis included a published systematic review detailing existing ways to measure satisfaction – which informed the choice of tools within the COFI study. Full details of the review can be found in the published paper Miglietta et al. (2018) Scales for assessing patient satisfaction with mental health care: A systematic review. Journal of Psychiatric Research, 100: 33-46.
A total of twenty-eight scales were identified in the review, each scale has been used at least twice in the literature. The scales were diverse, targeting different mental health services (inpatient, outpatients, supported accommodation), and varied in the number of items they included (3-45) and on the likert scales used (4-10 points). However there were some commonalities, with the contents of the scale covering “relationship with staff”, “staff competence”, “overall satisfaction”. The four most frequently used scales were reported. This helped inform the decision regarding the best way to measure satisfaction. Within the COFI study, two of the four most frequently used measures, the -Client Assessment of Treatment scale (CAT) and Verona Service satisfaction scale (VSSS) were used to measure satisfaction with inpatient and overall care respectively.
Unlike, inpatient satisfaction (presented above), there was no difference between the two types of care relating to overall satisfaction with services as indicated on the VSSS-32, although some sub-scales indicated a trend towards favouring integrated care.
Alongside patient satisfaction, quality of life and the experience of discrimination were also measured. For quality of life, the MANSA was used, whereas for discrimination, a modified version of the DISC-12 was used to separate actual and anticipated discrimination. Key findings of these analyses indicated that there were no differences between the two systems of care for either outcome. However, there was significant variation between the countries for anticipated and experienced discrimination, with patients in Belgium reporting higher levels of experienced discrimination, and individuals in Poland anticipating more discrimination. The findings also indicated that patients anticipated discrimination in all areas of their lives including finding a job, developing a close relationship and feeling the need to conceal their diagnosis. Finally, contrary to previous campaigns, which often focus on schizophrenia and psychosis, individuals with all conditions were likely to experience and anticipate discrimination. This was particularly the case for individuals with anxiety / somatoform disorders. This suggests that national stigma campaigns should not just focus on those with a diagnosis of psychosis; rather they should address the discrimination experienced and anticipated by all individuals with mental health conditions, including those with anxiety disorders.
Outcome 6) Adherence to guidelines and quality of care
To compare the systems of care with reference to adherence to treatment guidelines, we selected a very specific patient group, namely individuals with bipolar disorder hospitalised for a depressive episode. The review of existing guidelines highlighted the wealth of potentially relevant guidelines that could be followed within services. It also highlighted how different countries follow and adhere to different policies and practices even when considering the same diagnostic group. Full details of the guideline review can be found in the deliverable D7.1 Report on the comparison of adherence to guidelines of the interventions provided by the two systems
Based on the extraction of data from the different guidelines, a new measure of guideline adherence was created. This included the following aspects (which could be adapted and tailored for any other major psychiatric Axis I condition)
− Appropriateness of pharmacological treatment to patients’ diagnosis
− Adherence to standards of dosage and length of the treatment
− Appropriate provision of psychological and psychosocial interventions
− Short interval between discharge from the hospital and ambulatory mental health encounter
− Patient involvement in clinical decision-making
The grade of fulfilling the criterion was rated between 0 and 2 for each item, with a sum score calculated over the five aspects (max 10).
Consistent with the main findings, there were no significant differences in the scores between the two types of care regarding adherence to guidelines. For all items, and for the total sum score, the two types of care were equivalent, as shown in Table 6. What the analysis did however highlight, was the lack of availability for psychological therapies. Unlike all other items, where both systems scored 1.16 and 1.43 (out of a maximum of 2), appropriate provision of psychological therapies scored only 0.55 for functional care and 0.42 for integrated. This highlighted the need for better access to psychological therapies for patients across Europe.
Within the larger sub-sample (2181 patients), two further aspects of the quality of care were comparatively investigated between the two systems, namely therapeutic relationships and involvement in clinical decision-making. Therapeutic relationships were rated using the Helping-Alliance Scale (HAS). On average, a HAS score of 7.41 (SD 2.15) was calculated across all five countries. Consistent with the majority of outcomes reported, no significant difference in therapeutic relationship was found between the two care systems. A paper draft has been prepared: Schon et al. (in preparation) Predictors on therapeutic relationship: A study on patient’s view of health alliance in psychiatric care in five different/European countries.
A new tool was developed to assess clinical decision-making (shown in Box 1).
1. desire for information regarding your pharmacological treatment choices in a given situation.
a. I should be informed about all the different treatment strategies/alternatives.
• Strongly disagree, Slightly disagree, Neither agree nor disagree,
Slightly agree, Strongly agree
2. attitude towards decision-making regarding your pharmacological treatment.
a. I want to make the final decision.
b. I want to make the final decision after seriously considering my clinicians opinion.
c. My clinician and I should share responsibility for making the best decision for me.
d. My clinician should make the final decision, but seriously consider my opinion.
e. My clinician should make the final decision.
3. recall the last decision regarding your pharmacological treatment that was made.
a. The last decision affected
• beginning of pharmacological treatment
• discontinuation of pharmacological treatment
• dosage alterations
• addition of another drug
• switch to another medication
b. The last decision was made
• less than 1 month ago
• 1-2 months ago
• 3-6 months ago
• more than 6 months ago
4. indicate how much you agree with the statement.
a. I have been informed about different treatment strategies/alternatives
b. I had the chance to ask for as much information as I needed about the different treatment choices
c. The clinician took my expectations and concerns into consideration
d. I know the advantages and disadvantages of the different treatment options
• Strongly disagree, Slightly disagree, Neither agree nor disagree,
Slightly agree, Strongly agree
5. indicate which statement is true for you for this decision
• I made the final decision.
• I made the final decision after seriously considering my clinicians opinion.
• My clinician and I shared responsibility for making the best decision for me.
• My clinician made the final decision, but seriously considered my opinion.
• My clinician made the final decision.
Box 1 – Clinical decision-making tool
The measure assesses the concordance between the wishes of the patient and what they actually received. Although the study did not find any differences in decision-making between the two types of care, it did highlight a wider issue with patient involvement within mental health services. Although over 60% of patients strongly asserted their wish to be informed about all the different pharmacological treatment strategies and alternatives, less than 25% (22.8%) felt adequately informed.
Deliverable 7.2 Report on the comparison of patients' satisfaction with therapeutic relationship within the two systems provided full details of the analysis.
Outcome 7) Economic costs and unit costs of integrated and functional care across Europe
One outcome of the project with the potential for lasting impact related to the economic evaluation conducted as part of the project. Through the use of existing literature (e.g. Personal Social Services Research Unit annual compendium and NHS Reference costs or the Agency of Health Technology Assessment and Pricing) and a new data collection tool completed by the hospital and community services, unit costs for different mental health services have been calculated for each of the countries included in the study.
The resulting unit costs are shown in Table 7, which highlights the cost of each service in the local currency. These costs were then transformed into Purchasing Power Parity (PPP) adjusted costs (in Euros) for the purposes of the analysis.
Unit costs were combined with data on patient quality of life (collected as part of the sub-group) to determine the cost effectiveness of each system. Overall, the results indicated that there was no difference between the two types of care. This was in part due to the variation in both unit costs and resource use across the difference countries, and due to differences in quality of life measures. Table 8 summarises the main findings for each country.
The full detailed economic analysis is currently being prepared for submission, with full details included in the deliverables 8.1 Report on the assessment of mental health care costs within the two systems and 8.2 Report on the comparison of the cost effectiveness of the two systems in the total sample and patients' subgroups and presentation of different models of cost analysis.
Outcome 8) Understanding the experience of care from the perspective of patients and clinicians
To understand the experiences of each system of mental health care, both patients and clinicians were included in in-depth interviews. For the patients, the sample included individuals with a range of diagnoses and different experiences of hospital care, whilst the clinicians varied in their professional background and experience within services. Details of the interview procedure, topic guides and details of the thematic analysis conducted are included in the deliverable for WP9 – Report on qualitative analysis of in-depth interviews through thematic analysis.
Thematic analysis was used to analyse the data, with codebooks developed separately for clinician and patients interviews. The interviews generated a wealth of data, with the codebooks containing 266 codes (grouped into ten families) for patients and 245 codes (grouped into nine families) for clinicians. Within the deliverable and papers submitted for publication, full details of the codebook with example quotations are available. Here we have focused on the presentation of the main advantages and disadvantages reported for each system (presented in Table 9). Across the 188 patient interviews and 63 clinician interviews, one of the most striking results was the consistency of themes. Although interviewed separately with slightly modified versions of the topic guide, the themes present in the data were, in most cases apparent for both clinicians and patients, and across the five different countries, despite the differences in care organisation.
Potential Impact:
To maximise the impact of the project, one of the main outputs was the development of the COFI guidelines. The guidelines included a summary of the main findings and presented the recommendations from the study. These recommendations have the potential to impact on both the delivery of mental health services and on research into mental health care. The summary findings and the recommendations are presented below:
Summary of main findings and conclusions:
In summary, the main findings of the study are:
• Across the study, 35-40% of people were readmitted to hospital within a one-year period, although there was some variation within and across countries.
• The system of care e.g. functional or integrated care, had no significant impact on any clinical and social outcomes for patients at one-year following admission to a psychiatric hospital. This result was consistently found for the main sample and for different subgroups, including subgroups by country, diagnosis, age, and whether or not the person had previously been admitted.
• Patients were more satisfied with their inpatient treatment within integrated care systems; however, there was no difference in overall satisfaction (including both in and outpatient care) following one year.
• Patient and clinician experience indicated a mirroring of the advantages and disadvantages of each system, such that the advantages of one system mirrored the disadvantages of the other. The views on the advantages and disadvantages were consistent between patients and clinicians and across the different countries.
• Anticipated and experienced discrimination was apparent for all patients regardless of diagnostic group. Experienced and anticipated discrimination was particularly pronounced for individuals with a primary diagnosis of F2 (psychosis) and F4 (anxiety / somatoform).
• Overall, despite being a hotly contested topic, whether patients are treated by the same or different psychiatrists across in- and out-patient care had no significant impact on any of their outcomes as measured at the one-year follow-up.
Recommendations arising from the study
A. If the aim is to improve long-term outcomes for patients, costly reorganisations of mental health care systems should not focus on changing from integrated to functional care or vice versa.
B. If the aim is to improve patients’ experience of in-patient treatment, integrated care may be seen as preferable.
C. Both approaches have strengths and weaknesses – organisational considerations in the local context and preferences of patients and clinicians may favour one of the two approaches.
D. Patient preferences are particularly important when there is a choice between the two systems in the same service. Potential advantages and disadvantages of each system should be considered when making the choice.
E. Mental health services should consider ways to address the discrimination that psychiatric patients anticipate and experience, including patients with anxiety and somatoform disorders.
F. Research and policy may focus less on the organisation of care at a system level, but instead aim to improve the actual treatment within any given organisational system.
The guidelines and recommendations, and the project more widely will have the following anticipated impact:
Impact for services and policy makers
The recommendations of the COFI project suggest that changing the organisation of care by moving from functional to integrated systems or vice versa will not greatly benefit patients in terms of clinical, social and economic outcomes. Having high quality evidence highlighting the equivalent effectiveness in terms of the main outcomes has the potential to save resources for mental health services across Europe. Mental health care reforms and changes in the organisation of care consumes both human and financial resources. Such changes often occur in the absence of any evidence. The COFI study is the largest study to assess the impact of different service models in routine operations, and the results can be easily used by policy-makers and commissioners considering such a reform.
Recommendations B, C and D are particularly important when planning services, as they point to the advantages and disadvantages of both systems, and to the consideration of patient choice when planning care. At a local and national level, policy makers may wish to consider the features of mental health services identified in WP3, which were linked to either implementing functional or integrated care. The local context including the wishes of patients and clinicians should be taken into account when reorganising services. Recommendations A and F suggest that both policy and research should move away from studying change at the system level, but should instead focus on the content of care and the interventions delivered. For example, findings of the COFI project indicated that less than 25% of patients received sufficient information to enable informed and/or joint clinical decisions, whereas the assessment of adherence to guidelines highlighted issues with the provision of psychological treatments.
The guidelines were developed based on feedback from international experts and have been presented to Ministry of Health members as well as international experts at a number of COFI dissemination events. The dissemination plan focused on the presentation of the findings to service providers and policy-makers through the use of workshops and travelling roadshows. These events have been held in each country and well attended by a range of stakeholders.
Impact on the scientific community
The natural experiment conducted as part of COFI is one of the largest prospective studies conducted with psychiatric inpatients. The large multi-site nature of the study and the use of a natural experimental design ensured that the shortcomings of previous research were overcome. In particular, the COFI study provided new scientific knowledge regarding different outcomes of care for different patient sub-groups. This included a large population of individuals with a diagnosis of anxiety / somatoform disorder, who are often underrepresented within research studies. The large sample, choice of different European countries and wide inclusion criteria ensure the findings are applicable across settings.
Unlike previous studies conducted within the area, the COFI study was adequately powered to detect even small differences in outcomes. Therefore, the main finding that there was no difference between the two systems of care cannot be attributed to methodological limitations. The consistency of the lack of significant differences between any of the outcomes and for any of the patient sub-groups provides clear evidence in this area.
It is often common within research to recommend that further studies be undertaken to replicate or provide more evidence. However, given the size, scope and methodological strengths of the COFI study, we have recommended that further research focuses less on organisation at the system level (concerning this particular question) as presented in Recommendation F. Instead the findings could suggest that a) research should focus on the content of care – including effective interventions and practices to further reduce the burden of mental health disorders, b) on how systems may support the advantages and disadvantages of both models to provide continuity for patients c) overcoming the limitations identified within both systems such as lack of access to psychological treatments or the perceived lack of information regarding treatment choices and d) on other areas such as experienced or anticipated discrimination (Recommendations E, F).
The findings of the COFI study have been widely disseminated within the scientific community with dissemination events including workshops where we have discussed the methodology, and conferences to discuss the results. Five papers have now been published, with a further 15 close to submission. The dataset generated from COFI is unique, such that the range of outcomes and patients provide avenues for further investigation and hypothesis generation for the foreseeable future.
Impact of user involvement
The COFI study was regularly supported by the service user and carer organisation SUGAR (Service User and carer Group Advising on Research). SUGAR is an award winning public and patient involvement group part-funded by East London NHS Foundation Trust and City University London. SUGAR have been involved in the COFI study right from the design of the research proposal and research questions, through to dissemination. The lasting impact of this partnership is evidenced by the lay summary of the COFI project. The lay summary presents the methods, findings and recommendations of the project to a lay audience, by removing technical language and ensuring information is in bite-sized chunks. The lay summary was drafted in collaboration with SUGAR to ensure that only essential information was included.
The COFI findings, particularly regarding patient-reported outcomes and satisfaction with care were highlighted by SUGAR. The lack of access to psychological treatment, the discrimination experienced and anticipated by all patient groups and requirements for more information are tangible findings which highlight the need for future improvements within mental health services, regardless of their organisation (i.e. whether functional or integrated).
The patient voice was also captured in the wealth of qualitative interviews conducted with patients – both as inpatients (61 interviews) to understand their experience of inpatient care, and within the community (188 patients). The findings highlight the key advantages and disadvantages of both systems from the patient perspective, and should be used by service providers and policy-makers when planning services (Recommendations C and D).
To celebrate and widely disseminate the lay summary, we held an interactive workshop event at the Whitechapel Gallery. The gallery was chosen as an accessible and creative space in order to engage patients, carers, researchers, clinicians and members of the public.
Impact globally
Finally, the recommendations from the COFI study extend beyond the five countries that were included in the project. By including countries with a range of different care systems, traditions of mental health care and service provision, the results are more widely applicable to other countries within Europe and beyond.
The findings of the study have been presented internationally at seminars and workshops in Australia and Far-East Asia. We have presented to international guests (from Africa, South America, the Balkan countries and representative from the WHO) within a global health workshop held in London.
Aside from the main transferable results of the project, the project also provides valuable learning material for future international multi-site trials, including the need to understand the context of each country. Our analyses highlighted the variations that occur both within and across countries, and how these would need to be considered when drawing conclusions. The use of COFI as an example of international research was presented at a keynote lecture within the European conference on Integrated Care and Assertive Outreach in Hamburg.
Main dissemination activities:
The COFI project included a comprehensive dissemination plan in order to ensure maximum impact of the project findings in optimising mental health service delivery to ultimately reduce the burden of mental disorders. Dissemination activities focused on four main activities, i) peer-reviewed scientific publications, ii) presentations to a scientific audience, iii) presentations aimed at a wider audience including clinicians and patients and iv) larger-scale conferences and workshops.
Publications
Five peer-reviewed publications have so far been published as part of the study. The publications cover the background, protocol, methodology and baseline results of the study:
Omer S, Priebe S, Giacco D. Continuity across inpatient and outpatient mental health care or specialisation of teams? A systematic review. European psychiatry : the journal of the Association of European Psychiatrists. 2015;30(2):258-70.
Giacco D, Bird VJ, McCrone P, Lorant V, Nicaise P, Pfennig A, et al. Specialised teams or personal continuity across inpatient and outpatient mental healthcare? Study protocol for a natural experiment. BMJ open. 2015;5(11):e008996.
Miglietta E, Belessiotis-Richards C, Ruggeri M, Priebe S. Scales for assessing patient satisfaction with mental health care: A systematic review. Journal of psychiatric research. 2018;100:33
Dimitri G, Giacco D, Bauer M, Bird VJ, Greenberg L, Lasalvia A, et al. Predictors of length of stay in psychiatric inpatient units: Does their effect vary across countries? European psychiatry : the journal of the Association of European Psychiatrists. 2018;48:6-12.
Bird VJ, Giacco D, Nicaise P, Pfennig A, Lasalvia A, Welbel M, et al. In-patient treatment in functional and sectorised care: patient satisfaction and length of stay. The British journal of psychiatry : the journal of mental science. 2018;212(2):81-7.
A further 15 publications have either been submitted for publication or will be submitted within the next two months. The papers cover the main findings, with at least two papers per work package associated with data collection (WP4 – 9).
Presentations to scientific audiences
The COFI project has been widely presented at academic conferences and events. This has included hosting symposia, seminars and workshops at international and national conferences, including the World Psychiatric Association Congress, and European specific conferences such as the 4th European Conference on Integrated and Assertive Outreach. Seminar presentations focused on presenting the findings from each of the work packages, whilst workshops were hosted to discuss the implications of the project with different stakeholders including policy-makers and service managers.
Presentations to a wider audience
To maximise the impact of the project in influencing service delivery and improving mental health care, national workshops were held within each of the countries. The workshops or roadshows included a basic slide set providing information about the study methods and overall results. The contents were then tailored to the setting of the workshop, to ensure that the results presented were applicable to the audience. The workshops varied in size, and were conducted within the hospitals and community mental health services included in the study, and also at local universities and academic institutions.
International workshops
Five larger scale events/ international workshops were held in the final six months of the project to develop the COFI guidelines and recommendations, and to disseminate these widely to ensure their influence on research and practice. The five events were:
1) Brussels Research Day (6th and 7th December 2017) – Hosted by partners UCL in Brussels, this event brought together academics from different disciplines, clinicians, hospital managers and policy makers to discuss the main findings, including the features of the mental health systems which support continuity (within both integrated and functional systems).
2) International global scenario planning workshop (10th April 2018) – This workshop was hosted by QMUL and involved representatives from around the world in collaboration with the NIHR Global Health Research Group on the Effectiveness of Psychosocial Interventions for Community Mental Health Care. This included researchers and clinicians from institutions in Bosnia-Herzegovina, Colombia, Egypt and across the five study countries. Representatives from different NHS Trusts were included alongside members of the World Health Organisation (WHO) and people with lived experience of using mental health services. COFI was used as a starting point for the discussion surrounding the future of global mental health care. During the workshop, arts based methods were used to engage the workshop members, and potential futures for mental health discussed.
3) Poland International Dissemination Conference (9th and 10th May 2018) – Hosted by partners IPiN within Warsaw, this event presented the key findings of the project to a wide audience of experts, academics, clinicians, managers, policy makers and representatives from the Ministry of Health including the former Minister of Health and current Deputy-Minister. The interactive programme, which was simultaneously translated into English and Polish, provided a space for discussion of the results in the context of ongoing health care reforms within Poland. The conference ended with a panel discussion including the Ministry members who outlined their vision for improvements to mental health services in Poland.
4) European workshop on the COFI Guidelines and Recommendations (14th May 2018) - hosted by QMUL in Brussels, this workshop involved the discussion and development of the COFI Guidelines and Recommendations. The meeting was attended by all teams of the COFI project, and included international experts in functional and integrated care. The meeting was chaired by members of EUCOM who represented further European countries. Following this event, the COFI guidelines and recommendations were finalised.
5) Lay summary launch and celebration (21st May 2018) – the final COFI event was hosted by QMUL at the Whitechapel Gallery. It acted as a celebration of the COFI project and focused on disseminating the results and recommendations to a lay audience. The lay summary, which was produced in collaboration with SUGAR were launched at the event, which included researchers, clinicians, patients, carers and members of the public.
Slide set for future dissemination targeted to different stakeholder groups.
To add wider dissemination of the project and to ensure dissemination continues beyond the end of the project, a slide set has been created summarising the main findings and recommendations. The slide set aims to condense the information included in this complex programme into a digestible format for use at future events. The slide set outlines the main findings and recommendations of the project and is available for a scientific audience and for a lay audience as well.
List of Websites:
http://cofi.qmul.ac.uk/
Contact details:
Study Manager: Dr Victoria Bird, Queen Mary University of London, v.j.bird@qmul.ac.uk
PI: Professor Stefan Priebe, Queen Mary University of London, s.priebe@qmul.ac.uk