Comorbidity treatment service model evaluation

Discussion

Page last updated: August 2009

This section includes a summary of major findings; reflections on the significance of findings; and limitations of the evaluation. The summary of major findings highlights the elements of good practice that were found across the 17 treatment services. Recommendations are also included.

Summary of major findings
Significance of findings
Limitations of the evaluation

Summary of major findings

As reported in the literature, the complexities associated with comorbidity and methodological challenges have resulted in little agreement on what constitutes good practice in the delivery of treatment services to people with comorbid disorders. There is no consensus on the definition of comorbidity and treatment approaches. While co-occurring mental health and substance use disorders have attracted increasing attention from various levels of government and independent bodies, it is recognised in the literature as an area that still lacks a cohesive or comprehensive framework from which to address the issues of prevention, awareness, screening, assessment, treatment, and ongoing support for those with co-occurring disorders (see Appendix 2 for a summary of literature and systematic reviews for comorbidity treatment). Agreement on treatment integration might enable greater comparability of research, leading to strengthening of the evidence base, and greater understanding between service providers, leading to improved communication and coordination between providers.

Recommendation 1: That a consensus be developed in Australia on clinical, policy, and research-appropriate definitions for comorbidity and integrated treatment.

The treatment services that were evaluated reflect the diversity of service settings and approaches described in the literature. We did not expect to find identical treatment models. However, we found that the evaluated services and programs share a range of common characteristics. As noted previously, the evaluation did not focus on the performance of treatment services, but sought to identify elements of good practice. These common elements or characteristics of good practice are summarised below.
Top of page

Common elements of good practice

Element of good practice: Links/partnerships with other relevant services

The development and maintenance of links and partnerships with a diverse range of allied services to ensure specialised, coordinated treatment and continuity of care for clients is generally considered good practice (Farrell & Marshall, 2007; NSW Health, 2000b). We found this characteristic of good practice in the treatment services we evaluated.

The surveyed services reported a range of links with relevant organisations, including networking, coordinating, cooperating and collaborating relationships. Networking links were most commonly reported. However, coordination, cooperation, and collaboration were equally frequent. The most common collaborating relationships were those with AOD and MH treatment services, GPs, housing, and criminal justice. Medium-sized services reported the largest number of links with other organisations. Mostly, the work involved in linking and developing partnerships is not costed.

Most respondents suggested that barriers to inter-agency communication and treatment coordination are due to a lack of resources. It was reported to be common for links between child, adolescent, and adult services to be in place.

Recommendation 2: That partnership work continue to be promoted as an element of good practice and be made explicit in funding arrangements.

Element of good practice: Guiding principles of harm reduction and flexible and holistic approach

As we expected, treatment services differ in regard to their philosophies and guiding principles, though harm reduction or minimisation were noted most commonly. Working in a way that is holistic, flexible, and/or client centred was reported by the majority of services, while providing services that are 'evidencebased' or 'best practice' were mentioned only by two services. However, this was an open question and respondents may not have thought of evidence-based practice as a program philosophy, but rather a principle of professional practice.

Element of good practice: Processes/procedures/protocols for intake and comorbidity screening, client privacy, discharge, and client feedback

All 16 services that described their intake processes have mechanisms in place for intake and comorbidity screening; the latter either already being used or in the process of being developed. Screening for comorbidity occurs mostly during initial client assessment. While all services reported having intake mechanisms in place, these vary considerably. Due to the diversity of surveyed services in size and type, staffing and resource levels, we did not expect to find similar intake processes. Similarly, screening processes for comorbidity vary, with either validated tools or a combination of validated and purpose-built screening tools being used.

Recommendation 3: That following the development of an agreed definition of comorbidity, agreement on the use and type of screening tools be reached.

All surveyed services take client privacy very seriously, with respondents reporting that clear policies and procedures are in place and are being followed. This is important as research found that consumers and frontline workers sometimes do not support integrated treatment, fearing that increased communication and coordination between services would violate clients' right to privacy (e.g. Burnam & Watkins, 2006; Calderwood & Christie, 2008). Therefore, having clear and transparent policies and procedures relating to privacy, training staff in these, and communicating them to clients is good practice.

All surveyed services reported having discharge processes and procedures in place. Mostly, formal discharge plans are used, which sometimes are part of a client's overall care plan. As part of discharge planning, most services link clients with other relevant services.

All respondents reported having formal or informal processes for client feedback in place, encompassing a range of ways for clients to provide feedback and/or be involved on consumer committees or in the planning and development of programs.

Having clear policies and procedures in place is generally considered good practice. We found that the surveyed services have policies and procedures for intake, comorbidity screening, client privacy, and client feedback. As expected, the content of these policies and procedures varies. Their important feature is that they are in place and being implemented, and that they are transparent and communicated to clients.

Recommendation 4: That treatment services develop and implement clear and transparent policies and procedures for intake, comorbidity screening, client privacy, and client feedback. That staff are trained in these policies and procedures and that they are communicated to clients.
Top of page

Element of good practice: Active service promotion and referrals from a range of sources

Surveyed services promote their programs in a range of different ways and receive referrals from various sources. The most commonly nominated referral sources were self-referral and the criminal justice system.

Element of good practice: Well qualified staff and generous provision of supervision and professional training

Training is an important element of good practice service delivery. For example, research studies found that professionals who hold negative attitudes towards substance users often overlook substance misuse and fail to initiate or refer to appropriate further treatment (Chappel & Veach, 1987; Howard & Chung, 2000; Richmond & Foster, 2003).

Acknowledging the importance of training, the National Comorbidity Initiative has been working to increase the knowledge and skills of workers in the field of comorbidity about different approaches and understanding of comorbidity through initiatives such as DoHA's funding of clinical supervision of psychologists and social worker placements in AOD non-government organisation (NGO) treatment settings, and initiatives to improve clinical expertise in comorbidity and increase networking and communication between GPs, mental health and AOD NGOs.

In agreement with the literature, our survey found well qualified staff and high levels of professional development in the surveyed treatment services. The proportion of staff who have completed the required qualifications is very high. Further, many staff have a wide range of additional qualifications. This contrasts with reports in the literature, which suggest that staff in AOD services are less formally qualified than those in MH services.

Recommendation 5: That the importance of formally qualified staff continues to be promoted and supported.

We found that staff are trained in identification and treatment of clients with comorbid problems. Of the 17 surveyed services, 16 reported that staff are required to undertake continuing professional development. The training opportunities available included study leave, in-house training, workshops, seminars, forums, conferences, mentoring, clinical supervision and induction/orientation. With one exception, services reported that some or all of their staff had received training in identification and treatment of clients with comorbid problems during the previous 12 months. However, the type of training varied.

Recommendation 6: That training in the screening and identification of comorbidity be considered good practice for all AOD and MH services.

The majority of services noted that all or most staff are trained in relevant referral procedures.

Staff in surveyed organisations receive regular supervision, and in particular medium and large services tend to offer more than one type of supervision. Most clinical staff receive one to two hours of supervision per month. However, six services reported monthly supervision of between three and 15 hours' duration.

Most services reported that during the previous 12 months staff had received more than five days of professional training. The surveyed services have a good staff retention rate (mean 4.1 and median of 3.0 years for clinicians; 4.5 and 5.0 years for management). No substantial differences were found in regard to service size and type.

Our findings support the emphasis in the National Comorbidity Initiative which aimed to increase awareness and understanding of comorbidity in AOD and allied health workers.

Recommendation 7: That in the AOD and MH sectors efforts on increasing awareness and understanding of comorbidity continue.
Top of page

Element of good practice: Integrated treatment and the principle of a 'no wrong door' approach

In the absence of broad consensus, the literature review identified a preference for integrated treatment compared to parallel treatment and serial/sequential treatment. Further, the no wrong door approach is a guiding principle for providers to ensure that all individuals presenting for help are given appropriate treatment services (or referral), regardless of where they enter the treatment system. This approach may be adopted from within various treatment models.

While this approach or concept has been adopted, at least in principle, by a number of governments and agencies, including in Australia, the UK and the USA (Clark, Power, Le Fauve, & Lopez, 2008; Croton, 2005; DHS, 2007b; SAMHSA, 2002), the concept was not well-described in the literature and evidence of its effectiveness was not available. This approach seeks to ensure that all presenting clients are provided with appropriate treatment services, consistent with their treatment needs, regardless of where they enter into the treatment system (Gil-Rivas & Grella, 2005). A high level of coordination between services, both within and between the AOD and MH sectors, is needed to prevent lapsing into a 'wrong door' approach. Currently, for many service providers, the no wrong door approach may represent an ideal to which they aspire.

Most surveyed services reported that they provide integrated treatment. In several services, the decision about model of service provision varies depending on client need.

One of the participating treatment services was instrumental in establishing a cross-sector partnership of AOD & MH managers and clinicians in a regional area. This group, the Eastern Hume Dual Diagnosis Group, is committed to building a no wrong door, dual diagnosis capable service system. The approach of this group was described in the following way:

Specialist mental health and alcohol and other drug services establish effective partnerships and agreed mechanisms that support integrated assessment, treatment and recovery and ensure 'no wrong door' to treatment and care. Develop and maintain collaborative service relationships that result in a 'no wrong door' outcome for dual diagnosis clients seeking help from either service, by agreeing on regularly monitored, as part of quality assurance, referral pathways within and between services. Establish functional relationships with other service sectors that provide acute physical health care, housing, education and employment KPI2: Percentage of services in each sector that have in place partnership agreements and protocols that define client care pathways within and between service sectors (100% by Dec 2008).
Resources developed by this group can be found online at the dual diagnosis website (www.dualdiagnosis.org.au).

Recommendation 8: That a 'no wrong door' approach to the treatment of people with comorbid problems be promoted and supported.

Element of good practice: CQI programs

The majority of services reported having CQI programs in place or being introduced. The CQI providers/programs that were listed vary. Small services are less likely to participate in CQI programs.

Element of good practice: Individual treatment plans developed jointly with the client

Clients have individual treatment plans and these are developed together with clients and communicated to clients and/or carer. Sometimes, other providers and/or carers are also involved in the development of treatment plans.

Element of good practice: Feedback to referring professionals

The literature provides little guidance on good practice in regard to feedback to referring professionals. Whether or not feedback to referral sources is provided depends, for example, on program philosophy, privacy policies, and/or professional notions of client confidentiality. It could be argued that a well functioning partnership between services in the AOD and MH field requires a minimum of communication and feedback about client diagnosis, treatment(s) provided, and client progress. This may require development of and consensus on referral feedback protocols between AOD and MH organisations at a local, regional, or broader level.

Our survey found that feedback–formal or informal–is commonly provided to referring professionals. In some services, this depends on the type of provider (e.g. GPs and criminal justice system are most likely to receive feedback) and/or client consent to share information.

Recommendation 9: That feedback to referring professionals be regarded as desirable, and that the development of agreed referral protocols that include feedback be promoted and supported.
Top of page

Element of good practice: Collection of data on treatment success

Collecting and keeping data on treatment success is essential for determining treatment efficacy. However, our literature review found that the availability of relevant data is scant, variable in quality, and difficult to compare. For example, it has been argued that 'few national collections include information about treatment outcomes and it is difficult to comment on the effectiveness of services in terms of client outcomes' (Australian Institute of Health and Welfare, 2005, p. 53). Consistent with the literature, our evaluation found that the majority of services reported keeping data on treatment success. However, type of data and collection methods vary.

The diversity of service models and funding arrangements makes it difficult to achieve consistent data collections across services, and significant additional resources, tools and assistance with approaches would be required.

Recommendation 10: That the collection of data on treatment success be supported with tools and approaches.

Other findings

In some areas, few or no commonalities between surveyed services were found. This was to be expected given the diversity of services surveyed. For example, the services differed in regard to type of services provided, parent organisation and type of services provided by the parent organisation, type of disorders diagnosed in clients, and type and spread of staff roles.

While linkages with other relevant services are common, this part of the program or service is not funded in a consistent way. Some agencies reported that linking/partnership work is not funded in their service. This may reflect the range of different funding sources that were reported, although the main funding sources are Commonwealth and state and territory governments. Further, there was little consistency regarding the monitoring of treatment cost. It appears that little monitoring of treatment cost occurs apart from that required by funding bodies. We also found a diversity of arrangements for funding staff training.

The evaluation found that processes, procedures, and policies are in place for the major treatment processes. However, there are often few commonalities between these. For example, client feedback processes vary, the types of intake and comorbidity screening processes vary, treatment protocols vary, discharge processes and procedures vary, and CQI processes and programs vary. Further, a range of validated and/or purpose-built screening tools are used.

We found no commonalities in regard to frameworks used to classify comorbidity, and most services do not explicitly use a model at all. However, further probing may have revealed the existence of implicit frameworks. Some services exclude clients with particular problems from treatment (e.g. floridly psychotic clients), while others have a no wrong door approach.

Given the lack of agreement on the definition of comorbidity, it is not surprising to find that classificatory models of comorbidity were hardly used by the surveyed services.

The use of different classificatory systems for defining or diagnosing comorbidity contributes to the lack of comparability of research studies, particularly on efforts to estimate the prevalence of comorbidity (Todd et al. 2004). Further, the diversity of conceptual frameworks for comorbidity has implications for the delivery of treatment programs. Agreement on definitions to suit researchers, clinicians, and policymakers might enable greater comparability of research and understanding between service providers.

Recommendation 11: That consensus be developed on classificatory models of comorbidity.

Further, the variable prevalence of comorbidity that was reported by the surveyed services, which is also reflected in the literature, can be understood as a consequence of the lack of agreed definition of comorbidity and classificatory frameworks.

Treatment completion rates vary between services, and we found no relationship between completion of treatment and completion of significant treatment goals. Given the diversity of surveyed services and the complexity associated with comorbidity, this was to be expected. Overall, the description of 'successful treatment' varied, and consequently services appeared to have only vague notions of how successful treatments have been. Further, wide variations were found in regard to data on treatment outcomes and success. While all surveyed services keep data about treatment success, the type of data and collection methods vary. It is likely that this reflects the range of funding sources and required data collection for each funding source.

While staff in surveyed services are well-trained overall, the types of training and qualifications in regard to comorbidity vary. The majority of services reported not engaging volunteers. However, one mediumsized combined AOD/MH service reported having more volunteers than employed staff.
Top of page

Significance of findings

The scope of this project has, in part, been shaped by preceding projects undertaken through both the National Comorbidity Project and the National Comorbidity Initiative. The reports and monographs previously published under the umbrellas of the Project and the Initiative provide an overview of comorbidity in Australia, which includes reviews of diagnostic and screening instruments (Dawe, Loxton, Hides, Kavanagh, & Mattick, 2002); epidemiology, prevention and treatment options (Teesson & Proudfood, 2003); treatment of people in primary (McCabe & Holmwood, 2003) and tertiary care settings (Siggins Miller Consultants, 2003); the state of relevant data collections relating to comorbidity (AIHW, 2005); and the barriers and incentives for treatment (Holt et al., 2007). A gap in the literature was identified around service delivery models and outcomes for people with comorbidity (AIHW, 2005, p. 52). This project sought to address this gap but not duplicate existing work.

The notion of comorbidity as a 'wicked problem' has been suggested in the literature (Australian Public Service Commission, 2007; Kemm, 2006). Wicked problems are highly complex problems that go beyond the capacity of any one organisation to respond to, and there is disagreement about the best way to tackle them. Wicked problems require holistic thinking, multiple stakeholders, and collaborative approaches. The evaluation found that the surveyed services are well-connected to other relevant services and that collaboration between services is common, in particular between AOD and MH services. Thus, surveyed services meet the requirement for collaboration in tackling wicked problems.

Due to the complexity of wicked problems, solutions will vary. Variability in treatment approaches across the 17 treatment services was expected. However, there are common underpinning principles, such as the importance of well-trained staff and procedures and tools to identify comorbidity.

Limitations of the evaluation

This evaluation of 17 comorbidity treatment service models had a focus on the service structure and diagnostic and treatment methods. The focus was not on the detail of methods of assessment, diagnosis, interventions, and client outcomes, unless these are related to the type of treatment service model. All 17 services were perceived as good models of service provision to people with comorbid disorders. While these services represent the diversity of different types of treatment models quite well, this made it difficult to find commonalities in some areas, such as the content of protocols and policies.

This evaluation is based on self-report, and it could be argued that respondents were likely to provide the best possible presentation of their service. However, there were upfront responses and less than positive reflections that suggest respondents provided a frank assessment of their service.

The diversity of treatment service models and small number of services surveyed made it difficult to group the sample into sub-categories and arrive at statistically significant findings when comparing services by characteristics such as service size. Service size (small, medium, large) and service type (AOD and combined) were the only sub-categories used in the reporting of findings.

Footnotes

2 Key Performance Indicator

Top of page