Comorbidity treatment service model evaluation

Findings

Page last updated: August 2009

This report presents findings from the Comorbidity Treatment Service Model Evaluation survey. These are organised according to the program logic domains and sub-domains. Pertinent findings of the literature review are included in the relevant sections.

Staff, particularly those in larger services, indicated that some numbers are estimates. However, the survey responses provided were not clear about which numbers are actual and which are estimates.

Context
Inputs
Service system elements I (policies and procedures)
Service system elements II
Client impact
Outcomes

Context

The first program logic domain is concerned with the context in which treatment services operate. This section covers the following sub-domains: location of services and their catchment areas; density of the service system and workforce; and state regulatory or other issues with funding sources.

Location of services and catchment areas

Seventeen treatment services participated in the evaluation. Six of these are located in Victoria; two each in New South Wales, the Northern Territory, South Australia, and Tasmania; and one each in Queensland and Western Australia. One service covers NSW and the ACT.

The catchment areas of the surveyed services ranged from local government areas to one organisation with services based in two states and occasionally servicing clients in additional states. The majority of services cover a region (n=5) or a state/territory (n=6). One service covers rural and remote communities.

Density of service system and workforce

This section explores types of service links and partnerships; links between child, adolescent, and adult services; effective links and partnerships; costing of links and partnerships; barriers to inter-agency communication; and barriers to treatment.

Respondents reported whether they had a networking, coordinating, cooperating or collaborating relationship with a range of relevant services. Unsurprisingly, networking links were most commonly reported. However, the remaining three types of link were equally frequent. Medium-sized services were found to have the largest number of links with other services. In contrast with the literature review, which found that collaboration between AOD and MH sectors is very poor, the surveyed services appear to be well-linked with other relevant services. The links most commonly cited as being most effective were those between MH and AOD services, including links between an AOD and a MH service, as well as the links between one type of AOD service and other types of AOD services. Further, the majority of services reported relevant links with child, adolescent, and adult services. Most services reported not costing the work involved in building links and partnerships.

The most commonly reported barriers to inter-agency communication and treatment coordination included resource issues and differences in culture and ideology. Frequently listed barriers for clients were those relating to language and culture, waiting lists and limited hours of service, and transport (particularly in rural and remote areas).

State regulatory or other issues with funding sources

The main issues listed in regard to state regulatory requirements or other issues with funding sources as impacting on services included insufficient funding for infrastructure and a broad range of services, over-reporting, and duplication of services.
Top of page

Inputs

The sub-domains for the 'inputs' domain include service structure, funding sources and issues, workforce, and service promotion.

Service structure

Of the 17 treatment services surveyed, 15 belong to the non-government or private-not-for profit sector, while two are government organisations. Nobody described their service or program as a mental health service. Eleven were described as AOD services; three as combined MH and AOD services; and three were described as 'other'. The latter includes an ethnic community centre that provides various social welfare services; capacity building services to regional clinical mental health, psychiatric disability support and AOD services; and a youth health service with a focus on AOD, MH, and sexual health.

Of the 17 services, two are Indigenous services, and one service has a large proportion of Indigenous clients. Five services cater only for adolescents/young people, and nine services provide a residential program.

Taking into account the number of employed clinicians and other professional staff, services were categorised into small (n=2–20), medium (n=27–80), and large (n=124–403) services. The majority of surveyed services are small. Three services are medium-sized and two are large.

We also distinguished between AOD services and services that were described either as combined AOD and MH services or 'other'. For the purposes of this report, the latter will be referred to as 'combined services'. The surveyed services comprise 11 AOD and six combined services.

The types of treatment provided by the surveyed services includes withdrawal management/detoxification, rehabilitation, pharmacotherapy, counselling, support, information provision, assessment, group work, case management, community education to schools and other community agencies, community development, assertive outreach, family support, day programs, supported accommodation, referral, secondary consultation, clinical psychology, housing support, Needle and Syringe Program, and medical and psychiatric services.

Eleven of the services that participated in the evaluation are part of a larger organisation, while six are stand-alone services. Nine provide residential services, with capacity ranging from eight to 70 beds. The average occupancy rates during the last 12 months ranged from 67% to 95%. Services reported the shortest client stay as being less than a day to two days, while the longest stays ranged between 16 days and 19 months.

Twelve of the 17 services provide a community-based/outpatient service. These employ between three and 116 staff who provide clinical services. AOD services are slightly more likely to provide community-based services than combined services.

Sixteen services identified drugs of concern reported by their clients. Of these, alcohol, cannabis, and amphetamines were the most commonly nominated. Other drugs of concern included heroin, benzodiazepines, ecstasy, nicotine, methadone, solvents, and Subutex (prescribed and non-prescribed, injecting).

Services that cater for clients with a mental illness were asked about the disorders diagnosed in their clients. Thirteen agencies provided information about diagnosed disorders. The most common disorders include anxiety disorder, mood disorder, personality disorders, and substance use disorders.

Fifteen services provided data on the prevalence of comorbidity in their clients. Percentages ranged from 35 to 100. In regard to service type, combined services reported comorbidity of 75% or higher, while six out of the 10 AOD services reported proportions below 75%. The range of comorbidity in clients reported by AOD services was between 35% and 86%. Our literature review found that 50%–75% of AOD clients are reported to also have a MH problem.

The survey asked respondents to describe the philosophy or guiding principles of their programs. The most commonly suggested principle was harm reduction/harm minimisation.

Survey respondents were further asked whether they use a classificatory framework or model for comorbidity. Three reported that they use such a model, 11 advised that they did not use a model, and three were unsure about what the question meant. The models in use are not consistent, reflecting the diversity of frameworks for assessing or categorising comorbidity noted in the literature review.

All respondents reported having formal or informal processes for client feedback.
Top of page

Funding sources and issues

Twelve services provided information about funding sources. Unsurprisingly, services reported that the largest part of their funding is provided by Commonwealth and state/territory governments. Of the four services that listed self-generated funding, three are part of large parent organisations, while one is a small Indigenous organisation. Philanthropic trusts and other non-government funding bodies contribute to a very small extent.

Workforce

Respondents reported the following staff roles: AOD worker; counsellor; GP; MH nurse; nurse; psychiatrist; social worker; administration staff; manager; domestic staff; gardener; volunteer; and 'other professional'. As expected, medium and large services employ a greater range of staff compared to small services, although this is not true for all small services. The majority do not engage volunteers. However, one medium-sized combined AOD/MH service reported having more volunteers than employed staff.

Another survey question asked respondents to indicate the proportions of the following roles in their service: administrative; clinical; management; other support staff; and other roles. Overall, large and medium-sized services appear to have a smaller proportion of management roles. However, this is not true for all small services, and two small services did not list management roles at all.

All staff in seven services have the required minimum qualifications. Overall, the proportion of staff who completed the required qualifications was high. Almost all staff who were lacking the required qualifications were in the process of completing these.

Twelve services reported that their staff have additional qualifications. The additional qualifications listed include tertiary qualifications in psychology, social work, counselling, youth work, law, and business administration; certificates and diplomas in AOD, MH, motivational interviewing, and frontline management. Further, respondents reported training in a range of psychotherapies (e.g. narrative therapy, cognitive behavioural therapy), first aid, child protection, managing challenging behaviours, cultural awareness, and dual diagnosis training.

Overall, our survey found no substantial differences in the required minimum qualification for AOD workers and counsellors in AOD and combined services. This, and the overall high level of qualified staff, contrasts with findings from the literature review.

The survey found that the median number of years that (employed) staff have worked for the service ranged from three years for clinical staff to five years for staff in managerial roles. Surveyed services appear to have stable staffing which, according to the literature, has a positive impact on the quality of treatment.

Nine of the 17 services provided opportunities for volunteering, although two of these had no volunteers at the time of the survey. The nature of the volunteering included participation in committees of management or program steering committees, support work with clients, driving clients to appointments, work in opportunity shops, and administrative duties. The contribution made by volunteers ranged from less than one to 400 hours per week. The latter comprised mostly work in opportunity shops.

Service promotion

The main strategies for service promotion include participation in relevant networks, flyers, information stalls, and websites. Media releases, advertisements in local print media and newsletters were less common. Further strategies included word of mouth and promotion through other services, magazines, emails, the headspace initiative, community education and health promotion, and education of generalist services.

Surveyed services reported receiving referrals from different sources. Seven services selected all referral sources listed in the survey. There were no differences in the number of referral sources in regard to service size and type. The most commonly nominated referral sources were self-referral and the criminal justice system.
Top of page

Service system elements I (policies and procedures)

The service system elements I domain has a focus on policies and procedures. It comprises five subdomains: intake processes and screening for comorbidity; clear treatment protocols; processes/procedures for referral to and communication with other providers; staff training in service procedures; and staff training, skills and supervision in assessment, treatment and specialisation.

Intake processes and screening for comorbidity

Respondents were asked to describe their intake processes and 16 services responded to this question. Overall, all 16 services have intake processes; however, these vary considerably. Although not all respondents noted which staff are involved in intake processes, the following staff roles were referred to: counsellors; GP; practice nurse; clinical director; welfare director; case worker; intake workers; assessment counsellor; and direct care workers.

All surveyed services reported screening mechanisms for comorbidity, either currently being used or in the process of being developed. Most surveyed services screen clients for comorbidity, with 13 services screening all or most clients. At the time of the survey, two small services did not screen clients for comorbidity (one AOD service and one combined service). Mostly, screening for comorbidity occurs during the initial assessment.

Of those services that use screening tools, 10 used either validated tools or a combination of validated and purpose-built screening tools. Five used purpose-built screening tools. An Indigenous service that used validated tools reported also trying to validate other tools, as existing tools have not been validated for the Indigenous population.

The validated tools that were used by surveyed services included Kessler-10 (K-10), PsyCheck, Beck Depression Scale, Obsessive-Compulsive Disorder (OCD) screening (Yale-Brown Obsessive Compulsive Scale [Y-BOCS]) and Mood Disorder questionnaire, Brief Symptom Inventory (BSI), bipolar assessment, Indigenous Risk Impact Screen (IRIS), Disbility Assessment Schedule (DAS), Mental Health Screening Form–III (MHSF-III), Alcohol, Smoking and Substance Involvement Screening Test (ASSIST), HEADDSS assessment1, Mini International Neuropsychiatric Interview (MINI), and Hamilton Rating for Depression (validated by Brown's review 2003).

Treatment exclusion due to the presence of co-occurring disorders has been reported in the literature. Of the treatment services we surveyed, seven reported that clients with certain types of comorbidity are excluded from treatment. These include four small AOD services, one small and one medium-sized combined service, and a medium-sized AOD service. Exclusion criteria refer to, for example, florid psychosis, unstable clients, acute mental health issues, 'clients who become catatonic', clients at risk of harming themselves and/or others, and 'heavily medicated clients where medication would not allow the client to cope with the structure of the program. Clients with serious mental health issues that would not allow them to cope with a community-based setting with high level of social contact'. Individual treatment plans usually reflect that comorbidity has been identified and the client accepted for treatment.

Clear treatment protocols

Nine services reported having a range of treatment protocols and/or using guidelines and manuals. When asked about how services deal with client privacy issues, all respondents referred to existing legislation and/or their organisation's privacy policies and procedures.

Processes/procedures for referral to and communication with other providers

The majority of services reported using formal discharge plans, which are sometimes part of a client's overall care plan. Five services stated that reports, letters, or discharge summaries are sent to referring services, GPs, or the courts. Most respondents indicated that their discharge processes include linking clients with other relevant services.

Staff training in service procedures

The majority of services noted that all or most staff are trained in relevant referral procedures. Two small AOD services suggested that staff do not need to be trained in referral procedures, while one small AOD and one small combined service stated that staff should be, but were not, trained in referral procedures.

Staff training, skills and supervision in assessment, treatment and specialisation

Of the 17 services surveyed, 16 noted that staff are required to undertake continuing professional development. The training opportunities available included study leave, in-house training, workshops, seminars, forums, conferences, mentoring, clinical supervision, and induction/orientation.

Most services reported that during the previous 12 months staff had received more than five days of professional training. Overall, treatment services are generous in the allocation of professional training for their staff.

With one exception, services reported that some or all of their staff had received training in identification and treatment of clients with comorbid problems. Respondents listed a broad range of training, including workshops (e.g. on specific mental health issues such as depression, anxiety, self-harming), courses at TAFE and university level, and Network of Alcohol and Other Drug Agencies (NADA) dual diagnosis training (in NSW). Five services noted that all staff, or all clinical staff, had received such training.

All surveyed services reported providing supervision to their clinical staff, and 13 provide more than one type of supervision. The majority provide supervision on a regular basis and from an internal and/or external supervisor, as well as regular case presentations and discussions. Compared to small services, medium-sized and large organisations are more likely to provide a larger number of different types of supervision. Most clinical staff receive one to two hours of supervision per month.

Twelve services provided figures or percentages for their training budgets. Funding for supervision ranges between $200 and $1,000 per person per year with an average of $590, or between two and five per cent of the program budget. Several respondents reported that this is often part of the program budget or funded from a range of sources.
Top of page

Service system elements II

This domain has a focus on service system elements in regard to various practices, and includes the sub-domains 'appropriate' treatment, clear care/treatment plans, use of referral/communication/feedback for others involved in clients' care, client self-management post-treatment, and treatment cost.

'Appropriate' treatment

Three general service system management models for comorbidity treatment have been described in the literature: sequential (or serial) treatment; parallel treatment; and integrated treatment. There are distinct differences between a number of the integrated models, such as client/program level integration, service/system level integration, and single-sector integration. Further, although not a service system management model, the 'no wrong door approach' is a guiding principle that can be used within a number of service delivery models.

When asked about models of service provision, the majority of services surveyed reported integrated treatment (n=12), five reported parallel treatment, and four serial/sequential treatment. More than one type of model is sometimes used, depending on client need, expertise of the clinician involved, or severity of client's mental health condition. Serial/sequential treatment was used by combined services and not at all by AOD services, while parallel treatment was used more frequently by AOD services.

Fourteen services reported that they had a continuous quality improvement (CQI) program in place, and one respondent noted that the service is in the process of choosing a quality improvement provider. The three services that reported not having a CQI program are small services.

Respondents were also asked what changes could be made to their service's treatment model and/or service structure to improve treatment outcomes. Sixteen services made suggestions, with some of these offering several ideas for improvement. The suggestions related mainly to the areas of partnerships, training, data collection, and resources.

Clear care/treatment plans

The majority of surveyed services reported that clients have individual treatment plans. Two small AOD services noted that they do not develop individual treatment plans, and another small AOD service stated that some clients have treatment plans.

Nearly all surveyed services reported that they always involve clients and sometimes other providers in the development of treatment plans. Carers may be involved if clients consent; family meetings sometimes occur. Further, respondents noted that providers are involved for court-mandated clients, and case conferences occur for complex cases where multiple services are involved.

All services that use individual treatment plans communicate these to the client and/or carer. Some agencies noted that treatment plans are negotiated with the client, or that the client decides who has access to the treatment plan. Two services also provide treatment plans to the referring GP.

Use of referral/communication/feedback for others involved in clients' care

Respondents were asked whether they provide feedback to referral sources. Six noted that they always report back, nine stated that they report back sometimes, and three noted that they do not report back.

Client self-management post-treatment

Thirteen services reported that they support clients in self-management after discharge from the program/service. Three services noted that self-management or self-care skills are taught as part of the treatment, and three residential services commented that their outreach program supports clients after discharge.

Cost

When asked, only five respondents provided details of treatment cost. Several respondents stated that they had not costed the treatments provided.

Respondents were also asked to describe the processes for monitoring cost. Monitoring of treatment delivery cost was reported to involve financial reporting (and service delivery reporting) as required by funding bodies, external financial auditing, and/or regular (monthly or quarterly) budget reports. However, five services reported that no monitoring of treatment cost occurs. (Presumably, this can be interpreted as no monitoring other than that required by funding bodies.)
Top of page

Client impact

This domain deals with short-term impacts on clients and includes the following sub-domains: completion of treatment; achievement of treatment goals; continuity of care/post-service planning; client self-care knowledge; and referring health professionals.

Completion of treatment

Sixteen services provided data on treatment completion rates during the previous 12 months. Four of these reported not knowing the completion rates. Six services reported completion rates of up to 50%; the remaining six noted rates between 51% and 80%. The two services with the highest completion rates are youth services. Of those with the lowest completion rates, two are residential services and two provide a community-based/outpatient service. Treatment completion rates of up to 50% were reported by five small services and one medium-sized service, while the highest treatment completion rates were reported by a small and a large service.

Achievement of significant treatment goals

Services were also asked about the proportion of clients who have achieved significant treatment goals during the previous 12 months. The reported proportions range from '20% or less' to '91–100%'. Those that reported achievement of significant treatment goals for less than 50% of clients are small services, while the two large services reported proportions of 80% or higher.

When comparing completing treatment as planned with achieving significant treatment goals, it appears that, overall, the proportion of clients achieving significant treatment goals is slightly higher than the proportion of clients completing treatment as planned. Only five treatment services reported a proportion of treatment completion similar to that of achievement of significant goals.

The three services that reported not knowing rates of completion and achievement of significant goals are small AOD services. The service that reported not knowing treatment completion rates is a large combined service.

Respondents were also asked how they know whether the treatments provided are successful, and whether they use any routine outcome measures. Twelve services reported using routine validated or non-validated outcome measures, and another service advised being in the process of developing 'what would be an effective outcome measure'. With the exception of two services, all reported keeping data about treatment success.

The survey also asked respondents to describe the success of the treatment they provide, without giving a definition of success. As might be expected, answers varied widely, were vague, and some respondents found this question difficult to answer. Several services reported a 'good degree of success in achieving treatment goals', anecdotal evidence of treatment success, or similar response.

Continuity of care/post-service planning

Findings in regard to planning for discharge and the time after discharge from the service have already been reported in the section 'Processes/procedures for referral to and communication with other providers' (domain: Service system elements I). In summary, all services reported having discharge planning processes and procedures in place. The majority of services use formal discharge plans. Five services noted that reports, letters or discharge summaries are sent to referring services, GPs, or the courts. Ten respondents stated that their discharge processes include linking clients with other relevant services.

Client self-care knowledge

Findings in regard to client self-care after discharge have been covered in the section 'Client self-management post-treatment' (domain: service system elements II).

Referring health professionals

This sub-domain includes communication with the referring health professional. Respondents were asked whether they provide feedback to referral sources, and findings in regard to this question have been reported in the section 'Use of referral/communication/feedback for others involved in clients' care' (domain: service system elements II).
Top of page

Outcomes

The collection of information about long-term client outcomes was outside the scope of this evaluation. Judging by the responses to survey questions about the collection of data on treatment success (see section on 'Achievement of significant treatment goals' in the 'Client impact' domain), it can be assumed that services do not routinely collect such data.

Footnotes

1 A psychosocial interview for adolescents (Home & Environment, Education & Employment, Activities, Drugs, Sexuality, Suicide/Depression)