Comorbidity treatment service model evaluation

Client impact

Page last updated: August 2009

This domain deals with short-term impacts on clients and includes the following sub-domains: completion of treatment, achievement of treatment goals, continuity of care/post-service planning, client self-care knowledge, and referring health professionals.

Completion of treatment
Achievement of significant treatment goals
Continuity of care/post-service planning
Client self-care knowledge
Referring health professionals

Completion of treatment

Sixteen services provided data on treatment completion rates during the previous 12 months. Four of these reported not knowing the completion rates. Six services reported completion rates of up to 50%; the remaining six noted rates between 51% and 80% (see figure 6 below for further detail). The two services with the highest completion rates are youth services. Of those with the lowest completion rates, two are residential services and two provide a community-based/outpatient service.

Treatment completion rates of up to 50% were reported by five small services and one medium-sized service, while the highest treatment completion rates were reported by a small and a large service.

Figure 6: Proportion of clients who have completed treatment as planned during the last 12 months

Text equivalent below for Figure 6: Proportion of clients who have completed treatment as planned during the last 12 months
Top of page

Text version of Figure 6

Proportion of clients who have completed treatment as planned during the last 12 months:
  • 21-30% - 4 responses
  • 31-40% - 1 response
  • 41-50% - 1 response
  • 51-60% - 1 response
  • 61-70% - 3 responses
  • 71-80% - 2 responses
  • Don't know - 4 responses
  • No response - 1

Achievement of significant treatment goals

Services were also asked about the proportion of clients that have achieved significant treatment goals during the previous 12 months (see figure 7). The reported proportions range from '20% or less' to '91–100%' and are listed in table 33 below. Those that reported achievement of significant treatment goals for less than 50% of clients are small services, while the two large services reported proportions of 80% or higher.

When comparing completing treatment as planned with achieving significant treatment goals, it appears that overall the proportion of clients achieving significant treatment goals is slightly higher than the proportion of clients completing treatment as planned.

As table 33 below shows, only five treatment services reported a proportion of treatment completion similar to that of achievement of significant goals (see bold cells).

The three services that reported not knowing rates of completion and achievement of significant goals are small AOD services. The service that reported not knowing treatment completion rates is a large combined service.

Figure 7: Proportion of clients who achieved significant treatment goals during the last 12 months

Text equivalent below for Figure 7: Proportion of clients who achieved significant treatment goals during the last 12 months
Top of page

Text version of Figure 7

Proportion of clients who achieved significant treatment goals during the last 12 months:
  • 20% or less - 1 response
  • 21-30% - 2 responses
  • 51-60% - 2 responses
  • 61-70% - 2 responses
  • 71-80% - 1 response
  • 81-90% - 3 responses
  • 91-100% - 1 response
  • Don't know - 4 responses
  • No response - 1

Table 33: Completion of treatment as planned, by achievement of significant treatment goals (by clients during last 12 months)

20% or less achieved goals21-30% achieved goals31-50% achieved goals51–60% achieved goals61–70% achieved goals71–80% achieved goals81–90% achieved goals91–100% achieved goalsDon't know proportion achieved goalsTotal
21–30% clients completed treatment
1
2
0
1
0
0
0
0
0
4
31–40% clients completed treatment
0
0
0
0
0
0
0
0
1
1
41–50% clients completed treatment
0
0
0
1
0
0
0
0
0
1
51–60% clients completed treatment
0
0
0
0
0
0
1
0
0
1
61–70% clients completed treatment
0
0
0
0
2
0
1
0
0
3
71–80% clients completed treatment
0
0
0
0
0
1
1
0
0
2
Don't know proportion completed treatment
0
0
0
0
0
0
0
1
3
4
Total
1
2
0
2
2
1
3
1
4
16
Top of page

Treatment outcomes and success

Respondents were asked how they knew whether the treatments provided are successful, and whether they use any routine outcome measures. Twelve services reported using routine validated (n=6) or nonvalidated (n=6) outcome measures, and another service advised being in the process of developing 'what would be an effective outcome measure'.

Non-validated outcome measures included 'discussions with those involved', evaluation by clinician and client whether treatment goals were achieved, or client feedback surveys. 'Follow up studies and data collection' were also mentioned. A youth service reported the following:

We review treatment and outcomes based on the therapeutic and practical plan developed with the client. Young people are at the centre of the therapeutic relationship and set and determine what they consider success. This might change weekly. There are some basic outcome measures to judge success, they include things like: Reduced number of hospitalisations; reduced or managed drug use; resolved homelessness; increased self-esteem and self sufficiency; improved understanding and skills with coping with a dual disorder; ability to participate in community life; mood stabilisation; improved physical health; return to education or training.
Validated measures included HoNOS29, BASIS 3230, 'LSP-1631, mental state exam, Mirikai32 competency checklist (self-assessment tool)', Achenbach Child Behaviour Check-List, family support scale, satisfaction survey, strengths and difficulties scale, 'KPIs ... after 90 days at close of file (clinicians fill out KPIs), BTOM33, Psycheck34, IRIS35, and self-efficacy questionnaire'. One Victorian respondent reported using the 'Significant Treatment Goals document from DHS. Treatment is deemed as successful if one significant treatment goals is met during an episode of care'.

With the exception of two services, all reported keeping data about treatment success. However, the former advised that they had anecdotal reports on treatment success. The data kept by treatment services included client feedback forms, information from client focus groups, figures on attendees who completed the program, and statistics on various client characteristics such as 'AOD, mental health, family, crime and health'.

Further, data were recorded on SIMS36, CMI37, NADA38 database, SWITCH39. Some of these include data reported to funding bodies. One respondent noted that 'Quality Assurance Officer is responsible for collecting and collating data into evaluation and reports that are disseminated each six months, reviewed by entire clinical team and external best practice committee. In the past, we have also had some university input, through grants, to look at outcomes'.

The survey asked respondents to describe the success of the treatment they provide, without giving a definition of success. As might be expected, answers varied widely, were vague, and some respondents found this question difficult to answer. Several services reported a 'good degree of success in achieving treatment goals', anecdotal evidence of treatment success, or similar response. One service emphasised the link between success and integrated, holistic treatment in the following way:

Young people that engage in our program are more likely to return to treatment when in need given the way we engage and assist this target group. Aboriginal young people feel comfortable accessing our service because we have worked hard on making sure we are culturally appropriate. The service we provide is integrated and treats the individual and their needs rather than compartmentalising their problems.Top of page

Continuity of care/post-service planning

Findings in regard to planning for discharge and the time after discharge from the service have already been reported in the section 'Processes/procedures for referral to and communication with other providers' (domain: Service system elements I). In summary, all services reported having discharge planning processes and procedures in place. The majority of services (n=15) use formal discharge plans. Five services noted that reports, letters or discharge summaries are sent to referring services, GPs, or the courts. Ten respondents stated that their discharge processes include linking clients with other relevant services.

Client self-care knowledge

Findings in regard to client self-care after discharge have been covered in the section 'Client self-management post-treatment'. Thirteen services reported that they support clients in self-management after discharge from the program/service, and three services noted that self-management or self-care skills are taught as part of the treatment. The latter includes the provision of information material about relevant services and supports, such as counselling, drop-in support, transport, and accommodation. It was also reported that relaxation techniques, CBT techniques to deal with negative thoughts, and problem-solving strategies were being taught.

The types of services available to clients after treatment include home visits, individual counselling, an eight-week post-withdrawal group, post-withdrawal support worker, family counsellor, intensive playgroup, supported accommodation, phone link, drop-in options, transport, monthly support groups, links to job preparation, and day programs. Some services provide comprehensive support after discharge from treatment.

Referring health professionals

This sub-domain includes the communication with the referring health professional. Respondents were asked whether they provide feedback to referral sources, and findings in regard to this question have been reported in the section 'Use of referral/communication/feedback for others involved in clients' care'. In response to the question, six respondents noted that they always report back to the referring professional, nine stated that they report back sometimes, and three noted that they do not report back. However, of the latter, one provides feedback to the referring professional if this is a GP, and another service provides feedback if this is part of the contract with the client. Further, feedback is provided for court mandated clients. In summary, almost all services provide either feedback routinely or in specified circumstances.

Footnotes

29 Health of the Nation Outcome Scales
30 Behaviour and Symptom Identification Scale, a consumer self-rated outcome measure
31 Life Skills Profile
32 Mirikai Therapeutic Community Program, Queensland
33 Brief Treatment Outcome Measure
34 A mental health screening instrument designed for use by clinicians who are not mental health specialists
35 Indigenous Risk Impact Screen
36 Service Information Management System
37 Mental Health Client Management Interface
38 Network of Alcohol and Other Drug Agencies
39 Victorian Government Department of Human Services database

Top of page