Evaluation of the Bringing them home and Indigenous mental health programs
11.1 Key findings of the evaluation against the Terms of Reference
prev pageprev page| TOC |next page
Table of contents
- Acknowledgements
- Study Team
- Glossary
- Executive Summary
1. Introduction- 2. Methodology
- 3 Overview of the four programs
- 4 Literature review
- 5 Key outcomes and achievements of the programs
- 6 Key limitations of the programs
- 7 Management and administrative issues
- 8. Workforce issues
- 9. Issues related to the four programs
- 10. Likely future demand
11. Conclusions and suggested future directions- Appendix A: Stakeholders consulted
- Appendix B: Literature review
- References
11.1.1 Assess the impact of each program on its target client group(s), whether it is meeting the needs of Indigenous people affected by past Government policies of forced removal (including those identified in the Bringing Them Home Report, and the likely future demand for the services it provides
11.1.2 Examine how effectively and efficiently each program is being delivered. As part of this, assess the extent to which the programs are being delivered in ways that are consistent with the National Strategic Framework for Aboriginal and Torres Strait Islander Health 2003-2013 and the National Strategic Framework for Aboriginal and Torres Strait Islander Peoples' Social and Emotional Well Being 2004-2009
11.1.3 Identify any impediments that services are experiencing in delivering the programs and propose strategies for addressing these
11.1.4 Develop recommendations to inform future program objectives, directions and alignment, with a view to achieving greater synergies among the programs
11.1.5 With regard to SEWB RCs, assess their capacity to provide services to personnel who are widely dispersed within their state. Investigate and comment on whether the current locational and organisational arrangements are appropriate
11.1.6 Develop recommendations on strategies for strengthening coordination and collaboration among service providers at local and regional levels, with a view to achieving a more integrated, client-focussed service for Indigenous people affected by past Government policies of forced removal
11.1.7 Identify strategies for engaging other stakeholders who have a role to play in meeting the needs of this group. In particular, identify other Commonwealth and State Government programs that address the needs of this target group, and provide advice on how the BTH and Indigenous Mental Health Programs should relate to these
11.1.8 Identify best practice models and possible alternative service delivery models for consideration
11.1.9 Develop recommendations to improve reporting and accountability Top of page
11.1.1 Assess the impact of each program on its target client group(s), whether it is meeting the needs of Indigenous people affected by past Government policies of forced removal (including those identified in the Bringing Them Home Report, and the likely future demand for the services it provides
The four programs have had a number of positive impacts including:- The Link-Up and BTH Programs have provided services to a large number of Aboriginal clients.
- The Link-Up, BTH and SEWB RC Programs have provided services to people who in most instances would not have received services otherwise.
- The programs have generally provided services in a culturally appropriate manner.
- There is a generally high level of client satisfaction and positive outcomes for clients of the Link-Up, BTH and Mental Health Programs.
The BTH Program has also benefited the general Aboriginal community by providing generic Aboriginal SEWB services, even though this is beyond the program's intended target group.
The Link-Up, BTH and SEWB RC Programs have had much greater impact on the client groups in their immediate vicinity or within the boundaries of the ACCHS (in the case of BTH services), since the programs have often focused primarily or disproportionately on these groups due to limited resources. Their impact has been much weaker for clients located further away. This impediment is addressed by the strategies set out below.
The SEWB RCs have had some positive impact in meeting the professional development needs of the Aboriginal SEWB workforce, but have not reached their full potential at this stage (see further discussion below).
Future demand for each of the programs is likely to at least stay the same, or possibly to increase. Some key factors which could potentially increase the demand include more proactive marketing of the programs and targeting first generation members in particular, and BTH organisations providing services to meet the needs of a wider range of Stolen Generations members. Increasing public attention to Stolen Generations issues could also have an impact in some jurisdictions. However, demand for BTH services could potentially decrease if there were other general Aboriginal SEWB services available, and the services were better directed towards the intended target group of Stolen Generations members. Top of page
11.1.2 Examine how effectively and efficiently each program is being delivered. As part of this, assess the extent to which the programs are being delivered in ways that are consistent with the National Strategic Framework for Aboriginal and Torres Strait Islander Health 2003-2013 and the National Strategic Framework for Aboriginal and Torres Strait Islander Peoples' Social and Emotional Well Being 2004-2009
The data collected for this evaluation indicates that all four programs evaluated here are being delivered in a somewhat efficient and effective manner, in that they have achieved some significant outcomes (outlined below). However, there is considerable scope for improving the effectiveness and efficiency of all four programs.The main limitations on the efficiency and effectiveness of service delivery of the Link-Up and BTH Programs are that:
- The staff employed have variable skill and qualification levels, and in many instances inadequate access to training and professional support mechanisms to enable them to conduct their roles most effectively.
- Staff burnout and turnover adversely impacts on the efficiency of service delivery.
- An undesirably large proportion of the resources allocated to the BTH program is being expended on activities other than the core business of the program, in terms of supporting other ACCHS activities and providing general Aboriginal SEWB services.
- The two programs do not liaise together as well as they should, leading to services under the two programs performing some aspects of each other's role in some instances; this reduces the effectiveness of program delivery because it: diverts resources from the intended function of the programs, means staff are conducting work for which they may not be skilled/qualified; and contributes to staff burnout.
- The programs have more limited geographical coverage than intended, due to services concentrating inordinately on clients closer to the service delivery outlet.
- Most do little to proactively target and shape the nature of the client group they serve, and this approach does not work effectively for clients who should be the highest priority target group for the services (ie first generation members).
- There is a lack of national consistency in service delivery between services under both programs.
- they are focusing primarily on only one of the four core objectives and giving insufficient attention to their other three key roles (see further discussion below)
- they are not adequately servicing Aboriginal SEWB workers in areas further away from the Centres
- they are not inviting all Link-Up and BTH services to participate in their programs
- there is variable understanding, and lack of guidance, about their role. Top of page
- long waiting lists, and are not always able to respond to Stolen Generations members who approach the service
- limited physical access due to transport and other difficulties, and limited provision of outreach services
- limited capacity to respond to clients' full range of needs.
- cultural respect
- recognising and promoting Aboriginal concepts of holistic healing
- promoting community control of primary health care services
- responding to grief, loss, anger and Stolen Generations issues.
- effective coordination of services with other agencies and planning processes
- ensuring staff with appropriate skills are recruited, retained and supported through ongoing training
- collecting, analysing and publishing data to evaluate programs in a way that enables comparison across jurisdictions and use of data to improve service delivery.
11.1.3 Identify any impediments that services are experiencing in delivering the programs and propose strategies for addressing these
There are three major impediments to effective delivery of the Link-Up and BTH programs:- The lack of general SEWB services in the Aboriginal community. The considerable unmet need for such services makes it difficult for specialised, targeted services such as the BTH Program to focus on their intended core business ie Stolen Generations members. This is the most critical impediment identified for the BTH Program, and is addressed by the strategies set out below.
- Variable skills and qualifications of staff, compounded by variable access to training and professional support and supervision. This impediment to delivery of the BTH and Link-Up Programs is addressed by the strategies outlined below.
- The lack of national consistency in service delivery, flowing from major variability in the understanding of and implementation of the programs, and a lack of national guidelines. This impediment to all four programs is addressed by the strategies set out in the following section.
11.1.4 Develop recommendations to inform future program objectives, directions and alignment, with a view to achieving greater synergies among the programs
These are provided further in the following sections of this chapter.No changes are required to the program objectives, other than that there could be benefits in requiring BTH services to conduct at least some group activities. Top of page
11.1.5 With regard to SEWB RCs, assess their capacity to provide services to personnel who are widely dispersed within their state. Investigate and comment on whether the current locational and organisational arrangements are appropriate
To date the performance by SEWB RCs of their roles has been very variable. Most have focused on only one of their four core roles (curriculum development and training), and have not given enough attention to their other three roles (training needs assessments, provision of support to the health workforce in terms of professional supervision etc and development of cross-sector linkages).Overall, most SEWB RCs have only had very limited success in providing services to personnel who are widely dispersed within their State. Workers who are further away from the location where the SEWB RC is physically based are less likely to have their training and professional support needs met by the Centre. Exceptions include the Adelaide and Melbourne SEWB RCs, which have been more successful in this area, through strategies such as provision of training on an outreach basis to regional areas.
In terms of the current locational and organisational arrangements, it would be preferable to require all SEWB RCs to be Registered Training Organisations (RTOs).
The SEWB RCs also vary in whether they intended to cover the whole or only part of the State/Territory. The results of this evaluation indicate that it is harder for SEWB RCs to effectively cover larger areas such as the whole of the State. However, the fact that the Adelaide and Victorian SEWB RCs have done so demonstrates that this can be achieved. The most critical factors for success appear to be:
- SEWB RCs being clear about their roles and responsibilities
- effective planning and coordination in advance each year so that workers have plenty of notice of training opportunities
- attention by the Centres to developing and implementing strategies to effectively meet the needs of workers in areas further away from the Centre eg regularly conducting training on an outreach basis
- exploring alternative models for this such as videoconferencing, teleconferencing, web-based models
11.1.6 Develop recommendations on strategies for strengthening coordination and collaboration among service providers at local and regional levels, with a view to achieving a more integrated, client-focussed service for Indigenous people affected by past Government policies of forced removal
Some suggested strategies to enhance the coordination between the Link-Up and BTH services, and improve the operation of the SEWB RCs (including coordination with the Aboriginal SEWB workforce) are set out below.As noted the following section, the key alternative service delivery model involves co-locating Link-Up and BTH services (and preferably the SEWB RC where applicable) in a well-structured and governed Aboriginal-controlled service. Top of page
11.1.7 Identify strategies for engaging other stakeholders who have a role to play in meeting the needs of this group. In particular, identify other Commonwealth and State Government programs that address the needs of this target group, and provide advice on how the BTH and Indigenous Mental Health Programs should relate to these
The key Commonwealth and State organisations that address the needs of this target group in some way include both record-keeping/searching organisations and agencies providing support services to Aboriginal Australians.Record-keeping and searching organisations
One key group of agencies addressing the needs of the BTH/ Link-Up target group is Commonwealth and State record-keeping or searching organisations. These include, for instance, AIATSIS, Births, Deaths and Marriages, public records offices, the Koorie Heritage Trust and Koorie Family History Service in Victoria, immigration departments and State welfare departments (in relation to their past practices of removal). Suggestions on how to improve liaison by Link-Up with these agencies are set out below.Agencies providing support services to Aboriginal people
The other key group of agencies is Commonwealth or State agencies providing services to Aboriginal families as part of mainstream or Aboriginal-specific programs, particularly at a:- Commonwealth level, FaCSIA and DoHA
- State level, agencies with responsibility for health (including physical health, mental health, substance abuse services and hospitals), families/parenting, child welfare and justice-related programs/services, and State Privacy Commissioners and Departments of Aboriginal Affairs.
Mainstream mental health services obviously have a key role to play in meeting the mental health needs of Stolen Generations members (including in making referrals to, and receiving referrals from, BTH services). However, there is significant variation in how closely and effectively the BTH services liaise with them, and many mainstream services are not currently skilled at delivering services in a culturally appropriate manner.
OATSIH could develop national clinical guidelines which can be provided by BTH services to mainstream mental health and related services (eg substance abuse) to help facilitate closer liaison with these organisations and more culturally appropriate practices in working with Aboriginal people. A particular focus here should be on identifying and providing services to Stolen Generations members.
Similar guidelines in relation to Aboriginal drug-using clients are being developed by DoHA under the COAG initiative for use by Aboriginal and (primarily) mainstream services. Top of page
11.1.8 Identify best practice models and possible alternative service delivery models for consideration
There needs to be some caution in using the term 'best practice models' in this context since it can set an unrealistically high benchmark; it appears preferable to use the term 'good practice models'.A set of GPPs is provided below, which distils the key findings of the evaluation in relation to the good practice models identified.
The key possible alternative service delivery model identified is that there would be advantages in co-locating Link-Up and BTH services in a well-structured and governed Aboriginal-controlled service which is suitable to house both services in terms of its physical location, auspice agency etc. This would considerably enhance coordination between the two programs, which is critical to the effectiveness of both. For example, co-location of the Link-Up and BTH services (and RC) has assisted Nunkuwarrin Yunti in Adelaide to provide a greater level of support to both clients and the Aboriginal SEWB workforce. Top of page
11.1.9 Develop recommendations to improve reporting and accountability
Accountability would be improved by adding some further items to the current annual reporting and contractual requirements for the four programs. These suggestions should be considered in the context of the SDRF reporting process, to ensure consistency and synergy between the two.All four programs (Link-Up and BTH services, SEWB RCs and Mental Health Service Delivery Projects)
The following contractual conditions could be added:- compliance with the national guidelines developed for the program
- a specified level of outreach work must be conducted
- that any unspent program funds (eg due to staff vacancies) can be spent on other activities to directly support the program (and be reported to OATSIH).
- conducting evaluation activities on at least an annual basis, including collecting and reporting on data in a manner consistent with the Evaluation Framework and a supporting manual developed for each program
- back-filling of positions to cover workers attending training
- travel and other expenses associated with conduct of outreach work.
Link-Up and BTH services
The following reporting items could be added to the BTH Questionnaire:- The Stolen Generations status of clients, including whether they are first, second, third, fourth etc generation, or do not have a Stolen Generations history at all.
- A breakdown of client contacts according to whether they are 'active' or 'inactive' clients.
- Clients to whom services were provided in groups.
- The amount, level or range of training completed by each worker (against a standard system for counting this).
- The frequency and source of professional supervision and debriefing provided by a qualified mental health professional to staff, on either a group or one-to-one basis.
- The nature and extent of liaison with Stolen Generations organisations to inform the work of the service.
- For Link-Up services, the number of clients offered a referral to a BTH Counsellor, and the number who accepted this offer; and for BTH services, the number of clients referred from Link-Up or who are Link-Up clients. Top of page
- Reporting against targets developed by the service for activities to identify, appeal to and prioritise the needs of first generation Stolen Generations members.
- Minimum skill/qualification levels for workers and/or the skill/qualification mix that must be satisfied for the team as a whole.
- The maximum proportion of funds allocated to each worker that can be retained by the auspice organisation to cover management/administrative costs.
- All workers to undertake a specified minimum amount of training annually.
- The services must regularly liaise with any relevant Stolen Generations groups locally, or at a State/national level (to inform their work).
- The services must comply with a standard national protocol to be developed for referral between Link-Up and BTH services, and develop BTH/Link-Up protocols with the services they should be liaising with on a regular basis.
BTH services
A separate funding line item could be added to cover supervision by an external mental health professional, where such a person is not available within the auspice organisation.SEWB RCs
Performance measures should be established for the Centres, and the Centres be required to report against them as part of their contractual requirements. In addition, the following contractual conditions could be added:- The SEWB RC must be an RTO, and maintain this status for the duration of the contract. The majority of training offered must be accredited at either the national or State level.
- Training needs assessment of the Aboriginal SEWB workforce in the area covered by the SEWB RC must be conducted at least once a year. Priority should be given to staff with OATSIH-funded services.
- All SEWB RCs to ensure that they invite all Link-Up and BTH services (and Stolen Generations organisations where relevant and appropriate) to participate in all training activities organised.