Encouraging Best Practice in Residential Aged Care Program: Final Evaluation Report

Executive Summary

Up to Publications

prev page| TOC |next page

The Encouraging Best Practice in Residential Aged Care (EBPRAC) program is funded by the Australian Government with the aim of improving evidence-based clinical care in government-subsidised residential aged care facilities, including those providing low-level and high-level care. The EBPRAC program represents the most comprehensive, coordinated, approach to implementing evidence-based practice in residential aged care undertaken in Australia. Previous work to implement evidence-based practice within residential aged care in Australia has been limited, generally undertaken on a small scale and within short timeframes.

The program to date has consisted of two funding rounds, with a total of 13 projects. Round 1 commenced in December 2007 and concluded in December 2009, Round 2 commenced in December 2008 and concluded in December 2010. Each project required the support of management in participating facilities and focused on improving resident care by taking into account gaps in current care practices. The objectives of the program included improvements for residents, improvements in clinical care, improvements for staff, improvements in the system of residential aged care and increased consumer confidence. The program sought to take account of resident preferences, communicate changes required by the projects to residents and adopt a multidisciplinary approach.

In total, Round 1 and Round 2 involved residential aged care facilities in 108 locations in all states of Australia. Projects consisted of a lead organisation working with a group of facilities to implement evidence in one of nine areas of clinical practice – pain management, falls prevention, prn medications, oral health, nutrition and hydration, behaviour management, palliative care, wound management and infection control. Lead organisations included five universities, three research centres, three divisions of general practice and two service providers.

Each project was funded for two years and included a project-level evaluation, at the core of which was a ‘before and after’ design i.e. measuring a series of variables before implementation commenced and then measuring the same variables after implementation of the evidence. In the absence of control groups, there is a need for some caution in interpreting the results. For example, any judgements about impact on clinical care will be subject to the problem of attributing changes to what was done as part of each project, rather than other factors. Many activities were undertaken both to change practices and to collect data for an evaluation. The extent of data collection for the project-level evaluations was extensive. On average, 18% of project budgets were devoted to evaluation but this was skewed by three projects with much higher amounts spent on evaluation. The median amount spent on evaluation was 15%.

The evaluation of the program, as distinct from the evaluation of individual projects, was based on a framework to examine the delivery and impact of the program on residents, providers and the residential aged care system. The design of the evaluation was informed by a review of the literature which identified eight ‘key success factors’ that may influence the uptake and continued use of evidence. Data was collected for the program evaluation from interviews, six-monthly project progress reports, a tool to measure sustainability, visits to lead organisations and an economic evaluation which consisted of two questionnaires and a spreadsheet to collect data on project inputs, costs and project outputs.
Top of page
The implementation strategies adopted across the 13 projects were wide-ranging and consistent with what is found in the literature on evidence-based practice, including many different approaches to education. All projects adopted a multi-faceted approach to change, which is recognised as more effective than reliance on single strategies. The rationale for projects selecting the implementation strategies they used was underpinned by a mix of evidence, previous experience and available expertise.

Implementation of the program generally proceeded as planned. Some delays with implementation were experienced, usually during the initial ‘establishment’ phase because of under-estimating how long it would take to undertake some activities. In some projects full implementation in all facilities was not achieved. Changes in project scope usually involved an increase in scope, particularly regarding the development and delivery of education programs. An average of $59,000 was spent on implementation costs for each facility, including project salary costs, payments to participating facilities, travel costs and other operating expenses.

Some projects did not just implement ‘evidence’ but also added to the available evidence. One of the challenges for the future of EBPRAC is how to incorporate the dynamic nature of ‘evidence’ into ongoing work to maintain and improve evidence-based practice. There is scope for greater coordination to avoid duplication, facilitate consistency in the production of evidence, share knowledge about how best to implement evidence-based practice and link the various resources that are currently available.

Changes to the care received by residents were diverse. Many of the changes built on work that had been done previously in participating facilities and were relatively small scale and incremental in nature. In part, this reflects the focus of the program and the available evidence but is also indicative of the somewhat limited capacity of the sector to change. The capacity to change is dependent on the availability of resources, including the knowledge and skills of staff, the nature of daily work and the influence that a wide range of factors that are largely outside the control of those trying to bring about change can have e.g. turnover of facility managers which had a significant impact on some facilities and some projects. The nature of the changes (relatively small and difficult to measure) makes it difficult to judge the extent to which changes were implemented.

Engagement and participation of staff was extensive and generally in line with what each project expected. There was a strong emphasis on engaging the relevant stakeholders in all the projects. Facility managers had the most participation at the start of each project but were then joined by registered nurses, enrolled nurses and personal carers as the main groups participating. Participation by general practitioners was low, although this was generally anticipated.

Residents did not have a significant influence on project activities. The focus was more on keeping residents informed rather than seeking their opinion about what should happen. Various approaches were undertaken to achieve this including the use of posters, brochures, newsletters, speaking at resident meetings and media releases to local newspapers. Cognitive difficulties made communication with some residents difficult.

Involvement of residents’ families was variable, with projects tending to limit family involvement to keeping them informed via newsletters and meetings. Those families who did attend meetings were reported to have found the experience useful. The impact of the program on families was not evaluated by most projects. Those projects that did seek the views of families focused on family perceptions of how the project may have impacted on residents, rather than the family members themselves.

All projects offered facilities a financial incentive for participating in EBPRAC, with payments that ranged from $460 to over $60,000 per facility, averaging approximately $12,500 per facility. Only five facilities did not receive any payment. Facilities incurred some costs for which they were not reimbursed. The additional time spent by facility staff participating in each project does not appear to have been excessive, although this is difficult to estimate and is likely to be an under-estimate.

Almost 7,000 people were trained in the 13 projects, at a project salary cost ranging from about $100 to $300 per person trained. The full cost of training is higher, including as it does salary costs for the person attending the training and other miscellaneous costs.
Top of page
The program has resulted in the development of a significant volume of materials (education programs, tool kits, evidence summaries) which require some means of dissemination and regular updating. What has been learnt about changing practices needs to be incorporated into the daily life of facilities and the structure of the industry if it is not to be lost as ‘just another program’ that came and went.

The outcomes of the program for residents were difficult to measure. For many people residing in aged care facilities maintaining health status rather than improving health status may well be a satisfactory outcome. The three behaviour management projects produced the best evidence that resident outcomes improved. The pain management project in Round 1 had arguably the most comprehensive evidence that practices improved but was unable to show consistent reductions in pain. Two projects with a strong focus on prevention (oral health and wound management) were able to demonstrate improved outcomes (improved oral health and reduction in wounds).

Impacts on staff were mixed but generally included improvements in awareness, confidence, knowledge and skills. Staff had improved access to and use of evidence-based resources and tools. There was evidence in some projects of greater collaboration between nursing staff and personal carers, as well as with health and allied health in the planning and provision of care. Impacts on facilities included improvements to the physical environment, better access to equipment and outside services and improvements in key processes and systems of care.

Each project identified the main outcomes that it was designed to achieve over the course of the two years. Many of the intended outcomes were expressed in ways that made it difficult to determine whether the outcomes had in fact been achieved, which may have contributed to a lower rate of achievement than if the outcomes had been expressed more precisely. Projects had more success achieving intended outcomes for facilities and staff than for residents.

The seven objectives for the EBPRAC program were not well understood by projects in Round 1 but this improved in Round 2. Only one of the objectives was not well incorporated into project activities (‘Build consumer confidence in the aged care facilities involved in EBPRAC’). Four of the seven objectives were met, one was partially met and two not met. This is a good result, given that some of the objectives are ambitious or it is too early to fully assess the extent to which some objectives have been met.

An important part of the EBPRAC program was a series of six national workshops, attended by members of lead organisations and participating facilities. Feedback indicated that the workshops largely met the workshop aims, assisted in understanding how individual projects fitted within the program, were a worthwhile use of time and were a useful way of promoting networking, interaction and the sharing of ideas.

Dissemination about project activities was extensive, primarily at a local level but also more broadly with presentations at state and national conferences. Over 2,200 dissemination activities were estimated to have ‘reached’ over 200,000 people.

Sustainability is probably the most challenging aspect of any program. Use of a sustainability tool to measure ten factors that have been shown to influence sustainability indicated an increased likelihood of project activities being maintained, when results at the end of each project were compared with the results at the beginning of each project. Sustainability will depend more on factors within each facility (e.g. the presence of leadership and management support), rather than what was done by each project.

The factors most consistently identified by lead organisations and staff from participating facilities as ‘key’ to successful implementation were a receptive context for change (including leadership), adequate resources and stakeholder engagement. Being able to ‘see’ the benefits of change, either for residents or staff, was an important motivator for staff to either implement or maintain a change in practice.
Top of page

prev page| TOC |next page