Encouraging Best Practice in Residential Aged Care Program: Final Evaluation Report

7 - Sustainability

Up to Publications

prev page| TOC |next page

Sustainability is not well defined or well understood and can be ambiguous, with a natural tension between maintaining what is worthwhile and continuing to improve:
‘It may concern the stability of work methods, or the consistent achievement of performance goals independent of the methods deployed, and may also apply to the maintenance of a consistent trajectory of performance improvement. Maintaining methods and outcomes suggests a static view. The focus of an improvement strategy implies a more dynamic perspective’ (Buchanan, Fitzgerald et al. 2005, p 190).
Changes in work routines, patterns of behaviour, and mindsets are crucial for sustaining improvements after the end of any formal change program. Evaluation of sustainability is closely aligned with the issue of capacity building (e.g. increased capability and skills, increased resources) and any changes in structures and systems that ‘anchor’ or embed changes.

Definitions of sustainability, as applied to the EBPRAC program, focus on three main areas:

• Sustaining the benefits for clients (in this case, residents).
• Maintaining the program or its activities in an identifiable form within the facilities.
• Maintaining attention to the problem addressed by the program (Scheirer 2005).

As can be seen from Section 4.2 (impact on residents) the benefits of the EBPRAC program for residents are difficult to identify and even more difficult to quantify. Changes in the care of residents were more prevalent than improvements in resident outcomes.

As outlined elsewhere in this report (Section 2.1 and Section 3.4) changes implemented as part of the EBPRAC program include changes to the daily care of residents, changes to the ‘key processes’ of care (e.g. resident assessment, care planning) and more system level changes around changing attitudes e.g. to behaviour management or palliative care. These different changes are linked together in many ways – a change in practice will be influenced by attitudes and may be prompted by a formal assessment process or a change in attitude may be influenced by ‘seeing’ that a relatively minor change in practice is ‘working’.

Even changes to the physical environment, which might seem to have greater chance of being sustained (once the change is made it will stay there), depend on other factors and the issue of sustainability is not that simple, as pointed out in one of the project final reports:
The changes to the environment are likely to continue beyond the life of the project but will the additional amenities (e.g. extra social space, better outside space) actually be used? Again, this question can only be answered by further evaluation. (HC behav, p 47)
Each project was asked to collect data using a sustainability tool developed in the National Health Service in the UK which is designed to be used prospectively to assist project teams to gain knowledge about sustainability and incorporate this knowledge into the development of their projects (Maher, Gustafson et al. 2006). The tool uses a scoring system based on 10 factors, each with four options for scoring, grouped in three categories (Table 20). For each factor there is a maximum score, allowing for ‘gaps’ to be identified (between actual and maximum scores).

Table 20 NHS sustainability tool – categories and factors

Category	Factor
Process	Benefits beyond helping patients Credibility of the benefits Adaptability of improved process Effectiveness of the system to monitor progress
Staff	Staff involvement and training to sustain the process Staff attitudes toward sustaining the change Senior leadership engagement Clinician leadership engagement
Organisation	Fit with the organisation’s strategic aims and culture Infrastructure for sustainability

The factors match well with seven of the eight key success factors underpinning the program evaluation, the exception being the factors ‘model of change/implementation’. Projects were requested to complete the tool:

• for each residential aged care facility
• by those involved in each project who were best placed to rate the factors
• within the first two months of implementation commencing in each facility (or as soon as possible thereafter)
• within the last two months of implementation ceasing in each facility (or as soon as possible thereafter).

Data was received for 99 facilities pre-implementation and for 87 facilities post-implementation. The primary reason for the difference between these two numbers is that post-implementation data was not received from one project, two facilities merged and one facility withdrew after pre-implementation data collection. Data collection by one project was poor and has been excluded from the analysis, except for three facilities with complete data.

Figure 5 shows the combined results across all projects, comparing the average pre-implementation score and the average post-implementation score with the maximum possible score for each factor. For each factor the post-implementation score was higher than the average pre-implementation score and closer to the possible maximum, indicating a move towards greater sustainability by the end of the projects.Top of page
Figure 5 Sustainability tool – results for all factors for all facilities

The information below is a description of data provided in Figure 5 above.

	Pre	Post	Maximum score
Benefits beyond helping residents	5.9	6.8	8.7
Credibility of the evidence	7.0	7.9	9.1
Adaptability of improved process	5.8	6.3	7.0
Effectiveness of the system to monitor progress	5.2	6.1	6.8
Staff involvement and training to sustain the process	8.1	10.0	11.5
Staff behaviours toward sustaining the change	8.2	9.8	11.0
Senior leadership engagement	12.1	13.2	15.0
Clinical leadership engagement	12.2	13.7	15.0
Fit with the organisation's strategic aims and culture	5.8	6.3	7.4
Infrastructure for sustainability	6.4	8.0	9.7
	Factor score

The factor with the greatest improvement between pre and post-implementation was the factor ‘staff involvement and training to sustain the process’. The factors with the greatest potential for improvement by project end were ‘benefits beyond helping residents’, ‘infrastructure for sustainability’ and ‘senior leadership engagement’ (in that order). To give an indication of what this means in practice the explanation of the four levels for each of these factors is included in Table 21, with the aim being to move to the highest level (Level 1).Top of page

Table 21 NHS sustainability tool – areas with potential for greatest improvement

	Benefits beyond helping residents	Infrastructure for sustainability	Senior leadership engagement
Level 1	The change improves efficiency and makes jobs easier.	Staff, facilities and equipment, job descriptions, policies, procedures and communication systems are appropriate for sustaining the improved process	Organisational leaders take responsibility for efforts to sustain the change process. Staff generally share information with, and actively seek advice from, the leader.
Level 2	The change improves efficiency but does not make jobs easier.	There is an appropriate level of staff, facilities and equipment, but inadequate job descriptions, policies, procedures and communication systems for sustaining the change	Organisational leaders don’t take responsibility for efforts to sustain the change process. Staff generally share information with, and seek advice from, the leader.
Level 3	The change does not improve efficiency but does make jobs easier.	The levels of staff, facilities and equipment to sustain the change are not appropriate although job descriptions, policies, procedures and communication systems are adequate	Organisational leaders take responsibility for efforts to sustain the change process. Staff typically don’t share information with, or seek advice from, the leader.
Level 4	The change neither improves efficiency nor makes jobs easier.	The staff, facilities and equipment, job descriptions, policies and procedures and communication systems are all not appropriate for sustaining the change	Organisational leaders don’t take responsibility for efforts to sustain the change process. Staff typically do not share information with, or seek advice from, the leader.

The highest possible total score for the sustainability tool is 101. According to the authors of the tool preliminary evidence suggests that a score of 55 or higher offers reasons for optimism that sustainability will be achieved. Figure 6 shows the data for all facilities at the ‘pre’ and ‘post’ points in time. Given the small difference in the number of facilities for which data was supplied at the two points in time the data is expressed as percentage of facilities. As can be seen the percentage of facilities with a total score greater than 90 increased markedly over the course of the program. Only 11 facilities scored less than 55 at project commencement, with only 2 facilities having a total score of less than 55 at project end. One of the surprising aspects of the data is the high total scores at project commencement, suggesting that participating facilities were already favourably inclined towards achieving sustainability even early on.
Figure 6 Total sustainability scores – all facilities

The information below is a description of data provided in Figure 6 above.

Percentage of facilities
Pre

• less than 30 - n/a
• 30 to 35 - 1%
• 35 to 40 - 1%
• 40 to 45 - 2%
• 45 to 50 - 1%
• 50 to 55 - 6%
• 55 to 60 - 11%
• 60 to 65 - 2.5%
• 65 to 70 - 5.5%
• 70 to 75 - 9%
• 75 to 80 - 11%
• 80 to 85 - 14%
• 85 to 90 - 13%
• 90 to 95 - 6%
• Greater than 95 - 17%

Post

• less than 30 - 1.5%
• 30 to 35 - n/a
• 35 to 40 - n/a
• 40 to 45 - 2%
• 45 to 50 - n/a
• 50 to 55 - n/a
• 55 to 60 - 2%
• 60 to 65 - 2.5%
• 65 to 70 - 2.5%
• 70 to 75 - 7%
• 75 to 80 - 6%
• 80 to 85 - 7.5%
• 85 to 90 - 13%
• 90 to 95 - 23%
• Greater than 95 - 34.5%

The data from use of the sustainability tool indicates ‘reasons for optimism’ although experience with previous evaluations suggests that sustainability is challenging for a project-driven model of change. Many projects relied on dedicated funding for education which begs the question of how this will be maintained beyond the life of each project. The dependence on ‘champions’ and ‘link staff’ by most projects to drive improved clinical practice will require continued commitment by facility management, and the sector more broadly, if the positions are to be maintained.

The views of those involved in the projects regarding the issue of sustainability were quite mixed. Some were confident that sustainability was not a problem:
The thing that separates it from other projects that I’ve been involved in is the money was there initially to provide the education and all those things. You don’t require that finance to continue to provide the program. Once staff are educated, once you’ve got that model of care in place, it doesn’t cost anything to maintain it. It’s all about internal education. Simple little things like we’ve introduced the EBPRAC project on our orientation check lists for staff. We run sessions about it. When our best practice champion pulls up stumps and walks out of here on the last day of (month), there’s no reason why it can fall over because that person isn’t required to be … they’re not the person that’s doing the person-centred care. The staff that are on the floor, that are already rostered in the areas are the people that provide the person-centred care. The best practice champions introduced it, educated around it and monitored it as it’s gone along and when they step away it should be entrenched and that’s why I firmly believe that it’s a sustainable program. (F)
It’s part of our calendar now so it’s just normal every day therapy that they have and it’s on the calendar so it’s consistent so it causes no stress to the unit, it causes no extra, we don’t need any extra staff and it just flows into a therapy calendar. (F)
Others framed the issue of sustainability in terms of other factors such as establishing links with the funding of residential aged care, the perceived value of the changes and the ability to incorporate changes into normal practice:
We’ve embedded the three monthly assessments into the ACFI funding instrument determination which they have to do anyway. I think that sort of approach will make those assessments, you can almost guarantee that they will be done. (P)
It’s got to come down to the perceived value from the staff. If they see it making their job easier, better, then I think that’s going to almost guarantee the continuation. (P)
These projects are funded and then there’s no money to sustain it. How do you build it into your normal workplace? That’s what we have to do and that’s our struggle now. (F)
When the three behaviour management projects met at the EBPRAC workshop in February 2010 suggestions for achieving sustainability included having resource nurses to work with facilities across regions. It was not felt that the existing Dementia Behaviour Management Advisory Services could assist with this as the focus of that service is on the immediate needs of individual clients rather than working with staff in a systematic way.

When the three palliative care projects met at a workshop in Melbourne in July 2010 it was generally agreed that advance care plans, palliative care case conferences, and end-of-life care pathways could be sustained but that each was dependent on maintaining awareness and providing appropriate education. Palliative care case conferences require input from general practitioners and/or nurses with some expertise in palliative care in order to work effectively. As long as that input is available the case conferences should be sustainable. An interesting aspect of the end-of-life care pathways is that one reason staff like a pathway is that it is a ‘live’ paper-based document that is used regularly, as opposed to the standard care plans maintained by facilities which tend not to be referred to much on a day-to-day basis. With many facilities moving to electronic records it is not clear what will happen with use of paper-based end-of-life care pathways.

Comments from other projects in their final reports supported the importance of maintaining the infrastructure for sustainability and senior leadership engagement, as indicated in the results from the sustainability tool. For some projects an important issue is whether links established with outside services will be maintained. For example:

• Links between facilities and clinicians from the local community with expertise in wound management (wound management project).
• Links with local community palliative care services to provide input into case conferences and monthly reviews (palliative care project).

One important issue is that of staff turnover, with staff trained by their involvement in EBPRAC projects resigning from participating facilities, generally viewed as a ‘loss’ to what has taken place. However, some projects pointed out that many of those staff end up working elsewhere in residential aged care, a case of ‘our loss is someone else’s gain’.

The pain management project was notable for three of the participating five facilities allocating funding to employ someone to work in a dedicated pain management role, which should make a significant contribution to ongoing sustainability.

The project which achieved arguably the best and most consistent improvement in resident outcomes raised serious doubt about the sustainability of their achievements. The project was critically reliant on the use of mentors but the mentors (who gained a good knowledge of the capabilities of facility staff by their involvement with those staff) were unable to identify any facility staff with what they describe as the ‘three essential conditions required for the mentoring role’:

• The required skills and attitudes.
• The time to undertake the role.
• The distance from the day by day responsibilities and decision making that is required to enable staff members to see the mentors as a safe, neutral person able to hear their concerns without having to act (HC behav final report, p 47).

The conclusion from the project is that this combination of skills and contextual conditions is not readily available within residential aged care, thereby limiting its ability to be replicated.

One of the components of sustainability referred to earlier (maintaining attention to the problem addressed by the program) is in part the responsibility of individual facilities and the larger provider organisations to which they belong. The lead organisations also have a role to play. Many of the lead organisations, and the individuals involved in those organisations, have had involvement in either residential aged care or evidence-based practice in the past and are likely to maintain that involvement in the future. It is unlikely that the issue of evidence-based practice will fade away any time soon. There is a potential role for government in maintaining the momentum developed in the EBPRAC program which, in part, has already occurred with the decision to make EBPRAC an ongoing program.
Top of page

prev page| TOC |next page