This section addresses the overarching evaluation objectives of effectiveness, efficiency and appropriateness and includes a discussion on sustainability.
4.3.1 EffectivenessThe CHCI has delivered health services and treatment to Aboriginal people living in remote communities in the NT. Evidence suggests that those who received a health check were already accessing the health system through existing health screening programs available in the NT. This suggests that the CHCI was not an effective mechanism for reaching the unscreened population. Data on health improvements is currently limited and conflicting.
Inefficient processes are likely to have limited the effectiveness of the CHCI. More efficient processes for arranging and delivering hearing/ENT and dental follow-up services have the potential to contribute to the effectiveness of these services providing they are developed within the context of a comprehensive PHC approach. There remains a significant need for effective health service delivery to remote Aboriginal communities.
It is too early to assess whether the EHSDI has been effective in improving the health of remote Aboriginal and Torres Strait Islander people in the NT. The considerable increase in funding and the health workforce (FTE staff) suggest that, over time and with continued investment, the EHSDI has the potential to deliver health improvements.
4.3.2 EfficiencyThe CHCI evaluation found evidence of inefficient processes including inadequately developed administrative processes (such as for referrals) and delays in delivering follow-up services. Assessing the cost-effectiveness of the CHCI is problematic in the absence of clear evidence of effectiveness other than the services delivered.
Table 6 shows the average costs of delivering the child health checks, hearing/ENT and dental follow-up services. Average total costs range from $1,181 (dental) to $1,842 (hearing/ENT) per service delivered.
Table 6: Average costs of CHCI services (30 June 2010)
Service delivery costs to 30 June 2010
Total costs(a) to 30 June 2010
Number of children (CHC) or services (hearing/ENT and dental)
Average cost (service delivery only)
Average cost (total)
(a) Includes capital and infrastructure, administration, training, data collection and other costs (but not travel and accommodation). Excludes DoHA and DHF planning and administrative costs, and ADF support costs.
(b) Excludes costs of RAHC in managing dental follow-up services under the Child Oral Health Program.
Source: DoHA (4 August 2010), NT related health measures—allocation and expenditure in greater detail 2007–08 to 2011-12; AIHW and DoHA (2009).
There are fixed costs associated with setting up any new program and the total cost of a program per unit (in this case per child) can be expected to decline over time as the level of service use increases. The total costs for the CHCI were significant—an average of $1,691 per child for the child health checks—and given the short time frame for the program (two years for the health check component) the level of service use is capped. There will therefore be no efficiency gains other than for those follow-up services which are still being delivered.
Efficiency is a core objective of the regionalisation element of the EHSDI, although it is likely to be some time before this benefit can be realised. The ability of regionalised services to achieve gains in efficiency does not, however, appear to have been thoroughly analysed by the EHSDI partner organisations.
The lack of timeliness in allocating EHSDI funding for service expansion and infrastructure contributed to some inefficiencies; however, we acknowledge that the system as a whole is unlikely to have the capacity to spend large increases in funding in short time frames. The efficiency of the EHSDI was enhanced by its ability to build on existing reform initiatives and lessons, and to use existing governance structures and partnerships. For health service providers the EHSDI represents one of many sources of finance and was often viewed as ‘another grant on top of everything else’. Multiple funding sources and contracts can contribute to inefficiencies (such as from multiple reporting requirements and contractual negotiations) and distract providers from their core business of delivering services to the community.
The RAHC appears to offer similar efficiencies to existing service models for recruiting agency staff, although many DHF health managers would argue that staff recruited through the RAHC are not as effective as more experienced practitioners recruited through other means. top of page
4.3.3 AppropriatenessThe evaluation found that the CHCI was not an appropriate response to the health and wellbeing needs of remote Aboriginal communities in 2007. Clearly these needs were high—and continue to be—and the existing health service was not adequate in meeting these needs. The lack of participation from Aboriginal communities and the existing NT service providers and health professionals in the planning and delivery of the CHCI was not appropriate. The evaluation has found that there was insufficient focus on identifying and specifically targeting existing gaps in health service delivery. We also found that the biggest gap was at the primary care–referred services interface rather than access to PHC. There needs to be greater attention on improving a patient’s journey through the system.
There does not appear to be a single, clear explanation of whether the CHCI created a bottleneck at the primary care–referred services interface, or whether there was an existing bottleneck. Service providers already knew about the needs of the Aboriginal child population and a large proportion of this population was already accessing PHC services. We suggest that the explanation is probably a combination of the following points:
- before the CHCI, service providers knew about many of the children with the most serious needs and these children were accessing PHC and follow-up services; however, knowledge about the high needs of other Aboriginal children was at a population level, rather than knowledge of individual cases, so not all these children were accessing services
- before the CHCI many service providers were experienced in dealing with some follow-up needs themselves without having to refer to follow-up services
- before the CHCI, service providers may have considered referrals inappropriate because these services would not lead to a sustained benefit for an individual child if the child was going back to the same environmental conditions (such as overcrowded housing). Rather than taking a purely clinical perspective they were taking account of a broader population health perspective resulting in only the most serious cases being referred
- before the CHCI, service providers may have been discouraged from referring children because they were unlikely to be seen in time to benefit from the service, again resulting in only the most serious cases being referred
- some existing service providers and local health service staff may have become desensitised to the health needs of Aboriginal children and only referred the most serious cases
- the system was not coping with the existing number of referrals and so the bulge in referrals from the CHCI made the bottleneck to referred services more visible and severe
- it would appear likely that there were some over-referrals from the visiting child health check teams, probably due to several reasons. These may have included the inexperience of some doctors in remote Aboriginal child health and an inability of doctors to complete some follow-ups themselves because of the short time they were in the community. Together these may have led to doctors taking a precautionary approach to assessments and referrals.
While the CHCI and its associated data collections made the gap in existing service delivery between primary care and secondary and some other referred services more visible, the same cannot be said for other potential service responses to address child health. For example, the CHCI and its data collections did not effectively ascertain the level of need for a wide range of other PHC services such as parenting support, health promotion, mental health services and alcohol and other drugs services.
Similarly the CHCI was not used effectively to ascertain the need for social services including family, community and housing services. The CHCI data shows that, out of 9,373 children, just 43 were referred to mental health services, 53 to family and housing services and 65 were referred to a social worker (AIHW and DoHA 2009). We assume this significantly under-estimates the need for these services. The CHCI model was, therefore, inappropriate for ascertaining the need for these broader services. The content of the health check was certainly comprehensive. It was, however, often delivered by visiting doctors and nurses who were not familiar with the child, their carer, or their community. This contributed to a clinically and medically-focused screening program despite the efforts that many visiting teams and resident health staff made to work together to encourage a more holistic approach to the checks.
The EHSDI’s focus on core PHC services and building relationships with Aboriginal communities appears to be an appropriate approach. The components of the EHSDI are part of a long-term and ongoing reform process which builds on previous initiatives within the NT health system. This has facilitated the development of a context-specific approach to PHC reform, informed by past learnings and premised on engagement with local processes. The decision to decentralise the planning and governance of the EHSDI to the NT is in keeping with the aim of building a remote PHC system appropriate to the NT context.
Regionalisation, with its focus on Aboriginal community control and participation, has a sound rationale and is an appropriate way to plan and deliver health services in these parts of Australia. As discussed earlier, the partner agencies need to speak with a consistent voice about the aims of regionalisation. They need to work on all the components of regionalisation so they can fully implement the policy and change management process. top of page
4.3.4 SustainabilityFor many of the agencies and people involved in planning and implementing the CHCI and the EHSDI, sustainability has been a central consideration from the start of the NTER. Many have looked at how to build on the CHCI to achieve increased funding for the NT PHC system. The NT PHC system has demonstrated resilience—a key characteristic of a sustainable system—in responding to the ‘shock’ of the NTER.
The key issues affecting the sustainability of services and outcomes for the CHCI are:
- the need for ongoing resourcing (both funding and staffing) of the models of follow-up service delivery that have evolved from the program in order to complete all outstanding referrals as well as providing for new referrals
- the need to address the poor social conditions that limit longer-term impacts.
At the system planning and policy level, there is a need to sustain the benefits of the CHCI by increasing understanding of the health needs of the population and the ability of services to meet these needs. The further development and implementation of the NT AHKPIs, in addition to data collection and reporting on individual programs such as the GAA/Healthy Under 5 Kids, needs to incorporate appropriate measures to ensure this increased awareness is not lost.
The evaluation identified a number of areas of focus to enhance the sustainability of reforms through the EHSDI. There needs to be a continued focus on building strong relationships and partnerships within and between agencies to enhance the sustainability and continued development of the remote PHC sector. This includes between the three partner agencies of DoHA, DHF and AMSANT, service providers, health consumers and communities.
There is also a need for a long-term funding solution. The lack of long-term funding for the EHSDI is a concern as the gains made through the reform process cannot be sustained without continued financial investment. As well as financial resources, additional policy capacity will need to be directed towards implementing the reforms to sustain the pace of change.
Other aspects of sustainability that should be considered for the EHSDI include the need to engage more directly with other sectors to address social determinants of health. Any improvements to providing remote health services under the EHSDI will have limited long-term impact if there is no explicit engagement with these wider determinants of health, particularly housing.
There is a need to manage risks by planning for failures (such as service providers collapsing) to ensure the system does not fail as a result, and to ensure that the system adapts and learns from both successes and failures. top of page