In designing the evaluation, Allen and Clarke identified a set of evaluation questions based on the four evaluation objectives for the CHCI and the six evaluation objectives for the EHSDI. Table 1 lists the evaluation objectives for the CHCI and explains how we have organised the findings in this report on these objectives.
|1. Assess the extent to which the child health checks reached the target population.||This objective is addressed in Section 4.2. Our focus has been on comparing the group of children who had a child health check with the group of children who were eligible for a child health check but who did not have one.|
|2. Identify the prevalence and, if possible, the severity of the health conditions found through the child health check and validate these findings with data from other sources.||This objective has been largely addressed by the AIHW in the process of monitoring the progress of the CHCI. We considered validating the child health check data against data from additional data sources. This, as well as the results of the AIHW’s analysis, is discussed in Section 2.3.4. The severity of conditions has not specifically been assessed by the AIHW or this evaluation.|
|3. Assess the extent to which requested primary care, allied health and specialist follow-up services have been received, gaps in existing health service delivery, and barriers to the completion of follow-up treatment.||This objective is addressed in Section 4.3 where we discuss the referral and follow-up process, report on what services were requested and received, and the appropriateness of referrals.|
|4.1 Assess whether or not the CHCI has led to improvements in health service delivery for Aboriginal and Torres Strait Islander children.||This objective is addressed in Section 4.4 which looks at the impact of the CHCI on health staff, services and the wider PHC system in the NT.|
|4.2 Assess the health status of children in relation to the social determinants of health and access to comprehensive primary health care||These objectives are addressed in Section 4.5 where we discuss what treatment was provided as part of the CHCI, any change in health status that has occurred since the CHCI, and provide some commentary and baseline data on the social determinants of health.|
|4.3 Assess treatment outcomes.|
The organisation of the evaluation findings for the formative evaluation of the EHSDI is more complex. In addition to considering the evaluation objectives as an organising framework, we considered:
- the EHSDI funding components—expanded PHC services, development of regions and the move towards Aboriginal community control, the RAHC, capital and infrastructure and evaluation
- the five systems-wide policy issues identified in the first of our evaluation workshops as pillars of the NT PHC reform program—leadership/governance/regionalisation, core PHC services, workforce/infrastructure, NT AHKPIs and CQI
- the PHRG’s health systems planning framework based on the World Health Organization’s (WHO) framework which encompasses six building blocks (WHO 2007a)—leadership and governance, service delivery, access to medicines, vaccines and technology, workforce, financing and information
- other, less tangible elements of a health system discussed at our two evaluation workshops— partnership, capacity, communication and sustainability
- the overarching evaluation objectives of effectiveness, efficiency and appropriateness..
Section 7 of this report, which covers key findings and recommendations, fully adopts the PHRG’s health systems planning framework (based on the WHO’s building blocks), but with the omission of ‘access to medicines, vaccines and technology,’ which was not covered by this evaluation.
Top of page
Table 2 shows how we have organised the findings in this report to align with the evaluation objectives for the EHSDI.
|1. Assess the impact and sustainability of the EHSDI on primary health care service delivery and equitable distribution of resources. This will include measurement against indicators of the number, range, and accessibility of services compared with agreed standards for primary care across the NT.||This objective is principally addressed in Section 5.3 which looks at how the EHSDI has affected health services and remote health infrastructure and the impact of the EHSDI on the equitable distribution of resources. The sustainability of the EHSDI reforms is considered throughout the report, with the information drawn together in Section 6.4.|
|2. Assess the extent to which Aboriginal and Torres Strait Islander people were engaged and empowered to contribute to health service planning, governance and responsiveness of services.||This objective is addressed in two sections. Section 5.4 discusses leadership and governance in the NT PHC system. Section 5.5 looks at Aboriginal engagement through the process of regionalisation and Aboriginal community control.|
|3. Assess the impact and sustainability of the RAHC on health workforce availability and flexibility in the NT. This will include measurement against indicators of workforce supply in all locations and the effectiveness of clinical governance structures.||This objective is addressed in Section 5.6 which considers the performance of the RAHC workforce model and its impact on meeting local needs, quality and clinical governance, workforce capacity and the overall sustainability of the model.|
|4. Assess the efficiency of the EHSDI and how well it has maximised health service delivery with the available funds.||Efficiency is an overarching evaluation objective. Discussion on efficiency in health service delivery is in Section 5.3.|
|5. Assess the effectiveness of the EHSDI in achieving change in health status. This will include measurement against primary care related health indicators as developed through the NT key performance indicators (NT AHKPIs) project and the analysis of the NTER child health check program.||Effectiveness is also an overarching evaluation question. It was premature to fully address the question of the effectiveness of the EHSDI in changing health status, so the discussion focuses on the likelihood of the program contributing to improved health over the longer-term. Discussion of the NT AHKPIs is included in Section 5.7.|
|6.1 Assess the impact of the regional reform process on the efficient and effective operation of health services.||The program of regional reform had not progressed sufficiently to fully address these objectives. However, discussion on clinical governance (objective 6.2) and information systems (objective 6.3) is included in Section 5.7. The impact of the regional reform process more generally, is discussed in Section 5.5.|
|6.2 Assess the impact of the regional reform process on clinical governance, including quality of health services delivery.|
|6.3 Assess the impact of the regional reform process on information systems and planning capacity.|
Findings relating to the three overarching evaluation objectives of effectiveness, efficiency and appropriateness are incorporated, where relevant, in the sections that address each of the CHCI and EHSDI evaluation objectives. The findings on each of these overarching objectives are brought together in Section 6, along with issues relating to sustainability which emerged as a fourth overarching issue.
In addition to the evaluation objectives, DoHA asked us to consider the longer-term monitoring and evaluation approach for PHC system reforms and for future child wellness checks and services. Section 7.3 of this report includes theoretical program models for PHC system reform and child wellness checks. It includes important issues and questions to inform both future monitoring and evaluation approaches and the ongoing reform activity. Top of page