Evaluation of the mental health nurse incentive program
6 Effficiency: findings
Table of contents
6.1 Assessment scope
6.2 Summary of findings
6.3 Program management
6.4 Data collection
6.5 Cost analysis
6.1 Assessment scope
An examination of a program‟s efficiency seeks to ascertain:whether there are better ways of achieving these objectives, including consideration of expenditure and cost per output, project governance arrangements, and implementation processes.
In undertaking the assessment of the program‟s efficiency, the evaluation looked at the processes by which the program is delivered:
- management of the program, including:
- governance;
- implementation processes; and
- cost effectiveness.
6.2 Summary of findings
The key findings of the evaluation of MHNIP in relation to efficiency are summarised below:- Key Finding 19: based on the de-identified patient data provided by case study organisations (N= 267 patients), the cost analysis suggests that savings on hospital admissions attributable to MHNIP could on average be around $2,600 per patient per annum. This was roughly equivalent to the average direct subsidy levels of providing MHNIP, which ranged from an average of $2,674 for patients in metropolitan areas to $3,343 in non-metropolitan areas.
- Key Finding 20: there are a large number of uncosted and intangible benefits associated with MHNIP including the impacts of improved patient outcomes, enhanced relationships with carers and family members, and the effects on carer social security outlays. Examination of these impacts would require an extensive enhancement to existing data collection processes. The evaluation findings suggest a comprehensive economic analysis would find these benefits to be positive.
- Detailed finding #33: department staffing outlays for managing MHNIP are small (around 2.0 FTE) relative to overall program outlays. Consideration could be given to additional administrative activity in the areas of standard report generation and promoting program uptake in relevant sectors e.g. Aboriginal Medical Services.
- Detailed finding #34: a range of additional information should be collected using the claim form and an annual return by the eligible organisation.
Top of page
6.3 Program management
DoHA has overall responsibility for managing the operations of MHNIP. It has roles in the area of program management, marketing and administration, described below.- Program management activities: this role encompasses a range of activities, including:
- establishment of the program structure and elements, such as implementation of the requirement for mental health nurses to be credentialed;
- managing policy aspects of the program;
- managing Program Guideline content and parameters;
- resolving guideline ambiguity (largely via requests from DHS);
- monitoring whether the program is meeting its aims;
- responding to ad hoc queries;
- engaging with internal mental health experts and advisors; and
- responding to queries and provision of information to the Minister‟s office.
DoHA had regular consultations with the sector during the design development phase of MHNIP. Further dialogue with the sector has occurred more recently via the evaluation steering committee.
- Program marketing: marketing activities for MHNIP by DoHA included a media release in April 2007 introducing MHNIP. GPs, psychiatrists and AMSs received a letter about the program and an application package via mail in mid-2007. The DHS web page contains a collection of information on MHNIP, including the Program Guidelines, accessible by searching online.
The ACMHN has actively marketed and promoted MHNIP through their membership and web pages. Other peak bodies such as the Royal Australian and New Zealand College of Psychiatrists have made information on MHNIP available to their members through their web page.
- Program administration: DoHA has a memorandum of understanding with DHS that covers administrative arrangements for a spectrum of health related programs. MHNIP is one program covered by the business rules between the two agencies. Under these rules DHS is responsible for administering program funding. DoHA provides monthly payments of funds to DHS based on the level of session claims.
DHS is responsible for managing program data and data quality on MHNIP. DHS provides monthly reports on MHNIP activity to DoHA. There is scope to make greater use of these reports e.g. to monitor activity levels at a Medicare Local level.
During the evaluation HMA observed that engagement of AMSs in the program is limited. Two AMSs were registered as eligible organisations at 30 April 2012. Only one of these sites was actively involved in delivering MHNIP services.23 HMA visited this site during the evaluation. We examined the treatment and support given by a non-Indigenous mental health nurse and the linkage made to other primary mental health care in that AMS. This observation confirmed that the MHNIP model of care has validity in an AMS context, suggesting that additional promotional activities of MHNIP within the Aboriginal primary care sector could be explored.
Detailed finding #33: department staffing outlays for managing MHNIP are small (around 2.0 FTE) relative to overall program outlays. Consideration could be given to additional administrative activity in the areas of standard report generation and promoting program uptake in relevant sectors e.g. Aboriginal Medical Services.Top of page
6.4 Data collection
This section identifies additional data collection that could be considered for future MHNIP operations.As noted in the previous section, DHS is responsible for managing program data and collection. The main method of data collection is via the session claim process, which requires eligible organisations to fax a completed claim form for each session performed. The form collects a range of detailed information, by eligible organisation, mental health nurse and patient Medicare number.
The evaluation analysis activities sought a range of data from DoHA and DHS, of which some detail was unable to be provided.
Useful data, currently not available includes:
- identification of the patient‟s usual treating physician;
- patient indigenous status;
- indication of whether a mental health care plan has been prepared, and the date; and
- updated details of the eligible organisation, such as number of mental health nurses engaged and full-time equivalence, proportion of medical practitioners actively referring under MHNIP, number of patients participating in MHNIP (and proportion of total patient catchment).
This data could be collected through the existing claim form (electronic rather than fax), accompanied by an annual return to be submitted by each eligible organisation. Whilst this last point is a deviation from what currently occurs, it is consistent with information collection processes for other DoHA funded programs, such as the Access to Allied Psychological Services.
Detailed finding #34: a range of additional information should be collected using the claim form and an annual return by the eligible organisation.
6.5 Cost analysis
HMA undertook a cost analysis to assess the impact of MHNIP. This focussed on the level of resource use in treating patients with a severe and persistent mental illness under the MHNIP service delivery model and compared to what could have occurred in the absence of MHNIP services. Therefore, the study assessed the change in key-resource use of patients with a severe mental illness receiving services under MHNIP and those patients with a severe mental illness that do not receive services under MHNIP. The study employed a retrospective longitudinal study design.A full description of the study method and findings is at Appendix D.
6.5.1 Method
As part of the case study process, HMA sought de-identified information on up to 50 consumers of MHNIP services at each case study organisation. HMA received information on 464 consumers of MHNIP services from 15 case study organisations. Using pre-determined exclusion criteria for patients, a total of 267 patients were included in the analysis. The patients included in the analysis recorded 34 hospitalisations in the 12 months prior to entering MHNIP, and 30 hospitalisations in the 12 months after entering MHNIP.Top of page
6.5.2 Results
The study suggests that MHNIP had the potential to reduce mental health related hospital admissions by approximately 3 days (95% CI -5.57 – 0.078) per patient with severe mental illness and would be associated with a cost saving per patient of around $2,600 (95% CI -$5353 – $75). This finding was statistically significant at the 0.10 level (p<0.06). Caution should be taken when interpreting these savings, given the large confidence intervals. The variability of these results is the product of a small sample size, variability and the low rates of hospitalisations for these patients. The estimated savings are likely to be conservative, given that additional savings may also be derived from a changing pattern of claims for MBS item numbers, and reduced attendances to hospital emergency departments.While detailed information on the number and frequency of sessions these patients had with the mental health nurse was not available, a notional cost of providing services to patients ranged from $2,674 for consumers attending metropolitan practices, to $3,343 for those located in non-metropolitan areas in the 12 months following entry to MHNIP. Feedback from case study organisations indicated a common frequency of contact by patients with their mental health nurse was approximately one hour every week for the first six months following entry into MHNIP and fortnightly appointments thereafter.
The over-all effect on MBS Items claimed in the two periods was ambiguous and should be further explored in future analysis when data is available. Similarly, information on the changing profile of pharmaceutical use of patients both pre and post-entering MHNIP was not available. However, measurement of changes in pharmaceutical use might not be an adequate indicator of impact of the program, given that one of the most commonly reported outcomes related to pharmaceuticals was increased compliance and better management. The effects of increased compliance and better management of medications are likely to result in better patient outcomes, but may have an ambiguous effect on pharmaceutical spending.
Key Finding 19: based on the de-identified patient data provided by case study organisations (N= 267 patients), the cost analysis suggests that savings on hospital admissions attributable to MHNIP could on average be around $2,600 per patient per annum. This was roughly equivalent to the average direct subsidy levels of providing MHNIP, which ranged from an average of $2,674 for patients in metropolitan areas to $3,343 in non-metropolitan areas.
The design of the cost analysis did not enable accurate assessment of the impacts of MHNIP on MBS claim costs, the level of ED admissions, and the value of intangible benefits to patients such as improved patient outcomes, enhanced relationships with carers and family members, and the effects on carer social security outlays. Findings from the evaluation suggest that overall economic benefits of these uncosted impacts and intangible benefits would be positive.
Key Finding 20: there are a large number of uncosted and intangible benefits associated with MHNIP including the impacts of improved patient outcomes, enhanced relationships with carers and family members, and the effects on carer social security outlays. Examination of these impacts would require an extensive enhancement to existing data collection processes. The evaluation findings suggest a comprehensive economic analysis would find these benefits to be positive.Top of page