Better health and ageing for all Australians

Evaluation of the mental health nurse incentive program

Executive summary

Up to Publications

prev page| TOC |next page

Background to the evaluation
Evaluation scope
Summary of findings: appropriateness
Summary of findings: effectiveness
Summary of findings: efficiency
Overall evaluation findings
Possible ways forward

Background to the evaluation

Healthcare Management Advisors (HMA) was engaged by the Department of Health and Ageing (DoHA) to undertake an evaluation of the Mental Health Nurse Incentive Program (MHNIP). Initiated in July 2007, MHNIP provides payments to community based general practices, private psychiatric practices and Aboriginal Medical Services (AMS) to engage mental health nurses to:

......assist in the provision of coordinated clinical care for people with severe mental health disorders.

Mental health nurses work in collaboration with psychiatrists and general practitioners to provide services such as monitoring a patient's mental state, medication management and improving links to other health professionals and clinical service providers. These services are provided in a range of settings, such as clinics or patient's homes and are provided at little or no cost to the patient.1

Evaluation scope

The purpose of the evaluation was to:

"assess the effectiveness and appropriateness of the program and its current operational parameters as well as model future demand and growth patterns. Specifically, the scope of the evaluation will address patient outcomes, program uptake, program demand, cost benefits, program structure and compliance".2
HMA undertook the following steps for the project:
  • Prepared a situation analysis;
  • Developed an evaluation framework;
  • Modelled demand;
  • Conducted provider surveys;
  • Conducted 18 case studies; and
  • Undertook a cost analysis.
Top of pageThis process was guided by the underlying program logic for MHNIP:
  • Policy context: MHNIP was announced in July 2006 as part of the Council of Australian Government's National Action Plan on Mental Health;

  • Program objectives: the aims of MHNIP are to:
    • Improve levels of care for people with severe and persistent mental disorders;
    • Reduce the likelihood of unnecessary hospital admissions and readmissions;
    • Assist in keeping people with severe disorders feeling well and connected within the community; and
    • Alleviate pressure on privately practicing psychiatrists and GPs.

  • Program scope: the key program design features of MHNIP, including the financial, operation and service delivery characteristics are:
    • MHNIP is delivered by community based primary and private specialist health services, including GPs, private psychiatrists and Aboriginal Medical Services (AMSs) funded by the Office of Aboriginal and Torres Strait Islander Health;
    • Eligible organisations receive an establishment grant and payments for sessions of care provided to patients within the program target group;

  • Program requirements: the requirements for eligible organisations to implement MHNIP, include:
    • Development of patient management protocols;
    • Recruitment of a mental health nurse credentialed with the Australian College of Mental Health Nurses (ACMHN); and
    • Reimbursement via submission of claim forms to the Department of Human Services (DHS);

  • Implementation and service delivery: the journey for patients receiving support under the program includes: assessing patient eligibility, development of a mental health plan, and implementation of the treatment and support plan;

  • Outcomes: the expected overall outcome as a result of the intervention is increased health and wellbeing of people with severe and persistent mental illness.
This paper, the final evaluation report, assesses the program's impacts using the information collected through the evaluation process. The report makes 20 key findings, summarised below. There are a further 34 detailed findings that relate to the mechanics of the program operations presented in the body of the report.

Summary of findings: appropriateness

The key findings of the evaluation of MHNIP in relation to appropriateness are summarised below:
  • Key Finding 1: there is a sizeable group of people in the community with severe and persistent mental illness. Expert advice suggests this is in the order of 1.2% of the adult population aged 18 to 64 years. It is estimated that a little under half of this group is the size of the MHNIP target population - 0.6% of the adult population with severe and persistent mental illness primarily reliant on assistance from GPs and psychiatrists in the private sector.

  • Key Finding 2: the target group will always be bigger than realised demand under MHNIP eg some people will have exited the program because their condition has stabilised. Allowing for this, there is evidence demand exceeds the services currently available under MHNIP – an estimated 49,800 people in 2011-12.

  • Key Finding 3: there is a high level of support from medical practitioners for the model of care embedded in MHNIP whereby mental health nurses, working in conjunction with GPs and psychiatrists, provide treatment and support to people with severe and persistent mental illness living in the community.

  • Key finding 4: patients, carers and relevant peak bodies were also supportive of the model of care underlying MHNIP.

  • Key finding 5: General Practices and Medicare Locals (formerly Divisions of General Practice) accounted for the largest proportion of MHNIP services delivered (80.9%) and mental health nurses employed (76.4%) between 1 July 2009 and 30 June 2011

  • Key finding 6: there was evidence that medical practitioners are triaging patients to different Commonwealth funded programs supporting people with mental illness, based on clinical need. This included utilising MHNIP for patients with severe and persistent mental illness, and referral of patients with lower levels of disability to support from other appropriate services.

  • Key finding 7: until the application of session caps in May 2012, realised demand under MHNIP was driven by supply-side factors –the number of eligible providers and credentialed nurses. These program design features were not sustainable in a period of budget restraint.

  • Key finding 8: access to MHNIP services varies by jurisdiction. The supply-side driven design characteristics of MHNIP meant that service growth was not always linked to geographic areas where there was higher relative need for new services.
Top of page

Summary of findings: effectiveness

The key findings of the evaluation of MHNIP in relation to effectiveness are summarised below:
  • Key Finding 9: patients being supported under MHNIP are benefitting from improved levels of care in the form of greater continuity of care, greater follow-up, timely access to support, and increased compliance with treatment plans.

  • Key Finding 10: examination of a sample of MHNIP patients in the evaluation cost analysis showed a downward trend in their HoNOS scores, a measure of mental health and social functioning. This statistically validates qualitative perceptions that the treatment and support provided by mental health nurses improves the mental health and wellbeing of patients receiving support under the program.

  • Key Finding 11: based on an examination of a sample of MHNIP patients, the HoNOS score of patients using state and territory mental health services were on average at similar levels to the scores of MHNIP patients, affirming that the program is providing support to people with severe mental illness.

  • Key Finding 12: quantitative evaluation evidence showed overall mental health hospital admissions decreased by 13.3% for a sample of MHNIP patients in the 12 months following their involvement in the program. This was not true for all conditions: bipolar disorders showed a slight increase in the number of admissions.

  • Key Finding 13: for the same sample of patients, when they were admitted to hospital following their engagement in MHNIP, there was on average a reduction in their total number of admission days by 58% and the average length of stay fell from 37.2 days to 17.7 days.

  • Key Finding 14: there was some evidence of increased patient employment by MHNIP patients.

  • Key Finding 15: MHNIP has encouraged and facilitated patient's increased involvement in social and educational activities.

  • Key Finding 16: MHNIP has had positive flow on benefits to some carers of MHNIP patients.

  • Key Finding 17: MHNIP has had other positive impacts on patients, including improved family interactions and reductions in the number of emergency department presentations.

  • Key Finding 18: MHNIP has had a positive impact on medical practitioner workloads by increasing their time available to treat other patients and improve patient throughout.

Summary of findings: efficiency

The key findings of the evaluation of MHNIP in relation to efficiency are summarised below:
  • Key Finding 19: based on the de-identified patient data provided by case study organisations (N= 267 patients), the cost analysis suggests that savings on hospital admissions attributable to MHNIP could on average be around $2,600 per patient per annum. This was roughly equivalent to the average direct subsidy levels of providing MHNIP, which ranged from an average of $2,674 for patients in metropolitan areas to $3,343 in non-metropolitan areas.

  • Key Finding 20: there are a large number of uncosted and intangible benefits associated with MHNIP including the impacts of improved patient outcomes, enhanced relationships with carers and family members, and the effects on carer social security outlays. Examination of these impacts would require an extensive enhancement to existing data collection processes. The evaluation findings suggest a comprehensive economic analysis would find these benefits to be positive.
Top of page

Overall evaluation findings

Based on the commentary provided in the evaluation assessment we provide the following overview of our evaluation findings:
  1. Appropriateness: MHNIP is providing support to a sizeable group in the community – people with severe and persistent mental health illness who are primarily reliant for their treatment on GPs and psychiatrists in the private sector (around 0.6% of the adult population). There are still large levels of unmet need from this group. The model of care involving clinical treatment and support provided by credentialed mental health nurses working with eligible medical practitioners received strong endorsement. This came from patients, carers and medical practitioners using the program, along with relevant peak bodies.

  2. Effectiveness: the evaluation found that patients receiving treatment and support under the program benefitted from improved levels of care due to greater continuity of care, greater follow-up, timely access to support, and increased compliance with treatment plans. This was evidence of an overall reduction in average hospital admission rates while patients were being cared for, and reduced hospital lengths of stay where admissions did occur. There was also evidence that patients supported by MHNIP had increased levels of employment, at least in a voluntary capacity, and improved family and community connections. MHNIP has had a positive impact on medical practitioner workloads by increasing their time available to treat other patients and improve patient throughput.

  3. Efficiency: based on the de-identified patient data provided by case study organisations (N= 267 patients), the cost analysis suggests that savings on hospital admissions attributable to MHNIP were on average around $2,600 per patient per annum. This was roughly equivalent to the average direct subsidy levels of providing MHNIP, which ranged from an average of $2,674 for patients in metropolitan areas to $3,343 in non-metropolitan areas. There are a large number of uncosted and intangible benefits associated with MHNIP, including the impacts of improved patient outcomes, enhanced relationships with carers and family members, and the effects on carer social security outlays. Examination of these impacts would require an extensive enhancement to existing data collection processes. The evaluation findings suggest a comprehensive economic analysis would find these benefits to be positive.
Although the model of care underpinning MHNIP is well regarded and has positive outcomes, other design features of the program could be re-examined. This is particularly true of the current purchasing arrangements. These provide limited capacity to manage demand in line with program resource allocations and do not enable growth to be targeted at geographic areas of greatest need.

Possible ways forward

Observations on possible areas for enhancement of MHNIP are provided in Table E.1.

Table E.1: MHNIP design features – commentary and options to address program design issues

Table E.1 is presented as a list in this HTML version for accessibility reasons. It is presented as a table in the PDF version.

Model of care

Current MHNIP design feature
  • Target group: people in the community with a severe and persistent mental illness.
  • Credentialed mental health nurses work closely with GPs and psychiatrists to provide coordinated clinical services. It should be noted that GPs and Psychiatrists are the primary care givers.
  • The Program Guidelines outline functions that mental health nurses should undertake.
  • There is no cap on the number of sessions a nurse has with a patient
  • A nurse can be engaged to provide between one and ten sessions per week, per organisation, with an average nurse caseload of at least two individual services to patients per session.
Observations based on evaluation findings
Possible options for consideration, based on the evaluation findings
  • The Program Guidelines could be further revised to clarify roles and responsibilities of eligible organisations and mental health nurses, particularly in relation to responsibilities in managing the triage process, services provided and clinical governance
Top of page

Program participation

Current MHNIP design feature
  • Eligible (ie registered) organisations, comprising self-selected:
    • Private primary care services – general practices and private psychiatry practices
    • Medicare Locals
    • Divisions of General Practices
    • Aboriginal and Torres Strait Islander Primary Health Care Services funded by the Australian Government through the Office for Aboriginal and Torres Strait Islander Health (OATSIH).
Observations based on evaluation findings
  • There are varying degrees of program uptake across organisation types with GPs providing most MHNIP services and only a small number of Aboriginal and Torres Strait Islander Primary Health Services taking up the program. This self-selected, demand driven approach has resulted in inequitable service delivery (See Demand Management below) See section 4.4.2, section 4.6, key finding 5 and key finding 8.
Possible options for consideration, based on the evaluation findings
  • Investigation into the causes of unmet demand would assist in determining the reasons for service inequity. Some factors to consider include socioeconomic trends in each geographic area, patient drivers and Commonwealth and state and territory services that are available for people with severe and persistent mental illness in areas of perceived unmet demand.

Funder

Current MHNIP design feature
  • DoHA
Observations based on evaluation findings
  • N/A
Possible options for consideration, based on the evaluation findings
  • N/A

Purchaser

Current MHNIP design feature
  • DoHA is the funder and purchaser (based on retrospective payment of claims in arrears).
  • Purchasing intelligence: DHS reports.
Observations based on evaluation findings
  • There is therefore limited control over program expenditure levels (other than the current cap on sessions).
  • There is currently no mechanism to ensure equitable access to MHNIP services across geographies.
  • Based on derived figures of population with severe and persistent mental illness, there is evidence that demand exceeds services currently available (See also detailed finding 8).
  • As program is demand driven, supply side factors such as availability of nurses and perceived need by medical practitioners determine where services are provided. Due to this reason, service growth is not always linked to geographic areas where there was higher need for new services. See section 4.4.5, section 4.5, section 4.6 section 6.3, key finding 7 and key finding 8.
Possible options for consideration, based on the evaluation findings
  • Consider ways to ensure any new service provision is targeted to regions of unmet demand rather than being driven by supply side factors.
  • Ensure eligibility criteria on entrance and exit are clearly understood and complied with.
  • Facilitate more formalised patient pathways between MHNIP and other appropriate services.
  • Consider ways to manage program and regional expenditure levels.
Top of page

Demand management

Current MHNIP design feature
  • Nil, until application of the session cap in May 2012. Prior to this activity levels were driven by supply side factors:
    • number of eligible providers; and
    • number and availability of credentialed nurses.
Observations based on evaluation findings
  • There is therefore limited control over program expenditure levels (other than the current cap on sessions).
  • There is currently no mechanism to ensure equitable access to MHNIP services across geographies.
  • Based on derived figures of population with severe and persistent mental illness, there is evidence that demand exceeds services currently available (See also detailed finding 8).
  • As program is demand driven, supply side factors such as availability of nurses and perceived need by medical practitioners determine where services are provided. Due to this reason, service growth is not always linked to geographic areas where there was higher need for new services. See section 4.4.5, section 4.5, section 4.6 section 6.3, key finding 7 and key finding 8.
Possible options for consideration, based on the evaluation findings
  • Consider ways to ensure any new service provision is targeted to regions of unmet demand rather than being driven by supply side factors.
  • Ensure eligibility criteria on entrance and exit are clearly understood and complied with.
  • Facilitate more formalised patient pathways between MHNIP and other appropriate services.
  • Consider ways to manage program and regional expenditure levels.

Planning - practice level

Current MHNIP design feature
  • Triaging at the practice level.
Observations based on evaluation findings
Possible options for consideration, based on the evaluation findings
  • Clinical governance processes at a regional level could be developed to promote greater uniformity in the level of acuity of patients entering and exiting MHNIP. Processes may need to be varied in accordance with access to other support for people with severe and persistent mental illness in the area (eg access to public mental health services varies by geographic region).

Planning - other levels (regional; national)

Current MHNIP design feature
  • While nurse engagement and patient management is typically managed at the practice level, some eligible Medicare Locals triage at the sub-regional or regional level.
Observations based on evaluation findings
Possible options for consideration, based on the evaluation findings
Top of page

Clinical governance - practice level

Current MHNIP design feature
  • Some rules, as per the Program Guidelines, that are applicable to eligible organisations include the following:
    • The mental health nurse delivers services in collaboration with the medical practitioner.
    • The medical practitioner is required to practice formal protocols in managing patient mental health care, including the use of a GP Mental Health Treatment Plan, mental health nurse assessment of eligible patients at entry, every 90 days and when patient exit the program using the Health of the Nation Outcomes Scale, including the Child and Adolescent, Adult, and Older Person tools.
  • Other activities: ad hoc (eg nurse clinical supervision is determined on a site basis between the medical professional and the nurse; professional development is at the nurse's discretion, other than what is required to maintain credentialed status).
Observations based on evaluation findings
  • There is wide variability in clinical governance practices, including clinical supervision at a practice level. Quality could be improved if there was a more standardised approach. See section 5.5, appendix C and detailed finding 18.
Possible options for consideration, based on the evaluation findings
  • Program Guidelines could be further revised to clarify expectations of mental health nurses and medical practitioners in service provision.

Clinical governance - other levels (regional; national)

Current MHNIP design feature
  • The Program Guidelines provides program participants with guidance on patient, organisation and nurse eligibility criteria, administration of the program and guidelines that organisations registered to provide MHNIP should abide by.
Observations based on evaluation findings
  • No formal clinical governance arrangements, however the Program Guidelines provide a range of requirements that relate to governance type activities. See section 5.5 and appendix C and detailed finding 18.
  • Medical practitioners and nurses from the evaluation agree that the Program Guidelines are generally accessible; however there is scope to revise the Guidelines, in particular to allow for greater clarity in some areas including clearer description of reporting requirements and services that can be provided. See section 5.7.2 and detailed finding 20.
Possible options for consideration, based on the evaluation findings
  • Develop a standardised approach to clinical governance at the regional and national level, including advice on:
    • triage processes;
    • case management processes;
    • risk management;
    • patient and carer complaint mechanisms; and
    • identifying and supporting hard to reach population groups.
  • The Program Guidelines could be further revised to clarify roles and responsibilities and reporting requirements
Top of page

prev page| TOC |next page