Evaluation of the National External Breast Prostheses Reimbursement Program

2. Methodology

Page last updated: 05 November 2010

2.1 Overview
2.2 Sampling procedure for consultations
2.3 Timeline
2.4 Workplan

2.1 Overview

This evaluation involves two parts – a review of the administration of the program; and a review of the program's impact on the quality of life of women who have undergone a mastectomy as a result of breast cancer.

The review of the administration of the program will include an assessment of the efficiency, effectiveness and appropriateness of the program. These will be assessed specifically in relation to the following:
  • how efficiently the program is being administered, such as the extent to which resources are optimally used, whether financial, human or procedural
  • how effectively the program has been administered, for instance whether the resources, systems and processes lead to the achievement of the objectives or goals of the program
  • the appropriateness of the program, specifically alignment between the program's systems and intended outcomes.
The impact review will explore the impact of the program on improving the quality of life of women who have undergone a mastectomy as a result of breast cancer. This includes identifying any issues that may need to be addressed, and any modifications or opportunities for future improvements in order to strengthen the program's effectiveness, and maximise its potential for success.

Our evaluation takes into account the following:
  • the short period of time in which the program has been operating (data taken from 1 December 2008 to 1 December 2009)
  • the established business model between Medicare Australia and the department and the importance of efficiency, effectiveness and appropriateness indicators in assessing the program administration
  • the need for sensitivity in interacting with organisations, support groups, and individuals regarding breast cancer
  • the importance of seeking a range of views, including those of women who may not have used the service.
In summary, the methodology includes the following components:
  • development of an evaluation framework
  • assessment of the program's administration, including documentation and Medicare data analysis, and stakeholder consultation
  • assessment of the program's impact, including consultation by online survey, telephone and face-to-face interviews
  • data analysis and reporting.
The evaluation will run from July 2009 to August 2010. Top of page

2.2 Sampling procedure for consultations

We propose a multi-stage sampling process for consulting with stakeholders for this project. These are outlined as follows.

2.2.1 Administrative component

Face to face and telephone interviews will be conducted with the following stakeholders:
  • The department
  • Medicare Australia
  • Breast Cancer Network Australia (BCNA) – both administration and member groups
  • National Breast and Ovarian Cancer Centre (NBOCC)
  • Cancer Council Australia
  • State and territory Cancer Councils
  • Health care professionals – includes: breast care nurses, oncology nurses, care coordinators, social workers, general practitioners and breast surgeons.
  • State governments – program managers from Vic, Qld and WA
  • Breast prostheses suppliers
  • McGrath Foundation
Lists and contact details will be provided by the department. An interview guide will be developed and finalised in consultation with the department. Top of page

2.2.2 Impact component

Consultations with women who have had a mastectomy

An email survey will be distributed and telephone interviews and focus groups will conducted with women who have had a mastectomy and have accessed the program, as well as women who have had a mastectomy and have not accessed the program. Methods of sampling will be finalised in consultation with the department and consumer representative groups for women with breast cancer.

We will seek to consult with women across Australia, and in doing so will include women from a wide spectrum of the community:
  • age - between the ages of 18 and 80
  • location - urban, rural, remote; all states/ territories
  • cultural background - Aboriginal, Torres Strait Islander, European Australian, CALD background
  • eligible women who have not sought a prosthesis
  • eligible women who have not sought reimbursement
  • women who do not have internet connection
Possible access strategies:
  • work through Breast Cancer Network Australia and/or other support networks to invite eligible women to participate in study
  • place notices in newsletters, bulletin boards, and other media for women who are not connected to the internet and would not otherwise be included in the study
A number of considerations have been made in ensuring sensitivity and anonymity to potential participants. Women will be able to participate in three ways (completion of an email survey, participation in a telephone interview, and participation in focus groups), recognising that some women may not wish to speak to a stranger about their experiences, but may be willing to provide written commentary. In all instances, we will not contact any individual woman without her having first given permission for her contact details to be provided to us. Women will be asked to give consent to participate and will be able to withdraw their consent at any time; if they do so any identifying data held by the research team will be destroyed. Confidentiality will be respected at all times and only aggregated, anonymised data will be presented in reports. Top of page

Consultations with other stakeholders

Face to face and telephone interviews with the following stakeholders:
  • The department
  • Breast Cancer Network Australia (BCNA) – both administration and member groups
  • National Breast and Ovarian Cancer Centre (NBOCC)
  • Cancer Council Australia
  • State and territory Cancer Councils
  • Health care professionals – includes: breast care nurses, oncology nurses, care coordinators, social workers, general practitioners and breast surgeons.
  • State Governments – program managers from VIC, QLD and WA
  • Breast prostheses suppliers
  • McGrath Foundation
Lists and contact details will be provided by the department. Interview and focus group guides will be developed and finalised in consultation with the department. Top of page

2.3 Timeline

Project planning

Inception meeting - 9 Jul 2009 (billing milestone)
Submission of evaluation framework with project workplan - 31 Jul 2009 (billing milestone)
Department to send Medicare data and prepare initial correspondence - weeks commencing 3 Aug to 31 Aug 2009
Initial discussions with support organisations re consultations - weeks commencing 7 Sep to 30 Nov 2009

Impact review

Development and pilot of survey - weeks commencing 5 Oct to 30 Nov 2009
Mapping of access routes to women - weeks commencing 7 Sep to 30 Nov 2009
Survey of women - weeks commencing 7 Dec to 21 Dec 2009 and 11 Jan to 25 Jan 2010
Stakeholder consultations - weeks commencing 1 Feb to 29 Mar 2010
Data analysis - weeks commencing 1 Mar to 19 Apr 2010
Progress report 1 - 30 Nov 2010 (billing milestone)

Administration review

Document review - weeks commencing 7 Sep to 12 Oct 2009
Medicare data analysis - weeks commencing 5 Oct to 26 Oct 2009, 7 Dec to 21 Dec 2009, and 11 Jan to 25 Jan 2010
Stakeholder consultations - weeks commencing 7 Dec to 14 Dec 2009 and 11 Jan to 29 Mar 2010
Analysis - weeks commencing 1 Mar to 26 Apr 2010
Progress report 2 - 28 Feb 2010 (billing milestone)

Final reporting

Draft final report - 31 May 2010 (billing milestone)
Final report - 16 Jul 2010 (billing milestone) Top of page

2.4 Workplan

Activity

Days allocated

Team members responsible

Week beginning

Project management

Inception meeting2LK and NK.9/7/2009
Client liaison4LK project direction and oversight. NK to assist as required.Ongoing

Develop evaluation framework

Develop framework5LK and NK.31/7/2009

Develop research tools

Develop survey4.5SR to manage, NK to assist, LK to oversee.2/11/2009
Develop interview guides3.5KO and NK to develop, LK to oversee.16/11/2009

Stakeholder consultation - administration review

Interviews4Managed by KO. Assistance from LK and NK.7/12/2009
Write up3KO and NK.7/12/2009
Analysis of findings2KO, NK, LK.5/4/2010

Data analysis - administrative review

Document analysis3SR.7/9/2009
Medicare data analysis6SR with assistance from NK as required7/12/2009

Stakeholder consultation - impact review

Survey dissemination and management5NK to manage, LK to oversee.7/12/2009
Interviews and write up15KO, NK, LK.1/2/2010

Data analysis - impact review

Survey data entry3SR to manage.1/2/2010
Survey analysis4SR.8/2/2010
Interview coding and analysis7SR and NK, with LK to oversee.5/4/2010

Reporting

Progress report 13LK, SR, NK.30/11/2009
Progress report 23LK, SR, KO, NK.22/2/2010
Final draft report25LK, SR, NK.31/5/2010
Final report5LK, SR, NK.12/7/2010

Total number of days

107
Top of page

Study team

Linda Kurti (LK) has over 15 years of experience in health research and organisational development, working in the government, academic and non-profit sectors. Linda's experience covers a range of areas including strategic planning, organisational change and development, system development and evaluation, and health services research in both England and Australia. Her research interests include health system development, health equity, global health, and the intersection of praxis and policy. Linda has worked as a facilitator and consultant with a range of organisations, and has held both executive and strategic roles.

Samantha Ross (SR) is a Senior Consultant with Urbis. She is a highly skilled qualitative researcher and project manager. Since joining the firm Samantha has been a significant contributor to projects involving stakeholder, consumer and community consultation, data collection and analysis on a range of issues, including health, drugs and alcohol, welfare delivery, ageing and aged care, crime and justice, community services and community development, communication and education materials and campaigns, Indigenous Australians, housing and changes to the built environment.

Karen Olver (KO) is a unique and highly skilled practitioner with over twenty years experience in the fields of health care, mental health, disability, justice and aged care including their legislative frameworks, strategic directions and service quality issues.

Nichola Keevy (NK) is a social researcher at Urbis. Nichola has been involved in a wide range of studies that have required stakeholder consultation through telephone interviews, qualitative and quantitative data analysis, literature reviews, in-depth interviewing and report writing. Nichola has research experience across a range of fields including law and justice, social welfare, indigenous education, media communications, program evaluation, community consultation, and aged care.