The National Breastfeeding Helpline Evaluation report

Appendix A

Page last updated: 13 February 2013

Evaluation framework

Evaluation framework, questions and indicators

Intermediate Outcomes
Evaluation Questions
Indicators
Secondary Data Sources
Primary Data Sources
    1. Increased access to breastfeeding advice and support for mothers, partners and their families
1.1 Has the Breastfeeding Helpline service been implemented as intended?

Administrative arrangements:

Does the Breastfeeding Helpline operate as a national, 24-hour service

Is the Breastfeeding Helpline adequately and appropriately staffed

Is the data collection robust

How well is the service coordinated

Is the service efficient

What is the overall cost of providing the service

    • Time distribution of calls taken over the 24-hour period
    • Proportion of accredited Breastfeeding Counsellors working on the Helpline available to be rostered on duty
    • Average caller counsellor ratio over a 24-hour period
    • Occasions of referrals from the Breastfeeding Helpline to health professionals
    • Referrals to the Helpline by health professionals
    • Average cost per call
    • Production of an annual Breastfeeding Helpline data collection report
    • NBH data on Helpline operation
    • NBH protocols and counsellor support documentation
    • ABA Breastfeeding Helpline training data
    • ABA Breastfeeding Helpline expenditure data
    • Survey feedback from Breastfeeding Counsellors working on the Helpline and callers
    • Interview feedback from DoHA stakeholders, professional associations & jurisdictions
    • Cost effectiveness analysis
    • Literature review of effectiveness of telephone helplines to support breastfeeding
1. Increased access to breastfeeding advice and support for mothers, partners and their families1.2 To what extent have mothers and their families utilised the services of the Breastfeeding Helpline?

Callers:

What are the characteristics and needs of the callers

What type of assistance are the callers receiving

What strategies are in place to optimise caller access to counsellors

Consumers:

What is the level of consumer awareness of the Breastfeeding Helpline

    • Number and profile of callers to the national Breastfeeding Helpline
    • Calls to the Breastfeeding Helpline are appropriate
    • Proportion of callers who are connected to a counsellor
    • Caller wait times
    • NBH data on Helpline usage
    • Survey feedback from Breastfeeding Helpline callers and counsellors
    • Interview feedback from DoHA stakeholders, professional associations & jurisdictions
    • Focus groups with young mothers, Aboriginal and Torres Strait Islander mothers, mothers with a disability and mothers from culturally and linguistically diverse backgrounds
1. Increased access to breastfeeding advice and support for mothers, partners and their families1.3 Are callers satisfied with the support provided by the Breastfeeding Helpline?

Callers:

What is the level of demand for services? How is the service dealing with unmet demand

Has the service met the needs of the callers

Provider:

What factors have impacted on service delivery and capacity to meet objectives (staffing, availability of local support services, available training and tools)

Are there program performance/service targets

What quality assurance system is in place including formal reporting requirements

What governance arrangements are in place for the Breastfeeding Helpline and how are they supported

    • Reasonable waiting time for access to a counsellor
    • Ratio of counsellors to callers
    • Percentage of callers for whom the Breastfeeding Helpline support was relevant
    • Number, type and outcome of complaints to ABA about the Helpline
    • Development and monitoring of service targets
    • Development and implementation of a continuous improvement strategy
    • NBH data on Breastfeeding Helpline operation, usage and satisfaction survey
    • ABA Breastfeeding Helpline complaints information
    • ABA documented processes underpinning effective operation of the Breastfeeding Helpline
    • Survey feedback from Breastfeeding Helpline callers
    • Focus groups with young mothers, Aboriginal and Torres Strait Islander mothers, mothers with a disability and mothers from culturally and linguistically diverse backgrounds
    • Interview feedback from DoHA stakeholders, professional associations & jurisdictions
    1. Increased access to breastfeeding advice and support for mothers, partners and their families
1.4 How well is the Helpline utilised by priority population groups?

Callers & service providers:

How does the service target consumers from rural and remote communities and areas of socio-economic disadvantage

Breastfeeding Counsellors working on the Helpline & provider:

What are the critical success factors in engaging and supporting consumers with special needs such as those from culturally and linguistically diverse backgrounds, those with disability and young mothers

Is the service model appropriate in meeting the needs of all targeted population groups

Service providers & jurisdictions:

Is the service model appropriate to the wider service system

    • Proportion of callers to the Breastfeeding Helpline from priority population groups
    • Occasions of use of interpreters for callers contacting the Breastfeeding Helpline
    • Occasions of use of a telecommunications device for the deaf for callers contacting the Breastfeeding Helpline
    • Perception of the Breastfeeding Helpline as a relevant and accessible source of support for priority population groups
    • Implementation of strategies to improve Breastfeeding Helpline accessibility for priority population groups
    • NBH caller survey data
    • ABA administrative data regarding use of interpreters by Breastfeeding Counsellors working on the Helpline
    • ABA interviews for information about strategies to reach priority population groups
    • Online caller & counsellor surveys
    • Focus groups with young mothers, Aboriginal and Torres Strait Islander mothers, mothers with a disability and mothers from culturally and linguistically diverse backgrounds
    • Interview feedback from DoHA stakeholders, professional associations & jurisdictions
    • Literature review on models of good practice
    2. Agreed minimum standards for counselling on breastfeeding
2.1 Do the minimum qualifications for a Breastfeeding Helpline volunteer counsellor comply with requirements of the national regulator AQSA?

Are there appropriate bridging courses available to counsellors that meet the increased compliance requirements?

    • Counselling courses meet industry standards
    • Regular feedback and review processes in place to ensure currency and comprehensiveness of counsellor support documentation
    • ABA course and accreditation documentation
    • ABA interview feedback
    • Survey of counsellors
    3. Enhancement of breastfeeding counsellor skills
3.1 Is the training provided for Breastfeeding Counsellors working on the Helpline sufficient to ensure the supply of a skilled workforce and the sustainability of a quality, responsive Helpline service?
    • Numbers of accredited volunteer Breastfeeding Counsellors working on the Helpline
    • Number of active accredited Breastfeeding Counsellors working on the Helpline
    • Number of mentoring programs for new counsellors
    • Proportion of counsellors who have upgraded their skills to meet increased compliance requirements
    • Number, type and location of continuing education opportunities for counsellors
    • Provision of continuing education resources to volunteer counsellor workforce
    • Quality assurance processes
    • ABA documentation/reports on training for counsellors
    • ABA data on pool of accredited volunteer counsellors
    • ABA data on counsellor mentoring program
    • ABA information on counsellor resources
    • ABA quality surveys
    • ABA interview feedback
    • Survey of counsellors
    • Interview feedback from DoHA stakeholders, professional associations & jurisdiction
    4. Greater awareness among health professionals of the benefits of breastfeeding and available support services
4.1 Is there improved knowledge and understanding of breastfeeding and the role of the Breastfeeding Helpline?

Callers:

Is there more consistent information across services provided about breastfeeding

Are there appropriate sources of information about available support services

Service providers:

Is there clarity about the benefits of breastfeeding

What is the perception of the role the Breastfeeding Helpline fulfils

    • Number and type of breastfeeding skills development opportunities conducted by ABA for health professionals
    • Number and category of health professionals participating in breastfeeding skills development opportunities
    • ABA administrative data on skills development programs offered for health professionals and their participation
    • ABA Breastfeeding Helpline caller surveys
    • Survey of callers and counsellors
    • Interview feedback from DoHA maternity services stakeholders, professional associations and jurisdictions
    5. Expanded network of volunteers and health professionals aware of breastfeeding education opportunities
5.1 To what extent are ABA education and training opportunities taken up on the provision of advice and support for breastfeeding?

Provider:

How are education opportunities promoted and disseminated to the ABA volunteer workforce and relevant service providers

What strategies are used to encourage participation

Service providers (professional and volunteer workforce):

How relevant are the ABA education opportunities

How accessible are the ABA educational opportunities

    • Communication plan for promoting and publicising breastfeeding education opportunities
    • Strategies for encouraging participation in breastfeeding education

    • ABA administrative data and other documentation on rationale, frequency, nature, target and type of promotional activity undertaken to publicise breastfeeding education opportunities
    • Survey of counsellors
    • Interview feedback from professional associations and jurisdictions