The National Breastfeeding Helpline Evaluation report

2.4 The evaluation tools

Page last updated: 13 February 2013

Evaluation tools utilised in this project comprised:

  • Breastfeeding Helpline caller survey tool;
  • Breastfeeding Helpline counsellor survey tool;
  • stakeholder discussion guides; and
  • focus group questions.
Development and use of each of the tools is described below.

Caller survey tool

The caller survey tool was designed to be a principal source of new data for the evaluation, capturing the characterisitics and needs of callers to the Breastfeeding Helpline and their experience of using the service.

This new data contributed to information about utilisation of the Breastfeeding Helpline, levels of satisfaction with the service received, and the impact of the Breastfeeding Helpline in driving greater knowledge and understanding of the benefits of breastfeeding.

The design of the caller survey tool was informed by existing survey tools currently deployed by the ABA as part of their own quality assurance methods, and recent research conducted on behalf of the ABA by StrategyCo. This ensured a level of consistency between the data sets, but also minimised confusion amongst callers who may have responded to quality surveys undertaken periodically by the ABA.

Development of the caller survey tool was undertaken in close consultation with the Department and ABA. The final tool was approved by the Department.

Implementation involved extensive communication through the ABA to volunteer counsellors who were rostered on between 28 May and 17 June 2012, informing counsellors of the purpose and intent of the survey, and providing direction on how to alert callers to the existence of the survey.

The caller survey tool was hosted on the homepage of the ABA website. A short preamble outlined the intent and purpose of the evaluation and survey, and a link to the online survey was provided.

The caller survey tool is included at Appendix B.

Counsellor survey tool

The counsellor survey tool was designed to be a principal source of new data for the evaluation capturing the operation of the Breastfeeding Helpline from the counsellor’s perspective, the utilisation of the Breastfeeding Helpline by callers from different backgrounds, and providing an insight into skills, education and qualifications of the ABA counsellor workforce.

This new data assisted in determining how appropriate and effective the implementation of the Breastfeeding Helpline has been, utilisation of the Breastfeeding Helpline, levels of satisfaction with the service provided, adequacy of training and education for counsellors, and the impact of the Breastfeeding Helpline in driving greater knowledge and understanding of the benefits of breastfeeding.

The design of the counsellor survey tool was informed by existing survey tools currently deployed by the ABA as part of their own quality assurance methods. This minimised potential for confusion amongst counsellors who may have responded to previous surveys and built on quality measures determined by the ABA.

Development of the counsellor survey tool was undertaken in close consultation with the Department and ABA. Feedback and suggestions were incorporated where appropriate, before final endorsement of the counsellor survey by the Department.

Implementation of the survey involved extensive communication through the ABA to volunteer counsellors rostered on between 28 May and 17 June 2012. Emails were sent from the evaluation team to all the volunteer counsellors who had at least one shift rostered on during the survey period. Reminder emails were sent at weeks two and three of the survey period.

A link to the survey tool was emailed directly to each of the rostered on volunteer counsellors.

The counsellor survey tool is included at Appendix B.

Stakeholder discussion guide

The iterative and ongoing dialogue with members of the Department and stakeholders from the ABA during the life of the project was guided by the requirements of the evaluation project.

Discussion guides supported consultations with BJOG members and representatives from relevant professional associations. The guides provided a brief background to the project and outlined suggested areas of discussion.

These consultations sought to explore the level of coordination between the Breastfeeding Helpline and the broader service system operating in each jurisdiction, and whether the Breastfeeding Helpline had contributed to improved breastfeeding outcomes in the community.

The Department endorsed the discussion guides prior to their distribution. The discussion guides are included at 0.

Focus group questions

The purpose of the focus group discussions with different population groups was to examine utilisation of the Breastfeeding Helpline by mothers, and, where appropriate, to determine their levels of satisfaction with the Breastfeeding Helpline.

Accordingly, the discussion guide questions were designed to elicit information from the participants about their awareness and experience of the Breastfeeding Helpline. A further consideration for the focus groups was the expectation that mothers had not used the Breastfeeding Helpline. To address this issue, questions were also designed to explore other sources of information and support that mothers may have used to assist them in making decisions about how to feed their baby.

The focus group questions were designed to be exploratory in nature, and to suit the format of an informal group discussion. They were drafted by the evaluation team and endorsed by the Department prior to their distribution in facilitating focus group organisation and discussion.

A copy of the focus group discussion guide is included at 0.