Review of the effectiveness and validity of operations of the MAIF Agreement:Research Paper

Appendix A - Data collection and stakeholder engagement method

Page last updated: 18 June 2012

An outline of the data collection and stakeholder engagement method used to gather the necessary evidence to answer each of the review questions is provided below.

A.1 Data collection plan

Nous developed a detailed data collection plan to guide the collection of evidence needed to complete the Review. The data collection plan identified a hierarchy of review questions to address each of the review components included in the conceptual framework (see Figure 2).

Nous identified four likely sources of the data and other evidence that would be required to answer each of the review questions. The sources identified in the data collection plan included:
  • DoHA documentation
  • APMAIF complaints data
  • Literature review
  • Stakeholder consultations.

A.1.1 Stakeholder engagement plan

Nous developed a detailed stakeholder engagement plan to identify which stakeholders were likely to be able to provide the data and evidence required to answer the review questions. Eight categories of stakeholders were identified:
  • MAIF Agreement signatories
  • APMAIF members
  • Complainants
  • Government policy and regulatory agencies
  • Consumer groups
  • Health professionals and organisations
  • Industry bodies (including non-signatories)
  • Members of the general public.
For each stakeholder category, Nous considered the most effective mechanism of engagement to obtain the required data and other evidence. It was determined that a combination of targeted1 interviews and two surveys (one targeted and the other general) would ensure effective and timely engagement with all stakeholder categories.

Interviews

In collaboration with DoHA, Nous designed a structured interview guide to ensure consistency between interviews (see Appendix B) and to allow direct comparison of responses. Each interview was completed in accordance with this interview guide2 . The interview questions were designed to provide qualitative data to answer many of the review questions.

Nous conducted 24 interviews (which included the five APMAIF Members, 11 Commonwealth and State Government departments and bodies, five of the six3 manufacturers that are signatories to the MAIF Agreement, one industry representative, and two consumer groups). Twelve of the 24 stakeholders interviewed also completed the targeted survey (see below). A full list of interviewees is at Appendix A.

After each interview, Nous sent a summary of that discussion to the participant to review and to confirm the accuracy of the interview record.
Top of page

Surveys

In collaboration with DoHA, Nous developed two surveys – a targeted survey and a general survey – to elicit views from the stakeholders identified in the stakeholder engagement plan.

Nous ensured that the format and range of questions in both surveys facilitated ease of response and analysis (for example, by using agree/disagree questions to capture quantitative data). The surveys were also designed to enable respondents to freely express their views (qualitative responses), while focusing them on the specific purpose of the Review. A copy of the general survey is included at Appendix C.

Both surveys asked essentially the same questions, however, the general survey allowed respondents to remain anonymous or, alternatively, to provide their contact details to assist follow-up.

The first survey was sent to a targeted cohort of 40 stakeholders (the targeted survey).

The targeted survey was sent to the 26 organisations that were offered an interview. 12 completed responses were received from this cohort.

The targeted survey was also sent to a further 13 stakeholder organisations that were not scheduled to be interviewed by Nous but were identified by DoHA and Nous as interested parties that may be affected by the outcome of the Review, or be in a position to provide insights into the Review. These recipients were asked to respond to the survey on behalf of their organisation. Five completed surveys were received from this cohort.

The second survey provided an opportunity for individuals to contribute to the Review (the general survey).

A link to the general survey was made publically available on the APMAIF website by DoHA.

Nous also provided a link to the general survey to those stakeholder groups that were invited to complete the targeted survey (for those groups to provide to members that might wish to make a personal contribution to the Review).

Finally, the APMAIF Secretariat provided a link to the general survey to a representative sample of previous complainants (i.e. individuals that had previously submitted a complaint to APMAIF over the last five years). 4

Nous received 516 completed responses to the General Survey. Not all respondents completed each question. The majority of respondents were ‘consumers’ (326), followed by ‘health professionals’ (148). The remaining categories were ‘Consumer group’ (28) and ‘Other’ (14).

Draft Consultation Summary

At the conclusion of Stage 2 (Stakeholder consultation), Nous compiled a Draft Consultation Summary and sent this to all stakeholders that had participated in the Review and had also identified that they would be willing to be contacted by Nous/DoHA.

Recipients were invited to provide feedback on accuracy of the contents of the Draft Consultation Summary on the basis that any feedback received would be considered in compiling advice and recommendations to DoHA. Nous received 15 responses.

A.2 Other data sources

DoHA documentation

DoHA provided Nous with a range of background material that was directly relevant to the Review. This material included:
  • International Comparison Study into the Implementation of the WHO Code and Other Breastfeeding Initiatives, conducted by University of Sydney’s NHMRC Clinical Trials Centre (University of Sydney 2011). This report was commissioned by DoHA
  • Australian National Breastfeeding Strategy 2010-2015
  • Australian National Infant Feeding Survey Indicator Results 2010
  • APMAIF Annual Reports, including APMAIF Interpretations of the MAIF Agreement.
DoHA also provided a list of references that Nous might consider in undertaking the Review.

APMAIF complaints data

Nous was provided with the following data and documentation by the APMAIF Secretariat (DoHA):
  • Initial complaint forms (with identifiers removed) ranging from 2009 to 2010
  • Timelines of APMAIF’s handling of complaints in 2008
  • APMAIF complaints handling process
  • Summary of complaints (including out-of scope complaints) for 2010-11
  • APMAIF Meeting Minutes (July 2009 – November 2011).
Top of page

Literature review

Nous conducted a general literature review of a range of documents to inform the Review, including:
  • the WHO Code
  • previous reviews of the MAIF Agreement including the Knowles Review (2001) and the Best Start Inquiry (2007)
  • Australian and international journals, and thesis papers
  • Background material and additional references provided by DoHA
A complete bibliography is included in Appendix H.

A.3 Data limitations

The successful implementation of the robust data collection and stakeholder engagement plans ensured that sufficient data was collected to effectively answer each of the review questions. There were, however, four minor data limitations encountered.
1. Not all stakeholders agreed to participate in the interview process – one signatory to the MAIF Agreement declined to participate in an interview. It is possible that that signatory may have been able to offer additional insights to Nous in completing the Review.
2. Several of the 40 stakeholder groups invited to complete the targeted survey did not respond. It is possible that the groups who did not respond may have been able to offer additional insights to Nous in completing the Review.
3. A link to the general survey was provided on the APMAIF website and was also provided to those stakeholder groups that were invited to complete the targeted survey (for those groups to provide to members that might wish to make a personal contribution to the Review). It is possible that some consumers may not have seen or received the links and therefore not had the opportunity to contribute to the Review. It is possible that those individuals may have been able to offer additional insights to Nous in completing the Review.
4. Obtaining cost effective, quality information about the dynamics of the Australian infant formula market was difficult. The information contained within this Research Paper represents the detail deemed necessary to inform the Review and is not proposed to be a ‘market analysis’.

These minor data limitations did not affect the effectiveness of the Review.

A.4 Targeted stakeholder consultations

The following organisations were invited to participate in an interview and were also invited to complete the targeted survey:
  • APMAIF Members (All)
  • Australian Breastfeeding Association
  • Australian Consumer and Competition Commission
  • Department of Health and Ageing
  • Food Standards Australia New Zealand
  • Health Department - Australian Capital Territory
  • Health Department - New South Wales
  • Health Department - Northern Territory
  • Health Department - Queensland
  • Health Department - South Australia
  • Health Department - Tasmania
  • Health Department - Victoria
  • Health Department - Western Australia
  • Infant Nutrition Council
  • Lactation Consultants of Australia and New Zealand
  • National Health and Medical Research Council
  • Signatories to the MAIF Agreement – including:
    • Abbott Australasia
    • Bayer Australia
    • HJ Heinz
    • Nestle
    • Nutricia
    • Pfizer
The following additional organisations were invited to complete the targeted survey:
  • Australian College of Midwives
  • Australian Healthcare and Hospitals Association
  • Australian Nursing Federation
  • Bellamy’s Organic
  • Choice
  • Consumer Health Forum of Australia
  • Food and Grocery Council
  • Milk Powder Solutions
  • Murray Goulburn
  • Public Health Association of Australia
  • Royal Australian and New Zealand College of Obstetricians and Gynaecologists
  • Royal College of Nursing Australia
  • Tatura Milk
Top of page
1 Due to the broad range of stakeholders that have an interest in the operation of the MAIF Agreement, a decision was taken by DoHA to limit the targeted consultations undertaken to those stakeholders directly involved in the MAIF Agreement and policy makers.
2 The only departures from this consistent approach were the additional questions asked of the ACCC and FSANZ in relation to their respective regulatory roles.
3 One signatory declined to participate in an interview.
4 Complainant details were not disclosed to Nous. Participation was voluntary and not monitored.