All of the 14 RCSs and 11 UDRHs were consulted during the evaluation. Interview participants for the individual sites included Deans of medical and health faculties, professors and Heads of schools, other university and academic staff (teaching/research, administrative, IT and facilities), representatives of general practice networks (Divisions), staff of regional training providers, clinicians and managers of State-funded health services, staff of Aboriginal health services, representatives of rural workforce agencies, representatives from community boards of UDRHs and RCSs, community members, and students. Consultations took place in person or by telephone with follow-up interviews conducted as required.

In addition, a large number of stakeholders external to any individual institution, such as the Federation of Rural Australian Medical Educators (FRAME), the Australian Rural Health Education Network (ARHEN), professional bodies, rural workforce agencies, and others, were invited to participate, and a number of officers within the Department of Health and Ageing were also interviewed.

Appendix B summarises the range of participants included in the consultation.

2.3.1 Interview methodology

As Fontana and Frey (2005:695) point out, interviewing is 'inextricably and unavoidably historically,
politically, and contextually bound.' The process of undertaking an extensive consultation with a wide range of stakeholders, from students to academics to government officials to consumers, means that the process needs to be flexible enough to make each participant feel included and able to speak freely, while being bounded enough to ensure that information collected is consistent and amenable to aggregation and analysis. It is also essential, in an iterative process such as a consultation, that information is collected and analysed with rigour to ensure it reflects the multidimensional perspectives of those interviewed. For these reasons, a process was designed which required data to be analysed at several stages and with different members of the evaluation team, providing a number of opportunities for triangulation of data sources and perceptions. Triangulation refers to the comparison of qualitative data sources for credibility and, as has been noted (Walkerdine, Lucy and Melody 2002:189), this process is essential when different perspectives are provided regarding the same subject.

Detailed interview guides were designed to direct specific questions to participant groups; there were seven different interview guides designed for:
  • Deans, Heads of Schools/Departments and senior academics;
  • RCS academic and administrative staff;
  • UDRH academic and administrative staff;
  • key health stakeholders;
  • key community stakeholders;
  • students; and
  • government stakeholders.
The interview guides were structured specifically to facilitate an open-ended interview process, in which the participant was invited to provide her/his perspective and to focus on the areas of most concern. The process, however, was iterative, and key themes which arose from earlier interviews were often introduced into subsequent interviews in order to test their appropriateness to different contexts. Interview topics were condensed from the framework document under four broad headings:
  • national rural health workforce outcomes
  • relationships between the Programs and between the Programs and related initiatives
  • enabling and limiting factors
  • future directions.
The general interview guide, provided to all participants, is included in Appendix C.

2.3.2 Consultation schedule

The number of potential stakeholders who might have been included in this consultation was vast. In
order to control this within the time available, and to ensure that stakeholders who were most involved with the Programs were consulted, each UDRH or RCS site was asked to assist with the identification of participants. A list was drawn up naming potential stakeholders, from Deans of medical faculties through to staff and clinical stakeholders to community or consumer groups. The actual range of people consulted differed at each site depending on whom the site determined was most appropriate, and who was available during the timeframe of the consultation. Where the site was unable to arrange a schedule for interviews, Urbis staff made the arrangements after receiving a list of potential interviewees.

There was a danger of bias inherent in the decision to ask RCS and UDRH Heads to determine who should be included in the consultation. It was reasonable to expect that those people who were most involved in the Program and who might therefore be most enthusiastic and positive about the Program would be included. Urbis sought to minimise this potential bias through ensuring that a large number of external stakeholders were also consulted, such as Federal and State/Territory government staff, rural workforce agencies, rural health peak bodies, and regional training providers, and also to follow up on further suggestions of people to interview who might not have been readily included in the initial consultation. In practice most people were readily able to assess both the strengths and weaknesses of the Programs.

The schedule for consultation with the participating universities is provided in Table 4 below. (See also the consultation list at Appendix B.)

Table 4 – University consultation schedule

University (contract holder)

RCS

UDRH

Date

University of Western AustraliaKalgoorlieGeraldtonMay
University of Adelaide
University of South Australia
WhyallaWhyallaJune
Australian National UniversityCanberra -
University of SydneyDubboBroken Hill
University of New South WalesWagga Wagga and Albury -
Flinders UniversityRenmarkWarrnamboolJune/July
James Cook UniversityMackayMt Isa
Monash UniversityBendigoMoeJuly
University of MelbourneSheppartonShepparton
University of QueenslandToowoomba -
University of NewcastleTamworthTamworth
University of TasmaniaBurnieLaunceston
University of Sydney - LismoreAugust
University of WollongongWollongong -
Flinders UniversityAlice SpringsAlice Springs