Better health and ageing for all Australians

Interim Evaluation of the Northern Territory Aboriginal and Torres Strait Islander Community Aged Care Workforce Development Projects - Attachments

6. Evaluation Approach

Up to Publications

prev page| TOC |next page

6.1 GGJ Evaluation Approach

GGJ aims to conduct the evaluation of the Workforce Development Projects by analysing both quantitative and qualitative data.

Quantitative Data

Quantitative data was collected by the Registered Training Organisations (RTOs) and made available to GGJ for analysis and evaluation. Quantitative data will be collected for all service providers in the NT and all staff participating in workforce training. The quantitative data collected will provide a broad perspective of changes experienced by communities and individuals in the NT before, during and after the Workforce Development Projects.

Qualitative Data

GGJ employed a case study approach for the collection of qualitative data. Qualitative data collection strategies are interactive and allow the natural flow of events and processes to be discovered. Most qualitative research deals with people's individual and collective social actions, beliefs, thoughts, and perceptions. A number of communities were selected as case studies for the evaluation. These communities will be consulted before and after the workforce training; telephone consultations prior to training delivery and an onsite visit following training. During these interviews/visits GGJ will aim to interview:
  • Shire members
  • Service provider coordinators
  • Aged care workers
  • Health service personnel.
By interviewing a number of different people from each community, the evaluation will ensure the impact of the Workforce Development Projects is broadly explored.

The case study method will provide a snapshot of the experiences at both an aged care service provider and individual level. It will give a broad view of training and workforce development on services and individual workers and will allow the individual experience will be placed into context with their community and service provider.

6.1.1 Community Selection

Case study research often uses information-orientated sampling as opposed to random sampling. With this technique no generalisations to a population beyond cases similar to the one studied can be made. For the evaluation of the Workforce Development Projects, communities with differing characteristics will be selected as case studies.

Communities will be selected considering the following criteria:
  • Location (shire)
  • Size of community
  • Service shire operated or Independent
  • Type of programs funded (HACC, CACP, Flexi)
  • Program Management Issues.
  • Other information has been sourced to assist GGJ in understanding the current situation in the selected communities and the NT as a whole. Consultation with DoHA and state and territory government personnel, discussions with community aged care contacts and review of recent literature regarding visits conducted to a range of communities through other government projects such as the recent six monthly report completed by the Coordinator-General for Remote Indigenous Services has been conducted76. This report provides an overview of the challenges faced by the remote communities related to governance, early childhood, education, economic participation (including CDEP conversions), health, housing and community safety.
Three of the 29 priority communities included in the abovementioned report were scheduled to be included in the case studies (Maningrida and Galiwin’ku and Yirrkala). Due to the challenges faced by remote communities (such as lack of coordinators, rescheduling of training due to community needs etc), GGJ has had to adjust the selection of case studies; Maningrida and Galiwin’ku have now been replaced with Angurugu and Ti Tree.

6.2 Case Study Research

Case study is a research strategy that involves an in-depth longitudinal examination of a single case. It provides an in–depth study of a particular situation rather than a sweeping statistical survey and allows researchers to study complex events within their contexts77. Researchers have used the case study research method for many years across a variety of disciplines. When case study research is applied correctly it becomes a valuable method for health science research to develop theory, evaluate programs, and develop interventions78.

There are six steps to case study research, these include79:
  • Determine and define the research questions
  • Select the cases and determine data gathering and analysis techniques
  • Prepare to collect the data
  • Collect data in the field
  • Evaluate and analyse the data
  • Prepare the report.
Critics of the case study method believe that the study of a small number of cases can offer no grounds for establishing reliability or generality of findings80. On the other hand, it is argued that a case study provides more realistic responses than a purely statistical survey81.

Flyvbjerg 200482 identifies in his work entitled, Five misunderstandings about case-study research, that ‘the case study is a necessary and sufficient method for certain important research tasks in the social sciences, and it is a method that holds up well when compared to other methods in the gamut of social science research methodology.’83

Case study research is ideal when the objective is to find the greatest amount of information about a case, not necessarily a typical or average response. Random sampling emphasises representativeness and will seldom produce insight into ‘extremes’. Case study methodology has been selected to contribute to the evaluation as we believe that it offers the opportunity to explore the range of workforce issues and provide insight into the range of challenges that rural and remote communities face in delivering community care and developing their workforce.
Top of page

6.3 Case Studies Examples with Aboriginal Communities

Most research in Aboriginal communities has been based on survey methods. In 2004 Tchacos and Vallance84 conducted a study using a qualitative interview methodology with cultural sensitivity as its fundamental principal. Methods employed in this study included:
  • Face to face interviews with Indigenous people
  • Participant observation
  • Qualitative interviewing.
Qualitative research methods were utilised because they were considered appropriate and sensitive to the cultural and language differences that exist in Aboriginal communities in Western Australia85. The researchers believed that qualitative approaches were more likely to yield information that could provide an in-depth understanding of the questions posed by the study.

Another project utilising this methodology was The National Family Homelessness Project that was conducted 2002. This project was a longitudinal study of Aboriginal homelessness in Perth, Western Australia86. The project was 15 months in duration and examined the effects of homelessness on a target group of 61 Aboriginal families. This study employed both quantitative and qualitative data sources. Methods included87:
  • Relationship-based interviewing
  • Participatory observation
  • Collection of information through questionnaires and interviews.
These two examples of case study research support our methodological approach.

6.4 Ethical Research and Community Consultation

Aboriginal and Torres Strait Islander cultures operate within defined ethical frameworks. Human research ethics are a set of principles to help guide researchers to develop and conduct research in a way that is safe, respectful, responsible, and of high quality88.

The National Health and Medical Research Council have developed a set of guidelines for ethical conduct in Aboriginal and Torres Strait Islander health research. The six values at the heart of these guidelines are89:
  • Spirit and Integrity
  • Reciprocity
  • Respect
  • Equality
  • Survival and Protection
  • Responsibility.
Observing appropriate protocols when working with Indigenous people and their communities is critical to establishing positive and respectful relationships. Consulting with Indigenous communities should always be seen as a two-way process, with both parties learning together and from each other90.

Best practice methods for effective consultation have been developed by the Australian Housing and Urban Research Institute. They suggest the following as effective actions for cross-cultural consultation91:
  • Clarify the process of consultation with the community, regional agencies and service providers early on in project inception
  • Develop a clear methodology for communicating with communities including protocols for engagement being mindful of; community and family hierarchies, knowing who to talk to and when, living arrangements on site and modes of transportation
  • Provide immediate and ongoing feedback through informing the community of the methods and expectations of the consultation processes and developing visual materials to be left with the community for ongoing internal discussion as an outcome to every meeting
  • Ensure the relevance of planning meetings for the community and consultants and service providers through delivering on the promises agreed upon during consultation
  • Promote involvement ‘on the ground’ to effect relationship building and to encourage community ownership of projects
  • Consider developing consultancy frameworks with the involvement of cross-disciplinary teams to balance technically and socially determined planning priorities
  • Consider that best practice may not be about determining the ‘right’ outcome, but rather to propose directions to work towards improved outcomes. To recognise that consultation is being undertaken in culturally, environmentally and economically changing contexts
  • Understand that the local council is the forum for ideas where the continuation and transfer of culture from the old people to the young people occurs. Input and ownership of ideas occurs when the community congregate at the place where the elders sit, talk and draw
  • Understand that a likely outcome from a lack of consultation will be a dysfunctional building, unless built environment projects are worked out with the community to gain understanding of local factors such as seasonal issues and physical conditions.
The project plan developed by GGJ in the conduct of the evaluation demonstrates the extent of the community consultation and consideration that will be implemented in the project and reflects the abovementioned principles.
Top of page

76. Commonwealth of Australia 2009 Coordinator General for Remote Indigenous Services Six Monthly Report July-November 2009 Canberra
77. http://www.experiment-resources.com/case-study-research-design.html
78. Baxter P and S Jack 2008 Qualitative Case Study Methodology: Study Design and Implementation for Novice Researchers The Qualitative Report 13(4): 544-559
79. Flyvbjerg B 2004 Five misunderstandings about case-study research Qualitative Research Practice C Seale G Gobo J F Gubrium and D. Silverman London: 420-434
80. Soy S K 1997 The Case Study as a Research Method University of Texas at Austin: http://www.ischool.utexas.edu/~ssoy/usesusers/l391d1b.htm
81. http://www.experiment-resources.com/case-study-research-design.html
82. Ibid
83. Ibid
84. Tchacos E and R J Vallance 2004 Research in Aboriginal communities: cultural sensitivity as a prerequisite The University of Melbourne
85. Ibid
86. Roberts C 2004 National Family Homelessness Project- A Longitudinal Research Project on Aboriginal Homelessness in Perth Western Australia Centrecare
87. Ibid
88. Australian Government National Health and Medical Research Council, Australian Health Ethics Committee et al Keeping Research on Track: A guide for Aboriginal and Torres Strait Islander peoples about health research ethics
89. National Health and Medical Research Council 2003 Values and Ethics: Guidelines for Ethical Conduct in Aboriginal and Torres Strait Islander Health Research Investing in Australia's Health Commonwealth of Australia
90. Board of Studies New South Wales 2008 Working with Aboriginal communities: A guide to community consultation protocols Board of Studies NSW
91. Lee G and D Morris 2005 Best Practice models for effective consultation: towards improving built environment outcomes for remote Indigenous communities Australian Housing and Urban Research Institute

prev page| TOC |next page