Medical Research Future Fund Principles for Consumer Involvement Webinar – 3 August 2023

Dr Masha Somi:

Hello, everyone. I'm Masha Somi from the Health and Medical Research Office within the Commonwealth Department of Health. Welcome, everyone, to our latest Medical Research Future Fund webinar, and today we're focused on the Principles for Consumer Involvement in MRFF-Funded Research.

I'd like to begin by acknowledging the traditional owners of the lands we're all joining from today. I'm joining from Ngunnawal and Namburi Country in the Canberra region, however, acknowledge that we're all joining from lands right across the country. I pay my respects to elders and also acknowledge our Aboriginal and Torres Strait Islander colleagues who are here with us today. I also want to acknowledge the many strengths of Aboriginal-led research and researchers. And given today's topic for our seminar, I want to particularly call out the centrality of consumers in the community in Aboriginal-led research and acknowledge that we in the MRFF have taken many great learnings from these approaches and try to embed them across all MRFF funding opportunities.

So the presentation today is being recorded and it will be made available on the MRFF website.

Today, I'm joined by a wonderful panel, and that includes John Stubbs, Caroline Homer, Bernadette Brady, Yvonne Cadet-James, Merryn Carter and Tom Snelling. Both Caroline and I will do a brief introduction before handing over to Bernadette, Yvonne ... sorry, John and Caroline, and I'll do a brief introduction before handing over to Bernadette, Yvonne, Merryn and Tom for their presentations. I'll ask each speaker to quickly introduce themselves as they start their presentation and we'll have time for a Q&A session at the end of the presentations.

Many of you will know that we've been working to increase consumer involvement in research for many years. Within the MRFF, we took the step of appointing a Consumer Reference Panel early in 2022. The panel, which is led by John, provides advice on strategies for strengthening consumer involvement in MRFF-funded research. There are currently 12 members who are selected based on their skills, expertise and experience, existing and/or previous involvement in health and medical research, and their ability to represent the views of a balanced cross section of the community. More information about the panel is available on the MRFF website.

The panel has already provided invaluable input on MRFF processes in its first 12 months, and this includes the Principles that John will talk about shortly. In late 2022, the panel also helped us review and refresh the criteria for assessing MRFF applications. A key change was to make requirements for consumer involvement stronger and more explicit in the assessment criteria, including having a focus on lived experience, diverse populations and assessing consumer involvement across all aspects of research so that includes impact, methodology and capacity, capability and resources.

This is a quick overview of some of the great work that the Consumer Reference Panel has done in the last 15 months. Also, in addition to the work on the assessment criteria, the panel has also contributed to the design of the 2023 Consumer-Led Research Grant Opportunity. And as part of this, we're trialing an additional two-page statement focused on describing the roles that consumers will play in the proposed project, and also having multiple consumers on grant assessment committees, all as full scoring members. The panel has also been contributing to the joint NHMRC and MRFF Statement on Sex, Gender and Variations of Sex Characteristics in Health Medical Research. They've also contributed to reviews of our grant opportunity guidelines. They've provided advice on research priorities for investment, particularly through the MRFF Consumer-Led Research grant opportunity and some future investments that we're looking at, and provided advice on how to implement the Principles.

So I'll hand over to John in a minute to talk through the Principles. However, I just wanted to acknowledge that we see them as best practice principles and plan on working with stakeholders to support their implementation. And we'll also work with the National Health and Medical Research Council to align processes as much as possible. So I'll just hand it over to you, John. Thank you.

John Stubbs:

Thank you, Masha. And thank you to everybody who's joined for this very important seminar today. I'd just like to begin by acknowledging the traditional owners on the land in which I'm speaking from today, those in the Dharawal country, and I would also like to pay my respects to elders past and present. Thank you for that introduction, Masha.

So I'm chair of the Consumer Reference Panel and we've been operating since May of last year, and I have the honour to lead some 12 consumers who have great insight into the way in which consumer engagement in research should be undertaken in this country.

So we have a terms of reference and, first of all, to engage in why should consumers be involved in research in the MRFF? So the inclusion, the strengthening of the consumer voice will improve quality, relevance and impact of the MRFF research. And this is going to be through engaging with consumers to get more, safe, diverse and effective involvement. And I think recognising the value perspectives that of Australians who have a lived experience as patients, clients and carers and what that can bring to the research. And I think recognising the diversity of our population, the importance of including those with this varied life experience, and we have our members from Indigenous groups and also from the CALD community as members of my committee. And I think helping researchers and research organisations and give them some guidance into how they engage with consumers and what good consumer involvement would look like.

And I think the Principles for Involvement in this sector, we need to have a definition of what a consumer is, and so we've come up with a definition, and I don't think it's a new definition, but it's a coalescence of a lot of definitions out there. So a person with the lived experience as a patient, a client, potential patient, users of health service and/or providing support as a carer, family or community member. And I think the latter is very important, especially engaging with the family of patients because they understand what their loved one, what their patient has gone through, and they can bring that important perspective of being a carer to the research sector.

And so in the first 12 months, we've been able to establish Principles for Consumer Involvement in research, and these have been distributed to appropriate stakeholders, and we are awaiting feedback in relation to these. So we've developed six principles. We're asking that in every type of research, including basic science to involve with consumers. Consumers to be involved in all stages of research and especially from the outset of that research. Quite often in the past, consumers are considered as the last thing in relation to lodging applications and things, so we need to be brought in at the outset and at the start of this research.

And we want to be in partnership. We want to be co-partners with researchers through all of this research. And we have effective ways and means of engaging with the research community. And I think there's got to be sensitivity, there's got to be safety and engaging from not only a consumer or community point of view but also from the research point of view. And I think with broad diversity and equity in research in this country. If we look at the statistics of the number of people who have been born overseas and now making Australia their home, we have our Indigenous populations, and these community members are such an important part of the Australian community and therefore need to be part of the research community as well.

So thank you very much and I'd like to hand over to Professor Caroline Homer. Thank you.

Caroline Homer:

Thanks very much, John, and a warm welcome to everybody. It's wonderful to see so many people joining us online. My name is Caroline Homer. I'm the deputy chair of the Australian Medical Research Advisory Board, known as AMRAB. And I'm coming to you from Launceston today, which is Kanamaluka Land. I'd like to acknowledge the traditional owners of this beautiful part of Tasmania and acknowledge a long continuing history in this region.

We thought we would just give you a very brief outline of what AMRAB does. Many people probably won't be aware that there is advisory board that advises the Minister for Health and Aged Care on prioritising spending from the MRFF. We do this through a public consultation process to develop the Strategy and to develop the Priorities, and this is required of us under the MRFF Act.

So the work that we've been doing over the last three years, and there's a Strategy and Priorities that was released last year, and you can find that on the MRFF website, really had a strong engagement with consumers and took an important perspective around engaging with consumers. And so setting up the consumer advisory panel that John so ably chairs, and we have a fantastic group of people on that group really providing some really great advice going forward, and that's what we're going to talk through today.

AMRAB really supports the MRFF Consumer Involvement Principles that John has just taken you through. We had involvement and we're consulted by the group and asked for our advice. And there's been a very good conversation going backwards and forwards. And also, of course, with the National Health and Medical Research Council. There is now plenty of data that shows consumer involvement is really critical to get the greatest impact from our research and for us being funded under either NHMRC or MRFF, of course, this is taxpayer research and so it's really important that the community, that the consumers can see benefit.

The Principles look at what best practice looks like. We know for some researchers, maybe for many, this will be a challenge at this point and this will take time. As I think Masha said, it's not mandatory for now and it may never be mandatory, but it's a statement of future intent and a really good guide for us all to think about. And I think for all of us who do health and medical research start doing what we can now, there are lots of things that we can do now without considerable extra resources, without additional time perhaps. But it's really just thinking, where can I embed consumers all the way along in my research plan and not just at the end? And I think for me, this is what I've learned, is really engaging with consumers at an early stage, not at a late stage.

And look, it's learning and it takes time and it takes resources and it takes guidance as well. So while we'd like to get it all right, we also recognise that we can't let perfect be the enemy of good, and we can't wait until everything is perfect to actually do something. So we hope that you can get support from your organisations. This is not individual researcher only. This is the system, the funders, your organisations as well. And so I'm hoping there are lots of people on this webinar today who can take some of these principles and some of these issues going forward.

So thank you. That's all from me. I wish you all the best, and I'll hand it over to Bernadette Brady now.

Dr Bernadette Brady:

Thank you, Caroline, and thank you to MRFF for the invitation to come along and to speak in this webinar today.

So I'm a clinician researcher, a physio clinician researcher, and my area of expertise is really working in chronic disease and chronic pain management and specifically working with people from culturally and linguistically diverse backgrounds. I too would like to acknowledge the traditional owners of the lands on which I'm presenting from today, the Dharawal people, and I'd like to pay my respects to elders past, present and emerging.

And so I'm going to present today about our experience working together really as a partnership between people who are from or consumers from culturally and linguistically diverse backgrounds. And in particular, we have representatives from Arabic, Assyrian, Vietnamese, Samoan, Chinese and Fijian communities, our research partners and our clinicians to really deliver an MRFF consumer-led research funded project that seeks to improve the chronic disease management for people from CALD communities.

So before I go into a little bit about some of the how's and the what we're doing in terms of engaging consumers as part of our research program, I want to just give some context as to where this motivation emerged from and how we've transitioned to this phase. So for anyone on the call today who's not familiar with Sydney, where we work in Southwestern Sydney is considered one of the most multicultural areas in Australia where over 50% of the local population speak a language other than English at home, and this is higher in some of our local government areas.

We're also a region of considerable socioeconomic diversity, and some of our local government areas are considered among the most disadvantaged in New South Wales. We're a region of high refugee resettlement, and indeed one of our local government areas resettled more than three times the number of humanitarian entrants than any other local government area in New South Wales.

And so the point to this is really to say that with this broad social and cultural diversity comes challenges for health. And in particular, we can see this manifest in a disproportionate burden of disease, with specific culturally diverse communities, and we can see that there are differences in belief systems and values and that can impact on how people engage with healthcare. And ultimately, engagement is what drives outcomes, particularly around chronic disease management, which is where our research and our clinical areas focus. And so approaching a program of research from this perspective, knowing that culture matters and knowing that engagement is critical to outcomes, it's perhaps not surprising that we found ourselves in the thick of working with consumers and trying to engage communities as part of our research program.

And so what did we do? Well, I'm going to walk you through basically our process. I really liked that Caroline said, don't let the perfect be the enemy of the good, because when we began doing this, we didn't really know what we were doing. And we certainly didn't think at the time or plan ahead thinking that what we were doing was partnering with consumers as part of the research process. We really wanted to try to tackle the problem of chronic disease management. It probably isn't surprising to many people on the call today that I don't identify with a culturally diverse community and so already, I'm coming into this topic and this area as an outsider. When we're approaching communities, I'm also an outsider because I'm a healthcare practitioner and I'm a researcher. So there are differences in power relationships that we need to be mindful of.

And so our first step towards working with consumers and communities from diverse backgrounds was really about building relationships. And we spent the time, so  close to a year before we began our first research program or project was going out to different multicultural organisations, meeting with different groups and forums and trying to have a bit of a presence in the community and understand some of the health challenges they were having, particularly around the topics of chronic disease where my area of expertise was focused.

Now, having done this, it really helped to sensitise us that our health systems don't really incorporate culture that well or embedded as part of clinical service delivery. And so this really led to the second phase of our process towards production, which is to really understand those consumer voices and consumer perspectives. And so what we did to address this was we conducted a series of qualitative interviews and focus groups with diverse communities, so Arabic, Assyrian, Hindi and mixed culturally diverse communities to really explore and understand their perspective of the problem that we were focused on, which was, in this case, pain. Now, the point here is not to talk about all the different studies but to say that because we had spent some time building those relationships, we had broad reach to the communities to conduct these focus groups.

And the second part was that a key emphasis on qualitative research, which is the type of research that I've started doing and I spent about half my time doing now, is to go back to your participants in your interviews and focus groups and to talk with them and share with them about your interpretations of those interviews. And so it was through this process of member checking that we did on a larger scale by holding community forums and bringing community members back to hear about how we were understanding their perspectives and whether we were indeed accurately capturing that. That actually gave us the biggest ideas for change.

And so that's where we came to next, is because we were holding these discussion sessions and forums, we were able to get some ideas about how do we tackle this problem of chronic pain and chronic disease management. And so this led to two pilot studies that really looked at ways that we could improve healthcare delivery. One being a culturally adapted pain management program, and the second being the focus of where we're at with our MRFF application today, which is looking at a peer mentor approach to supporting people who have chronic disease as they engage in healthcare.

So thinking about where we are I guess at this point, we still have only started to transition from consumers as participants in the process. We've done some community consultation, but we're still not really at the point of true partnership in the full research process. And so about this time was when consumer engagement research started to really pick up momentum. And what we realised is that we actually had a really unique opportunity here. We had a strong presence with the community. We had engaged members who were with us from the beginning and were happy to have conversations with us from the beginning. And so we sought to embark on this particular pilot study, which is now the focus of our MRFF application, by actually meaningfully engaging consumer members in the research process and in the research activities.

We were lucky enough to secure some seed funding that allowed us to formalise this process of consumer engagement. And so we formed a consumer advisory group that helped us through all phases of the research design for our consumer-led research grant. So we'll talk a little bit about some of the activities and some of the things our consumers are working on that have actually helped make us successful in our current application and hopefully successful in the completion and outcomes from our program.

So what the consumer advisory group has perhaps not surprisingly directed some activities towards is many of you would be familiar with the participant information sheets and consent forms, and lots of consumers find these challenging and difficult to engage with. And so not only have we tried to adapt them in simple language enough to still meet the ethics requirements, but to be, I guess, better than our standard templates. But our consumers really wanted us to produce videos in language for our Arabic and our Assyrian and Vietnamese communities so that they could overcome some of the myths and misconception about research participation in the first place. And so these videos are really a complement to our traditional participant information sheets and consent forms, and that was done before we began the research project to describe what's involved in health research, to try to address some of those myths and misconceptions and to be visual aids of people that they felt that they could relate to.

Our consumer advisory group are very interested in the design of the intervention. And so in this case, what we're doing is a peer mentoring intervention. So here, we're actually using consumers as partners in the delivery of the intervention. They are actually our intervention, people who have lived experience of the chronic disease, people who identify with the target community that we're trying to improve outcomes for, and they are trained and then they support patients who are enrolled in our research program as they embark on their chronic disease management.

And so here we have consumers both embedded in the research to deliver the intervention, but we also have the consumer advisors to help shape how that program is being delivered. And so our advisors in particular helped us to design the intervention. They helped us to think about what supports and training people would need, how would we go about recruiting people to become mentors, to support ...

Dr Bernadette Brady:

... Go about recruiting people to become mentors, to support our health services because that was critical to the success of our project. And lastly, thinking beyond just getting the project done and getting the outcomes done is they want to share this as we go along with the community. And so, they've made us really think about and develop project promotional materials that they play an active role in sharing with their community to talk about what this particular health service is doing and how we're trying to work together to improve outcomes. And so, there are just a couple of examples of some of the activities our consumers have been meaningfully engaged in the research process, but this is a process and it's taken us a long time to get to this point where we've actually formalised consumers as partners in the research project, and not just participation, but it does require that continuous negotiation and renegotiation around roles and how to best work together. And so, we're still learning through doing.

So, what are the things that we've learned so far about successful co-production of research with consumers, particularly from diverse cultural backgrounds? And so, the first thing I've taken away from this whole process is that it really does... We do need to invest time, that working with consumers really does require you to invest and recognise the benefits are going to be best realised when you make the time and you invest in their development and meaningful engagement rather than coming in at the end and expecting people to be able to make a big contribution when they haven't been part of the process. It's about recognising that consumers have different motivations behind why they want to get involved in research. And so for us on our consumer advisory groups, we've got some consumers who are motivated by the intervention, giving back to the community, being able to provide peer support to patients.

For others, they're really motivated by learning about research because they see that as ways we can find new solutions or new avenues for optimising care. And so, recognising that not everyone has to look the same and not everyone has to have the same motivation, and that those different lenses really do improve how we look at the challenges and the problems emerging. It requires flexibility. And so, not all consumers want to receive written materials that they have to mull over and provide comments on. Some people prefer verbal conversations, lunch and have a chat about the different aspects and provide verbal feedback. Indeed, putting the MRFF application and getting our consumers on Sapphire was, again, a challenge in flexibility about how we would go about doing that and how we support consumers to fulfil these roles and ensure we get the most out of their unique skills and expertise. And the last point is really to recognise that it's about capacity and the literacy building.

So when we started working with consumers, consumers that we were working with, very few had any understanding or idea of what research was. And so, it's trying to find out again, what their motivations are and what opportunities do we have, what role do we have to upskill, so that we can harness their unique skills and expertise. And so again, that's providing opportunities for them to learn about the research process. Our consumers are learning, or a portion are learning to do some aspect of qualitative analysis to help improve our interpretation of the data.

And so, that's a bit of a quick whirlwind about our experience working with culturally and linguistically diverse consumers as partners in our MRFF research project. And I'd just like to conclude by acknowledging our wonderful consumer advisors who've been pivotal in getting us to this point and helping to shape the direction of our projects, as well as our community partners, multicultural health services, and our research bodies, so thank you.

Yvonne Cadet-James:

Good afternoon, everyone, and I'd like to thank the MRFF for the invitation to be part of this webinar today on a very important topic. I'd like to start by acknowledging the traditional owners and custodians of the land on which we meet today and where I present, the Gimuy Walubarra Yindinji and Yirriganydji people of the Greater Cairns area. Also, acknowledge the traditional owners and custodians of the lands across the nation where participants are located. I acknowledge ancestors and elders, the creators and holders of unique knowledge systems who have cared for this land over time and have made a significant contribution to the nation and its people.

As a way of introduction, I'm a member of the Gugu Badhun Nation and co-director of the Gugu Badhun Research Centre. I have a background as a registered nurse and midwife and spent 45 years in the field of education and health. After moving to academia, I've worked in health sciences, public health, and Indigenous health research. My research interests include empowerment, social-emotional wellbeing, mental health, maternal, adolescent, and child health, and research ethics and impact.

Today, I've been asked to talk about engaging Aboriginal and Torres Strait Islander people as consumers in research from the perspective of being a chief investigator on several MRFF grants. In relation to the MRFF Principles for Consumer Involvement in Research Funded by the Medical Research Future Fund, the honourable Mark Butler, MP Minister for Health and Aged Care has stated that the objective of the Medical Research Future Fund is to improve the health and wellbeing of Australians. Our collective ability to achieve this is enhanced by effectively involving consumers in the prioritisation, design, conduct, translation, and evaluation of research funded by the MRFF, as well as in the selection of funded research projects. Consumers bring a broad range of valuable perspectives and experiences such as diversity in culture, linguistics, gender, and ability that can improve the quality, relevance, and impact of research.

This is certainly true of Aboriginal and Torres Strait Islander people who continue to share their rich culture, their knowledge systems, science, and respect for the land and the environment. It is acknowledged that Indigenous people are the oldest researchers in the world, conducting research into all aspects of life for survival to gain and build on existing knowledge, and discover new knowledge. This included research that has resulted in inventions, innovations, and the development of unique knowledge systems in the areas of science, physics, engineering, medicine, astronomy, agriculture, and anthropology to name a few areas. Aboriginal and Torres Strait Islander people continue to use their research skills on a day-to-day basis to address some of the most complex issues they face in their everyday lives. And while involving consumers in research is a relatively new concept in the Western research world, in the Indigenous research world, this is usual practice given our social and cultural obligations and goals to achieve human rights, social justice and self-determination.

To explain my experience in engaging Aboriginal and Torres Strait Islander people as consumers in research, it is necessary to provide some context behind what it means to be an Aboriginal person working in the field of Aboriginal and Torres Strait Islander health research. Aboriginal and Torres Strait Islander consumers are our families and communities, so I'll use the term community rather than consumer. We belong to our nation or language groups as traditional custodians of the lands which are traced back to our ancestors. We are members of complex kinship systems, which govern our social organisation and determines our commitments and obligations within our kinship groups. We have in common connection to the land, sea, waters, animals, plants, and all parts of the environment. We are bound by common rules of social order, engagement and respect across nation and language groups, which underpins how we conduct research.

In my work, the common process for engagement of communities is to apply the principles of co-production. This means researchers, practitioners, and members of the community collaborating to talk about priorities and needs to make decisions and develop research plans. This involves researchers and end users for community working together as peers to ensure that the purpose of the research, the research methodology, and the application of research outputs are relevant to and appropriate for the end users. Community engagements throughout the research process is the key to successful collaboration. Aboriginal and Torres Strait Islander researchers will often have existing relationships in communities because of our kinship context or because family names are known or because we have made contact with key stakeholders and grown those relationships over time.

Because wherever we work, we have an obligation to community to assist in improving health and wellbeing outcomes. This means understanding community priorities and needs, community profiles and dynamics, and the historical, social, cultural, economic, and political context of the communities who may be involved in the research. The first point of call is making connections with elders, councils, other relevant stakeholders to discuss research possibilities, conducting presentations at group and community meetings, giving detailed information to communities, so they can make informed decisions about their willingness, readiness, and capacity to be involved in the research. If interest is confirmed, then groups and communities are involved in all aspects of the development of the research protocol, the ethics application, and the research process.

Crucial steps in the partnership is to identify the roles of community members in the research, including roles of chief and associate investigators, advisory group members, project and research officers and community researchers. Early steps include identifying employment opportunities along with what training is required in the areas of engagement, recruitment, data collection analysis, writing, knowledge translation, and strengthening capacity and capability of community members and researchers. Collaboration continues with the development of the research plan and research agreement, determining roles and responsibilities, engagement and feedback processes, milestone meetings, training implementation, and mediation processes. Collaboration and involvement is extended to research benefit, impact, and translation, and groups and communities are the key decision makers in determining how the research will benefit the community and/or broader society, what impact will the research have on the economy, society, environment, or culture, and what is the best way for integrated knowledge translation to occur throughout the research process for the co-production of knowledge is exchange and translation into action.

Last but not least, joint reflections on the planning and conduct of the research. What lessons have been learned, not only for the benefit of the researchers to improve practice, but also for communities to further their own research agendas? And while this might seem like a simple and streamlined process, it's of course complex and time-consuming, requires extensive planning and monitoring, being flexible to meet the needs of community. It means consideration for community politics, priorities and needs, and the extra burden of research on top of coping with demands on their everyday lives.

It's however extremely rewarding and a privileged to not only work on the ground with communities, but have the benefit of their lived experience, rich culture, and knowledge systems in making a difference. It's important to note that we talk about the need for equity in the power balance between researchers and consumer communities. This can't happen if the consumers or communities don't understand the research process to effectively engage in the process of making informed decisions. Part of my commitment is to provide research training sessions for Aboriginal and Torres Strait Islander traditional owner groups, organisations, and other groups, and assist them in developing their own research strategies and protocols, set their own research agendas so they can meaningfully engage in research based on their own terms. Thank you.

Merryn Carter:

Hi, my name's Merryn Carter. I'm a health consumer advocate and I'm speaking to you today from Wurundjeri land in the Naarm Melbourne area. Just firstly a bit about my history and how I came to be involved with health consumer advocacy. I was diagnosed in 2010 with HER2 positive breast cancer. And as part of my recovery experience I guess, and reflecting on what it meant for my life, I decided I would like to give back to the medical science that had provided the drugs that are responsible for my current state of health. So in 2014, I joined the WEHI, the Walter and Eliza Hall Institute Breast Cancer lab as a consumer advocate working with Professor Geoff Lindeman and Professor Jane Visvader and I work with them still today. And in 2018, I joined the consumer advisory panel of Breast Cancer Trials Australia, New Zealand, and I just attended my sixth annual scientific meeting with them last month in Auckland.

And then last year when the MRFF Consumer Reference Panel was getting going, I expressed interest in joining that and have been part of the work with the MRFF ever since. So, today I'm going to talk about how consumers are involved in basic health research, like the work I do with the Walter and Eliza Hall Institute and Breast Cancer Trials, and then to reflect on why that matters. So my two examples of consumer involvement, as I've already mentioned, with Breast Cancer Trials first. Breast Cancer Trials is one of the first research related bodies, I guess, to establish a consumer advisory panel, and that was done in 1998. So, they've been going a long time. Walter and Eliza Hall Institute established its consumer program in 2013, and I joined in 2014. That program has grown from an initial six consumers to currently requiring 103 consumers to be involved with more recruitment necessary because the demand is growing.

So, how are consumers involved in basic health research? So at Breast Cancer Trials, we on the consumer advisory panel review so much of the material from the early concept stage through the protocols, the lengthy dense descriptions of how the trials will proceed, and then the patient information and consent materials, which are increasingly in video form. The members of the consumer advisory panel share the load of the review work. Sometimes it's quite considerable and we contribute as our time allows because there are over seven or eight of us active, which is good. We try and get at least two or three people involved in each particular piece of work.

Some of the trial protocol documents can be extremely long and complex, and I want to acknowledge that my experience with the WEHI lab has provided great training for that work. They're scientific concepts that we are kind of required to understand the basis of, even though we don't understand the science of it. We also review trial designs and provide the critical participant perspective, which helps to ensure that participation is as comfortable, safe, and convenient as possible, which is important, obviously, for maximising recruitment to clinical trials.

And then when we review the patient information consent forms or the videos, we help ensure that they're as easy to understand as possible, which also increases the likelihood of trial participation. At the Walter and Eliza Hall Institute, researchers are now assigned teams of consumers to prevent consumer fatigue. So when I started off, I was the only consumer associated with the breast cancer lab, and that's supporting the work of Geoff Lindeman and Jane Visvader, but also all of their colleagues, researchers, their whole team of PhD and undergrad or PhD candidate researchers. So, it can be a lot of work at times, and I now have two colleagues who have joined me. So, there are three of us working with that lab. So, this allows us to support each other, to spread the workload, really important at grant application review time when there's a lot of detailed applications to go through.

And of course, we each bring different perspectives and skills to the table, so we get a range of comments from different perspectives. We're involved in reviewing the public presentations, for example, conference posters to ensure clear language. And we act as advocates for WEHI, we can act as mentors for early career researchers, and we can speak alongside our researchers when discoveries are made. We meet quarterly with our research teams to discuss what they're doing, keep up to date with research progress, and I really value those quarterly meetings. Sometimes there's a lot of dense science presented to us in as plain language as possible to help us keep on top of what they're researching.

So, the quarterly meetings ensure that our knowledge of the research aims, techniques, and outcomes is current, and it also enhances the researcher's ability to communicate their research in lay language, which is increasingly important for applications. This discussion provides us opportunities to ask questions, exchange ideas, ask for clarifications, and again, we share thoughts and experiences from different perspectives. And we do find ourselves sometimes reminiscing back to our own treatment and we ask the researcher how what they're doing might impact the experience of future breast cancer patients.

If this research proves successful, how will that change the experience that I went through? I think that's one of the biggest motivators is if we can avoid people going through onerous chemotherapy in the future. Having done that myself, it'd be wonderful to offer that to future patients. So, two of the projects from WEHI that have received recent MRFF funding include the Glimmer Research Program for glioblastoma, which is an aggressive brain cancer, which has very low survival rate, and a WEHI-led collaboration with Peter Mac to develop a cancer vaccine for patients with limited treatment options. So, there are high level consumer reference groups associated with both of those projects.

So, why are we involved in basic health research? So I think as was mentioned earlier in one of the intros, it's now widely accepted that consumers and community members add value to health and medical research and indeed have a right and responsibility to do so. And I think one of the principles that really chimes with me is there should be nothing about us without us, very important. It grounds the work of the researchers in real world perspectives and in our lived experiences of health conditions and health and medical research is after all a public good. We all pay for it with our taxes. So, accountability and transparency are really important and consumers are one way of providing that sort of public accountability and being involved.

And when consumer involvement is genuine, when there's mutual respect and we are well-supported, there's a really positive impact on both the consumers and the researchers on the research process and on the outcomes, and for WEHI and for Breast Cancer Trials, the outcomes are really significant in terms of research discoveries or drugs proven to be effective. So, there's a clear role for consumers to contribute to setting the agendas for the research, planning, and developing research direction, and then in communicating and disseminating the research outcomes. And also, of course, in my experience, consumer involvement helps researchers write better grant applications and communicate the research aims, plans, and results more clearly. And this helps better communicate the role of medical science and discovery to the public, which is a really important thing to do.

Thomas Snelling:

Thank you very much. I'm Tom Snelling. I'm a clinician researcher. I live and work on Gadigal] land and I pay my respects to the elders past and present. Most of my work is in helping other clinical researchers and public health researchers to design and implement their studies, and we do some studies of our own, and I'm grateful for the chance to share with you some of the experience that I've had over the past decade in working with consumers in research, and that experience has been at different levels. The MRFF Principles call for ongoing productive and two-way relationships, and I've been involved in projects where the involvement and engagement of consumers has been-

Thomas Snelling:

... Where the involvement and engagement of consumers has been just once off to inform very specific decisions or to address very specific questions about projects. I've also been involved in a number of studies where that involvement has been ongoing through the course of the project, through either research buddies, consumer buddies, attached to specific projects or as part of project specific consumer reference groups. But I've also been involved in projects where the involvement of consumers has been ongoing and beyond the life of a single project. And this is the very mature end of the spectrum of consumer involvement. Similar to what Merryn has just described, my greatest experience of this has been with the community reference group of the West Pharma Centre of Vaccines and Infectious Diseases. And that is a very mature reference group that's been going for a number of years now under the leadership of Cath Hughes. And they're involved not only in addressing specific questions about projects, but all the way through to making and helping to make decisions about what projects are supported, and even starting to set the agenda and priorities for research. I've also observed consumers acting purely in an advisory capacity to researchers, but I've also been involved in projects where consumers are not only advising, but they also have a seat at the steering committees for specific projects or research initiatives. So, they're actually directly involved in the decision making.

The MRFF Principles call for a diversity of perspectives to be reflected. And I just share my experience with BEAT CF, which is a big project in cystic fibrosis management that I'm involved with. That project is informed by consumer reference groups and under the guidance of Mitch Messer who coordinates the consumer involvement for that project. There are two reference groups. One is a reference group for adults who are directly affected by cystic fibrosis. And we have a separate reference group which is comprised of parents of children with cystic fibrosis, but also partners and carers of people with cystic fibrosis, acknowledging that they also have experience to contribute to guide research in cystic fibrosis. As part of those reference groups, there's been a solid attempt to ensure that we have both men and women on those reference groups and that we have people both from urban settings across Australia, but also from regional and more remote places as well.

I think when we aim for diversity of perspectives, we also have to be careful not to isolate or alienate people. We've had a lot of challenge with involving adolescents. For example, in our reference groups, we tried very hard to have a standalone adolescent reference group, which was difficult to sustain. So, now we have a couple adolescents on our adult group, but I'm mindful that they feel that that is a safe space for them and that they feel well-supported. And I'm sure that others, for example, people from culturally diverse settings may feel the same way, that if they're just a single person on a reference group, that they feel well and adequately supported in that role. Wherever possible, we try to have at least a couple members from minority groups.

I've experienced consumers being involved in priority setting for research. So, this is really at the most mature end of the spectrum of consumer involvement. I think as John mentioned right at the start, this really needs to be done well before grants are submitted and awarded. And this is where you do have a very mature model of consumer involvement, where their priorities are reflected in the sorts of proposals that are put forward in the first place. And this can be done either in a very informal way and that's what I've been most familiar with. But I'm also have observed very formal structures and procedures for priority setting and the James Lind Alliance, which is a UK based initiative, but also crosses borders.

I know have been involved in these large formal priority setting initiatives for things including in cystic fibrosis. And that's certainly been helpful with our own work. But when we do consider those priorities, we also have to consider whether those priorities are shared by local people, including Australians affected with the particular condition. I've been involved and observed consumers being involved in the design of clinical research. And perhaps one of the most fundamental ways I've seen consumers involved is in the framing of the fundamental research question. I think, as a scientist, clinician scientists often tend to look at problems in terms of testing a specific hypothesis, whereas it's more natural, I think for consumers to think about research informing decision making.

And I think that that is a very helpful perspective and can help drive better quality research. For example, the question that we might pose is, should a person with a specific condition ‘Y’ be treated with a new drug instead of the existing standard of care? So, this is a ‘PICO’ approach to framing research questions. And consumers can be involved in all aspects of picking apart that research question in terms of determining what it actually means. Consumers can also be helpful in terms of helping to frame whose decision we are trying to inform with the research that we're about to do. And I think of this in terms of the perspective of the policy maker who might be making a decision about an overall policy, or a guidance, or funding for a particular treatment across a whole population of people with a given condition.

Versus, the perspective of a clinician who might need to make a decision about that intervention for a specific patient with a condition, as opposed to a whole of population approach. Versus, the perspective of the consumer who may need to make a decision about whether they do take a treatment, if it's recommended for them or prescribed for them. And so, the perspective that you take helps to frame the question and helps to frame the design of the study. I've seen consumers involved in the determining the intervention that should be studied in a research project. Recent example is in a study I've been involved in which is offering financial incentives to people with hepatitis C to commence therapy.

And consumers have been involved in firstly advising on the appropriateness of that as an intervention, but also advising on the range of values of that incentive that have been used in that project. In another study, consumers have been involved in advising in the framing of text messages that are sent to parents to encourage them to vaccinate their children on time. So, consumers have been involved in crafting the nature of the intervention in those interventional studies. Consumers have also been involved in selecting the outcomes. So, this is how different interventions are evaluated. How do we make those comparisons, and in particular in determining outcomes that are relevant to them? So, this is patient-centred benefits versus toxicities and costs.

And a good example of this is the SONG initiative that the group involved in chronic kidney disease research have undertaken a large project, both with clinician stakeholders, but also with many people with chronic kidney disease, to define what outcomes should be assessed in studies of kidney disease. I've also experienced consumers involved in informing how research is conducted. So, this is the when, the where, the how and by whom participants in studies are identified, how they're notified of the project or referred to the project, how they're informed about the project and how they're consented to be in the study. Including how interventions are delivered in that study and how data about participants is gathered, including how testing is performed.

And a good example of where this has been very helpful to us is at the ORVAC project, which was a study amongst remote Aboriginal children in the Northern Territory, looking at evaluating a vaccine program which involved taking blood tests from babies to measure their responses to a vaccine. And it was originally intended or planned that many hundreds of babies would have blood tests in order to be able to identify the smallest important improvement in vaccine antibody responses with some certainty. And when we were challenged about the need for blood tests in so many babies, we were encouraged to go back to the drawing board and redesign this study in a way that allowed us to minimise the number of babies who would need blood tests.

And that involved having multiple analyses, interim analysis of the data with stopping rules to stop taking blood tests as soon as we had answered our question, rather than meeting an arbitrary enrolment number. And that I think greatly contributed to the success of that study, because it made it much more attractive to mums and dads to enrol their babies and to take part. And finally, of course, dissemination, the when, the where, the how and by whom the study results are reported back to the participants in the study, but also to important stakeholders, including people with the target condition, as well as the broader community. And consumers have really helped for us to work out the sequence of that process and how results are revealed in a meaningful way.

So, I will end there by thanking and acknowledging the dozens of consumers who have helped inform the various projects I've been involved in. But I'd like to mention a few people in particular who have really been instrumental in my approach and my journey over the past 10 years. Mitch Messer, who's really informed our involvement of CF consumers in our cystic fibrosis work. Ada Parry from the Northern Territory, who has strongly guided our approach to involving consumers and stakeholders in our projects in Aboriginal child health in the Northern Territory. Cath Hughes from Western Australia, who's informed a lot of our vaccine and infectious diseases research across Australia. And importantly, Anne McKenzie from Western Australia, who really started me off, encouraged me right from the start about how to involve consumers in research. Thank you.

Dr Masha Somi:

Thanks, everyone. What a great and insightful presentation. So, please use the webinar chat function to send through your questions. I've already got a couple that I'll start to work through. Also, if there's a particular presenter that you'd like to respond to your question, just include that in your question just so I know who I can refer it to. So, a quick question, asking if the recording will be available. Yes, we'll make the recording and the presentation available on the MRFF website as soon as we possibly can. And a question about ethics. And I will pass this one to Yvonne, if that's okay, given your interest in research ethics. So, the question is, I'm interested to know if and how people structure ethics approval for consumer engagement in research design. Often the grant application that people would like consumer input into happens prior to ethics approval of that specific project. Is there a specific umbrella ethics approval for consultation with reference group on projects or any other mechanism? Yvonne?

Yvonne Cadet-James:

Sorry, could you just repeat that again? There was a bit of noise in the background.

Dr Masha Somi:

Oh, I'm sorry, Yvonne. So, there's a question about how to structure ethics applications and approvals for inclusion of consumers in the research process, with a comment that often that the ethics approvals happens before or the consumer input is required before the ethics approval or after. So, is there a way to consider how to incorporate consumer involvement in ethics?

Yvonne Cadet-James:

Thanks, Masha. Yes, from an Aboriginal and Torres Strait Islander point of view, because it's an expectation that consumers or communities will be involved. It is actually part of the ethics process. And the question would be asked, if that didn't happen, why isn't it happening? And that set out certainly in all our guidelines about Aboriginal and Torres Strait Islander ethical procedures.

Dr Masha Somi:

Thank you, Yvonne. Perhaps I can ask this question of you, Bernadette. It's a question about how to include consumers, noting that a lot of the engagement needs to happen before the grant stage, the grant funding comes on. Are you able to talk a little bit about how you manage that part of the process that is to include consumers before the formal grant commences?

Dr Bernadette Brady:

Yeah, sure. Thanks for the question. I think it's a really good point. So, I think, again, just coming back to one of the key principles, is don't let the perfect be the enemy of the good. So, we didn't wait till we had all this infrastructure in place and these resources to begin consulting consumers. A lot of consumers want to give back and want to be able to contribute their time and their efforts if it's going to benefit their community, particularly from communities that are from what we call collectivist cultures. There the wellbeing of their community is very much highly valued. So, having some key representatives who are happy to volunteer and to answer one of the other questions, part of that has been we've onboarded them through our health district as volunteer consumer representatives, rather than participants at that point.

And so, it's trying to get that balance between what capacity people have to give and knowing that we have certainly at one point no resources to be able to reimburse or recognise their time, and making sure we're not overburdening them with what we're asking of them. And then to be actively trying to apply for funds and having that built into your funding application that you will recognise and reimburse their contribution if that comes to fruition, but otherwise minimising the burden or the impact that you have on them or the time required. And it's that negotiation process. So, it's trying to understand, well, what capacity do they have to give? Where do they see their strengths or what's the best contribution, the best use of their time? And showing, making sure it's a reciprocal relationship.

So, we may not always be able to pay people, but what else could they get out of that? What are they going to get from that relationship? So, sometimes it's in-kind support. So, I would go out to community groups and give a talk about physio, because that's what I can do. I can't pay them, but I can give something of myself to make that relationship reciprocal. In other situations, it's providing training for them. So, giving them opportunities to develop themselves that, again, if we can't pay someone, then there's trying to work out what their motivations are and how we can recognise them, and value that contribution, and show them that we value that contribution. And that overwhelmingly carries a lot more strongly then than providing someone with a gift voucher or monetary reimbursement.

Dr Masha Somi:

Thank you, Bernadette. And just building on that point, Tom, perhaps I might send a question your way. So, there's two questions that can be responded to probably together. One is around how do we make sure that consumers are fairly compensated for their time and lived experience or expertise that they bring to the research work? And coupled with that, a question about consumer related budget items that are appropriate to include. So, maybe just your experience on the type of consumer related budget elements that you've put into your proposals.

Thomas Snelling:

Look, I feel strongly that consumers should be compensated for the time and effort and expertise that they contribute. I think that's just respectful to do that, and not to expect that people will just volunteer their time without compensation. I've been very fortunate that a number of the reference groups that I have been involved in have been based at the Telethon Kids Institute. And they provided quite clear guidance around the expectation of what a reasonable level of compensation would be. So, we had a fairly well established practice of offering honoraria to consumer members. And I know I think that the New South Wales Health Consumer Forum also has recently provided advice about what a reasonable compensation looks like.

Thomas Snelling:

Yes, about what a reasonable compensation looks like for people to review documents and to sit on reference groups. But generally we would pay for a number of hours for a meeting attendance and a number of hours for any preparation for those meetings, and also offer compensation for time spent reviewing documents. I think for some areas there are well-established reference groups and those reference groups that will have procedures around how they recover their costs. And so that makes it relatively straightforward. But if you're setting up a reference group from scratch, for example, for a specific project, there is some advice that is available around what reasonable compensation looks like. I think one of the challenges that we've sometimes had is around setting up those payment processes. Some organisations make it easier than others, so sometimes it can be a lot of work to pay someone an honorarium for a little piece of advice. It can almost be more work to set it up than it's worth. But I think these are challenges that I've observed.

Dr Masha Somi:

Thank you Tom. Now John, I know this is a topic that's very dear to your heart. Would you like to comment on the issue of ensuring there's budget and payment and remuneration available for consumers?

John Stubbs:

Look, I think Tom's really nailed it. There are two areas of thought. Some organisations don't want payment for their consumers because they feel that that might muddy the waters and we do have to handle conflicts of interest in relation to that. But again, I think that if community has been asked to give its input and to give its advice, then yes there should be appropriate measures made to remunerate them in some way or another. One of the issues that arise with the age of some of the consumers there are a ATO, there are taxation elements that need to be considered by some of them. So that shouldn't negate paying something in some way.

And as I said, it might be an honorarium, but I think there is a general drive to ensure that consumers are recompensed for their engagement and their involvement in research. Whether the NHMRC can see whether or not an element of payment in the budget should be for consumer or community engagement. I know there are ways that some researchers engage with that and involve consumers in the payment. So I think we've started a very good conversation. I think there are elements that need to be addressed, but I think in the long term, consumers need to be compensated in one way, shape, or form, and remuneration is a good way. Thanks.

Dr Masha Somi:

Thank you John. Now Merryn, a question for you. This is a call-out from the audience, a specific question for you. What advice would you give to consumers who want to be involved in health and medical research?

Merryn Carter:

Oh, that's a great question. So I can't speak highly enough of the experience that I've had at WEHI because they've provided training for consumers from day one. I was also very fortunate to be mentored by the late Avis Macphee who was the consumer associated with the breast cancer lab at WEHI when I started. So I guess that's two things, is choose an organisation that has a great consumer training program that can provide the kind of support that you will need to develop the knowledge and the expertise. And if you can find a mentor, someone who's already an experienced consumer. Both of those things made a big difference to my getting involved and I guess almost unsaid it should be the area in which you have personal experience. So for me, breast cancer was the no-brainer because I'm a breast cancer survivor.

Dr Masha Somi:

Thank you Merryn. A question for Bernadette, a comment that it was a wonderful presentation and a question, did you always need an interpreter to engage with core consumers? What would be other ways around with around engaging core consumers with limited English proficiency when you do not have funding to hire an interpreter?

Dr Bernadette Brady:

Yeah, okay. That's a really good question. So I think it varies about the levels of engagement with our advisory group. Our advisory group are bilingual, so they have English language skills, that said they're not all proficient in written English language skills. So we do have to have flexibility in how we engage them in the different aspects of the research process. As we're engaging our consumers in understanding their perspectives and bringing their perspectives in to shape our questions and our priorities and our interventions, then it's partnering with the right support. So when I began my research career, I had no money, I had no budget. So part of going out to organisations is building a presence, forming networks who can help support that research, what bilingual community workers are invested and engaged in that and how can we help each other and work together.

And so forming partnerships with multicultural health, forming partnerships with organisations like core community services where there are bilingual staff that are interested in shared outcomes or shared vision for what you want to achieve can be a really useful way when you don't necessarily have a budget for interpreters at the outset. And then as soon as you do have opportunities to apply for funding, then that's the bulk of our budget, is about in either interpreters or bilingual staff that are trained and have that ability to provide that support. So I think just to summarise, it shouldn't be a barrier. It should be about, "Well what are the opportunities I can have to partner with people? How can I make that arrangement reciprocal? If I don't have money to pay them, then how do we have a shared output or a shared experience where I can give to them and they can help support that process?" So hopefully that answers the question.

Dr Masha Somi:

Thanks very much Bernadette. Tom, I might ask you this question and that's because you talked about in your presentation sort of continuing and ongoing relationships with your consumer groups. And the question is how do research groups keep consumers engaged and feeling valued over periods when their input is not actively required?

Thomas Snelling:

So I think there are almost always ways that consumers can be involved. So I think some of the groups that I've been involved in the past, being involved in the agenda setting and the priority setting has been very important to them. So I think there is a tendency for us to come to consumers for their input to review participant information sheets and consent forms. And I think while that's a very important way that consumers can be involved, it's very important to realise that consumers can also be involved in broader agenda setting, which can be a very interesting and rewarding part of their consumer involvement process. I also think, and that it's important as a researcher that when you have obtained advice from consumers that you first show that you've understood the advice that's been provided.

And that you feed feedback to that group or to those consumers, how the advice has been taken on board and what's happened as a result of it. It's not always possible to fully reflect their views or the perspectives of consumers sometimes for reasons of ethics or just practicality or logistics or whatever. And I think if that's the case, it's important to go back to those consumers to let them know why decisions have been made. But I would imagine that it would be frustrating to consumers when they've put a lot of effort and time and thought into providing advice. If that advice is not, doesn't appear to be reflected in the research. And if that's happened, I think it's just respectful to let them know and I think makes consumers much more likely to provide and continue to provide advice.

Dr Masha Somi:

Thank you very much Tom. And I think John, you might have some thoughts on this as well. Would you like to make some comments?

John Stubbs:

Yes, thank you. Look, I think Tom nailed it. It is about communication, just letting consumers know that we've received the information from you guys, we've assessed it, we've looked at it, thank you. On this occasion we can't do what is proposed but next time, et cetera. So I think it's just maintaining that communication and it can be just a brief email once every couple of months saying that the elements, the research organisation, sorry, the research projects that you're working on, we are developing more and more and we'll seek your input at a later. But I think just keeping that communication stream open is the key to this because you do wonder what's happened to your thoughts and your ideas and your suggestions because you put in time to make this information and feedback. So I think maintaining that communication stream is very important. And as I said, it just needs to be a brief email from time to time just saying that you're still involved and we still appreciate what you're doing. Thank you.

Dr Masha Somi:

Thanks John. I might go back to you with this next question as well that we've heard from a number of people with lived and living experience that they don't feel comfortable with the term consumer as this doesn't fully incorporate their role as contributors. And people may feel that the term consumer frames them in a passive role. Do you have any thoughts or ideas about how to navigate this?

John Stubbs:

Well, that's very interesting. Look, I think the term consumer has been thrown around in circles I think for about the last 10 years. But what other name can we come up with it? Can it be community involvement? I think the term consumer is in the lexicon across the whole health spectrum. And I think if we try and change it or maybe we need to think or come up with a better name, but at the moment there's not a lot that we can do. A consumer is a person who uses health services. Consumer engagement or consumer involvement probably is the better term because that means that you are involved and you are seen as a partner.

So a consumer standalone, no, consumer engagement, no, but consumer involvement I think is the key to making sure that you're seen as an equal partner. But that goes with the way that the process is done. If you're engaged from the outset or involved from the outset, then you are part of the team. If you are bought in as an adjunct or as a tick box well into the research program, then that doesn't work for a lot of consumers. So consumer involvement I think is the term that we need to use and involve us from the outset. And I think that will satisfy, I believe most people. Thank you.

Dr Masha Somi:

Thank you John. And Yvonne, I wonder if I can ask you comment on this topic as well. I noted in your presentation you didn't use the word consumer, I think you talked about community. Would you like to comment on this?

Yvonne Cadet-James:

Yes, thanks Masha. And commonly we talk about communities because that's part of who we are. Our families make up our community, so commonly we use the term community rather than consumer.

Dr Masha Somi:

Thank you. Okay, Merryn, a question for you. What strategies have you used at WEHI at the Walter and Eliza Institute to create a safe environment for consumer involvement that is sensitive to the other demands in their treatment journey?

Merryn Carter:

Wow, that's a good question. It's not really probably best directed at me because I'm just a member of the consumer group at WEHI. I'm not managing the program. We do have staff who manage the program, who do a fantastic job. And I mean one of the things that to ensure consumer safety, if you like, that's been introduced since I've been involved, I think training is really important. And the training that WEHI provides is not just for the consumers who are involved, it's also for the researchers. So it needs to be both sides training to understand that there needs to be mutual respect. Mutuality requires both sides. So I think providing that environment as WEHI has done where the researchers that we're being to engage with have also been trained to understand the role of consumers has helped. And also having multiple consumers associated with each researcher or each lab means that we are not under huge pressure when there's something that needs reviewing quickly.

Or indeed just maintaining that relationship and having the quarterly meetings if you're the only consumer and then if you do need time out for your own health, it's great because you've got colleagues who are sharing that load with you and that relationship with the researchers. So I think the multiple consumers per lab or per researcher has been a great innovation. There's also been lots of role descriptions and, if you like, paperwork developed around the whole consumer program since I've been involved. So it's become much more formalised and protected by the organisations, the way the organisation looks after all its staff and all the people who are engaged with it. So I think the roles need to be formalised, they need to be documented and in a way treated it a bit like you know are a valued team member, which you are just as you would look after your staff.

Dr Masha Somi:

Can I ask John, would you like to comment on that? So the question is do we move away from the term compensation and towards remuneration as a way of reframing that is done?

John Stubbs:

Well, as far as I'm concerned, remuneration has always been what... everything been discussed under the heading of remuneration. And remuneration includes time used but also reimbursement of expenses. No, I don't like the term compensation because I think that's something that's way out there. But you're being remunerated for the work that you've done in relation to this project and with these people. So therefore I think the term remuneration is much better and should be used.

Dr Masha Somi:

Thank you. We're running-

John Stubbs:

Thanks Masha [inaudible 01:26:24].

Dr Masha Somi:

Thanks John. We are really running close to time. I might ask, there's a question about what sort of strategies have worked really well for recruiting and engaging participants in research and Tom, I wonder if I can maybe hand over to you or ask you to comment on some of the strategies you've been able to use.

Thomas Snelling:

Look, I think this is very hard and I think it does depend a little bit on the specific condition. For some things like our work in cystic fibrosis, I know certainly having a champion or someone who's already well respected in that community helping to coordinate those activities has probably been the most useful thing that we've done. So I mentioned Mitch Messer, very well respected in the cystic fibrosis community, very well known. He became involved in our cystic fibrosis work very early on and really has been a champion for driving consumer involvement in that project. And obviously he already had a lot of... was very well-connected in that space and was really able to reach out to people. So I think that's probably, where possible, if you can identify someone within the community that you're trying to target who's already well known and established is successful. But having said that, I think it can be hard and we're almost always looking for people to involve in the various reference groups for the projects that we've been involved in.

Dr Masha Somi:

Thank you very much Tom, and a big thank you to the whole panel for the great presentations and also for the answers to the questions, I'll just hand over to John to do the thank you and farewell.

John Stubbs:

Thanks Masha. And to each and everybody online, I would like to thank you for first of all your interest and your engagement in the process and also the interesting questions that have come through and hopefully addressed to your satisfaction. And finally to my co- speakers and presenters for their time and engagement and to Masha for your great moderation of this webinar. Running a webinar of this size for us is easy because we're at the front, but there's a whole lot that goes on behind the scenes. And I would like to thank all those backend people, pardon the phraseology, but all of those people behind the scenes who've enabled this seminar to run so smoothly. That's where the work is done and I thank you for your efforts. And once again, thanks to all for being part of this, what I think has been a very interesting, engaging, and important seminar and I wish you well for the rest of the day. Thank you very, very much and cheers.