Medical Research Future Fund Research Administration Officer Webinar – 27 March 2024

Misha Hutchings:

Good afternoon and thank you for joining us today. I'd like to begin by acknowledging the traditional custodians on whose lands we are joining from today throughout Australia. And I'd also like to pay my respects to elders past, present, and emerging, and to recognise their rich cultural and educational practices that benefit all of us who get to call Australia home. My name is Misha Hutchings. I'm the director of the Grants Management Office for the Medical Research Future Fund, and I welcome you to the fourth MRFF Research Administration Officer or RAO webinar.

This is an ongoing MRFF offering that the Department of Health and Aged Care holds twice a year. The session today will cover responses to some questions that we receive directly from you, and these will be presented to you by MRFF directors. Toward the end of the webinar, we'll have a period for Q&A, so please begin sending through your questions now on the right-hand side of the screen so that we can get to as many as possible during that time.

We'll begin now with the intent and purpose of the webinars. So, the aim of these webinars is to provide practical information to connect MRFF staff with research administration officers and just to talk about in particular MRFF administrative arrangements. It's also to assist potential grantees to better understand MRFF funding requirements. Lastly, we like to seek feedback from RAOs with a view towards continuous improvement of MRFF grants administration, and really just to aid the implementation of any changes that we make as a result of the feedback that we received from you. I'll now hand over to my colleague, Ruth Griffiths, who is the director of the Patients and Infrastructure Section in MRFF.

Assoc. Prof. Ruth Griffiths:

Thank you, Misha. And hi, everyone. I'm really glad to be talking to you today about some topics, trying to address some of the myths and some of the questions that we get around our processes and procedures. So, I will be taking the lead on two of the sections in today's webinar. Unfortunately, a director of our Science Unit wasn’t able to join us today, so I’ll be taking on her section as well. But just to start off with, we’re going to look at some of the questions that we get around the application stage. So next slide, please.

So, one of the questions that we do get a lot is, why do we have two grants hubs, how do we make the decision which grant opportunities go with each grants hub? And it’s not a straightforward, well, it’s a bit boring actually, to be honest. It’s mostly administrative. All MRFF grants awarded from the Department of Health and Aged Care are required to use a grants hub. That’s actually the case across the Department. It’s a government policy that we use grants hubs, so we have to use a grants hub. You’ll have noticed that a lot of the number of grant opportunities have increased quite rapidly over the last few years, and so there’s actually a challenge with just loading them all on one grants hub at once.

And so, there's some practical reasons why we just have two grants hubs that we use. How do we decide which grants go through which hub? There's a few different things that play into that, but sometimes it's based on expertise. For example, our commercialisation-type grants go through the Business Grants Hub predominantly, but the others, there's a variety of reasons. We do have a number of grant opportunities that are repeated through grant hubs.

So if they've gone through NHMRC or BGH before then usually we use them again. But there's also capacity. We have to have a discussion with capacity with each grants hub about their capacity. So NHMRC sometimes comes back and says, "We're at capacity for the number of grant opportunities.” Business Grants Hub we have to also talk to as well. So it's not that interesting or exciting in many ways. It's just a reality that we have to go through a grants hub and then we decide which one we're going to go with.

There is an intent to try and harmonise those either just at the policy or process level, but also over time we flag that we are looking at whether we can move them all over to NHMRC or at least the majority of the grant opportunities. But it's not a small thing to shift a large amount of grant opportunities over to one hub. So that's why you might hear about it but not see anything happening for a little while. So I hope that helps. It's not that exciting, it's much more of an administrative reason. So next question, next slide, please.

We do get a number of questions around things about “can I include consumers as Chief Investigators?”, and other things about chief investigators as well. So, I've just sort of clumped these on one slide here. So firstly, there is a perception that you can't include consumers as Chief Investigators. Yes, you can. We actually want to promote true inclusion of consumers as true participants in the research. And so yes, you can. Yes, we know Sapphire is not set up for a consumer. What about the top five publications, et cetera, et cetera? So firstly, you can set up a profile in Sapphire as a consumer. If you are unsure how to do that, have a chat to the NHMRC Help Centre.

You can ignore the track record pieces around publications and grants and those sorts of things. It's not a requirement that they're there for a chief investigator. It's there to help the grant assessment committee assess the suitability of the team to deliver the project. So, you can actually use the one page per CI section to outline what the consumer does bring to the project. And it doesn't have to be publications, it doesn't have to be research impact and those sorts of things. Although if they have them, obviously they can include them. So yes, you can put consumers on as Chief Investigators.

Another question that we get is “can you claim salary for Chief Investigators or is there an unspoken rule that you shouldn't claim salary?” It's shown as being not being committed to the project or that sort of thing. We hear this kind of rumour sometimes. So just to be clear, yes, you can and should claim salary for your CIs if they're based in Australia, if it's needed. But only claim what you need.

We do sometimes see applications for 100% of the Chief Investigator's time when we know they're not probably putting 100% of their time towards this grant. So, you should only apply for what you need to deliver that project. But yes, absolutely, there is no problem with applying for salary for your Chief Investigators on your grant pending all the other eligibility kind of rulings around budget and costs that are able to be included, et cetera. Obviously, no salary for AIs. That's quite clear in our guidelines. But yes, trying to bust that myth, it's perfectly fine to claim salary for your Chief Investigators if they don't have salary from another source. Next slide, please.

So, another few questions that we've got through various webinars and other forums. Firstly, we've been asked, "Can you pay data scientists, analysts, health economists, et cetera through an MRFF grant?" Yes, absolutely. There's actually a number of different ways that you can pay for these types of personnel in your grant. They can be named CIs and as we've just discussed. You can claim a salary for them as a CI. You can put them in as unnamed personnel with PSP rates if you're with NHMRC, or name personnel with their salary if you're through the Business Grants Hub.

Or you can actually put sometimes services through as direct research costs depending on the role of the person on the grant and the significance of that role and the grant. So yes, absolutely you can pay for these different types of expertise on your grant and you can do that in a number of different ways. We also get asked, "Can you include consumer involvement and costs associated with that in your budget?" And yes, absolutely. True engagement with consumers comes with cost and we recognise that.

One of the things we are working on in the next guidelines refresh is to try and improve the language around that that would give you that added confirmation. Yes, absolutely we're committed to this and this is what we're talking about. But yes, you can put consumer involvement into your budget, whether that be time, travel, what they need. So we are working on added wording around that. But hopefully, that helps that question around consumer involvement in the budget. Next slide, please.

One of the other things that we have heard on the grapevine as well is there's this unspoken maximum amount that you can apply for in clinical trials grant opportunities. So that even though the guidelines say one thing, you actually won't get funded if you ask for too much and it's actually not correct, the maximum amount that you can apply for is what is in the guidelines.

The grant assessment committee does not have an unspoken limit in their minds. You can apply for what you need to do the project that you're proposing. So, there's two things to that though. The grant assessment committees do look at value for money so don't over ask. They know if you're padding out your budget excessively and they will score you down on overall value and risk. But on the flip side, if you under budget and you look like you're squeezing yourself too thin, they may score you down on overall value and risk because you are actually quite a high-risk project because your feasibility is compromised by under budgeting.

So, apply for what you need to do the trial with some realistic amounts, and then just put a good justification in for why it's needed. Don't over budget, don't under budget, but you can apply for the maximum amount that's available in the guidelines. Next slide please.

Another question we get a lot is overseas expenditure, especially around eligibility. So, there's a number of different areas in the guidelines, and I've just cut and pasted some of these here. They all work together. So, one of the key things you need to remember is that we are quite clear in the guidelines that you cannot use the grant to support research projects that are being undertaken outside of Australia.

The MRFF funds projects that are led by and conducted in Australia. Now, having said that, in some of our international clinical trial schemes, for example, you can ask for funding to support the Australia-based components of an international trial, but we will never allow you to spend money on trial sites overseas. It's just we don't do it. So just to keep that in mind, it has to be Australian-based research. If your project is predominantly being done overseas, this is not the grant opportunity for you.

A third thing that we do talk about just for completeness is that CIs based overseas are not able to draw a salary, but salary support is actually available for research support staff based overseas. But what you have to take into account is this final dot point as well where you can request funding for a component, not the whole project, a component of your project to be done overseas if the equipment or resources is not available in Australia, and that component is critical to the success of your grants.

So, it needs to be justified in your grant application. You can't just think that if it's under 10%, we're going to say yes. You have to justify why you can't access that in Australia, why it needs to be overseas. And in general, it's limited to 10% of MRFF funds, but it can go over if that is an absolutely critical part of your research.

So, it's not a hard and fast rule other than you can't be doing the bulk of your project overseas. So, I hope that has been helpful just to show you how to bring all of those things together. It's all about the justification. It's all about that primary policy that we support Australian-based researchers to do Australian-based projects with some leniency, acknowledging that there is necessity to go outside of Australia sometimes. But just to be clear, we will never fund clinical trial sites overseas and we do get asked that one a lot. Next slide, please.

So, now we're into some questions we get about the assessment stage, and this is where our director of our Science Unit was going to take over, but I'm just going to keep going. Apologies for hearing a lot from my voice today. So next slide, please. Let's start off with who actually assesses the MRFF grant application. So, what happens in our grant assessment processes? So, we'd always have independent Grant Assessment Committees, they're not decided internally within the department, they're not decided by the Minister. These assessment processes are carried out by an independent Grant Assessment Committee.

The members of those committees are expert reviewers. They come from a lot of different backgrounds. They're not just academic researchers and they bring different perspectives. We may have people from health services, we may have people from industry, we always have a consumer. We bring people together with different perspectives. So, researchers need to write their grant applications for accessibility and readability from a wide audience that come from different backgrounds. So, lots of different experiences that we tend to put in our assessment committees, they may be from academia, they may be from clinical, from health service delivery. We often have CEOs or administrative people from organisations as well because that's also important about whether something gets taken up into practice later.

We also have, as I said, consumers, we have industry, we have a whole variety of expertise. If you have researchers that are expressing interest in being on our Grant Assessment Committees or others, not just researchers, but noting this is a scenario webinar, you're probably going to be talking to researchers, point them to our expression of interest form on the website on NHMRC's website. We are always looking for more Grant Assessment Committee members and we'd be delighted if they signed up and gave us a little bit of a background about their expertise and what they could bring to a Grant Assessment Committee.

So that's who assesses our grant assessment, our grants. Why don't we move on to the next slide where we're going to talk a little bit about the Grant Assessment Committee process, like what happens? So basically, we assemble our Grant Assessment Committee when all the applications come in, A number of reviewers are assigned to each application for initial scoring. The Grant Assessment Committee meets to consider applications. If there's a small number of applications, sometimes they all go to the panel. If there is a large number of applications, we do shortlist. And we do that based on a couple of measures, one is just about the quantum of funding that's available.

If we have a certain amount, then we might go to 150% or a bit more of how much money we've got available to spend. But then we also have a process for identifying grants that have a wide disparity in scores and so to give them a fair chance at getting assessed, then they may go through to the panel as well. So, the processes we use are very highly similar to what NHMRC uses for their own grant activities, and the process is highly similar across the two hubs. So that's how you get assigned reviewers and then you get to a panel.

At the panel, each grant has spokespeople allocated to them and they are assigned to speak to its strengths, its weaknesses, and they basically take the lead on the discussion at the Grant Assessment Committee. But at the Grant Assessment Committee, all panel members have the opportunity to discuss and bring their expertise, and often we find that's a really good melting pot of hearing different perspectives about the value of a grant. And then each Grant Assessment Committee member scores the application on all the different assessment criteria.

So, you'll know we have three weighted scores and we have the score of overall value and risk. Just to be really clear, we prioritise the score of overall value and risk. So, if you are scored Excellent on OVR, we will prioritise that grant application over one that's scored Good. And if you're Marginal, you are deemed unfundable and you'll never get funded. So that gives you an outline about how the Grant Assessment Committee process works. It's very highly similar to other Grant Assessment Committee processes that happen already. And then after that, what happens is they provide the outcomes to us at the Health and Medical Research Office. So next slide, please.

So, what happens at the next stage in terms of deciding on outcomes, who gets grants funded and then provision of feedback to applicants, which is something we get asked about an awful lot. So, in terms of the process, once the Grant Assessment Committee has met, the recommendations are provided by the grants hub to the department delegate. Depending on the total value of grants being considered, it's at least the CEO or higher in the Department, but that's the delegate. And then the grant offers a signed off by the department delegate. We start at the top of the list and go work our way down unless there's other criteria such as we're going to fund one per state or territory or something like that. But other than that, we use the merit-based system given to us by the Grant Assessment Committee. We don't fiddle with that process at all.

So once the grant offers are signed off, they're signed off within the Department, they're not signed off by the Minister, they're signed off within the Department. And then we let the minister know who we are funding. And then once he's noted who we are funding, then we can let you know under embargo. We got a question actually recently about why does sometimes the embargo periods stretch out? it's because we're giving the Minister's Office the chance to announce the grants and we have to fit into his announcement cycle and schedule. He's got a lot of things on his plate, and so sometimes that's why the grant embargo period may be slightly long. But I shouldn't belabour that point because I think Misha is going to touch on this later as well. But yeah, that's why the embargo period is there, it's to give the Minister the opportunity to announce the grants.

Feedback to applicants. So, we are actually currently working in this space. Currently, we give feedback through the Business Grants Hub with some written feedback as well as scores. We are just working on revising that a bit to try and make that more useful to applicants. And you'll be really pleased to know that NHMRC is about to start rolling out written feedback to applicants through all the grant assessment processes that we run through them. So that will happen later this year, they're just starting with a couple of grant opportunities to fine-tune the process. But similar to the feedback you get through some of their other schemes, you'll be starting to get that through MRFF schemes as well.

So, watch this space. We know it's a big concern for the sector and we're working on what we can do and it is coming. Next slide, please. So, another question that we also get is the role of consumers on the Grant Assessment Committee and specifically what is the concept of lived experience and what constitutes experience in research? So, we funded the principles for consumer involvement in research funded by the MRFF, which was developed as part of our consumer reference panel and released last year. And in there, we define a consumer as a person with lived experience as a patient, client, potential patient, user of health services and or providing support as a care or family or community member. So, it's quite broad.

So, the term does relate to consumers and their involvement in every type and all stages of research. And also as a part of Grant Assessment Committees. So we do consider consumers valuable members of our Grant Assessment Committees, not just the grant applications because they provide perspective and experience from a consumer viewpoint, and they also provide informed advice on applications, which includes things like the value of research to consumers, and they participate in those discussions through our grant assessment committee processes. So, we do value them. That's how we think of what a consumer is.

In addition to that, all of our new MRFF grant opportunities containing the refreshed assessment criteria that you may have noticed, but they also incorporate explicit expectations for consumer involvement. So do have a look at those and encourage your researchers to have a look at those because we do want to strengthen the way that that involvement is progressed. And so, we are actually also, I was going to say playing around, but that's a bit too loose. We are testing, if you will, scoring, consumers actually providing scoring onto at the Grant Assessment Committee stage.

That's not across all our grant opportunities at the moment, but we're just starting to step into that space. So, there's a couple of grant opportunities where we have done that, one with them then been the 2023 Consumer-Led Research Grant Opportunity, and we're also looking at strengthening that through the 2024 Survivorship Care and Collaborative Research Prioritisation Grant Opportunity as well.

We're also looking at strengthening the requirements at that point, so requiring applicants to provide a consumer involvement statement, which is a two-page statement summarising the approach that that grant is going to take to consumer involvement. We're also testing having a more significant proportion of the Grant Assessment Committee being consumer members, not just one representative. So just to let you know, this is the direction we're heading. So certainly, that consumer involvement is actually a really critical piece that really needs to be taken into consideration. Next slide, please. And I think that might be handing over to Misha. So, thank you very much.

Misha Hutchings:

Thank you, Ruth. We'll now turn to some questions that we received about post-award and managing an MRFF grant. So, we received a few questions focused on initiating an MRFF project, which we've summarised into the following. What are the expectations around project initiation? So, starting up an MRFF grant, including the time it takes to establish the grant, to obtain approvals and to enter into contracts and agreements. Can you provide some guidance around contracting expectations with partners and ensuring that partners deliver on their obligations and rules in the grant?

So, the Department's expectations for initiation of MRFF-funded projects are pretty much in line with most other Commonwealth funders. As Ruth mentioned before, we have embargo conditions, and those are fairly standard, but we'll be releasing written advice soon just to remove any doubt. An embargo period means that you can't talk about an outcome publicly until the official announcement has been made and the embargo has been lifted.

During that period, if grant activity does start, if the grant start date does occur, you can begin spending some funds just as long as you don't publicly release information that links either to the MRFF, the grant or the grant opportunity, for example, such as in a job or a recruitment advertisement.

The Commonwealth Grant Rules and Guidelines also require us to monitor for what they call double-dipping, where a particular research activity can't receive funding from more than one source. With our grant hubs, we do have processes in place to monitor for and identify what we call duplicate grants. But your help is always appreciated. We'd much rather not rush around to deliver bad news and make you make a choice. It's better if you're on the front foot about that. So as soon as you receive the second or a duplicate outcome notification, you should contact the relevant grant hub as soon as possible.

In terms of starting the grant activity, the Department really expects grant activity to begin from the start date or the commencement date that’s outlined in the grant agreement or the grant schedule depending on which grant hub you applied through. So, the grant opportunity guidelines outline that you must notify the grant hub if the undertakings for project initiation aren’t being met. So, what that means, it just includes things like obtaining ethics approvals or finalising multi-institutional agreements.

To the question around helping to ensure that partners are on board and don’t delay things, you can help reduce the likelihood of some of this by pointing your collaborators to the indicative grant agreement on GrantConnect just to help set expectations and then you can come to agreement on things before the application is submitted, ideally. If delays do occur, just please don’t wait for the progress report to let us know. You have options, including contacting us, the grant hub or requesting a variation to defer grant commencement date or defer an in-progress grant variation types, which are both available in increments of one or more months up to a maximum of 12 months.

So, we also received some questions around post-award guidance and resources that we can make available that are specific to MRFF. So, can you provide a dedicated area for MRFF post-award help inquiries? Can you provide more guidance and advice about variation requests that will help RAOs and grantees manage their grant more effectively? Can you provide information about correct templates, formatting, and other requirements for grantees to follow? So last December, the Department launched—so, the first question is really easy for us to respond to and it's great, you set it up perfectly for us.

So last December, the Department launched a webpage that is a single location for MRFF post-award policies, forms, and templates for MRFF grants administered by both grant hubs. So not just BGH and not just NHMRC, it's all in one place. Soon we'll upload MRFF guidance documents that will assist you with your questions around project reporting, submitting variation requests, and the different variation types and other topics that are related to post-award and grants management.

So, earlier I mentioned we'll be releasing guidance on the embargo period and what you can do during that period. So, that advice will also be posted here on this web page. In addition, the Department is also holding regular RAO seminars in each state to give you RAOs—and that includes finance officers as well who are important to managing MRFF grants—an opportunity to meet the MRFF staff face-to-face and to ask and get responses to your questions directly. The primary RAO in your institution or the grant's primary contact will receive the notification about the next seminar being held in your state. I want to thank you. I'll now hand over to Cindy Thamrin, who's the Director of the Performance and Evaluation section in MRFF.

Assoc. Prof. Cindy Thamrin:

Thanks very much, Misha. And I speak to you from Noongar land today, and I am here as Misha indicated to talk to you about questions related to impact evaluation and reporting for MRFF grants. And so, I'll be covering a couple of topics, one is regarding performance indicators that we use to capture impact as well as a performance indicator survey, which you would've seen or received a heads up on us sometime last week. And the second topic I'll be covering is reporting of funded rates. So, we received this question at the last webinar about whether or not we have any types of measures to analyse impact of MRFF funding specifically around research career development.

So, the question was about whether, for example, whether we have been responsible for initiating or supporting careers, particularly in the biotech or the MedTech space, but more generally as well. And the second question which I'll address last is whether the Department provides information on number of applications for each grant opportunity in the future and whether success metrics like number applied versus number funded will be available for grant opportunities. And that's a question that has been asked at multiple prior webinars and we will provide an update towards the end of the session. To that first question about impact reporting, so the answer to that is yes.

In March last year, we published a series of performance indicators that go towards measuring the impact of the MRFF, and this was communicated at the webinar very close to that time as well. And this is all part of the implementation of the monitoring evaluation and learning strategy, which you may be aware of and applicants may be aware of when they're putting together their applications, particularly around the measures of success. And what that document outlines is how the MRFF intends to capture data that goes towards measuring and tracking impact of MRFF-funded projects.

I can take a step back and this figure will help illustrate that process and the intention. So, the monitoring and evaluation strategy already outlines for us what measures the MRFF uses to evaluate its impact. And because MRFF is a priority-driven fund with a particular focus on translation into health outcomes, the impact measures therefore reflect those as well.

So, the five MRFF impact measures as indicated on the right hand of this program logic diagram are better health outcomes, beneficial change to health practice, increase health efficiency, increase job and export potential and economic growth. And that document of the monitoring and evaluation strategy also outlines eight measures of success, which again, you may be already familiar with or your applicants may, your researchers may already be familiar with as well, as these are things that they need to address during the application process and are assessed against.

And those MRFF Measures of Success contribute towards these impacts. And so, they include increased focus on research on the areas of unmet need, demonstrating that more Australians are able to access clinical trials, more health technologies and more health interventions being embedded in health practice and in health in general, research community having greater capacity and capability to undertake translational research.

The MRFF is also hoping to be able to achieve the outcome of health professionals being able to adopt best practises faster and hopes to see the community engaging with and adopting new technologies and treatments and increased commercialisation of health outcomes. And so, you can appreciate that some of these are very high-level outcomes. And so, what the performance indicators were designed to do was to enable us to collect data in a quantifiable manner in order to capture progress towards these longer-term outcomes and longer-term impact measures.

And so you will see in a document that we published in March that we had a series of indicators that look at whether projects are cut targeting priority populations or emerging issues, some questions of indicators relating to clinical trials and the nature of them, some ones around workforce, the research workforce, some ones around some indicators around knowledge gain. Some indicators to assess the level of involvement in MRFF projects from consumers, not just during the beginning or towards the end, but throughout the pipeline, throughout the process, throughout the project Lifetime healthcare change indicators, commercialisation pathway indicators.

And we also use case studies to illustrate particular instances in which methods of success are being met. And so, the way in which we capture this data is through a variety of methods. And it so happens that, as I said, many of you will have received communications last week about the performance indicators survey. And we talked about the fact that we will be conducting a survey at the webinar as well as in the document, and it's one of the key methods of capture for that data to track that impact. So, the inaugural survey is the first time we're running this, it was sent to the lead investigators, the CIAs of all grounds on the 21st of March with a heads-up provided to RAOs a few days before that, and it will be open for four weeks. Recognising that there are other deadlines that the sector may be dealing with, we try to pick an appropriate time.

And we also are very mindful of research and burden on our RAOs as well. And the survey is designed, and it's probably worth pointing out, the survey is designed to capture data that's not easily obtained or not available for progress, final reports or other sources. Whilst the progress and final reports are designed to capture progress of individual projects and for the team to engage with the researchers on the progress of their individual projects, the performance indicators survey is designed to capture some of these longer-term metrics that sometimes won't occur until the project has completed.

Some publications and healthcare change and citation metrics and commercialisation indicators, as you can imagine, sometimes don't happen until many years after that happens, until the completion of the project. And so, this is what the survey is intended to capture and it's something that we will run regularly. We were planning every three years. And the questions, the format of the questions may evolve in future depending on what we learn from this initial inaugural survey.

So why should researchers do it? Well, first up, for accountability reasons, not so much from a researcher's point of view in this case, but from our point of view as well that we want to demonstrate that the MRFF is doing what it's supposed to be doing and provide that transparency and accountability to the sector, to taxpayers, to government. But more and just as important as that is that if researchers participating provides us with the data that we need to improve and help shape the MRFF, it helps us support what we might do for broader policy and program development, identifying if there are any opportunities for learning or improvement or gaps that we might not be covering in terms of what we're trying to achieve.

And more from the researcher's point of view, we hope that the whole process of collecting data, capturing data relating to impact will help researchers see better how to strategize their own impact more generally and also have a better understanding of what the MRFF prioritises as measures of impact, which may be considerations for themselves as they apply for MRFF grants and other grants that prioritise impact in the health and medical space.

So, what do RAOs need to do? Hopefully, not much. We just ask if you could encourage your lead investigators to complete the survey by the 18th of April, which is the closing date. We are looking at a lot of feedback that is... We have received some feedback about the survey and I'm very happy to address that if any are raised today. For instance, as you can appreciate, the MRFF captures a wide range of projects across a heterogeneous range of funding models. And we do appreciate that the survey in its current nature can be quite challenging for those projects that manage multiple sub-projects.

And so, we are mindful of these challenges. We are exploring an extended deadline for those particular projects. And we also advise you to answer. If researchers happen to be one of those that manage multiple sub grants, then we advise you to answer the survey on the basis of the collective wherever possible. The only exception to this is the question, the questions around clinical trials where we advise you to answer on the basis of the largest clinical trial that is supported by your grant. And where it's not possible, we advise you to use the free text option to indicate this to provide more information where you'd like.

So, as I said, the Department is taking all feedback on board. It is our first survey, so we anticipate lots of teething problems. But as I said, we are looking at the questions that we’ve received so far and exploring avenues for extension, particularly for some of the more complex grants. And also I think I want to address ahead of time, if some of you have grants, if some of your researchers have grants that are too early, just fill in the survey as best you can, and just be mindful that don't be afraid or don't worry if you don't have these outputs to report yet. This is something that we will hold regularly in future. And we also aim to not ask questions that we've already captured data for. So that is an evolving process as well.

However, any queries, feel free to lead your researchers, forward your researchers to mrff.evaluations@health.gov.au. And then going back to that original question that was posed, which was about whether we are supporting career development, there are several questions in the survey you'll see that are related to that. I've just provided one example, which is in regards to workforce capability and capacity activities, outputs or outcomes. And you can see that we do look at whether staff supported by the grant are being involved in research translation training, whether they're involved in exchange programs or placements within industry, particularly for those grants that sit in that space.

There are other questions as well where we ask about co-funding and capability building in other ways. So, these are things that we're very keen to build a picture around and some of these questions might evolve depending on the answers that we receive as well to try and unpack this a little bit more so that we can really speak to whether the MRFF is delivering on those fronts.

So, I've left the best for last. I know that there's been many questions in the past asking for funded rates and outcomes of grant opportunities. I'm very happy to tell you that thanks to the efforts of the office, this is now available on the website. It's on the same page where the list of grant recipients are announced, and so you'll find this available for download in Excel or PDF format and hopefully, that is helpful to you and your researchers. I think that is the end of it. I'll leave on this note and hand over back to the organisers.

Misha Hutchings:

Thank you, Ruth and Cindy, for your informative presentations. We're now going to head into the Q&A portion of the webinar. As mentioned before, we're triaging questions as you send them through. Since we may not have the appropriate directors online today to answer all of the questions, any that we aren't able to address in person here will be taken on notice and they'll be published on the MRFF website in the weeks to come. Alright, so I'll read out the questions and then one of the directors will provide a response. So, the first question is, Is there any reason why MRFF does not provide written feedback for unsuccessful applications? We have many requests from CIAs who want to improve their proposal for future submissions, and I think Ruth was happy to respond to that one.

Assoc. Prof. Ruth Griffiths:

Thanks, Misha. And I'm hoping I've provided you with some encouragement already that we are constantly reviewing processes. Why we have or haven't done things in the past often adjusts due to pragmatic limitations of a new program, getting up and running practical limitations related to really, really large grant opportunities. We want to be consistent across all our grant opportunities, but if we get 400 applications, then it's really hard to give things like specific feedback. So up until now, certainly through NHMRC, I believe people have got scores or know their NFFC. BGH more recently, you should have started getting scores and some written feedback that gives you outlines of what successful grant applications look like to the grant panel.

But as I mentioned, that's changing. So, moving forward, all grant applications through NHMRC, both successful and unsuccessful, should be getting some qualitative feedback or written feedback as well as the scores. We haven't implemented it yet, so I'm kind of hesitant about going out there with a lot of detail, but that is the intent that we do expand on what we're offering. And then we're also looking at what we're doing through BGH, but I think BGH has always got a little bit more anyway. So, no particular reason other than probably pragmatics and administrative reasons and limitations of systems, but we are working towards that and there will be changes coming later this year.

Misha Hutchings:

Thank you, Ruth. The next question, How long do embargo periods normally last? We've been waiting for a long time and yes, we've started recruiting and planning, but don't know when to appoint from, and so I'll respond to that question. We can't really answer except to say that embargo periods are at the discretion of the Minister's Office, and so they can vary. Sometimes they come out quite quickly once they're finalised and the delegate has made a decision, other times it might just take a while. As you noted in your question, you've started the hiring and recruiting and that's expected. In terms of not knowing when to appoint from, I think again, it's important to look back at the grant schedule and grant agreement to know what the grant start date is.

So, if your grant start date has passed, we're expecting that research activity or grant activity is taking place. So, you can appoint from that date. I know it's tricky, it's just around not publicising the activity.

Are there guidelines available for appropriate costing of consumer involvement? So, I think Ruth had a little bit to say on this one.

Assoc. Prof. Ruth Griffiths:

Sure. So, we are working on giving, I guess some more guidance as to what we might look for or expect, but I don't want you to think that there's hard and fast rules coming. It's always context-dependent, depends on the grant, depends on the grant opportunity, depends on the location of the research. So, the important thing is that the costs are well justified, and that's the guidance that we give you is what do you need to do to demonstrate true consumer involvement in your grant activity and what are the costs associated with that, and then putting them into your grant application.

But as I mentioned, we are looking to amend the wording in our guidelines, but also our consumer reference panel was also working with the team in the Department to try and generate some more information. But when they come out, please, again, there is no black and white way of putting this. It is context dependent and it is about the justification.

Misha Hutchings:

Great. Okay. The next question is, Can you clarify which measures of success have a greater weighting in a particular scheme or grant opportunity when described as short-term versus long-term? Should applications focus more on addressing the short-term or the long-term measures? I'll hand this one over to Cindy.

Assoc. Prof. Cindy Thamrin:

Thanks, Misha. Yes, I can clarify. I should mention that there is, just to state that there is no specific weighting attached to specific measures of success in the assessment criteria assessment. The measures of success being described are being described as short or long-term in the monitoring and evaluation strategy. But that is provided as a guide for evaluation rather than a criteria to address for assessment. But we recognise that different projects also will have different ways in which they address the measures of success. So, you should speak to your project and provide information on that, and use the long or short-term as a guide rather than a specific thing to address to.

Misha Hutchings:

Thank you, Cindy. Okay. Let's see. Next question. If grant commencement has been deferred, can MRFF grant funding be spent prior to the new grant commencement date? For example, to pay for salary of project staff employed to set up the project, negotiate partner agreements and prepare ethics applications? So, the answer to this one is no, and it is outlined in a couple of different places. So when the Department awards grant funding, and this is actually common to all funders, a start date and an end date are specified, and so you're allowed to spend funds between those dates or during the grant period.

If you requested a deferral of the grant commencement date, it is outlined in the MRFF Grant Variation Policy and the associated appendices that in order to apply for that defer grant commencement date variation type, you need to have not commenced spending the funding. If you have started spending the funding, then you can defer an in-progress grant variation type.

Let's see. So, the next question is, Can you please make the list of performance indicator survey questions available on a web page or send them to RAOs? This would help in preparation of responses before completing the online survey. I'll hand that one over to Cindy.

Assoc. Prof. Cindy Thamrin:

Thanks again, Misha. Yes, I'm very happy to provide a copy of the released survey to RAOs and we've also heard feedback from others to do so as well. We do note though that our records might not be 100% up to date. So, if you are an RAO and you haven't received a copy of the survey by early next week and you'd like a copy, do reach out to us. You can use the email address that's on the slide deck, it's mrff.evaluations@health.gov.au.

Misha Hutchings:

Thanks, Cindy. Okay. The next question is, Can people without a Ph.D., for example, students and consumers be considered as early to mid-career researchers? I'll hand that one over to Ruth to respond to.

Assoc, Prof. Ruth Griffiths:

Thanks, Misha. So there's a couple of parts to how to answer this, one is an early to mid-career researcher does tend to have a fairly standard definition that's used across a number of grant schemes, a number of universities to denote what an early career researcher is, and we need to make sure that we are, particularly for some of our eligibility criteria, we willing to prioritise promotion of EMCRs into CI positions and providing funding and those sorts of things. So, we do need a mechanism by which we can determine whether someone genuinely is an early to mid-career researcher or not.

So that actual definition of an early to mid-career researcher does have that definition of a Ph.D. date in there. Having said that, we do acknowledge that there are other people in the system that would be considered early to mid-career in their research career, if I can frame it that way, such as clinicians, but they may not actually have a Ph.D. So, we are starting to experiment in some of our new grant opportunities. For example, the most recent primary care grant opportunities, a definition of an early to mid-career clinician researcher, there is no requirement to have a Ph.D., but you are within five to 10 years of your, I guess, substantive date of getting your professional qualification.

I would suggest that students and Ph.D. students are probably not going to be determined to be early mid-career researchers in the near future. I think it just becomes very hard to manage that in some ways. But we are starting to think of other ways that we could acknowledge that not everyone has a Ph.D., but they may be a valuable part of that CI team. So, if I can put it that way, EMCR is a fairly standard definition, but we are looking at other definitions that we could use that incorporate people who may not have a Ph.D. yet. Keeping in mind that the CI team does have to be the people with the expertise, the experience, the skills and everything to deliver the project.

And so sometimes it may be more appropriate for a very early, I guess, researcher such as a student or maybe someone who's undertaking a Ph.D., maybe they'd be better off as an AI as that introduction to research depending on how early they are in their career. So, I guess that's how I would answer that one.

Misha Hutchings:

There's a question confirming Is it okay to start appointing personnel before the embargo is lifted? It is difficult to advertise positions without revealing identifying information about an embargoed grant. So just a nuance around that is just to not mention the specific funder and funding source. In most cases, research projects that are funded by an MRFF grant exist in a continuum of a greater research program. So, we wouldn't expect that details about the research to be conducted that might be included in a job advertisement need to be tied specifically to an MRFF grant opportunity. I think your HR teams are probably practiced in this, so you could work with them on how best to phrase things in the advertisement.

The next question is, Can expenditure occur on a grant if it is suspended, such as in a defer, an in-progress grant variation type? So, this is also covered, I believe, in the MRFF Grant Variation Policy and the associated appendices. During a deferral of an in-progress grant, we would expect that research expenditure would either come to a halt or it would be reduced commensurate with the activity that's being suspended. So sometimes a deferring in progress grant is requested because there's a delay in a portion of the research activity, but not all of it.

So then in that instance, we would expect to see a reduction in expenditure that's commensurate with the portion of the research activity that's delayed.

So, there was a follow-on question with regard to the Measures of Success. Further from that, could you let us know how reviewers are being advised on how to consider the long and short-term measures? Are they advised to look out for projects which include more of the long-term or short-term measures? So, happy for Cindy or Ruth to handle that one.

Assoc. Prof. Cindy Thamrin:

Yeah, I can do that one. Thanks, Misha. Yeah, so to clarify, reviewers are asked to consider the Measures of Success and they need to meet at least one. They're not being asked to consider short versus long-term. Again, that is something to guide evaluation and is not part of the assessment.

Misha Hutchings:

Next question, which is an easy one. Would this webinar be recorded and available for reviewing later? Yes. Yes, it will be. There'll be a transcript below the video as well as a recording of the video.

Let me take a look and see if in the last minute there are any other questions that we can respond to. Several of these would need to be responded to by our colleagues, and so you can expect to see responses in the FAQ published in the coming weeks. Alright, well, we'll take this last one that just came in. Can a CI, not based in Australia, be included on an application? Ruth, would you be happy to handle that? Thanks.

Assoc. Prof. Ruth Griffiths:

Yes, definitely. Absolutely. Yep, definitely they can be included on an application. It's incredibly common, particularly when you have overseas collaborators. They just can't draw a salary. But yes, absolutely they can be included.

Misha Hutchings:

That brings us to the end of the Q&A portion. I just wanted to put up here the different ways that you can get in touch with the MRFF team, as well as our grant hub partners. And then we've also included some links to some of the information that we talked about today during the webinar. We really want to thank you for your time and your continued feedback. We really appreciate it and it's helping us to make the MRFF better. Thank you very much.