Home > For Health Professionals > Health products and medicines > Health Technology Assessment Review

Health Technology Assessment Review

Discussion Paper 3 – Approaches to Evidence and Methodologies

This paper summarises the key concerns expressed by stakeholders about the current approaches by the Department of Health and Ageing (DoHA) to evidence and methodologies and the way that they are applied to HTA in Australia. It also presents a number of proposals to address these concerns, within the scope of the HTA Review.

Discussion Paper 3 – Approaches to Evidence and Methodologies (PDF 41KB)
If you have any difficulty accessing the PDF, please contact mbd.web@health.gov.au

Disclaimer

The proposals for reform to Commonwealth Health Technology Assessment (HTA) processes as outlined in this Discussion Paper represent a range of responses to the issues raised in submissions to the HTA Review and during stakeholder consultation. The Discussion Papers are intended to stimulate discussion at the forthcoming consultations and any proposal presented or omitted should not be taken to represent the policy position of the Government. The HTA Review is required to put forward proposals that can be sustained within existing funding levels and that are consistent with Government policy objectives.

Introduction

Commonwealth HTA processes for reimbursement purposes are undertaken by three separate HTA advisory committees: the Medical Services Advisory Committee (MSAC), Pharmaceutical Benefits Advisory Committee (PBAC) and Prostheses and Devices Committee (PDC). These advisory committees assess health technologies, including devices, pharmaceuticals, diagnostics and pathology tests. The manner in which these different advisory committees approach HTA (approaches to evidence and methodologies) varies, as do the ways each single advisory committee approaches different technology types. It is recognised that there is a strong need for consistency across and within HTA advisory committees where possible, as well as the ability to undertake fit for purpose assessments where appropriate.
This paper summarises the key concerns expressed by stakeholders about the current approaches by the Department of Health and Ageing (DoHA) to evidence and methodologies and the way that they are applied to HTA in Australia. It also presents a number of proposals to address these concerns, within the scope of the HTA Review. Top of page

Concerns raised in HTA Review Consultations

Concerns raised about approaches to evidence and methodologies for HTA processes during public consultations ranged across six themes:

Transparency of methods

• Stakeholders sought greater transparency of approaches that are used to assess technologies, including:
  • what methodologies will be used to assess technologies;
  • how those methodologies will be applied; and
  • how much evidence of what quality/level of evidence is expected.
• Stakeholders identified a need for better feedback about limitations in the evidence provided by sponsors (either prior to, during or following the assessment process).
• The consequences of the current lack of transparency were considered to be:
  • inconsistent or unpredictable application requirements, assessments and decisions;
  • duplication of work by sponsors and HTA advisory committees;
  • inefficiencies for government due to the receipt of insufficiently substantiated applications; and
  • poor public perception of the fairness of Commonwealth HTA processes.

Evidentiary requirements

• The levels of evidence required for assessments are regarded by some stakeholders as lacking clarity or consistency. Clarity of evidentiary requirements, consistent with a high likelihood of a positive HTA outcome, was important to stakeholders who have financial interests and desire some degree of certainty around HTA decisions.
• The perceived high number of HTA recommendations that did not support reimbursement for health technologies, was seen as being linked to unreasonably high evidentiary requirements, and was of concern for many stakeholders. Stakeholders stated that level 2 National Health and Medical Research Council evidence is difficult to obtain for many medical technologies.

Comparators

• Stakeholders identified a need for greater clarity regarding the determination of comparators in assessing new health technologies. Current levels of guidance, both formal and informal, were criticised with regard to the appropriate use of comparators, the scope of what can be considered a comparator, and what options are available if there is no obvious direct comparator.
• Societal values
• The use of ‘societal values’ in economic evaluations was felt to be lacking in the current system. Failure to consider wider economic benefits (other than health benefits) was thought to result in the undue exclusion of some technologies from reimbursement.
• Risk-based approach
• A risk-based approach to assessment methodologies was considered desirable in many submissions. The current system was criticised as being inflexible and unable to undertake ‘fit for purpose’ assessments in which the potential risks involved with a technology could determine the level of assessment (such as complex versus rapid assessments) and the level and quality of evidence required.

International harmonisation of methodologies

• The international harmonisation of HTA methodologies was supported by stakeholders, particularly for assessments of comparative safety and comparative clinical effectiveness. By ensuring economic evaluations are local, but HTA methods and some assessments are translatable internationally, harmonisation could add efficiencies to HTA and make better use of expertise and resources.

Top of page

Proposal 5 – Implementation of a Risk-Based Approach to Assessment

A more risk-based approach to assessment would mean ensuring that the intensity of assessment matched the risk of the technology. Such an approach would require the ability to vary the amount and level/quality of evidence for different technologies through the development and use of:

An HTA risk classification table

This would be a tool that, based on the characteristics and clinical performance of the technology, could produce an individualised risk rating (eg rating the fiscal risks and/or risks of harm to the patient). This could consist of a table or a series of tables comparing factors to come up with an individualised risk rating, akin to the Therapeutic Good Administration’s (TGA) risk classification system and the work of the Global Harmonisation Task Force (GHTF) on risk management principles. This table could enable applicants to easily identify the likely evidentiary requirements for an application (see next point below) and the likely level of assessment (eg full or streamlined). However, in order to ensure that a fit for purpose approach is taken, this table could only provide an indication of these expectations. The implementation of this approach could provide transparent and more consistent guidance on risk assessment and thus some certainty to applicants regarding the initial approach to assessment. It would provide no indication on the likely outcomes of the subsequent assessment.

Linking risks to evidentiary requirements

Once a technology had been risk-assessed, sponsors could then be formally advised of what type and quantity of evidence would be required for an individual application ¹. The risk-assessment could also determine the clinical and economic evaluation tools to be used, related to relevant outcome measures.
Different methods for grading evidence could be considered in the development of the final evidentiary requirements and associated guidance. These should take international requirements into consideration (see next point below). The implementation of this proposal could facilitate upfront discussion of limitations in evidence with sponsors (see also Discussion Paper 4).
The implementation of a transparent risk-based approach, alongside improved transparency of evidentiary requirements and improved communication with stakeholders around decisions, might better enable applicants to meet these requirements, and more clearly highlight the relationship between the strength of evidence and assurance in recommendations for public and private funding.

Consideration of harmonisation with international evidentiary requirements

The harmonisation of risk-based evidentiary requirements and methodologies to align with those of other countries where possible could provide applicants with some degree of certainty about possible outcomes, reduce duplication of effort and more efficiently use HTA expertise. This would be particularly helpful to multinational sponsors dealing with HTA expectations across different countries for the same health technology. Assessments would continue to be undertaken at the Commonwealth level, incorporating the local context.

Discussion

The implementation of these approaches to evidence and methodologies could involve a review of risk classifications, evidentiary requirements and harmonisation across HTA advisory committees, with consolidation where possible, and allow publication of more transparent requirements. The intent of this proposal is to address the consistently raised concerns around the lack of transparency and consistency, not necessarily to increase evidentiary requirements. These approaches may also address the identified need for greater simplicity in the HTA processes and may promote timeliness of assessments.
With regard to risk, the work undertaken on this proposal could define the types of risk to which it refers (eg fiscal risk and risk of harm) and explain their importance in the assessment process.
Top of page

¹ The levels of evidence required could have been set out in advance, at the time of the pre-lodgement meeting see Discussion Paper 2)

With regard to harmonisation with international requirements, the TGA has already adopted similar principles developed for the European Union in relation to the market regulation of devices. Whilst higher levels of harmonisation have been called for by some stakeholders than are currently proposed, it is important that this proposal enables the benefits of harmonisation (such as efficiencies) without jeopardising the ability to make HTA decisions based on local contexts, values and resources, and that harmonisation does not hinder the ability to undertake fit for purpose assessments. In addition, it would need to be ascertained whether HTA evidentiary requirements in other countries are compatible with the proposal to develop risk-based evidentiary requirements.
Stakeholders may dispute the defined risk for a health technology and consequential evidentiary requirements. However, this should be mitigated by a clearer articulation of the reasoning behind these evidentiary requirements (eg comparative safety and financial implications for government). It is also possible that a more transparent approach to risk classification and evidentiary requirements will further reduce the perceived inflexibility of the system to accommodate other relevant factors.

Proposal 6 – Improved Guidance on Methodolgies andMethodological Processes

Guidance for applicants that is consistent, transparent and publicly available. This could outline explicit HTA methodologies and methodological processes to address the following concerns including:
Perspectives considered in assessments
An assessment could be undertaken across the current HTA processes to provide a clear description of, and reasoning behind, the perspectives adopted for economic evaluations considered by these approaches (with consequences for the consideration of indirect costs). If any changes were agreed as a result of this assessment, further guidance could be developed about the way in which relevant issues are considered (including any caveats).

The use of comparators

Guidance for the selection of comparator technologies or services and their use in the assessment could include consideration of:

• the current clinical management option(s) that the proposed health technology would most likely replace or augment in practice;
• what is current best practice;
• what are similar technologies; and
• what are other comparators from other technology or service types.

The guidance documentation could also explain the way in which comparators are used in assessments.

Evaluation methodologies

An assessment could be undertaken across the current HTA evaluation methodologies, with a view to making them consistent wherever appropriate and explaining different evaluation methodologies as required. This could be conducted during the routine review of each of these methodologies, or as a single commissioned exercise. It could cover topics such as scope of the evaluation, development of research questions, and evaluation tools to be used. It could also include a clear explanation of how methodologies would differ for different types of assessments to ensure that they are fit for purpose, such as different HTA methodologies for treatments and for diagnostic tests. MSAC currently has different guidelines for these.

Specified points in time for obtaining further input leading up to a HTA recommendation

Guidance on the timing and number of opportunities for the applicant and other affected stakeholders to contribute information (eg evidence or feedback) during the HTA process. The reasoning around the number of evidence collection points could be explained clearly, for example the need to ensure adequate procedural fairness, as well as the tension around enhancing timeliness of assessments. This could link to the provision of timeframes identified in Discussion Paper 4.

Discussion

The use of economic evaluation as a method of allocating scarce health resources has wide acceptance, however the methodologies and perspective of such evaluations are often debated. An initial assessment of the perspectives adopted across MSAC and PBAC, for example, indicates a broad agreement, especially in relation to giving a lesser weight to indirect costs for policy, technical and equity reasons. These committees already include societal values about health consequences wherever health outcome changes relating to a new technology can be measured through Quality Adjusted Life Years (QALYs). However, many applications to MSAC have not been supported by evidence of health outcome changes, which makes societal values difficult to apply to such assessments.

A major focus of the HTA Review is improving the timeliness of assessments and decision-making. There is some tension between the desire for faster HTA processes and ensuring adequate procedural fairness. Shorter timeframes for the provision of information, or a restricted number of opportunities to provide further input, may be perceived as inadequate or unfair. However, shortening overall assessment time frames may ameliorate some of these concerns.

This proposal should also help address transparency, procedural fairness and improve communication with stakeholders around common areas of concern. It may also address the need to simplify the assessment of health technologies through enhancing the clarity around methodologies for applicants.

Some applicants might perceive a reduction in the flexibility of assessment processes if transparency of methodologies is increased, due to an enhanced understanding of processes even if no changes are made to the processes themselves. It is also likely that there will continue to be varying views on how societal values should be considered in assessments.

There may be some applications for which new evidence arises during the assessment process which could have an impact on the assessment of the health technology under consideration. Improved timeliness of HTA processes (and the submission of applications only when relevant evidence has become available) should decrease the likelihood or consequences of this occurring.
Top of page