'Beyond Bricks and Mortar - Building Quality Clinical Cancer Services' Symposium 2011

Quality in Pathology: An Overview - Dr Michael Harrison

Up to Radiation Oncology

prev pageprev page| TOC |next page

Chief Executive Officer and Managing Partner, Sullivan Nicolaides Pathology

Download powerpoint presentation by Dr Michael Harrison (Powerpoint 5237 KB)

Introduction by Norman Swan:

So this is a very important session moving forward. You've got the options people in front of you - Leigh will speak to that more specifically at the end of the day, but I will certainly use it to frame our discussion at the panel. And an important theme in that options paper if any of you have had time to look at it, is that others have been there before us. So there's no point in us trying to invent something that other people have worked through. Yes there are differences but what can we learn from others? And there are lots of worked examples in that options paper.

One of them, and probably one that's very useful for us is Pathology, and the standards assessment in - in a standards setting and assessment in Pathology is a very interesting example of how things have worked. And somebody who walks the talk in this area is Michael Harrison. He's been heavily involved in supporting the National Association of Testing Authority (NATA) and the College of Pathologists’ accreditation scheme. It's a sophisticated scheme and very, very thorough, because as you know, variation is the evil bug bearer in pathology testing and arises from a variety of sources, some of which are controllable and some of what are not, and you try and minimise the controllable ones.

Michael's a Consultant Pathologist and Chief Executive of Sullivan Nicolaides which is a huge pathology service in Queensland. Please welcome Michael.

Dr Michael Harrison:

Thanks very much Norman, and thank you Abel for asking me to talk today. I’ve been a pathologist now for 26 years and in my first year that I started my real job, my laboratory went through its first accreditation inspection, and so accreditation and assessment has been part of my professional life. Pathology was probably the frontrunner in this area and we’ve had accreditation of pathology laboratories now in Australia for about 26 years and it’s mandatory and it has been mandatory for a long time. I’ve been involved both initially as being a person who was assessed or whose laboratory was assessed, and as an assessor. I did get involved in the process of setting the standards; I’ve been a member of NPAAC, which is our standards setting body, and I’ve chaired the group, or the committee, that oversees the program in Australia – the Medical Testing Accreditation Advisory Committee. I do get to see lots of accreditation reports; I see all the initial reports that are done each year for labs starting up or having significant changes, and I also see any troublesome ones, and the proportion of those that stayed about the same throughout the whole time. Laboratories come and go in that area, but I don’t think there has been any significant change. Probably my most memorable time in that whole process was that I had to go to the Administrative Appeals Tribunal to contest an appeal of a laboratory that was going to lose its accreditation, and they went through the legal route to try to prevent that from occurring. I did at the time think that they didn’t pay me enough to do that, which was true because they don’t pay me anything, so that was quite easy.

So just a snapshot on Pathology – there’s a lot of pathology activity, a lot of small episodes of care as you know – 50 million a year in Australia and a little over 50% of the Australian population has at least one pathology test done each year, so it touches a lot of people; not as much as primary care, which does about 5 times that volume. It has a critical role in about 70% of medical decisions and of course it’s used as the basis of all cancer diagnoses. Its structure in Australia is interesting in that about 60% of services are provided by the private sector and about 40% by the public sector, but the funding is predominantly government – about 90% - and the rest comes from the private health funds and also as out of pockets which are pretty small in Pathology, the smallest of any of the medical services area. Total expenditure in Australia is about 3 billion dollars so that’s about 3 to 4 percent of the total health spend.

When I was thinking about risks associated with poor quality in Pathology, it’s something that we focussed on for a long time, but in fact if you look at medicolegal statistics, the major medicolegal problem in healthcare in fact is failure to diagnose. It’s the category that ranks highest and most important and of course Pathology is important there. In Pathology you would think that it’s probably making a wrong diagnosis on material that they’ve got, so an interpretative problem, but in fact it’s very low down the list and in fact it’s the very banal issues like mixing up specimens, or misidentifying specimens, or losing specimens. Incorrectly interpreting things comes well down the list in fact and analytical problems, which was the focus of accreditation when we first started this process, is way down the list, and in fact it’s not an issue of great concern except for a very few tests. But of course at the end of the day, when it comes to Pathology, still the greatest risk is the pathology result that’s not acted on. So that’s greater than any of the risks that we see actually in the whole testing process.

So what is quality? Well clearly it’s the right result. Is it the converse? Is it lack of errors? What about improved health outcomes? Surely that’s got to be the most important thing? And from the point of view of a health system, increased economy and better value for money is a quality aspect as well. I think there’s only one … and quality of course in most people’s minds is just getting things right; but the problem is getting people to agree on what is right, and that’s a process that you’re going through now, it’s a process that we’ve gone through in Pathology and are still going through because we still have our discussions and arguments about that. At the end of the day I personally think that there is only one quality marker that’s worth anything and that’s life expectancy, and Australia’s got the fourth best life expectancy in the world, so we must be doing something right and you may say it’s not all the healthcare system, but the healthcare system has some significant role in that. And even more importantly, cancer which we know is now a leading cause of death in our society, is treated very well in Australia so here’s a pat on our backs for doing this. And it’s the same list of countries that comes at the top of the life expectancy ranking that comes in the ranking of mortality from cancer and so Australia’s number 5 in this list. And then you can look at it from the point of view of cost effectiveness and Australia sits in the middle of the pack, as you know, and America’s over on the right. You’ll remember that America wasn’t even on that list of life expectancy. So I think we’ve probably got the balance pretty right. But when it comes to looking at quality and making judgements about quality and implementing changes like accreditation systems, there’s often resistance to that sort of proposal and that sort of process. And I think this really sums it up that if you’re an optimist, you see an opportunity in every difficulty and that’s what I think introducing an accreditation system is. It’s a difficult process but there are some real opportunities in this. Instead of focussing on the problems associated with bringing in changes, we should recognise the problems with the existing system and how the changes will improve that system.

So, this is just a brief rundown of the elements of the Pathology accreditation program. We have NATA which you’ve had some exposure to – the National Association of Testing Authorities – and it’s a not for profit organisation that sees this as its job, assessing organisations and accrediting them. And if you fall into the laps of NATA as part of a program as we have, then you pay a subscription to NATA for what they do for you, but you’re a member of NATA too, so it’s an inclusive club.

The College of Pathologists has equal billing here, in medical testing accreditation, and they’re very important and I’ll point that out in a second. There’s a very strong regulatory authority or framework – it comes via the power of the Health Insurance Act, and Medicare Australia has a significant administrative role there, but of course the Department of Health and Ageing has too. There are all the various standards that we use – we assess against those standards and there’s the actual process of assessment and accreditation. And now we’re moving closer, further into quality system accreditation, and I will talk a little bit about quality assurance programs. So this is the role of NATA and the RCPA – it’s a joint program established in the early 1908s; there is a formal agreement, a memorandum of understanding – it’s reviewed every five years, it’s just been resigned.

NATA basically provides the infrastructure, so it provides the secretariat; it provides lead assessors who are generally laboratory scientists and this area of medical testing, and the whole infrastructure behind the assessment process. Whereas the RCPA, the College of Pathologists, provides in particular professional advice, and it actually recruits or helps recruit the volunteer assessors and it does that a couple of ways, but one of the mechanisms is through this being a CPDP activity, that is, being an assessor, and it strongly supports the program.

If there are problems with pathologists in the assessment process, and these are laboratories that are being assessed, then it often falls on the College to make some judgements in relation to that and do something about it. It is a strong program, and having seen the programs in other parts of the world, like New Zealand and the USA, Germany and the UK, I think it is unique in world terms and it is for a number of reasons.
Top of page
So this is one of the reasons: it does have a strong regulatory framework. Prior to 1986 it was a voluntary program, but from 1986 through Act of Parliament, so Commonwealth law, it became a mandatory scheme through the Health Insurance Act. So you don’t have to have your laboratory accredited, but if you don’t have it accredited, you can’t claim Medicare benefits and it’s an extremely powerful tool as you might imagine. And in 2003 the powers were increased somewhat to allow further intrusion into Pathology as required. So Medicare Australia has a sort of a, not a policeman role, but more of a ticketing role in that the reports that come through the accreditation process, once a favourable report is received, it’s sent off to Medicare Australia. The laboratories have to actually apply to get their laboratories accredited, and their total organisation registered as well, and you cannot actually go forward unless all the parts of the jigsaw puzzle are presented to Medicare Australia. So that’s their prime role. We had tightened up the way in which we deal with new laboratories. So these are applicant laboratories – previously these laboratories could set up and start testing and go on for up to 6 months before they were assessed. We’ve now said that that’s not acceptable, that in fact they have to have an assessment before they actually start formally producing results and billing to Medicare. And so they have a different routine for applicant laboratories as opposed to renewal of existing laboratories.

And there’s a bunch of standards that I mentioned before we use to assess against. We have an international standard ISO 15189, which is specifically for medical testing laboratories or pathology laboratories. And this important from NATA’s point of view because it’s got accreditation as an accrediting body and that’s on the world stage. But that’s a fairly general standard and we need more specific guidelines and standards than those that are present in the ISO standard, and that’s where NPAAC comes in and I’ll mention that in a second. And even that in fact isn’t specific enough in some areas, and that’s where this document called the “Field Application Document”, which is managed by NATA, comes in, and the College has a lot of input into that. So here’s NPAAC and this is its website, it’s hosted by the Department of Health and Ageing. The Department of Health and Ageing provides a secretariat, and Abel was key in that for a number of years. There’s a lot of information on that website if you want to go and have a look at it; it’s got all the standards on it, it talks about the structure of NPAAC, what it does. It also talks about the standards that are under review, and it has those standards that are under review and are out for public comment. And everybody in the laboratory community can see what’s going on; you can also have input into that process. So it’s a very open process and it’s very contestable, and the drafting committees of NPAAC look at all those responses to any new standards or draft standard that comes out. It has a philosophy, and that is that it pitches standards and guidelines at an aspirational level, albeit those that are absolute requirements are minimum standards. But it would have elements within the guidelines that ask laboratories to stretch themselves up to the next level, and it has identified what it believes to be best practice pathology in its standards.

There’s a focus on safety and quality etc. and this is an example: so this is our main document “Requirements for Pathology Laboratories” and this is the format. And I won’t go through it – if you want to look at these documents, they’re all publicly available, they’re all downloadable off the website. There’s a standard one which is “Laboratory Ethics” and it has those elements that are prescribed so they’re things that laboratories must do, and they’re assessed against those. It has a commentary; it will have elements to it where it has ‘shoulds’ as the clause that relates to that requirement and that means that they’re optional but it’s better if they do do it. And there are a whole lot of documents, and there is a hierarchy of these documents in the NPAAC setting, and the top document of course that drives the whole process is the Health Insurance Act in Pathology. But then you have these different tiers – we’ve got our main two documents, which are the “Requirements for Laboratories” and “Supervision of Laboratories”. Then we have more general requirements that all laboratories would have to fulfil as well, which you see below there, and a good example of that is this one: “Requirements for Retention of Laboratory Records”. It’s quite clearly stipulated how long we have to keep specimens, how long we have to keep various records, and people are assessed against that every time they have an assessment inspection. And then we have these sorts of standards which don’t necessarily relate to every laboratory because it depends on what the laboratory is doing, and some of these are very specific. And examples of those are the ones in relation to gynaecological cytology and things like that.

And then you come further down the list, and these are the international standards so NPAAC doesn’t have any role in managing those. We have - the Australian laboratory community, has the ability to input into it, but they’re obviously key in the process. And then there’s a whole bunch of other stuff that laboratories use in guiding what they do. NPAAC does have a strategic plan and it reviews it regularly and at the moment its main focus is quality and safety. And I think we’ve come to realise that we don’t have to focus so much on the analytical issues in the laboratory, we need to focus more on those things that are associated with high level risk, and that’s things like specimen identification, mix-up, that sort of thing, and so there’s a focus on that at the moment.

So NATA’s role? You know a little bit about NATA now because you’ve had your pilot project, but they have this thing called a “Field Application Document”, so you know, what do you do in this circumstance? There’s got to be a general agreement about that. It’s not so high level that it would be in an impact document; it certainly isn’t in the ISO standard. NATA will go to the College, to the advisory committees of the College and ask them ‘what about this’? How often do you run quality control samples on this analyser? That sort of thing. But this is a public document, it gets reviewed and it’s available for everybody to see so it’s an important part of the standards that we assess against. And this is the review process for these standards, as I said before, it is an open process, all this information is publicly available, and the review process is something that everybody’s invited to be involved in. So via Standards Australia there’s a subcommittee that has input into our ISO standard – that’s ISO 15189. Through the NPAAC process there are a whole range of people who are involved in that process – they’re all voluntary in that capacity. And of course all the drafts that produced by that process are then open for public comment and are circulated for that, and then there’s the same process can occur with the “Field Application Document” which is an even, as I mentioned before, higher level detail.

So this is the process of accreditation, so all pathology laboratories are accredited, and they can’t attain Medicare reimbursement unless they are accredited and successfully so. And they’re assessed at least once every three years, and some laboratories, if they have a less than perfect accreditation assessment, will get assessed more frequently than that, and sometimes they’re assessed every year. These … we’re looking at a change in the timing of that process, and there’s now a smaller accreditation inspection, which occurs every 18 months in between the major ones. And this is the process basically, that laboratories are asked for information about what they’re doing about their structure, about their personnel, and all that sort of stuff is then sent into NATA. They look at that and do an initial sort. For an initial assessment of a new laboratory, they will then go and visit the laboratory and assess it against the impact guidelines in the FAD. For a full assessment we need a panel of people.

And that panel of people includes peer assessors, and so that includes pathologists and scientists as well as another staff officer. They go through the assessment – it’s usually a one day process for each laboratory, or each section of a major laboratory, and at the end of the assessment process, which is done by audit against these standards, there’s an exiting interview and an accreditation report is prepared. These days our report is actually prepared on the day and given to the laboratory or the department at the end of that day. It is a draft and it may be changed, but not greatly, after, and then sent to the laboratory. The laboratory then gets the report which has this sort of detail in it – it will have conditions, findings where something has been found that has to be addressed, minor conditions, things that should be addressed but they’re not as important, and observations where it’s just there for the laboratory’s interest. And the laboratory is then expected to respond to that report. And I just thought I’d give you an idea – I don’t expect you to read all this – but this a recent report on one of the sections of my laboratory, at the cytogenetics section of my laboratory, and it finishes off here saying that the assessment revealed that the department is operating at a satisfactory standard; there are a number of matters however in which the department has not complied.

So, I stand before you, non-compliant. Every NATA RCPA lab testing accreditation report generally has that comment on it, because something is always found, there is always something to correct. This was for an extension of scope, and this because we added this test to our set of tests, and we couldn’t charge Medicare benefits, or get Medicare benefits, for that test until we’d been accredited. So that’s 107126. A few things were found, and this was one of the things – it related to FISH testing, which some of you are probably aware of, and it said we needed to do some more validation of that. This was the response – we gave a detailed response to NATA; NATA send the response off to the peer assessor, the technical assessor, and they’ve said “that’s fine”. And on the basis of that and other responses, we received accreditation for that new test and for that section of our laboratory.

You then go on to the NATA website, and this is what you see – you see, and again this is publicly available information, this is my lab again; there’s the scope of accreditation. I downloaded this yesterday so the new test has not appeared yet because we haven’t got our final certificate of accreditation for that test. When it does come it will appear on there, and it will be there for anyone to look up and check that we’re accredited for that test. And what’s also on the website is the schedule of reassessments that are occurring for each part of my laboratory, and every laboratory that’s involved in the NATA system. So that’s the process, and I just thought I’d mention a little bit more, a little bit about QAP. There’s a very strong QAP program in Australia; it’s one of the reasons why the Australian program is different. Most of the QAP is run by a company which is owned by the College of Pathologists and it produces this sort of data, and so every test has to have QAP performed on it, that’s every test that’s done in a laboratory, and of course every test that’s in a laboratory has to have QC done. So this is the summary of the accreditation process, there’s NPAAC and there’s NATA; different roles obviously – NPAAC for standard setting, NATA for running the accreditation assessment process, and then there’s the regulatory framework associated with that.

So where are we now? We’ve been operating now the medical testing accreditation process - it’s been going for 26 plus years. In 2001 there were 640 accredited sites in Australia, and last year there were 28 new sites accredited, and there were about 380 assessments occurred last year, so they’re being done more than once every 3 years. There are 1040 peer assessors registered with NATA, and that’s a combination of pathologists and scientists. So these are people who’ve been selected generally because their laboratory has a good assessment history, but also because of them and what the system is assessed is being what they can contribute towards.

So this is where we are now; we do have a mature quality framework, we’ve seen quality improvement over the time, and generally with a reduction in costs, so quality doesn’t … or accreditation assessment and this sort of quality focus should in fact lead to a reduction in costs, not an increase in costs. But there are clearly other factors at work here too, and it’s a competitive environment of Pathology that’s probably driven some of that cost reduction as well. And we’re now looking more at this non-analytical side of pathology testing to see if we can reduce the risks there.

So these are my conclusions – the Australian Pathology accreditation system has produced excellent outcomes at reasonable cost, and NPAAC and NATA working with the professional groups, especially the College of Pathologists, have been streamlining the framework and dealing with these sorts of new challenges which are arising. So my summary of quality is this: "I am easily satisfied with the very best" – not my quote, Winston Churchill, but I think it sums it up. Thanks very much.

(applause)
Top of page
Norman Swan:

So, how much heartache was there in making it mandatory by legislation?

Dr Michael Harrison:

It was the thing that dragged people who were reluctant starters to the table, Norman, and in fact there was some fallout, there were people who closed their doors because they couldn’t meet accreditation requirements.

Norman Swan:

Really?

Dr Michael Harrison:

Yes.

Norman Swan:


And what influence has corporatisation had on this?

Dr Michael Harrison:

Corporatisation generally has led in Pathology to aggregation and centralisation and a lot of standardisation, so in fact, corporatisation has probably improved the overall results that are seen through the accreditation process.

Norman Swan:

Because it’s a bit more command and control?

Dr Michael Harrison:

Yes it is. There’s less variation.

Norman Swan:

People talk about, you know, it’s fine doing all this but you need the quote unquote "culture change" – sounds to me as if you’d say that’s bull?

Dr Michael Harrison:

If it was a voluntary system you certainly would; where it’s mandatory, you don’t need that, you either actually shape up or you ship out.

Norman Swan:

What’s your perspective … some people talk, you know, the whole issue, not just in medicine and healthcare, is about voluntary, but self-regulation versus regulation, and, you know, you take the pharmaceutical industry for a moment, it’s self-regulated under Medicines Australia rules. It is voluntary, but you get pharmaceutical companies complaining about it, not because they’re complaining about Medicines Australia rules, but because it’s not a level playing field. As soon as you’ve got voluntary regulation, it penalises the good guys.

Dr Michael Harrison:

Yes. An abbreviation that I used a lot when I was in clinical medicine was NAD, so it does mean ‘no abnormality detected’ but it also means ‘not actually done’ and in fact part of the audit process with labs that we know have got a bad accreditation history is to go directly to those things that they have to report, so they have to put their quality assurance program results in, they have to do their QC, they have to respond to any QC outliers, they have to log their instrument maintenance and invariably if they’re a poor performer you’ll find that they fail in those areas, so I think really if you don’t have a policeman for this sort of stuff, that you will get people who don’t have a quality focus, who don’t do it.

Norman Swan:

And, just to get the process right, so in fact there’s like a desktop audit first, their self-assessment, they put it in, and then you hold them to account on what they’ve said they're doing.

Dr Michael Harrison:

That’s correct. Yes.

Norman Swan:

Right – it’s scary stuff!

Dr Michael Harrison:

Yes - and occasionally you find things they’re doing that they didn’t declare, and then it gets very interesting.

Norman Swan:

Right – like eating their sandwiches in front of the analyser or something like that?

Top of page

prev pageprev page| TOC |next page