Better health and ageing for all Australians

Point of Care Testing Trial Report - Executive Summary

1. Is it safe to perform PoCT in general pratice?

Up to Point of Care Testing

prev page| TOC |next page

i) Performance in quality management


The methods used to establish a quality management system for the PoCT in General Practice Trial, and the results of quality testing undertaken to assess the analytical performance of the devices used in the Trial, are summarised below.

The quality management system incorporated;
(i) an ongoing training and competency assessment program to ensure PoCT Device Operators had the skill set required to conduct PoCT safely and effectively

(ii) the provision of Internal Quality Control (IQC) and External Quality Assurance (EQA) programs to continually monitor both operator competency and the analytical quality of the PoCT devices used in the Trial

(iii) a comparison between the observed performance for IQC and EQA testing with the profession-derived analytical goals that were established for each PoC test for the Trial. The PoCT Device Group was primarily responsible for training of practice staff and the implementation of an IQC program, while the Quality Assurance Program Group was responsible for the delivery of an EQA program. IQC and EQA are established quality practices undertaken by all Australian pathology laboratories.Top of Page

Key Findings

  • all 80 Device Operators passed their initial competency assessment prior to the commencement of the live phase of the Trial
  • the overall participation rate for IQC and EQA testing across the live phase of the Trial averaged greater than 90% for all POC tests
  • Device Operator competency skills were maintained throughout the live phase of the Trial with only one operator having their competency revoked
  • Device Operators conducted IQC testing to an analytical standard that met the goals set for the Trial for each PoC test (except HDL-C Level 1 QC where less than 60% of practices met the goal)
  • Device Operators conducted EQA to an acceptable level of accuracy when compared to pathology laboratories (84% to 98% acceptable results)
Device Operators conducted EQA to an acceptable level of precision that met the goals set for the Trial for HbA1c and urine ACR. The other tests, particularly total cholesterol and HDL-C were not able to meet all assessments of precision.

ii) Standards and accreditation for PoCT in general practice


A PoCT sub-committee first met in 2002 to begin developing standards to be followed in the Trial. It agreed that the National Pathology Accreditation Advisory Council (NPAAC) and the Royal Australian College of General Practitioners (RACGP) be approached to develop standards. NPAAC subsequently set up its own sub-committee to develop the standards with GP representatives of the RACGP included.

Members agreed to base the standards document on the headings outlined in the Quality Use of Pathology Committee PoCT Technical and Clinical Working Group Draft Guidelines for Use of Point of Care Testing on General Practice in Australia documents.

In March 2003, a first draft of the standards was circulated to various experts for comment. By June 2004, the standards had been finalised.

The standards were designed to support general practices in using PoCT to enhance the quality of patient care. The standards were intended to accommodate the expectation that PoCT and its clinical applications would significantly expand in the future, whilst the PoCT Trial would contribute to refining them to better meet the long-term need.

An evaluation of the (Interim) Standards for PoCT in General Practice and the accreditation program developed for the Trial was undertaken. As part of the analysis of the safety of PoCT in general practice, the results of practice compliance with the (Interim) Standards were also reviewed.

A Working Group developed the PoCT accreditation program based on the (Interim) Standards for PoCT in general practice. The accreditation program included: development of resources for practices participating in accreditation; development of an assessment process; and assessment through a site visit by a survey team and report on the outcome of the site visit.Top of Page

Key Findings

  • for the first round of accreditation, 90% of practices passed; 1 of the 3 practices which did not comply with the requirements subsequently met the criteria and was accredited. The remaining 2 practices voluntarily withdrew from the Trial
  • for the second round of accreditation, 100% of the practices complied fully with accreditation requirements
  • evaluation of the accreditation process indicated that practices and members of the survey team found the process appropriate
  • in terms of the (Interim) Standards, these were seen as realistic and achievable for general practice
  • accreditation surveyors reported that the (Interim) Standards provided an achievable minimum standard for general practice
  • GPs and Device Operators found the (Interim) Standards applicable and relatively easy to use.

iii) Comparison of PoCT and pathology laboratory test results


An analysis of the comparison of PoCT and pathology laboratory test results was undertaken as part of Phase I of the Trial. This analysis formed part of the assessment of whether PoCT was safe to perform in general practice.

Pathology laboratory test results were matched with PoCT test results for patients in the intervention group for the first six months of the Trial. Agreement was assessed using three methods: the Bland and Altman approach which includes mean differences and 95% limits of agreement followed by clinician review to determine the clinical acceptability of the results; analysis of the concordance between PoCT and laboratory results; and for INR assessment using published clinically relevant agreements. Top of Page

Key Findings

  • the estimated bias for INR was 0.0034. This means that on average PoCT results were 0.0034 above the corresponding pathology laboratory test results. The 95% limits of agreement were - 0.7851 and 0.7919
  • the estimated bias for HbA1c was -0.0504%. This means that on average, PoCT results were 0.0504% below the corresponding pathology laboratory test result. The 95% limits of agreement were -1.0658 and 0.9650%
  • the estimated bias for urine albumin was –0.7632mg/L. This means that, on average, PoCT results were 0.7632mg/L below the corresponding pathology laboratory test result. The 95% limits of agreement were -11.4719 and 9.9455mg/L
  • the estimated bias for ACR was –0.1513mg/mmol. This means that on average, PoCT results were 0.1513mg/mmol below the corresponding pathology laboratory test result. The 95% limits of agreement were -2.0488 and 1.7461mg/mmol
  • the estimated bias for total cholesterol was –0.2645mmol/L. This means that, on average, PoCT results were 0.2645mmol/L below the corresponding pathology laboratory test result. The 95% limits of agreement were
  • -1.0931 and 0.5641mmol/L
  • the estimated bias for HDL-C was –0.0694mmol/L. This means that on average, PoCT results were 0.0694mmol/L below the corresponding pathology laboratory test result. The 95% limits of agreement were -0.4899 and 0.3510mmol/L
  • the estimated bias for triglycerides was 0.2014mmol/L. This means that PoCT results were 0.2014mmol/L above the corresponding pathology laboratory test result on average. The overall 95% limits of agreement were -0.9540 and 1.3569mmol/L
  • concordance analysis found that 23.3% of INR results, 15.1% of total cholesterol results, 14.2% of HDL-C results, 10.6% of HbA1c results, 10.5% of ACR and triglyceride results and 2.5% of urine albumin results theoretically could have led to a different decision depending on whether the test was from the laboratory or PoCT
  • evaluation of INR results showed that, overall, 86% of dual INR measurements were within 0.5 INR units of each other. For laboratory INR <2.00, 2.0-3.0, 3.1-4.0 and >4.0, 94%, 90%, 69% and 67% of readings were within 0.5 INR units, respectively
  • for INR, clinical agreement occurred 91% and 89% of the time against published expanded and
  • narrow criteria, respectively. Top of Page

iv) Serious adverse events and incidents


An analysis of the serious adverse events (SAEs) and incidents reported during the Trial, which formed part of the assessment of the safety of PoCT was undertaken. SAEs were monitored by a Safety Subcommittee which assessed and made a recommendation to the Trial Management Committee regarding the likelihood or otherwise that a particular SAE was related to the Trial.

Weighted estimates of the number of SAEs, the number and percentage of patients experiencing one or more SAEs and the number of SAEs per 10,000 person-years were calculated both overall and by treatment group. A descriptive analysis of Trial incidents was also undertaken.

Key Findings

  • no SAE reviewed was assessed to be attributable to PoCT
  • for all three conditions, the number of SAEs per 10,000 person-years for the intervention (1319) group was lower than for the control (1446) group
  • overall, the proportion of patients experiencing one or more SAE was the same for both the intervention (13%) and control (13%) groups
  • a larger number of incidents was recorded by the Trial Management Group from patients and practices in the intervention group and that number reduced over the period of the Trial
  • most calls to the QC/QA hotline related to interpretation and recording of QC results
  • 15 test error codes were recorded during the Trial and these errors related to pre-analytical or operative factors.Top of Page

Key Conclusions (for i to iv)

  • in terms of performance in quality management it was safe to perform PoCT for HbA1c and ACR; however, the results were less clear for INR and lipids
  • the methods established for the implementation and delivery of training, competency assessment, IQC and EQA programs were appropriate for the General Practice Trial and Device Operators conducted PoCT to a generally acceptable analytical standard
  • the accreditation program developed by the Trial, based on the (Interim) Standards for PoCT, provided an acceptable framework for evaluating quality management for PoCT
  • practices were able to meet the (Interim) Standards and obtain accreditation
  • the (Interim) Standards for PoCT in general practice were acceptable to GPs and Device Operators and members of the survey teams
  • a comparison of PoCT and pathology laboratory test results determined that the mean difference in results and the 95% limits of agreement were clinically acceptable
  • PoCT did not result in a higher number of SAEs.

    prev page| TOC |next page