According to the CDC guidelines data quality “reflects the completeness and validity of the data recorded in the public health surveillance system”. Data quality is concerned with the accuracy of data for decision making purposes. It is related to the validity of the information that is recorded for the purpose at hand, and its completeness. One aspect of validity is the precision of the system in relation to prevalence of trachoma. It was determined earlier that the system does not need to be extremely precise, and it was established that it is reasonably precise in that the prevalence results are consistent with those of a recent study using the gold standard approach for measuring prevalence.
Another aspect of validity is how useful the data are for monitoring and controlling trachoma, which is the stated purpose of the current system. The feedback from stakeholders varied considerably. Some stated that the surveillance data are fairly useful apart from a few small gaps, however feedback of data from the system is an issue. Others considered that the data does not reflect the current situation due to resource limitations and therefore the data are never used. The evaluation team has already concluded that better systems and processes for feeding back information to local health services staff are required.
As stated in the CDC Guidelines, “the quality of the data is influenced by the performance of the screening and diagnostic test for the health related event, the clarity of hardcopy or electronic surveillance forms, the quality of training and supervision of persons who complete these surveillance forms, and the care exercised in data management”. Computer-based forms with cross-checking of information yield more accurate results than paper-based systems with no controls. However, data collection and entry is currently paper based, and there are inadequate systems for checking data quality. Computer-based data entry, if well designed, will contribute to resolving this problem.
In relation to training of staff collecting the data, the general consensus from stakeholders was that staff in trachoma surveillance and control roles are generally appropriately trained. Problems with training were encountered with staff from the local communities, where the interest levels and skills varied. It is important to engage local health services workers in screening for trachoma and consequently effective systems to engage and train staff local workers need to be designed and implemented. It is recognised that this task presents challenges particularly as health services staff in local communities are very mobile, but it is important that the training solution is systematic so that it functions independently of the people occupying roles at a point in time.
The flow of data can also facilitate data quality. The entry of data soon after (or preferably as part of) the screening event is considered much better than entering data some months after the screening event where recollections are not so good. Some delays in entering data were reported by stakeholders, as was re-handling (e.g. data were entered into systems more than once such as in local systems and then aggregated separately for national reporting purposes). Therefore, streamlining of reporting processes would improve data flow and lead to better quality data. This improvement can be achieved through more active efforts to obtain extracts from local information systems were data are entered at a person level.
There is no doubt that timely feedback on data allows the data entry person to review the data they have entered and consider whether it reflects their understanding of the current situation. This process also has the capacity to improve the skills of staff in capturing data with greater accuracy. The evaluation team has found that the feedback process is limited, and certainly there is little involvement of local level staff (often because feedback is not timely). The development of a facility for local generation of reports on the surveillance data would improve this situation and therefore enhance data quality.
Findings: With respect to the data quality component of the current national trachoma surveillance system, the evaluation concludes that:
- the current processes for data collection do not support data quality, and do not lend themselves to easily measuring the current level of quality (except for lack of completeness).
- the system can be enhanced through:
- web-based entry of data and/or extracts from local systems where controls are built into the system to assist with data quality (e.g. logical cross checks);
- developing training materials/packages (which could include web-based training) for training local staff in trachoma screening and control activities to assist jurisdictional level coordinators in this process; and
- creating a capacity to retrieve flexible user defined reports from the NTSRU access database, immediately following data entry.top of page