Final report - Risk factors for eye disease and injury

3.3 ‘Winnowing’ and assessing papers

Up to Eye Health

| TOC |next page

You may download this document in PDF format:

PDF printable version of 3.3 ‘Winnowing’ and assessing papers (PDF 16 KB)

Relevant articles were selected using a two-stage process (Figure 3.1). We first downloaded the results of the literature searches into individual Endnote databases. We then ‘winnowed’ the articles by considering the titles and abstracts, excluding any that were obviously irrelevant; for example, those that were speculative and contained no evidence from epidemiological studies, and those for which the topic under review was not the main focus of the study.

The next step was to obtain full copies of the remaining articles in each Endnote database. These were primarily articles describing epidemiological studies of the relevant risk factors, eye infections and eye tests, and other articles directly relevant to the topics outlined above. The most relevant of these papers were then selected for possible inclusion in the review.

Given the broad scope of the review, it was not possible to follow best practice and have each database examined independently by two people who would then compare the results. However, each database was looked at twice (ie once to winnow out obviously irrelevant or inappropriate studies, and once to select the most appropriate of the remaining articles). In most cases, a different person examined the database at each of these stages, providing some independent review within the selection process.

We initially selected systematic reviews and primary research articles involving clinical trials (studies of the highest level). However, where high-level studies were lacking, or where the question under review covered a broad category (eg sport), we also selected studies of a lower level, such as case series and cross-sectional studies. Each article retrieved was assigned a unique identifying number.

We reviewed the papers for a topic as a whole and selected those that were of highest quality and most relevant. Where there was a systematic review that was completed at an earlier date than the end date for this search (eg completed in 2004, when the search terminated in December 2006), we also considered any relevant papers published after the completion date. Every effort was made to include all relevant issues; however, given the breadth of the topics, we may have missed some relevant papers.

At each stage of the process, we kept electronic records of the search and analysis. Thus, for each combination of risk factor and condition, we produced an Endnote database showing the full search results, the papers winnowed out at the first pass and the final papers selected, with notes on type of study, level of evidence and reasons for including or excluding the studies. We also produced an Excel spreadsheet indicating the name of the person undertaking each stage, the date at which it was completed and the number of papers remaining at each stage. These electronic records have been provided to the NHMRC in electronic form (on a CD-ROM); hard copies of all included papers were also supplied to the NHMRC. A summary of the results of each search, from search terms to final number of papers used to determine the finding, is given in Appendix 3.

Note: The numbers given in the figure represent a single, typical search


Figure 3.1 Summary of process for selecting and assessing papers

| TOC |next page