Analysis focused on synthesising qualitative information from interviews and literature, validating findings from the various sources and evaluation methods used, and corroborating the quantitative analysis, particularly on the CHCI program, with the qualitative (largely case study) information. These processes occurred iteratively and involved identifying relevant concepts, themes, conflicting and converging ideas, assumptions and theories, interpretations and rival interpretations structured around the evaluation objectives.
The body of evidence—the story of the CHCI and the EHSDI—built progressively from single sources and methods (individual interviews or case studies) to multiple sources and methods (multiple interviews and cross-case analysis). The process included corroborating interview findings with program data, workshop discussions and research literature and, for CHCI, the information from the analysis of health datasets. We continually revisited our findings to check whether and how the supporting and relevant evidence fitted with
the emerging findings.
Generally, we considered data or evidence to be more valid, and therefore gave it more importance, when we were able to validate it with data collected from other sources or by other methods. We were very conscious of the importance of context and the risks of generalising. This required us to look for areas of convergence and divergence and then to assess whether this could be explained by different contexts.
The dynamic and emergent nature of the EHSDI made it difficult—and we would argue inappropriate—to set strict criteria against which to evaluate the program. There is no established program theory specifically for the EHSDI on which to base criteria. In the analysis we have sought to build up and test a theory for the EHSDI in a way that is useful both for future implementation of the program and for future evaluation activities.
These issues are also important for the evaluation of the CHCI. Again, there was no established theoretical model for the program and no clearly articulated outcome against which to judge success. There is, however, an existing and well-established evidence base for health screening and this did provide some criteria for evaluation (Wilson and Jungner 1968).
As a result of these challenges, the analysis focused on a process of explanation building. This process is similar to that of refining a set of ideas:
- making an initial statement or proposition
- comparing the findings against such a statement
- revising the statement—comparing other details of the findings against the revised statement
- comparing the statement to the other areas of analysis
- repeating as necessary.
and in the process dispel any rival explanations. This process fits with the iterative nature of our analysis, and
with the emergent nature of the EHSDI.
Top of page