Strengthening Nonrandomized Studies of Health Communication Strategies for HIV Prevention

This research makes the case for quasi-experimental designs to be used in evaluations of health communication (HC) interventions for HIV prevention when cluster-randomised trials are not feasible or appropriate and when observational studies may be biased. The article is from the JAIDS: Journal of Acquired Immune Deficiency Syndromes supplement addressing clinicians and public health scientists in the field of HIV prevention and treatment who might value information on health communication. (Footnotes removed by the editor.)

Factors that complicate impact evaluations of HC interventions include: diffusion of information throughout informal community networks; individual self-selection for exposure to these interventions; and communication initiatives that use multiple and layered intervention channels. "Cluster randomized controlled trial designs (cRCTs) respond to many of these issues. Broadly speaking, randomized trials control for confounding by design, whereas observational studies do so by analysis....cRCTs offer the least-biased and simplest approach to estimating and understanding the intention-to-treat (ITT) effect....The most significant problem for an evaluation seeking to estimate ITT effects in a manner analogous to a cRCT is residual confounding by unknown, unmeasured, and/or imprecisely measured factors. This article aims to improve and encourage the use of quasi-experimental designs in evaluations of HC strategies."

The discussion includes 4 quasi-experimental research designs (shown in Table 1 in the document) that are based on implementation scenarios in attempt to establish a “middle ground” of cluster-level quasi-experimental designs in situations where the following are clearly documented:

1. "The intervention components;
2. Criteria that determine which clusters are eligible to receive the intervention; and
3. Criteria that determine which eligible clusters will actually receive the intervention; we refer to this as the presence of an allocation scheme."

Design 1, Nonrandomized Controlled Comparison, may be applied when the implementation plan allocates some eligible clusters to receive the HC intervention but not others. Thus, there are comparison groups: "The evaluation design may exploit this variation between clusters. For example, community-based HC programs, such as those that involve community drama; peer educators, or other change agents recruited in the community; or other interpersonal channels of communication, are typically implemented in a subset of communities within an overall project area."

Design 2, Interrupted Time Series, may be applied where there are significant time differences in when an intervention is allocated. "This scenario may occur for programs relying primarily on national mass media channels, which typically have well-defined phases separated by periods of time with little or no activity, but little or no planned variation geographically. For example, in Brazil PRO-PATER implemented 3 separate mass media campaigns promoting vasectomy in 1983, 1985, and 1989, and the number of vasectomies increased markedly during each campaign period."

Design 3, Phased Implementation, may be applied when the implementation plan allocates clusters to initiate the intervention at different times, with eventual initiation in all clusters. "As an example, the Bridge Project in Malawi, between 2001 and 2008, used mass media and community-level interventions to communicate HIV prevention messages. Initially implemented in 8 of Malawi's 28 districts, it has since expanded to 11 more districts. When randomization determines when places are allocated to receive interventions, the evaluation design is known as a 'stepped-wedge' or 'phased-implementation' cRCT design."

Design 4, Implementation Strength, may be applied when the implementation plan may entail variation in the strength of the intervention allocated to clusters, with some clusters allocated a greater "dose" of activities than others. "For instance, the COMMIT project in Tanzania used both mass media and community-based activities to communicate messages promoting behaviors to reduce the transmission of malaria. The program's mass media messages reached all communities, but only some clusters had community-based group activities, and in even fewer communities the project recruited community members to serve as local change agents promoting malaria prevention. The allocation of these channels across communities would allow program evaluators to measure the dose of the intervention for each cluster."

These approaches, as stated here, emphasise whether the programme had an effect, not how it may have influenced behaviour. Assessing the theory of change to determine effectiveness of specific messages takes a comprehensive evaluation. A potential limitation of the approaches is that allocation of messages is measured, but exposure to messages might vary. However, external validity of these approaches may arise from delivery "at scale and with the budget and oversight of real-life implementation" possibly giving greater external validity than "a cRCT performed in limited conditions with an unrealistic implementation budget."