Download Chapter 8: Using the Study Results to Design

CHAPTER EIGHT
USING STUDY RESULTS TO DESIGN INTERVENTIONS
This chapter deals with selecting and designing interventions.
Decisions about intervention design should be influenced by what
you know about the effectiveness of different types of intervention,
together with what you have learned about the specific problem
you are addressing and the context in which it occurs. After a brief
overview of interventions, the chapter provides guidance about
using the information you collected to design interventions.
8.10: Intervention Strategies
The ultimate objective of the applied qualitative studies described in this manual is to design an
intervention to improve an identified drug use problem. To assist in this process, you carried out
targeted studies to answer a series of questions aimed at learning more about the underlying
causes of the problem, the motivations of prescribers and consumers, and possible constraints to
changing their behaviors. Armed with this new information, you are now ready to design an
intervention that you feel will be most likely to stimulate change.
8.11: Type of Interventions
When designing interventions, there are many options to choose from and a range of factors to
consider. Before launching into the design process, it is helpful to have an overview of the kinds
of interventions that have been tried in other settings. Interventions can be broadly grouped into
three categories: educational, administrative, and regulatory. Although it helps to organize our
thinking about interventions to group them in this way, it is important to note that the most
effective interventions combine elements from all three types of intervention.
1.
Educational Approaches
Educational approaches are based on communicating information and persuading health
providers or consumers to behave in a different way. They are most helpful when knowledge
deficits, mistaken beliefs, or access to biased information are contributing causes of the observed
problem. Because the behavior and opinions of peers can be very persuasive in stimulating a
8-1
change in behavior, educational interventions are also very promising when it is possible to
facilitate learning in peer groups of health providers or patients.
Examples of commonly-used forms of education include:
!
!
!
!
2.
printed materials -- clinical literature, newsletters, advertisements, etc.
training programs -- basic, post-graduate, and in-service
face-to-face persuasive approaches, either one-on-one or in small groups
media-oriented approaches, e.g., posters, radio, television
Administrative Approaches
Administrative (or managerial) approaches use processes and tools designed to guide decisionmaking by health providers or consumers. Administrative approaches can work either through
supportive processes that make it easier to perform a preferred behavior, or through barriers
against discouraged practices. These supports or barriers may include:
!
!
!
!
3.
guides for prescribing and dispensing -- limited lists of drugs, standard treatment
guidelines, clinical pathways, standard order forms, etc.
systems for improving the selection, procurement, and distribution of drugs
drug utilization review combined with feedback to providers
financial incentives -- practice budgets, patient cost sharing, drug pricing, etc.
Regulatory Approaches
Regulatory approaches are designed to restrict decision making -- to remove choices about drug
use behavior from the hands of prescribers, dispensers, or consumers and put them in the hands
of policymakers or managers. There are many possible areas of regulatory activity, including:
!
!
!
!
market controls -- banning drugs that are unsafe or of doubtful efficacy, refusing
to register products that are not cost-effective, etc.
licensing restrictions -- enforcing regulations about prescribing by non-physician
health workers, or pharmacy dispensing of prescription-only drugs, etc.
prescribing controls -- limiting certain drugs to particular types of prescribers,
changing products from prescription-only to over-the-counter, etc.
dispensing controls -- requiring adequate product labeling, mandated patient
counseling by dispensers, limiting the number of drugs dispensed per patient, etc.
8.12: Strengths and Weaknesses of Interventions
8-2
No intervention approach is equally effective at dealing with every problem or perfect for every
situation. In general, each of the three broad categories of intervention has its own strengths and
weaknesses, and particular interventions in each category may be more or less effective
depending on the circumstances. Table 8.1 summarizes some of the overall strengths and
weaknesses of the three basic intervention approaches.
Table 8.1: Strengths and Weaknesses of Intervention Approaches
Intervention
Strengths
Weaknesses
Educational
Approach
! works best if knowledge deficits
are an underlying problem
! best results if message is clearly
focused on specific issue
! more effective with single
individuals or small groups
! repetition and reinforcement of
messages strengthens results
! knowledge often cannot
overcome system barriers
! disappointing results with broad
messages and large groups
! can be labor intensive if there is
a large target group
! transfer of staff or counterpromotion by drug companies
can dissipate results
Administrative
Approaches
! works best when systems can be
set up to make it easier to
follow recommended behaviors
! can be used to support and
sustain educational programs
! very effective if target group
assesses own practices
! improved supervision can have
positive spin-off effects
! open to abuse if administrative
changes are not accepted by
target group
! formularies, guidelines,
protocols need periodic revision
! information systems may be
hard to establish and maintain
Regulatory
Strategy
! works best if safety is an issue,
and problem behaviors are easy
to isolate and eliminate
! frequently easy to implement
! can give powerful and rapid
results for certain problems
! best if combined with other
approaches
! frequently produces unexpected
negative results
! may be open to abuse
! often difficult to enforce
! impact difficult to measure
8-3
8.13. Caution About Regulatory Approaches
Regulatory interventions often cut with a blunt sword. Although they are frequently easier to
implement than educational or administrative interventions, they can have unintended side
effects. Examples of some of the ways regulations can fail to be effective include:
! prescribers substitute drugs that are clinically worse or more costly than the products
removed by the regulation
! consumers turn to Aunofficial@ health providers or to the black market to gain access to
banned drugs
! problem behavior changes while a regulation is in effect and enforced, but returns to old
patterns as soon as enforcement lessens
! prescribers or dispensers change their record-keeping practices to make regulatory
oversight difficult
One example of an unintended negative effect occurred in a country that banned all antidiarrheal
drugs for children. After the products were removed from the market, there was a reduction in
the use of antidiarrheals, but increases in the use of liquid antibiotic preparations and
metronidazole were also noted. It appears that prescribers had substituted antibiotics and
metronidazole, which they were allowed to use for infective diarrhea, for the antidiarrheals that
they were being educated not to use.
Implementing a regulations without taking into account the understanding and acceptance of
prescribers and consumers may cause unwanted side effects. For instance, generics prescribing
policies are in place in public health facilities in many countries. When prescribers are not well
informed about the advantage of generics, they may not comply with generic prescribing. They
may also transfer their negative perceptions to patients, which will further jeopardize the
implementation of the policy.
In addition, withdrawing drugs without proper information to the general public can create
confusion and uncertainty for consumers, especially if they use these drugs routinely. For
regulatory actions to be effective, there is usually a need to incorporate information and
education to health providers and consumers.
8.14: Combining Approaches
Interventions are usually much more effective when they combine a number of different
approaches to attack the same problem. Education of prescribers aimed at encouraging a new
approach to treating a certain problem can be combined with posters or pamphlets aimed at
patients and consumers supporting these changes. Administrative systems like formularies,
standard treatment guidelines, or utilization review can be put in place to support the changes
targeted in an education program. Regulations that remove some drugs from the market can be
8-4
combined with education for prescribers and consumers to explain why this action was taken and
to suggest alternatives. Each of the activities in a combined intervention reinforces, sustains, and
extends the overall impacts.
For example, a successful intervention occurred in Mexico City where the treatment of diarrhea
in social security clinics was unsatisfactory. The first component of the intervention was an
educational workshop for clinic staff to develop their own treatment algorithm for this problems.
After the workshop, treatment of children with diarrhea according to the algorithm improved
from 24.5% to 51.2% of cases. For the next six months, the physicians who attended the
workshop monitored their own practice and participated in a process of peer review, which
further improved appropriate case management to 71.6%. After 18 months the improvement had
been maintained. During this period, there were only minor changes in a control group of
prescribers who had not participated in the workshops and peer review.
8.15: Making Intervention Choices
When designing an intervention, the challenge is to choose an approach that will produce the
desired changes for the least cost without any unnecessary negative consequences. Some key
factors to consider in this choice include:
! Likelihood of Success: Choose an approach that is likely to succeed based on experience
elsewhere or on your own analysis of the situation.
! Cultural and Political Feasibility: Choose programs that fit well within your cultural,
political, and bureaucratic environment.
! Technical Feasibility: The availability of technical resources like trainers or data systems
must match the requirements of the intervention.
! Cost: Interventions that require large recurrent expenditures, or where savings on drug
expenditures do not clearly exceed costs, are not likely to be politically acceptable or
financially feasible.
! Potential for Broader Impact: Interventions that can be extended to other members of the
target group, to the rest of the country, or to other drug use problems should receive
higher priority.
! Potential for Negative Impact: Interventions that might result in negative outcomes -e.g., shifts to other expensive or unsafe drugs, increases in use for other services,
diversion of patients to other providers -- should be carefully evaluated before they are
implemented on a widespread basis.
In general, no intervention is final. There is always a need for sustained effort, further supportive
measures, and additional refinements. Interventions that do not succeed immediately can be
strengthened based on information you collect during evaluation. Therefore you may want to
begin with the intervention that is limited, feasible, and relatively simple to implement, measure
its effectiveness, and then extend it in ways that are more complex or demanding.
8-5
8.20: Using Applied Qualitative Data in Intervention Design
The previous chapter described a process for systematically answering a set of questions you had
posed to gain a more in-depth understanding of a problem. From discussions at a synthesis
meeting, you and the other investigators should have a clearer idea about the causes of the
problem and factors that will encourage or discourage change. The specific problem behaviors
should be clearly defined, as should the target groups of prescribers or patients likely to gain the
most from an intervention. You may have ideas about the general style of intervention that might
be effective. However, because until this point you have been exploring the problem rather than
designing an intervention, many details and practical issues may still be unclear.
You must now focus on using the information gained during your applied qualitative study to
design the details of an intervention. If time permits, you can begin this process as the last
activity in a synthesis meeting, so that the individuals involved in the applied qualitative studies
can contribute. However, that setting and group of people included in such a meeting may not
always be appropriate for this task. In addition, some aspects of intervention design may require
additional fact-finding or discussion with others who were not part of the applied qualitative
studies. Depending on your own situation, decide how best to undertake the activity described
below.
Table 8.1 contains a list of issues that you should consider when designing an intervention. For
interventions that will have an educational component, the issues involve: (1) defining target
groups and target behaviors; (2) selecting the best educational approach and materials; and (3)
implementing the educational program. For administrative approaches, the issues to consider
include: (1) the appropriateness of various supportive administrative systems, such as
formularies, clinical guidelines, audit and feedback, reminders, supervisory programs,
incentives, etc.; and (2) strategies for implementing administrative changes effectively. When
considering regulatory changes, important issues include: (1) strategies for promoting positive
changes following regulation and for minimizing unintended negative impacts; and (2)
implementation monitoring and enforcement. Finally, for all interventions, whatever the mix of
approaches, one key topic to consider during intervention design is monitoring, evaluation, and
dissemination of results.
One way to proceed with the intervention design process is to systematically consider each of the
questions in Table 8.1. These questions address a large number of design decisions ranging from
the early stages of intervention planning through implementation and evaluation. Determine if
each of the issues is relevant to your situation, and if so, how you will account for the issue
during intervention design. Refer throughout this process to the data lists and summary
statements you prepared during the synthesis session, since these should give you concise
answers to many of the major issues to be dealt with during intervention design.
8-6
Table 8.1 Issues to Consider in Designing an Intervention
Type
Educational
Interventions
Questions to Consider
Target Group and Target Behaviors
! Who is the target group for the intervention?
! What behaviors are expected to change and what alternatives will be
recommended?
! What specific knowledge deficits are associated with problem
behaviors?
! Why should target group members adopt the new behaviors?
! How will the members of target group be contacted?
! How will the objectives of the intervention be explained?
Educational Approach and Materials
! What is the best educational format: self-learning, one-on-one, small
groups, seminar?
! How can peer relations or opinion leaders be used to facilitate change?
! What are the main intervention messages?
! How can communication between health workers and patients be
improved?
! What print materials will be used to deliver the educational messages?
! Who will prepare printed materials?
! How and with whom will printed materials be pre-tested?
Implementation
! Who is the most credible sponsor of the educational/training program?
! Who will conduct the actual educational/training activities?
! How will the educators/trainers be trained?
! Who will plan and schedule the educational/training activities?
! Where will the educational/training sessions be carried out?
Administrative
Interventions
Designing Administrative Systems
! Would implementation of formularies, limited drug lists, or clinical
guidelines help to change practice?
! What incentives can encourage health workers to use formularies or
clinical guidelines?
! Do prescribing and dispensing practices very widely among health
workers or facilities?
! Are health workers aware of how their own practices compare with
those of their peers?
8-7
Type
Questions to Consider
! What would be the best way to collect and feed back data to health
providers about comparative practice?
! Are some poor practices due to forgetfulness by prescribers or failure to
consider alternatives?
! How could reminder systems be used, e.g., management information
systems, structured order forms, etc.?
! How can the supervisory system or self-monitoring be used to foster
and sustain improvement?
! How does the system or promotion or evaluation influence practice?
! Are there physical or infrastructural limitations that will prevent
change?
! What is the influence of financial incentives on practice?
! Do consumers have sufficient information about drug pricing to guide
their decision making?
Implementation
! What training will be needed for the prescribers and dispensers who
will be using the new systems?
! What training will be needed for the staff expected to implement and
monitor the planned systems?
! If formularies, drug lists, or clinical guidelines will be used, who will
participate in their development and how will they be kept up to date?
! What supporting printed materials are required?
! How can positive changes in practice be reinforced over time?
Regulatory
Interventions
Impacts of Regulatory Changes
! Are there existing regulations that contribute to problem practices?
! Which specific regulations need to change to improve practice?
! Would limiting access to certain drugs improve their use?
! What would be the impact of removing problem drugs from the market?
! How are providers and consumers likely to respond to the proposed
changes in regulations?
! If access to certain drugs is restricted, which drugs (or other services)
are likely to be substituted?
! What is the potential for shifts in utilization as a response to regulatory
changes, e.g., more use of traditional providers, increase in sales on the
black market, bypassing of the referral system, etc?
Implementation
! Which educational programs are needed to explain regulatory changes
8-8
Type
Questions to Consider
to health providers and consumers and prevent unintended effects?
! Would active enforcement of regulations improve practice?
! Which staff are available to enforce compliance with regulations?
! How will responses to regulatory changes be monitored?
All
Interventions
Monitoring, Evaluation, and Dissemination
! Who will supervise the implementation of the intervention?
! What data systems need to be developed to monitor changes in
practice?
! Who will collect and assess monitoring data?
! Who will be responsible for evaluating the impacts of the
intervention??
! What are the key outcome measures?
! When and how will outcome data be measured?
! What data will be collected to measure implementation cost, efficiency,
and cost-effectiveness?
! When will the target groups be informed about results?
! Who will prepare the report evaluating the intervention, and when?
! How will results of the intervention be disseminated to policymakers?
8.30. EVALUATING THE IMPACT OF INTERVENTIONS
Evaluating whether an intervention achieves its desired impact -- i.e., improving use of drugs in a
cost-effective manner -- is often neglected. In many situations, interventions are implemented
without prior field testing to demonstrate their effectiveness in the existing system. This risks a
waste of financial and human resources and may further increase the burden of health care
services. It is wrong to assume that if an intervention takes place, its behavior change objectives
have been accomplished. For example, the fact that clinical guidelines for primary care facilities
are produced does not mean that prescribers will utilize the guidelines, or that prescribing
practices will improve.
Evaluating impacts is an important component of any intervention strategy. The following points
should be considered in designing an evaluation:
! An evaluation plan should be developed along with the implementation plan. Before any
intervention is disseminated widely, it is imperative to test whether it can be successfully
implemented in the local health care setting.
8-9
! Impacts of an intervention should be evaluated according to the most relevant behaviors and
messages targeted. Depending on the objectives of the intervention, evaluation measures
may include changes in knowledge or perceptions about a specific practice, the process of
care, the types of drug prescribed, the cost of prescribing, changes in clinical outcomes, and
so forth. Indicators should also be selected according to their reliability and the feasibility of
collecting the required data.
! Valid evaluation of impacts requires appropriate methodology. At a minimum, this usually
includes measuring outcomes before and after an intervention, and comparing changes in a
group which receives the intervention with changes during the same period in a comparison
group which does not.
! Long-term sustainability of the impacts of an intervention is always desirable. Changes
observed immediately after an intervention has taken place often disappear after a period of
time unless there is repetition and reinforcement of the intervention messages. For this
reason, it is useful to measure key outcome indicators again after a longer period has elapsed
(1-2 years) to be sure that positive changes are sustained.
8-10