Download Module 12 – March 2010

Module 12 – March 2010
Monitoring and
Evaluation
Project Partners
Funded by the Health Resources and Services Administration (HRSA)
Module Overview
 Monitoring &
Evaluation (M&E)
framework and
components
 Records, registers and
reports
 Recording and
evaluating response to
TB treatment regimens
 Supervision
International Standards 13 and 21
Learning Objectives
At the end of this presentation, participants
will be able to:
 Describe what is meant by “Monitoring and
Evaluation”
 Discuss the importance of collecting data and
ensuring the accuracy of the data
 Explain ways in which the data are used to
evaluate treatment
 Describe how M&E activities can benefit both
TB and HIV/AIDS programs
Monitoring & Evaluation System
 A key element of the Stop TB Strategy
 Allows programs to:
• Monitor progress and treatment outcomes of
individual patients
• Evaluate the overall performance of the TB
program at various levels (local, district,
national)
• Identify areas of program improvement and
weakness
• Ensure accountability
Monitoring
 What is it?
• Routine tracking of
services and program
performance
Monitoring (2)
 How is it done?
• Through information collection, data input,
analyzing the data, and reporting what is
found in that analysis outcome report
 Why should we do it?
• To better assess how well a policy or program
is achieving its intended target
Evaluation
 What is it?
• Episodic assessment of results that can be
attributed to program activities
• Types of evaluation related to M & E:
➜ Process evaluation: assesses the progress in
program implementation and coverage
➜ Outcome and impact evaluation: measures the
effect of the program activity on the target
population
What are the Targets?
 Stop TB Partnership/WHO
• 70% TB case detection and 85%
cure rate by 2005
 Millennium Development Goals (MDG):
• Halt, and begin to reverse, the incidence of major
diseases such as HIV/AIDS and TB by 2015
 Decrease TB prevalence and death rates to 50% of
the 2000 estimates
 United Nations General Assembly Special
Session (UNGASS) – global targets
Where do Indicators Fit In?
 Indicator: a specific, observable, and
measurable characteristic or change that
shows the progress a program is making
toward achieving a specific outcome
 Indicators may be expressed in terms of:
• Number
• Rate
• Proportion
• Percentage
Limitations of Indicators
Indicators DO NOT:
 Measure everything
 Tell us why a problem may exist or how
to fix it
 Determine if problems identified are
amenable to intervention
 Tell us which interventions are most cost
effective
What are some possible uses
of data collected by the
National HIV/AIDS and
TB Programs?
Using and Disseminating Data
M & E can improve and enhance NAP and NTP
work by:
 Identifying areas of strengths and weaknesses
 Helping plot progress toward program goals
 Allowing a program to see trends and to identify high
risk groups in order to better target TB control efforts
 Providing justification for needed resources
 Identifying training and supervision needs
 Increasing public awareness about TB
 Advocating for policy changes and allocation of funds
Monitoring & Evaluation Framework
CONTEXT
Environmental, cultural, political, and socio-economic factors external to the programme
INPUT
Basic
resources
necessary
•
•
•
•
Policies
People
Money
Equipment
PROCESS
Programme
activities
•
•
•
•
Training
Logistics
Management
IEC/BCC
OUTPUT
OUTCOM E
IM PACT
Results at the
programme
level
Results at
level of target
population
Ultimate effect
of project in
long term
• Behaviour
• Safer practices
•
•
•
•
(measure of
programme activities)
• Services
• Service use
• Knowledge
Monitoring / Process Evaluation
TB incidence
HIV prevalence
Morbidity
Mortality
Outcome / Impact Evaluation
Types of M & E Activities
 Staff training
 Supervision
 Completion of reporting forms
 Discussions during staff meetings
 Ensuring medicine and laboratory stock
supplies
 Quality control activities
 Analysing data and preparing reports
Standard 13: Recording & Reporting
A written record of
all medications
given, bacteriologic
response, and
adverse reactions
should be
maintained for all
patients
Standard 21: Recording & Reporting
All providers must
report both new and retreatment tuberculosis
cases and their
treatment outcomes to
local public health
authorities, in
conformance with
applicable legal
requirements and
policies
Form 3: BASIC MANAGEMENT UNIT TB REGISTER – LEFT SIDE OF THE REGISTER BOOK
1
Name
Age
District
TB No.
Sex
M/F
4
Date of
registration
Address
1
Health facility
Site
Type of patient
Date
Treatment
Treatment
2 treatment
3
P
/
supporter
category
N R F D T O
started
EP
Facility where patient’s treatment card is kept. In case several copies are kept, the most peripheral facility should be entered.
including community worker/volunteer, family members or friends.
3
Enter the treatment category:
F=Treatment after failure – A patient who is started on a re-treatment
CAT I: New case
regimen after having failed previous treatment.
D=Treatment after default – A patient who returns to treatment,
CAT II: Re-treatment
e.g. 2(HRZE)S/1(HRZE)/5(RHE)
Chronic:patient sputum positive at the end of a re-treatment regimen.
positive bacteriologically, following interruption of treatment for 2 or
Chronic cases still alive and not started on Category IV treatment
more consecutive months.
T=Transfer in – A patient who has been transferred from another TB
should be re-entered at the beginning of each year. Patients who are
started on Category IV treatment should be entered in a separate
register to continue treatment. This group is excluded from the
Category IV register and separate Category IV treatment cards should
quarterly report on registration.
O=Other previously treated– All cases that do not fit the above
be used for them.
4
Tick only one column :
definitions. This group includes smear-positive cases with unknown
N=New – A patient who has never had treatment for TB or who has
outcome of previous treatment, smear negative previously treated, EP
taken antituberculosis drugs for less than 1 month.
previously treated and chronic case (i.e. a patient who is sputum
R=Relapse – A patient previously treated for TB, declared cured or
positive at the end of a re-treatment regimen)
treatment completed, and who is diagnosed with bacteriological (+) TB
(smear or culture).
2
Form 3: BASIC MANAGEMENT UNIT TB REGISTER – RIGHT SIDE OF THE REGISTER BOOK
Treatment outcome
& date
Results of sputum smear microscopy and other examination
Before treatment
Smear
result
1
Date/
Lab. No.
2 or 3 months
X-ray
Date/
4
Result
Smear
result
1
Date/
Lab. No.
5 months
Smear
result
Date/
Lab. No.
End of treatment
Smear
result
Date/
Lab. No.
TB/HIV activities
3
Date
Outcome
2
in text
HIV result /
Date/
No. HIV reg
ART
Y/N
Start date/
No. ART reg
CPT
Y/N
Start
date
Remarks
CAT 1 patients have follow-up sputum examination at 2 months; CAT II patients have follow-up sputum examination at 3 months. CAT 1 patients with
extended phase 1 to 3 months have follow-up sputum examination at 2 AND 3 months with results registered in the same box.
2
Enter the code (1-6) as follows:
1-Cure: Sputum smear positive patient who was sputum negative in the last month of treatment and on at least one previous occasion.
2-Treatment completed: Patient who has completed treatment but who does not meet the criteria to be classified as a cure or a failure.
3-Treatment failure: New patient who is sputum smear (+) at 5 months or later during treatment, or who is switched to Category IV treatment because
sputum turned out to be MDRTB. Previously-treated patient who is sputum smear positive at the end of his retreatment or who is switched to Category
IV treatment because sputum turned out to be MDRTB.
4-Died: Patient who dies from any cause during the course of treatment.
5-Default: Patient whose treatment was interrupted for 2 consecutive months or more.
6-Transfer out: Patient who has been transferred to another recording and reporting unit and for whom treatment outcome is not known.
3
+ positive, - negative, U unknown, ND Not Done. Documented evidence of HIV test performed during or before TB treatment is reported here.
4
+ : suggestive of TB, -: not suggestive of TB, ND: not done.
Why is accurate reporting and
record keeping important?
Data Quality Assurance
 Ensures that the information collected
adequately represents the program’s
activities
 Accurate data – measuring what it is
intended to measure
 Reliable data – collected and measured
the same way by all program personnel
over time
Reporting Forms and Registers
 Request for Sputum
Examination
 Quarterly Report on TB
Case Registration
 Tuberculosis
Treatment Card
 Quarterly Report on
Sputum Conversion
 Tuberculosis
Identity Card
 Register of TB Suspects
 Basic Management
Unit TB Register
 TB Laboratory
Register
 Quarterly Report on
Treatment Outcomes
 Yearly Report on
Program Management in
Basic Management Unit
Tuberculosis Treatment Card
Tuberculosis Treatment Card
Name:
Sex:
Age:
________________________________________________________
M
F
BMU TB Register No._____________
Disease site (check one)
Pulmonary
Date of registration: ____________________________
________
Health facility: _________________________________
Type of patient (check one)
New
Relapse
Transfer in
Address: ________________________________________________________
________________________________________________________________
Name / address of community treatment supporter (if applicable)
________________________________________________________________
I. INITIAL PHASE - prescribed regimen and dosages
Month
0
Self-referral
Community member
Public facility
Private facility/provider
Other, specify
Number of tablets per dose and dosage of S:
(RHZE)
S
Treatment after default
Treatment after failure
Other, specify ___________________
Date
ARV
Result
Lab No.
-------------------------------Cotrimoxazole
Weight
(kg)
Sputum smear microscopy
Referral by :
CAT (I, II , III):
Extrapulmonary, specify ___________
TB/HIV
Date
Result*
HIV test
CPT start
ART start
Other
* (Pos) Positive; (Neg) Negative; (I) Indeterminate; (ND) Not Done/unknown
Tick appropriate box after the drugs have been administered
Daily supply: enter
Day
Month
1
. Periodic supply: enter X on day when drugs are collected and draw a horizontal line (
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Please turn over for continuation phase
18
) through the number of days supplied. Ø = drugs not taken
19
20
21
22
23
24
25
26
27
28
29
30
31
Tuberculosis Treatment Card (2)
(RHE)
(RH)
II. CONTINUATION PHASE
(Other)
Number of tablets per dose
Daily supply: enter  . Periodic supply, enter X on day when drugs are collected and draw a horizontal line (
Day
Month
1
2
3
4
5
6
7
8
9
10
11
X-ray (at start)
HIV care
Date:
Results (-), (+), ND
Pre ART Register No.
CD4 result
ART eligibility (Y/N/Unknown)
Date eligibility assessed
ART Register No.
12
13
14
15
16
17
18
) through the number of days supplied. Ø = drugs not taken
19
20
21
22
23
24
25
26
27
28
29
30
31
Comments: _________________________________________
___________________________________________________
___________________________________________________
___________________________________________________
___________________________________________________
______________________________________________________________________________________
Treatment outcome
Date of decision ____
Cure
Treatment completed
Died
Treatment failure
Default
Transfer out
______________________________________________________________________________________
______________________________________________________________________________________
______________________________________________________________________________________
______________________________________________________________________________________
______________________________________________________________________________________
Name and address of contact person: ______________________________________________________________________________
Preparing a TB Treatment Card
Activity
Tuberculosis Identity Card
 This card contains an extract of information
on the treatment card
 It is given to the patient at the start of
treatment
 It is used to record daily DOT and must be
used during the intensive phase of treatment
 It also serves as a reference document for
TB status after treatment
 It should be presented to the doctor
whenever the patient falls ill in the future
Tuberculosis Identity Card (2)
Tuberculosis Identity Card
Name __________________ BMU TB Register No. _____
Appointment dates: ________________________________
Address __________________ Date of registration: _______
__________________________________________________
Sex:
__________________________________________________
M
F
Age ______ Date treatment start _______
Health facility: ______________________________________
Supporter (name and address) __________________________
__________________________________________________
__________________________________________________
__________________________________________________
Sputum smear microscopy
Month
Date
Lab No.
Result
Weight
(kg)
Disease site (check one)
Pulmonary
Extrapulmonary, specify _______
Type of patient (check one)
New
Relapse
Transfer in
Treatment after default
Treatment after failure
Other specify ______________
I. INITIAL PHASE
CAT (I, II , III):
Drugs and dosage:
(RHZE)
S
Other
II. CONTINUATION PHASE
(RH)
Drugs and dosage:
(RHE)
Other
REMEMBER
Basic Management Unit TB Register
 This revised register is the cornerstone of an
NTPs monitoring & evaluation system
 It records essential information for
notification & treatment outcome by district
 It should always be kept up to date with data
on sputum smear examinations and
treatment outcome
 Where electronic data collection systems are
available, the information from the register
should be entered into the database at least
once every month
Basic Management Unit TB Register – Left side of the register book
BMU
TB No.
Name
Age
Date of
registration
Sex
M/F
Type of patient
Address
Health
1
facility
Date
Treatment
treatment
2
category
started
Site
P/EP
N R F
3
D T O
Footnotes appearing on first page of the register only.
1 Facility where patient’s treatment card is kept. In case several copies are kept, the most peripheral facility should be entered. Use standardized type of health
facilities according to block 2 of the Yearly Report on Programme Management in BMU. Health facility is defined as any health institution with health care
providers formally engaged in any of the following TB control functions (DOTS): referring TB suspects/cases, laboratory diagnosis, TB treatment and patient
support during treatment.
2 Enter the treatment category:
D=Treatment after default – A patient who returns to treatment,
CAT I: New case of sputum smear microscopy positive, severe
sputum smear microscopy negative PTB & EPTB e.g.
positive bacteriologically, following interruption of treatment for 2 or
2(RHZE)/4(RH)
more consecutive months.
T=Transfer in – A patient who has been transferred from another TB
CAT II: Re-treatment e.g. 2(RHZE)S/1(RHZE)/5(RHE)
CAT III: New sputum smear microscopy negative PTB and EPTB
Register to continue treatment. This group is excluded from the
Quarterly Reports on TB Case Registration and on Treatment
e.g. 2(RHZE)/4(RH)
Outcome.
3 Tick only one column:
N=New – A patient who has never had treatment for TB or who
O=Other previously treated– All cases that do not fit the above
has taken antituberculosis drugs for less than 1 month.
definitions. This group includes sputum smear microscopy
R=Relapse – A patient previously treated for TB, declared cured
positive cases with unknown history or unknown outcome of
or treatment completed, and who is diagnosed with
previous treatment, previously treated sputum smear microscopy
bacteriological (+) TB (sputum smear microscopy or culture).
negative, previously treated EP, and chronic case (i.e. a patient
F=Treatment after failure – A patient who is started on a rewho is sputum smear microscopy positive at the end of retreatment regimen after having failed previous treatment.
treatment regimen)
TB Register in Basic Management Unit using Routine Culture and DST – Right side of the register book
Results of sputum smear microscopy and other examinations
Before treatment
Sputum
smear
X-ray
micros- HIV result3/
4
Result /
copy
Date
date
date/No./
2
Result
DST
Culture
date/No./
date/No./
6
5
Result
Result
1
2 or 3 months
Sputum
smear
Culture
microsNo./
copy
5
Result
No./
2
Result
5 months
Sputum
smear
Culture
microsNo./
copy
5
Result
No./
2
Result
Treatment outcome & date
End of treatment
Sputum
smear
Culture
microsNo./
copy
5
Result
No./
2
Result
Date
Outcome in text
7
TB/HIV activities
ART
Y/N
Start
date
CPT
Y/N
Start
date
Remarks
Footnotes appearing on first page of the register only
1 CAT I patients have follow-up sputum smear microscopy examination at 2 months; CAT II patients have follow-up sputum smear microscopy examination at
3 months. CAT I patients with initial phase of treatment extended to 3 months have follow-up sputum smear microscopy examinations at 2 AND 3 months
with results registered in the same box.
2 (ND): Not done; (NEG): 0 AFB/100 fields; (1-9): Exact number if 1 to 9 AFB/100 fields; (+): 10-99 AFB/100 fields; (++): 1-10 AFB/ field; (+++): > 10 AFB/ field
3 (Pos):Positive; (Neg):Negative; (I):Indeterminate; (ND):Not Done / unknown. Documented evidence of HIV test performed during or before TB treatment is
reported here. Measures to improve confidentiality should accompany recording of HIV status.
4 (Pos): Suggestive of TB; (Neg): Not suggestive of TB; (ND): Not Done.
5 (Pos): Positive; (Neg): Negative; (ND): Not Done.
6 (ResistR): Resistant to Rifampicin; (ResistH): Resistant to Isoniazid; (ResistE): Resistant to Ethambutol; (ResistStrept): Resistant to Streptomycin;
(ResistRH): Resistant to Rifampicin and Isoniazid;
(Suscept): Susceptible; (ND): Not Done.
7 Write clearly ONE of the following outcomes per patient:
Cure: Patient with culture or sputum smear microscopy positive at the beginning of the treatment who was culture or sputum smear microscopy negative in
the last month of treatment and on at least one previous occasion.
Treatment completed: Patient who has completed treatment but who does not meet the criteria to be classified as a cure or a failure.
Treatment failure: New patient who is culture or sputum smear microscopy positive at 5 months or later during treatment, or who is switched to Category IV
treatment because sputum smear microscopy turned out to be MDRTB. Previously-treated patient who is culture or sputum smear microscopy positive at the
end of his re-treatment or who is switched to Category IV treatment because sputum turned out to be MDRTB.
Died: Patient who dies from any cause during the course of treatment.
Default: Patient whose treatment was interrupted for 2 consecutive months or more.
Transfer out: Patient who has been transferred to a health facility in another BMU and for whom treatment outcome is not known.
Treatment Outcomes
 Cured
 Treatment
completed
 Treatment failure
 Died
 Default
 Transfer out
Treatment Outcomes
Cure
Patient whose sputum smear or culture was
positive at beginning of treatment but who was
smear- or culture-negative in the last month of
treatment and on at least one previous occasion
Treatment
Completed
Patient who has completed treatment but who
does not meet the criteria to be classified as a
cure or a failure
Treatment
Failure
Patient who is sputum smear-positive at five
months or later during treatment
– or –
Patient found to harbor a MDR strain at any
point of time during treatment, whether smearnegative or -positive
Treatment Outcomes (2)
Died
Patient who dies for any reason during the
course of treatment
Default
Patient whose treatment was interrupted for 1
month or more
Transfer
Out
Patient who has been transferred to another
recording and reporting unit and whose
treatment outcome is not known.
Treatment
Success
A sum of cured and completed treatment
(smear-positive or culture-positive patients only)
Supervision
How is supervision used in
your TB and HIV/AIDS
Prevention and Control
Programs?
Role of Supervision in M & E
 Supervision is a process of guiding,
helping, training, and enabling staff to
improve their performance in order to
provide high quality health care services
Purpose of Supervision
 Provide leadership and direction to staff
 Ensure effective program implementation
 Monitor operations and evaluate
achievement of goals
 Ensure adherence to laws and policies
 Avoid confusion or duplication of efforts
Purpose of Supervision (2)
 Monitor that all necessary tasks are
properly performed
 Ensure that resources are properly used
and are available to staff, including
training and supplies to carry out their
duties
 Ensure accountability
Barriers to Effective Supervision
 Lack of commitment
 Lack of proper planning and time
management
 Lack of tools for Monitoring & Evaluation
 Insufficient staff
 Problems with transportation
 Lack of confidence or preparation
Roles and Responsibilities
 National Level – NTP and NAP
• Planning, implementing, monitoring, and
evaluating Program at all levels
 Regional/District/Parish Levels
• Coordinating, supervising, planning,
implementing, monitoring and evaluating all
aspects of TB and HIV/AIDS Programs in the
region
How Can M&E Information be used in
TB and HIV Programs?
 Identify gaps in performance
 Monitor treatment outcomes
 Measure the impact of an intervention or
policy change
 Identify populations for enhanced control and
prevention efforts
 Identify local problems as they arise
 Ensure high quality TB and HIV prevention
and control strategies are consistently
provided
Summary: ISTC Standards Covered
Standard 13: A written record of all
medications given, bacteriologic
response, and adverse reactions should
be maintained for all patients
Standard 21: All providers must report
both new and re-treatment
tuberculosis cases and their treatment
outcomes to local public health
authorities, in conformance with
applicable legal requirements and policies
Summary
 Several approaches are
used to monitor and
evaluate TB and HIV/AIDS
programs including
supervision, training and
the keeping of records and registers
 Accurate and timely reporting and record
keeping is important. It allows true
assessment of Program achievements