Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Measurements Measure of Variability, Scale Levels of Measurements, Descriptive Statistics, Measures of Central Tendency Measurements need to: Produce valid and reliable results be sensitive and specific be able to identify clinically important changes have outcome measures and endpoints defined be easy to interpret Reasons for errors in measurement: Improper function or calibration of equipment patients providing misleading or dishonest answers to verbal/written questions Improper recording/transcribing of data Investigators recording or making inaccurate measurements Types of Errors Random error – Random in occurrence, often balancing out over course of study – mean or average of measurements still close to true value – Large patient size reduces random error Types of Errors Systematic error – represents bias in measurements and does not tend to balance out over course of study. – Bias can be knowingly or unknowingly – Good study design minimizes systematic error. Measurement Terms Validity- degree to which an instrument is measuring what it is intended to measure. – Predictive, Criterion, Face Reliability- reproducibility of a test Sensitivity- ability to measure a small treatment effect Specificity- how well the test can differentiate between the effect resulting from treatment and random variation Validity terms used in association with measurements: Predictive validity: – the extent to which a measurement or test actually reflects or predicts the true condition. Criterion (construct) validity: – the degree to which a measurement or test agrees with or obtains the same results as other proven tests designed to measure the same. Face validity – the extent to which a measure appears reasonable or sensible for measuring a desired outcome Reasons for False Positive Results Patient related – patients weren’t as ill as originally believed, and drug was more effective in mildly ill pts. – Patients were much more ill than originally believed, and drug was more effective in severely ill patients. – A few patients had a very large response, which skewed the overall results. – Patients gradually improved independent of drug treatment. False Positive Results Patient related – More medicine was absorbed than anticipated – Patients took excess medication. – Patients felt pressure to report a positive medicine effect – Concomitant non drug therapy or other drug therapy improved results False Positive Results Study Design and Drug Related – Blinding was broken or ineffective – open label study can sometimes produce a – – – – larger positive response no placebo control to help interpret error occurred in dosing patients- gave more drug than intended inadequate wash-out period, carry over effect inappropriate clinical endpoints, tests or parameters were used False Positive Results Investigator related – influenced response by great enthusiasm – chose inappropriate tests to measure Results and Data Related – systematic error- reporting large drug effect – high percentage of non-responders dropped out – not all data was analyzed False Negative Results Patient Related – were much more ill than realized – responded less to the drug than anticipated – study group had large number of non responders – non-compliance-- took fewer doses – concomitant medicines- interactions – exposed to conditions that interfered with study False Negative Results Drug Related – not adequately absorbed – kinetics were different in study group than in other patient groups Study Design Related – Too few of patients – inappropriate study design – insufficient drug dose was tested False Negative Results Study Design Related (cont.) – Ineffective tests or parameters used – Inadequate wash-out period in previous treatment period – Concomitant non-drug therapy interfered Investigator Related – influenced patients with skepticism displayed – chose inappropriate tests to measure effects False Negative Results Results and Data Related – Patients who improved dropped out leaving higher number of non-responders – systematic error resulted in reporting of an inappropriately small drug effect. Outcome Measures Example: A study is performed to compare the effects two antihypertensives, atenolol and propranolol in 2 groups of patients with mild high blood pressure. 2 types of outcomes measurements are selected for this study: measures of efficacy and measures of safety Measures of efficacy: BP, HR, symptom relief Measures of safety: adverse effects, blood glucose, electrolytes, serum lipids Criteria Used for Outcome Measures Presence or Absence criteria: Is sign, symptom present or absent? Graded or Scaled Criteria: the use of grading on a scale to measure clinical symptoms Relative change criteria- measured changes Global assessment criteria- Quality of Life Relative effect criteria- change in time to effect. Measurement Endpoints Endpoints are measurable points used to statistically interpret the validity of a study. Valid studies have appropriate endpoints. Endpoints should be specified prior to start of study (should be included in study design) Quality studies have simple, few and objective endpoints. Endpoints Objective- based on actual or measurable findings or events (heart rate, BP, Temp.) Subjective- based on thoughts, feelings, emotions (pain scale, mobility) Morbidity- quality or condition at the present-- quality of life Mortality- causing death or a death rate Endpoints Example In a study determining the effects of clonidine on quality of life, the researchers determine the number of days a patient misses work. Each patient is also asked to complete a rating scale to describe the degree of fatigue they experience. What type of endpoints are used? What type of criteria are used? Surrogate Endpoints These reduce the quality and validity of the study. Surrogate or Substitute endpoint examples: – CD4/CD8 ratios instead of “survival” in studies for treatment of AID’s. – Measuring volume of acne instead of proportion of patient’s cleared of acne. – Determining cardiovascular disease or atherosclerotic disease instead of measuring blood pressure in a study of antihypertensive drug treatment Hawthorne Effect Refers to the influence that a process of conducting a study may have on a subject’s behavior – Subject – Environment – Research design Reasons for Clinical Improvement in a Patient’s Condition Natural regression to the mean (most acute and some chronic conditions resolve on their own Specific effects of treatment (drug or intervention) Non-specific effects- attributable to factors other than specific drug/intervention effect. – Called a Placebo Effect Placebo Effect A placebo is an intervention designed to simulate medical therapy, but not believed to be a specific therapy for the target condition. A placebo is used either for it’s psychological effect or to eliminate observe bias. Placebo “response”= due to change in pt. Behavior following admin. of a placebo Placebo “effect” = change in pt’s illness due to the symbolic importance of a treatment. A placebo effect doesn’t require a placebo. Why do we see a Placebo Effect? Three different theories: 1. The effect is produced by a decrease in anxiety 2. Expectations lead to a cognitive readjustment of appropriate behavior. 3. The effect is a classical conditioned Pavlovian response. Placebo Effect Expectations lead to behavior change – Patient’s and providers expectations – Patient’s positive attitude toward provider and treatment – Providers positive attitude toward therapy – Provider interest in patient (sympathy, time, positive attitude) – Compliant patients have better outcomes than noncompliant patients even with a placebo. – The placebo response is stronger when stronger drugs are used. – Crossover studies show a stronger placebo response when given in the 2nd period of study. Appropriate Statistical Tests To determine whether appropriate statistical tests have been used, you must know 3 things: 1. The specific research question or hypothesis being addressed. The number of independent and dependent variables The scales or levels of measurement used for the dependent variables Variables in a Study: Dependent variables: – those variables whose value depends upon or is influenced by another variable. – It is the variable that is measured, and the one that changes as the result of a drug action. Independent variables – Those variables which modify a dependent variable (drug treatment) Example Patients given Lovastatin to lower cholesterol. Dependent variable- lowering of cholesterol Independent variable- Lovastatin There can be more than one independent and dependent variable in a study. Dependent/Independent Variables Example: A single blind study of 30 patients with poison ivy dermatitis were randomized to receive either topical hydrocortisone 1% or 2% and apply QID. Severity of the dermatitis was evaluated daily using a 5 point scale, where 5- severe and 0-none. What is the independent variable? Dependent variable? Example: A study was conducted to compare the efficacy of procainamide and quinidine for reducing ventricular arrhythmias. The number of ventricular ectopic depolarizations was determined in patients both before and during therapy with either drug. What is the independent variable? Dependent variable? Scales of Levels of Measurement Nominal Level – variables are grouped into mutually exclusive categories. Gender as female or male cured and not cured response and no response – include histograms (bar graphs) – weakest level of measurement – referred to as dichotomous data Scales of Levels of Measurement Ordinal level – ranked or ordered categories 1-2-3-4 severe, moderate, mild, none always, sometimes, never – stronger level than nominal – not measured quantitatively, but qualitatively – distance between groups need not be equal Scale Levels of Measurement Continuous Measurement Interval level: exact difference between two measurements is known and constant – – – – has arbitrary zero point highest level of measurement quantitative data Examples: BP (mm Hg) serum Theo levels (ug/ml), WBC (cells/cu mm) Continuous Level of Measurement Ratio level: – exact differences between measurements is known and constant – true zero point (Centigrade temp scale) – can make ratio statements (2:1) that denote relative size – Can be converted to an ordinal scale (but ordinal scale can’t convert to interval) Scale levels of Measurements Baseline Pain Assessment 0 1 2 3 (absent) (mild) (mod) (severe) Placebo 0 2 18 14 PainawayR 0 4 12 16 (number of subjects in each group with varying degrees of baseline pain intensity. What scale level of measurement? Scale level of Measurements Infectious Outcome Among 46 Patients Infection No infection Total Oxacillin 2 20 22 Placebo 0 24 24 Column total 2 44 46 What scale level of measurement? Types of Interval/Ratio Data Discrete scale of data (non-continuous): when a measurement has the interval characteristics but can only be assigned integer values. (HR, number of patients admitted to hospital/day) Non discrete (continuous) scale of data: each data point falls on a continuum with an infinite number of possible subdivisions (temp, BP, BG, weight) Data Distributions Once data is collected, it can be organized into a distribution, or graph of frequency of occurrence, or chart of the number of times that each measurement value occurs. – Bar Graphs – Bar Chart (Histogram) – Line Graphs Data Distributions Nominal and Ordinal level data use histograms (Bar charts) because data classified into distinct categories Continuous level data are distributed in the form of curves and line graphs (normal distributions and non-symmetrical distributions) Bar Chart (Histogram) 90 80 70 60 50 Flu cases 40 30 20 10 0 1st Qtr 2nd Qtr 3rd Qtr 4th Qtr Continuous Level Data Normal Distribution Gaussian Curve Non-Normal Distributions Bi-Modal Curve Weights of American Adults (women and men) Non-symmetrical distributions Non-normal distributions Continuous Distribution Examples: The distribution of GPA’s of college students: 1.0 2.5 4.0 Continuous Distribution Example Distribution of the ages of patients taking Digoxin 20 40 60 80 Descriptive Statistics Measures of Central Tendency Measures of Central Tendency Mean– mathematical average of a set of numbers. – Affected by extreme data points (outliers) – Useful for continuous level data (interval/ratio). – Ex: uric acid concentrations: 8,6,5,4,3,2,2,2. Total number of samples =8. Sum of measurements = 32. 32/8= 4 (mean). Measures of Central Tendency Median – “Middle” number of a group of numbers in which an equal number of responses above and below that point exist. (called 50th percentile) – Not affected by outliers. Useful for ordinal, interval and ratio data and non-symmetrical. – Ex: Uric acid concentrations: 8,6,5,4,3,2,2,2. Since even number, median lies between 4 and 3 or median= 3.5. How to Recognize “skewed” data If the magnitude of the difference between the mean and median is none or small, the data is approaching normal (symmetrical) distribution. If the difference between the mean and median is large, the data usually prove to be skewed. Measures of Central Tendency Mode – The most commonly or frequently occurring – – – – value(s) in a data distribution. Useful for nominal, ordinal, interval/ratio data. Only meaningful measure for nominal data. Can have more than one mode in set of data Ex: Uric acid concentration: 8,6,5,4,3,2,2,2. The mode = 2. Measures of Central Tendency Scale Level of Normal Non-Normal Measurement Distribution Distribution Nominal Mode Mode Ordinal Median=mode Median /mode Interval/Ratio Mean=med=mode Mean/med/mode Descriptive Statistics Measures of Variability Measure of Variability Two distributions can have the same mean, median and/or mode and yet be very different. Variability refers to how spread out (or close together) the data are. Example 2 groups of men w/ mean SBP in each group is 120 mmHg. Are they similar? First group: BP: 110,120,120,130 Second group: BP: 80,90,150,160 Both have mean= 120 Spread of data or range of data is much different Range Range: the interval between the lowest and highest values within a data group. Can be significantly influenced by outlying data (extreme values) Used for ordinal, interval or ratio data Interquartile Range Measure of variability directly related to the median. The median represents the 50th percentile. The interquartile range is that range described by the interval between the 25th and 75th percentile values. Used for ordinal, interval/ratio data that don’t have normal distributions Standard Deviation (SD) Standardized measure of the spread of scores around the mean. Useful for continuous (ratio/interval) data When reported in a study: +/- 1SD Needs normal distribution of data The mean +/- 1SD includes 68% of data points (34% on each side of the mean) Standard Deviation Mean +/- 2 SD include about 95% of data points (47.7% of the values on each side of the mean) Mean +/- 3 SD include about 97.7% of the data points (49.8% of values on each side of the mean) DBP 100 mmHg +/- 5 mmHg includes data points from 95-105 mmHg(assume 1 SD unless tells you differently) Standard Deviation (SD) The larger the SD, the further the data points deviate from the mean (more variable data). The smaller the SD, the closer the data points are to the mean (less variable data). Ex: 0.9 +/- 0.2mg% and 1.1 +/- 0.6mg% Which has more widely scattered data? Answer: 1.1 +/- 0.6 mg% (larger SD) Variance Variance is estimate of the study data. Obtained by calculating the differences between each individual value and the overall mean Needed for calculating the SD SD= variance SD2= variance Standard Error of the Mean (SEM) SEM is a way of estimating the variability of an individual sample mean relative to the population as a whole. SEM= SD/ variance or SD/ sample size SEM is used to calculate the Confidence Intervals (CI) Improperly used in place of SD because it is a smaller number and “looks better” SD versus SEM Mean Serum Theophylline concentrations of a group of patients was 13.6 +/- 2.1 (1SD). – Conclude about 68% of patients had conc. Somewhere in the range of 11.5-15.7. Serum Theo conc. of a group of patients was 13.6 +/2.1 (SEM) – could assume that if several add’l samples of pts with same characteristics were studied, their mean values would fall between 11.5 and 15.7 68% of the time. Review Variance= differences between each individual value and the overall mean. Variance used to calculate the SD SD= variance or SD2= variance SEM or SE is derived from the SD SEM= SD/ sample size Review…SEM SD= measure of the variability of individual values about the sample mean SEM/SE= measure or indication of the variability of individual sample means about the true but unknown population mean. SEM is used to estimate the reliability (precision) of a study sample in terms of how likely it is that the sample mean represents the true population mean. SEM is used to calculate the CI Example In a study of the effectiveness of Drug X on the Blood Sugar concentrations in 15 patients, the authors report the mean BS values in the patients as 150 +/- 2.3mg%. Would this represent the SEM or SD? Confidence Intervals (CI) Represents a range that has a high probability of containing the true population value. The likelihood that a study samples’ value reflects the true value of the population. Calculated for a desired level of probability (95%). A 95% CI means there is a 95% probability that the true population value falls within the CI range Confidence Interval Example The mean difference in healing rates between placebo and penicillin was reported to be 59% (CI = 24-72%). Confidence Intervals Can be calculated for nominal level data (proportions) and continuous level data 90% CI assoc. with narrower range of values (don’t need to be as confident) 99% CI assoc. with wider range of values (more confident the CI will contain true population value. CI is influenced by: 1. Level of confidence selected 2. SEM (larger SEM, wider the CI) 3. Standard Deviation (SD) of the study sample. (larger SD, then larger SEM, then wider the CI) 4. Size of the study group. (The larger the sample size, the smaller the SEM, and narrower the CI) Confidence Interval Example Two similar studies are published about efficacy of Pravastatin for reducing cholesterol. Both sets of patients are comparable. Study 1 enrolled 200 patients. Study 2 enrolled 50 patients. The mean +/- SD treatment reduction in cholesterol concentrations in the Study 1 patients was 15.2 mg% +/2.0 mg%. The corresponding values in the Study 2 patients was 17.1 mg% +/- 2.0 mg% Which mean values (15.2 mg% or 17.1 mg%) would most likely have a wider 95% CI associated with it? Confidence Intervals (CI) CI applies to continuous data, proportions (nominal data), medians, regression slopes, relative risk data, response rates, survival rates, median survival duration, hazard ratios, non-random selection or assignment between groups. Measures of Variability Level of Measurement SD SEM CI Nominal No No Yes Ordinal No No No Continuous Yes Yes Yes Statistical vs. Clinical Significance Example: A new antihypertensive drug is studied to determine whether it decreases the rate of myocardial infarction. The results indicate that the drug decreases MI by 11% with a 95% CI= -2-25%. Statistical vs. Clinical Significance The 95% CI for the relative risk of headache development with a new diabetes drug is reported as 1.20 (CI 0.95-1.50) and for a placebo drug as 1.25 (CI 0.88-1.76). Are these showing statistical significance? Ratios, Proportions and Rates Ratio expresses the relationship between two numbers. Men: Women (45:90) Proportion: specific type of ratio expressed as a percentage. 12% experienced cough when using this drug) (12% of total study population) Rate: form of proportion that includes a specific time frame. (18% died from influenza in the US last year) Incidence and Prevalence Incidence Rate = Number of new cases of a disease Total population at risk Prevalence Rate= Number of existing cases of a disease Total population at risk per time per time Descriptive Statistics Measures of Risk/Association Relative Risk Odds Ratio Relative Risk Reduction Absolute Risk Reduction Number Needed to Treat Number Needed to Harm Measures of Risk Relative Risk (RR) – the risk or incidence of an adverse event – – – – occurring or of a disease developing during treatment in a particular group. RR= # pts in treatment group w/ ADR Total # of pts in treatment group # pts in placebo group w/ ADR Total # pts in placebo group Relative Risk Example: A new drug is being compared to placebo to prevent development of diabetic retinopathy (DR). Treatment DR No DR Total New drug 50 75 125 Placebo 65 55 120 What is the risk of DR developing during treatment in patients taking the new drug? 50__ = 0.4= 40% Risk in placebo? 65 = 0.54=54% 125 120 RR= 0.4/0.54 = 0.74 or 74% Relative Risk (RR) RR = 1 : When the risk in each group is the same RR<1: When the risk in treatment group is smaller than the risk in the placebo group RR>1: When the risk in the treatment group is greater than the risk in the placebo group Relative Risk (RR) Example: The risk of an adverse event developing during therapy with an eye medication compared to the placebo group was listed as 1.5. What does this mean? Answer: That the eye med is 1.5 times more likely to cause an adverse event than the placebo being used. Relative Risk Example 92 men and women who were recovering from heart attacks were followed and surveyed a year later. 14 of the 92 patients had died. When death rates were calculated according to pet ownership, only 3 of the 53 pet owners (5.6%) were no longer living, compared to 11 of 39 (28%) patients who were without animals. Relative risk = 0.056/0.28 = 0.2 What does this mean? Relative Risk... Relative Risk does NOT tell us the magnitude of the absolute risk. Example: A RR of 33% could mean that the treatment reduces the risk of an adverse event from 3% down to 1% or from 60% down to 20%. These may or may not be significant depending on the population and adverse event (minor or major adversity) Odds Ratio (OR) Commonly reported measure in case control designs. Case control starts with outcomes. (looks back for risk factors) OR = # pts taking drug w/ ADR # pts taking drug w/o ADR__ #pts not taking drug w ADR # pts not taking drug w/o ADR Odds Ratio cont. The Odds Ratio could also be expressed as: Treatment A Deaths Treatment A Survival_____ Treatment B Deaths Treatment B Survival Odd’s Ratio (OR) Odds of developing a disease or ADR if exposed (to drug) Odds of developing a disease or ADR if not exposed (to drug) OR: Disease Present Absent Exposed factor A B Not exposed to factor C D OR= A/C = A X D B/D B XC OR = A/B = A X D C/D BXC Odds Ratio Example: A case control study reported that 35 of 120 chronic renal failure patients took NSAID’s compared to only 20 of 110 similar patients without renal failure. What would be the odds ratio of developing renal failure if taking NSAID’s? 35 (taking NSAID’s w/ RF) A=35 B = 20 20 ( taking NSAID’s w/o RF) C = 85 D= 90 85 (not taking NSAID’s w/ RF) 90 (not taking NSAID’s w/o RF) Renal Failure/NSAID’s A/C A/B B/D C/D 35/ 85 = 0.41 or 35/20 =1.75 20/ 90 = 0.22 85/90 = 0.94 0.41 = 1.86 1.75 = 1.86 0.22 0.94 Odds Ratio OR= l : The odds of developing an adverse event or disease in the exposed (treatment) group is the same as the odds in the nonexposed (non-treatment) group. OR<1: Odds of developing ADR in exposed group is less than odds in non-exposed. OR>1: Odds of ADR in exposed group greater than the odds in non-exposed. Odds Ratio (OR) Example: The odds that ASA was taken by children who developed Reyes Syndrome vs. the odds that ASA was taken by similar children who did not develop Reyes Syndrome was reported as OR=3:l. – The odds that Reyes Syndrome children had taken ASA was approximately 3 times greater than for the children who did not develop Reyes Syndrome. Interpreting the OR and RR 1. Degree of validity of the study design. 2. The confidence interval (CI) 3. Relative Risk Reduction (RRR) Relative Risk Reduction (RRR) Ex: If a new drug is shown to reduce the risk of cancer, what is the exact percentage of this reduction? RRR: measure of the reduction in the relative risk in the exposed group. RRR= Rate in control group-rate in tx group Rate in control group RRR= 1-RR Relative Risk Reduction (RRR) Incidence of cancer was 7% in treatment group and 12% in placebo( control) group. RRR= 12%-7% = 0.42 = 42% 12% Disadvantage of RRR- doesn’t discriminate between very large and very small actual incidence rates in the groups. Relative Risk Reduction Example A study is performed to determine the efficacy of a new LMWH, Drug “H” in preventing PE from post surgical patients. 299 post surgical patients are randomized to receive Drug H and 355 receive placebo. 43 patients developed PE in the placebo group, and 21 developed PE in the treatment group. What is the relative risk reduction by Drug H (reducing the risk of PE) Drug H Example cont... Incidence of PE in placebo group = 43/355 = 0.12 = 12% Incidence of PE in Drug H group = 21/299 = 0.07= 7% RRR =Rate in control group - rate in treatment group Rate in control group RRR= 12%-7% / 12% = 0.12- 0.07/ 0.12= 0.42 = 42% OR another way to calculate is RRR= 1-RR 1- 21/299 / 43/355 = 1- 7/12 = 12/12-7/12 = 5/12 = 0.42 = 42% or 1- 0.07/0.12 = 1-0.58= 0.42 Absolute Risk Reduction (ARR) ARR= Incidence rate in control group incidence rate in treatment group. Ex: cancer: treatment 7%, placebo 12% ARR = 12%-7% = 5% For serious conditions though, a small ARR can still be very clinically relevant. Number Needed to Treat (NNT) NNT: number of individuals that need to be treated in order to prevent one adverse event or one outcome. NNT = 1 ARR Ex: study determine efficacy of drug preventing cancer. Incidence of cancer in placebo 12%, in treatment group 7% 12%-7% = 5% 1/5% = 20=NNT (20 pts needed to treat to prevent 1 case of cancer NNT= 1/ placebo - treatment group Number Needed to Harm (NNH) NNH= 1/ treatment- placebo group Ex: Headache occurred in 25% of placebo patients and 75% of patients taking drug X. The NNH = 75%-25% = 50% 1/0.5 = 2 Only 2 patients would need to be treated with drug X in order to cause a headache occurrence. Review In a diabetes study, 4% of Glucotrol users and 18% of placebo pts. Developed CHF within 10 years. RRR= 18%-4% = 14 = 0.77 = 77% RR 18% 18 ARR = 18%-4% = 14% NNT = 1/0.14 = 7 pts In Glucotrol group 26% had HA vs. 3% in placebo. NNH = 26%-3% = 23% 1/0.23 =4