Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Corso di clinical writing What to expect today? Core modules • Introduction • General principles • Specific techniques • Title/Abstract drafting • Finding out relevant literature, and Introduction drafting • Nuts & bolts of statistics and Methods drafting • Practical session 1 – Appraisal of a published article Ultimate goal of research: appraisal of causation Methods of inquiry Statistical inquiry may be… Descriptive (to summarize or describe an observation) or Inferential (to use the observations to make estimates or predictions) Descriptive statistics 100 100 AVERAGE Inferential statistics If I become a scaffolder, how likely I am to eat well every day? P values Confidence Intervals Samples and populations This is a sample Samples and populations And this is its universal population Samples and populations This is another sample Samples and populations And this might be its universal population Samples and populations But what if THIS is its universal population? Samples and populations Any inference thus depend on our confidence in its likelihood Accuracy and precision true value measurement spread Accuracy measures the distance from the true value Precision measures the spead in the measurements Accuracy and precision Random and systematic errors Thus Precision expresses the extent of RANDOM ERROR Accuracy expresses the extent of SYSTEMATIC ERROR (ie bias) Bias Bias is a systematic DEVIATION from the TRUTH -in itself it cannot be ever recognized -there is a need for external gold standard and/or permanent surveillance An incomplete list of bias · Selection bias · Information bias · Confounders · Observation bias · Investigator’s bias (enthusiasm bias) · Patient’s background bias · Distribution of pathological changes bias · Selection bias · Small sample size bias · Reporting bias · Referral bias · Variation bias · Recall bias · Statistical bias · Selection bias · Confounding · Intervention bias · Measurement or information · Interpretation bias · Publication bias · Subject selection/sampling bias Simplest classification: 1. Selection bias 2. Information bias Sackett, J Chronic Dis 1979 Validity Internal validity entails both PRECISION and ACCURACY (ie does a study provide a truthful answer to the research question?) External validity expresses the extent to which the results can be applied to other contexts and settings. It corresponds to the distinction between SAMPLE and POPULATION) Validity Rothwell, Lancet 2005 Navigating through variables Statistical variables Variables CATEGORY nominal QUANTITY ordinal discrete continuous counting measuring Device diameter BMI Blood pressure Death: yes/no Race ordered categories SES ranks Categorical variables Exp Ctrl Event a b No event c d Absolute risk reduction (ARR) = [ a / ( a + c ) ] - [ b / ( b + d ) ] Relative risk (RR) = [ a / ( a + c ) ] / [ b / ( b + d ) ] Relative risk reduction (RRR) = 1 - RR Odds ratio (OR) = (a/c)/(b/d) = ( a * d ) / ( b * c ) Categorical variables Absolute risk reduction (ARR) 25% (25/100) - 40% (40/100) = -15% Relative risk (RR) Laparoscopic surg Open surg Bleeding 25 40 No bleeding 75 60 100 100 25% (25/100) / 40% (40/100) = 0.62 (given an equivalence value of 1) Relative risk reduction (RRR) 1 – 0.62 = 38% Total Odds ratio (OR) 33% (25/75) / 66% (40/60) = 0.5 (given an equivalence value of 1) Mean (arithmetic) Characteristics: -summarises information well -discards a lot of information Assumptions: -data are not skewed x x N – distorts the mean – outliers make the mean very different -Measured on measurement scale – cannot find mean of a categorical measure ‘average’ device diameter may be meaningless Median What is it? – The one in the middle – Place values in order – Median is central Definition: – Equally distant from all other values Used for: – Ordinal data – Skewed data / outliers – E.g. ………………… Comparing measures of central tendency Mean is usually best – If it works – Useful properties (with standard deviation [SD]) – But… Group 1 Lesion diameter (mm) Mean Median 17 19 19 17 18 18 18 Group 2 21 21 21 21 4 17.6 21 Comparing measures of central tendency It also depends on the underlying distribution… mean=median=mode Frequency Symmetric? Value Comparing measures of central tendency It also depends on the underlying distribution… Mean>Median>Mode Asymmetric? 30 Mode Median Mean Frequency 25 20 15 10 5 0 0 1 2 3 4 5 6 7 Number of clips implanted per patient 8 9 Measures of dispersion: examples Range 99% Confidence Interval (CI) 50 – Top to bottom – Not very useful 40 Interquartile range 75% CI 30 – Used with median 20 – ¼ way to ¾ way Standard deviation (SD) 10 SD – Used with mean 0 40 30 20 10 0 – Very useful Standard deviation Standard deviation (SD): – approximates population σ SD as N increases Advantages: – with mean enables powerful synthesis mean±1*SD 68% of data mean±2*SD 95% of data (1.96) mean±3*SD 99% of data (2.86) Disadvantages: – is based on normal assumptions 2 ( x x ) N-1 Statistical variables Variables CATEGORY nominal QUANTITY ordinal discrete continuous counting measuring Device diameter BMI Blood pressure Death: yes/no Race ordered categories SES ranks Comparisons Variables PAIRED OR REPEATED MEASURES eg Repeated-measures ANOVA Paired Student t test UNPAIRED OR INDEPENDENT MEASURES eg Unpaired Student t test Chi square test Statistical tests Are data categorical or continuous? Categorical data: compare proportions in groups Two or more groups, compare by chisquare test Non-normal data; use Mann Whitney test Continuous data: compare means or medians in groups How many groups? Two groups; normal data, same spread? Normal data; use t test More than two groups; normal data? Normal data; use ANOVA Non-normal data; use Kruskal Wallis Testing normality assumptions Rules of thumb 1. Referring to previous data or analyses (eg landmark articles, large databases) 2. Inspection tables and graphs (eg outliers, histograms) 3. Checking rough equality of mean, median, mode 4. Performing ad hoc statistical tests • Shapiro-Wilks test • Kolmogodorov-Smirnov test • … Alpha and type I error Whenever I perform a test, there is thus a risk of a FALSE POSITIVE result, ie REJECTING A TRUE null hypothesis This error is called type I, is measured as alpha and its unit is the p value The lower the p value, the lower the risk of falling into a type I error (ie the HIGHER the SPECIFICITY of the test) Alpha and type I error Type I error is like a MIRAGE Because I see something that does NOT exist Beta and type II error Whenever I perform a test, there is also a risk of a FALSE NEGATIVE result, ie NOT REJECTING A FALSE null hypothesis This error is called type II, is measured as beta and its unit is a probability The complementary of beta is called power The lower the beta, the lower the risk of missing a true difference (ie the HIGHER the SENSITIVITY of the test) Beta and type II error Type II error is like being BLIND Because I do NOT see something that exists Summary of errors Experimental study H0 accepted H0 rejected H0 true Type I error H0 false Type II error Truth Inferential statistics P values tell you whether there is a DIFFERENCE and its DIRECTION Confidence intervals tell you what is the MAGNITUDE (or SIZE) of such difference Power and sample size Whenever designing a study or analyzing a dataset, it is important to estimate the sample size or the power of the comparison SAMPLE SIZE Setting a specific alpha and a specific beta, you calculate the necessary sample size given the average inter-group difference and its variation POWER Given a specific sample size and alpha, in light of the calculated average inter-group difference and its variation, you obtain an estimate of the power (ie 1-beta) Questions? Materials and methods How was the problem studied? Materials and methods How was the problem studied? The answer is in the Methods Expanded IMRAD algorithm Introduction Background Limitations of current evidence Study hypothesis Methods Design Patients Procedures Follow-up End-points Additional analyses Statistical analysis Results Baseline and procedural data Early outcomes Mid-to-long term outcomes Additional analyses Discussion Summary of study findings Current research context Implications of the present study Avenues for further research Limitations of the present study Conclusions Structured approach • Study design • Patients (selection) • Procedures • Follow-up • Outcomes (ie end-points, definitions) • Additional analyses (eg IVUS, QCA, CT) • Statistical analysis Materials and methods • Describe what was done to answer the research question • Give full details of the methods • Include a clear statement of study design “The EXCITE study was a double-blind, randomized, parallel design … designed to compare the efficacy and safety of …” • Include a sentence about IRB approval, informed consent, or compliance with animal welfare regulations “The protocol was approved by the institutional review board, and all patients gave informed consent …” Materials and methods • State the protocol/procedures. Repeat the question. “We tested the efficacy of xemilonercept administered subcutaneously in a dose of 30 mg, given three times weekly for up to 6 months.” “There were 2 primary endpoints. The first was eventfree survival at 182 days, with an event defined as…” • Describe materials/methods or subjects adequately • Write in a logical order (usually chronological) • Describe analytical methods Materials and methods • Use subheadings • Do not include results in Methods • Include appropriate figures and tables • Write in past tense • Use active voice whenever possible • Place details in parentheses – BMI decreased 10% (from 32.6 to 29.4, p=0.027) Materials and methods • Use a figure for a complex design • Cite references for published methods • Describe others fully • Discuss learning curve implications • Enable the reader to a comprehensive appraisal of selection, performance, adjudication, and attrition bias Materials and methods • Briefly address questions you can anticipate from the reader, eg justify/clarify the design of your study or specific adjustments in your procedural protocol: “Bail-out open surgical resection was envisaged whenever laparoscopic surgical resection proved…” Materials and methods • Treat limitations of this study in a matterof-fact way: "These studies were performed as part of a routine clinical assessment, so that no attempt was made to ensure either fasting of the patient or performance of the test at a particular time of day." Design subsection • State clearly the design of the study • Was it retrospective or prospective? • Was it a registry or controlled study • Did you randomly allocated patients? • Did you follow a protocol (may add figure!)? You can also include here details of IRB approval Retrospective controlled study Prospective clinical study Study protocol Nelson et al, NEJM 2004 Patient subsection • State clearly how you selected patients • Specific inclusion criteria? • Specific exclusion criteria? You can include here details of written informed consent Patient data Ganio et al, BJS 2001 Procedure subsection • State clearly how you performed the procedure • Any novel approaches or devices? • May include lesion appraisal subsection • Can contain info on follow-up means • Complete with details on concomitant or postintervention medications You can include here pictures detailing what you did/use Procedural details Technical details Miccoli et al, AOHNS 2006 Outcome subsection • State clearly which outcomes you adjudicated • Define each of the most pertinent outcomes thoroughly • Define the timing of follow-up • Make sure you use validated definitions or classifications (if available, otherwise you are in trouble!) You can include here quality of life data Outcomes & definitions COLOR, Lancet Oncol 2005 Additional analyses subsection • Focuses on additional analyses that may be pertinent to the study – Nuclear scans – Pathologic classifications – Radiographic imaging – Ultrasound scans • Quote thoroughly for established methods • Define explicitly terms and ways to compute secondary variables Statistics subsection • Explain how you handled and reported categorical and continuous variables • Explain how you tested for significance at both univariate and multivariate analysis • Define tails and threshold p value • State width of confidence intervals • Provide sample size computation • Spell out which software package was used Quote extensively and be ready to defend yourself if you use sophisticated analytic tools Statistics subsection COLOR, Lancet Oncol 2005 Categorical variables • Categorical variables are probably the most important ones provided by a clinical study, as hard clinical end-points are always expressed so • Specifically, focus on: • Choose a few statistics, and used them consistently • Provide confidence intervals • May also provide number needed Continuous variables • Continuous variables are important for the appraisal of baseline/procedural characteristics (eg waist-hip ratio), or additional analyses (eg length of hospital stay) • Focus on these points: • Provide mean and standard deviation • Or median (interquartile range) if non-Gaussian • May check for normality assumptions P shows direction of difference Nelson et al, NEJM 2004 Confidence intervals (CI) show size of difference Nelson et al, NEJM 2004 Other examples Manouras et al, AJS 2008 Questions? Take home messages The most important points to remember when writing the Methods section are: Take home messages The most important points to remember when writing the Methods section are: 1. State exactly what you have done, no more than that 2. Concentrate on the primary aim of the study, not on the ancillary goals 3. Ensure reproducibility And now article appraisal…