Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Abstracts of the 19th Cochrane Colloquium Editorial Abstracts of the 19th Cochrane Colloquium Resúmenes del 19◦ Colloquium Cochrane 1 2 Oral sessions Supplement 2011 A. Knowledge translation Knowledge translation and communicating evidence (A1) Knowledge translation and communicating evidence (A2) Knowledge translation and communicating evidence (A3) Knowledge translation and education (A4) Knowledge translation and communicating evidence (A5) Knowledge translation in patient safety (A6) 3 6 8 9 11 13 B. Methods Diagnostic test accuracy review methods (B1) Searching and information retrieval (B2) Searching and information retrieval (B3) Methods for preparing reviews (non-statistical) (B4) Methods for preparing reviews (statistical and non-statistical) (B5) Statistical methods (B6) Investigating bias (B7) Investigating bias (B8) 14 16 18 21 22 24 25 27 C. Other topics Education and training (C1) Global health and equity (C2) Consumer issues and shared decision making (C3) Editorial processes and supporting review authors (C4) 29 34 36 37 Posters sessions Posters sessions 41 Workshops Workshops 205 Annex LatinCLEN Sesiones orales Sesiones de pósteres 235 241 Annex REDIBAM-GPC and JBI COnNECT+ Talleres REDIBAM-GPC Taller JBI COnNECT+ 249 251 Author Index Author Index 253 Abstracts available online at www.cochrane.org EDITORIAL Abstracts of the 19th Cochrane Colloquium maintaining some at their own. A similar approach is taken with the VI International Conference on Patient Safety, although this event only overlaps with the Colloquium the first day and a half. Apart from this book, the Colloquium Programme resumes the content of the different plenaries, which will be recorded and posted in the web afterwards, and the high number of workshops and meetings that have been scheduled. All together, it is a good demonstration of the vitality of the Cochrane Collaboration and the variety of topics that are being addressed. Finally, it is necessary to remark and appreciate the partnership of Wiley & Sons in the publication of this book. Dear colleagues, This is the Abstracts book where the oral presentations and posters of the 2011 Madrid Colloquium are described: there are 72 contributions that will be presented orally, grouped in 3 parallel sessions and with four presentations each. The 393 included posters have been assigned to one of three sessions that will take place during the morning coffee breaks. We really thank all those who evaluated the abstracts received, since it was a demanding task given the amount of work and the quality of most studies. You also can find an annex with the abstracts to be presented at the LatinCLEN (Red Latinoamericana de Epidemiologı́a Clı́nica) independent oral and posters sessions. LatinCLEN and REDIBAM-GPC (Red Iberoamericana de Guı́as de Práctica Clı́nica) celebrate their regular meetings as part of the Colloquium, sharing many activities but Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Xavier Bonfill Director Iberoamerican Cochrane Centre 1 Cochrane Database Syst Rev Suppl 1–1 (2011) DOI:10.1002/14651858.CD000003 EDITORIAL Resúmenes del 19◦ Colloquium Cochrane Se trata de un enfoque similar con el de la VI Conferencia Internacional sobre la Seguridad del Paciente, aunque este evento sólo se solapa con el Colloquium el primer dı́a y medio. Al margen de este libro, el Programa del Colloquium aporta el contenido de las sesiones plenarias, que serán grabadas y publicadas en el web más tarde, y el gran número de talleres y reuniones que se han organizado. El conjunto de estas actividades es una buena demostración de la vitalidad de la Colaboración Cochrane y de la variedad de temas que se abordan. Por último, es necesario destacar y valorar la contribución de Wiley & Sons en la publicación de este libro. Estimados colegas, Éste es el libro de resúmenes donde se describen las presentaciones orales y pósteres del Colloquium 2011 de Madrid: serán 72 las contribuciones que se presentarán oralmente, agrupadas en tres sesiones paralelas y con cuatro presentaciones en cada una. Los 393 pósteres incluidos han sido asignados a una de las tres sesiones que tendrán lugar durante las pausas-café de la mañana. Realmente queremos agradecer a todos aquellos que desde el Comité de Abstracts han evaluado los resúmenes recibidos, ya que la tarea ha sido difı́cil por la cantidad de trabajo y la calidad de la mayorı́a de los estudios. También publicamos un anexo con los resúmenes que se presentarán de forma independiente en las sesiones orales y de pósteres de LatinCLEN (Red Latinoamericana de Epidemiologı́a Clı́nica). LatinCLEN y REDIBAM GPC (Red Iberoamericana de Guı́as de Práctica Clı́nica) celebran sus reuniones regulares como parte del Colloquium, donde van a compartir muchas actividades aunque manteniendo algunas de las suyas aparte. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Xavier Bonfill Director Centro Cochrane Iberoamericano 2 Cochrane Database Syst Rev Suppl 2–2 (2011) DOI:10.1002/14651858.CD000003 Abstracts available online at www.cochrane.org Abstracts of the 19th Cochrane Colloquium A1O2 Using evidence to stop ineffective practice: The UK Cochrane Centre (UKCC) and the National Institute of Health and Clinical Excellence (NICE) work together Oral sessions A1 Knowledge translation and communicating evidence A1O1 Presenting the results of a Cochrane review to policy makers in a video format Docherty M1 , Garner S2 , Littlejohns P2 , Clarke M3 1 Institute of Psychiatry, Kings College London, UK; 2 NICE, UK; 3 Centre for Public Health, Queen’s University Belfast, Northern Ireland Rosenbaum S1 , Glenton C1 1 Norwegian Knowledge Centre for the Health Services, Norway Background: All healthcare systems face increasing pressure to control expenditure without compromising quality or patient safety. Disinvestment in clinically- or cost-ineffective interventions is thought to be key to transferring resources to higher-value care. Objectives: NICE and the UKCC undertook a pilot project using Cochrane reviews as an evidence base to identify low-value interventions for local NHS disinvestment. Methods: Over six months all Cochrane reviews concluding that an intervention could not be recommended or should be used only in research were identified by UKCC and evaluated by NICE. (Figure 1). The potential impact of stopping the intervention was assessed against four domains: quality of care, patient/carer experience, patient safety and productivity savings. (Table 1). Interventions were excluded if they: were not relevant to UK practice; required additional investigation; or were unlikely to achieve gains in any domains. Evaluated topics were published as reports suggesting disinvestment opportunities for local healthcare providers. (Table 2) Results: Of the 65 reviews appraised, 43% provided candidate interventions for local disinvestment. (Table 3) Quantification of potential productivity savings was difficult due to the absence of NHS usage data. Most could not be quantified; one suggested potential savings of £2 million. Many reviews required additional analyses by NICE. Furthermore, many interventions were no longer used or had insufficient evidence to refute potential benefits, making disinvestment recommendations either irrelevant or inappropriate. Conclusions: This is the first large-scale systematic application of a gold standard evidence base to the problem of disinvestment. It challenges assumptions that disinvestment can produce large savings, highlights the importance of robust processes to evaluate interventions when research evidence is inadequate, and demonstrates the need for detailed data on healthcare usage and expenditure. It identifies a translational gap between the requirements of decision makers and the content of Cochrane reviews, highlighting a need to improve the consistency and precision of Cochrane review conclusions. Background: There is growing attention to how The Cochrane Collaboration can best present the results of Cochrane reviews in a manner that is both reliable and accessible. A number of research and development projects have focused on the development of summary formats aimed at clinicians, consumers and policy makers. While these projects have brought us to presentation formats that are both reliable and accessible, a number of challenges remain, such as readers sometimes ignoring or misunderstanding important methodological concepts. Objectives: To develop a video presentation format that can: • help policy makers understand the results of a specific review • help policy makers understand general methodological concepts • attract a wider audience to Cochrane reviews • be easily accessible in multiple languages • be easily transferable to other reviews with a minimum of resources Methods: We used the knowledge gained from our developmental work with Summary of Findings Tables, Plain language summaries and SUPPORT summaries for policy makers. We selected the Cochrane review on the effects of lay health workers for maternal and child health and tuberculosis. We developed a 10-minute video using low-tech, low-cost solutions, and presenting what we know about the benefits and harms of this intervention, what we do not know, and how confident we can be in these results. We pilot tested the video among a relevant audience before developing a second version of the video and publishing it on YouTube. Results: Pilot testing led us to increase the information about what we do not know (i.e. where information was lacking for important subgroups and outcomes). Responses were largely positive, including appreciation of narration and use of tables, maps and photos. A few people were concerned about length. A second round of testing is being planned. Conclusions: Video is a feasible format for reliable, accessible and low-cost presentation of Cochrane Reviews. http:// www.youtube.com/watch?v=0zHHhpE5Rb0 Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. 3 Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 4 Figure 1 (A1O2). Flow chart of NICE review process. UKCC Manager Tech lead Deadline for report Tech support Send new DNDs For each DND QA Analyse and formulate recommendation Discuss recommendation DND DND rejected accepted Inform of reason Costing support Request costing and background Prepare Prepare background costing Prepare Cochrane statements For QA Send statements for QA Sign off QA statements Finalise statements and send for sign off Sign off statements for publication Table 1 (A1O2). Cochrane QP topic selection criteria. Criterion Examples Potential productivity and cash-releasing savings Direct cash savings: substituting prescriptions of minocycline for acne vulgaris with cheaper tetracyclines. Releasing human or physical resources: stopping routine fetal and umbilical Doppler ultrasound in normal pregnancy would free up clinic time for other patients or other activities. Potential impact of disinvestment on quality of clinical care delivered to patients and on clinical outcomes. The use of joint lavage alone, without debridement, in patients with osteoarthritis of the knee has not been shown to result in relevant benefits. Therefore, stopping it will result in improved quality of care. Potential impact on patient safety There is no evidence that selective serotonin reuptake inhibitors are effective as a treatment for children with autism and there is some evidence they can cause harm. Stopping the routine use of these medications in children may improve patient safety. Potential impact on patient and carer experience Stopping the routine replacement of peripheral venous catheters without clinical indication after 72–96 hours was predicted to reduce unnecessary patient pain and discomfort without compromising clinical quality or patient safety. A1O3 Wording GRADEd evidence statements and recommendations Kunnamo I1 , Kattainen A1 , Pelttari H1 , Alenius H1 , Atula S1 , Jousimaa J1 , Teikari M1 1 Duodecim Medical Publications Ltd, Finland Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: The GRADE process requires that the quality of evidence and strength of recommendation are assessed, and the reasons for downgrading or upgrading the evidence, and for giving strong or weak recommendations are described transparently. Since 2000, we have developed over 4000 evidence summaries containing evidence statements in PICO (Patient, Intervention, Comparison, Outcome) format, and recently we started to include recommendations. The main Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 5 Table 2 (A1O2). Summary of Cochrane Quality and Productivity topic reports. Section Explanatory notes Title Cochrane review title reworded to state intervention and patient population included in the Cochrane QP Topic. Introductory text A standard description of the Cochrane QP Topic process. Summary • Quotation from Implications for practice summary of Cochrane review • NICE summary of review authors’ conclusions • NICE comment Details of Cochrane review • Cochrane review title and hyperlink • Citation • Date review content was assessed as up-to-date • QIPP category • Relevant codes: ICD 10, programme budget. Background • Information about the condition and intervention Evidence • Summary of evidence (Information usually includes number of trials, outcomes measures used, inclusion criteria, etc.) • Relevant NICE guidance • Other NHS Evidence accredited guidance Potential impact on quality of NHS care • Impact on clinical quality • Impact on patient safety • Impact on patient and carer experience Likely ease of implementation • Time taken to implement • Healthcare sectors affected • Likely stakeholder support source for the evidence summaries are Cochrane reviews. Objectives: To develop standard expressions in the English language for the evidence and recommendations reflecting different grades of quality and strength, and stating reasons for downgrading and upgrading. Methods: Using the publications of the GRADE Group as a starting point, the EBM Guidelines Editorial Team developed standard wording in a consensus process. A web-based questionnaire was used in 2010 to examine the confidence of professionals in the evidence supporting an intervention when the same evidence statement used different verbs. The 132 respondents from several countries were mainly physicians involved in developing guidelines and systematic reviews. Results: On a 7-point Likert scale where 1 represented the highest and 7 the lowest confidence in the evidence, different verbs received mean scores as follows: is 2.12, seems 3.65, appears 3.67, may 4.08, might 4.39, might possibly 5.02. Examples of evidence statements and recommendations using standard expressions are shown, and a website is presented that contains all the expressions used in commenting on the evidence and the recommendations. Conclusions: There is wide variation in professionals’ interpretation of different wording, but wording that results in differences in confidence can be found. We selected the following verbs to express the four grades of the quality of evidence: high – is; moderate – appears; low – may; very low – might possibly. Standard expressions can make evidence and recommendations more transparent and easier to interpret by professionals. A1O4 Scientific evidence and practice: Bridging the gap. A European Monitoring Center for Drugs and Drug Addiction (EMCDDA) project to promote Best Practice in drug addiction Bo A1 , Allara E2 , Ferri M1 1 EMCDDA, Lisbon, Portugal; 2 University of Turin, Italy Background: Best Practice is the best application of the available evidence to current activities in the drug field. Evidence-based knowledge and practice, that is the experience and lessons learned from the implementation of evidence-based interventions, are two key dimensions of the Best Practice concept. Objectives: To develop a knowledge-base on the EMCDDA website providing reliable information on the latest scientific evidence as well as on the practical aspects of implementation throughout Europe. Methods: We search for systematic reviews and summarise resultsin plain language modules following the Patients-Interventions-Comparisons-Outcomes-Type of studies logic. Interventions are scored for the level of evidence based on the GRADE assessment system, and results are grouped by the level of impact on patients (the upper level being ‘Beneficial’ and the lower ‘Evidence of ineffectiveness’). Tables with summaries of ongoing studies are also available. The EMCDDA’s network Table 3 (A1O2). Summary of rationale for exclusion of reviews. Rationale for exclusion Number of reviews excluded Not licensed/ available in the UK 6 Clinical question not relevant to UK practice 5 Recommended change in practice was in line with standard UK practice therefore no ‘gains’ 3 Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Examples Intravenous alpha-1 antitrypsin augmentaion therapy for treating patients with alpha-1 antitrypsin deficiency and lung disease Hosptialisation and bed-rest for multiple pregnancy Hydralazine for essential hypertension Antibiotics to reduce post-tonsillectomy morbidity Heroin maintenance for chronic (treatment resistant) Heroin-dependent individuals Cervico-thoracic or lumbar sympathectomy for complex regional pain syndrome Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 6 Table 3 (A1O2). Continued. Rationale for exclusion Additional analysis (for example non-RCT data or safety data) would be required before a disinvestment conclusion could be confirmed. Number of reviews excluded 10* * For 4 of these, there were specific patient safety concerns about the consequences of disinvestment The review provided evidence that the intervention was potentially beneficial and thus not a potential candidate for disinvestment Complete or partial clash with existing NICE guidance e.g. The intervention may be recommended by NICE in particular circumstances which were not directly considered in the review Complex or controversial topic requiring more a) extensive analysis b) review of evidence or c) detailed explanation of findings than afforded by this process NICE guideline in progress specifically addressing topic 4 Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes Interventions for smoking cessation and reduction in individuals with schizophernia Drug therapy for the management of cancer related fatigue 5 Adenoidectomy for otitis media in children Rehabilitation interventions for improving physical and psychosocial functioning after hip fracture in older people Carbamazepine for acute and chronic pain in adults Glatiramer acetate for multiple sclerosis 1 Antibiotics for preterm rupture of membranes A2 Knowledge translation and communicating evidence A2O1 Impact of reporting the number treated needlesslyon perceived effectiveness and decision to adopt an intervention Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Teicoplanin versus vancomycin for proven or suspected infection Physical methods for preventing DVT in stroke Minimally invasive surgery versus open surgery for the treatment of solid abdominal and thoracic neoplasms in children Orthodontics for treating temporomandibular joint (TMJ) disorders Anticoagulant therapy for deep vein thrombosis (DVT) in pregnancy Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases Noninvasive positive pressure ventilation as a weaning strategy for intubated adults with respiratory failure 6 of national focal points regularly submits examples of interventions that are assessed according to the level of evidence. Results: The Best Practice Portal (BPP) (www.emcdda.europa.eu/best-practice) currently contains modules on treatment, and harm reduction and prevention. The modules are biannually updated with new evidence. A system for collecting relevant questions and commissioning Cochrane systematic reviews is being studied. Experience from practice is presented in a separate section, including a collection of European standards and guidelines and a collection of real-life projects from 30 European countries (EDDRA database). Conclusions: Identification of evidence and implementation experiences has important implications in highlighting research gaps as well as defining research priorities. The BPP is the result of our first effort to bridge evidence and practice, to disseminate and promote evidence-based interventions throughout Europe and foster a common approach. Nabhan A1 1 Ain Shams University, Egypt Examples Background: The success of evidence-based practice depends on effective communication of clinical trial results. It is suggested that trials should not only provide figures that indicate the positive impact of interventions, but also those that indicate how many would not benefit, the ‘number treated needlessly’ (NTN). Objectives: To assess the impact of explicit reporting of NTN on perceived effectiveness and decision to adopt an intervention among professionals and medical students. Methods: Cross-sectional survey with random allocation to different formats for presenting effect measure (absolute risk reduction (ARR), relative risk reduction (RRR), number needed to treat (NNT), and NTN). The clinical scenario was ‘‘use of statins for the primary prevention of cardiovascular disease’’. The outcomes were persuasiveness (measured as a decision to adopt the intervention) and perception (measured on an 11-point scale of perceived effectiveness). The study included 1472 participants. Results: The percentage of participants who decided to adopt the intervention was 92.4%, 87.8%, 67.7%, and 26.2% when RRR, NNT, ARR, and NTN were used, respectively. Comparing ARR with NTN, perceived effectiveness was smaller with NTN [Mean Difference (MD) −1.68 (95% Confidence Interval (CI) −2.06 to −1.30)] and Odds Ratio (OR) for decision to adopt the intervention was 0.17 (95% CI 0.12 to 0.23). Comparing RRR with NTN, perceived effectiveness was smaller with NTN [MD −3.88 (95% CI −4.27 to −3.49)] and OR for decision to adopt the intervention was 0.03 (95% CI 0.02 to 0.05). Comparing NNT with NTN, perceived effectiveness was smaller with NTN [MD −3.14 (95% CI −3.58 to −2.70)] and OR for decision to adopt the intervention was 0.05 (95% CI 0.03 to 0.07). Conclusion: Explicit reporting of the number treated needlessly has a significant impact on perceived Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 7 effectiveness and decision to adopt an intervention. It is likely to help people to make a more balanced, well-informed decision. A2O2 The use of synthesis methods in support of developing a new surveillance initiative Rajic A1 , Tuševljak N2 , Dutil L1 , McEwen S2 1 Public Health Agency of Canada, Canada; 2 University of Guelph, Canada Background: Research synthesis methods can be used to generate evidence-based inputs for policy and decision-making. We describe the use of three research synthesis methods: scoping study, systematic review (SR) and meta-analysis (MA) in support of developing a potential new surveillance program for zoonotic bacteria and antimicrobial resistance on seafood in Canada. Methods: In consultation with principal stakeholders, a scoping study was conducted to characterize the global primary research investigating zoonotic bacteria, antimicrobial use and antimicrobial resistance. Evidence-maps were evaluated for 8 selected zoonotic bacteria and all finfish, crustacean and mollusc species. SR-MA were prioritised to evaluate retail (point of consumer purchase) prevalence of E. coli, Salmonella, Aeromonas and Vibrio, which had the most underpinning research, on products frequently consumed based on Canadian consumption and import data; salmon, tilapia, shrimp/prawns, oysters/clams/mussels. Random-effects meta-analyses were conducted on subsets of homogenous data by aquatic species, bacteria, and sampling region when >2 lines of data were available, otherwise summarized by median/range. Results: Evidence maps included 1760 relevant papers and SR-MA was prioritized for 56. Pooled prevalence data for Aeromonas, E. coli and Salmonella for retail salmon were 13% (6 to 27%), 2% (0 to 11%) and 0% (0 to 5%) respectively. Many subsets were heterogeneous. The highest proportion of studies reported Vibrio prevalence on clams/oysters/mussels ranging from less than 10% to greater than 90%. Overall, there were few studies, and data gaps were identified for some large seafood exporting countries including Canada. Conclusions: Through the scoping study we have rapidly identified and evaluated the distribution, relevance and overall utility of research pertaining to this broad topic and prioritized focused SR-MA questions relevant to our stakeholders. The data gaps and limitations identified may be used to prioritize future research and the knowledge summaries from the scoping study and SR-MA may be used in the design of targeted, risk-based surveillance, prioritising initial baseline bacteriological surveys and as inputs for risk assessments. A2O3 Grading evidence and recommendations: Randomised experimental evaluation of four grading systems Perez G1 , Cuello C2 , Pacheco K3 1 Red Cochrane Iberoamericana, Mexico; 2 ITESM, Red Cochrane Iberoamericana, Mexico; 3 ITESM, Mexico Background: More than 60 systems for grading evidence and recommendations have been described, reflecting discrepancies among Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. those creating and synthesizing evidence. Objectives: To evaluate the effect of presenting a recommendation using different grading systems to determine to what extent the system changes the clinician’s eventual response to a particular clinical question. Methods: Randomised experimental study in clinicians’ offices and academic settings. Paediatricians and paediatric residents in private and public practice in Mexico were handed case notes of a child with diarrhoea and a question about clinician preference for using an anti-diarrhoeal medication. The same evidence was provided in a clinical recommendation but with different presentation according to the following grading systems: NICE (National Institute for Health and Clinical Excellence), SIGN (Scottish Intercollegiate Guideline Network), GRADE (Grading of Recommendations Assessment, Development and Evaluation) and CEBM (Centre for Evidence-Based Medicine, Oxford). The main outcome measure was the mean change in direction from baseline response (measured on a 10 cm visual scale and a Likert scale) and among groups. Results: 216 subjects agreed to participate. Most participants changed their decision after reading the clinical recommendations (mean difference (MD) 0.7 cm (95% confidence interval (CI) 0.29 to 1.0; P < 0.001)). By groups, mean change (95% CI) from baseline was 0.04 (−0.68 to 0.77) for NICE, 0.31 (−0.41 to 1.05) for SIGN, 2.18 (1.48 to 2.88) for GRADE and 0.08 (−0.52 to 0.69) for CEBM (P = 0.007 between groups). In a final survey, a small difference was noted regarding the clarity of the results presented with the GRADE system. Conclusions: The clinician’s decision to use a therapy was influenced most by the GRADE system. Trial registration number NCT00940290. A2O4 Understanding subjectivity in the meta-analytic process: Meta-analysts’ variation in managing variability Chan LS1 , Macdonald ME1 , Carnevale FA1 , Steele RJ1 , Shrier I1 1 McGill University, Canada Background: When meta-analysts arrive at discrepant conclusions from different meta-analyses, methodological differences are assumed to be the primary reason for the variation. Our research team has previously shown that discordant interpretations can exist even when meta-analysts are presented with identical data. Objective: To better understand the analytic processes meta-analysts employ when interpreting meta-analyses. Methods: For this mixed-methods study, we recruited authors of meta-analyses/systematic reviews using theoretical sampling and the following variables: gender, professional training, number of meta-analyses conducted. Fourteen participants have been recruited to date. After asking participants about their general meta-analytical approach, they were provided with simulated data sets (based on published meta-analyses) in relation to treatments for four different conditions. They were then asked to provide treatment effect estimates and confidence intervals after 3, 5, 10, and 20 studies for each of the four conditions. Using grounded theory methodology, we conducted semi-structured qualitative interviews to render explicit the different analytical processes employed. Results: Preliminary findings suggest that meta-analysts engage in the following analytical processes: 1) scrutinizing the data for ‘poolability’; 2) managing a ‘part-whole’ tension (i.e., shifting between a focus on individual Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 8 studies and the entire dataset); and 3) determining a treatment decision. Participants demonstrated variation within these processes as well as in the micro-processes they used to frame and formulate interpretations. Finally, we describe moral dimensions that shape meta-analysts’ analytic processes including: a desire to maintain the integrity of the whole; refusing to recommend a treatment or assign a point estimate because it would not be ‘right’; a concern about financial resources required to further evaluate a treatment; and complying with external standards. Conclusion: These results demonstrate that meta-analysts engage in subjective analytical processes, including important moral considerations, when interpreting meta-analyses. These processes should be considered to improve the format of meta-analysis presentation. A3 Knowledge translation and communicating evidence A3O1 Cross-sectional study of abstract quality Pawlak M1 , Bierling S1 , Dellavalle R1 1 University of Colorado, Denver, USA Background: Abstracts are often the only lasting evidence of research presented at conferences. Abstracts provide valuable information that may not be published outside of conferences. Minimum standard quality and data should be reported in abstracts so the readers can assess the research performed. Objectives: We set out to descriptively quantify the quality of abstracts presented orally at the 19th Joint Colloquium of the Cochrane and Campbell Collaborations. Methods: A modified version of the CONSORT for Abstracts was developed to measure the quality of abstracts for all study types. The checklist consisted of 11 items evaluating title, methods, results, and conclusion. Two independent reviewers used the checklist to review 126 abstracts presented orally at the 18th Joint Colloquium of the Cochrane and Campbell Collaborations in 2010. Descriptive analysis was used to characterize the quality of abstracts. Results: Of the 126 abstracts, 25 were not research in nature, and were excluded from the analysis. The mean score was 8.1 (range 1 to 11) and the median score was 8. The items most abstracts did not report were study design in the title (76%) and participant eligibility (37%). Outcomes were not clearly defined in 30% of the abstracts. Studies were incomplete in 22% of the abstracts at the time of submission. Conclusions: We recommend the use of a standardized methodology for reporting in abstracts to ensure the quality of the report. The findings of abstracts that were incomplete should be published in print or website versions as an addendum. We suggest that conferences provide abstract submission guidelines that are more concrete and descriptive. A3O2 Reporting of effect direction and size in abstracts of systematic reviews Beller E1 , Glasziou P1 , Hopewell S2 , Altman D2 1 Bond University, Australia; 2 Centre for Statistics in Medicine, UK Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: Access to the full text of journal articles is often limited by cost and difficulty of internet access, whereas abstracts are freely available. Abstracts need to be clear, and contain sufficient detail to enable use of the results. The abstract of a systematic review should clearly state the direction, size and statistical uncertainty of estimated intervention effects, and should not require expert statistical knowledge to interpret. Objective: To find out how many abstracts of systematic reviews present the results clearly and unambiguously. Methods: Two authors extracted details from abstracts on written and numeric presentation of results of 182 systematic reviews (64 Cochrane and 118 non-Cochrane) published in 2009. Selected text was extracted to provide examples of good and poor wording of results. Results: The direction of the intervention effect was described in words in 105 of 182 (58%) abstracts, with a further 34 (19%) deducible from the numeric results, and 43 (24%) not deducible from the abstract. The size of effect was given in words in only 22 (12%) abstracts. Cochrane and non-Cochrane reviews had similar quality of reporting results. 105 abstracts (58%) met a minimum standard for reporting of results, but only 11 (6%) included both direction and size of effect in words, size of effect numerically, and confidence interval. Reviews that had statistically-significant results were reported better than non-significant ones (76% vs 29% meeting the minimum standard). Across all abstracts 45 (25%) did not present any numerical summary of the results. Conclusions: The majority of abstracts of systematic reviews fail to communicate their message with regards to the size and direction of the intervention effect and its statistical precision. Examples of better reporting demonstrate that results can be expressed clearly and succinctly. Simple improvements to abstracts of systematic reviews would enhance their value. A3O3 Investigating and improving the understanding of Cochrane diagnostic test accuracy reviews (DTARs) Zhelev Z1 , Garside R1 , Hyde C1 1 Peninsula College of Medicine and Dentistry, Universities of Exeter and Plymouth, UK Background: The understanding and application of test accuracy information for clinical and policy decision making is difficult, and further complicated when primary test evaluations are systematically reviewed and meta-analysed. No research has yet explored health professionals’ understanding and interpretation of such reviews. At the 2010 Colloquium we presented the protocol for the current study, which is now underway. Objectives: We aim to explore how clinicians and policy makers, who vary in knowledge and experience with test accuracy and systematic reviews, extract and interpret information from Cochrane Diagnostic Test Accuracy Reviews (DTARs). We hope that the findings will help the Cochrane Collaboration to improve the presentation and accessibility of this type of review, and will ultimately result in increased use of test accuracy information in clinical and policy decision making. Methods: An online survey gathered preliminary information about clinicians’ and policy makers’ experience with test accuracy information and systematic reviews. Based on this, a purposive sample is being selected to take part in cognitive interviews, using one of the three existing Cochrane DTARs as material. The interviews are video-recorded and transcribed, and Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 9 qualitative analysis is carried out to identify patterns in participants’ ‘reading’ and interpretation of the reviews with a particular focus on the difficulties encountered. Study progress and preliminary findings: Final results will be available by the end of August 2011. Preliminary analysis of the survey and the number of interviews completed so far suggests, however, that there are a number of common difficulties experienced when reading a DTAR including statistical and clinical terminology, the way in which information is structured and presented, and navigating the document. This points to specific ways of improving DTARs’ presentation and accessibility. In addition, the study demonstrates the value of this innovative way of using cognitive interviewing and the possibilities of its application in other research. A3O4 To what extent is the clinical context considered in diagnostic test accuracy (DTA) reviews?: A methodological review Davenport C1 , Hyde C2 1 University of Birmingham, UK; 2 University of Exeter, UK Background: Statistical and methodological issues have, until relatively recently, dominated the test accuracy research landscape. Consideration of clinical context (the intended setting application and role of tests, the downstream consequences of test results and the use of test accuracy measures to convey contextual information) has only relatively recently received attention. Consideration of test accuracy in the absence of contextual information may mislead when making recommendations about test use. Objectives: To investigate the extent to which a representative sample of diagnostic test accuracy (DTA) reviews represent the clinical context in which an index test is to be used when formulating a review question, deciding on synthesis methods, reporting results and making recommendations, and on the basis of results to provide recommendations for the reporting of contextual aspects of DTA reviews. Methods: Published and unpublished reviews were sought by interrogation of the DARE database; the Cochrane Database of Systematic Reviews ; the database of systematic reviews hosted by the Aggressive Research Intelligence Facility, University of Birmingham; the UK NHS National Research Register; and contact with the Cochrane DTA working group. A final random sample of 100 reviews was included from 271 reviews eligible on the basis of title and abstract. Results: 100 included reviews represent 17 disease topic areas and include between 1 and 50 index tests. Preliminary scrutiny of included studies suggests: lack of clarity of question formulation, particularly with respect to index test role and current testing practice; a lack of consistency between question formulation, methods and reporting of results; and unsupported use of test accuracy outcome measures. Conclusions: The findings of this review will have implications for the development of standards for reporting of DTA reviews and the development of the ummary of results table for Cochrane DTA reviews. A4 Knowledge translation and education A4O1 Epidemiology, quality and reporting characteristics of systematic reviews of acupuncture interventions published in Chinese journals Ma B1 , Qi G2 , Li X1 , Wang T1 , Chen Z1 , Zhang Z3 1 Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou,China; 2 Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou,China; The First Clinical School of Lanzhou University, Lanzhou, China; 3 Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou,China; Ministry of Health of the People’s Republic of China, Beijing, China Background: Systematic reviews (SRs) of acupuncture have become increasingly popular in China and have been published in large numbers. This review provides the first examination of epidemiological characteristics of these SRs as well as compliance with the PRISMA and AMSTAR guidelines. Objectives: To examine epidemiological and reporting characteristics as well as methodological quality of SRs of acupuncture published in Chinese journals. Methods: Four Chinese databases were searched (CBM, CSJD, CJFD and Wanfang Database) for SRs of Traditional Chinese Medicine (TCM), from inception through December 2010. Data were extracted into Excel spreadsheets. The PRISMA and AMSTAR checklists were used to assess reporting characteristics and methodological quality, respectively. Results: A total of 88 SRs were identified, None of the reviews had been updated. Less one-third (27.3%) were written by clinicians and one-third (33%) were reported in specialty journals. The impact factors of 53.4% of the journals published in were zero. Information retrieval was not comprehensive in more than half (59.1%) of the reviews. Most (62.8%) did not reported information about quality assessment, while less than half (36.4%) reported assessing for publication bias. Statistical mistakes appeared in one-third (30.3%) of reviews. Though 93.2% reviews used the term ‘systematic review’ or ‘meta analysis’ in the title, no reviews reported conflicts of interest and were updated two or more years after they were published. Conclusions: While many SRs of acupuncture interventions have been published in Chinese journals, the quality of these reviews is troubling. As a potential key source of information for clinicians and researchers, not only were many of these reviews incomplete, some also contained mistakes or were misleading. Focusing on improving the quality of SRs of acupuncture, rather than continuing to publish them in great quantity, is urgently needed in order to increase the value of these studies. A4O2 Credence: creation of reliable evidence in developing countries by continuous enabling Mathew ME1 , Chandra S2 , Sriganesh V3 , Ejaz K4 , Sharma A5 1 Manipal University, India; 2 National Institute of Mental Health and Neuro Sciences, India; 3 Quality Medical Knowledge Foundation, India; 4 Aga Khan University Hospital, Pakistan; 5 Emory University, USA Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 10 Background: The Cochrane Collaboration has an efficient system in place, such that almost anyone interested, can write a Cochrane systematic review. The Cochrane Handbook of Systematic Reviews is so detailed that an author can find the answer to any question encountered. However undertaking a review is a complex and laborious task, and requires constant support. In countries like India, the few existing authors are dispersed geographically and do not have sufficient networking to form this support system to mentor new authors. Hence reviews with specific regional importance may be lacking. Intervention: In India, a group of medical professionals and students from around India, with support from the Cochrane South Asian Center and Network, have formed a network of authors. We call it ‘‘Credence: Creation of Reliable evidence in developing Countries by continuous Enabling’’; a training and support system for authors. The contact author is an interested novice student, so that he/she will get the maximum experience from writing the review. The main mode of communication is by email and Facebook. By writing the review, the novice authors learns and graduates to an expert by an activity-based learning approach. The new experienced author will serve to mentor more students. Result: In two months, we formed five teams with four to five members each, one with a registered title, one in the process of title registration and remaining three working towards a title. Between the authors we have six reviews in various stages of publication. Conclusion: We hope to kindle the mentor-student relationship, by bringing together experts and novices to a common platform using Internet networking technologies. This will generate the critical mass of experienced authors who will be able to write reviews which are more relevant to their setting. A4O3 Turning knowledge into action: Translating Cochrane reviews in the healthcare setting Koster MA1 1 Kaiser Permanente Southern California, USA Background: The Cochrane Library provides a rich source of evidence-based systematic reviews that can be used by healthcare organizations to improve the quality and delivery of patient care. However, little is known about the processes by which healthcare organizations translate Cochrane reviews into action within the healthcare setting. Objectives: To describe and evaluate evidence translation strategies used by a large US health maintenance organization to improve care quality for adults with chronic obstructive pulmonary disease (COPD). Methods: Existing knowledge syntheses from Cochrane reviews were used to develop and localize content for a clinical practice guideline on diagnosis and management of COPD. Interventions aimed at the practitioner, patient and systems levels were then tailored to facilitate awareness and implementation of the guideline recommendations. Practitioner interventions included a combination of passive and active strategies, including basic knowledge dissemination (distribution of guidelines via direct mail and online), interactive online continuing medical education, development of point-of-care job aids, and audit/feedback. Patient-level interventions included point-of-care education booklets, online resources for COPD self-management, and proactive outreach for spirometry testing. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Interventions at the systems level included development and provision of patient outreach lists and computerized decision support, as well as site visits to implement and evaluate workflows and outreach processes and obtain feedback from local leaders and champions. Monthly reporting and review of three measures (spirometry testing for assessment and diagnosis of COPD, and management of COPD exacerbations with systematic corticosteroids and bronchodilators) was conducted to monitor performance. Results: Over a two-year period (2008 to 2010), continued increases in the rates of spirometry testing and management of COPD exacerbations with systemic corticosteroid and bronchodilator medications were observed. Conclusions: Evidence translation strategies incorporating Cochrane systematic reviews have the capacity to significantly improve the quality and delivery of patient care. A4O4 Understanding how meta-analysts interpret meta-analyses Shrier I1 , Steele RJ1 , Carnevale F1 , Chan LS1 , Macdonald ME1 1 McGill University, Canada Background: Understanding how researchers/clinicians interpret evidence may improve communication of meta-analysis results. Objective: To explore discrepancies in treatment recommendations based on identical meta-analyses. Methods: For the quantitative component of our mixed-methods study, we simulated meta-analysis data after 3, 5, 10 and 20 studies for 4 different scenarios. Based on these data (included individual study results, fixed and random effects point estimates, I2 ), published meta-analysts estimated the treatment effect at each point, and whether the treatment was shown beneficial, or would be shown beneficial in the future, and if they would recommend treatment. For the qualitative component, we conducted individual semi-structured interviews with the meta-analysts to substantiate their quantitative responses. Results: These results are based on the 14 meta-analysts analyzed thus far. In general, participants did not believe a treatment was beneficial if there were ≤5 studies. An exploratory Classification and Regression Tree analysis suggested recommending treatment was mostly dependent on the meta-analyst believing the treatment would be proven beneficial in the future, which itself was determined by the treatment being shown beneficial at that time. The total number of events was the most important factor determining if treatment was considered beneficial, provided the confidence interval did not cross the null (86% recommended and 14% unsure). When the confidence interval included the null, 63% of meta-analysts did not recommend treatment. When total events were low, those with higher risk-taking behaviour were more likely to consider (Yes or Unsure) recommending treatment (71% vs. 16%). Our qualitative analysis corroborated the quantitative findings. Conclusions: Decisions to recommend treatment based on meta-analyses appear to be generally related to the number of total events in the meta-analysis, and whether the confidence interval crosses the null. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 11 A5 Knowledge translation and communicating evidence A5O1 What do we know about interpretation and application of test accuracy measures? Davenport C1 , Hyde C2 1 Birmingham University, UK; 2 Exeter University, UK Background: The widespread belief that decision makers have difficulty understanding and applying test accuracy information has not been based on a systematic interrogation of the evidence base to allow quantification or characterisation of the extent of the problem. Aims and objectives: To comprehensively ascertain literature pertinent to the understanding and application of test accuracy measures in order to identify facilitators and barriers to their use by decision makers. Methods: Bibliographic searches were conducted in 2003, 2005, 2007 and 2010, across 11 databases representing medicine, psychology and education. Searches were iterative, purposive and supplemented by reference checking included studies and contact with experts. A narrative synthesis of empirical and theoretical test accuracy and risk communication literature was undertaken. Results: 64 test accuracy and 21 risk communication papers were included. Research is characterised by self selected samples, lacks external validity and primary care is under-represented. Ability to define the most commonly used metrics (sensitivity, specificity, predictive values) is poor. Predictive values and test errors are promoted as most intuitive although there is no empirical evidence supporting the superiority of a single test accuracy metric for diagnostic decision making. Natural frequency and multiple presentation formats facilitate understanding. Verbal descriptions and negative test results may be less well understood. Self-reported use of measures varies: 80% for predictive values, 4% for sensitivity and specificity and 1% for ROC curves and likelihood ratios. Pre-test probability and test accuracy estimation is inaccurate and highly variable which has implications for probability revision. Conclusions: The emphasis in the literature has been on identifying the best single metric rather than identifying an optimal combination and understanding of meta-analytic summary measures has not been investigated. Investigation of contextual and motivational influences on test and test-treat thresholds is required to identify test accuracy magnitudes that will have most impact on diagnostic and therapeutic yield. A5O2 The challenge of assessing quality of evidence generated from mixed treatment comparisons and a potential solution Soares M1 , O’Meara S2 , Cullum N2 1 Centre for Health Economics, University of York, UK; 2 Department of Health Sciences, University of York, UK Background: Mixed treatment comparisons (MTCs) allow the synthesis of direct and indirect effectiveness evidence and can be Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. used, even when available data are limited, to help healthcare decision-makers interpret all available evidence. However, ensuring that estimates from these ‘sparse’ MTCs are interpreted correctly requires the development of appropriate quality of evidence assessments. We applied a MTC approach to the evidence for wound dressings for diabetic foot ulcers. In order to produce a ‘Summary of Findings’ table we attempted to adapted GRADE to allow for the MTC. Objectives: i) to determine the relative effects of alternative wound dressings on diabetic foot ulcer healing; ii) to assess how to reflect the volume and quality of research evidence in a MTC systematic review. Methods: Systematic literature searches and Bayesian mixed treatment comparison with sensitivity analyses and assessment of inconsistencies. The GRADE evidence rating approach was applied to direct network links and a modified GRADE approach applied to MTC evidence estimates based on: the limitations of the informing studies; impact of sensitivity analyses; the volume of direct and indirect data and data consistency; uncertainty in the resulting estimates and publication bias related to informing studies. Results: Fifteen RCTs (nine dressings) were included (Figure 1). Three dressings estimated as having the highest probabilities of being the ‘best’ for ulcer healing had the lowest quality direct evidence. Four dressings informed by higher quality evidence were estimated to have no probability of being the best dressing. A modified GRADE approach for MTCs helped systematically assess and communicate the quality of this MTC-derived evidence to aid interpretation. Conclusions: MTCs are an important evidence synthesis tool likely to be increasingly incorporated into systematic reviews and overviews of reviews. Quality assessment methods for such outputs are required. We suggest that adpated GRADE evidence quality rating methods should be considered for this purpose. A5O3 Communicating Cochrane: An update on the Collaboration’s communications initiative Tarbett L1 1 Canadian Cochrane Centre, Canada Background: The Cochrane Collaboration conducted a Strategic Review of the organization in 2009 in order to identify areas in which the Collaboration needed to grow, improve, or change. One of these areas was communications and marketing. To help meet this need a Marketing and Communications Working Group (MCWG) was established to assess the current communications situation and identify potential strategies. One of the recommendations was to hire a professional communications and marketing firm to aid in this important endeavor. Objectives: A Request for Proposal was circulated internationally to various professional firms and Delta Media (Ottawa, Canada) was approved for hire by the Steering Group at the mid-year meetings in 2011 in Split, Croatia. Delta Media will provide the professional expertise the Collaboration currently lacks internally in order to further increase the Collaboration’s profile globally and have the efforts of its 28,000 outstanding contributors recognized. Methods: Delta Media will use tools provided by the Collaboration (Scoping Document created by the MCWG)and interview various key players in order to understand the unique nature of the organization as well as its unique communications needs and strengths. Delta Media will draw on its nearly 20 years of experience, proven expertise and Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 12 Figure 1 (A5O2). A network summary of all dressing comparisons informed by direct trial data. Figures presented are the number of studies informing the link (n=), odds ratios and intervals around each ratio. QOE = Quality of evidence. current market research in order to advise the Collaboration on where to invest communications/marketing funds to receive the ‘best bang for our buck’. The firm will work closely with individual members and departments, such as the Web Team, to help the Collaboration reach its full communications potential. Results: Delta Media, will enhance the communications impact of the Collaboration markedly and thrust the organization into the mainstream when it comes to evidence-based research. Initiatives implemented or advised by Delta Media will be reported to Colloquium participants in order to apprise them of the progress made in this important initiative as well as future plans. A5O4 Approaches to estimate and present baseline risks: Recommendations for Cochrane reviews ‘Summary of findings’ tables Hayden JA1 , Tomlinson G2 , Ni A3 , Parker R4 , Cartwright J4 , van Tulder M5 , Altman D6 1 Dalhousie University, Canada; 2 University Health Network, Canada; 3 Canada; 4 Capital District Health Authority, Canada; 5 EMGO, Netherlands; 6 Oxford, UK Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: The ‘Summary of findings’ (SoF) table is a key tool to present the main findings of Cochrane reviews in a transparent and simple format. The SoF table presents an absolute measure of treatment effect, using a measure of baseline risk to convert relative effect sizes. This is desirable as an absolute measure provides an estimate of how likely an intervention is to work in a specific population, or in an individual patient. Objectives: We explored the relevant methodological literature, compared and identified strengths and weaknesses of different approaches to estimate baseline risk toward the goals of valid, feasible and understandable presentation of absolute measures of effect. Methods: Baseline risk estimates for health populations were extracted from randomized controlled trials, prognostic systematic reviews, and individual prognostic studies. The table presents data sources and estimate measures compared. Simulation modeling was used to assess the potential impact of more or less extreme estimates. Results: A summary of the methodological literature, results of exploratory analyses and simulation studies have been conducted. These analyses have demonstrated that the best estimate of baseline risk depends on the review question, including the size of the treatment effect, how well the trial control groups represent the population of interest, and the heterogeneity in the baseline risk values. It is appropriate to estimate the baseline risk from the median RCT control group rate in many situations; in some Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 13 Table 1 (A5O4). Approaches to estimate baseline risk explored. Data Sources – Selected RCT(s) (control group): largest study; clinically selected study – MA of RCTs (control group): studies from current review; related reviews – Selected prognosis study(ies): general population; defined by PFs or risk prediction models – MA of prognosis studies +/− clinical judgment Measure(s) of Estimate Presented – Measure of central tendency – For Subgroups – For Risk categories +/− probability distributions of estimates RCT = randomized controlled trial; MA = meta-analysis; PF = prognostic factor. cases estimation of baseline risk from prognostic evidence is more appropriate. Conclusions: We have developed several resources for reviewer authors and Cochrane Review Groups in making decisions about baseline risk estimates including: i) an algorithm to guide decision-making about appropriate choice for baseline risk estimate; ii) recommendations for identifying baseline risk estimates from prognosis studies; and iii) a tool to explore the impact of changes in baseline risk estimates on important absolute treatment effect. A6 Knowledge translation in patient safety A6O1 Implementation and evaluation of an international standardized operating protocol for the prevention of wrong site surgery in Germany Fishman L1 , Renner D1 , Kolbe M2 , Lessing C2 , Thomeczek C1 1 Agency for Quality in Medicine, Germany; 2 Coalition for Patient Safety c/o Institute for Patient Safety, University of Bonn, Germany Background: The objective of the WHO initiative ‘Action on Patient Safety: High 5s’ is to implement and evaluate standardized operating protocols (SOPs) in a multi-national learning community in order to improve patient safety in hospitals. High 5s SOPs have been developed for three topics, namely 1) correct site surgery (CSS), 2) safe management of concentrated injectable medicines, and 3) medication reconciliation. Methods: In Germany the SOPs are implemented successively, beginning with the CSS SOP. In 2009 to 2010, 17 German hospitals of various levels of care were recruited to participate in the CSS module. The core element of this SOP is a surgical checklist used both for SOP implementation and data collection. In hospital workshops the SOP and checklist, which were developed in the USA, were adapted nationally and locally to the preoperative processes in the participating Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. German hospitals while maintaining standardisation as according to the minimum High 5s criteria. The surgical checklists are distributed to the hospitals as carbon copies. After completion of the process, the copies are transferred for data analysis. Five internationally standardised process and outcome indicators are generated from the checklist data on a monthly basis. These allow a continuous assessment of the degree of realisation and the effects of the SOP, and are complemented by qualitative evaluation components addressing barriers and success factors for implementation. Results and Conclusion: Adaptations at both a national and hospital level are necessary in order to ensure acceptance and thus successful implementation of the SOPs. The comprehensive evaluation concept within the High 5s Project, which is applied in numerous hospitals in varying healthcare systems, provides an unprecedented opportunity to analyse the feasibility and impact of specific standardised patient safety solutions locally, nationally and internationally. First evaluation results will be presented at the conference. A6O2 Warnings for serious adverse events in drug labelling: Are same-class drugs treated the same? Panagiotou OA1 , Contopoulos-Ioannidis DG2 , Papanikolaou PN1 , Ntzani EE1 , Ioannidis JP3 1 Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece; 2 Department of Pediatrics, University of Ioannina School of Medicine, Ioannina, Greece; 3 Stanford Prevention Research Center, Stanford University School of Medicine, Stanford, USA Background: Adverse events carry major morbidity and are a considerable cause of mortality. Black box warnings (BBWs) are the strongest medication-related safety warnings in a drug’s labeling information and are placed on a drug label to highlight major risks. Absence of a BBW or asynchronous addition of a BBW among same-class drugs could create confusion in prescription patterns, or could differentially favor drugs in the same category and should, therefore, be justified appropriately. Objectives: We aimed to evaluate systematically the concordance in the BBW labeling between drugs of the same class. Methods: We tagged the 20 top-selling drugs for 2008 (10 with BBWs and 10 without BBWs on their label) that belonged to different drug classes. Labeling information on all drugs in these 20 classes was captured, and differences were recorded regarding the presence and timing of acquisition of BBWs for same-class drugs. Results: We assessed 176 different agents, belonging to the 20 evaluated drug classes. Seven of these agents had been withdrawn for safety reasons. The reasons for withdrawal became BBWs in other same-class agents in only 2 of the 7 cases. BBW-related differences were identified in 9 of the 20 classes, corresponding to 15 BBWs that were not present in all drugs of the same class. The information for 10 of the 15 different BBWs were included in the labels of same-class drugs as simple warnings or text, while it was absent entirely in 5 BBWs. The median interval from the time the BBW had appeared in another drug of the same class was 66 months. Conclusions: Differences in BBW labeling in same-class drugs are common and shape perceptions of safety among similar agents. BBW labeling needs to become more systematic, and requires clear justification for the differences observed within drug classes. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 14 A6O3 An investigation into the assessment and reporting of harms in clinical studies Smyth R1 , Williamson P1 1 University of Liverpool, UK Background: The recent ORBIT (Outcome Reporting Bias in Trials) study has shown that outcome reporting bias is an under-recognised problem that affects the conclusions of efficacy outcomes in a substantial proportion of Cochrane reviews [1]. Selective reporting of harms is expected, however there is a lack of research regarding whether standards for assessment and reporting of harms are being followed. Objectives: To examine the assessment and reporting of harms in clinical studies, and to examine the impact of selective reporting of harms in clinical trials on systematic reviews. Methods: We refer to three studies that demonstrate the problem and potential impact of poor reporting of harms: the ADRiC (Adverse Drug Reactions in Children) systematic review; further results from the ORBIT study; and a case study meta-analysis demonstrating the effect on the benefit-harm ratio. Results: Results from the ADRiC systematic review showed that the reporting of ADRs, in studies where detection of ADRs was a primary aim, was poor, with only 3/101 studies including data on causality, predictability and avoidability. The findings from the interviews from the ORBIT study provides an explanation of why selective reporting occurs in relation to harms. Poor reporting of gastro-intestinal bleeds in trials of aspirin is shown to affect the benefit-harm ratio. Conclusions: This presentation will provide a forum for discussion of the procedures for assessing and reporting harms in clinical trials. This will improve the ability of decision makers to make informed decisions that consider both benefits and harms of an intervention, ultimately improving patient safety. Reference 1. Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ (2010); 340: c365. doi: 10.1136/bmj.c365. A6O4 What is a rapid review? Harker J1 , Kleijnen J1,2 1 Kleijnen Systematic Reviews Ltd, UK; 2 CAPHRI, University of Maastricht, Netherlands Background: Commissioners of Health Technology Assessments (HTAs) require timely and cost-effective reviews to attain efficacious decisions on health care and treatments. In recent years, there has been an emergence of ‘Rapid Reviews’ within HTAs; however, there are no known published guidelines or agreed methodology. Objectives: In order to answer the research question: ‘What is a Rapid Review, and is methodology consistent in HTA Rapid Reviews?’ a study was undertaken in a sample of Rapid Review (RR) HTAs from the Cochrane Database to investigate equalities and/or differences in RR methodology used. Methods: The Cochrane Database of Systematic Reviews was searched, to locate RR HTAs from 2000 onwards. Each RR was Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. examined to investigate the individual methodology used for searching, inclusion screening, quality assessment, data extraction and synthesis. Methods of each RR were compared to investigate whether differences and similarities in methodologies used, in comparison with recognised methods for Systematic Reviews (SRs). Results: 46 Rapid Reviews were included. There was a wide diversity of methodology, with some reviews utilising well-established SR methods, but many others diversifying in one or more areas, i.e. searching, clarity of research questions, inclusion screening, quality assessment, data extraction, synthesis methods, report structure and number of reviewers. Conclusions: Despite the number of Rapid Reviews published within HTAs over recent years, there is no agreed and tested methodology and it is unclear how RRs differ from SRs. In a sample of HTA RRs from 2000 to 2011, there was a wide diversity of methodology utilised in all aspects of these reviews. There is scope for wider research in this area to investigate diversity of methods in more depth so that eventually, recommendations could be made for clear and systematic methods for Rapid Reviews; thus facilitating equity and credibility of this type of important review methodology. B1 Diagnostic test accuracy review methods B1O1 QUADAS-2: an updated quality assessment tool for diagnostic accuracy studies Whiting P1 , Rutjes A2 , Westwood M3 , Mallett S4 , Deeks J5 , Reitsma J6 , Leeflang M7 , Sterne J1 , Bossuyt P7 1 School of Social and Community Medicine, University of Bristol, England; 2 Division of Clinical Epidemiology and Biostatistics, Institute of Social and Preventive Medicine, University of Bern, Switzerland; 3 Kleijnen Systematic Reviews, York, England; 4 Centre for Statistics in Medicine and Department of Primary Health Care, University of Oxford, England; 5 Unit of Public Health, Epidemiology & Biostatistics, University of Birmingham, England; 6 Julius Center for Health Sciences and Primary Care, UMC, University of Utrecht, Netherlands; 7 Department of Clinical Epidemiology, Biostatistics and Bioinformatics, AMC, University of Amsterdam, Netherlands Background: QUADAS was developed in 2003 to assess the quality of diagnostic test accuracy (DTA) studies included in systematic reviews. It includes 14 items covering risk of bias, variability and reporting. Personal experience, anecdotal reports, and feedback via the Cochrane Collaboration suggested that some elements required improvement. Objectives: To update the QUADAS tool to produce QUADAS-2. Methods: We used a four-phase approach to develop QUADAS-2 (Figure 1): defining the scope, developing the evidence base, face-to-face consensus meeting, and refinement through piloting. We established a broad team including methodological experts and review authors working on DTA reviews. Results: We defined quality in terms of risk of bias and concerns regarding applicability. Based on the evidence we identified items for removal, items to be re-worked, and items to be considered for inclusion. Discussions at the face-to-face meeting informed decisions regarding the structure and content of QUADAS-2. After four rounds of piloting, agreement was reached on the final tool. QUADAS-2 consists of four domains Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 15 Figure 1 (B1O1). Development process for QUADAS-2. covering: patient selection; index test; reference standard; and flow of participants through the study and timing of tests. All domains are assessed in terms of the risk of bias. Patient selection, index test, and reference standard are also assessed in terms of concerns regarding applicability to the review question. QUADAS-2 is applied in four phases: (1) summarise the review question; (2) tailor the tool to the review including production of review specific guidance; (3) construct a flow diagram for the primary study; and (4) assess risk of bias and concerns regarding applicability. Conclusions: The development of QUADAS-2 was evidence based and included detailed piloting and refinement. We feel that QUADAS-2 is a considerable improvement on the original QUADAS tool and will allow more objective rating of the risk of bias and applicability of primary DTA studies. B1O2 Updating QUADAS: User experience informs the development of QUADAS-2 Westwood M1 , Whiting P2 , Rutjes A3 , Mallet S4 , Deeks J5 , Sterne J2 , Leeflang M6 , Reitsma J7 , Bossuyt P6 1 Kleijnen Systematic Reviews Ltd, UK; 2 University of Bristol, UK; 3 University of Bern, Switzerland; 4 University of Oxford, UK; 5 University of Birmingham, UK; 6 University of Amsterdam, Netherlands; 7 University Medical Centre Utrecht, Netherlands Background: QUADAS is a widely used, evidence-based quality assessment tool, developed for use in systematic reviews of diagnostic test accuracy (DTA) studies. The experience of the QUADAS steering group, anecdotal reports, and feedback via Cochrane Collaboration members suggested some problems with QUADAS. We therefore decided to revise QUADAS using experience of its application, and new evidence regarding sources of bias and variation to produce QUADAS-2. This abstract describes evidence derived from user experience. Objectives: To gather structured feedback from review authors who have used QUADAS, and to summarise the findings of studies that have evaluated QUADAS. Methods: We sought Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. information on users’ experiences of QUADAS using a web-based questionnaire to elicit feedback from authors of Cochrane DTA reviews and protocols, and authors of reviews on DARE which had used QUADAS. We conducted a systematic review of studies that had evaluated QUADAS. Results: 64 authors completed the questionnaire. The majority (70% to 89%) rated QUADAS as good or very good for coverage, ease of use, clarity and validity. Negative features highlighted were the mixing of questions relating to bias and applicability, and inability to use QUADAS to assess studies comparing multiple index tests. Approximately 15% of review authors reported omitting items on reporting quality from their QUADAS assessments. Despite guidance, 20% of review authors reported using QUADAS to calculate summary scores. Four studies evaluated QUADAS; key recommendations were that QUADAS-2 should be expanded to cover studies comparing multiple index tests and topics where follow-up is the reference standard. Conclusions: The experience of review authors using QUADAS informed development of QUADAS-2 and was largely consistent with the experience and conceptual decisions of the steering group. The experience of QUADAS users will continue to be important in informing future developments and ensuring continued relevance and wide up-take. B1O3 Forecasting yesterday’s weather: The risk of spectrum bias from the inclusion of already diagnosed/treated patients in studies of depression screening tools Thombs B1 , Arthurs E2 , El-Baalbaki G1 , Meijer A3 , Ziegelstein R4 , Steele R1 1 Jewish General Hospital and McGill University, Canada; 2 Jewish General Hospital, Canada; 3 University of Groningen, Netherlands; 4 Johns Hopkins University School of Medicine, USA Objective: Screening involves the use of a test or procedure to detect the presence of a disease in individuals not already seeking Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 16 treatment for symptoms and not already diagnosed with the condition. Studies evaluating the accuracy of screening instruments that include already diagnosed or treated patients are known to produce inflated estimates of screening sensitivity and case yield. The objectives of this study were to investigate: (1) the proportion of original studies included in systematic reviews and meta-analyses on the diagnostic accuracy of depression screening tools that appropriately exclude already diagnosed or treated patients; and (2) whether systematic reviews and meta-analyses of the accuracy of depression screening tools evaluate possible bias due to the inclusion of already diagnosed or treated patients. Design: Systematic review. Data sources: MEDLINE, PsycINFO, CINAHL, EMBASE, ISI, SCOPUS, and Cochrane databases were searched January 1, 2005 to October 29, 2009. Eligibility criteria for selecting studies: Systematic reviews and meta-analyses in any language that reported on the diagnostic accuracy of depression screening tools. Results: Only 8 of 197 (4.1%) unique publications from 17 systematic reviews and meta-analyses specifically excluded already diagnosed or treated patients. No systematic reviews or meta-analyses commented on possible bias from the inclusion of already diagnosed or treated patients, even though 10 reviews used quality assessment tools with items to rate risk of bias from patient sample composition. Conclusions: Studies of the accuracy of depression screening tools rarely exclude already diagnosed or treated patients, a potential bias that is not evaluated in systematic reviews and meta-analyses. This may result in inflated accuracy estimates on which clinical practice and preventive care guidelines are often based, a problem that takes on greater importance as the rate of diagnosed and treated depression in the population increases. B1O4 A novel approach to evaluate the diagnostic accuracy of a sequence of tests Novielli N1 , Cooper NJ2 , Abrams KR2 , Sutton AJ2 1 University of Birmingham, UK; 2 University of Leicester, UK Background: One diagnostic test is rarely sufficient to complete a diagnosis. Despite this, most diagnostic accuracy studies focus on the evaluation of an individual test, and therefore approaches for the meta-analysis of this diagnostic literature have also focused exclusively on estimating the accuracy of individual tests. Where the accuracy of sequences of tests has been modelled previously, the often dubious assumption of test independence has been made (1). Hence, there is a need to develop meta-analytic approaches to correctly estimate the accuracy of test sequence strategies. Objective: To develop a modelling approach to simultaneously synthesise studies of individual and multiple diagnostic test data in order to estimate the accuracy of test sequence strategies. Methods: The methodological framework developed is broadly applicable to contexts where multiple tests are of interest. Here we describe the approach using the motivating example of the Ddimer and Wells tests for diagnosing deep vein thrombosis. This random-effects modelling framework allows: 1) the inclusion of studies evaluating either test singularly and in combination (complete or partial reported data) and thus incorporates all available evidence; 2) for the Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. dependency between diagnostic tests; and 3) the incorporation of tests with multiple thresholds (Wells score). Results: When results of this synthesis are compared to simpler, but invalid, modelling which assumes the performance of both tests is independent, considerably different estimates for test strategies are observed. Conclusions: Accurate estimation of the accuracy of test sequences is critical for evidence-based decision making. For the first time a meta-analytic approach to do this has been developed. Reference 1. Novielli N, et al. How is evidence on test performance synthesised in economic decision models of diagnostic tests? Value in Health 2010; 13(8): 952–957. B2 Searching and information retrieval B2O1 Study selection by means of a web-based Trial Selection DataBase (webTSDB) Hausner E1 , Ebrahim S1 , Herrmann-Frank A1 , Janzen T1 , Kerekes MF1 , Pischedda M1 , Waffenschmidt S1 , Kaiser T1 1 Institute for Quality and Efficiency in Health Care (IQWiG), Germany The German Institute for Quality and Efficiency in Health Care (IQWiG) is an independent scientific institute that investigates the benefit and harm of medical interventions for patients. For this purpose IQWiG prepares systematic reviews on the basis of a systematic literature search. To minimize errors, the process of study selection is performed by 2 reviewers independently of each other. In order to achieve a comprehensible and efficient selection of studies, processes are continuously optimized by the Institute’s Information Management Section in cooperation with other departments. In addition to the establishment of a 3-step screening process aimed at improving external transparency, an MS ACCESS-based database for study selection was programmed and implemented, also prompted by the fact that suitable solutions were not available on the market. In 2009, due to positive experiences with this database, a web-based application was developed with an external service provider (web-based Trial Selection DataBase, webTSDB). The advantages of webTSDB are, in particular, the efficiency and transparency of the procedure, as well as the possibility of simple documentation of the whole process. General functions and possibilities of webTSDB are as follows: • simultaneous work on the project by reviewers, independent of location, • project-specific deposit of inclusion and exclusion criteria, • consensus function for citations with inconsistent classification, • deposit of comments, • preparation of reports and flow charts, • linking to full texts, as well as • automatic data-storage. WebTSDB has been used throughout the Institute since June 2010. The aim of this contribution is to present the advantages, functions and possibilities of this software. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 17 Figure 1 (B2O1). Figure 2 (B2O1). B2O2 Assessing the completeness of systematic reviews via the ‘related articles’ function and/or a simple structured Boolean search in PubMed – a pilot study Hausner E1 , Janzen T1 , Kaiser T1 1 Institute for Quality and Efficiency in Health Care (IQWiG), Germany Background: From July 2011 the Institute for Quality and Efficiency in Health Care (IQWiG) will assess the additional benefit of Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. newly-approved drugs in Germany on the basis of dossiers provided by manufacturers. This early benefit assessment includes an evaluation of the completeness of the evidence base provided. Objectives: To assess whether the ‘related articles’ function and a simple structured Boolean search (SSBS) in PubMed, alone or in combination, are efficient search tools to assess the completeness of an evidence base. Methods: Eligible citations were those included as relevant in 6 randomly-selected systematic reviews (SRs) on drug interventions published by IQWiG. Firstly, these citations were searched for in PubMed and the corresponding ‘related articles’ function was applied; 3 sets of articles (the first 20, 50, and all related articles) were exported. In a second step, an SSBS was performed for each SR and Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 18 the articles retrieved were exported. In a third step, the number of citations screened and number of relevant citations identified were determined for each search tool. In addition, sensitivity and precision were calculated to assess efficiency. Results: A total of 121 relevant citations were identified (4 to 55 per SR). The best sensitivity was achieved by the ‘all related articles’ function, but precision was very poor (100%; 0.44%). The combination of the ‘first 20 related articles’ function and an SSBS achieved the best optimization between sensitivity and precision (97% vs. 5%). The highest precision was achieved by the ‘first 20 related articles’ function (13%); however, this was accompanied by a decrease in sensitivity (80%). Conclusions: Our findings indicate that the combination of the ‘first 20 related articles’ function and an SSBS in PubMed is an efficient method to assess the completeness of an evidence base. Our approach will be applied and evaluated using the dossiers submitted. B2O3 Primary outcomes reported in abstracts and ClinicalTrials.gov – do they agree? Huynh L1 , Scherer R2 , Ervin A2 , Dickersin K2 1 Johns Hopkins Bloomberg School of Public Health, USA; 2 Johns Hopkins Bloomberg School of Public Health, US Cochrane Center, USA Background: The 2011 Institute of Medicine (IOM) standards for systematic reviews recommend handsearching conference abstracts to identify otherwise unpublished research results and address potential reporting biases. We hypothesized that the sparse description of randomized controlled trial (RCT) primary outcomes presented in conference abstracts could be supplemented using trial registry information. Objectives: To compare primary outcomes described in conference abstracts with those reported in ClinicalTrials.gov and assess the relationship between primary outcome reporting and industry sponsorship. Methods: Eligible conference abstracts described RCT results, had a valid ClinicalTrials.gov registration number, and were presented at the 2007 to 2009 Association for Research in Vision and Ophthalmology meetings. Two reviewers independently extracted information from the abstract and ClinicalTrials.gov. Discrepancies were resolved by consensus. Results: We identified 152 eligible abstracts; 40 explicitly reported at least one primary outcome in the abstract and primary outcome field at ClinicalTrials.gov. Of 80 primary outcomes reported in abstracts, 18% (14/80) agreed with one reported in ClinicalTrials.gov. Forty-nine percent (39/80) agreed partially, in that additional information (eg. related to time point) was present in ClinicalTrials.gov (17/39; 44%), the abstract (19/39; 49%), or both (3/39; 8%). Thirty-four percent (27/80) of primary outcomes reported in the abstract were not reported as such in ClinicalTrials.gov, although 6 were reported as secondary outcomes. Industry sponsorship was reported in 6/14 abstracts where a primary outcome was reported in the abstract but not in ClinicalTrials.gov, and in 14/26 abstracts where all primary outcomes reported in the abstract were also reported in ClinicalTrials.gov. Conclusions: Our data suggest that investigators may not be updating ClinicalTrials.gov trial information. Both conference abstracts and ClinicalTrials.gov provide unique primary outcome information not available in the other source. Systematic reviewers should obtain outcome information from both conference abstracts and registries. Study protocols with primary outcome information should be available from trial registries. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. B2O4 PRISMA in practice: keeping track of the literature search documentation. What are the problems? Mann M1 , Sampson M2 , Cooper C3 , Stanfield C4 , Rader T5 1 Support Unit for Research Evidence, Cardiff University, UK; 2 Children’s Hospital of Eastern Ontario, Ottawa, Canada; 3 Peninsula Technology Assessment Group (PenTAG), Peninsula College of Medicine & Dentistry, University of Exeter, UK; 4 EPPI-Centre, Social Science Research Unit, Institute of Education, University of London, UK; 5 Cochrane Cochrane Musculoskeletal Group, University of Ottawa, Canada Background: The PRISMA Statement (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) calls for detailed reporting of elements of the search process, and has been adopted by The Cochrane Collaboration with the intent of advancing the quality and transparency of Cochrane reviews. At a recent colloquium, Information Retrieval Methods Group participants expressed concern at the feasibility of implementing these reporting requirements. Objectives: This study examines adoption of PRISMA search reporting, and challenges related to PRISMA reporting standards in the context of new Cochrane reviews. Methods: We examined 19 reviews flagged as New’ in The Cochrane Library (Issue 4, 2011) for reporting aspects of PRISMA elements 7, 8 and 17; total retrieval size, number of unique records, the number of unique studies identified, as well as a flow diagram and transcript of the search. Separately, a survey of research communities, including Trials Search Coordinators (TSCs), Information Specialists, and authors, explored practice and perceived challenges in search documentation. Results: In our search results all the new reviews incorporated a transcript of the search strategies. Five reviews (26%) included a flow diagram, and these varied in the level of detail provided. Exclusion reasons were rarely provided for records found ineligible based on title and abstract. Survey responses provided insights into the difficulties faced by those responsible for search documentation. These included: lack of time, lack of tools, lack of clear understanding of the requirements, and a lack of clear assignment of responsibility for recording and reporting these elements. In our presentation we will fully discuss the findings. Conclusions: The Cochrane Collaboration adopted PRISMA in 2009; however, support for PRISMA flow diagrams was only available in RevMan version 5.1.1 and reporting compliance remains incomplete. Training and support for TSCs and search authors will help achieve full implementation of this standard. B3 Searching and information retrieval B3O1 Screen2Go: a pilot smartphone app for citation screening Huckvale K1 , van Velthoven M1 , Car J1 1 Imperial College London, UK Background: Although Cochrane’s RevMan software supports authors during the latter stages of a review, citation searching and Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 19 Figure 1 (B3O1). computer. Applied to a real-world screening of some 12,000 articles, there was no obvious deficit in the precision of screening decisions made using this workflow. Conclusions: A smartphone app may be an effective way to support the citation screening process by allowing users to review smaller batches of citations at convenient times. A future randomised crossover study will formally compare the performance of screeners using the tool against usual practice. B3O2 Could a search for a diagnostic test accuracy review be restricted to MEDLINE? van Enst WA1 , Scholten RJPM1 , Hooft L1 1 Dutch Cochrane Centre, Netherlands abstract screening are not part of the current or planned feature set. Screening up to several thousand citations can be a daunting task for a researcher with multiple demands on their time. Objectives: To develop a pilot smartphone and tablet app to explore the feasibility and acceptability of a convenience approach to citation screening. This would allow a user to screen batches of citations during otherwise unproductive periods of the day, for example when commuting to work. Methods: We used a rapid application development approach to build a prototype using the Microsoft Visual Studio SilverLight software development environment for Windows 7 Phone. Results: The prototype app was developed to support a simple workflow. Citations exported from bibliographic software, such as Endnote, can be uploaded to a secure website and are automatically downloaded to the phone for screening ‘offline’. When a mobile data connection becomes available, screening choices are sent back to the website where the user can then view and export batches of citations grouped by decision. The app proved highly acceptable to a small group of test users. It appeared most effective for rapidly excluding citations of obvious irrelevance. Citations with missing information had to be marked as ‘unsure’ and triaged in a separate step using a desktop Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: Often, a search for a review of diagnostic test accuracy studies (DTA review) results in a high number of references to screen. However, if the search could be limited to only the freely accessible database MEDLINE, the number of references needed to screen may be reduced and costs may be spared. Objectives: To assess the number of studies that will be missed when a search for diagnostic accuracy studies is restricted to MEDLINE and whether this will introduce bias in a meta-analysis. Methods: We collected diagnostic reviews published in high impact journals between 2006 and February 2011, that included a search in MEDLINE and in at least one other biomedical database, stated the search date, and did not apply language restrictions. For each meta-analysis, we identified the number of studies that could not be found in MEDLINE. When the meta-analysis included ≥10 studies, we repeated the meta-analysis, but including only those studies that were indentified in MEDLINE. We calculated the ratio of the diagnostic odds ratios (RDOR) based on the MEDLINE only studies versus all studies. Results: We identified 615 reviews of which 44 reviews with 76 meta-analyses were included. In 65 meta-analyses (85.5%) the studies were all identified in MEDLINE. In the remaining 11 meta-analyses, 87.5 to 99.0% of the studies were identified in MEDLINE. Of those, 10 meta-analyses included ≥10 studies. However, due to the very low proportion of non-MEDLINE studies we did not find repeating meta-analysis helpful. Therefore no RDOR was calculated. Conclusions: Surprisingly, the extra yield of a search for primary diagnostic accuracy studies in other databases than MEDLINE was very low. Based on our relatively small sample, searching in databases other than MEDLINE hardly seems to affect the outcome of a DTA review. B3O3 Endorsement of the ICMJE’s clinical trial registration policy by biomedical journals: A survey of instructions for authors Hooft L1 , Molenaar N2 , van der Haar S2 , Scholten R3 1 Dutch Cochrane Centre/Netherlands Trial Register, Netherlands; 2 Netherlands Trial Register, Netherlands; 3 Dutch Cochrane Centre, Netherlands Background: Publication and selective outcome reporting bias are recognized concerns in the medical literature. Prospective registration of trials may provide transparency in clinical trial conduct and reporting. Journals play an important role in encouraging investigators to Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 20 B3O4 A systematic review of studies that develop or evaluate search filters for the retrieval of diagnostic studies in MEDLINE adopt trial registration. In September 2005, the members of the International Committee of Medical Journal Editors (ICMJE) issued a joined registration policy that included that submitted manuscripts would only be considered for publication if they had been registered in a public register before the enrolment of the first patient. Objectives: To determine to what extent biomedical journals have adopted the ICMJE’s clinical trial registration policy and whether reported data are cross-checked with the prospectively registered data. Methods: Two researchers searched for items addressing prospective trial registration in the Instructions for Authors on the websites of journals following the ICMJE’s Uniform Requirements for Manuscripts (URM) and journals that endorsed CONSORT. Only Instructions for Authors available in English were analyzed. Results: We identified 866 biomedical journals. Of those 143 included non-English instructions, were discontinued journals, or journals without a website. Prospective trial registration was required by 319 of the remaining 723 journals (44%) and 145 of these (45%) referred to the WHO International Clinical Trials Registry Platform (ICTRP). No detailed information was given on whether or how submitted manuscripts will be compared with trial registry entries as a component of peer review. Results of an additional survey of journal editors about this topic will be available at the Colloquium. Conclusions: Although the number of biomedical journals requiring registration is encouraging, there is still room for improvement. The registration policy suggests that submitted manuscripts are compared with trial registry entries as a component of peer review; clear instructions to authors were mostly not given. Reviewers’ measure of registration quality should be more than verifying the registration number at manuscript submission. Beynon R1 , Leeflang M2 , Eisinga A3 , McDonald S4 , Mitchell R4 , Whiting P1 , Glanville J5 1 School of Social and Community Medicine, University of Bristol, UK; 2 Department of Clinical Epidemiology and Biostatistics, University of Amsterdam, Netherlands; 3 UK Cochrane Centre, National Institute for Health Research, UK; 4 Australasian Cochrane Centre, Monash Institute of Health Services Research, Australia; 5 York Health Economics Consortium Ltd, University of York, UK Background: Search filters are developed to improve the efficiency and effectiveness of searching and are created by identifying and combining search terms to retrieve records with a common feature (e.g. diagnostic accuracy). Objectives: To determine whether any existing methodological search filters designed to retrieve diagnostic test accuracy (DTA) studies perform well enough to be recommended for use in Cochrane DTA reviews. Methods: We conducted a systematic review of studies that developed and/or evaluated methodological filters designed to retrieve DTA studies in MEDLINE. Eligible studies had to report a measure of filter performance such as sensitivity or precision. Studies were identified through searching electronic databases and reference screening. Two review authors independently screened search results. Inclusion assessment, data extraction and methodological assessment were conducted by one author and checked by a second. Methodological assessment was undertaken using a Figure 1 (B304). Study selection process. 1683 hits Excluded 64 included on title/abstract 35 did not develop or evaluate a diagnostic filter 5 did not report on diagnostic filters in general 15 studies reported in 16 papers 5 did not report performance data 3 reported EMBASE filters 7 MEDLINE development studies 2 MEDLINE development and evaluation studies 6 MEDLINE evaluation studies 56 Filters reported 27 filters development data only Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. 12 filters development and evaluation data 17 filters evaluation data only Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 21 modified checklist. Data to calculate sensitivity, specificity, precision and the number of abstracts needed to read (NNR) to identify one relevant study were extracted for each filter evaluation and were summarized in a forest plot. Results were analysed separately for studies that developed the original filters (development studies) and studies that evaluated existing filters (evaluation studies). Results: Fifteen studies (16 papers) reporting on 56 filters were included: 9 development studies, 6 evaluation studies, and 2 studies that both developed and evaluated filters (Figure 1). None of the filters had consistently high sensitivity and low NNR in the evaluation studies. Estimates of sensitivity ranged from 29% to 100% and NNRs, where available, ranged from 9 to 100. Estimates of sensitivity in the development studies were higher than those reported in the evaluation studies in 28 of 33 comparisons, and estimates of NNR were lower in 5 out of 10 comparisons. Conclusions: None of the filters evaluated showed sufficient sensitivity and precision to be recommended for identifying DTA studies in MEDLINE. B4 Methods for preparing reviews (non-statistical) B4O1 The proposed Cochrane agenda setting and priority setting methods group: establishing a research program Nasser M1 , Welch V2 , Ueffing E2 , Tugwell P2 , Crowe S3 , Lieberati A4 1 Peninsula Dental School, University of Plymouth, Plymouth, UK; 2 Campbell and Cochrane Equity Methods Group, Institute of Population Health, University of Ottawa, Canada; 3 James Lind Alliance, Oxford, UK; 4 Italian Cochrane Centre, Italy Background: There is a continuing interest in defining priority topics for systematic reviews inside and outside of the Cochrane Collaboration. However, there is no consensus on the best methods for priority setting. We are establishing a new Methods Group in the Cochrane Collaboration to address this gap. Objectives: To identify methodological challenges that the Cochrane Collaboration is facing that can be addressed by the future Methods Group. Methods: We organized five events (workshops and meetings) at previous Cochrane meetings: 1. A workshop at the Cochrane Colloquium in Freiburg, 2008. 2. A workshop at the Cochrane Colloquium in Singapore, 2009. 3. An exploratory meeting in Singapore, 2009. 4. A workshop at the Cochrane Colloquium in Keystone, 2010. 5. A workshop at the UK Cochrane Contributors Meeting, Belfast, 2011. During these sessions, we provided the participants with the opportunity to discuss challenges they are facing to implement a priority-setting approach in their Cochrane entity. Results: We identified several challenges that the participants faced in planning, conducting, and implementing a priority-setting process in their Cochrane entity. We categorized them into methods and process challenges and implementation issues. The most important methods challenges included: • dealing with inequalities in developing and conducting priority setting. • uncertainties around whom to consult in the prioritisation process and challenges in involving multiple stakeholders in the process • identifying appropriate criteria for conducting a priority-setting strategy. • difficulties in evaluating, implementing, and disseminating prioritisation Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. processes. • methods to prioritise updating of Cochrane reviews. • how to maintain reviewers enthusiasm and making Cochrane reviews relevant and up to date Conclusions: There are several methods questions on how to prioritize topics for Cochrane reviews that the proposed Methods Group could address. We are currently conducting a number of methods studies to address these challenges. B4O2 Evaluating priority-setting approaches: tools for a Cochrane Methodology Review Nasser M1 , Welch V2 , Ueffing E2 , Tugwell P2 , Chalkidou K3 , Sawicki PT4 , Crowe S5 1 Peninsula Dental School, University of plymouth, Plymouth, UK; 2 Campbell and Cochrane Equity Methods Group, Institute of Population Health, University of Ottawa, Canada; 3 National Institute for Health and Clinical Excellence, London, UK; 4 Institute of Health Economics and Clinical Epidemiology, University of Cologne, Cologne, Germany; 5 James Lind Alliance Secretariat, James Lind Initiative, Oxford, UK Background: There is a continuing interest in defining priority topics and setting a research agenda for systematic reviews inside and outside of the Cochrane Collaboration. However, there is no consensus on the best methods of priority-setting. Therefore, we are conducting a Cochrane methodology review to address this gap. There is no agreement on the tools to critically appraise the quality (transparency, effectiveness, rigor, etc.) of conducting and reporting of the priority setting methods. Objectives: We aimed to develop critical appraisal checklists to evaluate the quality of the priority setting methods in the Cochrane methodology review. Methods: We did a search for studies on the criteria for good or successful priority-setting methods and studies on how equity can be incorporated in priority-setting methods. The results of the literature search were discussed in two workshops at previous Colloquia (Singapore and Keystone). The results of the discussions along with the consultation of experts in the field were used to develop two checklists for the Cochrane methodology review. Results: We developed two checklists, one of the checklists with eight questions to evaluate the transparency and rigour of the process of conducting and reporting of the priority setting methods. We also developed an equity lens with 17 questions across the different steps of a prioritization process. Conclusions: The checklists could be a potentially helpful tool to evaluate primary studies for our Cochrane methodology review and might even help primary researchers in planning and developing priority-setting methods. We will be working further to incorporate other aspects of critically appraising priority setting methods, especially differentiating priority setting methods with better outcomes like efficiency/value of money, and impact. The project also provides guidance for authors of methodology reviews on methods in situations where there is no agreed critical appraisal tool. B4O3 Modifications to the Risk of Bias tool: a case study of 204 trials Hartling L1 , Bialy L1 , Armijo-Olivo S1 , Ha C1 , Lacaze-Masmonteil T1 , Vandermeer B1 , Dryden DM1 1 Canada Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 22 Background: In 2008 The Cochrane Collaboration introduced the Risk of Bias (RoB) tool to assess internal validity of randomized controlled trials (RCTs). Modifications to the tool were released in 2011 after user testing and feedback. Objectives: To describe the results of applying the modified RoB tool. Methods: Two researchers independently applied the tool to 204 RCTs. Disagreements were resolved through consensus. We included several modifications to the original tool. First, we assessed blinding separately for participants, investigators, and outcome assessors. Second, we assessed potential influence of study sponsorship separately from ‘other sources of bias’. We assessed agreement across blinding domains using kappa. We assessed correlation between different blinding domains, and between funding and ‘other’ domains using Kendall’s Tau. Results: Risk of bias assessments varied among the three blinding domains. Of note was the higher frequency of low risk of bias for blinding of outcome assessors (51%) compared with blinding of participants (24%) and investigators (31%). The agreement for the blinding domains was fair (κ = 0.32). The pair-wise correlation for the blinding domains was moderate (τ = 0.39, participant versus investigator; τ = 0.38, participant versus outcome assessor; τ = 0.45, investigator versus outcome assessor). For other sources of bias, risk of bias was low in 118 (58%), high in 33 (16%), and unclear in 53 (26%). For sources of funding risk of bias was low in 77 (38%), high in 14 (7%), and unclear in 113 (55%). The correlation between ‘other sources of bias’ and ‘source of funding’ was weak (τ = 0.04). Researchers applying the RoB tool found it easier to assess blinding as three separate domains rather than a single item. Conclusions: This study provides evidence that risk of bias due to blinding varies depending on the targeted individuals. Risk of bias due to ‘other’ sources is different from inappropriate influence of study sponsor. B4O4 Consensus-based recommendations for investigating clinical heterogeneity in systematic reviews Gagnier JJ1 , Morgenstern H2 , Moher D3 1 Assistant Professor, Departments of Orthopaedic Surgery and Epidemiology, University of Michigan, Ann Arbor, Michigan, USA; 2 Professor, Department of Epidemiology, University of Michigan, Ann Arbor, Michigan, USA; 3 Senior Scientist, Clinical Epidemiology, Ottawa Health Research Institute, Ottawa, Ontario, Canada Background: It has been argued that systematic reviews (SRs) fail to inform clinical decision-making due to their results being far too general such that the findings do not apply to individual clinical cases. Also, few SRs investigate clinical reasons for heterogeneity and when they do often fail to use valid statistical or other methods. While there is some consensus on methods in SRs for investigating statistical and methodological heterogeneity, little attention has been paid to clinical aspects of heterogeneity. Objective: To develop recommendations for investigating clinical heterogeneity in SRs. Methods: We identified and invited potential participants with expertise in SR methodology, SR reporting, heterogeneity, statistical aspects of meta-analyses, or those who published papers on clinical heterogeneity. Three phases are being conducted: i) pre-meeting item generation, ii) face to face consensus meeting in the form of a modified Delphi process, and iii) post-meeting feedback. Results: A total of Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. 19 participants have agreed to attend the meeting that is being held 3–4 June 2011 at the University of Michigan, USA. Currently, we are performing the pre-meeting item generation that consists of phone calls with all participants, asking them to indicate important potential recommendations that need to be discussed during the face-to-face meeting. In addition, we have extracted recommendations from two recent comprehensive methodologic reviews in the area. We are collating and cross-referencing the phone call results and the extracted data from the methodologic reviews to construct items to be discussed during the face-to-face consensus meeting in June. We anticipate that by July 2011, we will have completed a manuscript that outlines the results of this consensus meeting. This manuscript will outline the first set of consensus-based guidelines for investigating clinical heterogeneity in SRs. Discussion: We expect that these guidelines for investigating clinical heterogeneity in SRs will improve the validity of such investigations resulting in SRs with more applicable results. B5 Methods for preparing reviews (statistical and non-statistical) B5O1 Intensity of a public health intervention in a systematic review: Assessing it Baker P1 , Francis D2 , Soares J3 1 School of Public Health Queensland University of Technology and Epidemiology Central Regional Services Queensland Health, Australia; 2 Epidemiology Central Regional Services Queensland Health, Australia; 3 Division of Nutrition, Physical Activity and Obesity, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, USA Background: Intensity of an intervention is not well defined in the public health literature and it may be one explanation for differences that exist in the outcomes between public health studies. Objectives: The aim of this study was to describe the approach used in a recent review about community wide interventions for increasing physical activity to categorise the intensity of interventions to assess whether this could help explain differences that existed in the outcomes reported. Methods: A subjective criterion, by which each study was rated by two independent reviewers, was developed. The criteria were based on six characteristics and attributes that we hypothesised would be important. Each characteristic was assessed as ‘more intensive’, ‘less intensive’ or ‘unclear’. It included: 1) development of community partnerships or coalition; 2) levels of intervention; 3) reach of strategies; 4) continuous provision of intervention; 5) cost; and 6) statement of intensity by the authors. The overall assessment of intensity for each study was categorised as ‘high’, ‘medium, ‘low’ or ‘unclear’. Results: The tool was simple to apply. Nine of the 25 studies included in the review were classified as being ‘high intensity’, 10 of ‘medium intensity’ and six of ‘low intensity’. Seven of the 25 studies reported some improved physical activity outcomes and the majority of these (n = 5) were classified as high intensity. Consistency between assessors will be investigated and reported. Conclusions: Interpretation of the studies in this review was challenging due to the overall poor quality of the included studies, however assessment of intensity is potentially useful for understanding heterogeneity of effectiveness between public health Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 23 studies. We recommend that intensity of a public health intervention be assessed in reviews of public health interventions where there is likely to be significant difference in the interventions and populations at which they are targeted. B5O2 The implications of different intervention definitions on a network meta-analysis of acupuncture and other physical therapies for the relief of chronic pain due to osteoarthritis of the knee Rice SJC1 , Woolacott NF1 , Corbett MS1 , Slack R1 1 University of York, UK Background: A systematic review and network meta-analysis of acupuncture and other physical therapies for the relief of chronic pain due to osteoarthritis of the knee was undertaken. There are several different physical therapies, variations of the same type of physical therapy, and combinations of therapies for osteoarthritis of the knee pain. The definition of treatment groups need to enable useful interpretation of the results for policy. Objectives: The objective of this communication is to present the implications of different intervention definitions on the network meta-analyses in this review. Methods: Given the multiple possible combinations of physical therapies for osteoarthritis of the knee, combinations of the main interventions were excluded from the analysis to make the project manageable. For one network analysis, 22 main interventions were defined and each one had the possibility of five adjunct treatments resulting in 110 possible treatment groups. For another network analysis, the five adjunct treatments were grouped into one group, leaving 22 interventions defined. For another, all physiotherapy interventions were combined into one group to reflect options available to general practitioners in the UK. Results: For the detailed network there were either no significant differences between the treatment effects of the interventions with the same principal physical therapy or the face validity was questionable. With more interventions defined, there were fewer trials per treatment comparison in the model and there was a low power to detect treatment effect differences. Results for the grouped interventions network were influenced by the new network structure and additional interventions and trials being captured within the broader treatment definition. Conclusions: In complex network meta-analyses decisions made about the definition of interventions are important. A too great an emphasis on differences between interventions results in a large network comprised of few trials per treatment comparison in the model, with low power to detect treatment effect differences. B5O3 Latent class bivariate model for the meta-analysis of diagnostic test accuracy studies Eusebi P1 , Reitsma JB2 , Vermunt JK3 1 Department of Epidemiology, Regional Health Authority of Umbria, Italy; 2 Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Center Amsterdam, Netherlands; 3 Faculty of Social and Behavioural Sciences, Department Methodology and Statistics, University of Tilburg, Netherlands Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: Several statistical methods for meta-analysis of data from diagnostic test accuracy studies have been proposed. The Bivariate Model is a rigorous method for this purpose by directly analyzing estimates of sensitivity and specificity. Objectives: Our research is motivated by a re-analysis of data of a published systematic review (1). In this meta-analysis three imaging techniques were compared for the diagnosis of lymph node metastasis in women with cervical cancer. Observing the heterogeneity amount on the data we aim at implementing new methods for helping the understanding on the relationship between sensitivity and specificity. Methods: We propose the Latent Class Bivariate Model, an extension of the Bivariate Model by means of a discrete latent variable for finding clusters of studies. Results: Several type of models were fitted with Latent GOLD software (2). The best model is the Latent Class Bivariate Model with the type of test as a nominal covariate. This model detected two latent classes of studies. Studies belonging to the first latent class show lower sensitivity but higher specificity and almost all the studies are X-ray computed tomography (CT) and magnetic resonance imaging (MRI); in that class sensitivity and specificity appear to be negatively correlated. Studies belonging to the second latent class show lower specificity but higher sensitivity and almost all the studies are lymphangiography (LAG); in that class sensitivity and specificity appear not to be correlated. Conclusions: What is added by the latent approach is that it provides an explanatory and confirmatory tool for investigating and testing different patterns of heterogeneity across studies. We tested the performance equivalence of CT and MRI and the different correlation between sensitivity and specificity in LAG and CT/MRI studies. Additional insight and data-driven hypothesis can be generated for future subgroup meta-analysis. References 1. Scheidler J, Hricak H, Yu KK, Subak L, Segal MR. Radiological evaluation of lymph node metastases in patients with cervical cancer. A meta-analysis. JAMA. 1997;278(13):1096–101. 2. Vermunt JK, Magidson J. (2008). LG-Syntax User’s Guide: Manual for Latent GOLD 4.5 Syntax Module, Belmont, MA: Statistical Innovations Inc. B5O4 A new strategy for meta-analysis of continuous covariates in observational studies Sauerbrei W1 , Royston P2 1 IMBI, University Medical Center Freiburg, Germany; 2 MRC Clinical Trials Unit, London, UK Background: Assessing the effect of a continuous covariate in a single study requires the determination of a dose-response relationship in a model adjusting for other covariates. Even with individual patient data (IPD), summarizing the results of several dose-response functions in a meta-analysis is not straightforward. Objectives: To describe a new procedure which determines in single studies an adjusted dose-response function for a continuous variable and which also summarizes such functions across studies. Methods: In single studies we determine functions using fractional polynomials, with or without adjustment for confounders. Three different methods to select the FP function are proposed. To average functions from several studies, a fixed or random effect approach can be used. Results: Using data Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 24 from the US SEER database in which each individual registry is treated as a single study, we derive adjusted overall estimates of the functional forms for the association between time to breast cancer death and a continuous covariate. Whereas the individual functions for nodes, a factor with a large effect, are similar across studies, those for age, a covariate with a weak effect, show considerable variability. Because of their different weights, whether to use a fixed or a random effects model affects the average function. Conclusions: For an ‘ideal’ situation with IPD and only minor variations between studies with respect to measurement techniques and confounders, our approach allows one to model dose-response relationships in single studies and to summarize them in an average function. Modifications are available if the data situation is less than ideal. Figure 1 (B601). Impact of future study on meta-analysis of trials for obstructive sleep apnoea. Oral appliance versus continuous positive airways pressure; outcome difference in Epworth score. B6 Statistical methods B6O1 Graphical augmentations to the funnel plot for assessing the impact of additional evidence on a meta-analysis Langan D1 , Higgins J2 , Gregory W3 , Crowther M4 , Sutton A4 1 Clinical Trials Research Unit, University of Leeds, UK; 2 MRC Biostatistics Unit, Cambridge, UK; 3 Clinical Trials Research Unit, University of Leeds, UK; 4 University of Leicester, UK Background: Methods to quantify the potential impact of future evidence on a systematic review are currently under-developed but would greatly aid decisions on both review updating priorities and recommendations for future research. Objectives: Develop novel overlays to the funnel plot to provide a visual illustration of the impact that a new study would have on a given meta-analysis. The additional features help inform: i) the current robustness of a meta-analysis; ii) sample size calculations for the design of future studies to be added to the meta-analysis; and iii) the update prioritisation strategy for a portfolio of meta-analyses (such as those managed by Cochrane Review Groups). Description of developed methodology: Several inter-related overlays to the funnel plot are described and illustrated using data from Cochrane reviews. These include: i) statistical significance contours, which define regions of the funnel plot in which a new study would have to be located in order to change the statistical significance of the meta-analysis (an example of these for a sleep apnoea review is included in the figure); and ii) heterogeneity contours, which show how a new study would affect the extent of heterogeneity in a given meta-analysis. The use of the aforementioned overlays simultaneously with further features including: pooled treatment effects, lines of no effect, confidence and prediction intervals, simulated new studies and sample size guidelines, are all discussed. Details of free software for creating the plots in both R and STATA are provided. Suggestions for how review editors, review authors and trial designers could use the plots will be considered. Conclusions: The authors believe the developed enhancements to the funnel plot provide a lot of accessible information about the impact of future evidence on an existing evidence base. It is hoped editors, systematic reviewer authors and trialists will find such plots useful. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. B6O2 Bayesian approach to meta-analysis. What can you gain? Walczak J1 , Borowiack E2 , Nikodem M1 , Siedmiogrodzki K1 , Zapalska A2 , Khan K3 , Meads C3 , Mol B4 , Oude Rengerink K4 , Thangaratinam S3 , Zamora J5 1 CASPolska Association, Poland; 2 Arcana Institute, Poland; 3 Queen Mary, University of London, Great Britain; 4 Academic Medical Center Amsterdam, Netherlands; 5 Hospital Universitario Ramon y Cajal, Madrid, Spain Background: The Bayesian approach gives new opportunities such as aggregating different types of data, combining direct and indirect comparison or assessing clinical significance. On the other hand, the variety of Bayesian models can beconfusing, and implementation difficulties can cause unwillingness to apply it. Objectives: The aim is to systemize the knowledge of application of Bayesian statistics in the area of meta-analysis, and to compare it with traditional statistical methods. We want to identify the situations in which use of the Bayesian approach is really worthwhile. Methods: Initially the systematic reviews were conducted using existing statistical methods for meta-analyses. Special attention was given to different Bayesian models which were then implemented in the WinBUGS environment and examined on different data sets. Results: In the case of regular meta-analysis of dichotomous data, applying basic Bayesian models leads us, in fact, to similar results of estimation as the Mantel-Haenszel or DerSimonian-Laird method. The real advantage of the Bayesian approach is noticed if we expect something more than typical meta-analysis, especially if we have to deal with the following problems: 1. Assessing the clinical significance – for instance, assessing the chance that relative risk is greater than 1.25 (or any other level of significance). 2. Combining data from different type of studies Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 25 -including extra information (e.g. results of non-randomized studies) to meta-analysis, keeping a moderate ‘level of conviction’ to this extra data. 3. Combining direct and indirect evidence (Mixed Treatment Comparison). Conclusions: Bayesian statistics give us technical opportunities to improve meta-analysis, especially in the area of aggregating multi-type data. On the other hand, there is no significant difference between the results obtained by the Bayesian and traditional approach in the case of simple meta-analysis of regular data. Moreover, if WinBUGS codes are prepared, conducting the calculations is not as difficult as one may think. B6O3 A Bayesian selection model for incorporating prior information for publication bias in meta-analysis Mavridis D1 , Cipriani A2 , Sutton A3 , Salanti G1 1 Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece; 2 Department of Public Health and Community Medicine, Section of Psychiatry and Clinical Psychology, University of Verona, Policlinico ‘G.B. Rossi’, Italy; 3 Department of Health Sciences University of Leicester, UK Background: The Copas parametric model is aimed at exploring the potential impact of publication bias via sensitivity analysis, by making assumptions regarding the probability of publication of individual studies related to the standard error of their effect sizes (Copas and Shi). Review authors often have prior knowledge about the extent of selection in the set of studies included in a meta-analysis. However, a Bayesian implementation of the Copas model has not been studied yet. Objectives: To present a Bayesian selection model for publication bias and to extend it to the case of multiple-treatments meta-analysis. Methods: We explored various models corresponding to the shape of the association between precision and probability of publication. We used a dataset of antidepressant studies submitted to the US regulatory agency (Food and Drug Administration), according to Turner and colleagues (Turner et al.). We took advantage of the greater flexibility offered in the Bayesian context to incorporate in the model prior information on the extent and strength of selection. To derive prior distributions, we used both external data and an elicitation process of expert opinion. Results: The Copas model assumes a likelihood approach which requires strong assumptions for the model to be identified and a sensitivity analysis is employed. Bayesian implementation using informative priors provides a solution to this problem by explaining adequately the selection process and giving clear-cut results. Extension of the model for multiple-treatments meta-analysis allows incorporation of comparison-specific priors beliefs and thus allowed a flexible class of models to be fit. Conclusions: A Bayesian framework utilizing expert opinion provided a powerful and easily understood approach. References Copas JB, Shi JQ. A sensitivity analysis for publication bias in systematic reviews. Stat Methods Med Res. 2001;10(4):251–65. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3):252–60. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. B6O4 The implications for meta-analysis of different methods of analysing count data Herbison P1 1 University of Otago, New Zealand Background: In randomised trials with count outcomes (such as number of falls/exacerbations of asthma) it is possible to analyse the results in many different ways. It is not known whether it is possible to combine the results of these different methods of analysis in a meta-analysis. Objectives: To see if different methods of analysis provide results that are alike enough to combine in a meta-analysis. Methods: Simulation study with count data with a range of means and amount of overdispersion. Different analyses were rate ratios, poisson regression, negative binomial regression, dichotomising the data (with and without the event), three forms of survival analyses, ratio of means and ratio of medians. Confirmation by the analysis of individual patient data from at least 14 randomised trials. Results: With a low mean, (about 20% have the event) most methods of analysis produced similar answers, except ratio of medians, which was not possible to calculate. As the mean increased, dichotomising the data increasingly underestimated the treatment effect, as did time to first event and this effect was noticeable even when 50% had the event. Survival models allowing for multiple events helped, but not completely. Negative binomial gave results very similar to poisson regression, even for considerable amounts of overdispersion, where these models were better fits to the data. These results were confirmed by the analyses of the individual patient data. Conclusion: It should be possible to combine more studies in a meta-analysis than was previously expected. Relative risks from dichotomised data and hazard ratios from time to first event analyses should only be included if the event rate is very low. Means can be converted to ratios and an approximate standard deviation calculated, but for the ratio of medians it is not possible to work out the standard deviation from published data. B7 Investigating bias B7O1 A descriptive analysis of risk of bias in neonatal trials and the association with effect estimates Bialy L1 , Lacaze-Masmonteil T2 , Dryden DM1 , Ha C1 , Armijo-Olivo S1 , Vandermeer B1 , Hartling L1 1 Alberta Research Centre for Health Evidence, Canada; 2 Children’s Hospital of Eastern Ontario, Canada Background: Research that quantifies the impact of different biases on effect sizes in neonatal randomized controlled trials (RCTs) has shown conflicting results. A meta-epidemiological study to quantify bias in neonatal RCTs will inform the design, conduct, and interpretation of research in this vulnerable population. Objectives: To describe a sample of neonatal RCTs in terms of methodology, risk of bias, and associations with effect estimates. Methods: We included all neonatal RCTs (n = 208) in the Cochrane Database of Systematic Reviews that examined the following key treatments: surfactant, corticosteroids, Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 26 nitric oxide, indomethacin, ibuprofen, and head or total body cooling. Risk of bias was assessed on nine domains by two independent reviewers. Meta-epidemiological methods will be used to quantify the association between pre-specified methodological characteristics and effect estimates. Results: The assessed RCTs were published between 1972 and 2009 with 52% conducted across multiple centres. A single or composite primary outcome was reported in 49%. Trials were supported by grants from academic institutions or governments (37%) or pharmaceutical industry (26%); 37% did not report a funding source. No RCTs had an overall ‘low’ risk of bias assessment; 42% were assessed as high risk of bias and 58% were unclear. For allocation concealment and blinding of participants and investigators most RCTs received an unclear risk of bias (69%, 74%, and 64%, respectively). Nineteen percent were assigned a high risk for selectively reporting outcomes and 60% were unclear. Analyses of association between risk of bias and effect estimates are ongoing. Conclusions: We have described the risk of bias for RCTs in 6 key areas of neonatal medicine. This body of literature shows methodological limitations in terms of selective reporting and documentation of blinding methods. It is critical to understand and quantify the potential impact of these biases on effect estimates in order to better ensure the delivery of optimal neonatal care. B7O2 Single-center trials show larger treatment effects than multicenter trials: evidence from a meta-epidemiological study Dechartres A1 , Boutron I2 , Trinquart L3 , Charles P4 , Ravaud P2 1 INSERM, Universite Paris Descartes, Hotel-Dieu Hospital (APHP), France; 2 French Cochrane Centre, INSERM, Universite Paris Descartes, Hotel-Dieu Hospital (APHP), France; 3 French Cochrane Centre, France; 4 Cochin Hospital (APHP), Universite Paris Descartes, France Background: For many physicians, multicenter randomized controlled trials (RCTs) are considered more valid than single-center RCTs but there is no formal evidence about this fact. Objectives: We performed a meta-epidemiological study to evaluate whether treatment effect estimates differ between single-center and multicenter RCTs. Methods: PubMed was searched for meta-analyses of RCTs with binary outcomes published between August 2008 and January 2009 and in the first 6 months of 2010 in the 10 leading journals of each medical specialty. One issue of the Cochrane Database of Systematic Reviews was also searched. All individual RCTs included in the meta-analyses were selected and searched. Data were extracted and quality assessed by use of the Risk of Bias (RoB) tool of The Cochrane Collaboration. The primary outcome was the ratio of odds ratio (ROR) quantifying the difference in estimated intervention effect between single-center and multicenter RCTs. A ROR < 1 would indicate larger estimates of the intervention effect in single-center trials. We performed sensitivity analyses with adjustment for sample size, risk of bias within RCTs and variance of the log OR to take into account publication bias. Results: We selected 48 meta-analyses including 421 RCTs: 223 were single-center and 198 multicenter RCTs. Single-center RCTs showed larger intervention effect than multicenter RCTs (combined ROR = 0.73, 95% CI: 0.64 to 0.83), with low heterogeneity across individual meta-analyses (I2 = 12.0%, p = 0.24). Adjustment for Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. sample size yielded consistent results (ROR = 0.85, 95% CI: 0.74 to 0.97), as did adjustment for ROB within RCTs, such as allocation concealment (ROR = 0.76, 95% CI: 0.67 to 0.86) and variance of log OR (ROR = 0.83, 95% CI: 0.72 to 0.96). Conclusions: Single-center RCTs showed larger treatment effects than multicenter RCTs, which was consistent in all sensitivity analyses. Our results suggest taking this item into consideration when interpreting the results of RCTs and meta-analyses. B703 PROSPERO: An international prospective register of systematic review protocols Booth A1 , Clarke M2 , Ghersi D3 , Moher D4 , Petticrew M5 , Stewart L1 1 Centre for Reviews and Dissemination, UK; 2 UK Cochrane Centre, UK; 3 International Clinical Trials Registry Platform, WHO, Switzerland; 4 Clinical Epidemiology Program, Ottawa Hospital Research Institute, Canada; 5 Department of Scoial and Environmental Health Research, London School of Hygiene and Tropical Medicine, UK Background: Support has grown for the prospective registration of protocols for systematic reviews to encourage transparency in conduct and reporting. Historically, systematic review protocol registers were limited to organisations such as The Cochrane and Campbell Collaborations. Following publication of the PRISMA statement, the UK Centre for Reviews and Dissemination (CRD) took the initiative to develop an international prospective register of systematic reviews with health-related outcomes. Methods: An international consultation was carried out to identify the key items for inclusion in the register. These were divided into required and optional items. A two-round electronic modified Delphi survey design was used. The panel was comprised of relevant experts, including commissioners, clinical and academic researchers, methodologists, statisticians, information specialists, publishers and users of systematic review evidence. Results: In February 2011, CRD launched PROSPERO, an international register of systematic reviews at the protocol stage. The required dataset, established through the consultation, consists of 22 required items and 18 optional items. The dataset captures the key attributes of review design as well as the administrative details necessary for registration. PROSPERO offers free registration and free public access. The dedicated web-based interface is electronically searchable and open to all prospective registrants. Cochrane protocols do not need to be registered individually as automatic uploading will be developed. Initially inclusion is restricted to systematic reviews of the effects of interventions and strategies to prevent, diagnose, treat, and monitor health conditions, for which there is a health-related outcome. The long term aim is to widen the scope to include all ongoing systematic reviews that have a health related outcome. Conclusions: PROSPERO should provide a way of assessing the risk of bias in systematic reviews, support the efficient use of funding and timely updating, and contribute to improving the quality of systematic reviews and the decisions that rely upon them. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 27 B7O4 Characteristics of physicians who receive large payments from industry and the accuracy of their disclosures in concurrent publications Norris SL1 , Holmer HK1 , Burda BU2 , Ogden LA1 , McAteer AM1 1 Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, USA; 2 Oregon Evidence-Based Practice Center, Kaiser Permanente Center for Health Research, USA Background: Authors’ financial conflicts of interest (COI) may bias the results of primary studies and therefore systematic reviews. There are few data, however, on the accuracy of COI disclosures in publications of primary research. Objective: To examine the accuracy of disclosures by individuals known to have received money from drug companies. Methods: ProPublica, an investigative journalism organization, identified 384 individuals (mainly physicians) who received more than $100,000 from one or more of seven major pharmaceutical companies for consultancies and other peer educational activities over varying periods between January 2009 and September 2010. We explored the demographic characteristics and clinical specialties of these individuals using several public databases, as well as disclosures in their concurrent publications identified in MEDLINE. Results: Of the 384 individuals, 373 were physicians, including psychiatrists (32%), internists (31%), and a variety of other clinical specialties. Seventeen physicians had received state disciplinary actions. The mean age of the physicians was 53.5 years and 11% were female. The median total payment was $126,724 (range, $100,047 to $303,558). Only 159 individuals had published in the relevant time period, authoring more than 700 publications. Of these, greater than half contained disclosures. Of publications with disclosures, approximately 15% had discrepancies between the disclosures and information provided by ProPublica. We will present additional data on the nature of the discrepancies. Conclusions: US medical providers who receive large sums of money from the pharmaceutical industry for speaking and other non-research engagements frequently do not report this information in concurrent publications. The systematic reviewer thus has incomplete information upon which to evaluate the validity of primary studies. Our conclusions are limited by the small number of pharmaceutical companies providing payment information to the public. B8 Investigating bias B8O1 Can trial quality be reliably assessed from a trial publication? Vale C1 , Tierney J1 , Burdett S1 1 Meta-analysis Group, MRC Clinical Trials Unit, London, UK Background: Assessment of risk of bias (RoB) using information from publications of randomised controlled trials (RCTs) is common practice for systematic reviews. It has been proposed as a basis for adjusting the results of meta-analyses to account for trials assessed at high risk. Objectives: We aimed to assess whether RoB assessments from trial reports are comparable to those using supplementary information Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. obtained from trialists. Methods: We sampled published RCTs included in our meta-analyses of individual participant data (IPDMA) for which trial protocols or forms detailing trial design were available. RoB was assessed using criteria outlined in the Cochrane Handbook and the Cochrane Risk of Bias tool (RevMan 5.1). Assessments were conducted for each trial based on trial reports only, or trial reports plus supplementary information from trialists, and were compared for consistency within domains and overall. Results: Preliminary results are based on 25 RCTs from 3 IPDMAs in cancer. Using trial reports, the RoB was judged to be low (14 trials) or unclear (11 trials) for sequence generation and low (10 trials) or unclear (15 trials) for allocation concealment. However, supplementary information resulted in low RoB assessments for all trials for sequence generation and for 24 trials for allocation concealment. Selective reporting bias was judged to be high (14 trials) or unclear (10 trials) from publications alone, however supplementary data led to 24 trials being assessed as low RoB. Conclusions: Supplementary information from trialists can alter high or unclear judgements of bias, particularly in regard to selective reporting of outcomes. Our results suggest review authors should be cautious in adjusting the results of meta-analyses based on RoB assessments from publications, particularly where assessments are unclear. Results will be updated to include 7 additional IPDMAs (76 RCTs) and the randomisation integrity will be further explored using IPD. B8O2 Inter-rater reliability of a new instrument for assessing potential for bias in prognosis studies Peterson K1 , Carson S1 , Carney N1 1 Oregon Evidence-based Practice Center, Oregon Health & Science University, USA Background: There is no gold standard for assessing the potential for bias in prognosis studies. The best available recommendations come from a 2006 ‘review of reviews’. However, very little formal testing has been done on applying those recommendations. Objectives: To assess inter-rater reliability of an instrument for assessing potential for bias in prognosis studies. Methods: We developed an instrument for assessing the potential for bias in prognosis studies based on the recommendations of Hayden et al. Our instrument consists of 24 items across 6 domains (patient selection methods, prognostic factor measurement, outcome measurement, follow-up, analysis and reporting methods, and measurement of confounders). The potential for bias was rated as low, medium, or high for each domain and for each study overall. Two reviewers independently assessed 37 studies included in a systematic review conducted for the Brain Trauma Foundation’s second edition of Early Indicators of Prognosis for Moderate to Severe Traumatic Brain Injury. Inter-rater reliability was estimated using Cohen’s kappa. Results: Inter-rater reliability was moderate for the overall potential for bias (weighted Cohen’s kappa: 0.78; 95% Confidence Interval (CI) 0.52 to 1.05). Although observed agreement was generally high across the individual domains (range 74% to 93%), kappa values ranged widely. For example, kappa values indicated ‘less than chance’ reliability for the outcome measurement domain (−0.03; 95% CI −0.31 to −0.26), but ‘almost perfect’ reliability for the analysis and reporting methods domain (0.83; 95% CI 0.52 to 1.14). Kappa values were lowest for domains having Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 28 distributions that were skewed toward ratings of low to medium potential for bias. Conclusions: Our instrument had moderate inter-rater reliability for assessing overall potential for bias in prognosis studies. Before our instrument can be used with confidence, however, more reliability and validity testing is needed on studies representing broader ranges of validity and clinical topic areas. B8O3 Risk of Bias Assessment tool for Non-randomized Studies (RoBANS): Development and validation of a new instrument Park J1 , Lee Y1 , Seo H2 , Jang B1 , Son H1 , Kim SY3 , Shin S4 , Hahn S5 1 Department of Health Technology Assessment Research, National Evidence-based Healthcare Collaborating Agency, Seoul, Korea; 2 Department of Nursing, College of Medicine, Chosun University, Gwangju, Korea; 3 Department of Family Medicine, College of Medicine, Hallym University, Seoul, Korea; 4 Department of Pulmonary Medicine, Ajou University, Suwon, Korea; 5 Department of Medicine, College of Medicine, Seoul National University, Seoul, Korea Background: Few validated checklist tools are available to assess the risk of bias (RoB) of non-randomized studies (NRS), although systematic reviews including NRS have increased in number, for many reasons. Objectives: The aims of the study are to develop and validate a new RoB tool for NRS. Methods: After deciding principles and reviewing relevant tools, we developed the Risk of Bias Assessment tool for Non-randomized Studies (RoBANS) tool. Subsequent testing involved the assessment of inter-rater agreement between reviewers, time to apply the RoBANS tool, degree of correlation for overall risk compared with overall quality scores, user-convenience, and content validity. Results: The RoBANS contains 6 domains including the selection of participants, confounding variables, measurement of intervention (exposure), blinding of outcome assessment, incomplete outcome data and selective outcome reporting (table). Inter-rater agreement of the RoBANS tool varied in individual domains (weighted κ = 0.38 to 0.74). The mean time to complete the RoBANS tool was less than for the MINORS (Methodological Items for NOn-Randomized Studies) (9.5 minutes (SD 3.39) per study vs. 10.45 (SD 3.54), P = 0.18). There was a moderate correlation between overall RoB compared with the MINORS (Kendall’s τ = 0.384). Reviewers evaluated RoBANS as ‘good’ in terms of user-convenience for 3 items (mean 5.7 of 7 as the Likert scale; 0.81 %). External experts (n = 8) evaluated RoBANS as ‘fair’ in terms of face validity for 7 items (mean 5.4 of 7 as the Likert scale) and all recommended its use to assess the RoB for NRS. Conclusions: RoBANS is a valid tool designed to assess the RoB of NRS. Since RoBANS is harmonized with the Cochrane’s RoB tool and GRADE (Grading of Recommendations Assessment, Development and Evaluation), and can be incorporated into RevMan and GRADEpro, it appears to be useful to people undertaking systematic reviews. B8O4 Initial experience with a pilot Cochrane tool for assessing risk of bias for non-randomized studies applying a web-based survey of content experts to derive criteria for imbalance Palmer DA1 , MacLennan SJ2 , Imamura M2 , Reeves BC3 , Clubb A1 , Duboy AJ1 , Neuberger MM1 , Dahm P1 1 Department of Urology, College of Medicine, University of Florida, Gainesville, Florida, USA; 2 Academic Urology Unit, University of Aberdeen, Aberdeen, UK; 3 Clinical Trials and Evaluation Unit, University of Bristol, Bristol, UK Background: To date, few tools exist for assessing the risk of bias (RoB) from confounding variables in non-randomized studies (NRS). Objectives: We tested a pilot Cochrane tool for RoB to assess its applicability and value in quantifying RoB in NRS. Methods: We applied the Cochrane tool to a systematic review of NRS evaluating the effectiveness of lymph node dissection in patients undergoing radical Table 1 (B803). The developed and validated version of RoBANS. Domain Description Risk of bias Selection of participants Selection bias caused by inadequate selection of participants Confounding variables Selection bias caused by inadequate confirmation and consideration of confounding variable Intervention (exposure) measurement Performance bias caused by inadequate measurements of intervention (exposure) Blinding of outcome assessment Detection bias caused by inadequate blinding of outcome assessment Incomplete outcome data Attrition bias caused by inadequate handling of incomplete outcome data Selective outcome reporting Reporting bias caused by selective outcome reporting Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Low High Unclear Low High Unclear Low High Unclear Low High Unclear Low High Unclear Low High Unclear Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 29 cystectomy. Three independent review authors abstracted data on methodological quality using the Newcastle-Ottawa instrument and the Cochrane tool for assessing RoB.[1] The Cochrane tool assesses the precision, imbalance, and adjustment for each confounding variable, and overall RoB from confounding, blinding, incomplete outcome data and selective reporting, on 1 (low ROB) to 5 (high RoB) point scales. We pilot-tested the data abstraction form and developed detailed guidelines to optimize inter-observer agreement. Results: Five NRS met the inclusion criteria. Confounders were identified in advance by literature search and responses to a web-based survey of eleven bladder cancer experts who were also asked to categorize what difference in the proportion of a confounding variable represented ‘no’, a ‘small’, ‘moderate’ or ‘large’ imbalance. To evaluate imbalance, each confounder was evaluated dichotomously (e.g. organ-confined versus non-organ-confined). Threshold values were derived using the median responses; e.g. for non-organ-confined disease, these were set at less than or equal to 2%, 2.1 to 5%, 5.1 to 7%, greater than 7%, respectively, thereby providing explicit anchors for imbalance criterion scoring. Conclusions: The novel RoB tool represents a valuable instrument for quantifying the RoB of NRS that deserves further formal investigation in a larger, more diverse sample of studies. The described survey method appears an effective and efficient method to define the necessary imbalance criterion thresholds. Results: We have co-authored one published SR, one protocol, one peer reviewed manuscript and are currently in the process of updating one SR and authoring two others. One poster was presented at the 2010 Cochrane Colloquium and an editorial published on The Cochrane library. Within the ambit of programme development, we have adapted a SR module for amalgamation within the University of the West Indies MSc Epidemiology and delivered a module to one cohort of students. We made links with the Pan American Health Organization, The International Initiative for Impact Evaluations and several Cochrane entities. We aim to achieve a mass of six trained individuals by conducting SR workshops and collaborating with INCLEN partners to host fellowships and establishing networks to help sustain activity. Conclusions: This experience responds to a growing and unique need for evidence based healthcare in developing countries. Capacity building activities such as fellowships and regional training workshops are viable avenues for developing the required cadre for the Caribbean region. The concept and execution could serve as a model for the developing world. Reference Struthers C1 , Noel-Storr A1 , Marcus S1 , McCleery J1 , McShane R1 1 Cochrane Dementia and Cognitive Improvement Group, UK 1. Reeves BC, Shea B, Wells GA. Classifying non-randomised studies (NRS) and the assessing the risk of bias for a systematic review. Workshop, 18th Cochrane Colloquium, Keystone 2010. C1 Education and training C1O1 Expanding the Cochrane Collaboration in low and middle income countries: A Caribbean experience Francis D1 , Pardo Pardo J2 , Benkhalti Jandu M2 , Tulloch-Reid M1 , Wilks R1 , Tugwell P2 1 Epidemiology Research Unit, University of the West Indies Mona, Jamaica; 2 Center for Global Health, University of Ottawa, Canada Background: The Caribbean region requires a cadre of trained health scientists to review existing research and provide evidence based recommendations to address regional needs. We report on efforts to establish a Cochrane branch through our membership within the International Clinical Epidemiology Network (INCLEN) to address this deficiency. Methods: A fellowship training programme facilitated through a scholarship grant from the Canadian Bureau of International Education and an INCLEN partner was used to train a suitable candidate for this process. Training was carried out at the Center for Global Health, University of Ottawa over a 4 to 6 month period and focused on systematic review (SR) author training, programme development in evidence based medicine and capacity building and networking. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. C1O2 Feasibility of recruitment and training of carers of people with dementia as a route to Cochrane-NHS engagement Background: Relatives of people with dementia often want to participate in research, but their time is limited because of their caregiving role. Objectives: The first, innovative part of the proposal is to recruit carers and former carers of people with dementia to a well-defined task: reading trial reports, extracting basic information, and entering it into the Dementia Group’s online register of studies – ALOIS. The volunteers are also invited to work through a series of modules about EBM using examples, all directly related to dementia. Methods: A range of publicity and outreach methods were used to recruit volunteers (see leaflet attached). Also an EBM skills workshop for carers was held in Oxford which was very well attended. We will conduct a before-and-after study where the participants’ skill at the coding task will be rated, and a questionnaire will be used to assess changes in involvement with dementia-related NHS activities, primary dementia research, and attitudes. Results: At the project half-way point (see poster attached), we had recruited 35 volunteers (20 carers or former carers). We have now reached 40, and this has translated into 65 new study records being coded by new volunteers and published on ALOIS, and a further 70 studies currently assigned to volunteers for coding. A slightly unexpected but welcome development is that schools are now interested in participating, and we are currently looking at how adapt the model for this purpose. We are also hopeful that the model might be further adapted as a Cochrane-wide ‘citizen science’ initiative in conjunction with the development of the Cochrane Register of Studies. Conclusion: Involving and training volunteer ‘expert carers’ in maintaining a trial register is feasible. We will explore whether the model can be adapted to bring wider benefits for public engagement with the NHS and participation in primary research. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 30 Figure 1 (C102). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 31 Figure 2 (C102). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 32 C1O3 A pilot study of an evidence-based practice training program for Filipino physiotherapists: emerging evidence on outcomes and acceptability Dizon JM1 , Grimmer-Somers K2 , Kumar S2 1 International Centre for Allied Health Evidence, University of South Australia, Australia; and University of Santo Tomas, Philippines; 2 International Centre for Allied Health Evidence, University of South Australia, Australia Background: Evidence-based practice (EBP) has been challenging for health professionals across the globe. Issues of implementing the evidence become even more challenging among health professionals in developing countries, due to cultural and contextual issues which compound the existing barriers reported in the literature. An EBP training (EBP for FilPTs) was developed for physiotherapists in a developing country (Philippines) based on the literature and local context exploratory studies. Objectives: This study reports the findings of pilot testing the EBP for FilPTs (Table 1). The study objectives are to: (1) Estimate the effect of the EBP training program Table 1 (C103). Outline of the training Schedule Topic Strategy Before the scheduled training 08:30 – 09:00 09:00 – 09:45 09:45 – 10:00 10:00 – 10:30 Baseline knowledge and skills assessment Introduction to EBP Hierarchy of evidence and study designs Break PICO Formulation Adapted Fresno test Questions on attitudes to EBP Lecture Lecture Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Participant’s Output Lecture on drafting PICO Practical session on: a. Drafting PICO from a case b. Drafting own PICO Own PICO Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 33 Continued Schedule Topic Strategy Participant’s Output 10:30 – 11:15 11:15 – 12:00 12:00 – 01:30 01:30 – 03:00 Designing the search Conducting the search Lunch Appraising the evidence Lecture -demo Practical session Own search strategy Acquire evidence Lecture Practical session Appraised paper 03:00 – 03:15 03:15 – 04:30 Break What does the evidence say and how applicable is it? How can it be applied? EBP Checklist Introduction to the online support Post program knowledge and skills assessment Lecture Practical session Answer to own PICO and completion of the EBP checklist 04:30 – 04:45 04:45 – 05:00 Demonstration Adapted Fresno test, Questions on attitudes to EBP and answer evaluation questions Table 2 (C103). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 34 in enhancing knowledge and skills in EBP; (2) Determine the sample size for a subsequent RCT; and (3) Test the acceptability of the EBP training program. Methods: Physiotherapists were randomly allocated to the intervention (EBP for FilPTs) and the waitlist control groups. An EBP Checklist (Table 2) specifically developed for the use of Filipino physiotherapists to apply the findings, was introduced to the intervention group. Pre-post intervention measures of knowledge and skills in EBP were assessed using the Adapted Fresno test. Participants and observers completed an evaluation summary and commented on the program’s objectives and strategies, and its acceptability in the context of practice. Results: The EBP for FilPTs resulted in significant and large improvement (Effect Size = 0.8) in knowledge and skills. A sample of at least 54 physiotherapists is required for the randomised controlled trial (RCT). Minor changes were required based on the evaluation. Overall, the EBP for FilPTs is a useful and acceptable training for physiotherapists in the Philippines. Conclusions: The EBP for FilPTs is considered to be an effective and acceptable medium to teach EBP to physiotherapists in the Philippines. Currently, the RCT is being conducted to assess the effectiveness of the EBP for FilPTs in improving knowledge, skills, attitudes and behaviour in EBP. C1O4 Evidence-based medicine training: Reaching the front line clinician in an integrated medical care system Ching K1 , Regidor D2 , Okawa G1 1 Hawaii Permanente Medical Group, US; Management Institute, US 2 Kaiser Permanente Care Background: A shared language surrounding assessment of medical literature and translation into evidence-based practice (EBP) is central to providing high-quality, coordinated, outcomes-based medical care. There are few structured evidence-based medicine (EBM) training programs targeting front line clinicians in a community setting. A large regional medical group (more than 400 physicians) embraced a program promoting a system-wide culture of EBM. EBM foundations training was developed to provide reinforcement through hands-on experience and system-wide dissemination through a regional grand rounds program. Objective: To create a culture of evidence-based practice in a single healthcare system through integration of an evidence-based foundations training course into a regional grand rounds program. Methods: A four-hour interactive training session targets the principles of EBM and EBP during grand rounds content development. Further EBM support is provided at follow-up sessions. In the grand rounds program, presenters must provide a rationale for key clinical recommendations by synthesizing the best available evidence using standard formats for evidence and recommendation grading. Physician clinics close for grand rounds, consistent language for medical evidence is used system-wide, and intranet posting reaches about 2000 healthcare professionals. Results: Since 2009, there have been 51 course participants, including physicians and allied health professionals. Grand rounds attendance averages 57% of the medical group. To date, over 68 graded recommendations were made across 21 clinical topics. Course evaluations show high satisfaction and achieved course objectives. Grand rounds participant evaluations demonstrate clarity of the evidence base for clinical recommendations (96% positive Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. response). Improved outcome measures from specific recommendations can be demonstrated as part of a broader, multi-faceted process, of which EBP is an important component. Conclusion: A structured, interactive EBP training program integrated into a regional grand rounds program can successfully reach front line clinicians, and advance evidence-based culture throughout a healthcare system. C2 Global health and equity C2O1 Lack of equity reporting in Cochrane reviews of effectiveness in chronic obstructive pulmonary disease Ospina M1 , Armijo-Olivo S2 , Bond K3 , Rowe B4 1 University of Alberta School of Public Health and Institute of Health Economics, Canada; 2 Alberta Research Centre for Health Evidence and Department of Pediatrics, University of Alberta, Canada; 3 Institute of Health Economics, Canada; 4 Alberta Research Centre for Health Evidence and Department of Emergency Medicine, University of Alberta, Canada Background: The importance of socioeconomic gradients in determining health status and treatment response is recognized worldwide. Hence, it is crucial to understand how Cochrane reviews account for potential variations in the distribution of harms and benefits, especially for conditions in which large socioeconomic disparities are documented, such as chronic obstructive pulmonary disease (COPD). Objectives: To assess to what extent Cochrane reviews for COPD describe individual study baseline data relevant to equity and incorporate these factors into the effectiveness analysis. Methods: Electronic searches of The Cochrane Library (Issue 4, 2011) were conducted to identify reviews of effectiveness in COPD. Baseline and study outcome data reported in the reviews were classified using the PROGRESS-Plus framework, which describes factors relevant to assessing health inequities. Results: Forty-three Cochrane COPD reviews were identified (26 pharmacological; 17 non-pharmacological). Eighty-four percent provided individual study baseline data by at least one PROGRESS-Plus factor: age (81.3%), sex (72%), place of residence, occupation, marital status and social capital (2.3% each; all in non-pharmacological reviews). Pharmacological reviews were not more likely to report equity-relevant baseline data compared to non-pharmacological reviews (odds ratio 0.2 (95% confidence interval 0.034 to 1.18)). Other equity factors (education and social exclusion) were described in the discussion of three non-pharmacological reviews but not analyzed. Three pharmacological reviews planned effectiveness subgroup analysis by age; however, none of the reviews incorporated any of the PROGRESS-Plus factors in the analysis of effectiveness. Conclusions: Equity is seldom addressed in Cochrane COPD reviews of effectiveness. While the characteristics of age and sex are often used to describe study participants, these and other potential equity factors are consistently excluded from the analysis of therapeutic effectiveness. There is an urgent need to better synthesize the evidence of how the effectiveness of chronic disease treatments may vary according to socioeconomic disparities, especially in COPD. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 35 C2O2 Synthesizing health systems evidence: The contribution of a network based on North-South collaboration Koehlmoos TP1 , Pantoja T2 , Kiwanuka S3 , Meng Q4 , Oxman A5 , Oliver S6 , Garner P7 , Bennett S8 , Ransom MK9 1 ICDDR, B, Bangladesh; 2 Escuela de Medicina, Pontificia Universidad Católica de Chile; 3 School of Public Health, Makerere University, Uganda; 4 Center for Health Management and Policy, Shandong University, China; 5 EPOC satellite for low and middle income countries, Norwegian Knowledge Centre for the Health Services; 6 EPPI-Centre, Institute of Education, UK; 7 Liverpool School of Tropical Medicine, UK; 8 Bloomberg School of Public Health, Johns Hopkins University, USA; 9 Alliance for Health Policy & Systems Research, World Health Organization, Switzerland Background: In 2007, a dearth of systematic reviews focused on, and review teams who were capable of, synthesizing health systems evidence to improve health and decision making in lowand middle-income countries (LMICs) led the Alliance for Health Policy & Systems Research (HPSR) to create a network of Centres for Systematic Review (CSRs) focusing on health finance, human resources, the non-state sector, and the advancement of HPSR synthesis methodology. Methods: After competitive bids, CSRs were established in Bangladesh, China, Chile and Uganda. Support for capacity building was provided by the Cochrane Effective Practice and Organization of Care Group, the EPPI-Centre and the Cochrane Infectious Diseases Group. The CSRs participated in two workshops, multiple individual training activities, and long-distance mentoring. Results: In four years the network produced more than twelve systematic reviews on priority topics including Cochrane reviews on dual practice, social franchising, and community health insurance, and more than ten presentations at high-profile international fora. Teams identified methodological challenges in the field and collected resources to address them. CSR members contribute to the global review community by serving as editors within the Cochrane and Campbell Collaborations and on The Cochrane Library Oversight Committee. They facilitate capacity building and extensive engagement with policy makers in their home countries and regions. The Pan-American Health Organization (PAHO) recognized one CSR-produced Cochrane protocol on emigration policy as the most policy-relevant protocol for the Health Agenda for the Americas in 2010. Conclusions: LMIC-based review teams mastered an array of review skills but more importantly pioneered new methodology, expanded capacity building networks, facilitated knowledge translation within their settings and contributed actively to the global synthesis movement. Maintaining this capacity requires sustainable funding for individuals and organizations but also investment in information, communication and technology to support partnerships and rapid access to synthesized evidence. C2O3 Do we need a PRISMA-Equity extension for systematic reviews? Welch V1 , Tugwell P1 , Petticrew M2 , Moher D1 , Ueffing E1 , Pardo J1 1 University of Ottawa, Canada; 2 London School of Hygiene & Tropical Medicine, UK Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: Equity-focused reviews are increasingly promoted and published by groups such as the Cochrane Public Health Review Group, the Cochrane Effective Practice and Organization of Care Review Group, the HIV/AIDS Review Group and the Campbell International Development Group. These reviews may have specific methods, participants or settings that need to be reported transparently which are not included in the existing, internationally-recognized reporting guidelines for systematic reviews PRISMA(Preferred Reporting Items for Systematic Reviews and Meta-analyses). Objectives: To assess whether equity-focused reviews require additional or modified items in the PRISMA reporting guidelines. Methods: We conducted a Cochrane methodology review and a methodology study and we assessed three equity-focused systematic reviews using PRISMA. Results: We found that equity analyses and applicability judgments were reported in insufficient detail to judge their credibility in our methodology study and Cochrane methodology review. We found 11 areas where PRISMA items needed to be modified or added to reflect special methods and concerns in equity-focused systematic reviews. These items related to the definition of health equity and rationale for assessing it, choice of study designs, search strategies, process evaluation, analysis of effects on equity in health status and judgments about applicability and generalizability. Conclusions: There is a need for modified PRISMA reporting guidelines for equity-focused reviews to reflect specific concerns and methods used. The Campbell and Cochrane Equity Methods Group is leading the development of an Equity-Extension to PRISMA, and invites Cochrane members to participate in a Delphi process and online survey to select and prioritize items. C2O4 What’s the story?: The Cochrane Library publicity, global media coverage and author experience Pentesco-Gilbert D1 , Stewart G1 , Beal J1 1 Wiley-Blackwell, John Wiley & Sons, Chichester, UK Background: Reviews from The Cochrane Library, published in the Cochrane Database of Systematic Reviews are often reported by newspaper and broadcast journalists, and bloggers. Press releases highlighting selected new and updated reviews are an important part of the publication process and marketing strategy for The Cochrane Library. Increasingly reviews are used as referential sources by media to help confirm what is known about a topic. Authors and editors of Cochrane reviews are involved in the authoring of the releases and media coverage. Objectives: The objective of Wiley–Blackwell’s publicity strategy is to disseminate the findings of Cochrane reviews and raise the international profile of The Cochrane Library through professional and consumer media, and to increase usage. Methods: A number of new and updated reviews are selected for publicity activity from the monthly issue publication of the Cochrane Database of Systematic Reviews. By analysing the resulting coverage, including a regional breakdown of where different reviews are reported, we can provide an interesting snapshot of the issues of interest to the international community. We also look for a corresponding increase in usage and citations for the articles that are most widely reported. Author feedback on their media experiences is collected post publication and will be summarized to identify opportunities to improve the quality Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 36 of media stories and author satisfaction. Results: An up-to- date analysis of international media coverage for the period January 2010 to June 2011 will be presented. Conclusions: Media coverage of new and updated Cochrane reviews raises the international profile of The Cochrane Library in both professional and consumer media, and increases usage of Cochrane reviews. A review of the impact of monthly publication (introduced in January 2010) and increased use of social medial tools together with author experiences will help to inform future publicity approaches. C3 Consumer issues and shared decision making C3O1 Consumer participation in The Cochrane Collaboration – comparison of two models Gyte G1 , Horey D2 1 Consumer Editor, Pregnancy & Childbirth Group, UK; 2 Editor, Consumers and Communication Group, Australia Background: Consumers have been integral to The Cochrane Collaboration since its inception in 1993, making the organisation a leader in consumer involvement across the world. However, we still do not know how to assess the contribution of consumers, or know how to get their input in the most feasible and acceptable way. It is likely that different models will work better for different entities. With the Collaboration looking to review its standards, it is important to ensure that involving consumers is done effectively so that consumer input is not just a token gesture. One Cochrane Review Group operated a complex model of consumer participation for ten years (from 1998 to 2007), overseen by a Consumer Panel Coordinator (a funded part-time post) and aided by two volunteer co-ordinators in USA and Australia. This model was evaluated in 2003 using interviews with editors, consumers and review authors. When funding to the Review Group was reduced in 2007, the Consumer Panel model ceased, and the Group embarked on an alternative model of involving consumers using staff at the editorial base (2008 to present). Objectives: To look at the advantages and disadvantages of the two models of consumer involvement used by the Cochrane Review Group between 1998 and 2011. Methods: Interviews will be conducted with editors and consumers with experience of both models of consumer input. Findings will be compared with those from the earlier evaluation. Conclusions: The lessons learned from both models of consumer participation will be presented. Ideas for other consumer involvement models and possible outcome measures will be proposed which may help the Collaboration continue to lead the world in involving consumers in health research. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. C3O2 Spreading Cochrane press releases to lay people and Italian journalists Colombo C1 , Pecoraro V2 , Banzi R1 , Moja L3 , Cima S4 , Mosconi P1 , Liberati A5 , Pistotti V2 1 Mario Negri Institute for Pharmacological Research, Milan Italy; 2 Italian Cochrane Centre, Mario Negri Institute for Pharmacological Research, Milan Italy; 3 University of Milan, Italy; 4 Zadig, Scientific Publishing Company, Italy; 5 University of Modena and Reggio Emilia, Italy and Italian Cochrane Centre, Mario Negri Institute for Pharmacological Research, Milan Italy Background: Since 2003, the Mario Negri Institute, the Italian Cochrane Centre (ICC) and the scientific publishing company Zadig, have developed ‘PartecipaSalute’, a national program to foster partnerships among patient organizations and healthcare professionals. The Italian translation of several Cochrane Reviews’ abstracts and plain language summaries are hosted in the PartecipaSalute website, with the objective to reduce users’ distance from evidence-based information. The Cochrane press releases are appealing for a general audience, and therefore could represent a preferred tool for dissemination. Objectives: To spread the Cochrane press releases to lay people and journalists and explore their popularity in Italy. Methods: Since 2007, the ICC receives Cochrane press releases from the Publisher Wiley a few days before the release of a new issue of The Cochrane Library. These are translated and adapted to the Italian setting, by researchers working on methodology and communication aspects. The press releases are then sent to a mailing list of 284 journalists and published on the PartecipaSalute portal. The access to this website section was monitored from 2007 to 2011 to explore the popularity among lay people visiting the portal. Monitoring of press coverage in a sample of journals is ongoing. Results: One hundred and ten press releases have been published and mailed to the journalists from January 2007 to April 2011. The median website visits per month were more than 28.6 (range 17.617 to 39.616). We will provide data on the Cochrane press releases section and about press coverage. Conclusions: Preliminary data show that the press coverage of Cochrane press releases is sensitive to hot media topics (eg. vaccines, zinc for the common cold, acupuncture). The impact of the press releases will be evaluated. C3O3 Involving young people in all stages of a systematic review Liabo K1 Science Research Unit, Institute of Education, University of London, UK 1 Social Background: A group of young people with experience of being in foster or residential care were invited to be involved in setting the question for a systematic review on a health-related topic, and to participate in all stages of the research. Objectives: To present the review and describe how the young people were involved in all stages of the research. To consider the negative and positive impact of their involvement, on the review and on the researcher. To discuss how consumer involvement may be optimised without compromising on the review quality. Methods: The systematic review followed standard Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 37 systematic review methods, but most of the decisions were made in collaboration with the young people. The young people were given some research training, and decisions were made using closed and open voting, following group discussions. The protocol was developed from a template, containing open and closed questions. The researcher recorded all meetings and analysed the interaction and decisions made, using thematic analysis. Results: The young people’s decision to address the effectiveness of educational support reflects a wider view of what constitutes health. They were successfully involved in developing the protocol, contributing to the searching and screening of 24% of all electronic hits. Their involvement in the critical appraisal of included studies and the synthesis was limited. Conclusions: Young people can be involved in most stages of a systematic review, but such participation requires the researcher to think of creative methods for presenting what a systematic review is, and explaining some of the key principles of reviewing. Some young people preferred to provide advice on the framework of the review, whereas others took part in research activities. C3O4 Do Cochrane systematic reviews report patient-important outcomes? Howell M1 , Tong A1 , Wong G2 , Howard K1 , Webster A1 , Craig J1 1 University of Sydney, Australia; 2 Westmead Hospital, Australia Background: Despite the proliferation of trial evidence, uncertainty remains about the effect of interventions on outcomes that are important and relevant to patients. Many studies report endpoints that have little meaning for patients. Objectives: To assess the concordance between the prioritised outcomes associated with immunosuppressant medications among kidney transplant recipients with outcomes reported in Cochrane systematic reviews. Methods: Identification and ranking of outcomes important to kidney transplant recipients was undertaken using the nominal group technique. Each group, which involved 6 to 10 participants, developed and ranked in order of importance a list of outcomes considered relevant to immunosuppressant medications. For the participant rankings, a maximum priority score was calculated and data presented as a percentage of the maximum possible priority score. The top 15 patient-important outcomes were compared with outcomes reported in Cochrane systematic reviews on immunosuppressive regimens in kidney transplantation. Results: Across the eight groups, we identified 48 outcomes considered relevant to kidney transplant recipients. The top ranked 15 were: graft loss, kidney function, damage to other organs, death, all cancers, diabetes, skin cancer, cardiovascular disease, susceptibility to infection, night tremors, weight gain, bone disease, impact on family, depression, and gastrointestinal problems. Across the five Cochrane systematic reviews, 5 (33%) to 9 (60%) of the top 15 patient ranked outcomes were reported. No reviews included damage to other organs, night tremors, impact on family and depression outcomes. Only one review addressed the following outcomes: skin cancer, cardiovascular disease, weight gain, bone disease and gastrointestinal problems. Conclusions: Patient-important outcomes following kidney transplantation include serious and fatal consequences as well as emotional and physical side effects. Strategies to integrate patient priorities for immunosuppressive treatment outcomes in clinical trials and systematic reviews are needed. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. C4 Editorial processes and supporting review authors C4O1 National Institute of Health Research (NIHR) Cochrane Collaboration Programme Grant Scheme – Gynaecological Cancer Review Group Williams C1 , Morrison J1 , Bryant A1 , Quinn G1 , Jess C1 , Hayes J1 1 Cochrane Gynaecological Cancer Review Group, UK Background: The Cochrane Gynaecological Cancer Review Group (CGCRG) was awarded an NIHR Cochrane Collaboration Programme Grant in September 2007. The Group decided to use the funds to help support authors preparing systematic reviews on topics of high priority to NHS patients which would also inform the gynaecological oncology community on an international level. The NHS in the UK has made major improvements in gynaecological cancer care and it is one of the most integrated and multidisciplinary areas in medicine with an active Improving Outcome Guidance Programme. This is important as there are seven tumour sites with multiple complex care pathways. The CGCRG is a highly collaborative multidisciplinary group with an excellent track record in developing teams to prepare reviews. The applicants of the award scheme all have strong links to all major professional and consumer organisations involved in gynaecological cancer. Objectives: Improvement of patient care by preparing systematic reviews, which inform national and international guidelines. Methods: After a major prioritisation project, titles were allocated to UK sub-speciality trainees who expressed an interest and demonstrated ability in conducting systematic reviews. The authors worked with experienced editors as well as with a dedicated methodologist who was employed throughout the grant period. All review authors received training and mentoring and were encouraged to meet regularly with their appointed editor and methodologist to progress the review throughout the entire process. The editorial base provided support, maintaining communication between the editors and authors, and managed the process of statistical, methodological and external peer review. Results: Thirty seven reviews and updates were produced in this programme and are being translated by British Gynaecological Cancer Society and Network Site Specialist Groups into guidelines and protocols to improve patient care. The Group has subsequently been awarded a second grant based on the success of this scheme. C4O2 Testing selective responses of Cochrane groups to the request of conducting a Cochrane systematic review: A crossover randomized controlled trial Ciapponi A1 , Glujovsky D1 , Rey Ares L1 , Garcı́a Martı́ S1 , Reveiz L2 1 IECS, Institute for Clinical Effectiveness and Health Policy Buenos Aires. Argentina; 2 Andean Cochrane Branch of the Ibero-American Cochrane Center Background: Thousands of people from around the world contribute to The Cochrane Collaboration. Although there are a number of steps that need to be followed to register a review, titles are Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 38 Table 1 (C4O2). Not-native English speaker request model. CRG X Dear Xxx Xxx My name is Yyy Yyy and I am an Physician from OOO. I have just completed a non-Cochrane review and am now interested in performing a Cochrane Review. Specifically, I am interested in your group’s topics. Would you please inform me if there are any available topics or issues that I may assist you with? Thank you for your time and attention. I look forward to hearing from you soon. Respectfully Yyy Yyy accepted at the discretion of each Cochrane Review Group (CRG) and criteria for accepting those titles are not explicit [1]. Previous studies found that authors reported having problems in getting titles registered [2,3]. Objectives: To assess if the first language or country location of ‘simulated newcomers’ influence the response patterns to register titles in CRGs. Methods: Two hypotheses were addressed: 1) native English speakers (NESs) receive more positive responses (+R) than non-NESs (NNESs); 2) responses will not differ by country income (low income countries (LMC) versus High Income Countries (HIC)). After ethics committee approval, we emailed substantially identical versions of a title request (Table 1) to 53 CRGs written by: ‘NES-LIC’, ‘NES-HIC’, ‘NNES-LIC’ and ‘NNES-HIC’. In this crossover RCT (one month of washout period), CRGs were randomized centrally by computer software. The answers were independently classified as ‘positive’ (offer review titles/updates, specific topics or mentors); ‘negative’ (explicit -R or requirements without help offer) or ‘neutral-R’ (the others) by two assessors blinded to interventions and e-mail senders/recipients. Results: No response was obtained in 23/106 (21%) of emails. The patterns of responses were: 24% positive, 40% neutral and 36% negative. NES received significant more +R than NNES (P = 0.031) as well as HIC than LIC (P = 0.031), being worst for NNES-LIC and best for NES-HIC (Table 2; Figure 1). Table 2 (C4O2). CRG’s response pattern by first language or location of ‘simulated newcomers’. NNES/NES: Not Native/Native English speaker; LIC/HIC: Low/High Income Country NNES N (%) NES N (%) LIC N (%) HIC N (%) Negative [20] 15 (36.59) 5 (11.90) 12 (25.00) 8 (22.86) Neutral [33] 14 (34.15) 19 (45.24) 24 (50.00) 9 (25.71) Positive [30] 12 (29.27) 18 (42.86) 12 (25.00) 18 (51.43) Total [83] 41 (100.00) 42 (100.00) 48 (100.00) 35 (100.00) Responses [N] Pearson chi 2 = 0.031 Postivie vs. Negative/No Reply Pearson chi 2 = 0.031 LIC: RR 3 (95% CI 0.98–9.20) HIC: RR 1.29 (95% CI 0.66–2.53) Figure 1 (C402). CRG’s response pattern by first language or location of ‘simulated newcomers’. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 39 Besides no response, frequent barriers included requesting curriculum vitae and newcomer’s publication list. Some CRGs had very positive, warm and collaborative responses. Conclusions: Although CRGs should focus on potential authors that will produce high quality and timely protocols and reviews, criteria for registering titles should be explicit and standardized. The Cochrane Collaboration should consider consistent ways of supporting authors from NNES-LIC. References 1. The Cochrane Collaboration. Registering Titles. Available at: http:// www.cochrane.org/cochrane-reviews/registering-titles. Access in April 2011. 2. Ciapponi A RL, Martı́-Carvajal A, Ortiz Z, Sanchez-Gomez LM, Delgado-Ramirez MB, Manterola-Delgado CG, Martı́nez-Pecino F. How difficult is to register a Cochrane title? An lberoamerican authors sample. XIV Cochrane Colloquium Dublin, Ireland 2006: [P003]. 3. D Gillies HM, K New, V Pennick, Z Fedorowicz, J Van Der Wouden, J Oliver, R. Scholten, A Ciapponi, J Verbeek. A collaboration-wide survey for Cochrane authors. XVI Cochrane Colloquium. Freiburg, Germany 2008: [P04-33]. C4O3 Impact, accountability, and sustainability of the Cochrane prioritization project – Eyes and Vision Group experience Li T1 , Dickersin K2 1 Cochrane Eyes and Vision Group US Project, Johns Hopkins Bloomberg School of Public Health, US; 2 Cochrane Eyes and Vision Group US Project, US Cochrane Center, Johns Hopkins Bloomberg School of Public Health, US Background: The Cochrane Eyes and Vision Group (CEVG) US Project was funded by the Cochrane Collaboration Prioritization Fund from 2007 to 2009 to test a framework for prioritizing systematic reviews. Objectives: To describe the impact, accountability, and sustainability of the project. Methods: A summary of research activities, professional collaborations, funding, and research opportunities. Results: We identified priority review topics related to open angle glaucoma using practice guidelines and a Delphi survey of clinicians. We searched for and appraised the methodological quality of existing systematic reviews for priority topics (Cochrane and non-Cochrane). For topics without any high quality review, we sought approval from the CEVG editor and posted the topics on the CEVG website. For priority topics with non-Cochrane reviews, we encouraged authors to convert them into Cochrane reviews as appropriate. We presented project findings at each stage, at the Cochrane Colloquia (2008 to 2010) and other international meetings. This project comprised one part of TL’s PhD thesis, and has resulted in two publications. An additional manuscript is under revision. We involved various stakeholders in the project, including CEVG and its Advisory Board, the US Cochrane Center, local biostatisticians and epidemiologists, guideline developers at the American Academy of Ophthalmology, and clinicians and vision research scientists from the American Glaucoma Society and Asian-Pacific region. Findings from this project also provided the groundwork for an additional grant award to our team from the National Eye Institute, to develop research priorities related to diabetic retinopathy and primary angle-closure glaucoma. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Two Master’s students are pursing the newly-funded projects as their thesis research. The American Optometric Association is considering collaboration with us to update their guidelines on diabetic eye disease. Conclusions: The impact of the Collaboration’s Prioritization initiative has been profound for our group and our research area. We have demonstrated accountability to the Collaboration and sustainability of the framework. C4O4 Capacity building in systematic reviews in South Asia: a South-North collaboration to expand the production and translation of synthesized evidence Koehlmoos TP1 , PIgott T2 , Thomson H3 , Ueffing E4 , McGowan J5 , Gazi R6 , Hossain S6 , Snilstveit B7 , Howard G8 1 Health & Family Planning Systems Programme, ICDDR,B, Bangladesh; 2 School of Education. Loyola University Chicago, USA; 3 Medical Research Council, Social & Public Health Sciences Unit, UK; 4 Campbell and Cochrane Equity Methods Group, Institute of Population Health, University of Ottawa, Canada; 5 Centre for Global Health, Departments of Medicine and Family Medicine, University of Ottawa, Canada; 6 Health Systems & Infectious Diseases Division, ICDDR,B, Bangladesh; 7 International Initiative for Impact Evaluation (3ie), UK; 8 South Asia Research Hub, Department for International Development, India Background: A substantial body of evidence exists to answer many of the questions asked by policymakers and development partners in low- and middle-income countries (LMICs). However, evidence is often scattered, inaccessible and rarely presented in a form that provides an indication of the quality of evidence. Systematic reviews in all sectors have the potential to contribute to improving the evidence base from which policies and interventions can be developed and implemented. The principles that underpin the international effort to prepare and use systematic reviews of controlled trials for evaluating clinical practice, particularly in high income countries, can be similarly applied to the challenges of poverty reduction and development in LMICs. Objectives: To support researchers in South Asia through the provision of capacity building in systematic review techniques using training, workshops, online training and support, and quality assurance. Methods: Health & Family Planning Systems Programme (ICDDR,B) in collaboration with the Campbell Collaboration, the Social and Public Health Sciences Unit of the Medical Research Council UK (SPHSU MRC), the Campbell & Cochrane Equity Methods Group and the International Initiative for Impact Evaluation (3ie) using funding from the Department for International Development (UK) will develop and deliver i) a programme of systematic review training in South Asia, ii) guidelines for development partners on formulating review questions, and iii) systems of remote partnership, networking and support to regional review teams. Results: The courses are being developed with a first run in June 2011. This section will be updated with details of participants, their feedback and any successful assistance/review protocols in October 2011. Success of the project will be measured by the number of regional review teams who receive support and bring in funding and/or complete reviews. Conclusions: LMIC-based researchers will be equipped to undertake systematic reviews to help inform decision making in different sectors of development. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 41 Posters sessions Searching and information retrieval Day: 20 Oct (P1A1–P1A55) P1A1 Is the retraction of journal articles in electronic journals and databases consistent and timely? A case study Wright K1 , McDaid C1 1 Centre for Reviews and Dissemination, UK Background: It is essential that the processes used for literature searching and selecting studies for inclusion reliably identify retracted papers so they are not unwittingly included in systematic reviews. There is, however, evidence that retracted papers continue to be cited without reference to the retraction. In 2009 multiple papers authored by Dr Scott S Reuben were retracted due to falsification of data. We used these citations to investigate whether databases and journals recorded the information on retraction appropriately. Objectives: To investigate whether three bibliographic databases recorded notices of retraction in a clear and timely way and whether electronic journals displayed information about retractions in a way that prevented inadvertent use of fraudulent research data. Methods: We searched MEDLINE, EMBASE and CENTRAL databases for 18 papers by Reuben that were known to have been formally retracted, based on retraction notices. We noted whether the database recorded that each of the articles was retracted. The 18 papers were published across nine journals. We retrieved one paper from each of the nine electronic journals and assessed whether it was possible to identify the study as being retracted. Results: In our case study all of the retracted papers identified on MEDLINE had been annotated appropriately as compared to 6% of EMBASE records and 80% of CENTRAL records. Similarly, there was a high level of variability across journals in how clearly they annotated that a paper had been retracted. Conclusions: Lack of consistency in how databases and electronic journals record retractions and the time taken to record the retractions make it difficult for researchers and information specialists to identify retracted studies and extra care is needed to ensure that they are not included in systematic reviews. P1A2 Comprehensive searching for systematic reviews: A comparison of database performance Beyer F1 , Wright K1 1 Centre for Reviews and Dissemination, UK Background: It is generally acknowledged that searching for studies to populate a systematic review should be as comprehensive as resources allow. Where choices have to be made between available databases, little evidence-based guidance exists about how to prioritize resources to search or when to stop searching. Objectives: This study aimed: (i) to compare the individual contribution of databases searched to identify studies meeting the inclusion criteria for a recently completed systematic review; (ii) to identify the best combination of databases to retrieve all relevant studies. Methods: A systematic review of Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. the non-pharmacological management of frozen shoulder was used as a case study. The total number of studies included in the review provided the reference standard. A spreadsheet was used to record for each included study a) whether it was retrieved by the appropriate search strategy and b) whether it was indexed on each database. Yields and number needed to read (NNR) from different combinations of databases were investigated. Results: The review included 27 randomized controlled trials (RCTs), one controlled clinical trial (CCT) and two case series. Of the 28 controlled trials, 27 were indexed in at least one of sixteen analysed databases. Recall for individual databases ranged from 0% to 90% (median 40%), and precision from 0.0% to 4.3% (median 1.2%). For the strategies used, the best combination of resources to retrieve all studies was either CENTRAL, EMBASE, Science Citation Index, and reference checking; or CENTRAL, EMBASE, MEDLINE (including PREMEDLINE), and reference checking. Conclusions: Of the databases searched, CENTRAL displayed the best combination of recall and precision. In terms of combinations of databases a minimum of three databases, including CENTRAL, plus reference checking were required to retrieve all included studies. P1A3 Threading the needle we found in the haystack: Identifying ongoing trials with the Pan African Clinical Trials Registry Abrams A1 , Pienaar E1 1 South African Cochrane Centre Background: Cochrane review authors face difficulties identifying ongoing trials in regions where trials registries do not exist. The Pan African Clinical Trials Registry (www.pactr.org) provides a platform to prospectively register all clinical trials conducted in Africa as the first regional World Health Organization (WHO)-endorsed primary register in the region. Objectives: 1) To categorise trial interventions registered on www.pactr.org and map these to published Cochrane reviews or protocols; 2) To demonstrate the value of searching www.pactr.org for review authors. Methods: Registered trial details were downloaded from www.pactr.org on 11 April 2011. Two independent investigators extracted trial data, including disease and intervention details. We searched the Cochrane Database of Systematic Reviews (CDSR) 2011, Issue 3 for published reviews or protocols evaluating the interventions reported in these trials. Results: Forty-six trials are registered on www.pactr.org. Thirty are randomized controlled trials of efficacy. We identified 20 trials evaluating treatments, eight evaluating prevention interventions and two diagnostic trials. The 20 treatment trials evaluated drugs for malaria (6), HIV (3), tuberculosis (2), anaesthesia (1), depression (1), fertility (1), hyperkalaemia (1), fluids for surgical irrigation (1), postpartum haemorrhage (1), an electronic decision making tool (1), provision of economic incentives (1) and effects of patient advocates (1) on adherence. The eight prevention trials evaluated drugs for malaria (5), HIV (1), tuberculosis (1) and a behavioural intervention for sexual risk-reduction in HIV-positive adults (1). One trial assessed diagnostic algorithms and another, a TB diagnostic test. Our search of CDSR identified thirteen reviews and two protocols where data from twenty of the above ongoing trials would be eligible for inclusion. No review reported these. Conclusions: www.pactr.org is a free, useful tool for identifying ongoing African trials to be included in updates of reviews, and for assessing topics where reviews should be undertaken. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 42 P1A4 Filled Gaps? Data after a year of the Pan African Clinical Trials Registry efforts to increase understanding and awareness of the child-focused clinical trial landscape Abrams A1 1 South African Cochrane Centre, South Africa Background: Cochrane reviews of child-focused interventions often find that trial data in children is insufficient to draw conclusions. In 2009, the Pan African Clinical Trials Registry (www.pactr.org) in partnership with the World Health Organization developed a Child Strategy as part of a multi-regional initiative to increase child-focused clinical trial activity and registration worldwide. www.pactr.org, based at the South African Cochrane Centre, provides prospective registration for all clinical trials conducted in Africa. Objectives: To describe trials affecting child-health registered on www.pactr.org. Methods: We downloaded details of trials registered on www.pactr.org on 11 April 2011. A single investigator extracted trial data including details of disease, trial intention (prevention, treatment, etc.) and participant ages. Results: 23 of 46 (50%) registered trials investigated topics that affect child health. Of nine trials in pregnant women, five researched malaria (three evaluated drug prevention; two treatment), one assessed patient advocacy for preventing mother-to-child HIV transmission, and three researched diseases other than HIV or malaria (one evaluated anaesthesia for Caesarian Section, one post-partum hemorrhage Figure 1 (P1A4). Disease-type Researched by Participant Age. 10 8 6 Other 4 TB 2 HIV/AIDS 0 Malaria Pregnant Women Infants Older Children Large age range Figure 2 (P1A4). Country location of trials including children. Egypt Mali The Gambia Guinea-Bissau Nigeria Ethiopia Cameroon Burkina Faso Benin Uganda Kenya Gabon Tanzania Ghana 1 trial 2 trials Zambia Rwanda Mozambique 3 trials 4 trials 5 trials treatment, and one, progesterone for unexplained infertility). In infants, two trials evaluated HIV vaccines. Of 9 trials in older children, three assessed antiretrovirals in HIV-infected children, four researched malaria (two evaluated drug treatment, one vaccine and one, post malaria anemia treatment), one assessed Tuberculosis treatment efficacy and another, the use of electronic decision making tools. A trial promoting adherence via economic incentives for TB, one analyzing malaria prevention, and one evaluating a meningitis vaccine included children and adults. Compared to last year’s baseline data there is a slight increase (from 48% to 50%) in the overall profile of child-focused trials registered. Child-focused trial activity is present in 19 African countries (an increase from last year’s 14; see map). Conclusions: Continued efforts of the Child Strategy, to encourage child-focused trial research and registration in the region, will ensure www.pactr.org’s value as a research tool for those affiliated with the Cochrane Child Health Field. P1A5 From final search to publication of HTA reviews: Does the time gap have an influence on the usefulness of their results? Parekh-Bhurke S1 , Cook A1 , Chatting D1 , Fairbank E1 , Clarke A2 1 NETSCC, University of Southampton; 2 University of Warwick Background: Systematic reviews (SRs) are important building blocks for clinical practice guidelines and health technology assessment (HTA) reports. With a rapid growth in evidence, it becomes necessary to update SRs regularly to maintain their importance in informing healthcare policy and practice. There is very little guidance available about when and how to update SRs. Moreover, the updating practices of some organizations that commission or produce SRs are unclear. Objectives: We examined the effect of the time gap between final search and publication of HTA systematic reviews on the usefulness of results and how these can be mitigated for HTA reviews to ensure their up-to-date status. We also examined the consistency in reporting of these reviews using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Methods: We identified and included 14 HTA reviews published from 2005 to 2007. Reviews including at least one randomized trial and at least one meta-analysis were considered. The date of publication, last search date for each review and the date of publication of all the included trials in the review was extracted by dual data extraction. We estimated the probability of trials being missed during the period between the date of last search performed by authors and the date of publication of the review. Each review was also checked against the PRISMA checklist. For reviews where significant numbers of trials had been missed, we repeated the original review searches to identify missed trials, and assessed their impact on results using cumulative meta-analysis. Results: The study identified that there were several reviews or meta-analyses included in a single report making it difficult to report if updates were required. We will make recommendations for conduct of systematic reviews by the HTA programme and others. Our study will also contribute to understanding best practice for updating systematic reviews. Malawi South Africa Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 43 Figure 1 (P1A6). Countries of first author. P1A6 Bibliometric analysis of 100 most cited Cochrane Systematic Reviews Uthman O1 , Okwundu C2 1 University of Birmingham, UK; 2 Stellenbosch University, South Africa Background: Systematic reviews can be very useful decision making tools because they objectively summarize large amounts of information, identify gaps in medical research and identify beneficial or harmful interventions. The number of citations an article receives after its publication reflects its impact on the scientific community. There have been a few recent attempts to identify and analyze the ‘most cited articles’ in various specialties. This is an update of a previous analysis of the top cited Cochrane systematic reviews which we reported two years ago. Objectives: To identify the 100 most cited Cochrane systematic reviews (CSR). To assess whether or not citation classics exist in CSR and to examine characteristics of frequently cited CSR. Methods: 100 most frequently cited CSR were identified using the database of the Science Citation Index of the Institute for Scientific Information in April 2011. Results: The most cited CSR received 174 citations, and the least-cited CSR received 33 citations. Six CSR were cited more than 100 times. These CSR were published between 2005 and 2009. The 100 originated from 19 countries (Figure 1), with the United Kingdom contributing 31 articles, followed by Canada (n = 13), USA (n = 11) and Australia (n = 10). Obstetrics and gynaecology, cancer, diabetes mellitus, neuromuscular disease and smoking cessation were the main topics covered. Considerable attention was also given to asthma, dementia, chronic inflammatory disease, obesity and hypertension. Conclusions: At present, only six CSR can be classified as a ‘citation Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. classic’ (with 100 or more citations). The United Kingdom, Canada, and United States have produced the greatest number of highly cited CSR. P1A7 User-friendly search strategies for geriatric evidence in Medline van de Glind EMM1 , van Munster BC2 , Spijker R1 , Scholten RJ1 , Hooft L1 1 Dutch Cochrane Centre, Netherlands; 2 Academic Medical Center, Department of Internal Medicine, Netherlands Introduction: Geriatric patients need specific treatments provided by doctors who choose these treatments based on existing evidence. However, evidence on many geriatric topics is hard to find. We aimed to create user-friendly search strategies with excellent operating characteristics to find geriatric evidence in Medline quickly and easily. Methods: We used a diagnostic test framework to create the search strategies. We compared the search strategies’ retrieved articles with a reference set as index test. This reference set consisted of 2255 articles from 23 biomedical journals in Medline. Each of these articles was labeled as relevant, not relevant or possibly relevant for geriatric medicine. From the relevant articles, we identified search terms to compose different search strategies. We compared the retrieved result of the different search strategies with the reference set to calculate the sensitivity, specificity, precision, accuracy and number-needed-to read (NNR). Results: The most sensitive search strategy had a sensitivity of 92.0%, a specificity of 86.9%, a precision of 71.1%, an accuracy of 88.2% and a NNR of 1.40. The most specific search strategy had Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 44 a specificity of 96.0%, a sensitivity of 69.6%, a precision of 86.1%, an accuracy of 89.2% and a NNR of 1.16. Conclusion: Our geriatric search strategies simplify searching for relevant literature and therefore contribute to a better evidence-based practice. The search strategies are useful to both the clinician who wishes a quick answer to a clinical question and to the researcher who wants to find as many articles as possible without missing too much relevant information. P1A8 Testing search strategies for geriatric medicine in the Cochrane Database of Systematic Reviews van de Glind EMM1 , Young C2 , Spijker R1 , Scholten RJ1 , van Munster BC3 , Stott DJ2 , Hooft L1 1 Dutch Cochrane Centre, Netherlands; 2 University of Glasgow, UK; 3 Academic Medical Center, Netherlands Background: Finding relevant information in databases is the cornerstone of evidence-based medicine. However, searching databases can be challenging. Recently, sensitive and specific search strategies were developed to find information in MEDLINE for geriatric medicine quickly and easily. These search strategies proved to have excellent sensitivity and specificity. Objective: To test the operating characteristics of geriatric search strategies in the Cochrane Database of Systematic Reviews (CDSR). Methods: A reference set was obtained from the CDSR. One assessor labeled all 6437 reviews and protocols published until March 2011 as either relevant to geriatric medicine or not. We translated the search terms from MEDLINE into Cochrane. Then we compared the reviews that were found with the search strategy with the labeled reference set obtained from the CDSR. Results: Only 373 (6%) of the articles in the CDSR were considered relevant to geriatric medicine. The most sensitive search strategy had a sensitivity of 65%, a specificity of 76% and a number-needed-to-read (NNR) of 7. The most specific search strategy had a sensitivity of 45%, a specificity of 98% and a NNR of 1.4. Conclusion: A small percentage of reviews in the CDSR contain information on the elderly. Compared to the performance in MEDLINE, the sensitive search strategy performed slightly worse in the CDSR. Although it found two thirds of the relevant reviews, there were a lot of false positive findings resulting in a higher NNR. The specific search strategy retrieved very few false positives. However, half of the relevant reviews were missed. Differences in the performance of the search strategies could be due to a lower prevalence of relevant articles in the CDSR as well as the use of different criteria for considering articles as relevant to geriatric medicine. By using the CDSR as another reference set we have proven that the geriatric search strategies are robust. P1A9 Reconceptualising searching and screening: How new technologies might change the way in which we identify studies Thomas J1 , O’Mara A1 1 EPPI-Centre, Institute of Education, UK Background: Typical reviews deal with the ‘information explosion’ by narrowing their search for studies (e.g., applying search filters). Relevant evidence can be missed through this approach. Current Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. methods to minimise the risk of missing relevant studies involve searching broadly and screening potentially tens of thousands of records, which is not always practical. Resource-efficient approaches that maximise sensitivity are needed. Objective: To evaluate whether new technologies allow us to search broadly without increasing the screening workload through semi-automated screening approaches. Specifically, we evaluate two types of text mining: a support vector machine using active learning (Wallace et al., 2010) and TerMine term clustering. Methods: Text mining techniques were employed in an ongoing review to prioritise records for screening and to classify the records automatically as includes or excludes. Screening prioritisation was assessed by comparison with a ‘baseline inclusion rate’ and through the novel application of power calculations. Classification was assessed through the stability of the classifier and the calculation of performance metrics (precision, recall, F-values). The classification procedure was also evaluated using simulations of completed reviews. Results: Screening prioritisation worked when sufficient information was provided to the text mining tool; in the ongoing review, only 25% of all records were screened manually to identify the expected total number of included studies. Classification reduced the manual screening required in all reviews evaluated, although it worked better for some datasets than others. Conclusions: Systematic reviews need to develop ways of handling the growing amount of evidence available. Text mining is a promising approach that shifts the emphasis of identification from the searching stage to screening. Reconceptualising searching permits broad searches to be conducted and allows reviewers to be more precise in estimating the number of potentially missing relevant studies than can be achieved by narrowing the search process. Areas for further development are suggested. P1A10 Sorting the wheat from the chaff: How does the Cochrane HIV/AIDS Specialized Trials Register compare to searching standard electronic databases? Zani B1 , Oliver J1 , Siegfried N1 1 South African Cochrane Centre Background: The Cochrane HIV/AIDS Specialized Register has been housed at the South African Cochrane Centre since 2004. Records from quarterly searches are coded manually and include a term for randomized controlled trials (RCTs). In this study, we test the accuracy of our Register in identifying RCTs for Cochrane reviews. Objectives: 1. Determine the precision and sensitivity of the Register compared to electronic searches for identifying RCTs for Cochrane reviews; 2. Identify limitations of the Register. Methods: We searched Archie to identify Cochrane HIV/AIDS reviews published in 2010. We extracted relevant keywords and used them to search our Register for RCTs relevant in each review. We calculated precision by comparing the number of RCTs identified with the number of records retrieved from the searches, and sensitivity by comparing the number of relevant RCTs identified with the number of RCTs included in each review. Results: We identified 18 reviews, and have analyzed three to date. For one review, four RCTs were identified from 103 records retrieved from standard electronic databases (precision: 3.9%). Searching the Register found all four RCTs from 10 records, and precision improved to 40%, with no loss of sensitivity. However, for the other two reviews, with precision increasing from 0.6% (13/2317) to 9.5% (8/84) when searching the Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 45 Table 1 (P1A10). The precision and the sensitivity of the Cochrane HIV/AIDS Specialized register for identifying randomized controlled trials for inclusion in Cochrane Reviews. Review Database Records retrieved from searches Massage therapy review 2010 E-databases The Register Home-based care review 2010 E-databases The Register Oropharyngeal candidiasis review 2010 E-databases The Register 103 10 2317 84 640 132 RCTs included in the reivew 4 4 13 13 36 36 Included RCTs identified from searches 4 4 13 8 34 32 Precision (%) 3.9 Sensitivity (%) 100 40 100 Register and from 5.6% (34/640) to 24.2% (32/132), with 38% (5/13) and 6% (2/34) loss in sensitivity, respectively. Reasons for loss of sensitivity included keywords not appearing in the title or abstract of the Register record (n = 1); records coded as awaiting assessment (n = 3); records not identified in the Register (n = 3). Conclusions: Searching the Register for HIV/AIDS trials greatly increased precision but some sensitivity was lost with reasons identified. The Register is not yet sufficiently robust to be the sole database searched and additional database searches are required to maintain review quality. P1A11 Finding public health evidence: How many stones need to be turned? 0.6 100 9.5 5.6 24.2 61.5 94.4 88.9 authors retrieved the included studies. Results: We will provide data on which resource retrieved each included study, if a study is unique to a particular database and the proportion that were identified from either Medline, Embase or the Cochrane Library. Conclusions: The results will be useful for trial search coordinators in providing guidance to authors and to authors themselves in planning their searches, and will include guidance on a ‘core set’ of databases to search for public health topics. P1A12 Are the searches of network meta-analysis comprehensive or well-reported? Morgan H1 , Bauschmann M2 , Weightman A1 1 Cochrane Public Health Group, UK; 2 Cardiff University, UK Tian J1 , Yang K1 , Liu A2 , Li L1 1 Evidence Based Medicine Center of Lanzhou University, School of Basic Medical Science of Lanzhou University, Lanzhou 730000, China; 2 The Health School of Gansu Province, Gansu, China Background: Identifying studies that are relevant to the public health field can be challenging. The public health field is broad, and interventions can be complex involving many varied settings, communities and populations. Also there is a lack of uniform terminology to describe and organise the field. Controlled trials are often not suitable for evaluating public health interventions and therefore controlled before and after and interrupted time series designs are frequently utilised. The combination of these elements can require the use of more complex search methods such as searching a broad number of databases that are relevant to the particular public health topic. Objectives: This current project follows on from a well received poster that was presented at the 18th Cochrane Colloquium (Mann M et al. Searching for success for a public health intervention review). Mann et al. investigated the need to search multiple databases to identify evidence for their review of collaboration between local health and government agencies, concluding that it was an important component of the review process. We will expand the investigation to 12 Cochrane reviews relevant to public health to further analyse if searching multiple databases is essential in conducting a public health review. Methods: From the 12 reviews we will identify which databases searched by the Background: Network meta-analyses (such as mixed treatment comparisons meta-analyses, multiple-treatments meta-analyses) could combine information from all randomized controlled trials (RCTs) when many treatment regimens already exist, to rank their benefits (and harms) to choose the best option for patients. Therefore finding all available RCTs is very important. Objectives: To evaluate whether comprehensive searches were well conducted or reported in network meta-analyses. Methods: We searched Pubmed, The Cochrane Library, Embase, ISI Web of Knowledge, and Google Scholar using ‘Network meta analysis’ OR ‘mixed treatment comparisons meta analyses’ OR ‘multiple treatments meta analysis’ in title/abstract without language, time, publication type restrictions. The first search was conducted on 2010 October 21, and updated on 2011 February 21. We included any network meta-analysis, regardless of the conditions or interventions. Letters or abstracts were excluded. Results: We found 37 network meta-analyses, of which six included RCTs from relevant systematic reviews. For the remaining 31 network meta-analyses, the median number of databases searched was 3 (IQR 2-4, range 1–12) and 58.1% searched three databases or more. 37.8% did nothing beyond searching databases. Only 12.9% included the full Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 46 electronic search strategy for at least one database. 41.9% did not present search terms, 12.9% did not report when they conducted their searches. Conclusions: Our study results demonstrate that the searches in network meta-analyses are not well conducted, which may lead to biased results, and this needs to be improved in the future. P1A13 What is the role of public trials registers in Cochrane Systematic Reviews? Blackhall K1 , Glanville J2 , Ker K1 1 London School of Hygiene & Tropical Medicine, UK; 2 Universtiy of York, UK Background: All member journals of the International Committee of Medical Journal Editors (ICMJE) now require that clinical trials must be registered in a public trials register before they are considered for publication. Such registration ensures that trial details are available to all researchers, including systematic reviewers, irrespective of the results, which may encourage improvements in trial methods and minimise publication bias. The introduction of the ICMJE requirement in July 2005 is likely to have led to an increased number of trials being registered and there is interest in the extent to which such registers can now be relied upon as a source of trials for Cochrane systematic reviews. Objectives: To explore the current state of trial registration and to assess whether it is adequate to search for trials, for inclusion in systematic reviews, using only public trials registers. To assess the number and type of trials which would be excluded if unregistered trials were not eligible for inclusion in Cochrane Reviews. Methods: We identified three recently updated Cochrane Injuries Group reviews with a large number of included trials published since 2005. We examined the proportion of the included trials which had been registered and the trials register in which they appeared. We explored the yield of searches in individual databases compared to searching via the ICTRP search portal. Results: Results will be presented at the Colloquium. Conclusions: Using this sample we have examined whether extensive database searches were necessary, or whether we might have found the included trials by searching trials registers alone. In addition we will be able to report on search approaches for trials registers and the number and type of trials which are unregistered. It is recognised that results may not be extrapolated to all reviews. P1A14 A survey of searching CBM and LILACS databases in Cochrane Systematic Reviews Chen Y1 , Yao L2 , Wang Z2 , Liang H2 , Liang F2 1 Evidence-Based Medicine Center of Lanzhou University, China; 2 School of Basic Medical Sciences of Lanzhou University, China Background: Ideally, unbiased systematic reviews should always search and include all relevant trials, independent of the language of publication. The inclusion of studies published in languages other than English could help avoid bias. The Cochrane Handbook recommends systematic reviewers to search CBM (Chinese Biomedical Literature Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Database) and LILACS (Latin American and Caribbean Health Science) besides MEDLINE, EMBASE and Cochrane Central Register of controlled Trials (CENTRAL). Objectives: To investigate how many Cochrane Systematic Reviews searched CBM and LILACS Database. Methods: We searched the full text in the Cochrane Database of Systematic Reviews (4594 reviews and 2008 protocols in Issue 4, 2011) using the terms ‘CBM’, ‘Chinese Biomedical Literature Database’, ‘LILACS’, ‘Latin American and Caribbean Health Science’. We then screened all included studies. Results: 222 (3%) systematic reviews or protocols searched CBM and 1353 (20%) systematic reviews or protocols searched LILACS. Only 41 (0.6%) systematic reviews or protocols searched both CBM and LILACS from the year 2002. Among these 41 systematic reviews, 28 (68%) have Chinese authors and 13 (32%) are about Traditional Chinese Medicine. Conclusions: Although searching for trials in CBM and LILACS has been demonstrated to improve the quality of systematic reviews, less than 1% of published Cochrane systematic reviews searched both of them. The number of Cochrane systematic reviews that searched LILACS is 6-fold the number of Cochrane reviews that searched CBM. We suggest more Cochrane systematic review authors search non-English databases, such as CBM and LILACS. P1A15 The sensitivity and precision of adverse effects search filters in MEDLINE, EMBASE and Science Citation Index (SCI) Golder S1 , Loke Y1 1 Adverse Effects Methods Group, UK Background: Systematic reviews should use search strategies which aim to identify as many relevant papers as possible. However, searching for information on adverse effects is challenging, not least because adverse effects terms (either specific such as ‘venous thrombolism’or ‘weight gain’ or generic such as ‘adverse event’ or ‘side effect’) may not be included in the title, abstract or indexing of bibliographic records in databases such as MEDLINE and EMBASE. Objectives: To assess the feasibility of using adverse effects terms when searching electronic databases to retrieve papers that report adverse effects data. Methods: 242 included studies from 26 systematic reviews on adverse effects were analysed to ascertain whether the corresponding records in MEDLINE, EMBASE and Science Citation Index (SCI) included adverse effects terms in the title, abstract or indexing. In addition published adverse effects search filters devised for MEDLINE and EMBASE were tested to assess how many studies would have been missed had these filters been applied. Results: Records in EMBASE (89%) were more likely to contain adverse effects terms in the title, abstract or indexing, than MEDLINE (80%) or Science Citation Index (SCI) (70%). The percentage of papers which would be missed with a combined search using adverse effects terms in MEDLINE and EMBASE was 8%. This figure is much lower than the 23% identified by a previous study in 2001. The sensitivity of published adverse effects search filters varied in MEDLINE from 1% to 93% and in EMBASE from 58% to 97%. Higher sensitivity was achieved when named adverse effects were included in the search strategies. Conclusions: The proportion of records that include adverse effects terms in the title, abstract or indexing appears higher than in 2001. Although no adverse effects search filters captured all the relevant records, high sensitivity could be achieved, particularly in EMBASE. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 47 P1A16 The usefulness of different information sources for retrieving adverse effects data for a systematic review Golder S1 , Loke Y1 1 Adverse Effects Methods Group, UK Background: Authors of systematic reviews tend to focus on searching MEDLINE and reference checking for identifying information on adverse effects. However, MEDLINE may not yield the most information on adverse effects. Objectives: To determine the relative value and contribution of searching different sources to identify adverse effects data. Methods: A systematic review of thiazolidinedione-related fractures in patients with type 2 diabetes mellitus was used as a case study. Searches were conducted in over 20 different sources including MEDLINE, EMBASE, Derwent Drug File, selected Internet sources and reference texts. A record was made for each relevant reference noting whether it was retrieved with a search for thiazolidinediones and fractures and whether it was indexed/available in each source at the time of searching. A record was made of the sensitivity, precision, and number needed to read from searching each source and from different combinations of sources. Results: The review included 58 relevant references. The highest number of relevant references (35) were retrieved from Science Citation Index (SCI), followed by BIOSIS Previews (27) and EMBASE (23). The precision of the searches varied from 0.88% (Scirus) to 41.67% (CENTRAL). With the search strategies used, the minimum combination of sources required to retrieve all references was: the GSK website, Science Citation Index (SCI), EMBASE, BIOSIS Previews, British Library Direct, Medscape DrugInfo, handsearching and reference checking, AHFS First, and Thomson Reuters Integrity or Conference Papers Index (CPI). However, high numbers needed to read were required for BIOSIS Previews and Medscape DrugInfo even when searched after other sources with duplicate records removed. Conclusions: In order to identify all the relevant references for this case study a number of different sources needed to be searched. The minimum combination of sources required to identify all the relevant references did not include MEDLINE. P1A17 A comparison of Embase systematic review filters Martin S1 , McHugh J1 1 BMJ Publishing Group, UK Background: The BMJ Evidence Centre has a transparent and rigorous methodology. Our products are based on systematic searches of Medline, Embase and the Cochrane Library for systematic reviews and randomised controlled trials. Our challenge was to develop new filters for Embase searches to increase their specificity, without losing a great deal of sensitivity, in order to increase our productivity for a variety of BMJ Evidence Centre products. Objectives: To compare the new Embase systematic review filter used for BMJ Evidence Centre searches against other published Embase systematic review filters. Methods: We have done an extensive literature search for all other available filters to use as comparisons and have based our methods on the relative recall method as reported by Sampson M. et al. (2006) Results: We will report specificity and sensitivity; highlight the strengths and weaknesses of our research methods; discuss considerations and outcomes and present our conclusions. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Reference 1. Sampson M et al. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Research Methodology 2006 6:33 [http://www.biomedcentral. com/1471-2288/6/33] P1A18 A comparison of Embase randomised controlled trial filters Martin S1 , McHugh J1 1 BMJ Publishing Group, UK Background: The BMJ Evidence Centre has a transparent and rigorous methodology. Our products are based on systematic searches of Medline, Embase and the Cochrane Library for systematic reviews and randomised controlled trials. Our challenge was to develop new filters for Embase searches to increase their specificity, without losing a great deal of sensitivity, in order to increase our productivity for a variety of BMJ Evidence Centre products. Objectives: To compare the new Embase randomised controlled trial filter used for BMJ Evidence Centre searches against other published Embase randomised controlled trial filters. Methods: We have done an extensive literature search for all other available filters to use as comparisons and have based our methods on the relative recall method as reported by Sampson M. et al. (2006). Results: We will report specificity and sensitivity; highlight the strengths and weaknesses of our research methods; discuss considerations and outcomes and present our conclusions. Reference 1. Sampson M et al. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Research Methodology 2006 6:33 [http://www.biomedcentral. com/1471-2288/6/33] P1A19 A comparison of two search methods for determining the scope of systematic reviews Forsetlund L1 , Kirkehei I1 , Harboe I1 , Odgaard-Jensen J1 1 Norwegian Knowledge Centre for the Health Services, Norway Background: One of the core activities of the Norwegian Knowledge Centre for the Health Services is the summary of research literature into systematic reviews on request. A pilot search to assess the scope of a new review is conducted before starting the reviewing process. These searches are conducted in pre-specified databases. However, new search facilities are sometimes launched, which challenge previous search methods. Objectives: To develop a pragmatic approach for systematically comparing two search methods for determining the scope of a systematic review. The first method (Direct Search Method) included performing direct searches in the Cochrane Database of Systematic Reviews (CDSR), Database of Abstracts of Reviews of Effects (DARE) and the Health Technology Assessments (HTA). Using the comparison method (NHS Search Engine) we performed searches Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 48 via the search engine of the British National Health System, NHS Evidence. Methods: We used an adapted cross-over design with a random allocation of the 55 requests for systematic reviews. The main analyses were based on repeated measurements adjusted for the order in which the searches were conducted. Results: The Direct Search Method generated on average fewer hits, 48% (95% CI 6% to 72%), had a higher precision, 0.22 (95% CI 0.13 to 0.30), and more unique hits than when searching by means of the NHS Search Engine, 50% (95% CI 7% to 110%). On the other hand, the Direct Search Method took longer, 14.58 minutes (95% CI 7.20 to 21.97), and was perceived as somewhat less user-friendly than the NHS Search Engine, −0.60 (95% CI −1.11 to −0.09). Conclusions: Although the Direct Search Method had some drawbacks such as being more time-consuming and less user-friendly, it generated more unique hits than the NHS Search Engine, retrieved on average fewer references and fewer irrelevant results. P1A20 Using a collaborative online platform to identify systematic reviews in LILACS Rada G1 , Ciapponi A2 , Glujovsky D2 , Rivera S1 , Roa M1 , Abdala V3 1 Evidence Based Health Care Program. Faculty of Medicine, Pontificia Universidad Católica de Chile, Chile; 2 Argentine Cochrane Centre IECS, Institute for Clinical Effectiveness and Health Policy, Argentina; 3 Servicios Cooperativos de Información – BIREME/OPS/OMS, Brasil Background: It is desirable that users have easy access to systematic reviews (SR) instead of potentially biased narrative reviews (NR). Methodological filters for databases are efficient tools for identifying SR but NR and other articles are also retrieved. The user demands some ability to discriminate between these, which is frustrating and prone to mistakes. Considering the limitations of database-specific strategies to perform this differentiation, we organized a collaborative web-based model to identify SR in LILACS. Objectives: To identify SR in LILACS, in order to facilitate their retrieval by users, including organisations appraising and disseminating SR (e.g. DARE). Methods: A search strategy developed for this project (based on Montori 2005 and the Ecuador Cochrane Center strategy) was deployed from January 2008 to February 2011. Study selection will be performed through EROS (Early Review Organizing Software), a web-based software for the initial phases of the SR process. Pairs of researchers will independently assess by title/abstract considering for full text assessment all potentially eligible SR (e.g. those mentioning at least a search in the methods section). They will assess SR as detailed in Table 1 and will also register if a meta-analysis is presented. Discrepancies will be solved by consensus. The search strategy will be run quarterly followed by the described process of SR identification. A filter and label of confirmed SR will be incorporated into LILACS. Results: The search strategy retrieved 1819 references. Results about SR identification and classification will be presented at the Colloquium. Once this project is in full operation, the users of LILACS will easily access confirmed SR. This collaborative model may be extended to randomized clinical trials and other study designs. P1A21 Searching the Chinese literature for public health interventions: Can it add value to a systematic review? Baker P1 , Sun J1 , Weightman A2 , Francis D3 1 School of Public Health Queensland University of Technology and Epidemiology Central Regional Services Queensland Health, Australia; 2 Support Unit for Research Evidence (SURE), Information Services, Cardiff University, UK; 3 Epidemiology Central Regional Services, Queensland Health, Australia Background: Identifying studies for inclusion in systematic reviews of public health interventions requires a comprehensive search strategy, usually including multiple databases. Increasingly, consideration has also been given to identifying studies only published in the Chinese Table 1 (P1A20). Definition of systematic reviews. *DARE: ≥4 criteria out of the first 5 (1–3 are mandatory) # Oxman and Guyatt 1991: 1–4 & 6–7 # Cochrane Collaboration, CRD, MOOSE, Potsdam Consultation, QUOROM, AHRQ: 1–4 & 6–8 1. 2. 3. 4. 5. Were inclusion/exclusion criteria reported? Was the search adequate? Were the included studies synthesized? Was the validity of the included studies assessed? Are there sufficient details about the individual included studies presented? 6. Was the data extraction process adequate? 7. Was the study selection process adequate? 8. Was ‘PICO’ used to focus the question(s)? *DARE. (Accessed 04/08/2011, 2011, at http://www.crd.york.ac.uk/cms2web/AboutDare.asp.) # Sander L, Kitcher H. Systematic and Other Reviews: Terms and Definitions Used by UK Organizations and Selected Databases. Systematic Review and Del-phi Survey. In: National Institute for Health and Clinical Excellence. London; 2006. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 49 language and only cited in Chinese databases. Searching such databases can be challenging for non-Chinese speaking authors as it requires the use of Chinese characters. Objectives: To describe the challenges, and assess the value of searching for studies of interventions in Chinese databases based on the recent Cochrane review ‘Community wide interventions for increasing physical activity’. Methods: In the published protocol 24 databases were listed but these did not include a specialist Chinese database. One expert advised that relevant studies were undertaken in China. Subsequently five Chinese databases were searched: CAJD, CCND, CPCD, CJSS, and CDFD through http://www.global.cnki.net/. Searching, screening, translation and data extraction were assisted by a Chinese epidemiologist. Results: The search (January 1995 to December 2009) yielded 124 hits and resulted in 4 included studies that were unique to these databases. These represented 16% (4 of 25) of the review’s included studies. Furthermore, the Chinese studies represented 33% (3 of 9) of the most intensive studies included and 67% of studies from low-income countries (4 of 6). The papers tended to be brief and required contact with authors for clarification and confirmation of important information. Conclusions: The additional searches of the six Chinese databases provided further complexity to an already complex and comprehensive search strategy. The search strategy was less sophisticated and was limited by challenges in translation of search terms from English to Chinese. Despite these challenges, the strategy added further studies of value to the review which would not have been included otherwise. We recommend that, where indicated, searching of Chinese databases be undertaken. P1A22 Hard-to-reach and difficult-to-define: searching for ‘hidden’ populations. An example from public health Cooper C1 , Levay P2 , Lorenc T3 , Craig G4 , Marrero-Guillamón I5 1 Peninsula Technology Assessment Group (PenTAG), Peninsula College of Medicine & Dentistry, University of Exeter, UK; 2 Information Specialist, National Institute for Health and Clinical Excellence, UK; 3 Department of Social and Environmental Health Research, London School of Hygiene and Tropical Medicine, UK; 4 City University, London, UK; 5 Associate Research Fellow, Department of Iberian and Latin American Studies, Centre for Iberian and Latin American Visual Studies, Birkbeck, University of London, UK Background: Writing efficient search strategies for systematic reviews requires defining, a priori, the topics of a review within a search. A difficulty arises when a population or set of interventions is unclear or unknown: how best to operationalise these points in the context of the search? Objectives: This paper presents a search designed for qualitative evidence on ‘hard-to-reach’ groups at risk of tuberculosis. It explores the methods we used to construct a search filter for this ill-defined population and whether the collaborative approach we used was an effective method for defining difficult-to-define concepts. Methods: Information specialists extensively tested potential terms and themes in a variety of resources and drafted the filter through database searching, citation chasing, expert/advisory contact, grey literature searching and working collaboratively with the review team. Hard-to-define populations were then analysed as a combination of specific risk groups (e.g. prisoners) coupled with risk factors (e.g. drug use) to ensure a highly sensitive population filter. The Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. original search combined three themes (TB + Population + Qualitative terms). To test the efficiency of the filter, we re-ran our original search in Medline without the population filter. The results of the new search (TB + Qualitative terms) were screened to see if any populations were retrieved which had not been defined in the filter. Results: The new search yielded 5454 studies not located in the initial search. No additional populations were identified after screening the extra studies. The sensitive population filter appeared to capture all relevant populations, suggesting that the time invested in creating the search strengthened the conceptual understanding of the population under review. Conclusions: Dialogue between information specialists, review teams and topic experts is an effective method for operationalising poorly defined concepts. A well defined search filter benefits both the retrieval of evidence and the scope of the review. P1A23 Word frequency analysis of over a million words to support the development of search strategies on ‘health-related values’ Petrorva M1 , Sutcliffe P2 , Fulford B3 , Dale J2 1 Egenis (ESRC Centre for Genomics in Society), University of Exeter, Exeter, UK and Health Sciences Research Institute, Warwick Medical School, University of Warwick, Coventry, UK; 2 Health Sciences Research Institute, Warwick Medical School, University of Warwick, Coventry, UK; 3 Institute of Clinical Education, Warwick Medical School, University of Warwick, Coventry, UK Background: Systematic reviews increasingly incorporate or are complemented with findings from research on ‘values’, understood broadly to include ethical values, beliefs, preferences, experiences, satisfaction, quality of life, etc. Research on such issues is scattered. The vocabulary for and the boundaries of such topics are also notoriously contentious. Objectives: This study used word frequency analysis to 1) generate a broad pool of search terms, along with data on their precision and sensitivity, to support systematic review searches; and 2) develop a ‘brief values filter’ for scoping searches. Methods: Datasets of MEDLINE records on Diabetes, Obesity, Dementia and Schizophrenia were used (2004–2006; 4,440 citations; 1,110,291 words). Word frequency analysis was performed using Concordance and SPSS. Text words and MeSH terms of high frequency and high precision were compiled into a search filter. It was validated on Dentistry and Food Hypersensitivity. Results: 144 unique text words and 124 unique MeSH terms of moderate and high frequency (≥20) and very high precision (≥90%) were identified. 19 text words and 7 MeSH terms had such excellent performance parameters across at least three topics. These were compiled into a brief values filter. In the derivation dataset, it had sensitivity of 76.8% and precision of 86.8%. In the validation datasets, its sensitivity and precision were, respectively, 70.1% and 63.6% (Food Hypersensitivity) and 47.1% and 82.6% (Dentistry). Conclusions: Both ‘health-related values’ and word frequency analysis-based approaches to search filter development are areas of substantial potential. A conceptualisation in terms of health-related values may underpin a systematic and coherent picture of the psychological, social, cultural, ethical and political factors associated with a particular healthcare concern. Word frequency analysis need not only precede search strategy development: the ideal Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 50 sample for it would comprise publications from completed reviews, with findings supporting search strategy improvements for subsequent reviews. P1A24 Unique trial identification numbers are underreported in biomedical publications van de Wetering F1 , Haring T1 , Scholten R1 , Hooft L1 1 Dutch Cochrane Centre, Amsterdam, Netherlands Background: To control for selective reporting, registering each study at inception in a public register is advocated. After registration an unique registration number is assigned to each study which enables identification of future publications and increases transparency. The International Committee of Medical Journal Editors (ICMJE) therefore recommends reporting of the registration number at the end of the abstract. Objective: To asses the reporting of unique trial identification numbers in biomedical publications. Methods: We searched the Netherlands Trial Register (NTR) for all randomized clinical trials (RCTs) registered in 2005 that notified study completion by August 31, 2008. We used the unique identification number to search MEDLINE for publications. If no publication was found, the author’s name(s) or elements of the title were used to identify possible publications or we contacted the primary author. We also evaluated whether the journals in which the studies were published were following the ICMJE’s Uniform Requirements for Manuscripts (URM). Results: We included 542 RCTs. Of those, 282 (52%) were published, of which 164 (58%) had reported the trial identification number (yet not necessarily at the end of the abstract). Of the remaining 118 trials, 55 (47%) were published in ICMJE journals. Conclusions: A substantial percentage of registered trials did not report the trial registration number in the abstract of their publication. Almost half of those were published in ICMJE journals that ideally should follow their own recommendations. Hence, there seems to be a need to promote accurate reporting of trial identification numbers to enable easy identification of published reports of registered studies. This could possibly be done by adding the unique trial identification number as an item to the CONSORT checklist. P1A25 Access to ongoing paediatric drug trials in the Netherlands Hooft L1 , Weber E2 , Hamelinck V1 , Offringa M2 1 Dutch Cochrane Centre, Netherlands; 2 Academic Medical Center, Netherlands Background: As the number of paediatric clinical trials increases it is essential that their existence is publicly documented and that eventually their results will be made publicly available. Registering clinical trials can bring us one step closer to an unbiased and efficient clinical world. Objectives: To determine a) the accessibility of ongoing paediatric trial data, b) how many retrospective clinical trial registers include paediatric drug trials, and c) the ease of searching for ongoing paediatric trials. Methods: We asked paediatricians and Medical-Ethical Review Boards of all eight paediatric University Medical Centres in the Netherlands to provide a list of all known paediatric drug trials, which were planned or ongoing between September 2005 and Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. December 2008. Simultaneously, we identified Dutch paediatric drug trials through the (inter)national prospective trial registers using the WHO Search Portal of the International Clinical Trial Registry Platform (ICTRP). Results: The field identified 172 paediatric drug trials, of which 97 (56%) had been registered prospectively. A total of 299 Dutch paediatric drug trials were found searching the WHO Search Portal. Most of these (70%) were registered in ClinicalTrials.gov, 25% in the Netherlands Trial Register (NTR), and 5% in ISRCTN.org. The WHO Search Portal facilitated searching for paediatric trials, resulting in a very low number needed to screen (1.2). However, not all trials (17%) registered in the NTR were found by using the WHO Search Portal. Conclusions: There are disparities between the number of ongoing trials, the completeness of registration and the sensitivity of the current search tools. It seems that ongoing paediatric drug trials are still not in the public domain because investigators and industry declare that they don’t want the information to be publicly available. Searching the WHO Portal for paediatric trials is easy, although there is reason to believe that not all registered trials can be found. P1A26 A central repository for data extracted in systematic reviews Hadar N1 , Keefe S1 , Parkin C1 , Iovin R1 , Lau J1 1 Tufts Medical Center, USA Background: Systematic reviews are labor-intensive and many overlap in the literature reviewed. Objectives: To describe the challenges involved in creating a repository for reusable extracted data. Methods: Web-based electronic data extraction tool and repository. Results: The availability of a freely accessible Web-based repository of systematic review data will minimize the unnecessary replication of effort where the details of the studies have already been previously extracted. Additional benefits include: a promotion of transparency and reliability in the systematic review process, enhanced cooperation and utility across related resources, a more efficient means of producing and updating systematic reviews, quick and simultaneous access to related reviews, and improved access to detailed information on review evidence by various stakeholders. The following requirements guided our development of this system: 1) ease of use with a low burden of contribution; 2) flexible and adaptable to all manners of research questions, and 3) interoperable with other existing systematic review databases. The repository’s Web interface would serve both as an archive and data extraction tool. Technical challenges included designing a Web-based system that balances ease of use with the availability of additional features, like importing and exporting data in different formats. Non-technical challenges primarily concerned development of governance and data quality assurance processes. A panel of technical experts knowledgeable in trial registries, databases, systematic review methodologies, and/or bioinformatics was convened to help address these issues. Pilot tests are underway to refine this system. Conclusions: A number of technical and non-technical issues need to be solved in implementing a data repository. The success of such an archive requires close collaboration with stakeholders to create a resource that researchers want to use and that encourages participation. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 51 Figure 1 (P1A27). Cochrane Database utilization analysis. P1A27 Cochrane Database utilization analysis: A 10 year trend of one medical center in Taiwan Chen KH1 , Chen C1 , Kuo KN2 , Tzeng PC1 , Chien LT1 , Liu HE3 1 Wan Fang Hospital, Taipei Medical University, Taiwan; 2 Taipei Medical University, Taiwan; 3 Chang Gung University, Taiwan Background: The Cochrane Database is one of the most valuable medical library resources. Since 2001, Wan Fang Hospital Medical Center in Taiwan started using the Cochrane Database. Objectives: To analyze the utilization of the Cochrane Database in a medical center in Taiwan and its correlation to EBM events. Methods: 1) We obtained the monthly log-in information for the Cochrane Database from January 2001 to December 2010; 2) We collected all major evidence-based medicine (EBM) events in Taiwan over the past decade. These events were categorized as: (1) Nationwide events, (2) Organizational regulation, and (3) EBM educational activities; 3. We identified the top 10 utilization months and the EBM events held at the time. Results: December is the annual top month in utilization of the Cochrane Database. It may be related to the factor of organizational regulation, as the annual evaluation in nurse promotion requires an EBM related report (Figure 1). The top five months in utilization of the Cochrane Database were all in 2010 when there were major EBM activities, including: (1) Nationwide events including online publishing of the traditional Chinese translation of all Cochrane abstracts by National Health Research Institutes and EBM contests held by Taiwan Joint Commission on Hospital Accreditation; (2) EBM educational activities, such as Cochrane systematic review workshop. The regular Cochrane Database training lectures and hospital accreditation did not Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. significantly correlate with the utilization of the Cochrane Database. Conclusions: The key factors for increasing the utilization of the Cochrane Database in a Taiwan medical center between 2001 and 2010 were: the annual nurse promotion evaluation, the publishing of the traditional Chinese translation of Cochrane abstracts, EBM contests, and Cochrane systematic review workshop. P1A28 Locating evidence for developing countries: A case study of three public health reviews Stansfield C1 1 EPPI-Centre, Social Science Research Unit, Institute of Education, University of London Background: Identifying relevant research evidence to inform decision-making in developing countries is not always straightforward. The recent programme of reviews commissioned by the UK Department for International Development offered an opportunity to evaluate search strategies for such a purpose. Analysing where the research studies they included were found is useful to inform search strategies in future reviews. Objectives: To assess the utility of different search sources in identifying research for three rapid systematic reviews that concern public health interventions for women in developing countries. The studies in these reviews used controlled trials and other study designs. Methods: A retrospective analysis was undertaken of the search source locations of the studies that were included in the three reviews. Review management software was used to determine how each study had been identified from the research literature, including Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 52 those found in multiple locations. Comparisons were made for the search sources across the three reviews. Results: A total of 34 studies were located across a range of international medical and social science bibliographic databases, specialised registers, and other sources. Over half, N = 19 (56%), were from one source only. For all reviews, handsearching websites and contact with authors contributed over a third of studies. Data is presented for each type of search source and for controlled trials and those of other study designs, for each review and collectively. Conclusions: Information resources in this area are evolving. This analysis highlights challenges in identifying public health research applicable to developing countries. For the three reviews it was important to seek out regional and topic-specific sources, contact other researchers and handsearch websites, in addition to searching large international databases in the fields of medicine and social science. P1A29 ‘Irrelevant’ or ‘neglected’? An analysis of RCTs in the ARI Group Thorning S1 , Van Driel M1 , Del Mar C1 1 Acute Respiratory Infections Group, Australia Background: Acute respiratory infections (ARI) are commonly managed problems in general practice. The Cochrane ARI group supports 120 reviews covering a wide range of topics relevant to primary care. Topics of Cochrane reviews are based on questions relevant to clinical practice. The ARI reviews include RCTs that have been indexed in the group’s specialized register in CENTRAL. It is unclear if RCTs that have not been included in reviews are ‘irrelevant’ or simply ‘neglected’. Objectives: 1. Identify the proportion of RCTs indexed in CENTRAL included in ARI systematic reviews. 2. Assess the number of available RCTs that are ‘irrelevant’ (not useful for clinical practice) or ‘neglected’ (relevant and not yet reviewed). Methods: We selected 3 topics relevant to primary care within the scope of the ARI group. We identified the relevant Cochrane reviews. Records were retrieved from CENTRAL on each of the selected ARI topics. Each record was checked for RCT characteristics and coded. A bibliometric analysis of included RCTs and number of potentially useful RCTs will be completed. Analysis by a panel of primary care clinicians of the non-included RCTs will classify questions covered by the available RCTs as ‘relevant’ or ‘irrelevant’. Results: Bibliographic analysis is currently being conducted and results will be provided at the Colloquium in October. Conclusions: Cochrane reviews cover clinically relevant topics, but include only a small proportion of available RCTs. Cochrane reviews could play a more proactive role in identifying topics to cover in future clinical research and ensuring their clinical relevance. produce recommendations on the use of new and existing medicines, products and treatments in the NHS. The STA process is open and transparent, and involves the submission of evidence by the manufacturer or sponsor of the technology. NICE contracts independent Evidence Review Groups (ERG) to appraise and assess the evidence submission. As part of the STA process, the ERG must appraise and critique the search methods which underpin the systematic review and economic model components of the evidence submission. This project describes how an ERG Information Team tested a standardised, structured and reproducible approach to reviewing search methods in manufacturer’s submissions (MS). Although an evidence based checklist [1] exists for peer review of individual search strategies, the ERG must appraise search strategies and the clarity of search method reporting. Based on the methods described by McGowan [1], Sampson [2,3] and NICE [4], we developed a tool for the assessment of STA search methods. Objectives: To create and pilot a search appraisal checklist designed specifically for evaluating industry evidence submissions. Methods: Data were collected on types of errors made in the study identification sections of two STA manufacturers’ submissions. This evidence was incorporated to create the ASSIST checklist, which will be piloted on further STA’s for fine-tuning and evaluation with an expected project conclusion in mid-July 2011. The pilot checklist included 33 items in 10 domains. Results and Conclusions: Preliminary findings during the pilot stage have shown that implementation of the ASSIST checklist enabled standardised appraisal of search methods. The ERG identified several shortcomings in MS searches, including typographical errors, incorrectly combined line numbers, inappropriate subject headings explosion and errors in use of study design filters. The finalised checklist and evaluation will be presented. References 1. McGowan J, Sampson M, Lefebvre C. An evidence based checklist for the peer review of electronic search strategies (PRESS EBC). Evidence Based Library and information practice 2010;5(1):1–6. 2. Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol 2009;62(9):944–52. 3. Sampson M, McGowan J, Lefebvre C, Moher D, Grimshaw J. PRESS : peer review of electronic search strategies. Ottawa: Canadian Agency for Drugs and Technologies in Health (CADTH), 2008 Available from: http://www/cadth.ca/index.php/en/hta/reports-publications/search/ publication/781 4. National Institute for Health and Clinical Excellence. Single Technology Appraisal: specification for manufacturer/sponsor submission of evidence. London: NICE, October 2009, 2009. 76p. P1A30 Appraisal of search strategies in industry submissions for technology appraisal (ASSIST): Reviewing search methods of industry submissions to NICE using a structured checklist P1A31 Syncope and other adverse events associated with cholinesterase inhibitors: Systematic review of the literature versus analysis of the FDA Adverse Events Database Allen A1 , Misso K1 , Riemsma R1 , Kleijnen J1 1 Kleijnen Systematic Reviews Ltd, UK López-Arrieta JM1 1 Cochrane Dementia & Cognitive Improvement Group, Spain Background: The UK National Institute for Health and Clinical Excellence (NICE) undertakes Single Technology Assessments (STA), to Background: Cholinesterase inhibitors (CI) are commonly prescribed to treat the symptoms of Alzheimer’s disease (AD). An aging population Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 53 will increase the number of CI users, who are more prone to the adverse effects (AE) of these drugs. Many of the AE are potentially severe, such as syncope and its consequences. Until recently their AE profile has received little attention. Objectives: 1) To compare the efficiency of methods of identifying and confirming the specific association of bradycardia and related serious consequences with CI (BRSCI) in a systematic review (SR) of the biomedical literature with that of the United States Food and Drug Administration (FDA) spontaneous Adverse Event Reporting System (AERS) database; 2) To determine which method could have detected the AE earlier; 3) To determine whether differences exist in these AE between the various CI. Methods: We performed a SR and descriptive analysis of primary, secondary (reviews) and tertiary literature (critical summaries) of the composite event of: bradycardia, bradyarrythmia, pacemaker insertion, complete atrio-ventricular block and hip fracture, associated with CI. We performed an analysis of the AERS database searching for the composite event (AERSDACE), using the Bayesian ‘Multi Gamma Poisson Shrinker’ method. Results: None of the clinical trials detected a single BRSCI adverse event. 13 patients with BRSCI were identified through case reports. Three large cohort studies yielded 831 individuals with BRSCI. A total of 246 patients were identified that possessed one or more of the defined composite AE through AERSDACE. A statistically strong signal of disproportionate reporting was observed by the Empirical Bayesian Geometric Mean, 7.07 (95%CI: 6.23–8.03), detected at one year after approval. Conclusions: Publication of case reports of CI’s AE are a less efficient form of detection and epidemiological studies take many years to test a hypothesis. Analysis of large spontaneous adverse event databases is the most efficient primary source to identify non-common AE. Table 1 (P1A31). Number of events constituting the bradycardia-related composite event, with MedDRA (PT) codes. MedDRA PT code Event Number of cases 10003673 Complete atrio-ventricular block 27 10049765 Bradyarrhythmia 10006093 Bradycardia 10040741 Sinus bradycardia 10016450 Femoral neck fracture 2 10016454 Femur fracture 3 10020100 Hip fracture 10007598 Cardiac pacemaker insertion 7 168 35 12 9 P1A32 Developing a search filter for identifying primary care studies in general medical journals Roberts NW1 , Gill PJ2 , Wang KY2 , Heneghan C2 1 Bodleian Health Care Libraries, University of Oxford, UK; 2 Department of Primary Health Care, University of Oxford, UK Figure 1 (P1A32). Flow chart of gold standard selection for primary care search filter. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 54 P1A33 The rate of publication of randomized controlled trials is near-exponential (2.5 times increase/decade with >1.5 million computer listed trials). A new strategy is required to fufil the Cochrane Challenge Chow TKF1 Hill Hospital, Eastern Health, Australian & New Zealand College of Anaesthetists, Monash University, Australia 1 Box Background: The number of randomized controlled trials has increased exponentially over the past 30 years. Although the Cochrane Collaboration’s achievements have been exemplary, the Cochrane Challenge cannot be fulfilled because we are bogged down by unscientific methods of identifying quality randomized trials. This prohibits the periodic update of systematic reviews. Objectives: To seek consensus on a search strategy to fulfil the Cochrane Challenge. Methods: The search strategies recommended in the Cochrane Handbook are a combination of handsearching and electronic search. The former is time consuming, expensive and impossible to duplicate and validate the authors’ findings. The electronic search, the Optimally Sensitive Search Strategy, is a 29-line algorithm shown to be highly sensitive in identifying randomized trials but lacks specificity. (Chow TKF et al. 2004) We present a simpler, single-line algorithm: (double blind$ or random$).af. We applied this strategy to a systematic review pertaining to post-thoracotomy pain (Joshi GP et al. 2008) to demonstrate its ease of use, high sensitivity Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Figure 1 (P1A33). Number of RCTs 1980–2009. 120000 100000 80000 Number of RCTs Background: Searching and identifying primary care relevant studies in the literature is challenging. Although specialist journals exist, relevant research is often published in general medical journals. To our knowledge, search strategies for finding studies relevant to primary care have yet to be developed and validated. Objectives: To develop and validate a search filter for identifying studies of relevance to primary care in Medline. Methods: We conducted a Medline search of articles published in five core medical journals at five yearly intervals (1998, 2003 and 2008). We excluded comments, editorials, letters and news publication types. All articles were randomly allocated to one of two sets: Reviewer A screened the titles and abstracts of the first set, while Reviewer B screened the second set. In cases of uncertainty the full-text was evaluated. The gold standard dataset was randomly divided into a development set and validation set. A textual analysis was conducted on the titles, abstracts and MeSH terms of the development set to extract frequently occurring words. Test search strategies were then developed and sensitivity/specificity data were derived using the validation set. The search strategies were further tested in Medline. Results: A total of 12,045 articles were retrieved in the search. After exclusion of specified publication types the title/abstract of 9,028 records were screened and a further 495 records required full-text evaluation. The gold standard dataset included 371 articles (Figure 1). The sensitivity, specificity and precision of different search strategies will be reported. Conclusions: Using textual analysis we have developed a range of Medline search strategies to identify articles relevant to primary care. However, inadequate description of the clinical research setting, inconsistent use of terminology to describe ‘primary care’ and the overlap between primary care and the public health or outpatient setting make the construction of a highly efficient search filter highly challenging. 60000 40000 20000 0 1975 1980 1985 1990 1995 2000 2005 2010 2015 Year of Publication Table 1 (P1A33). Applying Strategy (double blind$ or random$).af to identify postthoracotomy pain relief randomized trials compared to Joshi GP et al. in November 2008 and May 2011. Searches Results 2008 1 (double blind$ or random$).af 2 exp pain/ or exp analgesia/ or exp analgesics/ 3 (pain or analge$).af 4 1 and (2 or 3) 5 (thoracotomy or thoracic surgery).af 6 4 and 5 7 Remove duplicates from 6 Results 2011 1,050,252 1,250,315 1,469,054 1,640,096 783,298 152,708 107,370 1,176 866 1,096,662 207,555 146,020 1,593 1,096 Search 1 conducted on the 25th of November 2008: EMBASE 1980 to 2008 week 47 and Ovid MEDLINE 1950 to November Week 2 2008. Search conducted 2 months after publication of Joshi GP et al. 2008 Search 2 conducted on the 6th of May 2011: EMBASE 1980 to 2011 Week 17 and Ovid MEDLINE 1948 to April Week 4 2011. in identifying quality randomized trials, high specificity in excluding non-randomised trials and its advantages to updating systematic reviews in a timely fashion. Results: Applying (double blind$ or random$).af to Ovid EMBASE/MEDLINE, over the past 30 years (1980 to 2009) demonstrated that the number of randomized trials increased at a near-exponential rate (2.5 times per decade). At the presentation of this paper, there will be over 1.5 million randomized trials listed. Adding the strategy to a post-thoracotomy pain relief algorithm, we demonstrated that comparable conclusions to Joshi’s results could have Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 55 been achieved by reviewing 866 articles. Furthermore, this review can be updated currently (6 May 2011) by reviewing an additional 230 articles. Conclusions: (double blind$ or random$).af is reproducible, highly sensitive and specific. If adopted universally, it can manage the impending information overload and fulfil the Cochrane Challenge. P1A34 Where and how to find data on safety: What do systematic reviews of complementary therapies tell us? Pilkington K1 , Boshnakova A2 1 University of Westminster, UK; 2 Bazian, UK Background: Successfully identifying relevant data for systematic reviews with a focus on safety may require retrieving information from a wider range of sources than for ‘effectiveness’ systematic reviews. Searching for safety data continues to prove a major challenge. Objectives: To examine search methods used in systematic reviews of safety and to investigate indexing. Methods: Systematic reviews focusing on safety of complementary therapies and related interventions were retrieved from comprehensive searches of major databases. Data was extracted on search strategies, sources used and indexing in major databases. Safety related search terms were compared against index terms available on major databases. Data extraction by one researcher using a pre-prepared template was checked for accuracy by a second researcher. Results: Screening of 2563 records resulted in 88 systematic reviews being identified. Information sources used varied with the type of intervention being addressed. Comparison of search terms with available index terms revealed additional potentially relevant terms that could be used in constructing search strategies. Seventy-nine reviews were indexed on PubMed, 84 on EMBASE, 21 on CINAHL, 15 on AMED, 6 on PsycINFO, 2 on BNI and HMIC. The mean number of generic safety-related indexing terms on PubMed records was 2.6. For EMBASE the mean number was 4.8 with at least 61 unique terms being employed. Most frequently used indexing terms and subheadings were adverse effects, side effects, drug interactions and herb-drug interactions. Use of terms specifically referring to safety varied across databases. Conclusions: Investigation of search methods revealed the range of information sources used, a list of which may prove a valuable resource for those planning to conduct systematic reviews of safety. The findings also indicated that there is potential to improve safety-related search strategies. Finally, an insight is provided into indexing of and most effective terms for finding safety studies on major databases. P1A35 Coding clinical trials and systematic reviews in The Cochrane Library using standard international schemas Hawkins B1 , Ervin A1 , Dickersin K2 1 Cochrane Eyes and Vision Group, USA; 2 US Cochrane Center/Cochrane Eyes and Vision Group, USA Background: Registers and databases of reports of clinical trials are valuable resources. However, their utility for retrieval is limited by variations in terminology used by different medical specialties and Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. researchers. Coding them using standard international schemas may improve access. Objectives: Our goal was to evaluate in a pilot study coding of health conditions using the International Classification of Disease (ICD-10) and interventions using the beta-test version of the International Classification of Health Interventions (ICHI) of systematic reviews and clinical trials in the Cochrane Eyes and Vision (CEVG) register of clinical trials within the Cochrane Central Register of Controlled Trials (CENTRAL). Methods: Two raters coded all 43 CEVG systematic reviews published in The Cochrane Library by March 1, 2007. After discussing disagreements and agreeing on coding conventions, one rater coded a 10% random sample of approximately 10,000 entries in the CEVG register as of January 2007. Results: Inter-rater agreement using the ICD-10 was 81% for the systematic reviews. Of 981 trial reports in the random sample, 716 (73%) could be coded using titles, and abstracts when available. ICHI was inadequate for coding interventions and was supplemented by ad hoc codes for outpatient procedures and drug classes. Conclusions: We found ICD-10 to be a satisfactory schema for coding disorders of eyes and vision. However, we are seeking an alternative to the ICHI, designed primarily for hospital procedures; the majority of participants in trials in eyes and vision are outpatients. Also, we are conducting a pilot study to evaluate automated conversion of MeSH terms to ICD-10 codes. P1A36 Improving the efficiency of searches in Cochrane Systematic Reviews Thorning S1 , Del Mar C1 , Van Driel M1 1 Acute Respiratory Infections Group, Australia Background: Search strategies have become increasingly complicated with time. This increases effort for both searching and parsing effort. We wondered whether simpler searches would reduce unnecessary ‘hits’ while preserving the core ones. Objectives: To test whether simple searches will improve specificity without compromising sensitivity. Methods: We have searched past Cochrane reviews in the acute respiratory infections (ARI) Group with two or more search strategies to test whether included studies identified in the reviews were retrieved. Results: These are presented with recommendations about expanding or contracting either the aetiological agents; the symptoms; or the disease search terms. Conclusions: These recommendations should improve the efficiency of searching and parsing titles. P1A37 Accuracy of the MeSH term ‘Breast Neoplasms’: Ten years on Tai FM1 , Willson ML1 , Ghersi D1 1 Cochrane Breast Cancer Group, Australia Background: MeSH terms are designed to categorise articles into topic areas which allows us to search with some degree of specificity. In 1999, we analysed whether the MeSH term ‘Breast Neoplasms’ was correctly assigned to records in MEDLINE. We found that the MeSH term was accurately assigned to over 85% of the retrieved records. As our group’s search strategy has since been revised, the accuracy of the MeSH term ‘Breast Neoplasms’ needed to be reassessed. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 56 Objectives: To evaluate the accuracy of the MeSH term ‘Breast Neoplasms’. Methods: We applied our search strategy to MEDLINE for trials published in 2009/10. The retrieved records were imported into Reference Manager 11 and coded by two authors. Records were categorised as whether they were related to breast cancer, indexed with the MeSH term ‘Breast Neoplasms’ and if the trial was randomised. Full manuscripts were retrieved, if necessary. Any uncertainty while coding was resolved through discussion. Results: 2000 records were retrieved (87 were duplicates). 1473 records were tagged with the MeSH term ‘Breast Neoplasms’ of which 164 records were not relevant to breast cancer but on cancer in general (121) or incorrectly tagged (43). Of the 440 records without the MeSH term ‘Breast Neoplasms’, 36 were relevant to breast cancer and on reconstructive surgery, risk factors, diagnostics and endocrine therapies. There were 501 randomised trials on breast cancer, 21 of which were not assigned the ‘Breast Neoplasms’ MeSH term. These included trials on reconstructive surgery, mammographic density and drug therapies. This indicated that the MeSH term was relatively sensitive with minor (<5%) errors. Conclusions: MeSH terms are used to identify relevant publications when searching databases. Our analysis shows that the term ‘Breast Neoplasms’ is not always accurately assigned. Systematic reviewers should be aware of the limited accuracy of MeSH terms when searching for studies. P1A38 Inconsistent quality of reporting of searching clinical trials registries in Cochrane systematic reviews and protocols Ko H1 , Tai FM1 , Ghersi D2 , Askie LM1 1 Australian New Zealand Clinical Trials Registry, Australia; 2 Research Policy and Cooperation, World Health Organisation, Switzerland Background: Searching of clinical trials registries (CTRs) for registered clinical trials has not been mandatory for systematic reviews (SRs) and protocols published in the Cochrane Database of Systematic Reviews (CDSR), however some Review Groups are encouraging this practice. In the 2010 Cochrane Colloquium, Ghersi et al and Hooft et al reported there was low searching of CTRs. Objectives: To assess the quality and consistency of reporting of searching International Clinical Trials Registry Platform (ICTRP) and/or primary CTRs, as well as other methods to identify ongoing clinical trials, in SRs and protocols. Methods: SRs and protocols that used the keywords ‘trial registry’, ‘trial registries’, ‘trial register’, or ‘trial registers’ were retrieved on 29 March 2011. Two authors independently evaluated methods sections’ reporting of searching (1) the ICTRP for ongoing clinical trials, (2) primary CTRs listed in the WHO ICTRP, and/or clinicaltrials.gov, and whether (3) other sources were used to identify ongoing clinical trials. Disagreements were resolved by consensus. Results: 414 SRs and 228 protocols were retrieved. Overall, 247 (60%) SRs and 167 (71%) protocols reported searching ICTRP and/or CTRs. Table 1 shows the breakdown of CTR searching methods reported. SRs with their latest search performed from 2008 show >70% of SRs reporting searching CTRs, with most preferring to search a variety of CTRs or searching both ICTRP and CTRs. The amount and names of CTRs used were variable. SRs that have not updated their search since 2007 have minimal or no reporting of searching CTRs. Searching other sources such as experts or pharmaceutical companies about ongoing trials was very common. Conclusions: The quality and consistency of reporting Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Table 1 (P1A38). Reporting of CTR searching. Reporting of CTR Searching SRs Protocols ICTRP only 9 21 CTRs only 172 80 Both ICTRP and CTRs 66 61 Unclear for both ICTRP and CTRs 16 17 Searched ICTRP and unclear for CTRs 0 0 Searched CTRs and unclear for ICTRP 2 0 Unclear ICTRP and no CTR 0 0 Unclear CTR and no ICTRP 0 1 of searching CTRs for ongoing clinical trials is highly variable. More explicit and consistent search protocols across all SRs and protocols are needed, as well as updating of search strategies of older SRs. P1A39 The current status of clinical research in traditional korean medicine, South Korea Jang B1 , Go H2 , Lee Y1 , Park S2 , Kim J3 , Cho KH4 1 National evidence-based healthcare collaborating agency, Seoul, South Korea; 2 Dept. of Internal medicine, Hospital of Oriental Medicine, Semyung University, Chungcheongbuk-do, South Korea; 3 Dept. of Gastroenterology, Hospital of Oriental Medicine, Kyung Hee University, Seoul, South Korea; 4 Dept. of Cardiovascular and Neurologic Diseases (Stroke Center), College of Oriental Medicine, Kyung Hee University, Seoul, South Korea Background: The healthcare system in South Korea comprises of two systems: one is Western medicine and the other Traditional Korean Medicine (TKM). TKM has been criticized for uncertain evidence. Objectives: To examine the current status of clinical research in TKM and to assess ‘risk of bias’ (ROB) in randomized clinical trials (RCTs) in TKM. Methods: Korean Oriental Medicine Society (KOMS), a special committee for EBM, reviewed 17 journals related to TKM in South Korea (from the first issue to May 2010), the Cochrane Central Register of Controlled Trials (The Cochrane Library Issue 2, 2010), and PubMed (1966 to May 2010). TKM clinical trials reported by traditional medical doctors were included. Data were extracted from included articles on objectives, design, setting, participants, interventions, main outcome measures, main results and conclusions, using a predetermined form of structured abstract. We selected eligible TKM RCTs and assessed ‘risk of bias’. Results: We reviewed 12,653 articles from the 17 journals and 41 articles from CENTRAL and PubMed. 1,004 articles were left after non-clinical articles were excluded. Of these, 306 were TKM eligible studies. Of these studies, 57 were on musculoskeletal disorders and 55 on the circulatory system. 133 of the 306 TKM eligible studies were RCTs, of which 69 were on acupuncture and 25 on herbal medicine. We found a large proportion of ‘unclear’ risk of Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 57 Figure 1 (P1A39). bias in the criteria of ‘Allocation concealment’, ‘Blinding of participants and personnel’, ‘Blinding of outcome assessment’ and ‘Other bias’. On the other hand, we found a large proportion of ‘low’ risk of bias in the criteria of ‘Incomplete outcome data’ and ‘Selective reporting’. Conclusions: Clinical research in TKM is insufficient and many studies focused on musculoskeletal disorders and the circulatory system. There may be risk of bias in terms of ‘allocation concealment’ and ‘blinding’. For high-quality research in TKM, further research is needed with randomized and blind RCTs. the medical literature and for specialty areas and diseases the scatter follows the pattern predicted by Bradford’s law. The scatter has implications for both clinicians trying to keep up to date and for those performing systematic reviews. P1A40 How does the number and scatter of trials vary by discipline? Chen C1 , Chen KH1 , Kuo KN2 , Tzeng PC1 , Chien LT1 , Lo HL2 , Tam KW3 1 Wan Fang Hospital, Taipei Medical University, Taiwan; 2 Taipei Medical University, Taiwan; 3 Taipei Medical University Hospital, Taiwan Glasziou P1 , Thorning S1 , Erueti C1 , Hoffmann T1 1 CREBP, Bond University, Qld, Australia Background: The National Health Research Institutes in Taiwan has translated abstracts of the Cochrane Database of Systematic Reviews (CDSR) into traditional Chinese since 2008. This is the only contracted translation activity of CDSR abstracts in East Asia. During the past two years, more than 800 participants have joined the translation task. The translated Chinese Abstracts of CDSR (CACDSR) were open to the public online since 2010. (http://clc.nhri.org.tw/admin/clcmain1.aspx). Objectives: To analyze the utilization of CACDSR in three university hospitals in the past eight months. Methods: 1. Obtain the monthly users’ login information of CACDSR in three university hospitals from September 2010 to April 2011. 2. Analyze the trend of utilization among physicians, nurses, and other healthcare professionals. 3. Collect information on evidence-based medicine (EBM) events in these hospitals over the data collecting period. Results: 1. The utilization of CACDSR among nurses has increased significantly in the study period. The overall trend showed that the hit number in nurses is significantly higher than that of the other healthcare professionals. 2. The total hits of CACDSR from nurses are 9 times higher than that from doctors. When promoting a database, nurses are unnegligible. 3. The peak months of utilization had EBM activities at the time, such as an EBM workshop, an EBM contest and evidence-based quality improvement activities in the hospitals. These events may have stimulated the utilization in CACDSR. 4. The lowest month of utilization, i.e. Jan. 2011, can be explained by the influence of the long vacation around Chinese New Year. Conclusions: The study indicates the strong demands for Chinese translated databases while promoting evidence-based practice and continuous education for nursing professionals in Taiwan. EBM Background: Bradford’s law of scatter suggests that if the journals in a field are sorted by number of articles and divided into 3 equal sized groups the number of journals will be in the proportions 1:n:n2 . Bradford’s Law has been shown to apply to trials overall but little work has been done for specialties of diseases. Objectives: To assess whether and how Bradford’s law applies to trials and reviews overall and in different specialty areas. Methods: We downloaded all trials (using publication type) in MEDLINE for 2009, and classified trials in each of 14 clinical areas and specific disorders that are major contributors to the global burden of disease: mental disorders, depression, alcohol related disorders, heart disease, myocardial ischemia, nervous system diseases, cerebrovascular, dementia, otorhinolaryngologic, hearing loss, endocrine disorders, diabetes, cancer and lung cancer. Results: The number of journals covering all trials in an area was highly correlated with the number of trials (r = 0.97), but varied by clinical area. Depression had most with 318 trials in 214 journals – 1.5 trials per journal. Cardiology had least scatter with 1,459 trials in 374 journals – 3.9 trials per journal. The number of journals to identify half the trials in a condition ranged from 9 for lung cancer trials to 79 for stroke trials. The journal with the highest number of trials in each area was generally a journal within that speciality, e.g. Stroke for cerebrovascular disease and Diabetes Care for diabetes. However, general medical journals and journals in related subspecialties often occurred in the top 10. Conclusions: Trials are widely scattered in Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P1A41 Utilization analysis of Chinese abstract of the Cochrane Database of Systematic Reviews in three university hospitals Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 58 Figure 1 (P1A41). CACDSR monthly utilization and trend among health professionals. workshops, EBM contests, and evidence-based quality improvement activities may accelerate the utilization of evidence-based databases. P1A42 Development of search strategies to identify reports of off- label drug use in MEDLINE Mesgarpour B1 , Müller M1 , Herkner H2 1 Department of Clinical Pharmacology, Medical University of Vienna, Vienna, Austria; 2 Department of Emergency Medicine, Medical University of Vienna, Vienna, Austria Background: It has been reported that a large number of medications are used routinely for unapproved indications or in incorrect dosages, routes of administration or age groups. These are so-called ‘off-label’ uses. However, the efficacy and safety of off-label drug usage have not been sufficiently proven. Objectives: Because of variation in the description of using medications beyond their approval and diversity in study type, the off- label drug use studies are not easy to detect in electronic literature databases. This research aimed to develop a search strategy that facilitates finding the off- label drug use studies in a bibliographic database, Ovid MEDLINE. Methods: MEDLINE via OvidSP was searched from 1948 to 20 January 2011. Search queries, including search words and strings, were conceived based on definitions of off-label use of medications. The sensitivity and specificity of search queries and their combinations as well as the subject heading ‘Off-Label Use.sh.’ were determined by comparison Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. with a gold standard reference set. The gold standard was compiled from reports of off- label drug use identified by a sensitive search of MEDLINE and EMBASE. We attempted to achieve the highest possible sensitive search strategy and an optimal balance of sensitivity and precision. Results: Out of 4334 retrieved documents, 2177 (50.2%) off- label drug use studies were found. The most sensitive single term was ‘off label*.af.’ (sensitivity 40.9%, precision 84.4%). A top sensitive strategy was revealed by combining 31 search queries with sensitivity of 53.3% and precision of 60.3%. Maximizing precision, a top search strategy achieved a value of 84.0%, but with compromised sensitivity at 49.0%. Conclusions: We empirically developed two versions of highly sensitive search strategies, which can achieve high performance for retrieving off-label drug use documents in Ovid MEDLINE. P1A43 Developing software for combining search queries to design efficient search strategies Mesgarpour B1 , Mesgarpour M2 , Müller M1 , Herkner H3 1 Department of Clinical Pharmacology, Medical University of Vienna, Vienna, Austria; 2 School of Mathematics, University of Southampton, Southampton, United Kingdom; 3 Department of Emergency Medicine, Medical University of Vienna, Vienna, Austria Background: Diagnostic test methods usually have been applied in studies with search strategies development purpose to evaluate the Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 59 search terms and search strategies. These methods compile a set of relevant studies as the ‘‘gold standard’’ and determine the sensitivity, specificity, precision and accuracy of searches. If the gold standard has been produced by following the sensitive search in bibliographic databases, the sensitivity and precision of search queries and strategies have been evaluated and compared. Objectives: This study aimed to build and develop a computational program for combining search queries to design a highly sensitive search strategy, as well as an optimal sensitive and precise one. Methods: An algorithm has been introduced to measure the sensitivity and precision of search terms and their combinations with purpose of finding the most sensitive combination of search terms. Then, a C++ code is developed to capture user-defined search terms and their dependencies and constraints to derive optimized combinations. In order to test and improve the program, our former data on developing search strategies have been assessed continuously to identify reports of off- label drug use in MEDLINE and EMBASE via OvidSP. Results: Two versions of the software, SenPrecOptim, have been developed. They automatically perform calculations regarding the sensitivity and precision of search queries and all their possible combinations. The software outputs the optimized ranked list of search queries combinations based on the sensitivity in a .csv file. The best balance of sensitivity and precision strategy of optimized outputs can be obtained by finding the best-fit line on the scatter plot of sensitivity versus precision. Conclusions: The evaluation of search queries combinations can be enhanced by utilization of this software. The output of this yields the best-balanced sensitive and precise search strategies. P1A44 Identification of biomedical journals in Spain and Latin America Bonfill X1 , Bonfill A1 , José E1 , Garcia M2 , Raiher S2 , Aranda E3 , Rada G4 , Rincón D5 , Tristán M6 , Torres A7 , Hidalgo R8 , Simancas D8 , López L9 , Jiménez C10 , Piña M10 , Correa R11 , Rojas A12 , Loza C13 , Féliz E14 , Gianneo O15 , Patrón C15 , Martı́ A16 , Posso M1 , Osorio D1 Iberoamerican Cochane Centre and Network (1 Spain; 2 Argentina; 3 Bolivia; 4 Chile; 5 Colombia; 6 Costa Rica; 7 Cuba; 8 Ecuador; 9 Guatemala; 10 México; 11 Panamá; 12 Paraguay; 13 Perú; 14 R. Dominicana; 15 Uruguay; 16 Venezuela) Background: It is difficult to identify the existing journals belonging to a particular area. In Latin America, there are some specific databases (LILACS, SciELO, Latindex) which are very useful but many journals are not indexed and therefore not easy to detect by other means. Objectives: To identify all biomedical journals published in Spanish, both in Spain and Latin America in order to analyse and describe their main features. Methods: Biomedical journals that may publish original clinical research papers were selected. One or more people affiliated with the Iberoamerican Cochrane Network assumed the responsibility of identifying the respective national journals included into the specialised databases (Medline, LILACS, SciELO, Latindex), publishers’ catalogues or available at libraries. For each journal identified, the following characteristics were collected: name of the journal; ISSN; activity in 2009 (yes / no); period of activity; publisher; databases in which it is indexed; medical specialty. Further criteria were applied at the coordinating centre, in consultation with reviewers, for increasing consistency. Results: 2145 biomedical journals published in Spanish that met the inclusion criteria in 2009 were identified. After keeping apart Mental Health and Nursing and refining data, the distribution of Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. included journals (948) by country is: Argentina 155, Bolivia 85, Chile 90, Colombia 57, Costa Rica 16, Cuba 43, Ecuador 32, El Salvador 6, Spain 253, Guatemala 2, Honduras 3, Mexico 100, Nicaragua 0, Panama 4, Paraguay 5, Peru 32, R. Dominicana 7, Uruguay 18, Venezuela 40. Results will be presented by country, specialty, and database and activity period. Conclusions: Many biomedical journals are publishing clinical studies in Spanish although only a small proportion is indexed in bibliographic databases. A detailed analysis of collected data will permit making useful recommendations. Also, electronic and hand searches will be more efficient and comprehensive for identifying published randomised clinical trials and systematic reviews. P1A45 Systematic Review Service (SRS). China Xia J1 1 Cochrane Schizophrenia Group, UK Background: Cochrane groups have a problem with Chinese literature. China has modernized scientific research and the number of trials has hugely increased in the last decade (McDonald, 2006). There are problems with access and quality. Access is limited. For example, only 1% of trials in one area of health care are found on PubMed (Chakrabarti, 2007). China has five overlapping but distinct major biomedical databases. Collectively, they index thousands of Chinese biomedical journals (Xia, 2008) and these are largely ignored by Cochrane reviewers. While the quantity of Chinese papers has increased dramatically, quality is questionable (Lancet 2010). Quality assurance and governance must and will be improved, but until that researchers would have to manually select good Chinese studies from other less high quality work. If work from the whole of China is not to be disregarded – a systematic but clearly parochial approach contrary to the Collaborations’ founding principles – Cochrane reviewers must be helped identify, sift, quality assure and data extract work from mainland China. Conclusion: This work can only be undertaken efficiently by trained researchers fluent in Mandarin, scientific methods and Cochrane methodology. SRS (http://www.srs-org.com) is established by a team with a track record in production and maintenance of complex Cochrane reviews. SRS can i. help identify trials relevant to a specific review or area of interest; ii. select relevant trials for inclusion; iii. data extract; iv. work within RevMan to allow data to be used (or not used); v. provide full translation service (often unnecessary); vi. contact authors for additional information. Each service can stand alone or be integrated into a package. SRS will allow consideration of previously inaccessible trial data from China within Cochrane reviews and, by that, facilitate improvement of both reviews and trials. References Chakrabarti, A., Adams, C.E., Rathbone, J., Wright, J., Xia. J., Wong, W., Von Reibnitz, P., Koening, C., Baier, S., Pfeiffer, C., Blatter, J., Mantz, M. & Kloeckner, K. (2007). Schizophrenia trials in China: a survey. Acta Psychiatrica Scandinavica. Vol 116(1), pp. 6–9. The Lancet. (2010). Scientific fraud: action needed in China. Lancet. Vol 375(9709) pp. 94. McDonald, J. (2006). OECD: China to spend $136 billion on R&D. Business Week. Available from: http://www.businessweek.com/ap/ financialnews/D8LQ0OI00.htm (accessed 6 December 2006). Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 60 Xia, J., Wright, J. & Adams, C.E. (2008). Five large Chinese biomedical bibliographic databases: accessibility and coverage. Health Informationa and Library Journal. Vol 25(1), pp. 55–61. P1A46 Handsearching respiratory conference abstracts: a comparison with abstracts identified by an EMBASE search Stovold E1 , Hansen S2 1 Cochrane Airways Group, St George’s, University of London, UK; 2 Cochrane Airways Group, New Zealand Background: Conference abstracts make an important contribution to the Airways Group Register (AGR). The abstracts of key national and international respiratory conferences are manually handsearched for reports of trials. Identifying these abstracts manually is time consuming. The EMBASE database now includes conference abstracts from relevant respiratory meetings and these are being identified by the weekly EMBASE searches run for the AGR. Objectives: To compare abstracts found using the Airways Group EMBASE search strategy with the abstracts found by handsearching and determine if an EMBASE search would be sufficient in the future. The Chest 2010 meeting abstracts were used for the comparison. Methods: The Chest 2010 conference abstracts were handsearched via the Chest website. Each individual abstract was read and assessed for study design and the citation was downloaded for submission to the CENTRAL database. Those abstracts relating to the scope of the Airways Group were tagged and added to the Airways Group Register of Trials. When the weekly search alerts from EMBASE were downloaded, the Chest 2010 abstracts that had been identified by the Airways Group search strategy were extracted. They were manually checked to see if all the abstracts relating to the scope of the Airways Group already identified through handsearching were present. Results: The EMBASE search identified 151 abstracts from Chest 2010. Handsearching identified 47 RCT/CCTs, 34 of which were suitable for the AGR. A comparison of the two sets found 7/34 missing from the EMBASE set. Five were trials in pulmonary hypertension, and 2 of the missing abstracts were trials in asthma. Conclusions: The addition of conference abstracts to EMBASE is a welcome development, but handsearching the major respiratory conference abstracts is still worthwhile in view of the missing references. P1A47 Ankle brachial index to predict cardiovascular events in primary health care, a systematic review Galvão ALC1 , Stein AT2 , Pellanda LC3 , Marcadenti A4 1 Grupo Hospitalar Conceição; 2 Grupo Hospitalar Conceição / UFCSPA / ULBRA; 3 Instituto de Cardiologia – Fundação Universitária de Cardiologia / UFCSPA; 4 Grupo Hospitalar Conceição / Cardiology postgraduation course UFRGS Background: Cardiovascular diseases still remain the most frequent cause of death in many countries, despite all measures and efforts that have been made to control risk factors and therapeutic approaches. Approximately half of all coronary deaths are not preceded by cardiac symptoms. In addition near 50% of all patients who have acute Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. coronary syndrome are stratified, by risk score models, as just being an intermediate risk patient. The prevention of such events remains a serious public health challenge. Many previous studies have shown that the measurement of the ankle brachial index (ABI) may improve the accuracy of cardiovascular risk prediction. Objectives: The aim of this review is to provide an evidence-based approach to show how the measurement of the ABI, as an indicator of atherosclerosis, can improve risk assessment and predict cardiovascular events beyond the traditional used risk scores. Methods: The ABI is the ratio of systolic pressure measured by doppler at the ankle to that in the arm. Risk prediction using the Framingham risk score (FRS) was also reviewed. A search of MEDLINE, Scielo, Cochrane Library, LILACS and EMBASE was conducted using common text words for the term ankle brachial index combined with primary health care and cardiovascular risk. A review of reference lists and conference proceedings wasperformed. We have identified relevant studies published about this topic and have been doing a critical appraisal of the articles selected. Studies for inclusion were independently assessed by 2 collaborators and disagreement was resolved by a third reviser. Results: Preliminary results have shown that this question has not been completely replied in the literature and it is quite important for public health policies on this issue. Conclusions: Measurement of the ABI in addition to other cardiovascular risk score models can stratify patients more accurately and also may improve the accuracy of cardiovascular risk prediction. P1A48 Geographic and language distribution of trials in Cochrane Systematic Reviews related to acupuncture Tung Y1 , Hou M1 , Chen Y1 1 Changhua Christian Hospital, Taiwan Background: In the Cochrane Library, the systemic reviews of trials should be as global as possible and with no limitation of language or geography. When the topic is related to acupuncture, some basic diagnostic criteria of acupuncture must be included. Objectives: To study whether the Cochrane Database Systematic Reviews (CDSR) places language and geographical restrictions on searching for acupuncture, and whether the trials used the diagnostic criteria of ‘pattern identification’ in their researches. Methods: We searched ‘acupuncture’ published in the CDSR, years from 2009 to 2011 were all included. We first examined the published country and whether any language restrictions existed. Then we examined the reviews descriptions and whether the pattern identification was mentioned on their researches. We excluded the studies without direct relevance to acupuncture. Moxibustion, transcutaneous electrical nerve stimulation and Chinese herb were excluded. Results: There are 56 results out of 6,602 records for: ‘acupuncture’ in Record Title. One of the reviews in CDSR had imposed language restrictions in Chinese. The other review had language restrictions in English and Chinese. There are 25 reviews had no language restrictions. And the remaining reviews had unclear restriction on language. We also noticed all 56 studies are from two continents and 7 countries. There is one review from Germany, one from Norway, three from Canada, six from USA, and eight from Australia. With literal inspection on the type of participants, all of the included trials in 56 studies processed without basic diagnostic criteria of pattern identification. Conclusions: There appears to be most of the reviews do not have any language restrictions Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 61 on searches. And all the reviews are from very few countries. It reminds us there might not have enough diversity to show the real world. Besides pattern identification is not addressed enough, which would reduce the accuracy of the reviews for acupuncture. P1A49 Drugs costs before and after bariatric surgery: A systematic review Stein AT1 , Lopes EC2 , Heineck I3 , Athaydes G4 , Souto KP5 , Meinhardt NG5 1 Grupo Hospitalar Conceição/Ufcspa/Ulbra, Brazil; 2 Epidemiology Postgraduation Course, Ufrgs-Brazil; 3 Pharmaceutical Science Postgraduation Course, Ufrgs-Brazil; 4 Pharmaceutical Course, Ufrgs-Brazil; 5 Obesity Unit Grupo Hospitalar Conceição Background: Surgical treatment has shown to be the only effective method for keeping the weight loss for a long period in morbidly obese people. The surgical procedure requires long-term monitoring, both nutritional and pharmacologic. Objectives: To review the impact of bariatric surgery on drug costs. Methods: We have used the following MESh terms: obesity, bariatric surgery and drug costs, and have searched in nine databases (including Medline, Embase and Cochrane Library), until January 2011. In addition, reference lists were reviewed and contacted experts to identify additional published and unpublished references. Inclusion criteria: adults, BMI >35 kg/m2, drug use and comparision of costs before and after surgery. Articles were reviewed by two independent reviewers and differences were resolved by consensus. Results: The search identified 501 citations, and ten articles were identified as potentially relevant. Three articles were excluded, one that compared cost between obese and nonobese patients, another one in which compared total costs in health, although, cost had not been considered and the last one compared drug costs between surgical therapy and conventional therapy. The studies included in this review were cohorts, ranging from 50 to 965 patients. The systematic review has shown a decrease on drug expenditure between pre and post-operative, which was statistically significant (p < 0.05). Although, one of the studies had found an increase of 32.1% in expenditure after a 6-year monitoring. Conclusions: Even though, the studies that were reviewed have demonstrated cost reduction with regards to the use of drugs, the time framework in the postoperative period is short. The study that has followed patients for a longer period of time showed a different outcome in relation to others, ie an increase in drug spending after surgery. P1A50 The updating of clinical practice guidelines: A systematic review of the research evidence Martı́nez Garcı́a L1 , Arévalo-Rodrı́guez I2 , Solà I1 , Hemens B3 , Vandvik PO4 , Haynes B3 , Alonso-Coello P1 1 Iberoamerican Cochrane Centre, Biomedical Research Institute Sant Pau (IIB Sant Pau), Barcelona, Spain; 2 Clinical Research Institute, GETS, Universidad Nacional de Colombia, Bogotá, Colombia; 3 McMaster University, Hamilton, Canada; 4 Norwegian Knowledge Centre for the Health Services, Oslo, Norway Background: Scientific knowledge is in constant change. The flow of new information requires a frequent revaluation of the available Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. research results. Clinical practice guidelines (CPGs) are not exempted from this phenomenon and need to be kept updated to maintain the validity of their recommendations. Objectives: To identify, describe and compare methods for updating CPGs. Methods: Selection criteria: We included methodological research studies evaluating: 1) Strategies to evaluate if CPGs are out of date; 2) Strategies to update CPGs; and 3) Living CPGs strategies. Search strategy: The search strategy included MEDLINE (March 2011) and The Cochrane Methodology Register (the Cochrane Library Issue 3 2011). There were no restrictions concerning language or type of publication. Data collection and analysis: Authors independently selected studies and extracted data. The results are presented narratively due to differences in methodology between the studies. Results: The review included eight studies. Most of them described a method for updating one or more CPGs, except one study that applied it on 6 topics and another on 15 recommendations. Four studies described strategies to evaluate if CPGs were out of date, three studies described strategies to update CPGs and one study described a living CPG strategy. Five studies described a single method and three studies compared two methods. Regarding the literature searches for new evidence, those evaluating whether guidelines were out of date or those designed for the use in living guidelines applied more limited approaches. On the other hand strategies addressed to update CPGs included more exhaustive literature searches. Conclusions: There is scarce and heterogeneous research about the updating of CPG. The development of efficient strategies to identify new relevant evidence is needed to improve the timeliness and reduce the burden of updating CPGs. P1A51 Uncommon language: The challenges of inconsistent terminology use for evidence synthesis Parker RM1 , Hayden JA2 1 Capital District Health Authority, Canada; 2 Dalhousie University, Canada Background: Varied use of terminology around a common theme creates barriers to communication and presents obstacles to systematic literature retrieval. Concepts such as baseline risk and prognosis may be expressed in a variety of ways depending on the context including time, geographic region, and research discipline. Bias may be introduced into systematic reviews on methods and epidemiology topics if search terms used do not include all the possible ways of conceptualizing the topic. Conversely, inclusion of terms whose meanings vary depending on the context will impact the specificity of a search. Objective: Explore the context distribution and potential impact of different health concept terminology in peer-reviewed literature using the examples of baseline risk and prognosis concepts. Methods: For each concept, MEDLINE citations and the full-text of general medical and specialty journals will be searched using a set of terms identified from the literature and by consulting experts. Frequency of use for each term will be assessed in different contexts. Additionally, MEDLINE search results will be semantically analyzed to plot the frequency of citations containing the terms of interest across time, region, and discipline. Context analysis of term usage within full text and citations will be conducted using text mining of a corpus identified by the searches. Results: An initial search related to the concept of baseline risk (including absolute risk, overall prognosis, natural history, etc.) showed that the relative frequency of terms vary depending on the Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 62 journal. Semantic analysis of MEDLINE citation results also found discipline-specific term preference. Prognosis-related terms will be searched to assess terminology variability around another complex concept. Conclusions: The use of terms varies between disciplines and regions of practice. Complete results including context analysis and impact will be presented. P1A52 A snowballing technique to ensure comprehensiveness of search for systematic reviews: A case study Vedula S1 , Mahendraratnam N1 , Rutkow L1 , Kaufmann C1 , Rosman L2 , Twose C2 , Dickersin K1 1 Johns Hopkins Bloomberg School of Public Health, USA; 2 Welch Medical Library, Johns Hopkins University, USA Background: Increasingly, trial data are becoming available through internal company documents made accessible through litigation. We performed a systematic review of how such documents had been used in the past and how they were identified. Systematic reviews, such as ours, that address complex questions pose a challenge in terms of searching for potentially eligible studies. Objective: The objective of this study was to evaluate a ‘snowballing’ technique to iteratively revise a search strategy used for a systematic review of studies using data from internal industry documents. Methods: We identified a set of 19 ‘index’ articles that were eligible for inclusion in our systematic review. From this set of ‘index’ articles, we developed an initial search strategy for PubMed and Embase using a combination of indexing terms and keywords. After de-duplicating abstracts retrieved from the two databases, two reviewers independently screened the titles and abstracts, and subsequently the potentially eligible full text articles, against pre-specified eligibility criteria. We will examine the indexing terms and text words for all studies categorized as eligible and revise our initial search strategy using additional index terms and text words. We will compare the results of our ‘snowball’ approach to a ‘reference standard’ to estimate the recall and precision of our approach. Results: The initial strategy retrieved 7061 results in PubMed and 2244 results in Embase. Independent screening of titles and abstracts and full-text articles is in progress. Findings related to our analyses of recall and precision will be described in our presentation at the Cochrane Colloquium. Conclusions: Searches for systematic reviews addressing complex questions using a ‘snowballing’ approach could be tested for systematic reviews addressing other types of questions. P1A53 Searching far and wide for occupational prevention literature: Where do we go to find relevant studies? Van Eerd D1 , Mahood Q1 , Slack T1 1 Institute for Work & Health, Canada Background: The Institute for Work & Health (IWH) initiated a four-year pilot project to conduct reviews on the effectiveness of interventions to prevent workplace injury, illness or disability. As part Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. of the pilot the IWH has undertaken an examination of the methods of searching for and reviewing the prevention literature. One method of evaluating our searches is to analyze where studies come from to improve future search strategies and reviews. Objectives: To determine which bibliographic databases yielded the highest number of included studies in the area of occupational health and safety reviews and add to the research on health care review methodology. Methods: A retrospective analysis was done across 14 reviews, spanning a six year time interval, to identify which bibliographic resources yielded relevant studies for each review. In addition we will conduct a sensitivity and specificity analysis of our searches. Results: Preliminary results, based upon 9 reviews, indicate that those databases that typically yield the higher numbers of included studies in clinical reviews i.e. Medline and Embase, also yielded similar proportions in the occupational health and safety literature. Not surprisingly the results across 12 somewhat diverse bibliographic databases indicate the importance of searching multiple databases. No single database yielded all of the relevant studies. Data will be presented for each source and for individual reviews in order to determine the sensitivity and specificity of each. This analysis will represent over 100,000 studies from 12 different bibliographic databases. Conclusions: Our analyses will lead to more sensitive and specific search strategies in the area of occupational health and safety. To conduct a comprehensive systematic review on the effectiveness of interventions to prevent workplace injury, illness or disability it is important to include multiple databases. However, to capture relevant prevention studies it is necessary to screen many irrelevant studies. P1A54 Identification and analysis of search strategies and filters carried out in systematic reviews indexed in Lilacs. Overview Puga MEDS1 , Pereira RA1 , Figueiró MF2 , Melnik T1 , Carvalho APV1 , Riera R1 1 Brazilian Cochrane Centre, BR; 2 Institute of Education and Research Hcor-BR Background: The search methodology group from Brazilian Cochrane Centre has worked with the resources and tools available in the new LILACS (Latin-American and Caribbean Health Sciences Database) website (IAX format). This format displays the search results organized according to the study design and there are 1,218 systematic reviews (SR) – including Cochrane and non-Cochrane – currently available. Objectives: To identify and assess the search strategies and filters carried out in all systematic reviews indexed in the new Lilacs’ Database website. Methods: Through an independently way three researchers are assessing the methods section from each one SR aiming to identify: a) the search strategies and filters used for clinical trials, b) the grade of sensitivity and specificity of these filters, and c) the database and search terms (MeSH, DeCS or synonymous) used by the SR authors. Results: The results will be available in June 2011, but as potential findings, the researchers speculate that most of the reviews do not use LILACS database. Also, it is expected that synonymous are used instead of MeSH terms in non-Cochrane SR. Conclusions: The conclusions will be available in June 2011. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 63 P1A55 Title-abstract versus title-only citation screening strategies for systematic reviews and meta-analyses Mateen F1 , Oh J1 , Tergas A1 , Bhayani N1 , Kamdar B1 1 The Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA Background: Systematic reviews are foundational for evidence-based care, but require a substantial time investment by the authorship team. There are currently no data or guidelines regarding the most effective strategy for screening citations before full-text review. Objective: To compare title-abstract and title-only strategies for screening citations for a systematic review. Methods: A five-member group of physicians performed a systematic review and meta-analysis on light at night and breast cancer risk using Medline and Embase (January to March 2011). Group members were assigned in a round-robin fashion to two reviewer pairs. Each reviewer assessed whether citations included 1) primary data, 2) the exposure of interest, and 3) the outcome of interest. A titles-only screening strategy was undertaken first, followed by a combined titles-abstract strategy on the same citation list. Outcome measures included inter-rater agreement and reliability and articles ultimately included for review following both screening strategies. Results: 2965 records were screened following exclusion of duplicate citations. Agreement/reliability between the five reviewer pairs for a title-only screening strategy was 89–94% with kappa = 0.54, with 575 records remaining for abstract review. Agreement/reliability for a title-abstract screening strategy was 96–97% with kappa = 0.56, with 257 records remaining for full text review. The final systematic review and meta-analysis included 13 articles, all of which were identified by both screening strategies. Conclusions: Compared to titles-abstracts strategy, screening titles-only resulted in a higher number of disagreements but had a similar inter-rater reliability and did not lead to exclusion of any articles used in the final analysis. Initial screening via a titles-only approach may save considerable time in the systematic review process. Editorial processes and supporting review authors Day 20 Oct (P1A56–P1B80) P1A56 International activity within Cochrane Review Groups Allen C1 , Clarke M2 , Richmond K1 1 The Cochrane Collaboration Secretariat, Oxford, United Kingdom; 2 Centre for Public Health, Queen’s University Belfast, United Kingdom Background: Since 2000, a periodic count has been done of members of Cochrane Review Groups (CRGs), subdivided by country of residence. Objectives: To assess international activity within CRGs. Methods: Data were retrieved on 12 April 2011 for all active contributors to CRGs whose contact details were in The Cochrane Collaboration’s contact database (Archie). One author (CA) analyzed the data, by role within the CRG and country of residence. Unlike previous years when individuals were counted repeatedly each time they appeared in the audit, people listed more than once are now counted once only. This Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. made comparisons to absolute numbers in earlier years difficult, hence the focus on percentages in the following results. Results: Among 53 CRGs, there were 24 228 individual contributors from 118 countries. Of these, 20 246 (84%) contributors were listed as authors, which is a higher proportion than that estimated for recent years (when it was usually around 75%), and much higher than in earlier years (e.g. 52% in 2000). There are now 4618 contributors in countries with low- and middle-income economies (19% of all contributors, compared to 6% in 2000). Of these 4618 people, 4346 people were listed as authors (94%). Conclusions: The international diversity of people involved in the work of The Cochrane Collaboration continues to grow, and more than 24 000 individuals are contributing to its work. For many of these people, this involves direct involvement in the preparation and maintenance of systematic reviews, ensuring the continuing success of the Collaboration as the world’s largest producer of systematic reviews in health care. The growth in the contribution of people from low- and middle-income countries has been sustained, with more than 4000 authors in these countries. Other data analyses will be shown in the poster. The data will be available at http://www.cochrane.org. P1A57 Experiences of undertaking a diagnostic test accuracy review with the Cochrane Oral Health Group Walsh T1 , Littlewood A1 , Liu J2 1 University of Manchester; 2 University of Dundee Background: Oral cancer has been identified as a high priority area of research for the Cochrane Oral Health Group (COHG). Oral cancer is the sixth most common cancer globally, with incidence increasing worldwide. External funding from the National Institutes of Health (USA) to members of editorial staff of the COHG enabled four intervention reviews in the treatment of oral cancers and screening programmes for the early detection and prevention of oral cancer to be carried out. A diagnostic test accuracy (DTA) review for screening and diagnosis of oral cancer was identified as a priority review for the group. Objectives: To document the issues encountered by the review team and COHG editorial staff in undertaking their first DTA review. Methods: Editorial staff received training on the methods and processes involved for the production of DTA reviews. A multidisciplinary review team comprising clinicians, a statistician and a methodologist was formed to write the review, with input from an expert panel of oncologists. At least one review author was required to attend four DTA workshops. Results: Formulating the research question was difficult, as substantial prior clinical and methodological knowledge was required at the outset. Standard diagnostic practice was not always ‘standard’; index tests could be quite esoteric but still needed to be specified a priori; reference standard differed between comparisons. A scoping literature search and guidance from the Trials Search Co-ordinator were essential. From scoping two separate but related research questions emerged. Moving from basic analysis required detailed statistical knowledge and input. Detailed knowledge of study design (usually not randomised controlled trials) and clinical experience of target disease were needed to properly evaluate methodological quality. Conclusions: In our experience, DTA reviews are more difficult than intervention reviews, but not impossible. They require a very clear understanding of the nature of the problem and current clinical practice, and commitment from a multidisciplinary team. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 64 P1A58 Epidemiology, quality and reporting characteristics of randomized controlled trials on traditional Chinese medicine for gastric cancer published in Chinese journals Ma B1 , Ke Y2 , Su X2 , Xian L3 , Li Y2 , Yang K2 1 Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China; 2 Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China 2 The First Clinical School of Lanzhou University, Lanzhou 730000, China; 3 Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China 3 The School of Public Health, Lanzhou University, Lanzhou 730000, China Background: Randomized Controlled Trials (RCTs) of traditional Chinese medicine (TCM) have been produced in large numbers. But the epidemiological characteristics, reporting detail, scientific validity is unknown. Objectives: To provide the comprehensive information and guide the development of RCTs on TCM in China. Methods: Chinese Biomedicine Literature Database, Chinese Scientific Journal Full-text Database and Chinese Journal Full-text Database were searched to December 2010. Search terms included ‘stomach neoplasms’, ‘Traditional Chinese Medicine’ and ‘RCTs’ etc (MESH or EMTREE). Data were collected using an Excel form designed to examine the epidemiological and reporting detail. The CONSORT checklist was used to appraise the reporting quality. Results: A total of 102 randomized controlled trials (RCTs) published in 72 periodicals were included. Only 41.67% were published in specialty journals, 21.57% were included in Chinese Science Citation Database as cited by journals. Two studies reported informed consent. Two studies stated ‘randomization’ in title. 52.94% and 11.76% of trials reported the specific inclusion and exclusion criteria respectively. None of the studies stated the method of sample size calculation. Only 17 trials reported how the random allocation sequence was generated. A participant flow chart was not described in any of the trial reports. Only two articles stated the limitations of the study. None of the studies reported registration and protocol. Fund sources were not reported in 82.35% trials. Conclusions: A large number of RCTs were published in China, but the reporting information was insufficient and the scientific validity worrying. We suggest three measures to improve the quality of clinical trials in China. Firstly, enhancing the clinical trials registration policy. Secondly, the Chinese Medical Journal should endorse the CONSORT statement in their ‘Instructions to authors’. Finally, medical schools should introduce a registration policy and reporting guidelines as soon as possible. of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China; The First Clinical School of Lanzhou University, Lanzhou 730000, China; 3 Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China; The Second Clinical School of Lanzhou University, Lanzhou 730000, China; 4 Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China; Ministry of Health of the People’s Republic of China, Beijing 10044, China Background: A large number of randomized controlled trials (RCTs) in pediatrics have been published in China. However, there have been no systematic assessments whether the reporting quality has improved since the CONSORT checklists were introduced in China in 1997. Objectives: This study was undertaken to evaluate the reporting quality of RCTs published in pediatric journals in China. Methods: Six core pediatric journals were searched from inception through December 2010 which included Journal of Clinical Pediatrics, Chinese Journal of Contemporary Pediatrics, Chinese Journal of Practical Pediatrics, Chinese Journal of Evidence-based Pediatrics, Chinese Journal of Pediatrics, and Chinese Journal of Pediatric Surgery. The CONSORT checklists were used to assess the trial reporting quality. Data was collected using a standardized form. Analyses were performed using Excel and SPSS. Results: A total of 619 RCTs were identified, in which 157 RCTs were published before 1997. Compared to literature published before 1997, there was significant improvement of quality in some aspects of the trial reporting such as: title and abstract, introduction, trial design, statistical methods, recruitment, baseline data and ancillary analyses (P < 0.05). However, some important methodological components of RCTs were still incompletely reported such as: sample-size (0.63% vs.1.08%), sequence generation (3.18% vs. 7.58%), allocation concealment (0% vs. 1.08%), and blinding (0% vs. 0.87%), same as in funding (2.55% vs. 4.55%) and protocol (0% vs. 2.16%). Besides, no trials reported the registry. Conclusions: Overall, the reporting quality of RCTs in pediatrics has changed significantly in most of the aspects we analyzed since the introduction of CONSORT checklists in China; however, there is room for improvement in some aspects such as methodology components, registry, protocol and funding. Therefore, despite the advancements observed, there are still several areas that can be improved in pediatric RCTs. P1A60 Reporting standard for qualitative research: A good or a bad idea? Preliminary results from a Delphi study P1A59 Changes in randomized controlled trial reporting quality over time: A systematic review of pediatric journals from China Hannes K1 , Heyvaert M1 , Slegers K2 , Van den Brande S3 , Vandewiele W4 , Van Houdt S5 , Van Nuland M5 1 Centre for Methodology of Educational Research, K.U.Leuven, Belgium; 2 Centre for User Experience Research, K.U.Leuven, Belgium; 3 Interdisciplinary Centre for Religious Studies, K.U.Leuven, Belgium; 4 Criminal law and Criminology, K.U.Leuven, Belgium; 5 Academic Centre for General Practice, K.U.Leuven, Belgium Ma B1 , Li H2 , Liu W1 , Zhang Y3 , Hu Q1 , Zhang Z4 1 Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China; 2 Evidence-Based Medicine Center, Institute Background: For experimental, diagnostic and longitudinal research, standards have been developed with criteria guiding a researcher in how to report the research procedure and findings, in order to increase the quality of reporting and to facilitate the critical appraisal Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 65 of methodological quality. There have been attempts to develop such a standard for qualitative research as a result of individual, academic endeavour. However, the criteria identified have not been considerably taken up by researchers conducting basic qualitative research. This might be related to the variety of different paradigms and the broad range of designs, data-collection and analysis techniques that one could opt for when conducting qualitative research. Objectives: To explore the possibility of developing a consolidated standard for qualitative research. In the potential absence of a consensus on a general standard, to identify particular qualitative methodologies and/or approaches for which reporting guidance should specifically be considered or might be counter-productive. In case of agreement on the need for reporting criteria for particular designs, to reach consensus on specific reporting criteria. Methods: A Delphi study will be conducted to try and reach consensus between an international panel of experts selected from a broad pallet of approaches or designs. Approximately 25 experts will contribute to this study. A multidisciplinary team of researchers will assist in initiating discussion and analyzing data. Results: We will report the findings from the first Delphi-round. This round includes expert opinions related to the evaluation of the potential benefits, feasibility, appropriateness, potential effectiveness and meaningfulness of general standards of reporting for qualitative research. Depending on the progress of the research, we will also report on the selection of methodologies that are in need of specific guidance in reporting. Both issues will feed into the potential final round focusing on reporting criteria for particular designs. P1A61 Obtaining funding for IPD meta-analyses: Top tips for a successful application Rydzewska L1 , Tierney J1 , Vale C1 1 IPD Meta-analysis Methods Group, UK Background: Individual participant data (IPD) meta-analyses are usually more time and resource intensive than conventional summary data reviews; but benefit from improved data quality and analysis and are considered to be the ‘gold standard’ of systematic reviews. However, members of the Individual Participant Data Meta-analysis Methods Group (IPDMAMG) have highlighted difficulties in obtaining funding for these IPD projects. Objectives: To collate positive and negative feedback received from funders and provide tips for submitting new funding applications. Methods: We surveyed all 70 members of the IPDMAMG, asking them to provide details of negative and positive feedback received from both successful and unsuccessful funding applications. Feedback was collated and appraised to identify key themes according to the frequency of particular responses. Based on these identified themes, a list of ‘top tips’ was compiled. Results: We received responses relating to 18 IPD funding applications: nine successful, eight failed, and one ongoing at the time of the survey; and comments from funders were wide-ranging. As for any project; the originality, relevance and potential impact are important; the aims and methods need to be clearly stated and the group applying should either have appropriate experience or have formed a collaboration to achieve this. However, perhaps less obvious, is the need to clearly explain clearly why IPD are required, for example if summary data are insufficient to address the question, together with an indication of how likely it is that the IPD will be obtainable. Moreover, as IPD reviews do Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. cost more than standard reviews, ample justification for the resource requested is also vital. Conclusions: Surveying the members of the IPDMAMG provided a valuable resource for those seeking funding for IPD projects, which we hope will improve the chances of a successful application. This resource is available by contacting [email protected]. P1A62 Helping readers understand review update status: Pilot of a simple system Sinclair D1 , Garner P1 1 Cochrane Infectious Diseases Group, UK Background: The Cochrane Library’s unique selling point is that it contains up-to-date reviews. However, this is not always the case: some reviews address historical questions; in some reviews the weight of evidence is such that no further trials are justified; and in others, the authors are in the process of updating the review. It is not immediately explicit to The Cochrane Library readership what the update status of a review is. Objectives: To develop a clear, simple, transparent classification system for reviews to help readers understand whether our reviews are up-to-date or relevant. Methods: We established a classification system based on two parameters: relevance and status. For relevance we identified two options: historical question, where an intervention or policy had been superseded by new medical developments (such as a new drug), and a current question, which was still relevant to current policy or practice. Thus a review that was no longer relevant to current practice was labelled as ‘Historical: no update intended’. For reviews with a current question, we then used three status categories: ‘up to date’, ‘update pending’, or ‘no update intended’. There then follows a slot, which allowed the author or editor to provide a little more detail. For ‘update pending’ we might have given the number of trials waiting assessment and refer people to the ‘trials waiting assessment section’. For ‘no update intended’ we had two categories: ‘review findings conclusive’, and ‘topic covered in another review (see xxxx review)’. Results: We have applied this to over 80 of our reviews and are currently inserting this in our published reviews. We will provide examples to illustrate how this works. Conclusions: It is worth other groups considering adopting this approach. P1A64 A tool to improve efficiency and quality in the production of protocols for Cochrane Reviews of Diagnostic Test Accuracy Pennant M1 , Wisniewski S1 , Hyde C2 , Davenport C1 , Deeks J1 , on behalf of the Cochrane DTA ET3 1 The University of Birmingham, UK; 2 The University of Exeter, UK; 3 Cochrane Diagnostic Test Accuracy Review Unit, UK Background: The Cochrane Diagnostic Test Accuracy Editorial Team (DTA ET) organises methodological peer review of DTA protocols and reviews. In the first part of this process, submitted protocols are sent for peer review by DTA statisticians, search specialists and clinical epidemiologists, and discussed by the DTA ET. Feedback to authors highlights methodological issues that need to be addressed before the review can proceed. Over 50 protocols have passed through this system, and patterns of common problems have emerged. Objective: To review common errors in protocols for DTA reviews in Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 66 order to develop a structured tool to assist: authors, Cochrane Review Groups, and reviewers in preparing and reviewing submitted reviews. Methods: Areas of methodological weakness, considered important to the integrity of reviews of DTA, were identified by reviewing reports of peer reviewers and the DTA ET from the first submissions for a sample of 45 protocols. We reported on the nature and frequency of these problems, and produced a structured tool to help identify these problems, that links to the relevant DTA Handbook sections. Results: Important areas of DTA review methodology were identified as: definition of the research question (particularly describing the alternative diagnostic pathways), choice of reference standard, design of the search strategy, quality assessment of the included studies and the statistical methods for meta-analysis. Work continues to further evaluate the comprehensiveness of protocol submissions against these criteria in the sample of 45 first protocol submissions. Conclusions: This tool is likely to be helpful for both authors and DTA ET protocol assessors, making the DTA protocol development and assessment process more transparent, efficient and robust. P1A65 UK prioritisation methods for Cochrane Reviews Bailey SE1 , Stamp RJ1 , Clarke M2 1 NIHR Evaluations, Trials and Studies Coordinating Centre, Southampton, England; 2 Centre for Public Health, Queen’s University Belfast, Northern Ireland Background: The National Institute for Health Research (NIHR) is a strong supporter of The Cochrane Collaboration in the UK. With a planned investment of £21 million over five years up to March 2015, it is the largest single contributor to the infrastructure costs of Cochrane entities. However, despite this financial support, the potential review workload is always likely to be greater than the resource available to support it, creating the need to prioritise review topics. Objectives: To share methods for prioritising review topics as reported to their funders by UK-based Cochrane review groups. Methods: UK-based Cochrane review groups submit an annual report of expenditure and review production during each year. As part of this report, groups are asked to provide details about their prioritisation methods. Their answers have been grouped in to broadly consultative and analytical approaches. Results: The most frequently cited approaches to review topic prioritisation will be presented. This information should complement the results of other work in The Cochrane Collaboration, such as the projects commissioned by the Steering Group to develop prioritisation methods, and may be valuable to Cochrane Review Groups considering how best to utilise limited resource and capacity. Conclusions: Prioritisation of review topics at title stage and for updating is essential to ensure that limited resource and capacity are used in ways that best address key questions faced by patients, front-line professionals and by national decision-making bodies and other policy customers in health care. P1A66 Improving the participation of women from developing countries in the Cochrane Collaboration: A pilot study Koehlmoos TP1 , Nasser M2 , Kiwanuka S3 1 ICDDR, B, Bangladesh; 2 Peninsula Dental School, United Kingdom; 3 Makerere University School of Public Health, Uganda Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: There is increased interest around women’s involvement in the Cochrane Collaboration (CC). Broader participation is one of the principles of the Collaboration and increasing women’s involvement is one way to achieve this goal. Objectives: We aimed to determine the facilitators who are based in or hail from developing countries (DC), who facilitate women getting involved with the Collaboration. Methods: We developed a questionnaire and used snowball sampling to survey women from developing countries involved in the Cochrane Collaboration. Results: Nine women responded to our survey. They were from Iran, Bangladesh, USA, Argentina and Uganda and are based in the UK, USA, Bangladesh, Uganda and Iran. All of them were authors of Cochrane reviews. Other roles that they have filled were: centre staff, handsearcher, and referees. They have been involved with the CC two to six years. Only one of them had an influential role in the CC; however, the rest would be interested in taking a more influential role in the CC. They became involved either through a local initiative or another Cochrane contributor. The factors they saw as facilitating greater women involvement included: (a) Increasing opportunities for women to participate and network (e.g. engaging them in review teams with more equal treatment) (b) Scholarship for women to attend the Colloquium and other relevant events (c) Increasing training opportunities (d) Providing opportunities at a local level and building toward wider participation. (e) More long term and sustainable funding opportunities in developing countries. Conclusions: This pilot study provides a picture of the view of women from developing countries who are widely involved in the Cochrane Collaboration. We recommend that the Collaboration would more strategically involve these women in planning wider participation in the Cochrane collaboration. We will report the results from other women who will respond in October 2011. P1A67 From Barcelona to Madrid: History and quality of update reporting of Cochrane Reviews flagged as updates in 2003 and analysed for the Barcelona Colloquium Bastian H1 , Clarke M2 , Doust J3 , Glasziou P4 1 National Center for Biotechnology Information, National Library of Medicine, NIH, USA; 2 Professor/Director of all-Ireland Hub for Trials Methodology Research, Centre for Public Health, Queen’s University Belfast, Northern Ireland; 3 Professor of Clinical Epidemiology, Centre for Research in Evidence Based Practice, Bond University, Australia; 4 Professor of Evidence-Based Medicine, Director, Centre for Research in Evidence Based Practice, Bond University, Australia Background: Cochrane reviews are intended to be updated periodically, generally every two years. The Cochrane Handbook requires that reports of updates help readers to quickly and clearly identify the changes. Objectives: To assess the quality of update reporting and map the updating history of Cochrane reviews flagged as updated in 2003. Methods: Publication and updating histories of a cohort of reviews with update flags in 2003 (reported on at the Barcelona Colloquium) were compiled from The Cochrane Library (Issue 4, 2011) and Pub Med. Analyses were done of the time to each update. Results: Of 177 Cochrane reviews with update flags in 2003, 40 have been updated only once (23%). Seven (4%) have been determined to have stable evidence. If a review is assumed to be out-of-date after two years, only three (1.7%) of the reviews have been up-to-date ever since their original publication, and one of these was withdrawn in 2005. Based on an updating interval of two years, the 177 reviews were out Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 67 of date for 45% of their publication lives. If the interval was three years, the reviews were out of date for 28% of their lives; or 15% if the interval was four years. The median time to an update was between two and three years. A minority of the reviews reported all searches (33%) or whether additional studies were included (39%) in each update in their ‘What’s new’, history or notes section. Conclusions: Even among Cochrane reviews that had been updated at least once, most are not updated within two years or provide information in accordance with the recommendations in the Cochrane Handbook. The quality of reporting of the updating is poor. P1A68 Report on The Cochrane Empty Review Project Consensus Meeting: A proposal of empty review reporting guidelines and an invitation for feedback Montgomery P1 , Yaffe J2 , Shepard L2 , Hopewell S3 1 University of Oxford, UK; 2 University of Utah, USA; 3 University of Oxford and UK Cochrane Centre, UK Background: There is currently limited guidance in the Cochrane Handbook for the reporting of systematic reviews with no studies eligible for inclusion. This project has previously reported considerable inconsistency in the reporting of these so-called empty reviews across reviews and review groups. The Cochrane Empty Review Project convened an interdisciplinary consensus meeting of Cochrane Collaboration contributors in June 2011. The purpose of the meeting was to formally discuss the issues of empty reviews and to consider guidelines for the reporting of these reviews. Objectives: The aim of this presentation is to report the outcomes of The Cochrane Empty Review Project Consensus Meeting, and to invite further correspondence and feedback. Methods: Following the suggestions of Moher et al 2010 for the development of reporting guidelines, The Cochrane Empty Review Project identified the need for reporting guidelines for reviews without included studies by: reviewing the literature, identifying all empty reviews within the Cochrane Database of Systematic Reviews (CDSR), describing these reviews, examining how these reviews report implications for practice, and consulting with other systematic review groups to find existing guidelines for reporting empty reviews. This information was then presented for discussion at a formal consensus meeting. Results: The key proceedings from the June 2011 consensus meeting are presented in this oral presentation. Topics include: possible recommendations for the incorporation of the guidelines, the updating of empty reviews, the referencing of excluded studies, and the reporting of the implications for practice and research sections. Conclusions: The open discussion of Cochrane contributors regarding these proposed recommendations is invited; preparatory to the submission of the initial report and guidelines to the Cochrane Collaboration and the Cochrane Handbook Committee. P1B69 The Cochrane Empty Review Project: Prevalence and characteristics of systematic reviews without included studies in The Cochrane Database of Systematic Reviews Shepard L1 , Yaffe J1 , Montgomery P2 , Hopewell S3 1 University of Utah, USA; 2 University of Oxford, UK; 3 UK Cochrane Centre and University of Oxford, UK Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: Little is known about systematic reviews that report no included studies. These so-called ‘empty reviews’ have been primarily discussed by clinicians and policy-makers expressing frustration with the lack of implications for decision-making in these reviews. Objectives: The Cochrane Empty Review Project sought to describe the incidence, prevalence, and characteristics of systematic reviews without included studies in the Cochrane Database of Systematic Reviews (CDSR). Methods: Empty reviews within the CDSR as of 15 August 2010 were identified, extracted, and coded for analysis. We examined the numbers and proportions of empty reviews, date of first publication, and recency of the last update by Cochrane Review Group (CRG). Number of excluded studies, studies awaiting assessment, and on-going studies listed within empty reviews were also examined. Results: The Cochrane Empty Review Project identified 376 empty reviews in The CDSR, representing 8.7% of all active reviews. Forty-five (86.5%) of the 52 CRGs sustain at least one empty review, ranging from 0 to 35 (0 to 26.9%) per CRG. The average time since first publication is 4.2 years (SD 3.4). These empty reviews were updated within an average of 2.8 years (SD 2.2), with a range of 0 to 11 years since update. The identified reviews listed an average of 9.6 excluded studies (SD 14.5), but varied from 0 to 124 excluded studies listed. Empty reviews typically contained few, if any, studies awaiting assessment or on going studies, with respective averages of 0.1 (SD 0.68) and 0.14 (SD 0.49) per review. Conclusions: Reporting and updating of empty reviews appears to vary widely. This may or may not be explained by differences in editorial policy or practice across CRGs. Further analysis is warranted and development of more explicit guidance for the reporting of systematic reviews without included studies in The CDSR is recommended. P1B70 A comparative analysis of systematic reviews published in Chinese academic journals and the Cochrane Library Chen Y1 , Yao L2 , Liang F2 , Wang Q2 , Li J2 , Wei D2 , Xu H2 1 Evidence-Based Medicine Center of Lanzhou University, China; 2 School of Basic Medical Sciences of Lanzhou University, China Background: Should we conduct more than one systematic review for the same topic or question? How many systematic reviews are similar between Chinese systematic reviews and Cochrane systematic reviews? Does it make sense for identified Cochrane systematic reviews that some authors still conducted the same systematic review? Objectives: To detect the degree of similarity of systematic reviews between Chinese academic journals and the Cochrane Library. Methods: We searched four of the main Chinese medical databases WANFANG, China National Knowledge Infrastructure/Chinese Academic Journals full text Database (CNKI), VIP (a full text database of China) and China Biomedicine Database (CBM); using the term ‘systematic review’ or ‘meta analysis’ in the title from 1990 to 2010. Results: We identified 2648 Chinese systematic reviews about healthcare. We conducted a comparative analysis of these systematic reviews with all Cochrane systematic reviews using the PICO method. We found that about 20% of Chinese systematic reviews were very similar to Cochrane systematic reviews. Most of them were published after the corresponding Cochrane systematic review. Further investigation is ongoing. Conclusions: We will draw a distribution map between Chinese systematic reviews and Cochrane systematic reviews and identify blank and duplicate systematic reviews. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 68 P1B71 Review or original article? The manuscript category of systematic review and meta-analysis in high-impact biomedical journals Tam K1 , Tsai L2 , Kuo KN3 , Chen C3 , Lo H3 1 Department of Surgery and Evidence-based Medicine Center, Taipei Medical University Hospital, Taiwan. Center of Evidence-based Medicine, College of Medicine, Taipei Medical University, Taiwan; 2 Evidence-based Medicine Center, Taipei Medical University Hospital, Taiwan; 3 Center of Evidence-based Medicine, College of Medicine, Taipei Medical University, Taiwan Background: Unlike a narrative review, a systematic review involves the application of scientific strategies, in ways that limit bias, to the assembly and critical appraisal of all relevant studies that address a specific clinical question. A meta-analysis is a type of systematic review that uses a statistical strategy for assembling the results of several studies into a single estimate. However, when an author submits a systematic review and meta-analysis to journals, the manuscript category between a review and original article is indistinct. Objectives: To investigate the manuscript category of systematic reviews and meta-analysis in biomedical journals. Methods: Biomedical journals (impact factor >6) that consider systematic reviews and meta-analyses in the field of clinical sciences for publication were included. The Instructions to Authors of biomedical journals and the article category printed on the front page of the literature were reviewed for evidence of an editorial policy on the manuscript category. Results: 63 of 311 biomedical journals publish systematic reviews and meta-analyses of clinical issues. In the Instructions to Authors, 4.76% classified a systematic review and meta-analysis as an original article, 15.9% as a review, 20.6% as an independent type of manuscript, and 58.7% did not mention any policy on the article type for systematic review and meta-analysis. For the article category posted at the front page of the literature, 31.7% printed systematic reviews and meta-analyses as an original article, 9.52% as a review, 4.76% as a meta-analysis, and 39.7% did not reveal the article type on the front page. Conclusions: Most of the high-impact clinical biomedical journals did not mention their policy on classification of systematic reviews and meta-analyses in the Instructions to Authors. However, a relatively large proportion of journals recognize a systematic review and meta-analysis as an original article. P1B72 Evaluation of data-synthesis in systematic reviews of studies on outcome prediction models van den Berg T1 , Heymans MW1 , Leone SS2 , Vergouw D3 , Hayden JA4 , Verhagen AP5 , de Vet HC1 1 Department of Epidemiology and Biostatistics and the EMGO Institute for Health and Care Research, VU University Medical Centre, Netherlands; 2 Department of General Practice and the EMGO Institute for Health and Care Research, VU University Medical Centre, Netherlands; 3 Department of General Practice and the EMGO Institute for Health and Care Research, VU University Medical Centre, Netherlands; 4 Department of Community Health and Epidemiology, Dalhousie University, Canada; 5 Erasmus Medical Centre, Department of General Practice, Netherlands Background: Many prognostic models have been developed. Different types of models, i.e. prognostic factor and outcome prediction models, Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. serve different purposes, and this should be reflected in how the results of systematic prediction reviews are summarized. Objective: To investigate how authors of reviews synthesize and report the results of primary outcome prediction studies. Study selection: 41 systematic reviews of outcome prediction studies in English listed in MEDLINE between October 2005 and October 2009. Two review authors, using standardized forms for all reviews, independently assessed characteristics of the reviews and the primary studies that were included. Results: The type of primary studies included (prognostic factor or outcome prediction) was unclear in two-thirds of the reviews. A minority of the reviews reported uni-variable or multi-variable point estimates and measures of dispersion from the primary studies. Moreover, the variables considered for outcome prediction model development were often not reported, or were unclear. In most reviews there was no information about model performance. Quantitative analysis was performed in nine reviews; and 40 reviews assessed the primary studies qualitatively. In both analyses a range of different methods was used to present the results of outcome prediction studies. Conclusion: Different methods are applied to synthesize primary study results. Quantitative analysis is rarely performed, the nomenclature and reporting of primary studies is poor, and performance parameters of the outcome prediction models are rarely mentioned. Therefore, there is much room for improvement in reviews of outcome prediction studies. P1B73 Editorial policies of hematology and oncology journals: an analysis of author instructions Meerpohl JJ1 , Wolff RF2 , Grambauer N3 , Antes G4 , von Elm E5 1 German Cochrane Center, Institute of Medical Biometry and Medical Statistics & Pediatric Hematology & Oncology, Center for Pediatrics and Adolescent Medicine, University Medical Center, Freiburg, Germany; 2 Kleijnen Systematic Reviews, York, UK; 3 Institute of Medical Biometry and Medical Statistics, University Medical Center Freiburg, Germany; 4 German Cochrane Center, Institute of Medical Biometry and Medical Statistics, University Medical Center, Freiburg, Germany; 5 Cochrane Switzerland, CHUV and University of Lausanne, Switzerland & German Cochrane Center, Institute of Medical Biometry and Medical Statistics, University Medical Center, Freiburg, Germany Background: Complete and accurate reporting is a prerequisite for inclusion of original studies in systematic reviews. The continued debate about ethics and quality of biomedical publishing has led to recommendations aiming to enhance reporting quality and good publication practice. However, it is unclear to what extent these recommendations have been implemented by specialty journals. Objectives: We analyzed whether specific recommendations were included in author instructions of hematology and oncology journals and whether their endorsement was associated with journal characteristics. Methods: We identified 167 unique journals in the subject categories ‘Hematology’ and ‘Oncology’ of the 2008 Journal Citation Report publishing original research articles. From March to May 2010 we extracted information regarding endorsement of the Uniform Requirements for Manuscripts (URM) of the ICMJE and of five major reporting guidelines such as the CONSORT statement, disclosure of conflicts of interest and trial registration from the journals’ online author instructions. Two investigators collected data independently. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 69 We calculated a global mixed model for endorsement of editorial policies accounting for heterogeneity by publisher, followed by separate multivariate logistic regression models. Results: The URM were mentioned in the author instructions of 89 journals (53.3%). Endorsement of reporting guidelines was low: CONSORT was referred to most frequently (42 journals; 25.1%); each of the other four reporting guidelines were mentioned in less than 10% of author instructions. 132 journals (79%) explicitly required authors to disclose conflicts of interest, and 55 (32.9%) recommended or required trial registration. After model selection, only journal impact factor had a highly significant positive association with endorsement of these policies in the individual models (see Table 1). Conclusion: Author instructions of specialty journals serving the hematology and oncology research community do not yet reflect specific recommendations aiming to enhance the quality of biomedical publications. Journal impact factor is positively associated with endorsement of these policies. since publication of the original review, 41 new eligible studies were identified in addition to the eight included in the original review. When comparing endorsing and non-endorsing journals, items such as sequence generation, allocation concealment and participant flow were reported better in those endorsing CONSORT. Further details of the comparison between endorsers and non-endorsers as well as between trials published before and after CONSORT publication (both 1996 and 2001) will be presented. Conclusions: This review will provide further evidence on whether CONSORT is effective at improving the reporting of RCTs. This information will be helpful to authors, peer-reviewers and journal editors in helping to decide whether to endorse CONSORT. P1B75 A case report on updating Cochrane Reviews Nicola Maayan Enhance Reviews Ltd, UK P1B74 The influence of CONSORT on the quality of reporting of RCTs: An updated systematic review Turner L1 , Moher D1 , Shamseer L1 , Weeks L1 , Peters J1 , Plint A2 , Altman D3 , Schulz K4 1 Ottawa Hospital Research Institute, Ottawa, Canada; 2 Children’s Hospital of Eastern Ontario, Canada; 3 University of Oxford, UK; 4 FHI, USA Background: The Consolidated Standards of Reporting Trials (CONSORT) statement was developed in response to concerns about the quality of reporting of randomized controlled trials (RCTs). It is an evidence-based minimum set of recommendations for reporting RCTs, intended to facilitate complete and transparent reporting and aid in critical appraisal and interpretation. A 2006 systematic review examining the effectiveness of CONSORT for improving the reporting of RCTs in endorsing journals (i.e. those which, at minimum, recommend that authors use CONSORT), found CONSORT endorsement to be associated with better quality of reporting, despite poor methodology of some included studies. Five years on from the publication of that review, an update is needed. Objective: To update the systematic review of CONSORT effectiveness by Plint et al. Methods: Conventional systematic review methods employed in the original review were followed. The search for new comparative studies evaluating the quality of reporting of RCTs spanned August 2005 to March 2010. Two reviewers independently screened studies for eligibility; data extraction and study validity assessments were conducted by a single reviewer and verified by a second reviewer. Results: In the five year period Background: Updating reviews is a challenge faced by all review groups within The Collaboration. Many groups have tried to create mechanisms to help authors update their review, but the challenge of updating increases as more Cochrane reviews are published every year. Objectives: We report on a partnership between the Cochrane Schizophrenia Group (CSzG) and a small company specialising in systematic reviews (Enhance Reviews Ltd) on a project to update various reviews. Methods: The CSzG set up a list of priority reviews to be updated. Our team worked on tasks involved in updating the review, in consultation with the authors. Results: Six reviews were updated over seven months; and the updates differed on their level of complexity, with some more challenging than others. One challenge involved updating changes in the methods of systematic reviews, in particular new areas such as: Risk of Bias, Summary of Findings and GRADE. A further challenge was that it was difficult in some reviews to measure the amount of work involved; sometimes it was necessary to return to the original studies to complete tasks, such as updating the Risk of Bias Tables to the new format. The project allowed us to calculate the amount of work required according to the number of included studies and tasks involved. Conclusions: Given these challenges, and based on our experiences, we suggest that there should be an evaluation process before embarking on an update to estimate the scale of the tasks involved. Updates should involve partnership with authors so they can indicate which tasks would be most helpful to them in supporting the update of their review, and allow them to deal with specific issues in their field, such as the discussion. We encourage editorial teams to act as an intermediary between authors and those performing the update. Table 1 (P1A73). Association of journal impact factor with editorial policies. Editorial policy URM Any reporting guideline Disclosure of COI Trial registration Odds ratio per change in log IF 2.01 7.17 3.25 4.51 95% CI [1.28; 3.32] [3.32; 15.52] [1.80; 6.41] [1.87; 11.79] Odds ratio for change of IF from 1 to 2 1.62 3.92 2.27 2.84 95% CI [1.18; 2.30] [2.30; 6.69] [1.50; 3.63] [1.54; 5.53] p-value <0.01 <0.001 <0.001 <0.01 COI – Conflicts of Interest; JIF – Journal Impact Factor; URM – Uniform Requirements for Manuscripts of ICMJE. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 70 P1B76 When should systematic reviews of effects restrict studies to those conducted in low- and middle-income countries: A proposal? Munabi-Babigumira S1 , Lewin S1 , Glenton C1 , Oxman A1 1 Norwegian Knowledge Centre for the Health Service, Norway Background: The Norwegian satellite of the Cochrane Effective Practice and Organisation of Care (EPOC) group produces reviews that address health system questions relevant to low- and middle- income countries (LMICs). It is not always clear if review authors should restrict searches to LMICs or focus on the global evidence. Objectives: To suggest issues to consider when deciding whether or not to restrict systematic reviews to LMICs. Methods: Reasons to restrict the focus of a review to LMICs were developed through discussions among the EPOC editorial team. Results: Suggested reasons to restrict a review to LMIC are: i) The intervention(s) and/or problem that the review addresses is highly relevant in LMIC and of little or no relevance in high-income countries (HICs), and; ii) There are compelling reasons to believe that the problem or the intervention(s) or the outcomes of interest are different in LMICs and HICs, and the intervention(s) would be expected to function differently in LMICs and HICs, so that the evidence would be unlikely to be transferable between LMICs and HICs. Reasons that do not adequately justify restricting the focus of an EPOC review to studies from LMICs include: i) The interventions are common in LMICs and uncommon in HICs, but there are not compelling reasons to believe that the problem or the interventions or the outcomes of interest are different in LMICs and HICs, and the intervention(s) would not be expected to function differently in LMICs and HICs; and ii) The researchers or those who commissioned the review are particularly interested in evidence from LMICs. Conclusions: When proposing a review that is restricted to evidence from LMIC, authors need to justify why. Further work is needed to examine whether there are differences in effects for similar interventions implemented in LMICs and HICs. P1B77 Using program theory to understand heterogeneity in systematic effectiveness reviews Cargo M1 , Stankov I1 , Thomas J2 , Hannes K3 , Saini M4 , Mayo-Wilson E5 , Rogers P6 1 University of South Australia; 2 Evidence-based Policy and Practice Information Centre; 3 K.U. Leuven; 4 University of Toronto; 5 Oxford University; 6 Royal Melbourne Institute of Technology Background: Program theory can provide reviewers with a structured approach for outlining the action model(s) (i.e., characteristics of implementer, implementing system, ecological context, partnering organisations, participants, intervention), change model(s) describing the intended sequence of proximal to distal outcomes in participants, and identifying relevant process evaluation variables (i.e., fidelity, dose). Integrating this information can strengthen attributions of effectiveness and inform knowledge translation. Objectives: To assess the extent to which and how reviewers account for aspects of the action model, process evaluation measures and change model(s) underpinning their reviews. Methods: Twenty-seven systematic effectiveness reviews Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. on social welfare, criminal justice and education published by the Campbell Collaboration (some co-registered with Cochrane) with reported outcomes on children or youth were reviewed. An assessment tool, informed by a program theory approach, was developed to assess elements of the action model, change model and process evaluation measures within reviews. Two raters completed extractions using a 7-point rating scale and noting comments in EPPI-Reviewer Version 4.0. Results: Within the action model, the majority of reviews accounted for one or more aspects of the intervention (i.e., format) and participant characteristics with aspects of the implementing organisation, implementer and partnerships considered less often. Overall, action models could have been more clearly defined. Intervention dose was the most common process evaluation measure. For the change model, most reviews were ‘black box’ with change processes remaining undifferentiated. Reviewers expressed interest in identifying active ingredients of interventions and integrating qualitative evidence. This presentation will feature exemplars to illustrate how program theory can be integrated into reviews to understand heterogeneity. Conclusions: Program theory can add value to reviews by informing the review objectives, inclusion/exclusion criteria, search strategies, and the operationalisation/analysis of variables related to the action model(s), change model(s) and process evaluation. Resources, training, and guidance are needed to support reviewers. P1B78 Analysing the growth in impact of the Cochrane Database of Systematic Reviews (CDSR) Pentesco-Gilbert D1 , Stewart G1 1 Wiley-Blackwell, John Wiley & Sons, Chichester, UK Background: The Impact Factor is calculated and published as part of the Journal Citation Report by a commercial company, Thompson ISI, each year in June. The Cochrane Database of Systematic Reviews (CDSR) received its first Impact Factor in 2007. In a recent article in The Lancet Richard Horton noted, ‘In 2007, CDSR had an impact factor of 4.65 (14th place in the general medicine category, the same group as The Lancet ). In 2008, it rose to 5.18 (12th ). In 2009, it rose again to 5.65 (11th ). The CDSR is improving its quality year-on-year.’(1) Methods: Citation data taken from the ISI Web of Science covering the years 2007 to 2010 (data for 2010 will not be available until June 2011) will be analyzed in order to identify which individual articles, subject areas and review groups have contributed most to the rise in the Impact Factor of the CDSR. Results: Can the rise in impact factor be attributed to a small number of review groups or has each individual review group contributed? Has the CDSR impact factor relied on a small number of highly cited articles? Should highly cited articles influence the topic areas where future review articles should be commissioned? Conclusions: Conclusions will be drawn from the analysis once the data is made available from Thompson ISI in June 2011. Reference 1. Horton R. (2010) Offline: what about the football? The Lancet, 376 (9735), 80. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 71 P1B79 Does one click access matter? Global use of The Cochrane Review: 2010 analysis Global health and equity Day: 20 Oct (P1B81–P1B102) Wilkins S1 , Stewart G2 , Pentesco-Gilbert D2 1 Wiley-Blackwell, Oxford, UK; 2 Wiley-Blackwell, Chichester, UK P1B81 Ivermectin and benzimidazoles for treating strongyloides infection Background: Evidence Aid, The Cochrane Collaboration and WileyBlackwell first introduced the free, one-click access policy for Cochrane Reviews in 2007. This was for countries in the World Bank list of low-income economies. The initiative was extended in September 2010, allowing direct no-cost access to people in a greater number of countries all over the world. People in more than 100 countries, in some of the most impoverished parts of the world, now have free access to The Cochrane Library through the Internet. Methods: The number of unique visits to www.thecochranelibrary.com, number of visits by country, and most accessed Cochrane Reviews on Wiley InterScience/Wiley Online Library (from August 2010) for 2010 were taken from the Wiley Usage Data Warehouse, an analysis tool that stores and tracks usage data. Another set of usage data was obtained from all other licensed providers of The Cochrane Library, showing the number of abstracts and full text articles that were accessed during 2010. A subset of data will be collected and interrogated for low- and middle-income countries with one-click access covering access denied data and the comparative access periods (From switch-on September 2010). Results: Data taken from all providers of The Cochrane Library show that during 2010 use increased. Full text use on the Wiley platform increased by 17%. Use by the World Bank list of low-income economies increased by 74% from 2009 data. Countries in this programme with the most downloads included: Vietnam, Pakistan (free access through Evidence Aid after flood disaster), Ecuador, Nigeria, and Kenya. Conclusions: Global use of The Cochrane Library continues to grow. The initiative for access for low- and middle-income countries is a major development, above and beyond the provision of knowledge for natural disasters. This positions The Cochrane Collaboration, Wiley-Blackwell and Evidence Aid at the forefront of information provision. Echevarria J1 , Gotuzzo E1 , Samalvides F1 , Terashima A1 1 Instituto de Medicina Tropical Alexander von Humboldt-Lima, Peru P1B80 How to involve consumers: A guide for Cochrane Editors McIlwain C1 Consumer Network 1 Cochrane Objectives: A new guide is available to help editors use consumer input in systematic reviews. You will learn techniques for: • locating consumers with review-specific interests, • using available resources to facilitate involvement, • encouraging repeat participation by meeting consumer needs, and • reporting on consumer involvement. Description: Funding organizations have been making a major commitment to consumer involvement in research; yet the best methods to find and include consumer input are still unclear. Some materials are available for training and involving consumers, but none pertain specifically to the Cochrane review process. This presentation will use a new Cochrane guide ‘How to Involve Consumers in Cochrane Systematic Reviews’ to chart your way through consumer participation in research. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: Strongyloidiasis is a common and globally distributed infection caused by strongyloides stercoralis. This infection is capable of transforming into a serious and sometimes fatal illness in people with comorbilities (malnutrition, immunocompromised, etc). The benzimidazoles and ivermectin are the only two types of drugs available for strongyloidiasis. Objectives: To evaluate antistrongyloidal drugs for treating strongyloides infection. Methods: We searched the Cochrane Infectious Diseases Group Specialized Register (July 2010); the Cochrane Central Register of Controlled Trials (CENTRAL), published in The Cochrane Library; MEDLINE (1966 to July 2010); EMBASE (1980 to July 2010); LILACS (July 2010). We also searched the metaRegister of Controlled Trials (mRCT) using ‘stronglyloid∗ ’ as a search term. Randomized controlled trials of benzimidazoles (including thiabendazole and albendazole) versus ivermectin for treating strongyloides stercoralis were included. Results: Ten trials (992 participants) met the inclusion criteria. Seven trials compared benzimidazoles vs Ivermectin, showing an effect in favour of Ivermectin 87% vs 60% (RR 1.40, 95% CI 1.11 to 1.78; I2 = 87%, Chi2 p-value =< 0.00001. There was no association of failure of therapy with type of benzimidazole. There were several adverse clinical events reported, but no deaths, after administration of benzimidazoles and ivermectin. The most serious adverse event reported in the thiabendazole group was nausea and disorientation. There were more adverse events in the benzimidazole group (overall RR: 1.44, 95% CI 1.23 to 1.70, I2 = 84%, Chi2 = 48.98). Conclusions: Both ivermectin and benzimidazoles are efficacious for treating strongyloidiasis, but ivermectin was superior in terms of efficacy and adverse events. Using ivermectin as a single doses improves adherence. It is not possible to conclude whether ivermectin or benzimidazoles has an effect on morbidity caused by infection. Thiabendazole might be less tolerated than ivermectin or other benzimidazoles based on serious adverse events (disorientation and nausea). P1B82 Sex- and gender-based analyses in Cochrane Reviews: Methods and lessons learned Ueffing E1 , Pardo Pardo J1 , Welch V1 , Borkhoff CM1 , Kristjansson E1 , Petticrew M2 , Tugwell P3 1 Campbell and Cochrane Equity Methods Group, Canada; 2 Campbell and Cochrane Equity Methods Group and London School of Hygiene and Tropical Medicine, UK; 3 Campbell and Cochrane Equity Methods Group, Dept of Medicine, Dept of Epidemiology and Community Medicine, Faculty of Medicine, University of Ottawa, Canada Background: The Canadian Institutes of Health Research (CIHR) notes that ‘Accounting for sex and gender in health research has the potential to make health research more just, more rigorous and more useful’. As awareness around equity and justice issues grows, formal sex and gender policies have been established. For example, Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 72 CIHR-funded researchers must address sex and gender, as appropriate, when designing their research while the American National Institutes of Health (NIH) guidelines require funded clinical research to examine differential effects of interventions by sex/gender and race/ethnicity. Objectives: To report on findings from studies evaluating the degree to which systematic reviews have addressed sex and gender. To describe how methods developed by international equity researchers can be used for sex- and gender-based reporting and analyses. Methods: Recently, teams have published guidance on how to address equity and the social determinants of health in systematic reviews. These methods include logic models, subgroup analyses, and process evaluations; sex and gender considerations are crucial for each of these methods. Results: Specific methods for including sex and gender considerations in systematic reviews will be presented, with examples from international research reports, national guideline panels, and published reviews. Conclusions: Sex- and gender-based analyses can help researchers to ensure that: 1) systematic reviews are suitably designed; 2) results from systematic reviews are available for men and women, males and females where appropriate; and 3) that policies and decisions based on systematic review evidence are just and relevant. P1B83 Methodological quality of randomized controlled trials on gastric cancer treated with traditional Chinese medicine published in Chinese journals LI H1 , HU Q1 , LIU W1 , MA B1 1 Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China Background: Randomized controlled trial (RCTs) on gastric cancer treated with Traditional Chinese Medicine (TCM) have been published in large numbers. However, methodological quality of these RCTs were unknown. Objectives: The purpose was to assess methodological quality of RCTs on gastric cancer treated with TCM. Methods: Chinese Biomedicine Literature Database (CBM), Chinese Scientific Journal Full-text Database (CSJD), Chinese Journal Full-text Database (CJFD), and Wangfa Database were performed in the end of the Dec. 2010. The search terms included ‘Traditional Chinese Medicine’ and ‘Chinese herbs’ (Mesh or EMTREE), ‘Gastric Cancer’, ‘Stomach Cancer’ and ‘Stomach Neoplasms’, (Mesh or EMTREE). Besides, Google were searched and the range of snowballing method was used. The Cochrane Handbook 5.0.1 was used to evaluate the methodological quality of included studies. Results: Totally 102 RCTs were identified and published in 72 different Chinese Journals. Less half (41.67%) were published in specialty journals. Less one-third (21.57%) were published in journals cited by Chinese Science Citation Database (CSCD). The sample of mostly trials (91.18%) less than 100 cases. Only 52.95% and 11.76% trials reported the specific inclusion and diagnostic criteria, 88.24% did not reported exclusion criteria. All of them did not register in any clinical trials registry platform. Only 17 studies reported the correct randomization method. All trials did not report allocation concealment. Only 3 studies reported blinding, but no one reported whose were blinded. Statistic mistakes appeared in one-third (29.3%) studies. Conclusions: RCTs on TCM were published in large number in Chinese Journals in recently years, but the quality of these review were worrying. The information reported from these RCTs were not comprehensive and even some were mistake. It did not only provide evidence for clinicians but also misled them. Hence, the most urgent and important thing is to improve the quality of RCTs on TCM, not accelerate the quantity in China. P1B84 Financial health equity. Intervention for balance and financial stability of institutions providing national health, health promoters and insurers Kadar I1 1 Professor of epidemiology and health care systems, Antonio Narino University, Colombia Background: The quality of health systems is measured and influenced by limiting factors such as access, timeliness, coverage and equity in service to the population. France is known for its good health system, as the country spent $ 4056 (USD) per year per person on health, which is equivalent to 11.1% of gross domestic product (GDP); this allocation of fundsmay have contributed to France having the highest life expectancy with age 81. In contrast, the Republic of Congo invested just $ 42 (USD) per year per person, corresponding to 2.1% of GDP, and the population has a life expectancy of only 54 years (WHO 2006, Table 1). These investments constitute a limit for health management. Developing countries have a very limited amount of resources to establish efficient and effective health care systems. Objectives: • achieve efficient cost-effective management. • reward healthcare companies who (a) work toward prevention and reduction of prevalent diseases that generate high costs, and (b) reduce spending generated by the advanced stages of preventable diseases. Methods: • develop standardized guidelines for the management of diseases with evidence-based scientific rigor and at an acceptable cost. • develop a monitoring system for health care promotion. • develop a method of monitoring insurance companies’ and health service institutions’ (e.g. hospitals) use of disease prevention and control of risk factors Analyze the results including financial implications to generate internal policies. Results: Building financial stability for all companies that comply with health prevention policies, delivering a budget increase for those healthcare promoting companies whose load of high-cost patients exceeds the standard deviation and decrease the budget for Table 1 (P1B84). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 73 those who have reduced load. This will reward only those companies who implement an effective intervention and prevention procedure that has reduced the cost of these diseases over time. P1B85 Agreement and alignment: Current status of clinical practice guidelines within the Southern African Development Community Kredo T1 , Gerritsen A2 , van Heerden J3 , Conway S4 , Siegfried N1 1 South African Cochrane Centre, South Africa; 2 Epi Result, South africa; 3 Benguela health, South Africa; 4 Southern African Regional Programme for Access to Medicines, South Africa Background: Reducing the burden of disease relies on availability of evidence-based clinical practice guidelines (CPGs). There is limited data on availability, quality and content of guidelines within the Southern African Development Community (SADC). This evaluation aims to address this gap in knowledge and to provide recommendations for regional guideline development. Methods: We prioritised five diseases: HIV in adults, malaria in children and adults, pre-eclampsia, diarrhoea in children and hypertension in primary care. A comprehensive electronic search, supported by email contact with SADC Ministries of Health was used to locate guidelines. The AGREE II tool was applied by independent reviewers to evaluate 6 quality domains reporting the guideline development process. Individual domains were scored and percentages calculated. Alignment of the evidence-base of the guidelines was evaluated by comparing content with key recommendations from accepted reference guidelines, identified with a content expert, and percentage scores were calculated. Results: We identified 30 guidelines from 13 countries, between June and October 2010. Publication dates ranged from 2003 to 2010. Overall the ‘scope and purpose’ and ‘clarity and presentation’ domains of the AGREE II instrument scored highest, median 58% (range 19–92) and 83% (range 17–100). ‘Stakeholder involvement’ followed with median 39% (range 6–75). ‘Applicability’, ‘rigour of development’ and ‘editorial independence’ scored poorly, all below 25%. Alignment with evidence was variable across member states, the lowest scores occurring in older guidelines or where the guideline being evaluated was part of broader primary healthcare CPG rather than a disease-specific guideline. Conclusion: This review identified quality gaps and variable alignment with best evidence in available guidelines within SADC for five priority diseases. Future guideline development processes within SADC should better adhere to global reporting norms requiring broader consultation of stakeholders and transparency of process. A regional guideline support committee could harness local capacity to support context appropriate guideline development. P1B86 Neglected tropical diseases: Are global policies based on reliable evidence? Nagpal S1 , Garner P1 1 Cochrane Infectious Diseases Group, Liverpool School of Tropical Medicine, UK Background: Neglected Tropical Diseases is a term that has appeared in the last 20 years, and is attracting global donor funds to implement big programmes in tropical countries. With expanding investment, it is particularly important that policies are informed by reliable Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. research. Objectives: To analyse the evolution of the term ‘neglected tropical diseases’ through bibliographic analysis; and to assess whether policies promoted for investment are underpinned by reliable syntheses of research. Methods: We retrieved all articles on MEDLINE to April 2011 containing the words ‘neglected tropical disease(s)’ in the title or abstract. We used content analysis to map the concept, the organisations and personalities involved in its evolution, and the diseases it includes. We examined the percentage of articles that contained research data. Finally, we took the top ten cited articles in this field, summarised the policies and strategies that were being advocated, and if reliable research evidence (systematic reviews or trials) is available for each, and if it is cited. Results: ‘Neglected tropical diseases’ is a term that was coined in 2005, and refers to tropical infections once the ‘big three-malaria, tuberculosis and HIV’ have been taken out. The first publications with the mention of NTDs appeared in 2005 and 2006, since then there has been a steady increase in the number of articles up to 83 in 2010. We are currently analysing the top 10 cited policy papers against existing reliable research evidence. Conclusions: We will present an assessment of the evidence-base behind current intervention strategies for neglected tropical diseases, and provide an agenda for further synthesises and primary research. P1B87 High 5s project. Hospital survey on patient safety culture results from Germany Kolbe M1 , Lessing C1 , Fishman L2 , Renner D2 , Thomeczek C2 1 Institut for Patient Safety, University of Bonn; 2 German Coalition for Patient Safety Background: In the World Health Organisation initiated High 5s project a multi-national learning community has set itself the goal of reducing three serious patient safety problems by implementing standardized operating protocols (SOP). One component within the scope of the project is to conduct a survey on patient safety culture in participating hospitals on a voluntary basis. Method: The present study is based upon the Hospital Survey on Patient Safety Culture (HSOPS) of the Agency for Healthcare Research and Quality. The survey was validated with more than 1,400 hospital employees from 21 hospitals across the US. For the realisation of the survey in German hospitals within the High 5s Project, the HSOPS was translated into German and adapted according to national data protection regulations. The statistical evaluation is carried out by the research group Westat. In the scope of the High 5s Project, the survey is to be conducted at two points in time. Results: In the first inquiry period, the HSOPS was conducted at some participating hospitals. The results of the HSOPS grouped according to the 12 safety culture dimensions show that Communication Openness has the highest rate of positive response and Organizational Learning - Continuous Improvement is rated lowest. Conclusions: About a third of the participating hospitals conducted the survey in spite of the higher workload for hospital staff. Based on the safety culture dimensions, certain organisational factors concerning patient safety can be described in addition to the High 5s SOP indicators. First trends are apparent as measured by the rate of positive response. The second survey period will demonstrate if conclusions can be reached regarding a change in the perspectives of patient safety by employees as a result of implementing the SOP. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 74 P1B88 Cutaneous Leishmaniasis International Observatory: An initiative for Evidence-based Global Health González U1 , Ruiz-Baqués A2 1 Research Unit for Evidence-based Dermatology, Hospital Plató, Barcelona, Spain; 2 Topping Medical Research, UAB Research Park, Barcelona, Spain Background: Cutaneous Leishmaniasis (CL) is a neglected mosquitoborne protozoan disfiguring and stigmatizing infection that causes a great suffer of the patients and their families. It is a global increasing public health problem but still does not receive the necessary attention. It is a disease that is present in 82 countries and its incidence is estimated at 10 million of affected people. An international team has developed with the support of the WHO, three Cochrane systematic reviews focused on the role of control strategies for CL. Objective: To develop a Cutaneous Leishmaniasis International Observatory (CLIO), as a non-for-profit model of knowledge management, based on the scientific evidence and for improving the lives of people with CL. Methods: Our main need is to seek contacts, ideas and interest of institutions, organizations, companies and individuals to lead and develop this action and to help complete the project in the best conditions. Of particular interest is to periodically maintain and actively disseminate a comprehensive and rigorous analysis and synthesis of the information derived from updated Cochrane reviews. Results: We have collected the opinion and support of different people and organizations. We have developed a strategic plan for the commitment of CLIO to reach real and significant impact in guiding and promoting practice, policy decisions and in helping to identify research priorities around the world. We want healthcare decisions in CL to be informed by reliable, up-to-date, scientifically defensible and relevant evidence. Conclusions: The major interests of CLIO are the update of the information and to develop networking activities and communication actions for actively disseminate what is best known about the effects of control interventions in CL. P1B89 The role of the medical information database in managing the East Japan disaster Nango E1 , Naito T2 , Toyoshima Y3 , Yoshimoto H4 , Nakayama T5 1 Tokyo-kita Social Insurance Hospital, Japan; 2 Fukuoka Dental College, Japan; 3 The Dai-ichi Life Insurance Company, Japan; 4 Mie University Graduate School of Medicine, Japan; 5 Kyoto University Graduate School of Medicine, Japan Background: On 11 March 2011, a massive earthquake measuring 9.0 on the Richter scale hit northeastern Japan, triggering a 10-meter-high tsunami that engulfed the northeast coast of Japan. The number of deaths has reached 14,435 and the number of missing persons has been estimated at 11,601 (as of 27 April 2011). The total number of evacuees approximately 335,000 (as of 20 March 2011). There is concern about health conditions of the evacuees due to the insufficient infrastructure, medical services, lack of privacy and anxiety about the future. Medical support teams are also facing unusual medical demands such as infectious diseases, cardiovascular diseases, trauma, post-traumatic stress disorder and radiation-related injuries. Objectives: To define what types of medical information are needed for the workforces involved in this disaster and to improve the dissemination of information. Methods: We searched the medical information services that had been provided via the internet for this Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. disaster and counted the number of people who were accessing the databases. Results: Cochrane Evidence Aid for earthquakes started to provide service on 12 March 2011. A Japanese translation of their information was prepared by Kyoto University School of Public Health and publicized on 13 March. The number of pages accessed increased sharply after the provision of the Japanese translation and the notice by an internet discussion group for medical personnel. UpToDate and DynaMed started to provide free access to Japanese medical personnel on 14 March. The most frequently accessed content was ‘Treatment of radiation injury in the adult’ provided by UpToDate. Conclusions: Although there are many useful evidence-based databases, almost all information is written in English, and is hard to use for non-native users of English. Translation and localization of databases and their dissemination are therefore also considered to be important factors for the timely provision of information. P1B90 A systematic review identifies substantial heterogeneity of studies evaluating effects of rural-to-urban migration on cardiovascular risk factors in low and middle income countries Hernandez AV1 , Pasupuleti V1 , Deshpande A1 , Bernabe-Ortiz A2 , Miranda JJ2 1 Cleveland Clinic, Ohio, USA; 2 Universidad Peruana Cayetano Heredia, Lima, Peru Background: Limited information is available about the effects of rural-to-urban within-country migration on cardiovascular (CV) risk factors in low and middle income countries (LMIC). Objectives: We performed a systematic review of studies evaluating these effects and having rural and/or urban control groups. Methods: Two teams of investigators independently searched observational studies published between 1960 and June 2010. Studies evaluating international migration or those from developed countries were excluded. Three investigators extracted the information stratified by gender. We captured information on 17 known CV risk factors. Results: Eighteen studies (n = 58,536) were included. Studies were highly heterogeneous with respect to age, study design, sampling frame for migrants, years of urban exposure for migrants, and reported CV risk factors. Fourteen studies were cross-sectional; only six studies adjusted for confounders. In migrants, commonly reported CV risk factors – systolic and diastolic blood pressure, body mass index, obesity, total cholesterol, and LDL – were usually higher or more frequent when compared to the rural group and usually lower or less frequent when compared to the urban group. This gradient was usually present in both males and females separately. Anthropometric (waist-to-hip ratio, hip/waist circumference, triceps skinfolds) and metabolic (fasting glucose and insulin, insulin resistance) risk factors usually followed the same gradient, but conclusions are weak due to the paucity of information. Hypertension, HDL, and other uncommonly reported risk factors such as fibrinogen and C-reactive protein did not follow any pattern among groups. Conclusion: In LMIC, studies investigating the effect of rural-to-urban within-country migration on CV risk factors are heterogeneous. A gradient, higher or more frequent in migrants than in the rural groups but lower or less frequent than the urban groups, was observed in most but not all CV risk factors. Such gradients may or may not be associated to differential CV events and long-term evaluations remain necessary. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 75 P1B91 Integrating prevention of mother-to-child HIV transmission programmes to improve uptake: A systematic review Tudor Car L1 , Van Velthoven MH1 , Brusamento S1 , Elmoniry H1 , Car J1 , Majeed A1 , Tugwell P2 , Welch V2 , Marusic A3 , Atun R4 1 Department of Primary Care and Public Health, School of Public Health, Imperial College London, UK; 2 Centre for Global Health, University of Ottawa, Ottawa, Canada; 3 University of Split School of Medicine, Department of in Research in Biomedicine and Health, Split, Croatia; 4 Imperial College Business School, Imperial College London, UK Background: Prevention of mother-to-child transmission (PMTCT) of HIV interventions can significantly reduce transmission rates, responsible for more than 90% of HIV infections in children. The World Health Organisation recommends integration of PMTCT programmes with other healthcare services to increase access and improve uptake of these interventions. Objectives: To assess the effect of integrated PMTCT interventions compared to stand-alone or partially integrated services on PMTCT uptake in low- and middle-income countries. Methods: We searched for experimental, quasi-experimental and controlled observational studies in any language in 21 databases and grey literature sources. We also scanned reference lists of included articles and relevant systematic reviews. Results: Out of 28 654 citations retrieved, five studies met inclusion criteria (Figure 1; Table 1). A cluster randomised controlled trial reported higher probability of nevirapine uptake at the labour wards implementing HIV testing and structured nevirapine adherence assessment. A stepped wedge study showed marked improvement in antiretroviral therapy (ART) enrolment and initiation in integrated care, but the median gestational age of ART initiation, ART duration or 90 days ART retention did not differ significantly. A retrospective cohort study reported no significant difference either in the ART coverage or eight weeks of ART duration before the delivery between integrated, proximal and distal partially integrated care. Two before-and-after studies assessed the impact of integration on HIV testing uptake in antenatal care. The first study reported that significantly more women received information on Figure 1 (P1B91). Study selection flow-diagram. Citations identified by search strategy (n=28 654) Duplicates removed (n=4032) Titles and abstracts screened (n=24 622) Articles excluded becuse not relevant (n=24 596) Full-text articles screened (n=26) Full-text articles assessed for eligibility (n=15) Conference proceedings excluded because of lack of data (n=11) Articles excluded becuse not relevant or second report (n=10) Articles included (one second report) in the systematic review (n=5) Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 Before and after Stepped wedged Killam, 2010 Zambia Jul-07–Jul-08 Study design Kasenga 2009 Malawi Jan-05–Dec-07 Author Year of publication, Country Study duration Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Women in ANC Intervention: 17,619 Control: 13,917 Women in ANC Intervention 01,063 Control 196 Participants and sample size Table 1 (P1B91). Characteristics of included studies. • Women received ART in ANC, twice per week. at ANC • HIV testing of women • Women received ARV prophylaxis in LW • Safe delivery • Women received infant feeding counselling • HIV testing and counselling & services integrated within ANC Intervention 2. Percentage of HIV positive women who underwent to CD4 control • Women referred for ART from ANC to ART clinic 5. Percentage of women on care at 90 days of follow up 4. Percentage of women receiving HAART as part of PMTCT 3. Percentage of women enrolled in ART clinic 1. Percentage of women who tested in ANC ANC • HIV testing of women at ANC Partially integrated • Women received ARV prophylaxis in LW • Safe delivery • Women received infant feeding counselling 1. Percentage of women tested in ANC • Non-integrated HIV testing of women not provided in ANC but in separate VCT unit 2. Percentage of women receiving ARV prophylaxis in LW ANC Primary outcomes reported Non-integrated Comparison 5. 91% vs. 88%, p = 0.3 4. 14% vs. 33%, p < 0.001 3. 25% vs. 44%, p < 0.001 2. 85% vs. 85.1%, p = 0.98 1. 98.4 vs. 97.9% p = 0.51 Control vs. intervention: 2. 92% vs. 97%, p = 0.404 1. 53% vs. 79%, p < 0.001 Control vs. intervention: Results 76 Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Stinson, 2010 RSA Jan–Dec 2005 Megazzini, 2010 Zambia Oct-05–Jan-06 Author Year of publication, Country Study duration Cohort cRCT Study design Table 1 (P1B91). Continued. Fully integrated one point: 4,823 Partially integrated proximal: 4,783 Partially integrated distal: 5,381 Women in ANC Intervention: 2,435 Control: 2,106 Women at LW Participants and sample size • Women received ART at the ANC, twice per week. • HIV counselling, testing and ARV prophylaxis. • Formal nevirapine adherence assessment • Nevirapine administration as indicated • HIV testing of women of unknown serostatus in LW • Women received ARV prophylaxis at ANC • HIV testing of women at ANC Intervention • Women were referred to receive ART to the HIV clinic in another health facility within 5 Km. • HIV counselling, testing and ARV prophylaxis. Partially integrated distal: 2. Percentage of HIV positive women who underwent to CD4 control • Women were referred to receive ART to the HIV clinic in the same health facility 4. Percentage of HAART eligible women receiving ARV prophylaxis 3. Percentage of women starting HAART as part of PMTCT 1. Total: 88.1% 1. Percentage of women who tested in ANC • HIV counselling, testing and ARV prophylaxis. 5. 26% vs. 28% vs. 29% 4. 48% vs. 47% 3. 55% vs.48% vs. 47% 2. Total: 97% Integrated vs. proximal vs. distal: • Intervention: from 42% to 54% (difference ranged −10-+33%) Partially integrated proximal: ANC • No formal nevirapine adherence assessment • Women received ARV prophylaxis in ANC • Percentage of women/infants receiving ARV prophylaxis • Control: from 53% to 43% (difference ranged −13-0%) • HIV testing of women at ANC Baseline period/intervention period: Results LW Primary outcomes reported Partially integrated Comparison 77 Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Before and after Study design Intervention: 4,089 Control: 4,142 Women in ANC Participants and sample size 2. Percentage of women tested • Counselling not provided in ANC but in a separate location by a counsellor • Infants ARV prophylaxis in LW • Women received ARV prophylaxis in LW • Infants ARV prophylaxis in LW • Women received ARV prophylaxis in LW • Safe delivery in LW • Women received infant feeding counselling from the counsellor • ARV prophylaxis for women and infant provided from the counsellor at 34 weeks 1. Percentage of women who received information on PMTCT • HIV testing of women not provided in ANC but in off-site laboratory 4. Percentage of women receiving ARV prophylaxis LW 3. Percentage of women who received their test result ANC: Primary outcomes reported Partially integrated Comparison • Safe delivery in LW • Women received infant feeding counselling at ANC • ARV prophylaxis dispensed any time from the 2nd trimester • Infant ARV prophylaxis dispensed at ANC • Women received ARV prophylaxis at ANC • HIV testing of women provided in ANC • Nurse counsellor provided all the PMTCT interventions in ANC Intervention ANC = antenatal care; LW = labour ward; ARV = antiretroviral; VCT = voluntary counselling and testing ART = antiretroviral therapy; PMTCT = prevention of mother to child transmission; RSA = Republic of South Africa. Van’t Hoog, 2005 Kenya Nov-01–Aug-03 Author Year of publication, Country Study duration Table 1 (P1B91). Continued. 4. 57% vs 70%, p < 0.001 3. 89% vs 90%, p = 0.233 2. 62% vs 76%, p < 0.001 1. 77% vs 92%, p < 0.001 Control vs intervention: Results 78 Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 79 PMTCT, were tested and learned their HIV status. The second study also reported significant increase in HIV testing uptake after integration. Conclusions: Limited evidence supports the effectiveness of PMTCT programmes compared to partially integrated care. More research measuring coverage and other relevant outcomes is urgently needed to inform the design of services delivering PMTCT programmes. P1B92 Evidence-based African first aid guidelines and training materials Van de Velde S1 , De Buck E1 , Vandekerckhove P2 , Volmink J3 1 Expertise Centre, Belgian Red Cross-Flanders; 2 Belgian Red Cross-Flanders; 3 South African Cochrane Centre; Faculty of Health Sciences, Stellenbosch University Background: In Sub-Saharan Africa, 40% of the burden of disease and injury can potentially be addressed by emergency care. Adequate pre-hospital care is vital, but often unavailable in Sub-Saharan Africa. Initial help and emergency transport often depends on basic first responders. The World Bank identified first aid training as a cost-effective way to save lives in Sub-Saharan Africa. Although well-intentioned, first aid training in Sub-Saharan Africa often depends on didactical materials from non-African settings. This may lead to misdirected, inadequate or even harmful training instructions. Objectives: To produce and implement evidence-based guidelines on how to train basic first responders to manage emergency situations in an African context. Methods: Two authors searched 8 databases (including MEDLINE, EMBASE, The Cochrane Library, AFRICAN INDEX MEDICUS) to identify relevant evidence on the effectiveness, safety, and feasibility of various first aid procedures. A separate search was done for studies on African cultural remedies and preferences. A multidisciplinary panel of eleven African experts discussed each recommendation until they reached agreement. The quality of evidence and strength of recommendations were determined according to GRADE. Four peer reviewers revised the guidelines. To implement the guidelines we developed a flexible didactic materials kit and an implementation guide. Between June and December 2010 we piloted the training materials and implementation guide in Uganda and Swaziland. Results: Overall we screened 24,000 references and selected 143 publications for the guideline. It is planned to implement AFAM in the Red Cross national societies from Namibia, Mozambique, South Africa, Malawi, Uganda, Swaziland, the Comoros and Kenya. At the colloquium we will present results of the guideline and discuss experiences with its implementation. Conclusions: This project allowed us to learn about successful first aid practices that made the best use of the limited resources available. P1B93 Criteria for judging the effectiveness of public health interventions Denison E1 , Vist GE1 1 The Norwegian Knowledge Centre for the Health Services, Norway Background: Judging the effectiveness of public health interventions (i.e., health promotion and prevention) includes assessment of both Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. the methodological quality of evaluations and the relevance of interventions. Many interventions are complex and may involve different sectors and agencies, and it is sometimes argued that methodologically rigorous evaluations may miss anticipated effects. However, to our knowledge, it has not been described which criteria for study design and methodological quality are actually used by researchers who systematically review the effects of public health interventions. Objectives: To describe criteria used to evaluate methodological quality used in systematic reviews (SRs) of effects of public health interventions. Methods: A systematic literature search was carried out in 13 electronic data bases from January 2005 to January 2011 to identify SRs addressing effects of public health interventions. We included non-medical interventions targeting children, adults and older persons where health related outcomes were reported. We extracted data on 1) inclusion criteria for study design, 2) quality assessment of included primary studies, 3) meta-analysis, 4) grading of results and 5) first author affiliation. The most recent review from research milieus that had published two or more SRs was included. Results: The literature search identified 7,850 references, 801 were read in full text, 644 publications were assessed as SRs. Data from the 125 selected SRs show that 1) inclusion criteria for study design involved randomised controlled trials (24%), studies with a control group (36%), no design limitations (29%) and no specification reported (11%), 2) quality assessment of primary studies was performed in 69% of the SRs, 3) meta-analysis was performed in 34% and 4) grading of the results was done in 7%. Conclusions: Criteria regarding study design and assessments of methodological rigour were to a large extent applied in SRs evaluating the effect of public health interventions. P1B94 Using ‘Implications for practice’ from Cochrane reviews to promote worldwide improvements in healthcare Newbatt E1 , Gholitabar M1 , McGuire H1 , Ullman R1 1 National Collaborating Centre for Women’s and Children’s Health, UK Background: The National Collaborating Centre for Women’s and Children’s Health (NCC-WCH) is contracted to produce clinical guidelines for use in the UK National Health Service (NHS). These guidelines provide specific granular evidence and/or consensus-based recommendations for the care of women and children using the NHS. Because of this, our recommendations may not be translated easily into clinical practice in other countries.Cochrane reviews adopt a more ‘holistic’ approach by including studies from all countries and writing ‘Implications for practice’ based on the findings of the review. However these ‘Implications for practice’ are not always directly relevant to both developed and developing countries, and an alternative approach might be to use sub-group analyses to inform specific ‘Implications for practice’ for different countries. This should help promote global improvements in the quality of healthcare through the increased use of more context-specific ‘Implications for practice’ in Cochrane reviews. Objectives: To evaluate how the results of systematic reviews, and the ensuing ‘Implications for practice’ and recommendations, are affected by the inclusion or exclusion of evidence from different countries. Methods: We identified systematic reviews, in NCC-WCH guidelines, that included evidence from all countries, and then performed sub-group analyses to examine the effect of Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 80 including only evidence from developed or developing countries. We then examined the potential implications for creation of more context-specific practice recommendations. Results: Results will be presented at the Colloquium. Conclusions: Conclusions will be presented at the Colloquium. P1B95 Evidence-based medicine as a safety parameter for the judicialization of health policies Marin dos Santos D1 , Puga ME1 , Bartolomei C1 , Marchewka T1 1 Brazilian Cochrane Centre and Federal University of São Paulo, Brazil Background: In Brazil and other countries, the judiciary branch plays an active role in health policies, due to the phenomenon called judicial activism or judicialization of health. Objectives: To establish useful parameters to support the judicial power in cases involving health policies and drugs. Discussion: It is known that scientific evidence is associated with a lower rate of errors. Though, it is expected that legal practitioners recognize evidence as a basis for decision-making. However, one of the greatest challenges on implementing this premise is the broad discretion held by the judges, as there is no hierarchy of legal evidence (Principle of free persuasion of the judge). Conclusions: The Brazilian Cochrane Center, from the experience gained in tackling the issue, believes it is possible to establish three precepts, which would ensure that judicialization of health policies could be conducted safely. First of all, when the object of the lawsuit reveals a high level of scientific evidence, the judicial intervention will be legitimate. The inertia of the government, in such situations, can be characterized as faut du service, legitimizing judicial mediation. In the other hand, it there is no available evidence, the judiciary branch may, as a rule, dismiss the case. However, in such situations, the judge can conduct an extensive analysis of the peculiarities of the case, trough the performance of forensic expertise. Finally, if there is scientific evidence that points against the intervention, the judicialization of health policies may cause damage to the collective and the individual’s health. By ignoring evidence, the Judiciary Branch will be denying the constitutional right to health; though, for reasons unrelated to science, believes to be ensuring it. P1B96 A novel approach to building evidence for improving communication about childhood vaccinations in low- and middle-income countries: the ‘communicate to vaccinate’ (COMMVAC) project Lewin S1 , Hill S2 , Kaufman J2 , Galak N2 , Kis-Rigo J2 , Anderson L3 , Bosch-Capblanch X4 , Hussein L5 , Lin V3 , Mahomed H5 , Rhoda L5 , Robinson P3 , ben Saude de Castro Freire S6 , Waggie Z5 , Wiysonge C5 1 Norwegian Knowledge Centre for the Health Services and Medical Research Council of South Africa, Norway; 2 Centre for Health Communication and Participation, Australian Institute for Primary Care and Ageing, La Trobe University, Australia; 3 La Trobe University, Australia; 4 Swiss Tropical and Public Health Institute, Switzerland; 5 Institute of Infectious Disease and Molecular Medicine, University of Cape Town, South Africa; 6 International Union for Health Promotion and Education, France Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: Effective provider-parent communication can improve childhood vaccination uptake and strengthen immunization services in low- and middle-income countries (LMICs). However, demand-side interventions to improve vaccination uptake have been neglected and existing rigorous research is often not readily found or easily applicable to LMICs. This makes it difficult for policy makers to use evidence to inform policies and practice. Objectives: To describe the approaches used by the COMMVAC project to explore, evaluate and disseminate evidence on strategies for improving communication about childhood vaccinations with parents and communities in LMICs. Methods: COMMVAC uses an innovative combination of methods. First, we are producing a systematic map of communication interventions. Systematic maps use the same rigorous methods as systematic reviews of effects but focus on describing the range of interventions. For each intervention identified, we are extracting information on the populations targeted, settings, intervention purpose and delivery (and evaluation design and outcomes in trials). Second, we are developing a taxonomy of interventions to improve communication around childhood vaccination so as to: (1) understand the relations between different types of interventions; (2) facilitate conceptual mapping of these interventions; and (3) clarify the key purposes of interventions. Third, we will hold deliberative fora with key stakeholders to discuss priorities for systematic reviews of effects, informed by the systematic map and taxonomy. Fourth, we will conduct systematic reviews on high priority topics. Finally, we will produce web-based evidence summaries that translate the review findings into accessible messages for LMICs and allow users to add implementation commentary. Conclusions: COMMVAC takes a novel approach to building knowledge resources and making more effective use of existing research and practice descriptions. Key outputs will include high quality evidence on the scope and effects of interventions to improve provider-parent communication around vaccination and knowledge resources tailored for LMICs. P1B97 VIP (Very Integrated Program) Project on alcohol and drug abusers in Southern Sweden – RCT of efficacy of the adding the health promotion and rehabilitation for treatment of alcohol and drug abusers Hovhannisyan K1 , Ehrnström M1 , Kovacs J1 , Skagert E1 , Hau Skibelund D1 , Thornqvist K1 , Tønnesen H1 1 Lund University, Sweden Background: Alcohol and drug abuse are followed by tremendous physical, psychological and social problems as well as early death. Heavy smoking, poor nutrition, physical inactivity and chronic diseases (co-morbidity) are often part of these problems and illnesses.There seems to be a large potential for a better outcome by including smoking cessation, physical training, diet and nutrition as well as co-morbidity treatment in a multi-disciplinary setting – a potential not used yet. This Very Integrated Program (VIP) is inspired from the rehabilitation offered to patients with chronic diseases and surgical patients having a likewise unhealthy lifestyle and similar co-morbidities. Aim: To evaluate the effect of adding the VIP program to the usual alcohol and drug intervention on the outcome for alcohol and drug abusers compared to the usual intervention alone. The VIP project consists of 3 steps: 1. To map the health status and estimate the potential improvement for 400 alcohol and drug abusers 2. To pilot Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 81 test the VIP program 3. To evaluate it compared to the daily routines for 260 alcohol and drug abusers with a poor health status in a randomised controlled trial. Main outcome: Change from abuser to non-abuser. Secondary outcomes are health status; quality of life, use of health services, time return to work (or similar activity level), harm reduction, and cost-effectiveness. The results will be published in scientific journals, reports and in the public media. The VIP project takes place at the Addiction Centre Malmö, Psychiatry Skåne, Sweden in collaboration with a multidisciplinary staff group. It is organised by Lund University Clinical Alcohol Research Center at the University Hospital Malmö, Sweden in collaboration with WHO Collaborating Centre for Evidence-Based Health Promotion in Hospitals & Health Services, Bispebjerg University Hospital, Copenhagen, Denmark. P1B98 The international distribution of RCTs in Cochrane dementia intervention reviews Marcus S1 , Noel-Storr A1 , Struthers C1 1 Cochrane Dementia and Cognitive Improvement Group, United Kingdom Background: There were an estimated 35.6 million people with dementia in 2010, the numbers nearly doubling every 20 years, with an expected 65.7 million in 2030 and 115.4 million in 2050. 58% of all people with dementia worldwide live in low and middle income countries, rising to 71% by 2050 (Alzheimer’s World Report, 2009). Objectives: This study sets out to assess the international distribution of both included and excluded studies within published Cochrane Dementia and Cognitive Improvement Group (CDCIG) intervention reviews. Methods: Over 100 reviews are to be assessed for this study. The following information will be extracted from each: • the country of origin of both included and excluded studies • the country or countries of recruitment of participants from included and excluded studies • the absolute number of participants from each included and excluded study • the main diagnostic criteria used for each included study. Results: The final data collection for this study will be in July 2011. It is expected that less than 10% of included Cochrane dementia studies will be from low and middle income countries yet this is where proportionate increases in the number of people with dementia will be much steeper compared with high income countries. P1B99 Assessing applicability, scaling up and equity issues in Cochrane reviews: A worked example from a review of lay health worker interventions in primary and community health care for chronic conditions Lewin S1 , Munabi-Babigumira S2 , Glenton C2 , Scheel I2 , Bosch-Capblanch X3 , Daniels K4 , van Wyk B5 , Aja G6 , Zwarenstein M7 1 Norwegian Knowledge Centre for the Health Services, Norway and Medical Research Council, South Africa; 2 Norwegian Knowledge Centre for the Health Services, Norway; 3 Swiss Centre for International Health, Swiss Tropical and Public Health Institute, Switzerland; 4 Health Systems Research Unit, Medical Research Council, South Africa; 5 School of Public Health, University of the Western Cape, South Africa; 6 Department of Health Sciences, Babcock University, Nigeria; 7 Combined Health Services Sciences, Sunnybrook Health Sciences Centre, Canada Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: In addition to evidence of effectiveness, review users often require information on the equity impacts of interventions; the extent to which interventions can be applied in other settings and whether similar effects might be expected (applicability); and factors likely to influence scaling up. However, these issues are often not discussed by review authors. Objectives: To assess the extent to which evidence is available to facilitate assessment of applicability, scaling up and equity issues from RCTs included in a review of the effects of lay health workers (LHWs) for chronic conditions. Methods: For each included RCT, data was extracted on intervention context and on the groups targeted by the intervention, including their geographic, demographic and socio-economic details. We used these data to consider issues of applicability, scaling up and equity. Results: Overall, the included studies provided little data regarding the differential effects of the interventions across different populations making equity effects difficult to assess. Most studies were undertaken in high income countries and it is not clear whether the effects could be transferred to other settings with different health care systems. However, most of these studies were directed at low income or ethnic minority groups in these settings. To the extent that these groups share similar constraints in accessing health care with service users in low- and middle-income countries (LMICs), the interventions may be applicable in LMICs. Consideration of factors affecting the scaling up of the interventions was limited by poor intervention descriptions. Conclusions: Assessing applicability, scaling up and equity issues for effective interventions continues to be hampered by limited data in trial reports. Guidance for trialists on the reporting of such data needs to be applied. P1B100 Clinical practice guideline: Dental care for patients with epidermolysis bullosa Krämer SM1 , Villanueva J1 , Serrano C2 , Galvez P3 , Zillmann G3 , Araya I1 , Oliva P4 , Carrasco A1 , Brignardello R1 , Yanine N1 , Cornejo M1 1 Centro adherido Cochrane de la Facultad de Odontologia, Universidad de Chile, Chile; 2 Universidad de Valencia, Spain; 3 Universidad de Chile, Chile; 4 Universidad del Desarrollo de Concepción, Chile Background: Epidermolysis Bullosa (EB) is a rare disease with multiple oral manifestations requiring a special approach from the dentist. The scientific literature is scarce, making it difficult for dentists with no experience on EB to know how to approach patients in a safe manner. The present guideline has been developed using a standard methodology. Objective: Provide the users with information on the special care people with EB require before, during and after dental treatment. Methods: A systematic review of the literature was performed. The information gathered was discussed at a two day consensus meeting including dentists from different countries and patient representatives. To formulate the recommendations the SIGN system was used. The draft document was sent for external review to dental professionals and patient groups. Results: Key recommendations include: A preventative protocol is the dental management approach of choice. Patients with EB should be referred to the dentist for the first consultation at the age of 3 to 6 months. A dietary caries-prevention programme should be instigated at early age. Patients with the severe generalized RDEB subtype of EB require the most treatment modifications. Extreme care of fragile tissues is important: little pressure (compressive forces) can be applied, but no sliding movements should be used. Lips should be lubricated before any procedure is performed. Bullae occurring during treatment need to be Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 82 drained to avoid spreading. Sutures can be used safely in all patients. Conclusions: This guideline will help dentists treating patients with EB to provide safe and evidence based treatments. P1B101 Examining ‘transportability’ issues in systematic reviews: The example of parenting interventions in developing countries Gardner F1 , Knerr W1 , Lucie C2 1 Oxford University, UK, Dept of Social Policy & Intervention; 2 Oxford University, UK; University of Cape Town, South Africa Background: Parenting interventions have been shown to improve parent/child outcomes in high-income countries. Many are potentially applicable across cultures and countries, but little is known about their effectiveness in low-and middle-income countries (LMICs). Prior reviews identify few trials in LMICs and even fewer trials of interventions with prior established efficacy. Global interest in parenting interventions has not been met by empirical guidance on whether/how interventions translate between countries. Objectives: To investigate effectiveness of parenting interventions for reducing harsh/abusive parenting and increasing positive parenting in LMICs; to explore methods for sysnthesising characteristics of ‘transported’ interventions, as a step towards further understanding transportability mechanisms. Methods: A range of databases were searched; unpublished reports were hand and online-searched, and experts contacted. Included were randomised trials with no-intervention, alternative-intervention, or services-as-usual comparisons. High heterogeneity precluded meta-analysis, but characteristics of included studies were discussed narratively according to type of delivery mode, outcome, level of evidence in country of origin, and extent of adaptation for current setting. Results: Twelve studies (N = 1580) in nine countries reported results favouring intervention on a range of measures, but overall validity is unclear due to significant/unclear risks of bias. Results of the two largest, highest-quality trials suggest interventions may be feasible and effective for improving parent–child interaction and parental knowledge/ attitudes in LMICs. Results provide initial evidence on feasibility of coding factors related to transportability across countries, although this was sometimes limited by poor reporting, and lack of efficacy evidence from countries of origin. Conclusions: While limited conclusions can be drawn, there is promising evidence that parenting interventions can be implemented and may be effective in improving positive parenting in LMICs. The results raise key methodological and implementation issues for wider investigation, e.g., how best to synthesise evidence about transportability; relative benefits of ‘transported’ versus indigenous interventions; adaptation to new service contexts. P1B102 Re-analysing equity in a Cochrane review: Smoking cessation in pregnancy O’Mara A1 , Oliver S1 , Chamberlain C2 , Caird J1 , Barnett-Page E1 , Thomas J1 Institute of Education, London, UK; 2 Centres Collaboration, Women and Children’s Program, Southern Health, Clayton South, Australia 1 EPPI-Centre, Background: A Cochrane review about smoking cessation in pregnancy concluded that interventions can reduce the proportion of smokers and the incidence of low birthweight and preterm births Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. (Lumley et al. 2009). Another systematic review (Arblaster and colleagues 1996) suggested a number of intervention features that are promising for reducing inequalities, such as the involvement of peers. Objectives: To assess the impact of interventions on smoking cessation in pregnancy for disadvantaged groups, and to evaluate the relative effectiveness of intervention features suggested by Arblaster and colleagues for reducing health inequalities. Methods: Various meta-analytic models (meta-regression, multivariate multilevel modeling) assessed the relative impact of interventions on women differing in their socioeconomic status (SES) and ethnicity. The effectiveness of intervention features at reducing inequalities was also explored: using incentives; addressing the expressed concerns of pregnant smokers; involving participants’ peers; and consulting women to design interventions. Some variables were addressed descriptively (e.g., young mothers as a disadvantaged group; structural measures as an intervention feature). Results: Interventions reduced the number of women smoking in late pregnancy regardless of SES or ethnicity. Interventions with incentives were more effective than those without; no other intervention features were statistically significant predictors of effectiveness. Interventions were relatively more effective at increasing birthweight than stopping smoking altogether. Conclusions: Smoking cessation programmes are effective for all women and should be widely implemented. Analysis is ongoing to identify promising elements for smoking cessation interventions to reduce inequalities. References 1. Lumley J, Chamberlain C, Dowswell T, Oliver S, Oakley L, Watson L. (2009) Interventions for promoting smoking cessation during pregnancy. In: the Cochrane Database of Systematic Reviews, 2009: Issue 3. Art. No.: CD001055. DOI: 10.1002/14651858.CD001055. pub2. 2. Arblaster L. (1996) A systematic review of the effectiveness of health service interventions aimed at reducing inequalities in health. Journal of Health Services Research and Policy 1: 93–103. Knowledge translation in patient safety Day: 20 Oct (P1B103–P1B117) P1B103 The building blocks of safety: Creating multi-disciplinary micro-systems and departmental PSOs as the foundation for a successful patient safety program Shaw K1 , Rickard S2 , Reay T3 , Thomson D4 1 Department of Pediatrics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA; 2 Center for Complexity and Systems Thinking, Lund University, Lund, Sweden; 3 Department of Strategic Management and Organization, School of Business, University of Alberta, Edmonton, Canada; 4 Cochrane Child Health Field, Department of Pediatrics, University of Alberta, Edmonton, Canada Background: The Children’s Hospital of Philadelphia (CHOP) has created an infrastructure to foster patient safety (PS) and communication across disciplines, across units, and between unit-based and executive leadership. The patient safety program at CHOP: Patient safety programs must be designed to consider the following organizational factors: Structures (formal reporting relationships and grouping of individuals). The Chairs of the Academic Departments and hospital Chief Operating Officer (COO) supported the formation of Departmental Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 83 Patient Safety Officers (d-PSOs) from the key clinical areas who work with both hospital leaders and unit-based clinical leadership. Systems (ways tasks are organized and completed). To support collaborative work and learning, multi-disciplinary micro-systems were created at the unit level with nurse/physician partnerships, Quality and PS Committees, quality coordinators, and systems which allow staff of all levels time to participate in PS meetings and walk-rounds. Leadership (support from multiple champions from different areas of the organization). Physicians, key members of PS teams, were provided with protected time and financial support to partner with nursing staff at the unit level and to create d-PSOs to promote communication between the units and the hospital administration. Culture (beliefs and assumptions that guide activities). A ‘just culture,’ where individuals feel free to report and learn from safety events (without feeling threatened by the consequences), has been introduced. The Unit-based PS walk-rounds and unit-based Quality and Safety Committees allow all staff, clinical and non-clinical, to identify safety issues and practices, foster communication, and promote analysis of problems. Politics (use of power to accomplish particular goals). The PS program was facilitated by recognizing and working with varying power levels within units (micro-system) and the hospital leadership (macro-system). Conclusions: At CHOP, the alignment of organizational structures, systems, leadership, culture and power have established the foundation for becoming a high reliability organization in regards to patient safety. P1B104 Assessment of hospital capacities in patient safety in the Philippines Acuin J1 De La Salle University, Philippines 1 Background: Despite growing public concerns and more stringent government regulation, Philippine public hospital programs to ensure patient safety are perceived to be hampered by severe infrastructural limitations. Little attention, however, is paid to human resource and programmatic constraints in aligning hospital cultures with organizational goals. Objectives: To assess human resources, programs and patient safety climates in government hospitals. Methods: Four 200- to 400-bed public hospitals (one children’s hospital (A), one transplant center (B), one general hospital in an island province (C) and another in Manila (D)) that had begun patient safety programs were included. Human resource and programmatic markers, based on Department of Health hospital accreditation standards implemented since 2010, were determined from key informant interviews and policy document reviews using the Institute for Healthcare Improvement (IHI) framework for achieving patient safety and high reliability. The 19-item IHI Safety Climate Survey was administered to managers and staff in high-risk patient care areas. Summary estimates were calculated for each hospital. Results: Except for hospital C, the hospitals generally had designated officers and working committees. Hospital C had the lowest nurse to patient bed ratio (1:38). Programmatic markers were generally absent in all four hospitals. Thus, even when working committees are present, the scopes of responsibilities in ensuring patient safety appear limited and underdeveloped. Except for hospital A, the hospitals had no patient disclosure processes. About 60% of the 131 respondents (range, 50%–82%) viewed the safety climate as positive. Overall and patient safety means were highest in hospital B. The lowest scored items involved management action on staff’s safety suggestions and management’s willingness to compromise safety concerns for productivity. Conclusion: The number of hospital staff engaged in patient safety appears to be adequate. However, they have limited formal programmatic roles and have fair regard for their hospitals’ safety climates. P1B105 Safe care for children: A scoping review of what we know, and what we need to find out Thomson D1 , Seto I1 , Scanlon M2 1 Cochrane Child Health Field, Department of Pediatrics, University of Alberta, Edmonton, Canada; 2 Department of Pediatrics, Critical Care, Medical College of Wisconsin, Milwaukee, WI, USA Table 1 (P1B104). Patient Safety Markers and Safety Climate Scores in 4 Philippine hospitals. Hospitals Patient safety markers and safety culture scores A B C D Average Designated QI / PS Officer Working Patient Safety Committee Working Therapeutics Committee Working Infection Control Committee Infection control nurse to bed ratio Adverse event tracking systems Healthcare associated infection surveillance Patient falls prevention Standardized patient disclosure processes Overall Mean—All Questions & Respondents∗ Safety Climate Mean∗ Percent Respondents Viewing Safety Climate as Positive: Number of respondents Yes Yes No Yes 1:250 Yes No No Yes 3.97 3.91 0.53 40 Yes Yes Yes Yes 2:400 Yes No No No 4.35 4.24 0.82 38 No No No No 4:150 No No No No 4.07 3.98 0.50 16 Yes Yes Yes Yes 2:350 No No No No 4.10 3.93 0.51 37 – – – – – – – – – 4.13 4.02 0.60 131 ∗ based on this scale: 1—disagree strongly; 2—disagree slightly; 3—neutral; 4—agree slightly; 5—agree strongly Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 84 Background: Children differ from adults in many profound ways. These unique aspects to childhood affect all aspects of medical care, including patient safety. Therefore, any study of pediatric patient safety must be carried out on the basis of research which explicitly incorporates an understanding of the physiological and psychosocial stages of childhood. Objectives: To map the research published in pediatric patient safety in the period 1999–2010. To establish what this research demonstrates about the characteristics of childhood that affect patient safety, as well as the gaps that remain. Methods: We carried out a scoping review of literature in pediatric patient safety from the period 1999–2010. Scoping reviews illustrate the volume, nature and characteristics of a body of research. They identify and map all relevant literature, regardless of study design, and identify research gaps (Arksey and O’Malley 2005). We searched for all literature, and then identified a subset concerned specifically with pediatrics. We analysed this subset to identify what is known about pediatric characteristics of patient safety. Results: Preliminary results: pediatric trials are between 1.47% and 7.21% of total published studies each year, whereas children in the 0–19 age range make up 34.9% of the world’s population (US Census Bureau). Final results will be discussed at the Cochrane Colloquium. We found two broad categories to pediatric aspects of patient safety: 1) Nature of childhood (varies across age groups), e.g. physiological, developmental, social/family context; and 2) Aspects of the system of care (varies across settings), e.g. medication prescribing and administration, organizational approaches (including staffing, education and training), and technology (e.g. computerized physician order entry systems). Conclusions: Addressing pediatric patient safety needs to be based on evidence that considers the particular physiological and psychosocial aspects of childhood and adolescence, as well as the settings in which pediatric care is administered. P1B106 Six Sigma applications for patient safety in a university hospital Mejia HM1 Management Chief University Hospital Fundacion Santa Fe de Bogota Colombia 1 Clinical Background: The Fundación Santa Fe de Bogota University Hospital is a private high-complexity institution with 205 beds. Since its foundation in Bogotá in 1983, the hospital has been recognized for its commitment to safety, innovation and the search for excellence in all its processes, and it was awarded accreditation by the Joint Commission International. Objectives: The Six Sigma improvement tool was implemented in 2003. With its structured problem-solving approach and the zero-error goal, it has had a positive impact on patient safety through different project implementations. Methods: At present, we run our own independent training process and there are already 488 green belts, 41 black belts and 4 master black belts. Our portfolio of projects consists of 88 projects, 57% of which focus on patient safety. Results: Examples of projects on patient safety: 1. Prevention of ventilator-associated pneumonia (VAP) in critical care units, 2007–2010: preventive measures were standardized and effective controls were put in place, improving from Sigma levels 1.7 to 2.5. 2. Optimization of the blood transfusion process, 2007–2010: improvement by means of process standardization and the implementation of controls translated into a Sigma level improvement from 4.0 to 4.7. Conclusions: Because of their extreme variability, healthcare processes are a great opportunity for improvement. The Six Sigma training and project development methodology support patient safety culture and help create open minds. Six Sigma has taught us that well-chosen projects, well-integrated teams and a committed leader are the keys to success. Six Sigma strengthens teamwork and the search for excellence in patient safety. Figure 1 (P1B106). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 85 Figure 2 (P1B106). Figure 3 (P1B106). P1B107 A survey analysis of patient for patient safety in China Zhang Q1 , Li Y1 , Zhang L1 , Yin Q1 , Shang L1 , Wei X1 , Zhang M2 1 West China School of Medicine, Sichuan University, Chengdu, China; 2 Chinese Cochrane Centre, West China Hospital, Sichuan University, Chengdu, China Background: Patient safety involves not only medical institutions and medical workers but also patients. Much research has been done to promote patient involvement in patient safety recently. But very Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. little is known of this area in China. Objectives: To investigate the knowledge, attitudes and actions of patients toward patient safety and to provide evidence on how to promote patient safety in China. Methods: A questionnaire was developed on patient safety and distributed to patients by a group of volunteer medical students. SPSS and MS Excel were used to analyze the data. Results: The volunteer medical students distributed 350 questionnaires at random to patients, of which 326 were completed with some items left blank. The sample of patients had a mean age of 38.8, 42.2% were male and 57.8% were female, and over 88% had completed less than undergraduate Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 86 Figure 4 (P1B106). Figure 5 (P1B106). Figure 6 (P1B106). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 87 Table 1 (P1B107). Representation of patients’ knowledge, attitudes and actions of patient safety in China. (Zhang et al.). Characteristic All patients If you have insurance about your health? Yes No What is your biggest worry in your medical process? High cost If the treatment is effective The adverse effect of the treatment The quality of the hospital The quality of the doctors Do you know what is a medical error? Yes No Do you know the your drug’s adverse reaction? Yes No Do you know that it is possible that you could be infected in hospital? Yes No Do you think that the doctors/ nurses should wash their hands before doing examinations on you? Yes No Do you think that the medical device the doctors/nurses use is disinfection? Yes No I don’t know Will you remind the medical staff if you find something unhygienic? Yes No Will you take legal action if you or your family member is the victim of the medical error? Yes No Patients No. (%) 326 (100) 326 (100) 261 (80) 65 (20) 326 (100) 62 (19) 134 (41) 52 (16) 7 (2) 22 (7) 326 (100) 110 (34) 216 (66) 324 (100) 103 (32) 224 (68) 326 (100) 104 (32) 222 (68) 325 (100) 287 (88) 38 (12) 322 (100) 241 (75) 9 (3) 75 (22) 324 (100) 291 (90) 33 (10) 326 (100) 238 (73) 88 (27) P1B108 Patients for patient safety research in The Cochrane Library Zhang Q1 , Zhang M2 1 West China School of Medicine, Sichuan University, Chengdu, 610041 China; 2 Chinese Cochrane Centre, West China Hospital, Sichuan University, Chengdu, 610041 China Background: Patients for patient safety has been increasingly important and much research has been done in the last decade. Research has focused on assessing the patients’ attitudes toward patients’ engagement and to evaluate interventions to encourage patients in safety-related actions. Objectives: Our study aims to investigate the current research status of patients for patient safety in The Cochrane Library. Methods: We developed a search strategy with MeSH terms including iatrogenic disease or medical errors/ prevention and control, medical errors/ adverse effects, safety management or cross infection/ prevention and control. The Cochrane Library Issue 4, 2011 was searched. We selectively reviewed the evidence that was likely to assess patients’ attitudes or evaluate interventions to promote patient participation in patient safety by first screening identified titles and then re-reading the abstracts. Results: Searches of The Cochrane Library yielded 236 candidate articles, of which only three studies met our needs. There were no studies designed to assess patients’ attitudes to patient safety. Three interventions from three studies published from 2008 to 2010 were evaluated. They were treatment education programs, secure electronic messages and a decision aid about risk of acute coronary syndrome. All of these interventions aimed at increasing the knowledge and information of patients, which could promote patient involvement in patient safety. Two of them were in specialist areas: one on occurrence of hemorrhagic or thromboembolic events and another on acute coronary syndrome. Except for one study that is still in progress, other two studies showed improvement in patient safety. Conclusions: Patient safety research, which is very important, is very limited. Greater attention should be paid by the Collaboration to this field to ensure that safer health care will be provided for patients. P1B109 Patients for patient safety: A systematic review Li Y1 , Huang Y1 , Nie Y1 , Li J1 , Zhang M2 1 Dept. of evidence-based medicine and clinical epidemiology; 2 Chinese Cochrane Center education. About 80% of patients had health insurance. Most patients thought that effectiveness of the treatment was their biggest worry during the medical process, while only 16% of patients worried about the adverse effect of the treatment. Of the patient respondents, 66% do not understand the concept of a medical error, 68% do not know the adverse effect of their drugs, and 68% do not know the possibility of infection in hospital. However, 88% of patients know that medical staff should wash their hands before doing examinations and 90% of patients would remind the medical staff if they find something unhygienic. If they or their family members were victims of a medical error, 73% of patients would take legal action. Conclusions: Patient safety information needs to be disseminated among patients. Most patients trust doctors and do not care about their safety on their own initiative. However, most patients show enthusiasm to protect themselves when medical error happens. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: Patients play an important role in improving patient safety. However, little is known about how and when to involve patients, and the effects after involvement in patient safety. Objectives: To systematically review the literature, summarize and compare the information on patients for patient safety at home and abroad, including patients’ knowledge, attitude, influencing factors and the effects after participation in order to provide evidence-based suggestions for education, training and research of patients for patient safety in China. Methods: We systematically searched the Chinese Biomedical Literature Database, EMBASE and MEDLINE to identify primary studies about patients for patient safety. Two researchers independently identified the eligible studies, assessed the risk of bias using the Cochrane Handbook 5.0 and Critical Appraisal Skills Programme, and extracted data. A meta-ethnographic approach was used to summarize the results. Results: We identified 28 eligible studies with Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 88 Table 1 (P1B108). Details of studies that evaluated interventions to engage patients in safety-related action. (Zhang et al.) (n=3). Reference Pernod, G., J. Labarère, et al. (2008) Weingart, S. N., H. E. Hamrick, et al. (2008) Pierce, M. A., E. P. Hess, et al. (2010) Patients Diagnosed as having deep vein thrombosis or pulmonary embolism requiring therapy with a vitamin K antagonist for 3 months or more. 267 subjects who received and opened a MedCheck message from April 2001 to June 2002. Adults presenting to Emergency department (ED) with a complaint of chest pain who are being considered for admission for prolonged ED observation in a specialized unit and urgent cardiac stress testing. Intervention Outcome Main findings Interventional group received a specific oral anticoagulation treatment educational program. The occurrence of hemorrhagic or thromboembolic events. Main outcome criteria were observed 5 times in the experimental and 15 times in the control group. (p < 0.01). Patient education using an educational program reduced VKA-related adverse event rates. MedCheck, a medication safety application, sent patients a secure electronic message 10 days after they received a new or changed prescription. We analyzed patient and clinician response rates and times, examined patient-clinician communication about medications, and identified adverse drug events (ADEs). Patients and physicians responded promptly to patient-directed electronic medication messages, identifying and addressing medication-related problems including ADEs. Receiving CHEST PAIN CHOICE (a decision aid that communicates the short-term risk of acute coronary syndrome). Six outcomes, including the rate of delayed or missed ACS. Patients often identified problems filling their prescriptions (48%), problems with drug effectiveness (12%), and medication symptoms (10%). Patients experienced 21 total ADEs; they reported 17 electronically. In progress. five conducted in China. Except for one randomized controlled trial (RCT), the studies had cross-sectional designs. 1. Patients’ perceptions and attitudes: patients’ knowledge of patient safety was generally poor, but patients were eager to participate. 2. Factors affecting patients’ involvement: patient’s characteristics (gender, age, race and education), disease-related factors (severity of disease, symptoms, treatment and prognosis), medical staff-related factors (attitude of medical staff), and emotional factors (feeling uncomfortable). 3. How to participate: improving patients’ knowledge, acquiring therapeutic information, cooperating, reminding and communicating with medical staff, etc. 4. Outcomes after involvement: patients’ participation can effectively standardize the performance of physicians, prevent unsafe incidents, reduce harms, and promote patient safety. Conclusions: Current studies cover a wide range of topics related to patients engaging in patient safety, but research in China is very limited and covers only the patients’ knowledge and willingness. Future studies Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Evaluative method This pilot randomized trial will develop and evaluate a decision aid for use in ED chest pain patients at low risk for ACS, which could reduce the rate of delayed or missed ACS. should emphasize the ways to promote patients’ involvement in patient safety and the outcomes after involvement. * This study is funded by NSFC 70973083 and CMB. P1B110 Knowledge translation in patient safety: What barriers are present to assess the evidence in practitioners? Perez S1 , Rodriguez V1 , Cattivera C1 , Pardo J2 , Garcı́a Elorrio E1 1 IECS, Argentina; 2 Centre for Global Health, Institute of Population Health, University of Ottawa, Canada Background: The gap between evidence and practice is important for patient safety. Besides resistance to change, scarce leadership support, resource availability, access and appraisal of evidence are also contributing factors. To reduce this gap, a collaborative forum in Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 89 Table 1 (P1B110). Barriers identified Lack of economical support Language (english and others) Deficiency of training in meta analysis interpretation Absence of protected time during daywork Lack of formal training in EBM Lack of use of the Cochrane Library Absence of support from the Board of director Absence of support from the colleagues % (n/40 surverys) 92,50% (37/40) 45% (18/40) 32,5% (13/40) 37,5% (15/40) 35% (14/40) 27,5% (11/40) 25% (10/40) 22,5% (9/40) hospital patient safety was recently launched in Argentina, integrating patient safety practitioners and patient representatives. Experiences are shared and evidence is disseminated at the forum. Objective: To provide an exploratory description of the barriers to accessing and assessing the evidence in a professional community of interest in patient safety. Methods: After a literature review on instruments to reveal the presence of barriers, we developed a preliminary 25-item survey that was tested to confirm understanding. No qualitative techniques were conducted at this time, given the exploratory purpose of this report. The survey was distributed among the participants of the forum, who completed it online, to ensure confidentiality, after giving consent. Results: 40 surveys were collected (response rate 41%). The distribution of responders was 55.81% physicians, 27.91% nurses, 11.63% pharmaceutics and others 4.65%. 73.71% have more than 15 years working at the institute. 36.59% belonged to hospitals larger than 200 beds. We will describe the frequency of responses on the presence of certain barriers. Knowledge of The Cochrane Library was 72.5% but only 52.5% knew how to interpret a meta-analysis. Internet connection was not an issue for any responder. Conclusions: Many barriers were present among patient safety practitioners. Several findings will be confirmed in a larger study, which will also promote the use of knowledge translation techniques that will surely reduce the gap. This line of work may help to prioritize systematic reviews in this field to reduce the impact of such barriers. P1B111 Pediatric patient safety: the distinctive nature of childhood must be part of the equation Thomson D1 , Matlow A2 , Rickard S3 , Scanlon M4 , Williams K5 1 Cochrane Child Health Field, Department of Pediatrics, University of Alberta, Edmonton, Canada; 2 Medical Infection Prevention & Control Programme, The Hospital for Sick Children; Centre for Patient Safety, University of Toronto, Toronto, Canada; 3 Center for Complexity and Systems Thinking, Lund University, Lund, Sweden; 4 Department of Pediatrics, Critical Care, Medical College of Wisconsin, Milwaukee, WI, USA; 5 Cochrane Child Health Field; APEX Australia Chair of Developmental Medicine, University of Melbourne; Department of Developmental Medicine, Royal Children’s Hospital, Melbourne, Australia Background: Children’s health is affected by several characteristics: developmental change over time; dependency on adults; and a distinct epidemiological profile of disease. These items, when combined Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. with healthcare provider and system factors, create vulnerabilities for pediatric care that must be taken into account in pediatric patient safety (PPS) programs. The above-noted issues affect the risk of iatrogenic harm in many ways, of which the following are a few examples. 1) Children’s developmental processes affect their responses to medications, and medication dosages. However, there is a paucity of evidence that is directly relevant to children, and children of different ages, on the effectiveness and efficacy of drug interventions. The epidemiology of adverse events differs between adults and children. 2) Children’s dependency on adults can affect the risk of harm. For example, preverbal children cannot advocate for themselves. Moreover, family dynamics (e.g. multiple caregivers) can make it difficult to get a complete or accurate medical history. 3) In many countries, some health care is delivered in school and community settings, yet we know little about the potential for harm in these environments. More research is also needed on how delivery of care to children in lower-and-middle-income countries affects PPS. 4) Much health care for children is given by providers without specialized training in pediatrics (e.g. family physicians, staff in general emergency departments), who may not recognize differing trajectories of disease in children, which creates conditions for errors in diagnosis and treatment. Conclusions: The dynamic nature of childhood influences the accuracy of diagnoses, effectiveness of treatments and risk of harm to children. Including more specific information that is relevant to these issues in systematic reviews will advance PPS endeavours. More research is needed on PPS in various settings of care and on global health issues in PPS. P1B112 Evidence evaluation in ranking target areas of interest for a national patient safety campaign Lauvrak V1 , Norderhaug IN1 , Saunes IS2 , Krogstad U2 1 Norwegian Knowledge Centre for the Health Services (NOKC), Department of Evidence-Based Practice, Specialist Health Care Unit, Norway; 2 NOKC, Department of Quality Measurements and Patient Safety, Secretariat for the Norwegian Patient Safety Campaign, Norway Background: In 2009, the Norwegian Knowledge Centre for the Health Services (NOKC) was commissioned by the Ministry of Health and Care Services to prepare a national patient safety campaign. A broad group of health care experts was established to use a formalized consensus process to rank eleven possible target areas according the following set of criteria: 1) Potential for improvement in Norway; 2) Interventions documented as effective; 3) Access to data and methods to evaluate the effect of interventions; and 4) Support among health care professionals in Norway. Target areas were: safe surgery, postoperative infections, urinary tract infections, CVK-infections, medication errors and harm, stroke, waiting time cancer patients, patient falls, decubitus ulcer, early rehabilitation hip fracture, and rapid response team heart failure. Objectives: Provide information on the effect of specified interventions. Methods: For one area, a systematic review of primary studies was prepared. For eight areas, short reports, based on systematic reviews identified in The Cochrane Library, the CRD – databases and by hand searches, were prepared. The most updated (2005–2010) high quality systematic review for each predefined intervention was selected. The level of evidence for predefined patient safety relevant outcomes was graded according to the GRADE approach. Methods and results were communicated Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 90 written and orally to the expert group who scored each area against all four predefined criteria. Results: Most interventions were supported by low or very low quality of evidence. No intervention was rated high quality, and only a few were rated moderate quality. For some interventions, relevant scientific evidence was not found. Nevertheless, the consensus process resulted in ranking four areas (safe surgery, postoperative infections, medication errors and harm, and urinary tract infections) as appropriate for the campaign. Conclusion: Uncertainty of effectiveness did not in itself disqualify an area from being ranked as appropriate for the campaign, but it was transparently acknowledged together with the other three assessment criteria. P1B113 Opioids for chronic pain: evidence of effectiveness, consistency of use, and public health issues Harris J1 1 The Permanente Medical Group, USA Background: There is increasing concern about the personal and public safety of prescription opioids for chronic pain. In the United States (US), opioid prescription rates, emergency room visits, and mortality rates have risen sharply in the last 10 years. However, it is not clear that opioids are effective for chronic pain. There are high levels of adverse effects and psychiatric comorbidities among patients using chronic opioids. Consistent adherence to evidence-based best practices would significantly improve patient safety. Objectives: 1. To review and assess the literature on the relative effectiveness, risks and benefits of opioid medications for chronic pain. 2. To review and assess the literature on management of opioid use for chronic pain. 3. To suggest a research agenda to support more effective chronic pain management. Methods: We searched The Cochrane Library, other databases, and evidence generated to support guidelines for systematic reviews on opioid use in chronic pain. We then classified and critiqued the evidence. We also reviewed the evidence and behavior change techniques used to support safe prescribing and pain management at Kaiser Permanente Medical Centers. Results: Most literature in this area consists of observational studies that are low quality according to the GRADE system. Virtually all studies lasted less than 3 months, which is not considered chronic. Retrospective cohort studies described high levels of psychiatric comorbidities and adverse effects of opioids. Pilot studies have shown wide variance in opioid prescribing patterns, which seem stable over time despite educational interventions. Conclusion: The effectiveness, efficacy and safety of opioids for chronic pain are not clear based on current evidence. Increasing use of chronic opioids is associated with public health problems. More consistent, evidence-based prescribing patterns would improve effectiveness and safety. Research designs could be improved. P1B114 Evidence to support patient safety improvement Harris J1 1 The Permanente Medical Group, USA Background: The U.S. Institute of Medicine estimated that over 100,000 Americans die each year because of medical errors, and many more sustain illnesses or injuries. Adherence to evidence-based Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. best practices and reduction in unwanted variation in care would significantly improve patient safety. Objectives: 1. To review and assess the literature on effective practices to prevent events that should never happen, such as objects left in the body after surgery, medication errors, and central line and other hospital-acquired infections. 2. To review and assess the literature on effective implementation of safety improvement projects. 3. To suggest a research agenda to support safety improvement. Methods: We searched The Cochrane Library, major data bases, and evidence generated to support guidelines for systematic reviews on patient safety improvement. We then reviewed and critically appraised the literature we found. We also reviewed the evidence and behavior change techniques used to support effective safety improvements at Kaiser Permanente Medical Centers in Northern California. Results: Most literature in this area consists of observational studies, including cohorts and case series that would be classified as low quality in the GRADE system. Some studies are ‘action research,’ applying checklists and protocols and tracking results. Nevertheless, use of protocols and guidelines has resulted in ongoing, measurable improvements in several areas. Conclusion: The lower quality studies, checklists and organizational interventions reduced adverse event rates substantially. Because they might deny one cohort protection from adverse events, randomized controlled trials (RCTs) may not be ethical in safety sensitive areas. Research should use other designs to prove or disprove the effectiveness of processes to protect patient safety, and focus on methods to ensure consistent application of proven procedures. P1B115 Extracting clinically relevant messages from drug class review Melvin C1 , Carey T1 , Ranney L1 1 University of North Carolina at Chapel Hill, USA Background: Evidence from drug class reviews is often not accessible to practicing clinicians nor is it presented in a way that allows clinicians to use the information to guide treatment and prescribing decisions. Information on drug safety and efficacy is often difficult to interpret clinically, especially in the absence of head-to-head trials. Nevertheless, information from such reviews can be very helpful to clinicians as they evaluate evidence of drug efficacy for a particular medical condition. We describe a process for extracting key concepts and clinically relevant messages from a drug evidence review. Our outcomes of interest were remission, recurrence, and safety and adverse events associated with the use of anti-epileptic drugs for bipolar disorder. Objectives: Our objectives were to update and review evidence from a drug effectiveness report on the efficacy of anti-epileptic drugs for the treatment of bipolar disorder and to distil this evidence into clinically relevant messages for communication to prescribers through a national dissemination effort. Our overall goal was to improve treatment of patients with bipolar disorder. Methods: We began with a published drug effectiveness report and used standard systematic review techniques: an expert, scientific panel with no ties to pharmaceutical companies to update the drug effectiveness report, grade and synthesize evidence found in the updated review and published report, and determine clinical implications; and audience research with prescribers to inform the development of key concepts and clinically relevant messages. Results: Our approach reduced a 735-page review to ten key concepts and four clinically relevant messages while maintaining scientific integrity and Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 91 credibility. These messages were used to inform national dissemination strategies. Conclusions: It is possible to use established approaches to literature review and synthesis in combination with scientific expert review and targeted audience research to distil volumes of evidence into clinically relevant, short messages that busy clinicians can use with confidence. P1B116 Instruments and methodologies used for detecting and characterizing adverse events in ambulatory care: A systematic review Gaitán Duarte H1 , Vallejo Ortega MT1 , Garcı́a Dieguez M2 , Esandi ME2 , de Luca M2 , Milberg M2 , Cho M3 , Monserrat-Capella D3 , Reveiz L3 1 Universidad Nacional de Colombia, Colombia; 2 Academia Nacional de Medicina, Argentina; 3 Organización Panamericana de la Salud, Estados Unidos Background: Unintentional damage to a patient caused by health care is known as an adverse event (AE).The World Health Organization (WHO) published a review about methods and measures used in research to assess patient safety in primary care. The search was limited to studies published in English up to 2007; the review included only studies that evaluate one type of adverse event. A systematic review updating and complementing previous evidence is proposed. Objectives: 1. To describe methods, sources of information and tools used to estimate the frequency of adverse events. 2. To describe the adverse events reported in studies, according to their type, frequency, origin, preventability, and seriousness. 3. To compare instruments focusing on their constitutive domains, resources and applicability. Methods: An advanced search was performed in MEDLINE (PubMed), The Cochrane Library, LILACS, SciELO and EMBASE to look for randomized controlled trials; cohorts; case control studies; before and after studies; and cross-sectional studies. Studies had to include at least two different types of AE on ambulatory care regardless age or underlying condition. Tools to detect, qualify and classify AE were identified and compared. Two authors independently will review titles and abstracts to identify the eligibility criteria. Full text will be assessed to check the inclusion criteria and to extract data. Results: The search in MEDLINE identified 1836 articles, EMBASE 3846, LILACS 24, the Cochrane Database of Systematic Reviews 12, and DARE 37. Full results of the assessment of the identified Cochrane Reviews will be completed by October 2011. Conclusions: Findings will be relevant for further investigations on AE. Results of this systematic review will be used as a basis for the elaboration of the Investigation Protocol on Adverse Events in Outpatient Care for Latin America and Caribbean countries supported by PAHO. P1B117 Update of Cochrane Systematic Review of audit and feedback demonstrates stability in findings. Will future trials be helpful? Ivers N1 , Jamtvedt G2 , Flottorp S2 , Young J3 , French S4 , O’Brien MA5 , Johansen M2 , Odgaard-Jensen J2 , Oxman A2 , Grimshaw J6 1 Department of Family Medicine, Women’s College Hospital, Canada; 2 Norwegian Knowledge Centre for the Health Services, Oslo, Norway; 3 Surgical Outcomes Research Centre (SOuRCe), Central Sydney Area Health Service, Camperdown, Australia; 4 Primary Care Research Unit, University of Melbourne, Carlton, Australia; 5 School of Rehabilitation Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Science, McMaster University, Hamilton, Canada; 6 Clinical Epidemiology Program, Ottawa Health Research Institute, Ottawa, Canada Background/Objective: This review updates a previous Cochrane review aiming to assess the effects of audit and feedback (AF) on healthcare provider professional practice, which found a median 5% improvement in compliance with desired practice across a range of dichotomous outcomes. Methods: We searched MEDLINE, EMBASE, CINAHL and the Cochrane Effective Practice and Organization of Care (EPOC) database up to December 10, 2010 for randomized trials of AF. We defined AF as a summary of clinical performance given to health care providers. For this update, trials were included only when AF was judged to be the core, essential feature of the quality improvement intervention; thus, some previously included trials were excluded. Data were double-abstracted independently. For each comparison, we calculated the risk difference (RD) of compliance with desired practice, adjusted for baseline compliance. Results: 144 trials were included in the review; 91 were from the previous review. The interventions were heterogeneous in their design and addressed a variety of clinical problems. Considering only trials with dichotomous outcomes, when AF alone was compared to usual care (41 comparisons), there was a 4.1% increase in compliance with desired practice. When AF was considered a core, essential aspect of a multifaceted intervention (56 comparisons), the median increase in compliance was 5.3%. In trials comparing combined educational outreach plus AF to AF alone (16 comparisons), the median increase in compliance was 4.4%. Conclusions: AF has a small but important positive effect on clinician behaviour. This effect may be increased by concurrent interventions, especially educational outreach. This update led to minimal changes in the estimate of effect despite a substantial increase in the number of studies. The stability in effect estimate suggests that future trials comparing AF to usual care may not provide valuable new knowledge, unless carefully crafted to assess specific aspects of intervention design. Education and training Day 20 Oct P1B118 A survey of the awareness rate of GRADE in China Chen Y1 , Yang K1 , Wang Z2 , Yao L2 , Wang X2 1 Evidence-Based Medicine Center of Lanzhou University, China; 2 School of Basic Medical Sciences of Lanzhou University, China Background: Evidence based medicine and systematic reviews are becoming better known among clinicians, researchers and policy-makers; it is imperative to promote the development of GRADE in China. Although it has been ten years since the GRADE system first emerged, it is still a relatively new concept in China. Objectives: To investigate the awareness of the GRADE system in China. Methods: We conducted a two-part survey of health professionals. The first part using a survey questionnaire given to attendees on the 6th Asia-Pacific Evidence Based Medicine Conference on September 25; the second using a web-based survey on http://www.dxy.cn (The biggest medical community site in China) from October 13 to 28. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 92 Results: Of the 245 respondents who completed a questionnaire, 118 (48 %) had heard of the GRADE system. Among those who had heard of GRADE, 53 (45 %) had accessed the web site of GRADE; 20 (17 %) had used the GRADEpro program; 91 (77 %) didn’t knew how many levels existed for grading the quality of evidence and 102 (86 %) didn’t how many levels existed regarding the strength of recommendations in the GRADE system; 6 (5 %) knew of all the upgrading factors and 39 (33 %) knew of all of the degrading factors of ‘quality of evidence’; 8 (7 %) correctly answered factors affecting the strength of recommendations; 114 (96 %) thought GRADE was important or very important to clinical practice in China and 115 (97%) thought it was necessary to introduce and apply the GRADE system in China. Conclusions: The survey yielded useful information about respondent’s knowledge and perceptions of the GRADE system. Researchers and doctors have limited familiarity with the concept of GRADE but most of them thought the GRADE approach was important to clinical practice, and it that it is necessary to introduce and apply the GRADE system in China. P1B119 Building capacity in EBHC: What is happening in the African region? Rohwer A1 , Young T2 1 Stellenbosch University, South Africa; 2 Stellenbosch University, South African Cochrane Centre, South Africa Background: Evidence based Health Care (EBHC) is recognized internationally as a key competency for health care professionals. The practice of EBHC involves converting information needs into answerable questions; finding the best evidence; critically appraising the evidence for validity and usefulness; applying the results of the appraisal into practice; and evaluating performance. To inform EBHC capacity development initiatives, we assessed current training initiatives in the region. Objectives: To identify current training activities in the field of EBHC. Methods: We contacted medical schools, organizations and institutions as well as known experts in the field and searched the Internet. Information obtained was summarized in a table format and circulated to key contacts to validate. Results: Of 64 medical schools in the region, only 40 had available e-mail addresses. Of the 40 e-mails sent, mail delivery failed in six cases (15%) and four responded (10%). Nineteen institutions and organizations were contacted by sending 38 e-mails to various contact persons. Mail delivery failed in two cases (5%) and 14 responded (37%). Additional information on two training initiatives came from the Internet. We identified 37 training programs. Seven universities and 10 other organizations present these programs; 16 (43%) of the courses are presented in South Africa. Three are postgraduate degrees (Clinical Epidemiology (two), Applied Clinical Research and EBM,); one is a postgraduate diploma (Research Methodology); six are modules within university degrees (two undergraduate and four postgraduate); 19 short courses (≤1 week) and eight intensive courses (≥2 weeks) on EBHC. Two of the courses are e-learning courses, while three are blended. Conclusions: Despite low response, the findings indicate a paucity of EBHC capacity development programs in the African region. There is a need to use innovative strategies to increase the reach of EBHC capacity development programs. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P1B120 Reporting quality of randomized controlled trials on gastric cancer treated with traditional Chinese medicine published in Chinese journals Qi G1 , Lin X1 , Chen Z1 , Wang T1 , Ma B1 1 Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China Background: Randomized controlled trials (RCTs) of gastric cancer treated with traditional Chinese medicine (TCM) have become increasingly popular in China and have been published in large numbers. However, the quality of reporting in these RCTs was unknown. Objectives: To assess the reporting quality of RCTs of gastric cancer treated with TCM. Methods: We searched the Chinese Biomedicine Literature Database (CBM), Chinese Scientific Journal Full-text Database (CSJD), Chinese Journal Full-text Database (CJFD), and Wangfa database to the end of June 2010. The search terms included ‘Traditional Chinese Medicine’ and ‘Chinese herbs’ (MESH or EMTREE), ‘Gastric Cancer’, ‘Stomach Cancer’ and ‘Stomach Neoplasms’ (MESH or EMTREE). We also searched using the Google search engine using the snowballing method. The reporting quality of included studies was evaluated based on the CONSORT 2010 statement. Results: A total of 102 RCTs were included. Only 12 (11.8 %) RCTs included studies that reported specific inclusion and exclusion criteria. Some important methodological components of RCTs, such as: sample size calculation (0 %), randomization sequence (16.7 %), allocation concealment (0.0 %) and blinding (2.0 %), were incompletely reported. Only half of the trials (54.9 %) defined the criteria of subjective measures, and none of them registered in any clinical trials registry platform or agency. Conclusions: Although many such RCTs have been published, the reporting quality is troubling. As a potential key source of information for clinicians and researchers, not only were many of these RCTs incomplete, some contained mistakes or were misleading. Focusing on improving the quality of RCTs of TCM, rather than continuing to publish them in great quantity, is urgently needed in order to increase the value of these studies. P1B121 An effective model for training social workers in evidence-based practice in Korea Hwang S1 1 Kyungpook National University, South Korea Background: There has been a growing interest in evidence-based practice (EBP) among social workers in Korea. Objectives: This study was designed to evaluate the impact of various continuing education training models on the EBP process with community practitioners. Methods: A replicated one-group pre-test-post-test design was used on an outcome measure that assessed the level of workshop participants’ attitudes toward the EBP process. Results: There were significant changes between pre- and post-measurements and moderate to strong effect sizes (0.48 to 0.95) for the dependent variable in the desired direction. Conclusions: Overall, the results support the effectiveness of all three types of EBP training models for Korean social work professionals. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 93 P1B122 The evaluation of the effectiveness of a postgraduate training program of systematic review development Weng YH1 , Shih YH2 , Lo HL3 , Kuo KN3 , Chen CF4 , Chiu YW2 1 Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Taiwan; 2 Institute of Population Health Sciences, National Health Research Institutes, Taiwan; 3 College of Medicine, Taipei Medical University, Taiwan; 4 Department of Surgery, Taipei Medical University-Wan Fang Hospital, Taiwan Background: Systematic reviews provide critical exploration, evaluation, and synthesis of the unmanageable amount of information; and separate the insignificant, unsound, or redundant deadwood in the medical literature from the salient and critical studies that are worthy of reflection. They have been regarded as the highest quality of evidence for clinical practice. Objectives: The National Health Research Institutes of Taiwan provided a series of education for postgraduate health professionals to learn how to conduct systematic reviews. The curricula of the two-day workshop included training on how to perform meta-analysis using RevMan 5. The aim of this study was to investigate the effectiveness of this training program. Methods: A pre- and post-survey of participants was carried out to examine views related to systematic reviews including changes in beliefs, attitudes, knowledge, skills, and perceived barriers. Results: The participants’ professions included: physicians, nurses, pharmacists, nutritionists, and public health experts. After educational training, participants’ knowledge and skill in the production of systematic reviews significantly increased (P < 0.001). They were more likely to believe that systematic reviews can improve the quality of patient care (P < 0.001), and thus to support the implementation of systematic reviews (P < 0.001). The prevalence of perceived barriers to produce systematic reviews also declined (P < 0.001). In addition, the most common barrier to meta-analysis was lacking English language skills. Conclusions: The belief, attitude, knowledge, and skill in relation to systematic reviews were improved after the training program. This suggests that education is useful in disseminating systematic reviews. P1B123 Complex interventions required to comprehensively educate allied health practitioners on evidence-based practice Dizon JM1 , Grimmer-Somers K2 1 International Centre for Allied Health Evidence, University of South Australia, Australia and University of Santo Tomas, Philippines; 2 International Centre for Allied Health Evidence, University of South Australia, Australia Background: There is currently no strong evidence regarding the most effective training approach for allied health professionals that will support them to consistently apply the best research evidence in daily practice. Current evidence-based practice training tends to be ‘one size fits all’ and is unlikely to be appropriate for all allied health disciplines, because of the variability in their tasks and scope of practice. The scant body of evidence regarding the effectiveness of evidence-based practice training for allied health practitioners provides some support for improving knowledge and skills, but equivocal evidence about Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. influencing behaviours and attitudes. Findings: This paper is a reflection of the findings of our systematic review in relation to issues associated with training allied health professionals in evidence-based practice (EBP), particularly the training required to change their attitudes and behaviours towards research evidence implementation. There are key issues to consider when designing effective EBP training programs for allied health practitioners, such as; disciplines differ in terms of their contexts and scopes of practice, their clinical training and the research that matter to their day-to-day practice. We propose a new model of evidence-based practice training, based on the Medical Research Council’s concept of complex interventions and Hawe and colleagues’ work regarding standardizing complex interventions by considering the interventions as fixed (core, constant) components and variable components. We suggest that EBP training should be similarly considered, as a complex intervention, with fixed and variable aspects. Conclusions: We believe that by offering training in evidence-based practice based on complex interventions relevant to the needs of the attendees, using fixed and variable components, there may be greater success in significantly influencing knowledge skills, attitudes and behaviours. P1B124 Evidence based practice workshop for senior managers in Ethiopia Abdulwadud O1 , Young T2 , Hailu K3 , Rangwala A4 , Midekssa M3 , Volmink J2 , Garner P5 1 Ministry of Health, Ethiopia; 2 SA Cochrane Centre, SA; 3 AIHA-TC, Ethiopia; 4 AIHA, Washington USA; 5 Liverpool STM, UK Background: Ethiopia is a low-income country and has free access to The Cochrane Library (CL). There is only one Cochrane review author in Ethiopia and awareness about the Cochrane Collaboration (CC), the South African Cochrane Centre (SACC), the CL and evidence based practice (EBP) is poor. Objectives: To raise senior managers’ and partners’ awareness about EBP, the CC, SACC and lay a foundation to strengthen EBP. Methods: After four months of planning, a one-day national awareness-raising workshop was conducted on December 1, 2010 in Addis Ababa by the American International Health Alliance (AIHA), the SACC and Liverpool School of Tropical Medicine (LSTM). Results: Sixty-five senior managers from Ethiopia, South Africa, Nigeria, Namibia, Zambia, and Tanzania attended the workshop. The interactive workshop consisted of five sessions. It started with international and African initiatives in EBP including the work of the CC and the SACC in promoting EBP in Africa. Participants then presented on how EBP is promoted or used in their institutions and during the 4th session, a demonstration of the CL and how to search for evidence was given. Lastly, participants discussed how to advance EBP in their countries. Conclusions: The national workshop was very successful and participants learnt from it. However EBP faces many challenges in Ethiopia: there is only one Cochrane review author in Ethiopia, awareness is poor, and no institution is actively leading and supporting EBP activities. To address the challenges, a committee of key stakeholders was established to take over the leadership role to advocate for EBP and coordinate all the efforts to promote EBP in Ethiopia. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 94 P1B125 Measuring the quality of end-of-life care: Palliative care of cancer patients KO W1 1 National Institute for Cancer Research, Italy Background: Among various chronic diseases, cancer is one of which could cause enormous pain to a patient during the final stages of one’s life. The recent Hospice Movement since the 1960s marked a revival of palliative care as an alternative option for terminally ill patients. While a lot of palliative care emphasises the role of pain soothing, there are also substantial publications that try to measure the quality of the end-of-life care1,2 . Objectives: The quality of the end-of-life care is important as it enhances the capability of cancer patients to die in a dignified and preferred manner. However, existing indicators have different focuses and hinder cross-country comparisons. Methods: This paper attempts to do a systemic literature review of the European publications on the measurement of the concept of end-of-life care, with a particular attention given to cancer patients. Despite the differentiations on the symptoms and severity of pain suffered by cancer patients at the final stage of their lives, the paper aims at surveying the existing criteria employed in measuring the quality of the care offered. This is a precursor for attempting to find how a standardization of the measurement is viable for comparisons. Conclusions: The paper concludes with a discussion on the possibilities of deriving a comparable measurement on the quality of end-of-life care of cancer patients. References 1. Engelberg RA, Downey L, Wenrich MD, Carline JD, Silvestri GA, Dotolo D, et al. (2010) Measuring the quality of end-of-life care. Journal of Pain and Symptom Management, 39(6):951–71. 2. Singer PA, Bowman KW (2002) Quality end-of-life care: A gloabal perspective. BMC Palliative Care, 1(1):4. P1B126 Development of a systematic review unit of the Botucatu Medical School (FMB), São Paulo State University in Brazil El Dib R1 , Rudge MV1 , Schellini SA1 , Barretti P1 , Cataneo A1 , Nogueira CR1 , Fecchi D1 , Betini M1 1 São Paulo State University, Brazil Background: One of the aims of the Botucatu Medical School (FMB) is to introduce the principles of Evidence-Based Medicine (EBM) to clinicians, physician residents, graduate and postgraduate medical students, and health professionals. The new model of self-education allows the production of systematic reviews and meta-analyses and to help in the design and conduction of randomized clinical trials at the FMB Clinical Hospital. Objective: We present the development of a systematic review unit (SRU). Methods: A systematic review unit was established consisting of a scientist in EBM (RED) trained at McMaster University and worked in the Brazilian Cochrane Centre, two surgeons (SAS, AC), an information retrieval specialist (MB), three clinical physicians (MVR, PB, CRN), and one researcher (DF). Weekly meeting discussion of all members of the SRU is performed to access relevance prior to conducting a systematic review. Systematic reviews Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. and meta-analyses are conducted according to the principles of the Cochrane Collaboration and written based on the PRISMA statement. Furthermore, methodological studies are performed dealing with the problem of lack of clinical trials. All teachers, health professionals and students are welcome to perform systematic reviews. They are trained in EBM by workshops coordinated by Prof. Dr. Regina El Dib. Results: 22 projects for systematic reviews are currently underway (six are registered in the Cochrane groups). Other three completed systematic reviews were done and one of them paved the way for the successful design and conduct of a pragmatic trial. This trial and review were funded by the National Council for Scientific and Technological Development (CNPq). The other two reviews were also funded by CNPq and by São Paulo Research Foundation (FAPESP). Three clinical trials have been designed according to the reviews protocol. Conclusions: A SRU increased the production of systematic reviews and clinical trials to help patients make healthcare decisions. P1B127 Introduction to evidence based practice in the Humanitarian Services of the Belgian Red Cross-Flanders: Experiences from a newly developed workshop Dieltjens T1 , Adriaenssens L1 , Pauwels N1 , Van de Velde S1 1 Belgian Red Cross-Flanders, Belgium Background: Evidence-based practice (EBP) has become an integral part of the activities and programmes developed by the Belgian Red Cross-Flanders. The Humanitarian Services of the Belgian Red Cross-Flanders work with volunteers and staff that are, for the most part, unfamiliar with EBP. Effective communication and training is essential to avoid misunderstandings about the concept of EBP and to engage them towards EBP. Objectives: To evaluate a workshop designed to introduce EBP to a public with very different backgrounds. Methods: The main learning objectives were that the participants understand the importance of EBP and have a critical attitude towards the validity and usefulness of various sources of information. We will use brief presentations combined with an interactive approach and group work to facilitate the learning of the critical components of EBP. The workshop is designed in collaboration with didactical experts. A questionnaire was developed to measure the attitude of the participants towards EBP before and after the course. Results/Conclusions: The workshop will be piloted in June 2011. We will present the format of the workshop and the results of the pilot study during the Cochrane Colloquium. P1B128 Educational models and their application in medical research training: A meta-analysis Ejaz K1 , Zia N1 , Shamim MS2 1 Aga Khan University Hospital, Pakistan; Pakistan 2 Memon Medical Institute, Background: Different educational models by theorists take into account several elements of adult learning. In medical research training such models are seldom reported. Models by Kolb and Dunn & Dunn have proven to be successful in adult education. With reference Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 95 to medical research training the application of these models may prove to be helpful. Objectives: To assess the impact of various educational models on medical research training. Methods: This is an ongoing meta-analysis of observational studies. Our inclusion criterion is original research from all years and languages. Only those articles which have full-text assess are included. A search strategy based on our inclusion criteria was developed. To-date, ten databases have been searched. Data is being extracted using MeSH and key words, ‘medical’, ‘research’, ‘education’, ‘teaching’ and ‘educational models’. The retrieved articles are being reviewed and tabulated using a standardized form. The form includes: study description, rationale for selection, coding details, assessment of confounders, study quality and heterogeneity. The description of statistical methods, tables and figures are also being assessed. Results: We are currently in the process of identifying and screening the articles according to our inclusion criteria. So far 50,000 titles, from ten major databases, have been reviewed. Five met our inclusion criteria. We are searching further. We are removing the duplicate records and abstract-only articles. The full-text articles not fulfilling our inclusion criteria will be excluded and the reasons for their exclusion will be noted. Conclusions: As this is still an ongoing project we cannot make a final conclusion. However, the team’s current observation is that a minimal amount of original work is reported in online databases. Original research projects which involve medical research teaching use ad-hoc methods instead of standardized models. This may result in a future development of educational model for medical research training. P1B129 Mentoring clinicians in conducting systematic reviews through a clinical fellowship program Chau M1 , Pitt V1 , Green S2 , Gruen R1 , Bragge P1 1 National Trauma Research Institute & Monash University, Australia; 2 Australasian Cochrane Centre & Monash University, Australia Background: Knowledge of current clinical evidence combined with clinical experience is pivotal to evidence-based practice. However, successfully engaging clinicians in the process of: searching for, appraising, and interpreting research evidence is challenging due to competing clinical commitments. Objectives: The Neurotrauma Evidence Translation (NET) Clinical Fellowship Program aims to build capacity for evidence-based practice and clinical leadership through mentoring clinicians to conduct systematic reviews to inform clinical practice. Methods: Planning the design and implementation of the NET Clinical Fellowship Program involved a review of similar programs used to engage clinicians in systematic reviews. The program employs a research fellow and research assistant with extensive systematic review experience who are responsible for recruiting appropriate candidates and supporting them in identifying a clinical question, title registration, one-to-one training and mentorship through the entire systematic review process. Topic selection is supported through a related neurotrauma evidence project, The Global Evidence Mapping (GEM) Initiative, which has identified high priority neurotrauma topics where systematic reviews are lacking. The Cochrane Injuries Group has also been consulted to identify potential review topics (including reviews in need of updating). Results: The program commenced in 2010 and has recruited four clinical fellows to date. Potential challenges, and strategies for addressing these, are presented based on similar Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. reported experiences in the literature. Conclusions: Programs that engage clinicians in systematic reviews can support implementation of evidence into practice and promote clinical leadership. Consideration should be given to: how clinicians are recruited and engaged within programs, what topics should be addressed, how adequate training and support will be provided to meet candidates’ needs, and how the research findings can impact on local and international practice. P1B130 Current practice of Filipino physiotherapists in Metro Manila, Philippines Dizon JM1 , Grimmer-Somers K2 , Kumar S2 1 International Centre for Allied Health Evidence, University of South Australia, Australia and University of Santo Tomas, Philippines; 2 International Centre for Allied Health Evidence, University of South Australia, Australia Background and Objectives: Country-by-country differences exist in which physical therapists make decisions regarding treatment. In the Philippines, there is no published information about physical therapists and what underpins their decisions in patient treatment. Thus, we conducted this study aimed at creating a profile of physical therapists in the Philippines and their bases for selecting of treatment. Methods: We used the Physical Therapy Profile Questionnaire (PTPQ) from our previous study. The PTPQ was administered in two ways. Paper copies were distributed in the national capital region of the Philippines, where 71 % of physical therapists practice and, a link to an electronic copy of the questionnaire via the Survey monkey software. Results: A total of 133 (response rate of 61.5 %) physical therapists took part in this survey. Seventy-three (55 %) were females and 60 (45 %) were males. Only 12 (9 %) completed a master’s degree related to physical therapy. The largest percentage of respondents had been practicing for five years or less, N = 89 (67 %). The majority of respondents were in general areas of practice. Most physical therapists were based in hospitals and majority performed clinical work with/without other roles in practice. The most common bases for treatment interventions in daily practice were: (1) medical prescription, (2) clinical experience and (3) undergraduate education. When faced with a new or difficult case, the most common bases for treatment decisions were: (1) medical prescription, (2) clinical experience and hospital protocol (equally rated) and (3) journal articles. Conclusions: Physical therapists in the Philippines are composed of a young population and are mostly based in hospitals. Treatment is based on the doctors’ referral, which includes a medical prescription of treatment. This challenges their capacity to deliver treatment with the best evidence. Therefore, training on evidence-based practice may be useful for them to make recommendations for treatment with the best evidence. P1B131 Access alone does not guarantee use: The need for training to use evidence based resources Sriganesh V1 1 QMed Knowledge Foundation, India Background: In the last few years, access to evidence based resources has increased in developing countries. However there is not a Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 96 proportional increase in the qualitative and quantitative use of such resources. Observation: As information professionals helping health professionals in a developing country, we noticed that many of them were not aware of important resources and their access rights. If they knew about them, they did not know about their key features. For instance they were not aware of MeSH and the Single Citation Matcher function in PubMed, or about the importance of forest plots in Cochrane reviews. Our intervention: In the last three years we: delivered lectures about evidence based resources, informed about free access to the Cochrane Library and conducted training programs in the use of PubMed. This sensitization has resulted in requests for more such programs and also better utilization. Recently the Medical Council of India has also stressed the need for such programs in medical school programs. Steps ahead: We are planning to work in partnership with medical schools to introduce ‘Train the trainer’ programs for the effective use of evidence based resources. Through these programs, we wish to create pools of trainers in every medical school. We would like to measure the impact of our programs and see if the awareness promotion and training makes a difference in the quantitative and qualitative use of EBM resources; especially the Cochrane Library, for which we have a country wide license. Conclusions: We believe that if students learn to use evidence based resources correctly, it would become routine for them to use them when they become residents, medical teachers or practitioners. It is very important that if country-wide access to resources like the Cochrane Library is provided, it must be supplemented with awareness and proper training to translate to evidence based healthcare! P1B132 Igniting minds: Use of Cochrane resources as educational tools for training medical students in developing countries Chandra S1 1 INFORMER/South Asian Cochrane Network, India Background: Academic centers of excellence which focus equally on research, education and patient-care are needed to cater to current healthcare demands. Developed countries have been successful in integrating research into their medical curriculum, transitioning from a textbook-based system to an evidence-based one. Developing countries like India lack the necessary infrastructure, training and policies to implement such a system. Intervention: Based on this need to groom the next generation of healthcare professionals, a national platform to advocate and encourage the work of young medical researchers was formed under the banner of ‘The Forum for Medical Students’ Research, India’ (INFORMER). In 2009, at the Forum’s annual congress, Dr Prathap Tharyan, director of the South Asian Cochrane Network delivered a lecture on Evidence-based medicine (EBM). As a continuation, collaboration was forged between INFORMER and the Cochrane Centre based in Vellore. This collaboration was furthered during the Third South Asian Regional Symposium on Evidence-Informed Healthcare, at Christian Medical College, Vellore. Here representatives from both organizations deliberated and, with input from the Symposium participants, enlisted measures which they could adopt to increase student participation in the Cochrane Collaboration’s activities. Results: The measures included the setting up of a formal students’ group linked with the regional Cochrane Centre, journal club discussions, introductory workshops on Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. evidence-informed healthcare and Cochrane reviews, training modules targeted at teaching the basics of EBM to students, involvement of students in writing and various other aspects of Cochrane reviews, and establishing a database to link novices and mentors with similar research interests. Since the implementation of the aforementioned methods, there has been a surge in Cochrane activity from the Indian subcontinent. Conclusion: The Cochrane Collaboration can serve as a good quality, accessible and affordable educational resource for medical students, especially in developing countries which have inadequate academic centers of excellence. Information and communication technology Day 20 Oct (PB133–PB134) P1B133 Do e-interventions for medication management in ambulatory care pose a threat to patient safety? A systematic review Carling C1 1 Norwegian Knowledge Centre for the Health Services Background: Enthusiasm for the implementation of e-interventions for medication management to decrease adverse events is richly supported by empirical studies. However, there is concern that they can introduce new problems and facilitate or cause events harmful to patients. Objectives: To identify and summarize empirical evidence of e-intervention outcomes in ambulatory settings that represent a risk for patient safety. Methods: We conducted a systematic search of the literature for studies in which e-interventions, i.e., electronic patient record, clinical decision support, electronic transmission of prescriptions to pharmacies, or electronically-generated paper prescriptions, were employed in the process of medication management in the ambulatory setting. We included studies with quantifiable outcomes and a wide range of designs to increase the probability of finding seldom or previously unknown adverse outcomes in studies not necessarily designed or powered to identify them. Results: We identified 87 studies that met inclusion criteria whereof 55 were randomized controlled trials (RCTs) and two quasi-randomised. None set out to reveal threats to patient safety. One observational study found overall worse outcomes, i.e. more new e-prescriptions needed pharmacist intervention than new paper prescriptions. Two other observational studies reported that some out-patients did not retrieve their e-prescribed drugs from pharmacies. One RCT reported more major bleeds in computer-generated anti-coagulant dosing, although computer dosing resulted in better clinical events overall. Otherwise, e-interventions did not result in overall inferior patient or process outcomes. Conclusions: Evidence of threats to patient safety in the identified studies was scant and mainly anecdotal. Observational studies may identify previously unknown or seldom outcomes that are threats to patient safety. Studies using robust designs such as the randomized controlled trial powered to reveal seldom outcomes are needed to test hypotheses generated based on findings from observational studies. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 97 P1B134 Are computerized clinical decision support systems effective at improving the process of care and patient outcomes, and what makes an effective system? A decision-maker-researcher partnership systematic review Roshanov P1 , Hemens BJ1 , Nieuwlaat R1 , Souza N1 , Sahota N2 , Wilczynski J1 , Fernandes N1 , Haynes RB1 1 McMaster University, Canada; 2 University of Saskatchewan, Canada Background: Despite decades of research on computerized clinical decision support systems (CCDSSs), results from rigorous evaluations remain mixed and little is known about what makes an effective system. Previous systematic reviews seeking characteristics important for success were limited by small sample sizes and poor quality of primary studies. Objective: To determine if CCDSSs are effective at improving care processes or patient outcomes, and to identify characteristics associated with effectiveness. Methods: A systematic review of randomized trials comparing use of CCDSSs to usual practice or non-CCDSS controls in clinical care settings. We conducted literature searches to January 2010 in bibliographic databases and scanned reference lists. Guided by partnerships with clinicians and senior hospital administrators, we considered over 50 trial and system characteristics. Authors of all included primary studies were contacted to provide additional information and to help select features potentially associated with effectiveness. Two outcome categories were analyzed independently: ‘process of care outcomes’ (such as appropriate monitoring of disease) and ‘patient outcomes’ (such as values of test results). Results: In the 166 included trials, success rates for process of care and patient outcomes averaged 60% and 24%, respectively, across 6 categories of application: toxic drug monitoring and dosing, acute care, primary preventive care, chronic disease management, diagnostic test ordering, and drug prescribing. We will use univariate and multivariate analyses to identify features significantly associated with effectiveness. Conclusions: CCDSSs affect the process of care but have limited benefit for patients. Our analysis will provide empirical guidance on effectiveness, optimal design and implementation. Methods for preparing reviews (non-statistical) Day: 20 Oct (P2A135–P2A196) P2A135 The COMET (Core Outcome Measures in Effectiveness Trials) initiative Williamson P1 , Altman D2 , Blazeby J3 , Clarke M4 , Gargon E1 1 University of Liverpool, UK; 2 University of Oxford, UK; 3 University of Bristol, UK; 4 Queens University Belfast, UK Background: Systematic reviews are hampered by inconsistencies in outcomes assessed and reported in otherwise eligible studies. Many meta-analyses have to exclude key studies because relevant outcomes were not reported. Much could be gained if an agreed minimum set of appropriate and important outcomes was measured and reported in all clinical trials in a particular area. Why standardise outcomes? Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. The design of new trials would be simplified, their risk of measuring inappropriate outcomes would be reduced, and selective reporting of outcomes would be less likely. It would be easier to compare, contrast and combine studies in systematic reviews, and the choice of primary and secondary outcomes for reviews would be more straightforward. Core outcome sets would help review authors to present their findings clearly and succinctly, for example within Summary of Findings tables. Aims: COMET initiative brings together researchers interested in standardised outcome sets, with well-attended international meetings in 2010 and 2011. COMET aims to foster and facilitate research by providing guidance on developing a core outcome set, methods to include user involvement in this process, and preparing reporting standards for such projects. Work is ongoing to identify, collate and maintain relevant resources in a searchable database on the internet. More than 50 completed projects in various areas of health/health care have been identified. Several examples of planned and ongoing work have also been recorded. The database will be demonstrated and progress to date will be presented. Implications for The Cochrane Collaboration and systematic reviews: If successful, COMET will help review authors choose outcomes, and will increase the likelihood that these outcomes have been measured in eligible studies, thereby decreasing the likelihood of important studies being excluded. By improving the evidence base for reviews, COMET will make it easier for people to make well-informed decisions in health care. P2A136 Purity or pragmatism: how does systematic review methodology from health care translate to the field of development? Stewart R1 , van Rooyen C2 , de Wet T2 1 EPPI-Centre and Consumers and Communication Review Group, UK; Centre for Language and Culture, University of Johannesburg, South Africa; 2 Dept of Anthropology and Development Studies & Centre for Language and Culture, University of Johannesburg, South Africa Background: Systematic review methodology pioneered in health care has been applied increasingly to questions of importance to lower- and middle-income countries. The development industry itself is also engaging with the evidence-based approach with initiatives such as the Abdul Latif Jameel Poverty Action Lab (J-PAL) and the International Initiative for Impact Evaluations (3ie). However, the evidence-based approach does not necessarily translate smoothly to the field of development. Systematic review methodology is based on the gold standard of randomised controlled trials. This gold standard is now widely accepted in health care, but such a narrow approach to evidence is not so easily accepted in a field where trials are often lacking, solutions urgently required, and regional and local variations common. Methods: In the last twelve months, the UK Department for International Development (DFID) has undertaken a pilot, commissioning academics to undertake systematic reviews in development. We undertook one such review and in doing so made a number of pragmatic methodological compromises. Having completed and disseminated our review, we have now reflected on these compromises to assess what might be learnt for future reviews. Results: We have identified lessons for future commissioners and researchers employing systematic review Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 98 methodology in development. These relate to multi-disciplinary teamwork, application of regional filters, drawing on evidence from a range of study types, and exploring mechanisms for change through the development and testing of causal pathways. Conclusion: There are still unanswered questions for reviewers in the field of development, particularly surrounding questions of study design, generalizability and applicability. Whilst adhering to the highest standards of systematic review methodology may not be possible (or supported by all), through transparent and complete reporting, we believe these discussions can advance and a balance between purity and pragmatism achieved. P2A137 Reviewing psychometric properties of HRQL measures in mental health: Practical experience Papaioannou D1 , Brazier J1 1 ScHARR, University of Sheffield, UK Background: Generic health-related quality of life measures (HRQL) are claimed to be applicable across disease areas. It is important to have robust methods of reviewing evidence that HRQL measures are valid and responsive in individual conditions. Objectives: To review the methods of systematically reviewing the literature on the validity and responsiveness of the SF-36, SF-12, SF-6D and EQ-5D in five mental health conditions. Methods: Systematic reviews were undertaken in schizophrenia, depression, anxiety, bipolar disorder and personality disorders. Ten databases were searched up to August 2009. Studies were appraised and data extracted. A narrative synthesis was performed on construct validity including known groups validity (KGV) (ability to detect differences in HRQL scores between groups), convergent validity (CV) (strength of association between generic HRQL and other related measures, e.g. symptoms or function) and responsiveness (R) (i.e. changes in scores in responders/non-responders to treatment and correlation with changes in related measures). Results: There were several problems in the review process including determining eligibility at the abstract level and lack of relevant quality assessment criteria. The greatest problem was determining how to synthesize the data from included studies, particularly in relation to the robustness of methods used to assess KGV, CV and R. For example, schizophrenia studies demonstrated crude KGV by detecting a difference between scores of the general population and individuals with schizophrenia. The methods used to assess CV by measuring the correlation between HRQL and clinical measures might not truly indicate its validity or otherwise. Conclusion: There are no guidelines on how to perform such reviews. However, such reviews are essential to determine if generic HRQL measures are valid for use in cost-effectiveness analyses for all diseases/conditions. Determining the methods that are appropriate to demonstrate KGV, CV and R is essential to developing the methods of systematic reviewing psychometric properties of HRQL measures. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P2A138 External validity and generalizability of systematic reviews in primary health care: Adapting the methodology to conduct the review Nasser M1 , van Binsbergen JJ2 , van de Laar FA2 , van Weel C2 , Buechter R3 1 Peninsula Dental College, University of Plymouth, Plymouth, UK; 2 Department of primary care and community care, Radboud University Medical Centre, Nijmegen, Netherlands; 3 Department of Health Information, German Institute for Quality and Efficiency in Health care (IQWiG), Cologne, Germany Background: Systematic reviews of health care interventions can be helpful sources to inform clinical decision-making in primary care but need to provide adequate information for decision makers to make judgments about the external validity of the included studies and about the applicability and transferability of the results to their individual patients. Objectives: We aim to introduce a guide on how systematic reviewers can adapt the methodology of their systematic reviews to make them more relevant to primary health care. Methods: We developed a conceptual framework based on a literature search, guided by an extrapolation checklist for systematic reviews, the experience of the author team, and the Wonca definition of primary care for systematic reviews. The framework was discussed at the Heelsum workshops that aim to raise discussion on the alliance between primary care and public health. We piloted the checklist using 10 systematic reviews. Results: Based on the literature search, we identified sixteen questions across different steps of conducting a systematic review from identifying questions, searching literature towards quantitative synthesis that could guide the systematic reviews. The questions also cover the different aspects of primary care that we identified in the definition provided by Wonca for family medicine: diversity of patient groups, contextual factors, and person-centred approach. Conclusions: The 16 identified questions can help systematic reviewers in improving the relevance of their review to primary care. We are currently evaluating the external validity of 100 systematic reviews including Cochrane reviews using the conceptual framework and will present those results in the Cochrane Colloquium, October 2011. P2A139 Prioritising whether and when to update Cochrane reviews Hopewell S1 , Takwoingi Y2 , Sutton A3 1 UK Cochrane Centre, UK; 2 Public Health, Epidemiology and Biostatistics, University of Birmingham, UK; 3 Department of Health Sciences, University of Leicester, UK Background: Given the increasing workload of Cochrane Review Groups and review authors, there is a need to develop an evidence-based approach for updating Cochrane reviews based on priority rather than the current arbitrary and often unmet two years. Objectives: To develop and validate a decision tool to determine when a Cochrane review should be prioritized for updating. Methods: We have refined and amalgamated two complementary methodologies proposed for prioritizing systematic review updates namely: i) a qualitative tool based on a broad range of updating signals including: publication of new study, information from existing studies, changes in methodology, user feedback, and clinical question; and ii) formal Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 99 statistical methods which assess the inclusion of new studies and the likelihood that a review’s conclusions will change, if updated, based on the primary outcome meta-analysis within the review. Formal piloting was used to refine and test the validity and reliability of the enhanced tool based on a retrospective sample of reviews. All Cochrane reviews flagged as updated during 2009 were identified. For each of the reviews, the following data were extracted: i) whether the conclusions of the update had changed; ii) the meta-analysis of the primary outcome; and iii) the corresponding meta-analysis in the previous version of the review. In doing this, new trials added to the review in the most recent update were identified. The statistical methods (programmed using STATA macro metarank) to evaluate the quantitative signals for the need to update a Cochrane review for each meta-analysis dataset was then run on all identified meta-analyses. We then compared the results of the statistical analyses with results of the updated reviews to determine priorities for updating. Results: The results of this formal pilot and a demonstration of the decision tool will be presented at the Colloquium. P2A140 Uncertainty in control group risk can, in systematic reviews, undermine confidence in estimates of effect Guyatt G1 , Vandvik P2 , Spencer F1 , Akl E3 , Falck-Ytter Y4 , Schunemann H1 1 McMaster University, Canada; 2 Norwegian Knowledge Centre for the Health Services, Norway; 3 State University of New York at Buffalo, USA; 4 Case Western Reserve University, USA Background: In general, systematic reviews focus on estimates of relative effect when presenting binary outcomes. Ratings of confidence in estimates (quality of evidence) therefore reflect confidence in relative effect estimates. Treatment decisions, however, must be based on trade-off in absolute effects, often calculated by applying relative risk to estimates of control group risk. Objectives: To explore how uncertainty in control group risk influences confidence in estimates of effect in the context of evidence summaries prepared for clinical practice guidelines. Methods: We surveyed the evidence summaries prepared for the ninth iteration of the American College of Chest Physicians Antithrombotic Guidelines looking specifically at sources of estimates of control group risk and possible sources of uncertainty in control group risk. Results: We found numerous instances when uncertainties in estimates of control group risk potentially compromised confidence in estimates of absolute treatment effect. Examples included the following. Risk of bias in estimates of symptomatic thrombosis risk arose in randomized trials that used screening venography to diagnose asymptomatic thrombosis, and in observational studies that failed to report antithrombotic prophylaxis. Risk of bias also increased in observational studies that used billing codes to determine thrombotic outcome diagnoses. Small sample size in observational studies resulted in imprecision of estimates of control group risk. Reliance on estimates from randomized trials raised issues of directness or applicability to clinical populations. Inconsistency in results from observational studies sometimes undermined confidence in estimates of control group risk. Conclusions: Limitations in the quality of evidence regarding estimates of control group risk is an under-appreciated source of decreased confidence in estimates of absolute treatment effects in evidence summaries from systematic reviews. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P2A141 Estimating risk differences in patient-important events when studies have focused on surrogate outcomes: Application to systematic reviews and practice guidelines Guyatt G1 , Akl E2 , Murad MH3 , Falck-Ytter Y4 , Gould M5 , Eikelboom J1 1 McMaster University, Canada; 2 State University of New York at Buffalo, USA; 3 Mayo Clinic, Rochester, USA; 4 Case Western Reserve University, USA; 5 Keck School of Medicine of the University of Southern California, USA Background: Four chapters in the ninth iteration of the American College of Chest Physicians antithrombotic guidelines addressed thrombosis prevention. Assessing risk differences (RDs) in symptomatic thrombosis proved problematic because most studies focused on a surrogate outcome, asymptomatic thrombosis. Objectives: To implement optimal strategies assessing RD. Methods: Panels considered three approaches: i) using estimates of symptomatic thrombosis from randomized controlled trials (RCTs) for both control group risk (CGR) and relative risk (RR); ii) using estimates of symptomatic thrombosis from RCTs for CGR and estimates of a composite of asymptomatic and symptomatic events from RCTs for RR; and iii) using estimates of symptomatic thrombosis from observational studies for CGR and estimates of a composite of asymptomatic and symptomatic events from RCTs for RR. Results: All three approaches proved feasible and applicable; examples follow. The chapter dealing with prophylaxis in hospitalized medical patients used approach A and estimated RD −34 symptomatic thrombotic events per 1000 (95% CI −51 to 0), limited by imprecision. The chapter addressing orthopedic surgical patients, using approach B, chose an untreated CGR from the low-molecular-weight heparin (LMWH) arm of recent RCTs after accounting for the effect of LMWH against placebo in earlier studies. Using the RR from RCTs for the combined endpoint of symptomatic and asymptomatic deep venous thromboses (DVTs), they estimated the effect of unfractionated heparin as RD of −10 (95% CI −9 to −12). Using asymptomatic events required rating down for indirectness. The chapter addressing non-orthopedic surgery thromboprophylaxis used approach C for the intermittent pneumatic compression recommendation. An observational study provided an estimate of 26 events per 1,000. Use of RR from RCTs resulted in an RD estimate of −13 per 1000 (95% CI −7 to −20) rated down for indirectness because asymptomatic events were used for RR. Conclusions: A variety of approaches estimating risk differences when RCTs have focused on surrogate endpoints proved useful. P2A142 The evidence base for interventions delivered to children in primary care: An overview of Cochrane systematic reviews Gill P1 , Wang KY1 , Mant D1 , Hartling L2 , Heneghan C1 , Perera R1 , Klassen T3 , Harnden A1 1 Department of Primary Health Care, University of Oxford, Oxford, UK; 2 Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, Edmonton, Alberta, Canada and Cochrane Child Health Field, Department of Pediatrics, University of Alberta, Edmonton, Alberta, Canada; 3 Manitoba Institute of Child Health, Department of Pediatrics and Child Health, University of Manitoba, Winnipeg, Manitoba, Canada and Cochrane Child Health Field, Department of Pediatrics, University of Alberta, Edmonton, Alberta, Canada Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 100 Background: As a first step in developing a framework to evaluate and improve the quality of care of children in primary care, there is a need to identify the evidence base underpinning interventions relevant to child health. Objective: To identify all Cochrane systematic reviews relevant to the management of childhood conditions in primary care and to assess the extent to which Cochrane reviews reflect the burden of childhood illness presenting in primary care. Methods: We used the Cochrane Child Health Field register of child-relevant systematic reviews to complete an overview of Cochrane reviews related to the management of children in primary care. We compared the proportion of systematic reviews with the proportion of consultations in Australia, US, Dutch and UK general practice in children. Results: We identified 396 relevant systematic reviews; 358 included primary studies on children while 251 undertook a meta-analysis. Most reviews (n = 218, 55%) focused on chronic conditions and over half (n = 216, 57%) evaluated drug interventions. Since 2000, the percentage of pediatric primary care relevant reviews only increased by 2% (7% to 9%) compared to 18% (10% to 28%) in all child-relevant reviews. Almost a quarter of reviews (n = 78, 23%) were published on asthma treatments which only account for 3–5% of consultations. Conversely, 15–23% of consultations are due to skin conditions yet they represent only 7% (n = 23) of reviews. Conclusions: Although Cochrane systematic reviews focus on clinical trials and do not provide a comprehensive picture of the evidence base underpinning the management of children in primary care, the mismatch between the focus of the published research and the focus of clinical activity is striking. Clinical trials are an important component of the evidence base and the lack of trial evidence to demonstrate intervention effectiveness in substantial areas of primary care for children should be addressed. P2A143 Variability of outcome measures among trials of topical Chinese herbal medicines for musculoskeletal injuries Wang L1 , Yuan Q1 , Li X1 , Li Y1 1 Chinese Cochrane Centre, China Background: Traditional Chinese herbal medicines showed promising effects for musculoskeletal injuries. However, most trials addressing the effects of topical Chinese herbal medicines are of low quality, often with variable and subjective outcome measures. Objectives: To assess the variability of outcome measures among trials of topical Chinese herbal medicines for musculoskeletal injuries. Methods: We searched PubMed, EMBASE, CENTRAL, Chinese Biomedical Database, CNKI, and VIP from the inception to December 2010. Randomized trials of topical Chinese herbal medicines for musculoskeletal injuries were included. We analyzed the outcome measures of the trials. Results: Forty-five trials were included. Seventeen trials reported the overall efficacy with four categories of ‘‘cured’’, ‘‘excellent’’, ‘‘good’’, and ‘‘no effect’’, and nineteen trials with three categories of ‘‘excellent’’, ‘‘good’’, and ‘‘no effect’’. However, criteria to determine the efficacy varied among trials. Among seventeen trials, ten reported curing using ‘‘symptoms and physical signs of patients disappeared, and physical function recovered to normal level’’; three adding ‘‘overall improvement >95%’’; two using ‘‘>90%’’. Two other trials did not report any definition of curing. The thresholds of ‘‘excellent’’ efficacy of overall improvement ranged from ‘‘>75%’’ (1 trial), ‘‘>70%’’ (6 trials), ‘‘>2/3’’ (11 trials), to Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. ‘‘>50%’’ (2 trials). Twelve trials described excellent as ‘‘symptoms disappeared and physical function recovered’’, coinciding with the definition of ‘‘curing’’. Four trials described ‘‘curing’’ as ‘‘symptoms and physical signs significantly improved’’. Eight trials reported pain score reduction. Among them, five trials used visual analogue scales, three used a self-categorized numerical rating scale, and one used the WOMAC index. Conclusions: The outcome measures of topical Chinese herbal medicines for musculoskeletal injuries greatly varied among trials, which may greatly influence the data extraction and synthesis, even mistakenly combining data for specific outcomes. Criteria to define the outcome measures and validated pain scores for Chinese herbal medicines are urgently required. P2A144 Still too little qualitative research to shed light on results from reviews of effectiveness trials: A case study of a Cochrane review on the use of lay health workers Glenton C1 , Lewin S2 , Scheel I3 1 Nordic Cochrane Centre, Norwegian branch, Norway; 2 Cochrane Effective Practice and Organisation of Care Review Group, Norway; 3 Norwegian Knowledge Centre for the Health Services, Norway Background: Qualitative research alongside trials of complex interventions can be used to explore processes, contextual factors or intervention characteristics that may have influenced trial outcomes. This research may also shed light on the results of systematic reviews of effectiveness by identifying factors that help explain heterogeneous results across trials. In a Cochrane review on the effects of lay health worker programmes, we identified 82 trials. These trials showed promising benefits but results were heterogeneous. Objectives: To use qualitative studies conducted alongside the included trials to explore factors and processes that might have influenced intervention outcomes. Methods: Qualitative research carried out alongside the trials was identified by contacting trial authors, checking references, searching PubMed for related studies, and through citation searches. For included qualitative studies, we extracted information regarding objectives, methods, and results. Results: For 52 (63%) of the trials, no qualitative research had been conducted. For 16 (20%) trials, qualitative data collection had been done but was unavailable or had been done before the trial. For 14 (17%) trials, qualitative research had been done during or shortly after the trial. Most of this qualitative research aimed to elicit trial participants’ perspectives and experiences of the intervention. A common theme was participants’ appreciation of the lay health workers’ shared circumstances, for instance, with regard to social background or experience of the health condition. In six studies, researchers explored the experiences of the lay health workers themselves. Issues included the importance of supervision and health professionals’ support or lack of support. Overall, descriptions of methods and results were often sparse. Conclusions: Qualitative studies carried out alongside trials of complex interventions could help us understand heterogeneity in systematic reviews of effectiveness. However, for interventions of lay health worker programmes at least, too few such studies exist for these opportunities to be realised. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 101 P2A145 Guidelines for conducting and reporting reviews of reviews: Dealing with topic relevance and double-counting O’Mara AJ1 , Jamal F2 , Parry W3 , Lorenc T4 , Cooper C5 1 EPPI-Centre, Institute of Education, UK; 2 Institute for Health and Human Development, University of East London, UK; 3 Centre for Longitudinal Studies, Institute of Education, UK; 4 Department of Social and Environmental Health Research, London School of Hygiene and Tropical Medicine, UK; 5 Peninsula Technology Assessment Group (PenTAG), Peninsula College of Medicine & Dentistry, University of Exeter, UK Background: Reviews of review-level evidence (tertiary reviews) are desirable when a research question is time-sensitive and/or the scope is broad. However, reviews included in a tertiary review often have only partial overlap with the tertiary review’s research question and, consequently, not all included studies are relevant to the tertiary review. Additionally, the reviews can include some of the same primary studies (known as double-counting). These concerns might lead to biases in the evidence base. Objectives: To explore the issues of relevance and double-counting in a tertiary review and to present guidelines for identifying and addressing potential related problems. Methods: We examined data from a completed systematic tertiary review on a public health effectiveness topic. We established the relevance of the included reviews by determining how many of the primary studies included in each review met our inclusion criteria and by analysing the included reviews’ synthesis sections using the PICO elements (population, intervention, comparison, and outcome) in relation to the tertiary review research question. We demonstrate graphical (plot-based) approaches and a matrix-based approach to establishing the extent of double-counting. Results: Of 20 reviews that met our inclusion criteria, 10 reviews had less than 50% of the primary studies included that were relevant to our research question. Exploring the synthesis sections of included reviews using a PICO framework was useful in establishing the degree of relevance of the findings to the tertiary review research question and yielded a review ‘utility’ rating. Our graphical and matrix-based approaches allowed us to evaluate the extent of double-counting across reviews; 14 primary studies were included in more than one review, with some studies appearing in four reviews. Conclusions: Issues of relevance and double-counting need to be assessed in tertiary reviews, but are often overlooked. The guidelines proposed can help identify potential biases and attempt to address them. P2A146 Using EpiData software to extract data for Cochrane systematic reviews Liu Z1 , Li X1 , Visconte D2 , Lewith G2 , Liu J1 1 Center for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, China; 2 Complementary and Integrated Medicine Research Unit, Primary Medical Care, Southampton, UK Figure 1 (P2A146). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 102 Background: Data collection is very important for systematic reviews. Data collection forms should be designed carefully to target the objectives of the review. Instruments for data collection and procedures used to collect data affect the accessibility and accuracy of extracted data. Objectives: We explored the feasibility of using EpiData software for data collection in a Cochrane systematic review. Methods: We used EpiDatatocollect data forthe review ‘Chinese herbal medicines for hypercholesterolemia’. We designed the data extraction form in MS Word based on the suggested model inthe Cochrane Handbook to include factors that should be considered in research methodology and study outcomes. We converted the Word document into EpiData questionnaires and reviewed and checked the information before creating a data file. The EpiData file was used for data extraction to ensure that all required data were obtained from the included studies. We also designed a Word document to extract text data longer than 80 characters. Results: Examples of items we extracted are illustrated in Figure 1. The extracted EpiData database was converted to a SPSS dataset and used for statistical analysis. It is noteworthy that the number of text characters should not exceed 80 inthe EpiData database; which is why we used aWord document to extract text data such as aims of included studies for tabular presentation in elements such as ‘Characteristics of included studies’. Conclusions: The EpiData application is suitable for review data extraction for quantitative data or text data less than 80 characters. It has many advantages over other tools used for data extraction such as Word or Excel formats. An intuitive user interface madeaccurate data extraction easy. Users can double enter data using the ‘match records by fields’ option to avoid data entry errors. We found that data extracted usingthe EpiData application could be easily converted to another dataset for statistical analysis. P2A147 Indirect meta-analysis of uncontrolled Studies about two independent regimes: A case report Sun T1 , Wang J1 , Fang J1 1 Shanghai Institute of Digestive Disease, Shanghai Jiao-Tong University School of Medicine Ren-ji Hospital, Shanghai 200001, China Background: Conducting randomized controlled studies inpopulations with rare diseases is challengingbecause of difficulties in participant recruitment. There are far fewer challenges in conducting uncontrolled studies in these populations, even if participant numbers are relatively small. Objectives: To investigate if indirect meta-analysis of uncontrolled studies in two independent regimeswas possible and whether the results werecredible. Methods: Systematic searches were performed in PubMed, The Cochrane Library, EMBASE and ISI Web of Knowledge to identify randomized controlled trials (RCTs) in gemcitabine-cisplatin compared with gemcitabine alone for advanced biliary tract cancer (BTC), or phase II trials reporting on the efficacy detected in gemcitabine alone or in combination with cisplatin for advanced BTC. The pooled results for each outcome of each regime were calculated, and compared using Chi2 analysis. Direct comparisons were also conducted to compare gemcitabine-cisplatin with gemcitabine. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Results: The pooled overall response rate (ORR), T-cell receptor (TCR), mean progression-free survival (PFS) and overall survival (OS) of 10 phase II studies (330 patients) in gemcitabine-cisplatin for BTC was 25.45% (95% CI 20.75% to 30.15%), 52.12% (95% CI 46.73% to 57.51%); 9.02 months, 13.65 months, respectively. The pooled ORR, TCR, mean PFS and OS of eight phase II studies (208 patients) in gemcitabine for BTC was 20.19% (95% CI 14.74% to 25.65%), 55.77% (95% CI 49.02% to 62.52%); 5.94 months, 4.98 months, respectively. Results of indirect comparison from 18 uncontrolled studies (538 patients) showed that there were no significant differences in efficacy (ORR, TCR, PFS and OS) between the groups. Results of direct comparison from three controlled studies (579 patients) showed gemcitabine-cisplatin combination prolonged the median time to progression, substantially increased the OS and produced a statistically significant increase in TCR compared with gemcitabine alone. Conclusions: From this case report, discrepancies between indirect and direct meta-analysis were found. Although indirect meta-analysis might be possible, we must be cautious about the results produced. P2A148 A rapid systematic review versus a Cochrane systematic review: An empirical comparison Rizzo M1 , Llewellyn A1 , Martin A1 1 Matrix Evidence, UK Background: There is increased pressure to conduct systematic reviews as comprehensively as possible within the constraints of a timescale that meets the needs of policy makers. Objectives: We aimed to evaluate how the findings and conclusions of a full Cochrane systematic review differ when certain aspects of the methodology are limited in order to conduct a more rapid systematic synthesis of the evidence. Methods: Blinded to the results of a Cochrane systematic review on the effectiveness of directly observed therapy for treating tuberculosis, we limited three key features of the Cochrane review protocol. 1) The literature search: using the same key words as those used in the full Cochrane review, we reduced the number of databases searched; 2) study selection: using the same inclusion criteria as the Cochrane review, we used only one researcher to select studies; and 3) data collection and synthesis: using the same data collection and synthesis procedures to meta-analyse the data, we used only one researcher to extract the data. Any queries regarding study selection and data collection were resolved with a second researcher. Results: The results compare how limiting certain aspects of the methodology of a full systematic review impact on: 1) the studies identified from the search; 2) the studies included into the review after screening; and 3) the results of the meta-analysis after data collection. Conclusions: The results of this more rapid systematic synthesis of data are compared with those of the Cochrane review to understand how limiting methodological aspects impact on the conclusions which can be drawn for policy makers. Based on these findings, the benefits and limitations of each approach are discussed, as are the limitations of the study. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 103 P2A149 Development and Evaluation of a Study Designs Typology for Medical Literature of Intervention Seo HJ1 , Kim SY2 , Park JE3 , Lee YJ3 , Shin SS4 , Hahn SK5 1 Department of Nursing, College of Medicine, Chosun University, Republic of Korea; 2 Department of Family Medicine, Hallym University College of Medicine, Republic of Korea; 3 Department of Health Technology Assessment Research, National Evidence-Based Healthcare Collaborating Agency, Republic of Korea; 4 Department of Pulmonology, Ajou University Hospital, Republic of Korea; 5 College of Medicine, Seoul National University, Republic of Korea Background: It is difficult to adequately classify the study designs of included studies in the process of performing systematic review. Objectives: To develop study designs typology in systematic reviews of intervention and test the inter-rater reliability, feasibility and comparison with an existing typology tool (Figure 1). Methods: To develop the typology, our study was conducted as follows: 1) establish a principle of development, 2) investigation of the existing study design typology, 3) develop new typology making up for the weak points in existing typology, 4) develop instructions for application of the tool to intervention studies, and 5) consult with experts about systematic reviews and clinical practice guidelines. To test the validity of the new typology, we used a sample of 70 studies. To assess the reliability of agreement among the three raters for all study design classification ratings and the agreement between the new typology and the National Institute for Health and Clinical Excellence (NICE) typology, we calculated the weighted kappa. We measured time and feasibility to apply the new typology tool compared with NICE. Results: Inter-rater reliability was almost perfect among the testers (κ = 0.82) and the mean time to complete the new typology was significantly shorter than for the NICE typology (113.4 seconds (95% CI 99.4 to 127.4) vs. 132.6 seconds (95% CI 116.1 to 149.1), P = 0.046 for paired t-test). To evaluate the clarity and simplicity using 7 point Likert scale, completeness, and user-friendliness, the new typology was rated with mean scores 6, 6.6, and 6.3, respectively. Conclusions: We suggest the use of the new typology to classify study designs of systematic reviews and guideline development for interventions. Application of the new tool to classify study designs should be accompanied by adequate training and decision-making based on explicit instruction. P2A150 Validation and application of a systematic review and narrative synthesis within mental health services research Bird V1 , Leamy M1 , Williams J1 , Le Boutillier C1 , Slade M1 1 Institute of Psychiatry, King’s College London, UK Background: The importance of combining qualitative and quantitative data when designing complex interventions for patient benefit is growing in recognition. Methods for systematically synthesising qualitative and quantitative evidence have been developed Figure 1 (P2A149). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 104 and are increasingly utilised in health services research. Objectives: To demonstrate both the validity of a systematic review and narrative synthesis and the application of the results in designing and testing a complex intervention for mental health service users. Methods: We combined qualitative and quantitative data by conducting a systematic review and narrative synthesis using a multistage literature search aimed at identifying mainstream and grey literature relating to patients’ experience of recovery. Three stages were included in the narrative synthesis with tabulation, cross-tabulation, in-depth thematic analysis and vote counting used to develop the preliminary synthesis. Relationships between studies of 1) differing quality and 2) different populations were explored using conceptual and methodological triangulation. We tested the validity of this approach through critical reflection, expert consultation with different stakeholder groups (patients, carers, expert researchers, health care professionals and individuals from black and minority ethnic backgrounds) and patient focus groups. We applied the results of the synthesis to develop a complex intervention. Results: 97 papers including 87 models of recovery were included in the synthesis. The validation phase suggested that the synthesis was a robust model which captured the experiences of patients. The synthesis has been used to inform development of a standardised measure (INSPIRE) and an intervention and training package for frontline mental health staff (REFOCUS). Conclusions: Narrative synthesis offers a robust and valid way of synthesising evidence, particularly where qualitative evidence predominates and for topics where patient experience is key. The narrative synthesis has been successfully applied to the design and development of tools currently being evaluated in a multisite cluster randomised controlled trial. P2A151 Systematic review and synthesis of quantitative and qualitative research: A worked example to assess community preferences Tong A1 , Howard K1 , Jan S2 , Cass A2 , Rose J1 , Chadban S1 , Allen R1 , Craig J1 1 University of Sydney, Australia; 2 The George Institute for Global Health, Australia Background: The synthesis of qualitative research and mixed method synthesis are important and emerging methodologies in health research. It is increasingly recognised as a useful approach for understanding consumer and community preferences for health services and policy. Objectives: To describe our methodology for appraising and analysing the findings of qualitative and quantitative studies on community preferences for the allocation of solid organs for transplantation, and to describe the utility of this approach. Methods: For the quantitative survey studies, we appraised study reporting according to a comprehensive list of criteria we developed based on items reported in the included studies and additional items in published guides for designing survey studies. We synthesised the survey results by extracting items assessed by the survey and the respective participant responses. For the qualitative studies, we used the consolidated criteria for reporting qualitative studies (COREQ) framework which included criteria specific to the research team, study methods, context of the study, analysis and interpretations. We performed a textual synthesis of the results and conclusions reported by the primary author of each study. Results: Fifteen studies were included (n = 5563 respondents). We identified 15 attributes relating to community preferences for Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. organ allocation and identified consistencies across survey studies in community preferences for prioritising for ‘younger age’, ‘time on waiting list’, and ‘better prognosis’ after transplant. Seven main themes underlying their preferences were identified from the textual synthesis: maximum benefit, social valuation, moral deservingness, prejudice, fair innings, first come, first served, and medical urgency. Conclusions: Using this approach, we found that community preferences for organ allocation hinge on a complex balance of efficiency, social valuation, morality, fairness, and equity principles. Pairing qualitative and quantitative studies within a review can generate more complete data, corroborate findings across studies, and enhance or explain insights attained with the complementary method. P2A152 Causal mapping for systematic reviews of complex social interventions Lorenc T1 , Clayton S2 , Neary D2 , Petticrew M1 , Whitehead M2 1 London School of Hygiene & Tropical Medicine, UK; 2 University of Liverpool, UK Background: There is increasing interest in the use of theory and logic models to inform systematic reviews of complex interventions. However, there is limited methodological guidance as to how theory can be integrated into systematic reviews. Objectives: To describe the use of causal mapping as a theory-based framework for a systematic review relating to crime, fear of crime, health and the built environment. Methods: Our causal map was intended to provide a framework for an evidence synthesis project including systematic reviews of intervention effectiveness and of qualitative evidence. We utilised a non-systematic iterative method to locate relevant theoretical and empirical literature, and synthesized findings and theories narratively using the emergent conceptual framework, in consultation with the review Advisory Group. The map was used to inform the development of methods for the systematic reviews. Results: The causal map included a wide range of factors and drew on theory and data in several different areas. It provided an outline structure of both the conceptual relationships of key review terms and their potential causal interactions. It also helped to identify potential pathways of intervention effectiveness; to clarify methodological issues in the primary evidence; and to situate the systematic review findings within a broader context. Purely theoretical literature was of less value than empirical observational and qualitative evidence, although the mapping methodology may not be well suited to the integration of insights from qualitative research. The validity of the mapping process is limited by the non-systematic methods; however, there are several reasons why full systematic review methods may not be appropriate for causal mapping. Conclusions: Causal mapping may be a valuable tool in systematic reviews of complex interventions. However, there remain a number of questions about appropriate methodologies for collecting and synthesizing theoretical and empirical literature for causal maps. P2A153 A worked example of ‘bestfit’ framework synthesis: A potentially more consistent or reliable form of qualitative data synthesis? Booth A1 , Cooper K1 1 University of Sheffield, UK Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 105 Background: A variety of different approaches to the synthesis of qualitative data are advocated in the literature, but none is recognised as the preferred method. Objectives: To describe the application of a pragmatic ‘best fit’ framework synthesis approach to qualitative evidence synthesis. Methods: An evaluation of a novel version of framework synthesis as an approach to qualitative systematic review. The case study was a systematic review exploring adults’ views about taking various potential agents for the primary prevention of colorectal cancer. An existing model was identified from the literature that conceptualised attitudes of adult women to the taking of several of these agents. The model identified did not entirely match the topic under study, but was a ‘best-fit’ and provided a relevant pre-existing framework and themes against which to map and code data. Results and Discussion: Twenty papers from North America, Australia, the UK and Europe met the criteria for inclusion. Fourteen themes were identified a priori from the previously identified model, which were then used to code the extracted data. The production of clear definitions for these themes reduced the scope for subjectivity of interpretation and thus enabled rapid and consistent coding of the studies’ data by more than one reviewer. The final synthesis also required some secondary thematic analysis to develop themes for data not covered by the a priori framework. The result was a more sophisticated model with additional themes. Conclusion: This novel and pragmatic ‘best fit’ approach to framework synthesis was found to be fit for purpose. It offered a means to reinforce, critique, and develop an existing published model, conceived for a different but relevant population, and produced a relatively consistent or reliable process when compared to more interpretative forms of synthesis. Future research should seek to test further this approach to qualitative data synthesis. P2A154 Mixed-methods synthesis for community-level public health interventions: Some considerations from a review of skin cancer prevention Lorenc T1 , Jamal F2 , O’Mara A3 , Cooper C4 1 London School of Hygiene & Tropical Medicine, UK; 2 University of East London, UK; 3 EPPI-Centre, Institute of Education, UK; 4 Peninsula Technology Assessment Group (PenTAG), Peninsula College of Medicine & Dentistry, University of Exeter, UK Background: The potential value of using systematic reviews of qualitative research and mixed-methods synthesis in conjunction with reviews of the effectiveness and cost-effectiveness of interventions is generally recognised. However, few studies have directly compared different synthesis methodologies. Objectives: To explore methods for mixed-methods synthesis in the context of community-level interventions, and identify potential challenges and directions for future research. Methods: A systematic review of the effectiveness of community-level skin cancer prevention interventions was conducted by CRD1. Our systematic review of qualitative evidence, funded by NICE, was conducted in parallel and aimed to identify barriers and facilitators to such interventions. We used thematic synthesis to identify themes in the qualitative evidence. After the completion of the NICE review, we carried out further analysis comparing two methods for cross-review synthesis: constructing a matrix of themes from the qualitative research and identifying themes addressed (or not addressed) in the intervention Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. research; and constructing causal pathways for selected interventions and using these to identify relevant qualitative themes. Results: Both models displayed the potential for qualitative synthesis to enhance the evidence base. However, analysis using a cross-study matrix for community-level interventions encountered some limitations, due to the lack of explicit cognitive content in the interventions, and to the limited extent of the effectiveness data. Causal pathway analysis provided a richer account of the relationships between the two forms of data, but does not enable rigorous testing of hypotheses, and requires potentially questionable interpretation of the effectiveness findings. Conclusions: Different approaches to mixed-methods synthesis have different strengths; there may be a trade-off between rigour and informativeness. The findings suggest broader questions about the role and purpose of mixed-methods synthesis. Reference 1. McDaid C et al. (2010). Sun protection resources and environmental changes to prevent skin cancer: A systematic review. York: Centre for Reviews and Dissemination.www.nice.org.uk/nicemedia/live/11871/ 49660/49660.pdf P2A155 Overviews of systematic reviews often do not assess methodological quality of included reviews Büchter R1 , Pieper D2 , Jerinic P3 1 Department of Health Information, Institute for Quality and Efficiency in Health Care (IQWiG), Germany; 2 Section EbM/HTA, Institute for Research in Operative Medicine (IFOM), Faculty of Health, Department of Medicine, Witten/Herdecke University, Germany; 3 University of Cologne, Germany Background: There is now often more than one systematic review on the same or a similar question, which has led to the development of systematic reviews of systematic reviews (overviews). Overviews offer the opportunity of providing decision makers with a broader summary of evidence and contrasting the results of different reviews on the same question. Little is known about their characteristics so far. Objectives: We set out to examine a sample of overviews in terms of different descriptive and methodological characteristics, including which databases were searched and how the quality of the included systematic reviews was assessed. Methods: We searched MEDLINE, DARE, the CDSR and different HTA databases for overviews of systematic reviews. Two reviewers independently screened titles and abstracts of the retrieved citations for eligible publications. Data extraction was conducted by one reviewer and checked by a second. Results: A total of 99 overviews were eligible. The median publication date was 2008 (interquartile range [IQR]: 2005–2010). The overviews included a median of 15 reviews (IQR: 7 to 35). Typically, four electronic databases were searched for systematic reviews (IQR: 2 to 6). The quality of the included reviews was systematically appraised in a good half of the overviews (56/99 [57%]). To assess methodological quality, most overviews used the Oxman&Guyatt Index (24/99 [24%]), followed by AMSTAR (8/99 [8%]) and PRISMA/QUORUM (5/99 [5%]). Primary studies published after the search date of the most recent included Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 106 review were included in 5 overviews [5%]. Conclusions: Overviews should use rigorous methods, if they ought to be a useful tool in health care decision making. The use of a quality assessment tool can help to determine if eligible reviews meet minimum quality requirements. In our sample only a small proportion of overviews used a validated quality assessment tool. P2A156 The data was unobtainable. A survey of the reporting of missing data in a sample of Cochrane reviews Struthers C1 , Noel-Storr A1 1 Cochrane Dementia and Cognitive Improvement Group, UK Background: It is widely acknowledged that publication bias is one of the most difficult to avoid in the preparation of systematic reviews. There are many factors that lead to data from clinical trials being unavailable for a meta-analysis, and this single source of bias significantly undermines the power of a Cochrane review to reflect the true picture of an intervention’s effectiveness. Objectives: We decided to investigate the occurrence and extent of unobtainable results data in studies included in Cochrane reviews. We hope that our findings will lead to a more robust, proactive and standardized reporting of unavailable data that will reduce its occurrence, and thereby improve the quality of Cochrane reviews. Methods: We developed a search strategy using terms designed to find all Cochrane reviews where data from trials eligible for inclusion were incomplete or unobtainable. We selected a random sample of 100 full reviews, and updated reviews. We also randomly selected Cochrane reviews with studies awaiting classification, or ‘ongoing’ to see if that masked underlying publication bias. Results: We analysed the reviews in both sample sets using a number of measures, including the proportion of eligible trials where the data were missing for the reasons outlined above. We also looked at the efforts made to obtain data, and how their absence was reported in the reviews. Discussion: As part of the development of a Cochrane-wide citizen science initiative, we are considering how to include the tracking of results data in a portfolio of volunteer tasks. We are looking forward to a future where public involvement can bring pressure to bear on research authorities to make sure that all clinical trials are registered, protocols adhered to, and results data are made freely available for analysis by review authors and others in a timely manner. P2A157 Mixing with the ‘unclean’: Including non-Cochrane reviews alongside Cochrane reviews in overviews of reviews Foisy M1 , Becker LA2 , Chalmers JR3 , Boyle RJ4 , Simpson EL5 , Williams HC6 1 Cochrane Child Health Field, Canada; 2 Cochrane Comparing Multiple Interventions Methods Group, USA; 3 University of Nottingham, UK; 4 Imperial College London, UK; 5 Oregon Health and Science University, USA; 6 Nottingham University Hospitals NHS Trust, UK Background: Overviews of reviews are ‘systematic reviews of systematic reviews’ that examine all possible interventions for a certain condition. To date, no overviews published in The Cochrane Library or by the Child Health Field have included non-Cochrane reviews. Our most recent overview on eczema prevention is the first overview to include both Cochrane and non-Cochrane reviews. Objectives: To discuss the methodological challenges we faced, how we overcame these challenges, and what we learned from the process of including non-Cochrane reviews in overviews. Challenges: 1) Two of our three potentially included non-Cochrane reviews did not contain data that enabled calculation of risk ratios and were therefore excluded; 2) many non-Cochrane reviews analyzed data from Cochrane reviews we had already included in our overview; and 3) we debated using scores from the AMSTAR tool (Assessment of Multiple Systematic Reviews) as an inclusion criterion. Our main challenge, however, centred on the fact that Cochrane and non-Cochrane reviews often examined the same interventions and contained many of the same trials. Deciding which reviews to include in our overview involved a trade-off between inclusivity and ensuring the project did not become too unwieldy or large for the end user. Solution: We used a consensus-based process to concurrently develop two sets of inclusion criteria: one assessed the eligibility of each review individually, and the other ensured that only one eligible review per topic area was included in our overview (Table 1). Conclusions: Six Cochrane reviews and one non-Cochrane review were included in our overview. Data extraction proceeded largely without difficulty, mostly due to our development of logical a priori inclusion criteria which other groups undertaking overviews might find helpful. Including non-Cochrane reviews allowed us to consider a broader range of interventions, although this was limited by our inability to extract usable data from two potential non-Cochrane reviews. Table 1 (P2A157). Criteria for considering reviews for inclusion. General inclusion criteria for individual reviews: Participants must have been 0-18 years of age. Search for trials must have been conducted no more than five years ago. Trials in reviews must have been randomized controlled trials (RCTs). Trials examining breastfeeding and pet avoidance were not subject to this restriction because RCTs in these topic areas are ethically difficult to conduct. Reviews must have contained at least one piece of quantitative data to enable calculation of a risk ratio and confidence interval. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Inclusion criteria to select one eligible review per topic area: All Cochrane reviews meeting our general inclusion criteria were included. A maximum of one non-Cochrane review was included in each topic area that did not have an included Cochrane review. If two or more non-Cochrane reviews covering the same topic area satisfied the general inclusion criteria, we included the most up-to-date review. Non-Cochrane reviews were excluded if they analyzed data from one or more Cochrane reviews that we had already included in our overview. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 107 P2A158 Risk difference estimation and reporting in systematic reviews Alonso-Coello P1 , Neumann I2 , Carrasco-Labra A3 , Brignardello-Petersen R4 , Irfan A5 , Solà I1 , Dahm P6 , Glujovsky D7 , Johnston B8 , Martinez L1 , Ramirez-Morera A9 , Sun X10 , Vandvik PO11 , Akl E12 , Tikkinen KA13 , Iorio A14 , Santesso N8 , Brozek J8 , Schünemann H14 , Guyatt G14 1 Iberoamerican Cochrane Centre, Institute of Biomedical Research (IIB Sant Pau) Barcelona, Spain; 2 Department of Medicine, Pontificia Universidad Catolica de Chile, Santiago, Chile; Department of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Canada; 3 Evidence Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Chile; Department of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Canada; 4 Evidence Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Chile; Department of Health Policy, Management & Evaluation, University of Toronto, Canada; 5 Internal Medicine Residency Program, University of Illinois at Urbana-Champaign, USA; 6 Department of Urology, College of Medicine, University of Florida, Gainesville, Florida, USA; 7 Institute for Clinical Effectiveness and Health Policy, Buenos Aires, Argentina; 8 Department of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Canada; 9 IHCAI Foundation, Costa Rica; 10 Center for Health Research, Kaiser Permanente Northwest, Portland, USA; 11 Norwegian Knowledge Centre for the Health Services, Oslo, Norway; 12 Department of Medicine, State Univeristy of New York at Buffalo, NY, USA; Department of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Canada; 13 Department of Urology, Helsinki University Central Hospital, Helsinki, Finland; Department of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Canada; 14 Department of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Canada Background: Trading off desirable and undesirable consequences of alternative patient management strategies requires knowledge of estimates of both relative and absolute measures of effect (AME). It is unclear to what extent and how authors of systematic reviews (SRs) determine and report AME. Objectives: To determine the proportion of SRs reporting AME and to evaluate how these are derived and interpreted. In particular, we will evaluate the proportion of SR that use relative measures for benefit outcomes and for harm. Additionally, we will test if AM reporting is associated with prespecified study characteristics. Methods: We will include SRs of interventions that report dichotomous outcomes. We will search MEDLINE for a random stratified sample of Cochrane and non-Cochrane SRs. From a pilot study of 50 SRs, we will calculate the final sample size to test our hypothesis, which is that a considerable proportion of SRs do not report AME and that the methods used are often inadequate. We will determine the proportion of SRs reporting AME and how they are obtained (e.g. meta-analytically combined across studies or computed based on relative effect measures and assumed baseline risk) and reported (e.g. risk difference or number-needed-to-treat). We will also assess how baseline risk was obtained (e.g. as mean or median from included trials or from representative observational studies), and whether authors provided AME for more than one risk group. Finally, we will conduct multivariable logistic regression analyses to examine the relationship of reporting AME with a set of prespecified study characteristics (e.g. type of SR [Cochrane vs not Cochrane], statistical significance or source of funding). Discussion: The project results will be presented at the Colloquium. Our results will inform the extent of limitations of determining and reporting AME in SRs and lay the foundation for further studies about the implications for decision-making. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P2A159 Using qualitative evidence to inform a review of effectiveness: A case study of integrating qualitative data in a systematic review of effectiveness of ECT for adult depression Berg R1 , Leiknes KA1 , Hoie B1 1 Norwegian Knowledge Centre for the Health Services, Norway Background: The issue of combining data from different traditions is of increasing importance as qualitative research is now being integrated within Cochrane reviews. Objectives: To describe the process of and results derived from conducting a systematic review in which we, as one of the first Cochrane registered reviews, integrated qualitative data. Methods: The Cochrane guidance on including qualitative evidence in effectiveness reviews was used to design an integrative methodological approach, which aimed to inform and extend the review, as defined in Chapter 20 of The Cochrane Handbook. We present the first phases of the review, in which the integrative approach was used with the aim of informing the effectiveness review question and Patient Intervention Comparison Outcome(PICO) elements. We conducted a systematic literature search and used content analysis to identify key concepts in included studies. A data recording form was used to identify text units pertaining to review elements and determine the frequency of categories. Dominant categories were used to form suggestions for refinement of the protocol. Results: The literature search for qualitative studies to inform the review yielded 66 studies of which 54 were examined in fulltext. Five studies met our pre-defined inclusion criteria. Based on these studies several dominant categories emerged within the PICO element areas population and outcome. Thus, we refined the protocol with respect to these elements by adding subgroup analyses and outcomes not already included. Challenges encountered during the process were skills required for synthesis of qualitative evidence, and practical integration of modifications within the Archie system. The merits of our approach included upholding expected standards of rigour and transparency, applying systematic methods, and heeding the importance of patient perspectives in health care. Conclusions: This review illustrates how qualitative evidence can add value to a review of effect and shows how incorporating qualitative material need not compromise the trusted methods of the Cochrane Collaboration. P2A160 Methodological perspectives on a realist synthesis: An explanatory analysis of the effectiveness of female genital mutilation-cutting abandonment efforts Berg R1 , Denison E1 1 Norwegian Knowledge Centre for the Health Services, Norway Background: Realist synthesis is an emerging approach to systematically reviewing research evidence on complex social interventions, which provides an explanatory analysis of how and why they are (or not) effective in particular contexts or settings. While there is considerable interest in conducting realist syntheses there are few exemplars of how to do this. Objectives: We present results of and methodological perspectives on a realist synthesis of FGM/C abandonment efforts. Methods: The Cochrane Handbook guidance Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 108 on including qualitative evidence in reviews of effects was used to inform the development of a realist synthesis approach and conduct of our review. Exhaustive searches sought two types of studies: 1) controlled studies to examine the effectiveness of FGM/C abandonment programs; 2) studies using qualitative and other methods to describe the factors related to the continuance and discontinuance of FGM/C (‘views’ studies). We combined quantitative data with data from views studies and the results of the synthesis were described in a model explaining how efficacy (outcome patterns) of an intervention varied depending on the particular configuration of its constituent mechanisms and contexts. Results: The searches yielded 7517 records. For the effectiveness synthesis, we included eight controlled before-and-after studies of low methodological quality (7042 participants). There were four types of interventions, covering seven intervention areas in Africa. Few studied the field’s reality before rolling out the intervention. Four meta-analyses showed limited effectiveness and considerable heterogeneity affected the interpretation. For the views synthesis, we located 26 studies, covering all but one intervention area. The synthesis of these studies was used to illuminate the effectiveness findings. In general, sexual morals and religion were poorly addressed and the degree to which the interventions were appropriate responses to the populations’ situation regarding FGM/C varied. Conclusions: Realist synthesis is a valuable but time-consuming approach requiring advanced reviewer skills in several assessment and analysis areas. P2A161 New and emerging methods of research synthesis: Are they given sufficient attention? Petrova M1 1 Egenis (ESRC Centre for Genomics in Society), University of Exeter, Exeter, UK Background: In spite of meta-analysis remaining the cornerstone of evidence synthesis, numerous alternative methods of statistical and non-statistical synthesis have recently emerged. Their proliferation has been driven by needs that the ‘exemplar systematic review’ meets incompletely, such as the need to address broad research questions and heterogeneous study types or to incorporate multiple perspectives and levels of analysis. Healthcare quality and patient safety are prime examples of areas where such demands arise. The field of emerging research synthesis methods is poorly mapped but lively. Four review publications identify over 40 such methods among themselves. These methods have developed in relative isolation. Their terminology is highly varied. Some of their constitutive practices are at odds with traditional systematic review ones. Many of them simply extend single study designs and lack the level of abstraction, articulation and detail required for wide usage. There are thus substantial barriers to their uptake. Yet those methods try to solve important research synthesis concerns. Are we giving them sufficient attention? Objectives: This presentation will begin with an overview of the field of emerging health research synthesis methods. It will then address the higher-order dimensions along which they tend to be compared and the lines along which they translate desiderata for systematicity, transparency and robustness into specific procedures. Finally, it will propose further criteria for evaluating the capacity of a synthesis method to instantiate such desiderata. Methods and data sources: A case study of the contents and structure of a heterogeneous pool of research papers Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. on cancer. Ideas from the philosophy and social studies of science. Conclusions: The current edition of the Cochrane Handbook offers limited guidance to non-standard synthesis methods. A debate is needed as to whether extending the Handbook in this direction is required or whether there are positive reasons for taking aconservative attitude. P2A162 Feasibility of fast track systematic review for policy makers: Interchangeability between pneumococcal conjugate vaccines Ciapponi A1 , Garcı́a Martı́ S1 , Rey-Ares L1 , Glujovsky D1 , Bardach A1 , Valanzasca P1 , Lee A1 , Cafferata ML1 1 IECS – Institute for Clinical Effectiveness and Health Policy, Argentina Background: Streptococcus pneumoniae is a leading cause of serious illness among children worldwide. Pneumococcal conjugate vaccines (PCV) including 7 to 15 serotypes have been developed but there is uncertainty about the effects of interchange schemes among them. The Pan American Health Organization (PAHO) committed us a systematic review (SR) for an evidence based decision that would be taken in the next month at an expert meeting. Objectives: To assess the efficacy, cost-effectiveness, immunogenicity and safety of interchangeability among PCV schemes in one month. Methods: A systematic search was conducted in December 2010 on MEDLINE, EMBASE, LILACS and CCTR. Databases containing regional proceedings were also searched. No language or temporal restriction was imposed. We included randomized controlled trials (RCTs), economic evaluations, SR and meta-analysis evaluating antibody response, cost-effectiveness and effectiveness of the interchangeability among PCV. Study selection was performed using EROS (Early Review Organizing Software), a web-based software to serve in the initial phases of a SR. Four pairs of reviewers independently selected, assessed the quality and extracted the data of the studies. Discrepancies were solved by consensus of the team. A librarian uploaded fulltext during the project as soon each agreement was obtained. Results: 21 of 159 studies were included. Direct information on interchangeability among PCV for primary series was not available. Some studies showed similar immunogenicity and safety between PHiD-CV and PCV7. PhiD-CV and PCV13 were consistently more cost-effective than PCV7 at a constant price. PHiD-CV vs. PCV13 comparitive results varied according to price, indirect effects and indirect costs. PHiD-CV gains more quality-adjusted life years due to the prevention of more frequent yet less severe events such as otitis media; and PCV13 prevents less frequent events but more costly as invasive diseases. Conclusions: Considering the absence of direct evidence, the PAHO meeting recommended not interchanging PHiD-CV with PCV7 unless necessary. The web-based software facilitated this intense process. P2A163 Use of WOMAC for the Assessment of Treatment Benefit for the Pain of Osteoarthritis of the Knee Woolacott N1 , Corbett M1 , Slack R1 , Rice S1 1 CRD, University of York, UK Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 109 Background: For the purposes of meta-analysis and network meta-analysis the use of standard outcome measures is ideal. In the field of osteoarthritis research The Western Ontario and McMaster University Osteoarthris Index (WOMAC) was developed as an osteoarthritisspecific measure of disability. It comprises three components: pain, stiffness, physical function, which can be reported separately or as an overall index. In 1994 a consensus meeting recommended the use of WOMAC as a primary measure of efficacy in osteoarthritis. Objectives: Within the context of investigating the efficacy of physical interventions for the relief of the pain of osteoarthritis of the knee (OAK), we investigated to what extent WOMAC had been adopted. Methods: We conducted a systematic review of physical therapies for pain relief in OAK. A range of sources were systematically searched in December 2009–January 2010. Results: A total of 138 original trials formed the basis of the review. Pain was measured using a variety of scales, with WOMAC pain scores making up 41% of the studies: pain was recorded using a WOMAC Likert scale in 29 studies and using a WOMAC visual analog scale in 28 studies. However, in many cases unexplained modifications and transformations of the WOMAC scores had been used such that comparability across trials, purportedly using the same measure, could not be assumed. Only 26 studies reported an overall WOMAC index (19%); 25 reported (some if not all) individual WOMAC subscores in addition to pain. Conclusions: The limited use of WOMAC, coupled with the wide range of other tools used in studies in our review, was a source of heterogeneity between studies. Further standardisation of the use of the WOMAC scoring system, with improved reporting of it, in trials of OAK is required. Caution in the pooling of the different pain scales and other similar score data is needed. P2A165 Defining study designs in The Cochrane Collaboration: Do all Cochrane Review Groups talk the same language? López-Alcalde J1 , Calcerrada N1 , Callejo D1 , Reza M1 , Gracia FJ1 , Maeso S1 , Escalona S1 , Nieto B1 , Fernández de Larrea N1 , Valentı́n B1 , Blasco JA1 1 Laı́nEntralgo Agency, Spain Background: Cochrane systematic reviews (SR) focus on randomized clinical trials (RCT) but some Cochrane Review Groups (CRG) also consider non-randomized studies (NRS) as eligible. However, NRS cover designs defined with different labels not consistently applied among researchers. Objectives: To describe the study designs considered as eligible by the CRG and the definition provided for NRS. Methods: We consulted CRG websites and their corresponding modules of The Cochrane Library (March 2011). At least two researchers assessed the information provided by each CRG. Any disagreement was resolved through discussion and, if necessary, with the involvement of a third researcher. The inter-rater reliability in the assessments of key items was rated by using the kappa statistic. We registered the following information for each CRG: 1) Are NRS eligible for inclusion? 2) If so, is this justified with any reason? 3) What ‘study design label’ is used (if any) for each study design? 4) What definition (if any) is provided for each study design? 5) How are key ‘study design features’ described? We considered as key study design features those detailed in Tables 13.2a and 13.2b of The Cochrane Handbook: Which was the unit of allocation? Was there a comparison? How were participants allocated Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. to groups? Which parts of the study were prospective? On which variables was comparability between groups assessed? Results: We identified 53 CRGs. The preliminary results show that around 60% of the CRGs analysed considered NRS as eligible, although this decision was justified with explicit reasons by fewerthan 50% of the CRGs. The CRGs analysed did not use common study design labels and did not explicitly describe allstudy design features suggested inChapter 13 of The Cochrane Handbook. Conclusions: Consistent definitions of key study design features for NRS are needed in The Cochrane Collaboration. P2A166 Why and when children are different: Guidance on appropriate inclusion of child health data in systematic reviews Thomson D1 , Williams K2 , Hartling L3 1 Cochrane Child Health Field, Department of Pediatrics, University of Alberta, Edmonton, Canada; 2 Cochrane Child Health Field; APEX Australia Chair of Developmental Medicine, University of Melbourne; Department of Developmental Medicine, Royal Children’s Hospital, Melbourne, Australia; 3 Cochrane Child Health Field and Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, Edmonton, Canada Background: Children’s responses to health care interventions often differ significantly from those of adults. Consequently, systematic review (SR) authors must consider the physiological, developmental and psychosocial aspects of childhood. Guidance for SR authors in this area remains limited. Key considerations: We identified resources to assist decision-making in the following areas: Population. A key decision at the outset of the review is whether to focus on child data only, or to include both adult and child data. This decision is often based on clinical expertise about the condition and its trajectory at different ages; care guidelines can also be used. Outcomes: The selection of a priori outcomes should be based on sound knowledge of the disease trajectory and children’s development. Relevant short-term outcomes may differ for children and adults, and similar outcomes may be assessed using different measurement tools. Recognition that the benefits and harms of treatment for children may unfold across subsequent decades of life may influence the choice of outcomes to be assessed and study designs to be included. Subgroups: Within the age range of 0–18 years, one can expect differing reactions to interventions. Authors should analyse age groups where there may be important similarities or differences in terms of physiological and psychosocial development. Searching: Databases have varying definitions of ‘child’ and other age-based terms, and may be inconsistent in indexing pediatric material. The assistance of a librarian experienced with using pediatric-search filters will help to identify relevant material. Conclusions: Key stages of carrying out SRs in children are affected by the dynamic nature of childhood. Authors are urged to carefully consider how children’s psychosocial and physiological development along the age span will influence their response to interventions. Work is under way to define the key developmental stages of childhood and how they intersect with different disease conditions. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 110 P2A167 Systematic reviews of complex interventions: From bare bones to greater policy relevance Snilstveit B1 , Waddington H1 1 International Initiative for Impact Evaluation (3ie), UK Background: Systematic reviews have traditionally focused on synthesising quantitative evidence, typically from randomised controlled trials. For questions regarding the effectiveness of simple interventions, such as effectiveness of a drug, ‘bare bones reviews’ are invaluable. However, when applied to systematic reviews of complex interventions the bare bones approach to systematic reviewing has a number of limitations, including its usefulness for informing policy and practice. This has been particularly highlighted by the recent rise in demand for systematic reviews in the field of international development. Objectives: Outline an approach to systematic reviewing that is appropriate for reviews of complex interventions, with the aim of enhancing the usefulness of such reviews for policy and practice. Methods: Based on a review of the methodological literature in this field and the experience of 3ie’s systematic reviews program, this paper provides an outline 3ie’s methodological approach to systematic reviews. Results: Theory-based systematic reviews, which summarise evidence on what works, when and why, can improve the policy relevance of systematic reviews of complex interventions. This approach situates reviews in the broader context of the underlying programme theory, reporting and synthesising evidence on all assumptions and links in the causal chain. Reviews that answer these questions draw on a range of study types. Answering the what works and what doesn’t questions means drawing on effectiveness studies, conducted to standards of high quality impact evaluation. But formulating answers to the when and why questions, may require a broader range of evidence from both quantitative and qualitative research. Conclusions: Systematic reviews of complex interventions should draw on a program theory, and when appropriate and feasible, use a mixed methods approach to collect and synthesise evidence along the causal chain. P2A169 Efficacy and safety of atypical antipsychotic drugs (quetiapine, risperidone, aripiprazole and paliperidone) compared with placebo or typical antipsychotic drugs for treating refractory schizophrenia: Overview of systematic reviews Melnik T1 , Soares B1 , Puga E1 , Atallah A1 , Rufino C1 1 Brazilian Cochrane Center, São Paulo, Brazil Background: According to some cohort studies, the prevalence of refractory schizophrenia (RS) is 20% to 40%. Objectives: Our aim was to evaluate the effectiveness and safety of aripiprazole, paliperidone, quetiapine and risperidone for treating RS. Methods: This was a critical appraisal of Cochrane reviews published in The Cochrane Library, supplemented with reference to more recent randomized controlled trials (RCTs) on RS. The following databases were searched: MEDLINE (1966–2009), CENTRAL: Controlled Trials of the Cochrane Collaboration (2009, Issue 2), EMBASE (1980–2009), LILACS (1982–2009). There was no language restriction. Randomized Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. controlled trials, systematic reviews and meta-analyses evaluating atypical antipsychotics for treating RS were included. Results: Seven Cochrane systematic reviews and 10 additional RCTs were included in this review. The data generally showed minor differences among the atypical antipsychotics evaluated and typical antipsychotics, regarding improvement in disease symptoms, despite better adherence to treatment with atypical antipsychotics. Risperidone was specifically evaluated in patients with RS in one of the systematic reviews included, with favorable outcomes, but without definitive superiority compared with other drugs of proven efficacy, such asamisulpride, clozapine and olanzapine. Conclusions: The findings underscore the difficulty in treating these patients, with high dropout rates and treatment patterns of modest improvement in assessments of effectiveness. Atypical antipsychotics have advantages over typical antipsychotics mainly through their better safety profile, which leads to better adherence to treatment. A combination of antipsychotics may also be an option for some refractory patients. P2A170 Non-randomised studies in reviews of effective professional practice and the organisation of health care: Do authors consider them and do they find them? Glenton C1 , Lewin S2 , Mayhew A2 , Scheel IB3 1 Norwegian Branch, Nordic Cochrane Centre, Norway; 2 Cochrane Effective Practice and Organisation of Care Group, Canada; 3 Norwegian Knowledge Centre for the Health Services, Norway Background: The Cochrane Effective Practice and Organisation of Care Group (EPOC) accepts inclusion of several study designs in addition to randomised trials, including quasi-randomised controlled trials; interrupted time series; and controlled before-after designs. But how often are these study designs included and is it worth the effort? Objectives: To explore: 1) How many EPOC reviews include non-randomised studies; 2) How many and what type of non-randomised studies they find; 3) Whether there is a connection between the proportion of non-randomised trials identified and the review topic. Methods: We examined the 65 EPOC reviews published before March 2010, and recorded the types of study designs considered. For reviews that included nonrandomised studies, we calculated the proportion of included studies that were nonrandomised. We categorised the reviews according to topic, using the categories adopted by the Health Systems Evidence Database, and explored whether there were differences across categories in the proportion of nonrandomised studies. Results: We found that: 75% of all EPOC reviews include nonrandomised studies in their inclusion criteria; the proportion of non-randomised studies in each review varied widely (median: 33%, range: 0 to 100%). The proportion of non-randomised studies varied according to intervention type: Healthcare delivery interventions, median 18.5% (range 0 to 100%); implementation strategies, median 33% (range 0 to 100%); financial interventions, median 50% (range 33 to 100%); and governance interventions, median 100% (range 66 to 100%). Conclusions: The majority of EPOC reviews do search for non-randomised studies. However, the degree to which they find such studies varies greatly, and may be influenced by intervention type. Given that non-randomised studies are often at higher risk of bias, and that including them entails extra effort, review authors should consider whether the benefits justify this Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 111 decision. Research should explore whether it is more useful to include non-randomised studies in reviews of some intervention types than others. P2A171 Determining appropriate inclusion criteria for reviews of public health interventions: Skin cancer prevention as a case study McDaid C1 , Sowden A1 , Paton F1 1 Centre for Reviews and Dissemination, UK Background: We undertook a systematic review to evaluate the effects of provision of sun protection resources (e.g. shade, sunscreen) combined with information provision in the prevention of skin cancer, commissioned by Centre for Public Health Excellence at NICE. Broad inclusion criteria were applied; studies were included regardless of design or quality. Due to limitations in the included studies it was difficult to determine the contribution of various intervention components to outcome. This raised the question of whether it is appropriate to adopt broad inclusion criteria for answering a policy relevant question. Objectives: To assess the impact on review conclusions of varying the inclusion criteria where the intervention includes multiple components. Methods: We used a completed systematic review as a case study. We investigated the inclusion of studies according to three sets of criteria: i) studies with appropriate assessment of the sun protection resource and information components; ii) studies where the main intervention components were of at least equal importance and iii) the original criteria. Narrative synthesis was undertaken. We assessed whether choice of criteria influenced the number of studies included and the overall conclusions. Results: Applying the narrowest inclusion criteria resulted in a single included study; the second set resulted in five included studies; 27 studies were included in the original synthesis, using the broadest criteria. The overall review conclusions remained the same, irrespective of criteria applied. The review would have been less resource intensive with tighter inclusion criteria, although the commissioner would not have had access to the same breadth of evidence, regardless of its limitations. Conclusions: Careful consideration should be given to the inclusion criteria selected for reviews in public health as decisions will affect the number of studies included, the amount of data to be extracted, quality assessed and synthesised, without necessarily impacting on the conclusions reached. P2A172 Piloting realist review methods to assess cost-effectiveness within he context of a systematic review Anderson R1 , Pearson M1 , Shepperd S2 , Pawson R3 , Hunt H1 , Cooper C1 , Shemilt I4 1 Peninsula Medical School, University of Exeter, UK; 2 University of Oxford, UK; 3 University of Leeds, UK; 4 University of Cambridge, UK Background: Realist review has been used to explain how and why complex health programmes are effective or not. However, this approach has not been extended to explaining the resource use, cost and cost-effectiveness of programmes or changes to services within Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. the context of evidence synthesis. Some intervention mechanisms, their contexts and outcomes may be inherently resource-related, while others may have a behavioural focus and be harder to define in terms of resources—and therefore be harder to cost. The development and testing of more explanatory systematic review methods has been strongly advocated by the Campbell & Cochrane Economic Methods Group. We have been funded by the NIHR Service Delivery and Organisation programme (UK) to conduct a systematic review that summarises the evidence of the effectiveness and cost-effectiveness of ‘step-down’, ‘hospital at home’ and other forms of intermediate care that aim to provide an alternative to acute inpatient care. Objectives: To present the methods (protocol) and early findings of this realist review, with a particular focus on the adaptation of methods for assessing and explaining the cost-effectiveness of intermediate care. Methods: We will present the protocol for this review, describe any amendments to the planned methods, and reflect on the emerging challenges and insights from half-way through the 10-month project. Our intention is, as a first stage, to identify key intervention mechanisms and their related contexts which explain effectiveness, and then assess their resource requirements and opportunity costs as the second and third stages of analysis. Results: The review protocol is in development and will be finalised by June 2011, preliminary findings will be available by October 2011. Conclusions: On the basis of this review we will comment on how realist review methods may complement established Cochrane methods for systematic reviews of effectiveness. P2A173 Amalgamating individual patient data for meta-analysis: An IT perspective Hilken N1 , Middleton L1 , Champaneria R1 , Daniels J1 1 University of Birmingham, UK Background: Individual patient data (IPD) meta-analysis requires raw data from authors of primary studies, which can be provided in various formats. An IPD meta-analysis to evaluate the relative effectiveness of hysterectomy, endometrial ablative techniques and Mirena for heavy menstrual bleeding illustrates potential problems in collating disparate data sets. Objectives: To describe the data management and synthesis aspects of an individual patient data meta-analysis from an IT perspective. Methods: We requested IPD from 30 RCT authors and received responses from 17. Data were transformed into Third Normal Form (3NF) for each study, and translated to use homogeneous naming and coding conventions. A data import specification was created using Microsoft Excel that used VBA macros to create the necessary SQL scripts. The method provided a single authoritative source recording all data transformations. All import code was directly generated in a swift and repeatable manner from the specification itself. Results: IPD from 17 studies was received over a 16 month period, as spreadsheets, databases, SAS, SPSS files or on paper, mostly lacking descriptive meta-data. The specification spreadsheet defined all coded lists for discrete variables, and mappings for each data set to a common coding convention. Automated code generation allowed collaborative effort from staff without programming skills on the import specification. VBA macros create SQL scripts for creating tables, relationships, views and data inserts, checks, and SAS files necessary to consume the data. Recommendations: An IPD meta-analysis is a substantial undertaking, particularly where non-standard ordinal data variables Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 112 such as ‘satisfaction’ are included. We recommend the creation of a master relational database, with automated data import to maintain data integrity and quality. Trialists should also consider further data use, apply accepted data standards, use relational databases and thoroughly document datasets, and ensure meta-data are stored along with data in the long term. P2A175 A descriptive analysis of overviews of reviews published between 2000 and 2010 Hartling L1 , Chisholm A1 , Thomson D1 , Dryden DM1 1 Canada Background: Overviews of systematic reviews (overviews) compile data from multiple systematic reviews (SRs) and are a new method of evidence synthesis. Objectives: To describe the methodological approaches in overviews of healthcare interventions. Methods: We searched four databases from 2000 to May 2010. We included studies if they: stated a clear objective; examined an intervention; used explicit methods to identify SRs; collected and analysed outcome data from the SRs; and intended to include only SRs. Two researchers independently screened studies and applied eligibility criteria. One researcher extracted data with verification by a second. We conducted a descriptive analysis. Results: The search yielded 1661 citations; 70 overviews were included. The number of overviews increased from 1 in 2000 to 10 in 2009. The interventions were pharmacological (n = 17), non-pharmacological (n = 42), or both (n = 11). Inclusion criteria were clearly stated in most overviews (n = 63). The majority searched at least two databases (n = 47), reported the years and databases searched (n = 48), and provided key words (n = 48). Eighteen overviews included Cochrane SRs only. Two reviewers independently screened and completed full text review in 27 overviews. Methods of data extraction were reported in 32. Quality of individual studies was extracted from the original SRs in 11 overviews. Quality assessment of the SRs was performed in 32 overviews; at least 13 different tools were used with the Oxman and Guyatt tool most frequent (n = 12). The quality of evidence was assessed in 11 overviews. Most overviews provided a narrative or descriptive analysis of the included SRs; a quantitative analysis was conducted in only two cases. One overview conducted indirect analyses. Publication bias was discussed in 19 overviews. Conclusions: This study shows considerable variation in the methods used for overviews. There is a need for methodological rigour and consistency in overviews, as well as empirical evidence to support the methods employed. P2A176 Methodological challenges associated with network meta-analyses: The experience of the Musculoskeletal Group Tanjong Ghogomu E1 , Maxwell L1 , Singh J2 , Christensen R3 , Wells G1 , Buchbinder R4 , Tugwell P1 1 Cochrane Musculoskeletal Group, Canada; 2 Cochrane Musculoskeletal Group, USA; 3 Cochrane Musculoskeletal Group, Denmark; 4 Cochrane Musculoskeletal Group, Australia Background: Many traditional meta-analyses summarize the results of studies that compare an intervention with placebo. Often there is Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. limited or insufficient evidence from direct head-to-head comparisons of competing interventions. A network meta-analysis enables the comparison among different interventions in the absence of direct head-to-head comparisons. Evidence from a network meta-analysis is reliable if studies with very similar study characteristics and eligibility criteria (study designs, populations, interventions, comparisons and outcomes) are combined. Objectives: To discuss the methodological and process challenges involved in network meta-analyses and how to overcome them. Methods: We used the example of two Cochrane overviews of reviews on biologics involving network meta-analyses to illustrate the methodological challenges encountered and how we overcame them. We compared the efficacy and safety of six different biologics in patients with rheumatoid arthritis in the first overview. We found limited evidence on the adverse effects such as tuberculosis, life threatening fungal infections and lymphomas that may be common with biologics, so we undertook a second overview to assess the adverse effects of nine biologics across different indications of use. The unit of analysis was different in the two overviews: Cochrane systematic reviews in the first, and clinical trials in the second. Results: We faced different methodological considerations, including eligibility criteria and methods of assessment of risk of bias and analysis depending on the type of overview most appropriate to answer our question. Challenges included managing a large volume of data for screening and extraction; achieving consensus with a large team of extractors; preparing data for analysis; and presenting results using the Review Manager overview template. Conclusions: Our experience showed that where limited evidence is available for direct comparisons of different interventions, well-conducted indirect comparisons based on network meta-analyses are challenging to undertake, but necessary to guide health decision making. P2A177 Are all Cochrane reviews born equal? Methodological quality across different Review Groups Pantoja T1 , Moreno G2 , Mayhew A3 , Rivera S2 , Claro JC4 1 Departamento Medicina Familiar, Pontificia Universidad Católica de Chile/ Cochrane EPOC group; 2 Departamento Medicina Familiar, Pontificia Universidad Católica de Chile; 3 Cochrane EPOC group; 4 Departamento Medicina Interna, Pontificia Universidad Católica de Chile Background: High quality systematic reviews are increasingly recognized as providing the best evidence to inform healthcare decisions at different levels. The quality of a review—and then the confidence that decision-makers put on its findings—depends on the extension to which its design will generate unbiased results. Published reviews vary considerably in their quality, with Cochrane reviews rating consistently better than non-Cochrane reviews. However, little is known about the quality of reviews across different Review Groups in the Collaboration. Considering that the review production process in the Collaboration is highly decentralized some variation is expected, but its magnitude is not known. Objective: To compare the methodological quality of systematic reviews produced by different Cochrane Review Groups. Methods: All new reviews published in issues 3 and 4, 2011 of The Cochrane Library were selected for appraisal. Each selected review was independently assessed by two evaluators using the AMSTAR tool. The percentage of reviews from each Review Group that met each of the 11 AMSTAR items and the proportion that met a ‘high-quality’ Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 113 score were calculated. Data were analyzed using non-parametric approaches. Results: Fifty reviews were selected for appraisal (31 from issue 3;19 from issue 4). In our initial assessment of a subsamplesome differences in ‘quality’ were found. However, we have not completed the assessment of the full sample in order to attribute those differences to the Review Groups where they were produced. Conclusions: It might be differences in the ‘methodological quality’ of reviews produced by different Review Groups. If this is verified in the analysis of the whole sample of reviews more work should be done within the Collaboration to assure appropriate quality standards of its main product through changes in the editorial processes of review groups. P2A178 Systematic reviews of the effectiveness of quality improvement strategies and programmes: Some methodological issues Pantoja T1 , Mayhew A2 , Grimshaw J2 1 Departamento Medicina Familiar, Pontificia Universidad Católica de Chile, Chile; Cochrane Effective Practice and Organisation of Care Group, Canada; 2 Cochrane Effective Practice and Organisation of Care Group, Canada Background: Well conducted systematic reviews are increasingly seen as providing the best evidence to guide choice of quality improvement strategies in healthcare. But in contrast with reviews of pharmacological interventions, a number of methodological challenges should be addressed by reviewers undertaking reviews in this area. Objective: To discuss methodological issues relevant to the conduct of reviews of quality improvement strategies and programmes. Methods: We built on our experiences in the Cochrane Effective Practice and Organisation of Care (EPOC) Group and on a focused review of the methodological literature in the area. Results: We have selected the following issues for a further analysis: i) Formulating a relevant question and inclusion criteria: study designs, populations and outcomes; the use of non-randomised evidence for effectiveness questions; the lack of an accepted taxonomy for quality improvement strategies; and the broadness/narrowness of the review question (the ‘lumping versus splitting’ debate). ii) Identifying and screening evidence sources: published versus nonpublished literature; search strategy considerations. iii) Methods of synthesis/analysis: appropriateness of meta-analysis; other synthesis approaches (range of effect sizes); dealing with expected heterogeneity; identifying effect modifiers. Conclusions: We have discussed issues that authors will face when conducting reviews of the effectiveness of quality improvement strategies. Additionally we have presented some of the current methodological developments in order to contribute to more sound and informative reviews in this field. P2A179 Minimal clinically important differences in acute pain Tendal B1 , Ortner N1 , Gøtzsche PC1 , Hróbjartsson A1 1 The Nordic Cochrane Centre, Norway Background: Pain is an important outcome in a large number of clinical trials and systematic reviews. Trials investigating interventions Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. for painful conditions frequently present their findings as the difference between the mean pain score. However, it is unclear when such a difference is large enough to be clinically relevant, making interpretation of such measures challenging. An attempt to solve this problem is to empirically determine a minimal clinically important difference (MCID) i.e. the smallest difference in a score which patients perceive to be beneficial or harmful and which would lead the clinician to consider a change in treatment. The MCID is also important in order to estimate adequate sample sizes in clinical trials. Empirical studies aimed at determining MCIDs report differing results. Objectives: To assess the typical minimal clinically important difference in acute clinical pain and to examine reasons for its variability. Methods: We searched PubMed, EMBASE and The Cochrane Library with no language restrictions. We included empirical studies examining MCID in acute clinical pain on one-dimensional pain scales (e.g. visual analogue scale or numeric rating scales). Studies using retrospective pain assessment were excluded. Two independent observers extracted data. We plan to estimate the average MCID with random effects meta-analyses, and to perform subgroup analyses to examine the effects of co-factors (baseline pain, gender and age). If studies are considered too heterogeneous for a meaningful quantitative summary, we will summarise results qualitatively. Results: We have preliminary included 23 studies and analyses are ongoing, to be presented at Cochrane Colloquium. P2A180 Does patient expectation affect treatment outcomes? A preliminary investigation on expectations and outcomes Lee H1 , Zhang X1 , Cho S2 , Lee H1 1 Acupuncture & Meridian Science Research Centre, Kyung Hee University, Korea; 2 Hospital of Korean Medicine, Kyung Hee University Medical Centre, Korea Background: Non-significant findings from sham/placebo-controlled trials are usually interpreted as the intervention being merely a placebo in spite of a range of influencing factors, including expectation. Although patients’ belief or expectation regarding the benefit of treatment is regarded as a potentially important factor for yielding positive outcomes, data from clinical trials have shown inconsistent findings. Objectives: To investigate the impact of patients’ expectation regarding treatment on outcomes in randomised sham/ placebo-controlled trials reporting pain-related outcomes. Methods: Electronic searching was conducted in Ovid MEDLINE in April 2011 to include: (1) randomized sham/placebo-controlled trials or secondary analysis of relevant RCT data; and (2) studies measuring general or treatment-specific expectations and the association with analgesic outcomes. We excluded laboratory-based experiments involving healthy volunteers or studies that did not report analgesic outcomes. As substantial heterogeneity between studies was expected, a qualitative review was conducted. Methodological quality assessment was performed using the modified Oxford quality scale. Results: Of 174 studies, 5 reports including 6 RCTs and 4 secondary analyses of 7 relevant RCTs involving 2645 patients met our inclusion criteria. Interventions included acupuncture (n = 6), manipulation (n = 1), and drugs (n = 2) in mainly chronic pain conditions. All included studies were assessed as of highquality (≥4 points). At different time points, 7 studies measured patients’ expectations or belief in the intervention, while the other 2 studies additionally sought physicians’ Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 114 expectations. Trials where no difference was found between tested intervention and placebo control, reported heterogeneous findings, i.e. high expectation does not always lead to better analgesic effect. Blinding success, how expectation is measured, and expectation measurement time-points varied across the studies, which may have contributed to conflicting results. Conclusions: The current evidence doesnot fully support the notion that expectation produces less pain, calling for further investigation of this issue. A careful approach to interpreting non-significant findings of sham-controlled trials is needed in systematic reviews. P2A181 Consulting with young people to inform systematic reviews: An example from a review on the effects of schools on health Jamal F1 , Harden A1 , Wells H2 , Bonell C2 1 University of East London, Institute for Health and Human Development, UK; 2 London School of Hygiene and Tropical Medicine, Department of Social and Environmental Health Research, UK Background: There has been increasing interest in involving the public in systematic reviews to help ensure the review is focused on issues that are important and relevant to them. Case examples of this involvement are rare, especially for reviews focused on children and young people. Objectives: This study aims to describe the process and outcome of consulting with young people in a review of the effects of school environment interventions on children and young people’s health. Young people’s perspectives were sought alongside the perspectives of teachers, policy-makers and academics to determine the scope and priorities for review. Methods: Consultations were conducted with a pre-existing group of young people brought together to advise on public health research. Their views were sought at three stages: 1) at the beginning when general views relating to the policy problem under study were elicited; 2) half-way through when views on the map of identified evidence were sought; and 3) near the end when the review results were presented and discussed. Face-to-face to consultation methods were used, supplemented with online social networking for young people to continue discussion. Results: Each consultation stage helped shape the review process. For example, the first and second consultations helped researchers to understand the policy problem in its current context and informed the development of inclusion criteria for priority review areas. Face-to-face consultation was better at engaging young people but social networking gave the review team greater flexibility to elicit further views when unanticipated issues arose that needed immediate input. Conclusions: Consulting with young people alongside other stakeholders added considerable value to the review, especially in terms of helping the review team to prioritise in a relevant and meaningful way. The use of a pre-existing group facilitated engagement, but the use of social networking requires further evaluation. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P2A182 Using systematic reviews to study adverse drug reactions: The example of heparin-induced thrombocytopenia Junqueira DRG1 , Carvalho MDG2 , Perini E1 1 Centre of Drug Studies, School of Pharmacy, Federal University of Minas Gerais, Brazil; 2 Department of Clinical and Toxicological Analysis, School of Pharmacy, Federal University of Minas Gerais, Brazil Background: Methodology to systematically review drug efficacy based on randomized controlled trials (RCT) is well-established. Nevertheless, evidence-based healthcare may be supported by balanced evidence on benefits and harms of healthcare interventions. Development of standard recommendations to systematically review harm effect with the same emphasis on treatment benefit is of remarkable importance. Objectives: To describe the experience of conducting a systematic review for an adverse drug reaction (ADR). Methods: We conducted a systematic review based on standard methodology drawn on the Cochrane Handbook for Systematic Reviews of Interventions. The review was adapted to evaluate evidence about the risk of developing an ADR, heparin-induced thrombocytopenia (HIT). HIT is a paradoxical and unpredictable reaction diagnosed by clinical and laboratory assessment. Results: The review question focussed on a specific ADR in a selected, susceptible population. This was a pragmatic decision mostly concernedwith the quality evaluation of the ADR and impact of the clinical decision. We considered both blinded and unblinded RCTs for inclusion, and we did not limit our search strategy to ‘adverse effects’.. Search strategies considered the intervention (anticoagulant) and the outcome (HIT). The quality assessment applied the Handbook directions, but special attention was paid to the appropriateness of HIT diagnosis according to its clinical and serological profile. We included assessment of adequacy of ADR monitoring as an additional bias criterion. Only one study was included. Most studies were excluded because the definition of HIT or diagnosis process was inappropriate. Conclusions: The few studies included in the review illustratethe limitations of RCT to evaluate unpredictable adverse events induced by drug use, such as HIT. Characteristics of some ADRs, together with poor quality reporting, analysis and discussion, may justify a systematic review focused on specific adverse outcomes and inclusion of observational studies. P2A183 A network meta-analysis of randomized controlled trials of target therapy for mRCC Chan A1 , Leung H2 1 Chi Mei Medical Centre, Tainan, China; 2 Wan Fang Hospital, China Background: there are six new target therapeutic drugs approved for the treatment of metastatic renal cell carcinoma (mRCC) since 2006. At present, no direct head to head trial to compare their efficacy is available. Methods: A systematic literature search of the MEDLINE, EMBASE, CANCERLIT and CENTRAL databases was performed. All randomized clinical trials of sorafenib, sunitinib, everolimus and pazopanib used to treat mRCC were included. The study selection, data extraction, and quality assessment were performed independently by two reviewers, with all disagreements resolved by consensus. The main outcome was benefit (defined as an improvement in progression median disease-free survival or overall survival). We Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 115 did the network meta-analysis using mixed-effects logistic regression to perform an indirect comparison of the treatment effects between target therapeutic drugs. The quality appraisal was analyzed using the Jadad scale. Results: Compared with placebo or interferon-α, target therapeutics may improve survival benefit (Hazard ratio (HR): 0.51, 95% CI 0.05 to 0.98). The proportion of patients who discontinued the study drug due to adverse events was similar between sorafenib and pazopanib; but everolimus was higher than sunitinib (10% in everolimus and 6% in the sunitinib). Sorafenib is likely less effective than other target therapeutics, although this difference was statistically significant for the comparison with sunitinib (HR: 0.47, 95% CI 0.316 to 0.713) and everolimus (HR: 0.159 95% CI 0.091 to 0.276). However, the effectiveness of sorafenib was not statistically different from pazopanib by using placebo as a comparator (HR: 0.957, 95% CI: 0.657 to 1.393). The results of quality assessment showed that 60% of studies were scored 5 and 40% were scored 3. Conclusions: Our findings suggest that most of the targeted therapeutic drugs included in this study have reportedly favorable PFS or overall survival for patients with mRCC compared to IFN-α or placebo. Given the limitations of indirect comparisons, sorafenib is likely less effective than sunitinib and everolimus, but have similar effectiveness to pazopanib. Sunitinib was safer than everolimus, sorafenib and pazopanib. More RCTs are required to confirm these findings and investigate the clinical effectiveness of targeted therapeutics in the treatment of mRCC. P2A184 How useful is GRADEpro for assessing evidence from epidemiological or diagnostic accuracy studies Gholitabar M1 , Ullman R1 , Newbatt E1 , Fields E1 , Kenny J1 1 National Collaborating Centre for Women’s and Children’s Health (NCC-WCH), UK Background: GRADEpro software was developed to provide a transparent and logical approach to grading evidence and strength of recommendations or implications for practice for interventional studies. GRADEpro uses a default for rating the quality of evidence based on the study design (RCT or observational) and this is downgraded or upgraded based on other factors including limitations, inconsistency, indirectness and imprecision. Objectives: To examine the challenges of using GRADEpro to different types of reviews and present some possible solutions to the problems. We used GRADEpro to assess the quality of evidence for two NICE clinical guidelines. Methods: Three worked examples illustrating the problems of using GRADEpro in different types of reviews will be presented. These are: i) assessment of non-comparative data using GRADEpro, ii) rating imprecision where the clinical cut-offs for potential benefits and harms are unclear, and iii) presentation of findings where denominators differ between studies. Results: Some practical solutions to commonly encountered reviewing problems will be presented and potential for application of the GRADE approach across a range of study types discussed. Conclusions: Although GRADE is a methodologically rigorous method that has been well developed for use in intervention studies questions remain about its consistency and objectivity, especially when applied to different study types. Further development will help to improve the GRADE approach further but this needs to be undertaken with extensive field testing as this is the only way to ensure its methods stand up across a wide range of situations. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P2A185 Systematic reviews in dentistry have problems including split-mouth and cross-over trials in meta-analysis Carrasco-Labra A1 , Brignardello-Petersen R2 , Araya I3 , Yanine N3 1 Evidence Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Chile, Department of Clinical Epidemiology & Biostatistics, McMaster University, Canada; 2 Evidence Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Chile, Department of Health Policy, Management & Evaluation, University of Toronto, Canada; 3 Department of Oral & Maxillofacial Surgery, Evidence Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Chile Background: Split-mouth and cross-over trials are frequently used in dentistry. There is some evidence about systematic reviews’ authors having problems trying to incorporate these types of studies in meta-analysis. Objectives: To determine to what extent authors are including the evidence from split-mouth and cross-over trials in systematic reviews and meta-analysis in dentistry. Methods: All the systematic reviews published by the Oral Health Group in the Cochrane Database of Systematic Reviews up to the issue 3, March 2011 were included. Two researchers examined independently and in duplicated the references list of included and excluded studies. Each reference was searched in MEDLINE to determine whether the trial corresponded to a split-mouth or cross-over trial. When one of these studies was detected, the reasons for exclusion from the review were registered. When these studies were included in the systematic review, it was evaluated whether they were included in the meta-analysis. Also the methods section of each review was assessed and the approaches for including split-mouth and cross-over with the rest of the evidence were recorded. Results: A total of 114 systematic reviews from the Oral Health Group were retrieved. From these, 51 (44.7%) reviews had split-mouth or cross-over trials in the reference list of included or excluded studies. Of the reviews that included these types of design 19 (57.6%) were not able to include them in the meta-analysis. The most frequent reasons for exclusion due to methodological flaws were inadequate data analysis or inappropriate design (n = 40) and the condition of non-randomized allocation of the treatment to time/side (n = 20). Conclusions: Authors face many challenges when trying to incorporate the evidence from split-mouth or cross-over trials in systematic reviews. Even when these studies are included, most of the times they do not contribute to the estimation of the treatment effect. P2A186 Designs and analysis of N-of-1 trials: A systematic review Perdices M1 , Barrowman N2 , Sampson M3 , Shamseer L4 , Bukutu C5 , Vohra S6 1 Royal North Shore Hospital and University of Sydney, Australia; 2 Children’s Hospital of Eastern Ontario Research Institute, Canada; 3 Children’s Hospital of Eastern Ontario, Canada; 4 Ottawa Hospital Research Institute; 5 Alberta Centre for Child, Family & Community Research, Canada; 6 University of Alberta, Canada Background: Well designed N-of-1 trials provide a valuable alternative methodology for evaluating the therapeutic efficacy of pharmacological or behavioural interventions when randomized controlled trials (RCTs) Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 116 are not feasible. Objectives: To evaluate methodological design, analysis and meta-analysis used in N-of-1 trials. Search strategy: We searched the following databases from their inception to August 2007: MEDLINE, PsycINFO, ERIC, Sociology Abstracts, PsycBITE, AMED, EMBASE, CINAHL, and the Cochrane Database of Systematic Reviews. Reference lists of included studies were checked. Selection criteria: Original studies specifically dealing with methodological design and statistical analysis of N-of-1 trials. Reports of N-of-1 trials themselves were not eligible for this review. Data collection and analysis: Two reviewers independently assessed the suitability of studies for inclusion. For each paper, two authors independently extracted content information for 64 predefined items specifically related to design, methodology, statistical analysis and meta-analysis. Results: We identified 95 papers that satisfied the inclusion criteria. Of these, 51 were general reviews discussing strengths, limitations and sources of bias of basic N-of-1 designs (e.g., A-B-A-B, multiple baseline). Nine of these specifically reviewed randomisation test procedures as an adjunct for increasing methodological robustness. An additional 44 papers discussed specific data analysis techniques. Of these, 22 dealt with statistical analysis techniques exclusively, 3 with visual analysis exclusively, and 11 with comparison of statistical and visual analysis, highlighting the strengths and weaknesses of each approach. Eight papers discussed methods for quantitative synthesis of data. Conclusions: Although consensus exists about aspects of basic design and analysis of data for N-of-1 trials to ensure internal and external validity, issues related to generalisability of results and meta-analysis of data remain contentious. P2A187 A coherent framework for choosing ‘fit for purpose’ review methods Oliver S1 , Thomas J1 , Gough D1 1 EPPI-Centre, Institute of Education, UK Background: Systematic review methods span qualitative syntheses (e.g. meta-ethnography, thematic synthesis), mixed methods syntheses (e.g. realist synthesis, framework synthesis) and quantitative synthesis (e.g. statistical meta-analysis and meta-regression). They differ in terms of how they identify studies, appraise their quality and synthesise findings. Review methods have been applied to different types of questions and different types of studies, often developed by different teams working independently without a clear methodological overview. Objectives: To make sense of the diversity of review methods available with a coherent framework for choosing ‘fit for purpose’ review methods. Methods: Systematic reviews from a purposive sample that either configured or aggregated findings (1) were analysed in terms of the source and type of their questions, their purposes (generating, exploring or testing theory), data (qualitative, quantitative, or mixed), methods (linear, iterative; qualitative, quantitative, or mixed), and products (syntheses of concepts or measures). Results: A series of visual representations of the diversity of review questions, purposes, data and methods evolved through discussions to provide a coherent framework for choosing between review methods. Conclusions: Three major pathways for reviews begin with: clear conceptual frameworks (for testing hypotheses), partially developed frameworks (for exploring theory); or tentative concepts (for generating theory). These pathways differ in terms of whether: they aim to find all relevant studies or Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. all relevant concepts; the search strategy is linear or iterative; they appraise the study methods, findings or both; they aggregate or configure the findings of studies; they comment on the risk of bias or coherence of findings; and how well they are suited to instrumental models of knowledge transfer, or interactive or enlightenment models of knowledge exchange. Reference Sandelowski M. Reading, writing and systematic review. J Adv Nurs. 2008;64(1):104–10. P2A188 Incorporating the GRADE approach in overviews of systematic reviews: an example from an overview in neonatal respiratory care Rojas M1 , Lozano J2 , Solà I3 , Bonfill X3 1 Pontificia Universidad Javeriana, Colombia; 2 Florida International University College of Medicine, USA; 3 Iberoamerican Cochrane Centre, Spain Background: Evidence from systematic reviews (SR) addressing different comparisons for the same treatment could provide effect estimates of different quality. Overviews of SR incorporating GRADE to rate the quality of evidence would provide readers with summaries of evidence and classification of its quality. The accuracy and reproducibility of the quality evaluation process entails some challenges for overview authors. Objectives: To describe the reliability aspects in the evidence synthesis process applying the GRADE framework, in the development of an overview of SR in the field of respiratory care in neonates. Methods: The selection process was completed independently by two authors. A third author assessed the methodological quality of eligible SR using the AMSTAR. Two authors independently summarized the evidence for each comparisons/ outcomes in GRADE profiles. Consensus about the importance of outcomes for patients was reached ad hoc. Results from trials published after the SRs’ search dates were added to the profiles. A third author compared the quality ratings provided by authors for each outcome and checked the accuracy of the overall process. Measurement of inter-rater agreement was done using the Cohen’s kappa coefficient. Reasons given by authors to rating down the quality of evidence for each outcome were also compared and observed agreement presented. Results: The presentation will describe the above mentioned methods implemented in an overview assessing the effect of different ventilation strategies in preventing bronchopulmonary dysplasia and other neonatal outcomes in preterm infants with respiratory distress syndrome. A total of six SR addressing eight different comparisons between ventilation modes and strategies were included, and seven outcomes assessed. Only one additional trial was found and incorporated in the meta-analyses for each outcome. Conclusions: The example illustrates some challenges in rating the quality of evidence from the GRADE framework in the context of developing overviews of SR, and the need of integrating reliable consensus methods among reviewers. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 117 P2A189 A pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies reviews Pluye P1 , Cargo M2 , Robert E3 , Bartlett G1 , O’Cathain A4 , Griffiths F5 , Boardman F5 , Gagnon M6 , Rousseau M3 1 McGill University, Canada; 2 University of South Australia, Australia; 3 Université de Montréal, Canada; 4 University of Leeds, UK; 5 University of Warwick, UK; 6 Université Laval, Canada Background: A new form of literature review has emerged, mixed studies review. These reviews include qualitative, quantitative and mixed methods studies. The Mixed Methods Appraisal Tool (MMAT) is intended to be used as a critical appraisal tool for systematic mixed studies reviews (SMSRs). In SMSRs, reviewers appraise studies that use different methods allowing them to obtain in-depth answers to complex research questions. However, appraising the quality of studies with different methods remains challenging. To facilitate this, the MMAT has been developed. The MMAT has been pilot tested using 32 evaluation studies for efficiency and reliability. On average, it took 14 minutes to appraise a study. Agreement between reviewers was moderate to perfect with regards to MMAT criteria, and substantial with respect to the overall quality score of appraised studies. Content validation: The current 2011 version of the MMAT is based on a literature review of MSRs, a mixed methods framework, and has been revised using feedback from four 90-minute workshops. The workshops involved diverse audiences such as graduate students enrolled in a mixed methods research course, researchers and research professionals with experience in qualitative, quantitative, and mixed methods research, and members of the Cochrane collaboration with experience in SMSRs or in systematic reviews of qualitative studies, e.g., meta-ethnography, or of quantitative studies, e.g., systematic reviews of randomized controlled trials. Results: The current 2011 version of the MMAT (checklist and tutorial) contains two screening questions, and 19 questions corresponding to four types of studies (methodological domains): qualitative research, randomized controlled trials, non-randomized studies, quantitative descriptive, and mixed methods studies (http:// mixedmethodsappraisaltoolpublic.pbworks.com). Conclusion: The MMAT is validated and unique for appraising the most common types of study designs, including mixed methods studies. The development of the MMAT is supported by an international research team with expertise on qualitative, quantitative and mixed methods research. P2A190 Systematic reviews of adverse events. Is screening by title and abstract enough? Zorzela L1 , Punja S1 , Joffe A1 , Hartling L2 , Loke Y3 , Vohra S4 1 University of Alberta, Canada; 2 Univerity of Alberta, Canada; 3 Univeristy of east Anglia, UK; 4 University of Alberta, Canada Background: Papers selected for systematic reviews are generally screened by title and abstract. Since harms are reported poorly, screening using title and abstract may miss relevant papers when compared to screening full text. Objectives: To identify if systematic reviews of harms using the traditional method of screening papers Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. based on title and abstract will omit relevant studies, in comparison to screening based on full text. Methods: We are proceeding with two systematic reviews of adverse events associated with propofol infusion in pediatric patients. Only clinical trials will be selected (randomised or quasi-randomised). In the first review, papers are screened by title and abstract, looking for reports of presence or absence of adverse events. In the second review, the papers are screened using full text for reports of adverse events. Results: Both reviews are underway and full results will be shared at the Cochrane Colloquium. A total of 109 papers fulfilled the inclusion criteria and were selected, of those, 59 publications reported adverse events in title or abstract and 14 only reported at the results section. Thirteen publications had not reported the presence or absence of adverse events. A total of 23 non-English publication are in the process of being translated. Conclusions: Although at a preliminary state, the partial data suggest that clinical trials do not consistently report the presence or absence of adverse events. Systematic reviews of harms may need to screen studies based on full text in order to capture all applicable studies and reduce type II error. P2A194 Challenges facing reviewers preparing overviews of reviews Flodgren G1 , Shepperd S1 , Eccles M2 1 University of Oxford, UK; 2 Newcastle University, UK Background: An overview of reviews compiles evidence from multiple reviews into one accessible document, thus helping decision makers to choose effective healthcare interventions. Overviews are challenging. For instance, results from individual studies are not always adequately presented in reviews to allow optimal compilation of data. Objectives: To describe the difficulties of preparing an overview of reviews. Methods and Results: We summarised evidence from systematic reviews on the effect of financial incentives on professional practice and patient outcomes, including both Cochrane (CR) and non-Cochrane reviews (NCR). From a large number of seemingly eligible reviews, only 2 CRs and 2 NCRs, presenting quantitative data and fulfilling the DARE criteria for an adequate search, were included. Excluded reviews typically did not present numerical data or failed to present data at the level of individual studies. Comparison of the quality of the evidence reported in included reviews was difficult because they used different assessment tools. We used the AMSTAR tool to assess the methodological quality of reviews, and found the quality to be high even though the presentation of numerical data was incomplete for many outcomes. We attempted to retrieve missing data by consulting individual studies or their authors. Still, we could not retrieve the information needed for meta-analysis. Instead, results were summarised using vote counting within financial incentives across type of outcome, and within outcomes across types of financial incentives. Conclusions: Inadequately reported data in reviews is a major problem in the preparation of overviews over reviews. Differences in the tools used by reviewers to assess study limitations, is another. This makes it very difficult to summarise and evaluate large amounts of heterogeneous data, in order to present it in an accessible format. Researchers and reviewers alike must improve the reporting of data. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 118 P2A195 GRADE in Cochrane systematic reviews: An investigation Xiao J1 , Zhen H1 , Chen Y1 , Yuan J1 , Ke W1 , Li Y1 , Li M1 , Zhao X1 , Li X1 , Liu Y1 1 Evidence-Based Medicine Centre, Lanzhou University, China Background: GRADE system aims to evaluate the quality of evidence. Objectives: To assess the use of GRADE in Cochrane systematic reviews (CSRs). Methods: We searched The Cochrane Library (2011, 3 issue) for CSRs. Two reviewers independently selected CSRs which used GRADE. Then the data was extracted and analysised. Results: In total 2,169 CSRs were searched. 307 CSRs have been identified using GRADE to evaluate the quality of evidence. Most (81.11%; 249/307) reviews were published after 2008, published by the Cochrane Musculoskeletal Group, Cochrane Neuromuscular Disease Group and Cochrane Schizophrenia Group. There were a total 1,213 outcomes which were assessed by GRADE. Up to 14.59% (177/1,213) outcomes were rated as high quality; 33.8% (410/1,213) outcomes were rated as moderta quality; 28.20% (342/1,213) outcomes were rated as low quality; 12.12% (147/1,213) were rated as very low quality.The outcomes were downgraded mainly because study limitations and imprecision. Conclusions: Nowadays, few CSR uses GRADE system. Study limitations was the main factor which might decrease the quality of evidence. P2A196 Drug reviews should state industry-related bias in abstracts and plain language summaries Napoli M1 , Simi S2 1 Center for Medical Consumers, New York, USA; 2 Italian National Research Council (CNR), Institute of Clinical Physiology, Pisa, Italy Background: The abstract and plain language summaries (PLS) are the sections from Cochrane reviews most read, including many clinicians. Many, if not all, trials included in Cochrane drug reviews are conducted by drug makers. The withholding of serious adverse events data from industry-funded trials is not unusual. Such industry-related biases may be noted within the text of a Cochrane drug review; yet they rarely appear in the abstract or the PLS. Objective: This concern was initially aired on the Cochrane Consumer Network e-discussion list. We now bring this discussion to the larger community of Cochrane contributors, many of whom were in the forefront of exposing bias in industry-funded research. Review authors should address industry-related bias in the abstract and PLS when relevant. This would be consistent with the Cochrane handbook which suggests the abstract and the PLS be written as stand-alone documents. Interventions: Our poster will come with handouts that show how two Cochrane review groups have already described industry-related biases in the abstract and PLS. During ‘poster prime time’ we will be present for discussions with poster viewers and we will provide a feedback sheet to record their interest in this topic. Conclusion: If Cochrane drug reviews fail to address industry-related bias in the abstract and PLS, consumers and physicians are denied information needed to make informed decisions. This is particularly critical now that millions of healthy people worldwide are on long-term drug therapy merely because they have a risk factor like bone loss or high cholesterol. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Investigating bias Day 21 Oct (P2B197–P2B260) P2A197 Association between risk of bias and the effects of interventions on mortality in randomized controlled trials in critical care medicine Unverzagt S1 , Prondzinsky R1 1 University Halle, Germany Background: Methodological deficiencies are known to affect the results of randomized controlled trials (RCTs). Differences in the risk of bias may indicate that the effect estimates of some trials are more biased than others. Objectives: This work concentrates on reviews within emergency and critical care medicine and estimates the extent of bias on mortality. RCTs in patients with acute and partially rare disease are typically small and heterogeneous because of organizational, ethical and judicial problems. High risk of bias can result from long periods of recruitment with changing standard treatment or problems in funding. Risk of bias was assessed using the Cochrane Collaboration’s tool and six additional indication-specific quality measures: early stopping, pre-intervention, conflict of interest, baseline imbalances, cross-over trials, and insufficient length of follow-up. Methods: Subgroup analyses, logistic regression models, and fixed and random effect models were used to investigate whether quality measures explain any of the heterogeneity of odds ratios in 73 randomized trials from eight meta-analyses of Cochrane reviews of patients with sepsis, or septic or cardiogenic shock. Results: The percentage of trials with low risk of bias varied between 23% in early stopping and 81% in addressing incomplete outcome data. Overall, odds ratios were underestimated in trials with inadequate allocation concealment, selective reporting, or if a contra-active or similar supporting pre-randomization intervention could have influenced the effect of the randomized intervention or cross-over could not be ruled out. Odds ratios were overestimated in RCTs with incomplete reporting of mortality. Full data will be presented at the colloquium. Conclusions: Investigated sources of bias can lead to exaggerated or reduced estimates of the intervention effect in this medical area. These associations will support the assessment of risk of bias in Cochrane reviews on interventions in intensive care and emergency medicine. P2A198 Extraction error or interpretation: A case study of event data extraction in systematic reviews from three high-impact, high-quality peer-reviewed sources Carroll C1 , Scope A1 , Kaltenthaler E1 1 University of Sheffield, UK Background: Data extraction is a crucial but under-researched area of systematic review process and methods. Data extraction errors may occur in all variables extracted for a review, but outcomes appear to generate the most errors. Objectives: To assess differences in the event data extracted and analysed for three key outcomes in three systematic reviews on the same topic published within 18 months of one another in the following sources: The Cochrane Library Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 119 (CL); the British Medical Journal BMJ); and the authors’ own Health Technology Assessment review. Methods: We compared event data from reviews assessing the effectiveness of total hip replacement versus hemiarthroplasty for the treatment of displaced intracapsular hip fracture. We analysed the event data extracted and analysed for the outcomes of dislocation rates, one-year mortality rates, and revision rates, that were common across all three reviews, from those trials common to all three reviews. Differences and possible bias were investigated. Results: Across the three outcomes, extraction errors accounted for between 8% and 25% of the event data extracted and analysed by the BMJ review and between 0% and 17% in the CL review, respectively. A further 8–33% differences in the BMJ review and 8–17% in the CL review might be explained by issues of interpretation, eg. applying apparent ‘best case’ scenario analysis, but not justifying or explaining such choices. These differences did lead to small differences in meta-analysed relative risks, but none was significant. Conclusions: Systematic reviews require full implementation of its required processes to offer a robust method of evidence review and synthesis. The presence of errors in key outcome data suggests that full and proper implementation is not always achieved, even with apparent ‘double data extraction’ within high-quality sources of systematic reviews. Secondly, reviewers should make every effort to clarify or explain their choice of data. P2A199 Heterogeneity in systematic reviews on spinal surgery: A meta-epidemiological study Jacobs W1 , Kruyt M2 , Verbout A2 , Oner C2 1 Leiden University Medical Center, The Nethrlands; 2 University Medical Center Utrecht, Netherlands Background: Methodological design characteristics of trials in meta-analyses have been shown to influence pooled effect sizes and are therefore recognised as potential sources of heterogeneity in several medical fields. However, for spinal surgery, the presence and the strength of the effect of effect modifiers have not been assessed. Objectives: The goal of this study was to identify the influence of design characteristics on effect size in studies on effectiveness of spinal surgery. Methods: Searches were performed in MEDLINE, CDSR and EMBASE. Methodological quality of the included meta-analyses was assessed with the Amstar tool. The effect sizes of trials included in the meta-analyses were assessed. The differences in effect sizes were calculated as risk differences (RD). Relation of RDs to sponsoring, randomization, allocation concealment, blinding and study size was assessed with meta-regression adjusted for multiple testing. Results: Seven reviews consisting of 76 studies and 8191 patients were included. Data provided by the systematic reviews alone was insufficient to analyse heterogeneity. Meta-regression analysis of the individual trials though, showed that sample size, strict randomization and outcome blinding were significant factors influencing study effect. Validly randomised studies showed an increase in RD of 5.4% (95% CI 1.2 to 9.6; p = 0.044) compared to not validly randomised and observational trials. Studies with adequate observer blinding showed a 7.2% decrease in RD (95% CI 0.8 to 13.7; p = 0.049). For each increase of 100 patients, the RD decreased 3.6% (95% CI 0.4 to 6.8; p = 0.098). Conclusions: Contrary to basic methodological assertions, formal and strict randomization appeared to increase the Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. risk difference significantly in spinal surgery research. Sufficient sample size and observer blinding, on the other hand, can decrease this risk difference. This may imply that by trying to limit bias, strict randomization may actually cause bias. P2A200 Attributes of non-randomized studies associated with bias Sandhu L1 , Tomlinson G1 , Kennedy E1 , Wei A1 , Baxter N1 , Urbach D1 1 University of Toronto, Canada Background: Randomized controlled trials with inadequate allocation concealment or a lack of double-blinding are associated with biased estimates of treatment effects. However, empirical evidence of bias associated with the methodologic attributes of non-randomized studies is lacking. Objectives: To identify methodologic attributes of non-randomized studies that are associated with biased estimates of treatment effect. Methods: We identified English language randomized and non-randomized studies comparing laparoscopicassisted and conventional surgery for colon cancer. We estimated odds ratios of the risk of post-operative complications as the treatment effect of interest. Meta-regression was used to determine the association between systematic outcome assessment—an explicit statement of a standardized protocol for the collection of outcome data—and estimates for treatment effect. Results: We identified 155 comparative studies from 6,261 abstracts. Of which, 62 studies (41 NRS and 21 RCTs) involving 857,132 patients provided estimates of the risk of post-operative complications. Studies where outcomes were not assessed in a systematic manner estimated larger treatment effects; odds ratios were exaggerated by 26% (95% CI, 4%–44%) as compared with studies with systematic outcome assessment. Conclusions: Non-randomized studies with systematic outcome assessment are less prone to bias. P2A201 Reporting of participant flow diagrams in published reports of randomized trials Hopewell S1 , Hirst A2 , Collins G2 , Mallett S2 , Yu L2 , Altman D2 1 Centre for Statistics in Medicine, University of Oxford, UK; UK Cochrane Centre, UK; 2 Centre for Statistics in Medicine, University of Oxford, UK Background: Reporting of the flow of participants through each stage of a randomized trial is essential to assess the generalisability and validity of its results. Objectives: To provide a comprehensive assessment of the type and completeness of information reported in CONSORT flow diagrams published in current reports of randomized trials. Methods: A cross-sectional review of all primary reports of randomized trials which included a CONSORT flow diagram indexed in PubMed core clinical journals (2009). We assessed the proportion of parallel group trial publications reporting specific items recommended by CONSORT for inclusion in a flow diagram. Results: Of 469 primary reports of randomized trials, 263 (56%) included a CONSORT flow diagram of which 89% (237/263) were published in a CONSORT endorsing journal. Reports published in CONSORT endorsing journals were more likely to include a flow diagram (62%; 237/380 versus 29%; 26/89). Ninety percent (236/263) of reports which included a Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 120 flow diagram had a parallel group design, of which 49% (116/236) evaluated drug interventions. Eighty-one percent (191/236) reported the overall number of participants assessed for eligibility, 71% (168/236) the number excluded prior to randomization and 98% (231/236) the overall number randomized. Reasons for exclusion prior to randomization were more poorly reported. Ninety-four percent (223/236) reported the number of participants allocated to each arm of the trial. However, only 40% (95/236) reported the number who actually received the allocated intervention, 67% (158/236) the number lost to follow up in each arm of the trial, 61% (145/236) whether participants discontinued the intervention during the trial and 54% (128/236) the number included in the main analysis. Conclusions: Over half of the published reports of randomized trials included a diagram showing the flow of participants through the trial. However, information was often missing from published flow diagrams, even in articles published in CONSORT endorsing journals. P2A202 Quality of studies reporting patient-reported outcomes identified within a systematic review on low-dose rate brachytherapy for localized prostate cancer Peinemann F1 , Grouven U1 , Sauerland S1 1 IQWiG Background: Permanent interstitial low-dose rate brachytherapy (LDR-BT) is a treatment alternative for patients with localized prostate cancer. As evidence from research on effectiveness or safety may not be decisive, patient-reported outcomes (PRO), such as function, bother, and generic health-related quality of life (HRQL), could have an impact on medical decision making. Methods: Bibliographic databases (MEDLINE, EMBASE, and The Cochrane Library) were searched until June 2010 for randomized as well as non-randomized parallel group studies. Validated questionnaires, 70% or higher response rate, and 80% or more data with relevant diagnosis and therapy were required for inclusion of studies. For non-randomized controlled studies, baseline data were required to be comparable between treatment groups or results were required to be adjusted for major confounders. Results: We identified only 1 randomized study among 61 studies potentially relevant for inclusion. PRO were reported in 32 of 61 and 7 of these 32 studies were finally included. Thus, 25 studies were excluded because baseline data were not comparable and not adjusted or stratified (16 studies), response was below 70% (4 studies), or data were not reported separately for the accordant groups or outcomes (5 studies). Of the 7 studies, 7/5/5 studies evaluated function/bother/generic HRQL scores using 7/2/3 different instruments, respectively. Conclusions: We found that the majority of identified non-randomized controlled PRO studies showed selection bias too high for inclusion. Included PRO studies lacked statistics about differences across treatment groups and various scales complicated the synthesis of results. P2B203 Systematic reviews of paediatric surgical interventions: quantity, coverage and quality McGee R1 , Craig J2 , Webster A2 1 Sydney School of Public Health, University of Sydney, Australia; 2 Cochrane Renal Group, Sydney, Australia Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: The practice of evidence-based surgery depends on high quality systematic reviews being readily available. Previous research has highlighted the lack of high quality research in paediatric surgery although systematic reviews have not been assessed. Objectives: To determine the quantity, coverage and reporting quality of systematic reviews of paediatric surgical interventions. Methods: MEDLINE, Embase, DARE, PROSPERO and the Cochrane Database of Systematic Reviews were searched for any systematic reviews of paediatric surgical interventions published before 2011. The PRISMA checklist was used to assess reporting quality. Results: Fifteen reviews were identified of which 13 were Cochrane reviews. The median year of publication was 2010 and the earliest was 2001. The most commonly assessed area was the use of surgical treatments for the management of otitis media (four reviews). Reporting quality was mixed (see Figure 1). In particular, one-third of included reviews did not declare their funding source or the role that funders played in the conduct of their review. No review utilised the PRISMA flow diagram to illustrate the results of their literature search. Conclusions: Systematic reviews are unavailable for many areas in paediatric surgery. Surgeons trying to assess the safety and efficacy of an intervention will therefore be required to conduct their own assessments of the primary literature or extrapolate from adult systematic reviews. To reduce the discrepancy between best reporting quality as defined by PRISMA and the Cochrane Collaboration, the Cochrane Handbook for Systematic Reviews of Interventions and the PRISMA statement should be more closely aligned. Already some progress has been made in this regard with the recent upgrade to the Cochrane collaboration’s Review Manager Software, which now includes the ability to create a PRISMA flow diagram. P2B204 Core outcomes in systematic reviews of paediatric surgical interventions McGee R1 , Craig J2 , Webster A2 1 Sydney School of Public Health, University of Sydney, Australia; 2 Cochrane Renal Group, Sydney, Australia Background: Recent initiatives have highlighted the importance of standardised outcome measures in clinical trials to reduce reporting bias, misinformation and wasted resources. Although it has received less attention, the principles behind these efforts remain the same for systematic reviews. Objectives: To determine which outcomes are assessed and how they are defined in systematic reviews of paediatric surgical interventions. Methods: MEDLINE, EMBASE, DARE, PROSPERO and the Cochrane Database of Systematic Reviews were searched for any systematic reviews of paediatric surgical interventions published before 2011. All specified outcomes and related definitions were recorded. Results: Fifteen reviews were identified of which 13 were Cochrane reviews. The mean number of outcomes per review was six, with a range of two to 14 and a mean of two primary outcomes. Outcome definitions were provided in three reviews. Time-points for outcome assessment were only pre-specified in one review. Mortality, adverse events (only one review was specific about which adverse events were to be recorded) and quality of life (only one specified for whom quality of life was being assessed) were the most commonly reported outcomes (five reviews each). Conclusions: Outcomes assessed in systematic reviews of paediatric surgical interventions are Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 121 Figure 1 (P2B203). Star plot of the percentage of paediatric surgery reviews adequately reporting PRISMA items. inconsistent, poorly defined and reported, and are therefore susceptible to a high risk of outcome reporting bias. While we have assessed reviews covering a range of surgical treatments, there is a core set of outcomes that should be assessed in all reviews of paediatric surgery, e.g. mortality, adverse events and quality of life for both the child and carer. As the Cochrane Collaboration publishes the most systematic reviews of paediatric surgical interventions, we suggest that they develop and promote a set of core outcomes for inclusion in systematic reviews of paediatric surgical interventions. In addition, a set of standardised outcome definitions may improve the reporting quality of systematic reviews. P2B205 ‘Fatal flaws’ in randomised controlled trials of interventions for low-back pain Henschke N1 , Ostelo R2 , van Tulder M3 1 Musculoskeletal Division, The George Institute for Global Health, Australia; 2 EMGO Institute for Health and Care Research (EMGO+), Netherlands; 3 Department of Health Sciences, VU University, Netherlands Background: When performing a systematic review, it is possible that a study could be included even if it has a serious methodological flaw which can render the results implausible. These ‘fatal flaws’can include Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. excessive attrition or a large proportion of drop-outs and withdrawals. However, there is no consensus regarding which methodological flaws can be considered ‘fatal’, and what we should do with trials which have them. Objectives: The aim of this study is to identify, via consensus among a panel of experts, which features of randomised controlled trials (RCTs) should be considered ‘fatal flaws’ and to develop recommendations for dealing with them. The focus for the study will be RCTs on interventions for low-back pain. Methods: A modified-Delphi technique using three survey rounds was used to select and reduce a list of methodological characteristics. Experts in low-back pain, systematic reviews, and methodology were invited to participate in the study. These experts were asked to individually rate the list of characteristics according towhether they consider the item to be a serious methodological flaw that could render the results of a study implausible. During three surveys, participants were given the opportunity to provide reasons for their decision, to add items, or to modify the wording of items. Consensus was considered to be reached when the rating of at least two-thirds (67%) of the panel members indicated agreement with an item on a five-point scale. Results: The respondent panel consists of 68 experts. At present, the first two rounds of the Delphi technique have been completed and the third round is underway. Final results will be presented at the colloquium. Conclusions: The identification of serious methodological flaws and how to deal with trials which have them may lead to improved validity of the results in systematic reviews. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 122 P2B206 Trends over time in the size and quality of randomised controlled trials of interventions for chronic low-back pain Henschke N1 , Kuijpers T2 , Rubinstein S3 , van Middelkoop M4 , Ostelo R3 , Verhagen A4 , Koes B4 , van Tulder M5 1 Musculoskeletal Division, The George Institute for Global Health, Australia; 2 Dutch Institute for Healthcare Improvement CBO, Netherlands; 3 EMGO Institute for Health and Care Research, Netherlands; 4 Department of General Practice, ErasmusMC, Netherlands; 5 Department of Health Sciences, VU University, Netherlands Background: Despite the high prevalence and enormous burden of chronic low-back pain (CLBP), there remains uncertainty regarding the most effective form of therapy. This continuing uncertainty is often considered to be due to methodological shortcomings or a lack of adequate reporting. In light of the increasing number of randomised controlled trials (RCTs) and continuing initiatives to improve the quality of clinical research, it could be expected that the methodological quality of RCTs in the field of CLBP may have improved during recent years. Objectives: This study aimed to identify and describe trends over time in the study design characteristics and risk of bias in CLBP trials performed over the past 30 years. Methods: We extracted 157 randomised trials of interventions for CLBP from four recently published systematic reviews. The reviews included RCTs on physical and rehabilitation interventions, injection therapy and denervation procedures, complementary and alternative therapies, and pharmacological interventions for chronic LBP. Study level data were extracted and analysed for trends associated with year of publication. Results: Overall, the mean sample size in the RCTs was 141 (median 70; range 17 to 3093). There was a slight increase in the median number of risk of bias criteria fulfilled from trials published prior to 1995 to those published after 1996. The analysis showed that in more recent years RCTs of medical interventions were more likely to be successfully blinded than RCTs of non-medical interventions. Conclusions: Over the past 30 years, the number of RCTs for CLBP has increased exponentially. However, there does not appear to be a corresponding increase in sample size or a decrease in risk of bias. Further research is needed into specific risks of bias within RCTs for CLBP and the effect they have on the plausibility of the results. P2B207 The risk-of-bias and standards of reporting of published randomized controlled trials of medical education (Med Ed) research Horsley T1 , Rabb D1 , Campbell C1 , Hamstra S2 , Cook D3 1 Royal College of Physicians and Surgeons of Canada; 2 AIME University of Ottawa; 3 MAYO College of Medicine Background: Controlling bias within randomized trials (RCTs) has been empirically shown to reduce errors in estimates of treatment effects. In theory, the RCT is less susceptible to bias than any other research study design. It is this ‘susceptibility to bias’ that is the focus of this work. Objectives: We aimed to establish the risk of bias and standard of reporting of RCTs of medical education (Med Ed) research. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Methods: Records were independently assessed until a representative sample (N = 150) was achieved. Publications were evaluated in duplicate using the Cochrane risk of bias (ROB) assessment tool and the CONSORT reporting guideline for RCTs. Results: Reporting of components of the CONSORT guidelines (including, but not limited to, sample size calculation, blinding, baseline characteristics, and sequence generation) are woefully inadequate. Med Ed RCTs were predominately rated to be of ‘unclear’ risk of bias. Conclusions: The Med Ed discipline should strongly consider using CONSORT for improving reporting of RCTs. A proposal for developing an extension for RCTs of education for consideration by the CONSORT consortium is underway. Considerations for risks of bias should be made when interpreting Med Ed RCTs. P2B208 Are overviews of (systematic) reviews really of good quality and transparent information? Li L1 , Yang K2 , Han D3 , Tian J2 , Sun T4 1 Evidence Based Medicine Center of Lanzhou University, School of Basic Medical Science of Lanzhou University, Lanzhou 730000, China; The first Clinical Medical College of Lanzhou University, Lanzhou 730000, China; 2 Evidence Based Medicine Center of Lanzhou University, School of Basic Medical Science of Lanzhou University, Lanzhou 730000, China; 3 The second Clinical Medical College of Lanzhou University, Lanzhou 730000, China; 4 Shanghai Institute of Digestive Disease, Shanghai Jiao-Tong University School of Medicine Ren-ji Hospital, Shanghai 200001, China Background: Overviews of (systematic) reviews, or umbrella reviews, are designed to compile evidence from multiple systematic reviews of interventions into one document and provide the reader with a quick overview relevant to a specific decision. Objectives: To evaluate reporting and methodological qualities of overviews of reviews. Methods: Searches were conducted using (‘overview’ AND (‘meta analys*’ OR ‘systematic review*’)) OR ‘umbrella review’ in title/abstract in PubMed, the Cochrane library, EMBASE, ISI Web of Knowledge. All searches were conducted in February 2010, and updated in September 2010. We included those overviews that only included systematic reviews or meta-analyses. We developed an 18-item assessment tool to assess the methodological and reporting qualities of overviews of systematic reviews. Two independent reviewers assessed the qualities with this checklist, and resolved differences with a third reviewer. Results: We found 41 overviews of systematic reviews, whose mean total reporting and methodological score was 10.78 (SD 3.84) of 18 items and 3.05 (SD 2.09) of 8 items. Some necessary items for the reporting and methodological qualities in the overview were not reported, for example 69% mentioned inclusion and exclusion criteria, 76% mentioned information sources, 49% mentioned review selection, 44% mentioned data collection, 7% mentioned reporting quality assessment, 46% mentioned methodological quality assessment, and 20% mentioned quality of evidence assessment. Conclusions: The reporting and methodological qualities of overviews of systematic reviews were very poor, and there is still much room for improvement. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 123 P2B209 What are the reporting and methodological qualities of network meta-analysis? Li L1 , Sun T2 , Han D3 , Yang K4 , Tian J4 1 Evidence Based Medicine Center of Lanzhou University, School of Basic Medical Science of Lanzhou University, Lanzhou 730000, China; The first Clinical Medical College of Lanzhou University, Lanzhou 730000, China; 2 Shanghai Institute of Digestive Disease, Shanghai Jiao-Tong University School of Medicine Ren-ji Hospital, Shanghai 200001, China; 3 The second Clinical Medical College of Lanzhou University, Lanzhou 730000, China; 4 Evidence Based Medicine Center of Lanzhou University, School of Basic Medical Science of Lanzhou University, Lanzhou 730000, China Background: Network meta-analysis can combine information from all randomized controlled trials when many treatment regimens already exist, in order to rank the regimens to determine the best option for patients. But is is not clear what are the reporting and methodological qualities of network meta-analysis. Objectives: To evaluate the reporting and methodological qualities of network meta-analyses. Methods: We searched Pubmed, the Cochrane library, Embase, ISI Web of Knowledge, and Google Scholar using ‘Network meta analysis’ OR ‘mixed treatment comparisons meta analyses’ OR ‘multiple treatments meta analysis’ in title/abstract without language, time, or publication type restrictions up to 2010 October 21 and updated up to 2011 February 21. We included any network meta-analysis, regardless of the conditions or interventions. We evaluated the reporting qualities of network meta-analysis using the PRISMA statement and methodological quality was assessed using OQAQ as a network meta-analysis was considered as a meta-analysis. Two trained authors independently selected studies, abstracted data, and assessed the qualities, and disagreement was resolved by a third author. Results: We found 37 network-meta analyses. Thirteen items were reported in less than 75% articles, in which seven items were reported in less than 50%. Fifty-one percent tried to avoid bias in the selection, and 54% reported criteria used for assessing validity, but 24% were assessed appropriately. Conclusions: Some necessary methodological and reporting quality items were not performed or reported adequately, so there is much room to improve. P2B210 Epidemiology and quality of reporting of conference abstracts related to randomized controlled trials in renal transplantation Wang X1 , Jiang J2 , Li L3 , Tian J3 , Tian H3 , Ying X4 1 Evidence Based Medicine Center of Lanzhou University, School of Basic Medical Science of Lanzhou University; The second Clinical Medical College of Lanzhou University, Lanzhou 730000, China; 2 Evidence Based Medicine Center of Lanzhou University, School of Basic Medical Science of Lanzhou University, The second Clinical Medical College of Lanzhou University, Lanzhou 730000, China; 3 Evidence Based Medicine Center of Lanzhou University, School of Basic Medical Science of Lanzhou University, Lanzhou 730000, China; 4 The second Clinical Medical College of Lanzhou University, Lanzhou 730000, China Background: Transparent and accurate conference abstracts of randomized controlled trials (RCTs) are important, especially when they are used to make health-care decisions. However, the reporting Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. quality of conference abstracts related to RCTs in renal transplantation is unknown. Objectives: To assess the reporting quality of conference abstracts of RCTs in renal transplantation and to identify factors associated with high quality. Methods: Web of science was searched to identify RCTs of renal transplantation and limited to conference proceedings. Their qualities were evaluated with a modified version of CONSORT for reporting randomized trials in Journal and conference abstracts. The percentage of trials that scored ‘yes’ on each quality item and the 95% confidence interval (CI) was calculated. Differences of the reporting quality of different items (region, center, published year) were tested using Chi-square analysis expressed as odds ratio (OR) with 95% CI with Mantel-Haenszel method. Results: We included 112 RCTs. More than half of conference abstracts reported specific objective (86%, 95% CI 91% to 77%), recruitment (55%, 95% CI 64% to 46%), number of participants randomized (63%, 95% CI 71% 53%), primary outcome (51%, 95% CI 60% to 42%), and general interpretation of the results (90%, 95% CI 94% to 83%). All of the abstracts reported the contact details for the corresponding author. Only 2% of the abstracts provided details of blinding. No abstract explained the methods of random sequence generation, allocation concealment, source of funding or trial registration. Compared to abstracts not reporting center, the quality of single-center (OR 1.93, 95% CI 1.25 to 2.97) and multi-center (OR 1.84, 95% CI 1.44 2.35) was significantly higher. The reporting quality of European is significant lower than North American (OR 0.68, 95% CI 0.50 to 0.93). But statistically significant differences were not found between publication before and after 2008. Conclusions: The reporting quality of conference abstracts of RCTs in renal transplantation was suboptimal, especially for methodological items where improvements are needed. P2B211 The relationship between risk of bias and effect size: randomized controlled trials Dolores F1 , Héctor M1 1 University of Valencia Spain Background: Randomized controlled trials (RCTs) are the gold standard for trials assessing the effects of therapeutic interventions and provide the best evidence to inform and guide clinical decision-making. Therefore, it is important to understand how they are conducted and how to evaluate the strength of evidence. Bias is a systematic tendency to produce an outcome that differs from the underlying truth. Bias in clinical trials falls into four categories: selection bias, performance bias, detection bias and attrition bias. There are numerous tools to assess methodological quality of primary studies. Understanding the relationship between the risk of bias and the effect size can help plan research to monitor the presence of such bias. Objectives: Our work has three objectives. First, provide an analysis of the relationship between bias and effect size. Second, analyze the bias and reporting checklists (example CONSORT). Third, we review and analyze the Cochrane Collaboration’s tool for assessing risk of bias. Methods: Systematic review. Results: The methodological quality of studies can have a substantial impact on estimates of treatment effect, which may affect the validity of the conclusions. Conclusions: Our work helps to identify areas of strength and weakness in the existing evidence and to formulate recommendations to improve the conduct and value of future research. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 124 P2B212 Definitions of intention-to-treat in the face of missing outcome data: A systematic survey of methodology papers Alshurafa M1 , Guyatt GH1 , Akl EA2 , Haines T1 , Gentles S1 , Rios L1 , Tran C3 , Lamontagne F4 , Moayyedi P5 , Walter SD1 , Briel M6 1 Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada; 2 Departments of Medicine and Family Medicine, State University of New York at Buffalo, Buffalo, USA; 3 Chau Tran Consulting, Toronto, Canada; 4 Centre de recherche clinique Étienne-Le Bel, Université de Sherbrooke, Sherbrooke, Canada; 5 Department of Medicine, McMaster University, Hamilton, Canada; 6 Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Basel, Switzerland Background: Intention-to-treat (ITT) has been established as a quality label for randomized clinical trials. However, surveys of randomized trials showed that trialists hold a variety of views on the relationship between missing outcome data (MOD) and ITT and how MOD should be handled under ITT. Objectives: We surveyed articles addressing the methodology of ITT in relation to MOD. Methods: We systematically searched MEDLINE up to December 2008 for methodological articles written in English that devoted at least one paragraph to ITT and two other paragraphs to either ITT or MOD/loss to follow-up. We excluded original trial reports, observational studies, or clinical systematic reviews. Working in teams of two, we independently extracted relevant information from each eligible article. Discrepancies between data extractors were resolved by consensus or by third party arbitration if necessary. Results: Of 1007 titles and abstracts reviewed, 66 articles met our eligibility criteria. Five (8%) did not provide a definition of ITT; 25 (41%) mentioned MOD but did not discuss its relationship with ITT; and 36 (59%) discussed the relationship of MOD with ITT. These 36 articles described one or more of three distinct relationships: full-follow-up is required for ITT (58%); ITT and MOD are separate issues (17%); and ITT requires a specific strategy for handling MOD (78%); 17 (47%) described more than one relationship. The most frequently mentioned strategies for handling MOD were using the last outcome carried forward (50%), sensitivity analysis (50%), and use of available data to impute missing data (46%). Most articles (81%) specifically excluded complete case analysis under ITT. Conclusions: We found that there is no consensus on the definition of ITT in relation to MOD. For conceptual clarity, we suggest considering deviations from randomization separately from issues of MOD when reporting randomized trials. P2B213 Methodological and reporting quality of systematic reviews on influenza Jiang J1 , Wang X1 , Tian J2 , Lin Q1 , Li L1 1 Evidence Based Medicine Center of Lanzhou University, School of Basic Medical Science of Lanzhou University; The second Clinical Medical College of Lanzhou University, Lanzhou 730000, China; 2 Evidence Based Medicine Center of Lanzhou University, School of Basic Medical Science of Lanzhou University, Lanzhou 730000, China Background: Influenza is becoming more and more threatening to the world, and many systematic reviews about preventing and treating influenza are conducted. But the methodological and reporting quality Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. of these systematic reviews (SRs) is unknown. Objectives: To assess the methodological and reporting qualities of SRs on influenza and compare these qualities of Cochrane systematic reviews (CSRs) with non-cochrane systematic reviews (NCSRs). Methods: PubMed, EMBASE, Web of science and the Cochrane Library were searched for SRs on influenza using ‘systematic reviews’, ‘meta-analysis’, ‘influenza’, ‘flu’ in 2011 February. We assessed the methodology quality with OQAQ and AMSTAR, and the reporting quality using PRISMA. The result of ‘yes’ for AMSTAR or PRISMA was scored ‘1’, and other conditions was marked ‘0’. The total scores of AMSTAR and PRISMA for each SR were calculated. For OQAQ, 1, 3, 5 or 7 was marked for each SR if there were extensive, major, minor or minimal flaws. The data was expressed with mean ± SD and T-test was used to test the differences of the qualities between CSRs and NCSRs by SPSS 17.0. Results: In total, 28 SRs (15 CSRs and 13 NCRs) were identified. The methodological quality score of all SRs using OQAQ and AMSTAR were 4.21 ± 0.99 and 7.86 ± 1.67, respectively. The reporting quality score of all SRs using PRISMA was 20.18 ± 4.09. Compared with NCSRs, CSRs showed better qualities in reporting by PRISMA (MD 5.67, 95% CI 3.14 to 8.20) and in methodology by OQAQ (MD 1.25, 95% CI 0.67 to 1.83) and AMSTAR (MD 1.23, 95% CI 0.01 to 2.45). Conclusions: The reporting and methodological qualities of SRs on influenza were not very low, but they need to be improved. The methodological and reporting qualities of CSRs were better than those of NCSRs; therefore, the PRISMA and AMSTAR/OQAQ should be recommended for NCSRs especially. P2B214 Characteristics of randomized controlled trials (RCT) published in Latin America and the Caribbean’s (LAC) in 2010 Reveiz L1 , Glujovsky D2 , Pinzon CE3 , Asenjo-Lobos C4 , Cortes M5 , Canon M6 , Bardach A2 , Comandé D2 , Cardona AF7 1 Policy and Research, Health Systems Based on Primary Health Care. Pan American Health Organization, WDC, USA; 2 Argentine Cochrane Centre IECS, Institute for Clinical Effectiveness and Health Policy Buenos Aires, Argentina; 3 Instituto de Investigaciones, Fundación Universitaria Sanitas, Bogotá D.C., Colombia; 4 Centro Rehabilitación Oral Avanzada e Implantologı́a (CRAI) Universidad de Concepción (Centro Adherido Chileno de la Red Cochrane Iberoamericana) – Chile; 5 Chilean branch of the Iberoamerican Cochrane Network, Universidad Católica de la Santı́sima Concepción, Chile; 6 Facultad de Medicina, Fundación Universitaria Sanitas, Bogotá D.C., Colombia; 7 Grupo Oncologı́a Clı́nica y Traslacional, Instituto de Oncologı́a, Fundación Santa Fe de Bogotá, Bogotá D.C., Colombia Introduction: Few studies have evaluated the characteristics of RCTs conducted in LAC countries. Methods: A review of RCTs published in 2010 in which the author’s affiliation was from LAC was performed in PUBMED-LILACS; the RCT was included if it explicitly used the word ‘random’ or variations thereof. Two reviewers independently extracted key data and assessed the risk of bias (RoB). Results: We identified 1695 references including 526 RCTs (N = 73.513) recruiting 139.8 (SD = 284.53) participants on average. English was the dominant publication language (93%); with Spanish (3.4%) and Portuguese (2.9%) following. We found RCTs with affiliations representing 19 LAC countries; 70% were from Brazil. Maternal health represented 6.7% and neglected diseases-HIV-tuberculosis-malaria 3.8%. We Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 125 assessed the following characteristics in a sample of 326/526 RCTs. Settings/Authorship: 7.7% were multinational and 12.0% multicenter. 10.6% included a co-author from a different country (the USA being the most frequent). Participant’s sex: 75.3% of RCTs recruited men and women; 19.6% exclusively women. Overall, 57.4% of participants were women. Gender analysis was performed in 6.8% of studies. 19% were <18 years. Type of intervention: Drugs (47.3%) and procedures (24.5%) were the most frequent. Follow-up: Short term (<1 month) (38%) and 1-6 months 39% were the most frequent. Risk of bias: Scores were low (25.2%), unclear (60.1%), and high (14.7%). Ethic/Conflict of interest: 95.1% reported approval by ethics review committee and 93.1% presented an informed consent process. Conflicts of interest were declared by at least one author in 11.5% of RCTs; 42.5% failed to provide a declaration. Funding: Public 33.9%; private (i.e. pharmaceutical) 14.4%; not reported 36.6%; others (i.e. mixed-NGO) 15.1%. More frequent funders were FAPESP, CAPES and CNPq from Brazil. Conclusion: There is plenty of room for improvement in South–South collaboration and RCTs addressing regional priorities. Methods and result reporting of RCTs should be improved to meet international standards. P2B215 Impact of the trial registration initiative in Latin America and the Caribbean: a study of randomized controlled trials published in 2010 Reveiz L1 , Bonfill X2 , Glujovsky D3 , Pinzon CE4 , Asenjo-Lobos C5 , Cortes M6 , Canon M7 , Bardach A3 , Comandé D3 , Cardona AF8 1 Policy and Research, Health Systems Based on Primary Health Care, Pan American Health Organization. WDC, USA; 2 Servicio de Epidemiologı́a Clı́nica y Salud Pública, Centro Cochrane Iberoamericano, Hospital de la Santa Creu i Sant Pau (UAB), Barcelona, España; 3 Argentine Cochrane Centre IECS, Institute for Clinical Effectiveness and Health Policy Buenos Aires. Argentina; 4 Instituto de Investigaciones, Fundación Universitaria Sanitas. Centro Colaborador de la Colaboración Cochrane, Bogotá D.C., Colombia; 5 Centro Rehabilitación Oral Avanzada e Implantologı́a (CRAI) Universidad de Concepción (Centro Adherido Chileno de la Red Cochrane Iberoamericana) – Chile; 6 Facultad de Medicina, Centro Colaborador de la Colaboración Cochrane, Universidad Católica de la Santı́sima Concepción. Chile; 7 Facultad de Medicina, Fundación Universitaria Sanitas, Bogotá D.C., Colombia; 8 Grupo Oncologı́a Clı́nica y Traslacional, Instituto de Oncologı́a, Fundación Santa Fe de Bogotá, Bogotá D.C., Colombia Introduction: Adherence to clinical trial registration in Latin American and the Caribbean (LAC) countries is yet to be evaluated. Objective: Determine the prevalence of registration of randomized controlled trials (RCT) published in PUBMED-LILACS in 2010 from LAC in the International Clinical Trial Registry Platform (ICTRP) compliant databases; and to compare methodological characteristics of registered vs. non-registered RCT. Methods: A search for detecting RCTs in which at least the first author had a LAC’s affiliation was made. The RCT was included if it explicitly used the word ‘random’ or variations thereof. Trials not including human subjects were excluded. No language restrictions were applied. We assessed the full text of the reports to determine if the RCT had been registered in a register contributing to the ICTRP. Data from studies were independently extracted by two authors. We assessed the risk of bias (RoB) in all Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. registered RCTs (n = 89) and in a sample of non-registered RCTs (n = 237). Results: The search identified 1695 references; we included 526 RCTs from 19 countries of which 16.9% (89/526) were registered in the ICTRP. Prospective registration was done in 21 (4.0%) RCTs. We found a significant difference in the overall RoB assessment between registered (5.1% high; 60.3% unclear; and 34.6% low risk) and non-registered RCTs (high 18.5%; unclear 59.3%; and 22.2% low risk); significant differences were also found in the proportion of ‘allocation concealment’ and ‘other types of bias’ domains of the RoB. The number of RCTs reporting conflict of interests, approval by an ethics committee and the provision of informed consent were significantly higher in registered studies. No significant differences were found when analyzing only prospectively registered vs. non-registered RCTs. Conclusion: There is a significant association between clinical trial registration with lower RoB and with a better quality assessment. There is room for improvement in trial registration adherence. P2B216 Does the sensitivity of tests for Down Syndrome really decrease with maternal age? A case-study illustrating the potential impact of loss to follow up on review conclusions Pennant M1 , Guo B1 , Alldred K2 , Neilson J2 , Alfreivic Z2 , Deeks J1 1 The University of Birmingham, UK; 2 The University of Liverpool, UK Background: In a Cochrane Diagnosis Test Accuracy review of maternal screening for Down’s Syndrome, we observed that the estimated sensitivity of the 2nd trimester serology based triple test appeared to be 20% lower in studies of women aged >35 compared to studies unrestricted by age. Objectives: To investigate whether the study design, and associated risk of loss to follow up, could explain the observed relationship. Methods: Methodological characteristics of the included studies were evaluated and considered. Results were plotted in summary ROC space identifying studies according to their methodological characteristics as well as maternal age. Bivariate random effects meta-regression models were used to estimate the effect of study characteristics on test performance. Sensitivity analyses were used to investigate whether differential loss to follow-up could explain the observed relationship with age. Results: Two distinct groups of studies were evident: those in which the reference standard diagnosis involved follow-up to birth (risking loss to follow-up) and those where all patients received invasive confirmatory testing (minimising loss to follow-up). Only studies of older mothers used the latter design. Comparison of test accuracy between these groups for the largest dataset showed, for a false positive rate of 5%, significantly higher sensitivity (68.6%; 95% CI 62.3–74.3%, n = 11 versus 48.4%, 95% CI 40.7–56.2%, n = 13, p < 0.0001) for follow up compared to confirmation studies. Sensitivity analyses showed that the observed relationship with maternal age could partially (but not totally) be explained by differential loss-to follow-up in studies undertaken using follow-up to birth. Conclusions: The apparent increase in sensitivity in follow up compared to confirmation studies may partially be an artefact of differential loss to follow-up, where test negative women who had Down’s syndrome fetuses (false negatives) are more likely to miscarry. In confirmation studies, all false negatives were likely to be identified, giving more reliable estimates of test accuracy. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 126 P2B217 Split-Mouth and crossover trials in dentistry are neither well differentiated, nor well analyzed or reported Brignardello-Petersen R1 , Carrasco-Labra A2 1 Evidence Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Chile, Department of Health Policy, Management & Evaluation, University of Toronto, Canada; 2 Evidence Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Chile, Department of Clinical Epidemiology & Biostatistics, McMaster University, Canada Background: Crossover and split-mouth randomized controlled trials (RCTs) are designs commonly used in dentistry. Although their statistical analysis should be similar, they have different fundaments and are indicated when the population and interventions under study have specific characteristics. However, it seems that many researchers are not aware of the methodological differences between them and tend to describe split-mouth trials as crossover, failing to accomplish crucial methodological requirements of the former design. Objectives: To determine if the requirements of the design and analysis of crossover and split-mouth RCTs are being considered in the field of oral and maxillofacial surgery (OMFS). Methods: Systematic Review. A comprehensive searching in MEDLINE was used to retrieve all the split-mouth and crossover trials published in the four journals of OMFS of highest impact factor. RCTs classified by the authors as split-mouth or crossover, or if the methods section described the methodology of one of these designs were selected. Two evaluators extracted the data independently. Results: A total of 152 articles were retrieved from the electronic searching. The final sample was composed of 41 articles, from which 82.93% were classified as crossover by the authors. Of these trials, only 6% had a crossover design. None of the trials considered the possible carry-over or carry-across effect that could bias the results. Of the whole sample, 43.9% analyzed the results without considering the paired nature of the data. The presence of a statistician, the journal, and the year of publication showed no statistical association with the reporting of a proper analysis (p > 0.05). Conclusions: In general, authors of RCTs in OMFS do not recognize the differences between crossover and split-mouth trials. There is room for improvement in the particular aspects of these designs. P2B218 Are there discipline-specific quality measures? A systematic review of meta-epidemiological studies Jacobs W1 , Kruyt M2 , Moojen W1 , Verbout A2 , Oner C2 1 Department of Neurosurgery, Leiden University Medical Centre (LUMC), Leiden, Netherlands; 2 Department of Orthopedic Surgery, University Medical Center Utrecht (UMCU), Utrecht, Netherlands Background: Several studies with a meta-epidemiological approach have been published on effect of quality measures in clinical research. The results from these studies show some variations. Objectives: The goal of this systematic review of meta-epidemiological studies was to systematically search and evaluate the evidence for effect of quality measures in clinical research and to determine whether differences in effect of quality measures could be attributed to type of intervention. Methods: Systematic search of Medline, EMBASE Web of Science, Cochrane methodology register, and reference lists Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. for meta-epidemiological studies with estimates of effect of quality measures. References were selected by predefined criteria and appraised with the Amstar tool adjusted for the purpose. Selection and quality evaluation were performed in duplicate. We anticipated, but were not limited to, effects of study size, study type, randomization, concealment, blinding (care-provider, patient and observer), and sponsoring. We made a distinction between surgical, pharmaceutical, and treatment programs (psycho-therapeutical, physio-therapeutical) as separate types of interventions. Heterogeneity was calculated to identify differences between types of intervention. Results: We found 23 meta-epidemiological studies from which we were able to extract 50 effect estimates of 11 different quality measures. Eleven of these estimates were statistically significant. All of these confirmed the common assumptions of smaller effect estimates in studies with ‘better’ quality, defined as larger studies, use of randomised allocation, proper allocation concealment, and double blinding. The only exception is a surgical study that found an increased effect estimate in randomised studies. Conclusions: Study size and allocation concealment were more consistently found to have an influence on effect size than other quality measures. For randomization and allocation concealment there are indications that the effect size might be different between different types of interventions. P2B219 Epidemiology and publication history of randomized controlled trials that were discontinued Blümle A1 , Kasenda B2 , Saccilotto R2 , Briel M2 , von Elm E3 1 German Cochrane Centre, Department of Medical Biometry and Statistics, University Medical Center Freiburg, Freiburg, Germany; 2 Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Basel Switzerland; 3 Cochrane Switzerland, Institute of Social and Preventive Medicine, CHUV and University of Lausanne, Switzerland Background: Little is known about the epidemiology and publication history of randomized controlled trials (RCTs) discontinued due to futility, harm, or insufficient recruitment. We hypothesized that the data already collected are often not made available to the scientific community. The lack of reporting potentially enhances the problem of wasted research efforts and raises ethical concerns, because trial participants consent on the premise of contributing to the advancement Table 1 (P2B219). Reasons for discontinuation in 64 RCTs. Reasons for discontinuation N (%) Slow recruitment Futility Harm Funding problems Others∗ Total 26 (41) 8 (12) 5 (8) 9 (14) 16 (25) 64 (100) ∗ This category includes the following: parallel study in USA showed no effect (N = 1), unknown (N = 2), not clearly specified (N = 5), protocol violation (N = 1), loss of study data due to technical defect (N = 1), closing of liver transplant program (N = 1), change in researchers’ career (N = 5). Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 127 of medical knowledge. It may also lead to bias in meta-analyses if relevant data from discontinued trials are not considered. Objectives: To describe the epidemiology and publication history of discontinued RCTs. Methods: We included protocols of RCTs approved by the research ethics committee of the University of Freiburg (Germany) between 2000 and 2002. We searched electronic databases and surveyed trialists for information about project/publication status. In case of discontinuation, further information about the reasons was collected. Results: Of 418 approved RCTs, 194 (46%) were published in full, and 224 (54%) were not. Overall, 300 (72%) RCTs were either completed according to protocol, ongoing, or not started at all; for 54 (13%) we received no survey response, and 64 (15%) were definitely discontinued. Of the latter, 8 (13%) were published and 56 (87%) were not. The reasons for discontinuation were slow recruitment (41%), futility (12%), harm (8%), funding problems (14%), and others (25%) (Table 1). Conclusions: About half of all approved RCTs were not published in full and at least 15% were discontinued. Most of the data collected until discontinuation were not published and thus not easily available for meta-analyses. Slow recruitment was the most frequent reason for discontinuation. Our findings emphasize the necessity for a screening tool to identify trials at risk of being discontinued because of recruitment problems. This could help reduce the waste of scarce resources in clinical research. P2B220 Sponsors’ influence on reporting of results in industry-sponsored trials: a comparison of protocols and published papers Lundh A1 , Krogsbøll LT1 , Gøtzsche PC1 1 The Nordic Cochrane Centre, Denmark Background: Bias in industry-sponsored trials is common and interpretation of results may be distorted in favour of the sponsors’ products. Therefore many industry-sponsored trials are coordinated by seemingly independent steering committees. However, this may not prevent the sponsors’ influence on reporting of results. Objectives: To investigate the sponsors’ influence on reporting of results in industry-sponsored trials. Methods: We included a sample of industry-sponsored trials published in The Lancet in 2008–2009 and corresponding trial protocols provided by the editors. For each protocol and published paper, we extracted information on coordination of publication, sponsors’ publication rights and involvement of medical writing assistance. We compared the information from protocols with the published papers. Results: We identified 169 papers of randomised trials and included 69 (41%) industry-sponsored trials. An additional 12 (7%) fully industry-funded trials with trial conduct apparently independent of the sponsor were analysed separately. In 24 out of the 69 trials (35%) the sponsor or a hired contract research organization were involved in coordinating writing of the manuscript, in 10 (14%) the sponsor was not involved and in 35 (51%) it was not described. In 64 trials (93%), the sponsor had influence over publication of the results through co-authorship or an explicitly stated right to approve or comment on the paper, and 37 trials (54%) had evidence of medical writing assistance supported by the sponsor. For the 12 apparently independently conducted trials 4 described the sponsors’ involvement in reporting of results, without this being stated in the published papers. Conclusions: The sponsors are usually involved in reporting Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. of results in industry-sponsored trials. Even for fully industry-funded trials that appear to be conducted independently, the sponsors are also sometimes explicitly involved in the reporting of results. P2B221 Critical appraisal checklist for systematic reviews of clinical prediction models Moons KG1 , Bouwmeester W1 , Collins G2 , Mallet S2 , Altman D2 , Reitsma J1 1 Julius Center, UMC Utrecht, Netherlands; 2 Centre for Statistics in Medicine, Oxford, UK Background: The introduction of evidence based medicine resulted in a clear shift from implicit to explicit reasoning in medicine, including the appreciation of multivariable diagnostic and prognostic prediction models. This is reflected by a sharp increase in published clinical prediction models. Systematic reviews try to assess and summarize the evidence. There is yet no tool to appraise clinical prediction studies. Objective: To provide a comprehensive list of items that are relevant for systematically reviewing and critically appraising publications on clinical prediction models. Methods: To indentify the relevant items for critically appraising clinical prediction research, we studied existing reporting guidelines (including in other medical research areas), various quality assessment tools, systematic reviews of prediction research, and methodologic key publications. Subsequently, experts were consulted for additional items. Results: Items that are most important for systematically reviewing publications on clinical prediction research include study design, subject selection methods, assessment, definition, and coding of outcomes and candidate predictors, statistical power, statistical techniques used, the reporting and handling of missing values, predictor selection approaches, assessment of predictive performance and validation of the final model, and model presentation. Conclusions: This overview of items can assist systematic reviewers in the appraisal of clinical prediction studies. P2B222 Comparison of adverse effects data derived from different study designs Golder S1 , Loke Y1 , Bland M2 1 Adverse Effects Methods Group, UK; 2 Department of Health Sciences, University of York, UK Background: There is considerable debate as to the relative merits of using randomised controlled trial (RCT) data compared to data from observational studies in systematic reviews of adverse effects. Objectives: This meta-analysis of meta-analyses aimed to assess the level of agreement or disagreement in the estimates of harm derived from meta-analysis of different study designs. Methods: Searches were carried out in 10 databases in addition to reference checking; contacting experts; citation searching; and handsearching key journals, conference proceedings and websites. Studies were included where a pooled relative measure of an adverse effect (odds ratio or risk ratio) from one study design could be directly compared, using the ratio of odds ratios (RORs), with the pooled estimate for the same adverse effect arising from another study design. Results: Thirty-nine Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 128 studies, yielding 111 meta-analyses were identified for inclusion. The pooled ratio of odds ratios of RCTs compared to observational studies was estimated to be 1.03 (95% CI 0.93–1.15) and there was less discrepancy with larger studies. Other meta-analysis of meta-analyses of different types of observational studies (such as cohort studies and case-control studies) also indicated no significant difference in estimates of adverse effects derived from different study designs. In almost all instances, the estimates of harm from meta-analyses of the different study designs had 95% confidence intervals that overlapped. In terms of statistical significance, in nearly two-thirds of the meta-analyses the results were in agreement (both studies showing a significant increase or significant decrease or no significant difference). In only two meta-analyses was there opposing statistical significance. Conclusions: Empirical evidence from this overview indicates that, on average, there is no difference in the risk estimate of adverse effects derived from meta-analyses of different study designs. This suggests that systematic reviews of adverse effects need not be restricted to specific study designs. P2B223 How do systematic reviews of insulin and drug therapies for diabetes deal with heterogeneity of results? Sun X1 , Troung J2 , Dattani N3 , Wong E2 , Kapend P2 , Bianca H4 , Sherifali D2 , Wang L5 , Nereberg K2 , Guyatt G6 1 Center for Health Research Kaiser Permanente Northwest, USA; 2 McMaster University, Canada; 3 University of Toronto, Canada; 4 Copenhagen University Hospital, Denmark; 5 Chinese Evidence-based Medicine Center, Sichuan University, China; 6 Department of Clinical Epidemiology and Biostatistics, McMaster University, Canada Background: The problem of how to deal with heterogeneity of results is ubiquitous in systematic reviews. Appropriate approaches include the use of a small number of pre-specified hypotheses, and the interaction test. Objective: To determine the rigor with which recent systematic reviews of therapy for diabetes explore heterogeneity. Methods: We searched Medline, EMBASE, and Cochrane Database of Systematic Reviews for systematic reviews, published in any language from January 2008 to May 2010, that, first, assessed effects of insulin and anti-diabetes drug therapies, both in single use or combination, for any type of diabetes mellitus; and, second, reported a meta-analysis of two or more randomized controlled trials (RCTs). Two methods trained reviewers, independently and in duplicate, conducted study screening and data extraction. Results: Of 46 reviews that proved eligible, 29 (59%) explored heterogeneity by subgroup analyses (n = 27, 59%) and/or meta-regression (n = 5, 11%), and 21 out of 29 reviews (72%) reported heterogeneity p < 0.1 or I-square ≥ 50%. Of the 29 reviews exploring heterogeneity, 6 (20%) clearly pre-specified heterogeneity variables, and 9 (31%) clearly used the interaction test. Explorations of heterogeneity tests a median of 5 (Inter-quartile: 2-6) variables. Variables explored included patient characteristics (17 of 29, 59%), intervention characteristics (24 of 29, 83%), and applied risk of bias and study characteristics (e.g. duration of follow up) (11 of 29, 38%). Many of these characteristics were also used as factors for sensitivity analyses (data not shown). Two of 5 studies including meta-regression, each having 14 and 22 RCTs, also undertook multivariable analyses of 4 independent variables. Conclusions: Exploration of heterogeneity was poorly performed in systematic reviews of insulin and drug therapies for diabetes. In particular, systematic reviewers often used patient baseline Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. characteristics for exploring heterogeneity, which may be problematic. Systematic reviewers also did not well differentiate subgroup analyses and sensitivity analyses. P2B224 Sponsorship of drug and device studies and research outcome Lundh A1 , Lexchin J2 , Sismondo S3 , Busuioc OA3 , Bero L4 1 The Nordic Cochrane Centre, Denmark; 2 School of Health Policy and Management, York University, Canada; 3 Department of Philosophy, Queen’s University, Canada; 4 Institute for Health Policy Studies & Department of Clinical Pharmacy, University of California-San Francisco, USA Background: Previous systematic reviews have documented that funding of drug studies by drug companies is associated with findings favourable towards the sponsor’s product. However, meta-analyses of drug studies frequently fail to disclose the underlying funding of included studies or to investigate whether the source of funding influences results. To determine if funding bias should be taken into account in meta-analyses, it is important to know its extent. This study updates previous reviews showing the association of funding and research outcome to determine if the association persists and additionally investigates device studies. Objectives: To investigate whether funding of drug or device studies by drug or device companies is associated with results and conclusions favourable to the sponsor. To investigate whether studies funded by drug or device companies differ in their risk of bias compared with studies with other sources of funding. Methods: We searched major bibliographic databases, reference lists, previous systematic reviews and author files. Two reviewers included studies and final inclusion was by consensus of all authors. Two reviewers extracted data from included papers and assessed studies for risk of bias. Authors were contacted in order to obtain missing data. Data was analysed in RevMan. Results: We identified 2,579 studies in the search, 72 of which appeared to meet our inclusion criteria and were retained for final assessment by all authors. Sixty-three were published after the search period of our previous systematic review. Additional results will be presented at the Colloquium. Conclusions: It appears that the evidence of funding bias has grown, but our conclusion awaits results from data analysis. P2B225 Adding unpublished Food and Drug Administration (FDA) data changes the results of meta-analyses Hart BL1 , Lundh A2 , Bero LA1 1 University of California, San Francisco; US Cochrane Center, San Francisco; 2 Nordic Cochrane Center, Denmark Background: Studies examining selective reporting of drug trial data show that positive results are more likely to be published than null or negative results. Consequently, meta-analyses based only on published data may over-estimate drug efficacy and under-estimate drug harms. Food and Drug Administration (FDA) reviews of drug trials are publicly available, making them a good source of unpublished trial data for inclusion into meta-analyses. Objective: This study examines the effect of including unpublished FDA trial data on the results of published meta-analyses across a variety of drug classes. Methods: We studied Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 129 24 drugs approved by the FDA between 2001–2002 with previously identified unpublished outcome data. We performed a systematic search of PubMed, Embase, and the Cochrane library in November 2010 to identify relevant meta-analyses for each of the study drugs. Two authors independently screened meta-analyses and selected one for analysis when multiple meta-analyses for a drug were included. Two authors independently extracted data from meta-analyses and the FDA reviews. We calculated summary statistics both with and without unpublished trial data. Results: After excluding 11 drugs in which no relevant meta-analyses were identified, 13 drugs and 42 meta-analyses met our inclusion criteria. Each drug had 1 to 8 corresponding meta-analyses (median = 2). To date, we have completed analyses for 4 drugs and recalculated summary statistics for 15 outcomes. For 11 outcomes, the change in summary statistics after inclusion of unpublished data demonstrated less benefit or more harm of the drug (range in percent change of summary statistics: 4% – 41%). Four outcomes showed more benefit of the drug after inclusion of unpublished data (range in percent change of summary statistics: 4% to 9%). Discussion: This preliminary analysis suggests that the inclusion of unpublished trial data in meta-analyses affects their results. The magnitude and direction of the effect varies between drugs. P2B226 Overestimation of treatment benefits in single-centre osteoarthritis trials may be related to sample size: Meta-epidemiological study Nüesch E1 , Trelle S1 , Reichenbach S1 , Jüni P1 1 University of Bern, Switzerland Background: Single-centre trials might differ from multicentre trials in characteristics of included patients, implemented interventions, methodological quality and sample size, which may result in different estimated treatment benefits. Objectives: We examined whether estimated treatment benefits differ between single-centre and multicentre osteoarthritis trials and whether this can be explained by components of methodological quality and sample size. Methods: We performed a meta-epidemiological study of 13 meta-analyses with 154 trials that compared therapeutic interventions with placebo or non-intervention control in patients with hip or knee osteoarthritis. We calculated standardised mean differences (SMDs) from the differences in means of pain intensity between groups at the end of follow-up divided by the pooled SD and compared SMDs between trials with and without multiple centres. We used stratification by components of methodological quality and sample size to derive differences between single and multicenter trials adjusted for these characteristics. Results: The 80 trials with multiple centres did not differ clearly from the 74 trials without multiple centres in their methodological quality, but randomised more patients (P = 0.027). On average, multicentre trials showed less beneficial effects compared to single-centre trials (difference in SMDs −0.20, 95% CI −0.38 to −0.02, P = 0.028) with small to moderate heterogeneity between meta-analyses (tau2 = 0.05, P for heterogeneity 0.002). Differences in treatment benefits were robust when analyses were adjusted for adequate generation of random sequence (−0.17, 95% CI −0.36 to 0.02, P = 0.07), adequate allocation concealment (−0.17, −0.33 to −0.01, P = 0.034) and blinding of patients (−0.19, −0.34 to −0.02, P = 0.029); were attenuated when adjusted for intention-to-treat analyses (−0.13, Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. −0.27 to 0.01, P = 0.08); and disappeared when adjusted for sample size (−0.07, 95% CI −0.27 to 0.14, P = 0.51). Conclusions: There is an overestimation of treatment benefits in single-centre osteoarthritis trials. This overestimation is not evident after adjustment for sample size. P2B227 Assessing the quality of non-randomised comparative studies: Our experience of using the Cochrane Collaboration’s risk of bias tool Robertson C1 , Ramsay C1 , Gurung T1 , Mowatt G1 , Pickard R2 , Sharma P1 1 Health Services Research Unit, University of Aberdeen, UK; 2 Institutue of Cellular Medicine, Newcastle University, UK Background: We describe our experience of using a modified version of the risk of bias (RoB) tool for non-randomised comparative studies. Objectives: – To assess inter-rater agreement for RoB assessment; – To assess time to complete RoB assessment; – To explore the association between RoB and treatment effect size. Methods: Our sample comprised full text primary reports included in a systematic review comparing laparoscopic versus robotic prostatectomy for localised prostate cancer (n = 49). We excluded non-English language publications and conference abstracts. Three teams of two reviewers assessed individual categories as high, low or unclear RoB. Joint agreement was then reached for overall RoB. We explored differences in treatment effect size according to RoB for the outcome positive surgical margin (n = 9). Inter-rater agreement was assessed using the Kappa statistic. We weighted the analysis to reflect higher disagreement between high and low risk compared to high/low and unclear judgements. We timed individual RoB assessment and time to reach joint agreement. Results: Twenty five studies were judged as having high overall RoB, 13 were judged as low RoB and 11 were judged unclear. Standard and weighted Kappa values for inter-rater agreement were 0.34 and 0.35 respectively. The median (range) time for individual assessment was 30 minutes (10 to 49 minutes). The median time for reaching agreement between reviewers was 10 minutes (2 to 38 minutes). The effect estimate for all studies was 0.61 (95% CI 0.46 to 0.83). The effect estimate for low risk studies was 0.73 (95% CI 0.29 to 1.75). Conclusions: We achieved fair agreement between reviewers for RoB assessment of non-randomised studies. Although the process was time consuming, using a modified version of the Cochrane (RoB) tool proved useful for demonstrating conservative effect estimates in our systematic review. We suggest including more RoB levels and further validation could improve inter-rater agreement. P2B228 A meta-epidemiological approach for evaluating bias and small study effects in networks of interventions Chaimani A1 , Schmid C2 , Vasiliadis H3 , Salanti G1 1 Department of Hygiene and Epidemiology, School of Medicine, University of Ioannina, Greece; 2 Sackler School of Graduate Biomedical Sciences, Tufts University School of Medicine, Boston, MA, US; 3 Molecular Cell Biology and Regenerative Medicine, Sahlgrenska Academy, University of Gothenburg, Sweden Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 130 Background: Investigation of the impact of study-specific biases typically requires a large number of studies. Meta-epidemiological approaches consider collections of independent pairwise meta-analyses, where the impact of bias in each one of them might be different, depending on the clinical area. Network meta-analyses include pairwise meta-analyses assumed ‘exchangeable’ in everything but the comparison being made. Hence biases can be studied simultaneously by borrowing strength across comparisons even when each pairwise meta-analysis comprises a small number of trials. Objectives: To suggest a meta-epidemiological technique for exploring study-specific bias sources and small-study effects using a collection of published networks of interventions. Methods: We searched for and collected all star-shaped networks of interventions published until March 2011. We applied various multiple-treatment meta-regression models that used as covariate a) the probability that a trial is at risk of bias in terms of adequacy of sequence generation, allocation concealment and blinding b) a measure of precision or variance to explore small study effects. We assume exchangeable bias parameters across comparisons within each network and across networks for similar outcomes. The fit of the different models and the amount of heterogeneity explained were compared. An average effect per bias source was estimated. Results: The adjustment for small study effects or bias items showed in some networks a slight improvement in the fit of the model and a respective reduction in heterogeneity. Borrowing strength across networks showed that a) small study effects exhibit an important amount of heterogeneity across networks b) bias sources overestimate the effectiveness of the interventions compared to common comparator, in agreement with previous findings. Conclusions: This meta-epidemiological approach enables investigation of bias and heterogeneity while adjusting for differences in effectiveness between treatments. As networks of interventions become popular, large-scale investigation of bias should consider them as source of evidence. P2B229 First results of a Systematic Review of Meta-analyses on the Efficacy of SSRI and SNRI in the Treatment of Major Depression Koesters M1 , Zhang Y1 , Weinmann S2 , Cipriani A3 , Barbui C3 , Becker T1 1 Department of Psychiatry II, Ulm University, Germany; 2 Deutsche Gesellschaft fuer Internationale Zusammenarbeit, Germany; 3 Department of Public Health and Community Medicine, Section of Psychiatry and Clinical Psychology, University of Verona Background: Systematic reviews and meta-analyses are the gold standard to assess efficacy and acceptability of treatments. In the field of antidepressant trials, hundreds of studies are available and many systematic reviews have been carried out so far, sometimes with conflicting results. It is known that the quality of the reviews may play an important role. However, this issue has not been yet systematically investigated. Objectives: To examine the quality of meta-analyses of serotonin reuptake inhibitor treatment of depression and to shed light on the influence of methodological aspects on the results of these meta-analyses. Methods: A comprehensive literature search for systematic reviews including meta-analysis of RCTs on SSRI and SNRI treatment in major depression was conducted. Studies Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. comparing SSRI and SNRI to placebo or all kinds of pharmacological antidepressant treatments were included. Results were screened for inclusion or exclusion by two independent investigators. Relevant data was extracted from the included studies, and quality assessment of systematic reviews was performed. Influence of a variety of factors on the results of reviews was analysed by regression analyses and t-tests. Results: The analyses included 134 reviews. Depending on the rating scale, the average quality of included studies was moderate or good, with a significant increase of study quality over the past 20 years. Blinding of reviews regarding authors and results had no impact on quality ratings. The results pointed out that there is room for improvement of current quality measurements for systematic reviews. Only involvement of drug companies in the publication of meta-analyses showed an impact on the effects of the review. All other factors tested failed to reveal statistically significant relationships with the results of the review. P2B230 What can we learn from Chinese randomized controlled trials? A systematic review of Chinese venlafaxine studies Koesters M1 , Zhang Y1 , Ma YC2 , Weinmann S3 , Becker T1 , Jin WD2 1 Department of Psychiatry II, Ulm University, Germany; 2 Tongde Hospital, Zhejiang Province, China; 3 Deutsche Gesellschaft fuer Internationale Zusammenarbeit, Germany Background: Despite an impressive increase of Chinese studies the interest in these studies is still relatively low in Western countries, and Chinese randomised controlled trials (RCTs) are rarely included in Western systematic reviews. Objectives: We systematically reviewed blinded Chinese venlafaxine RCTs, based on a search in Chinese and western databases. This study aims at enhancing insight into the quality of Chinese RCTs and to investigate if venlafaxine is an effective treatment option in Chinese populations. Methods: Chinese databases (CNKI/VIP) and western databases were searched for blinded randomized controlled trials. Trials comparing venlafaxine to other antidepressants or placebo were included if the patients had a diagnosis of depression according to CCMD, DSM or ICD. Effect sizes were calculated as Hedges’ g for rating scale scores and Mantel-Haenszel risk ratios (MH RR) for response and remission data. Effect sizes were combined in a fixed-effects model. Results: Twenty-five studies were included. Nine trials compared venlafaxine to SSRIs; placebo-controlled trials were lacking. Quality was at best modest. All trials were underpowered. There were more responders (MH RR, 1.08; 95% confidence interval [CI], 1.02–1.15) and remitters (MH RR, 1.12; 95% CI, 1.02–1.24) in venlafaxine groups compared with those in TCA groups. HAMD end point scores in the venlafaxine groups were lower (Hedges’ g = 0.16; 95% CI, 0.04–0.27), and venlafaxine was better tolerated than tricyclic antidepressant (Hedges’ g = 0.56; 95% CI, 0.37–0.74). There were no significant differences between venlafaxine and selective serotonin reuptake inhibitor on any of these parameters. Analyses of publication bias were inconclusive. Conclusions: Chinese researchers have published a number of randomized controlled trials comparing venlafaxine to active comparators, but study quality was found to be low. To make optimal use of their research potential Chinese researchers will have to improve trial reporting and the peer-review process. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 131 P2B231 Industrial funding of research is associated with practice of survival analysis Vlassov V1 , Gemjian E2 1 First Moscow State Medical University, Russia; 2 National Haemathology Center, Moscow, Russia Background: Industrial funding of the biomedical studies is associated with practices which increase the possibility of bias, including subgroup and per protocol analyses, and testing of multiple hypotheses or multiple variables. Objective: To find out if industrial funding associated with the use of survival analysis (SA) to estimate the cause-specific survival (CSS). Methods: We searched MEDLINE by ‘survival analysis’. CSS studies were found by ‘specific survival’. Only human-related studies were selected. To evaluate the source of financial support, the full texts of the articles were checked. The primary analysis was limited to year 2009. Results: In 2009,12787 reports were classified as using SA, and only 423 (3%) employed analysis of CSS. All 104 articles using CSS and available in full text were evaluated, as well as 104 of the first articles of 3813 using SA but not CSS and available in full text. Evaluation of the sources of funding were done by two authors independently with 30% overlap to estimate the agreement (K = 0.67). Of 104 studies using CSS, 16 (15%) were funded by industry, while of 104 studies using SA without estimates of CSS, 7 (6.7%) reported industrial funding. The 95% confidence interval for the difference is 0.001 to 0.175. Conclusions: The possibility of finding significant differences in specific-survival when total survival is not different between groups may be an attractive perspective. This possibility is used by scientists ‘torturing’ the data in search of statistical significance. Authors of systematic reviews need to know about this possible source of bias in estimates of survival. The full presentation of this study will include the data on use of specfic-survival in clinical trials. P2B232 True methodological quality of trials are not reflected in their reporting Mhaskar R1 , Magazin A2 , Soares H2 , Kumar A1 , Djulbegovic B1 1 University of South Florida, Center for Evidence based Medicine, USA; 2 Moffitt Cancer Center, USA Background: Biased results from poorly designed and reported RCTs can mislead decision making. Whether publications accurately reflect the actual methodological quality of randomized controlled trials (RCTs) has not been comprehensively evaluated. Additionally, it has been hypothesized that trial sample size is associated with better methodological quality. Objective: To compare the reported methodological quality of a RCT as reflected in publications with actual methodological quality as depicted in the protocols, and assess association of RCT sample size with published versus actual methodological quality. Methods: All consecutive published phase III RCTs conducted by 8 National Cancer Institute sponsored Cooperative Groups (NCICOG) until year 2006 were eligible for inclusion. Data on methodological quality domains relevant to minimizing bias and Figure 1 (P2B232). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 132 random error were extracted from protocols and publications for each study. Results: A total of 429 RCTs met the inclusion criteria. Overall reporting of elements of methodological quality domains relevant to minimizing bias (Figure 1A) and random error (Figure 1B) was poor and did not reflect the actual high methodological quality of RCTs. Similarly, the results showed a positive correlation between RCT sample size and reported methodological quality domains of adequacy of blinding procedures, ITT analysis and the choice of the comparator (p value < 0.001). However, this correlation disappeared when actual methodological quality was considered. Conclusion: This largest study to-date comparing published versus actual methodological quality of RCTs shows that poor quality of reporting by NCICOGs does not reflect the actual high methodological quality. Similarly, we found no association between RCT sample size and its actual methodological quality. The positive correlation between methodological quality of reporting and RCT sample size is misleading. Our findings underline the need for adherence to the CONSORT statement by authors and journal editors for transparent evaluation of RCTs and highlight the importance of publication of RCT protocols in the public domain. P2B233 Impact of reported vs. actual methodological quality assessment on treatment effect size Mhaskar R1 , Magazin A2 , Kumar A1 , Djulbegovic B1 1 University of South Florida, Center for Evidence based Medicine, USA; 2 Moffitt Cancer Center, USA Background: Discrepancy between reported methodological quality and the actual methodological quality of randomized controlled trials (RCTs) has been documented. However, the relative impact of reported versus actual methodological quality on the treatment effect size (ES) has not been assessed. Objective: To assess the impact of reported methodological quality versus actual methodological quality on ES. Methods: All consecutive terminated phase III RCTs published by 8 National Cancer Institute sponsored Cooperative Groups until year 2006 were eligible for inclusion. Data on methodological quality domains relevant to minimizing bias and random error were extracted from protocols and publications for each study. ‘Actual quality’ was assessed based on data from either protocols or the publications. The hazard ratio (HR) for overall survival was used as the ES. Association between methodological quality (reported vs. actual) with ES was conducted using standard meta-epidemiologic methods. Results: A total of 429 RCTs met the inclusion criteria. There was no statistically significant difference between associations of ES and actual vs. methodological quality for the following domains: adequacy of randomization sequence generation, description of drop outs, intention to treat analysis, pre specification of alpha and beta errors (Figure 1). However, on average, poorly reported allocation concealment exaggerated the ES by 6% (Ratio of HRs: 0.94, 95% CI: 0.88–0.99). Also, poorly reported blinding inflated the ES by 24% (Ratio of HRs: 1.24, 95% CI: 1.05–1.43). Nonetheless, in the ‘actual quality’ assessment, no significant association between ES and any of the methodological quality domains was detected. Conclusion: Our study results show that the assessment of the impact of quality on the ES based on the quality of reporting only can produce misleading results. These findings are important for users of research evidence, Figure 1 (P2B233). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 133 systematic reviewers, meta-epidemiologic research, as well as further development of the Cochrane ‘risk of bias’ table which relies on reported methodological quality. P2B234 Observer bias in randomised clinical trials with continuous outcomes: An analysis of trials with both blind and non-blind outcome assessors A1 , ASS1 , F2 , B2 , J3 , Hróbjartsson Thomsen Emanuelsson Thomsen Hilden Ravaud P4 , Boutron I4 , Brorson S5 1 The Nordic Cpochrane Centre, Denmark; 2 The Nordic Cochrane Centre, Denmark; 3 University of Copenhagen, Denmark; 4 The French Cochrane Centre, France; 5 Herlev University Hosptal, Denmark Background: The typical degree of observer bias in randomised clinical trials with non-blind outcome assessors of continuous outcomes is not known. Previous analyses which normally involve comparisons between trials with blind assessors and ‘similar’ trials with non-blind assessors, are susceptible to confounding. In an earlier work, based on trials that use both blind and non-blind outcome assessors of the same binary outcome, we found that substantial observer bias could result from misclassification of few patients. Trials with continuous outcomes may have a different susceptibility to observer bias, and an estimate of the typical degree of bias in such trials is important for the assessment of risk of bias. Objectives: To estimate the degree of observer bias in randomised clinical trials with continuous outcomes, and to identify factors associated with large observer bias. Methods: A systematic review of randomised clinical trials with both blind and non-blind assessment of the same continuous outcome. We searched PubMed, EMBASE, PsycINFO, CINAHL, The Cochrane Trials Register; HighWire Press and Google Scholar. We plan to extract data from one outcome per trial, and to calculate the difference between the standardized mean difference (SMD) based on blind assessments and the SMD based on non-blind assessments: dSMD. We plan to pool the dSMDs with inverse variance meta-analysis using random effects models. Furthermore, we will study whether the size of dSMD is associated to a) high degree of outcome subjectivity, e.g. global clinical improvement; b) close trial involvement of the non-blind assessors (e.g. they also treat patients); and c) high degree of sensitivity to reporting and behavior of non-blind patients (e.g. symptom interviews). Results: At the time of abstract submission we had identified 23 eligible trials. We expect full outcome data from 17 trials. Conclusions: Pending. P2B235 Pooled standardised mean differences and estimated heterogeneity between trials depend on the type of assessment tool analysed: Meta-epidemiological study da Costa BR1 , Rutjes AWS1 , Johnston BC2 , Nüesch E1 , Reichenbach S1 , Guyatt G2 , Jüni P1 1 Division of Clinical Epidemiology & Biostatistics, ISPM, University of Bern, Switzerland; 2 Department of Clinical Epidemiology and Biostatistics, McMaster University, Canada Background: Investigators performing systematic reviews and meta-analyses are typically faced with data derived from different assessment tools to measure a common concept such as pain. A common method is to combine data regardless of instrument using standardised mean differences (SMD). However, it is unclear whether the estimated SMD and statistical heterogeneity will depend on the choice of instrument used. Objectives: To determine whether the two most frequently used instruments to assess knee or hip pain in osteoarthritis trials, the WOMAC Pain subscale (WOMAC) and Pain Overall measured on a visual analogue scale (VAS), result in different pooled estimates of treatment effects and statistical heterogeneity between trials. Methods: Meta-epidemiological study of meta-analyses of large-scale osteoarthritis trials comparing active treatment with placebo, sham, or non-intervention, reporting at least one follow-up for both, WOMAC and VAS, with at least 100 patients per group. We pooled SMDs for each instrument using an inverse-variance random-effects model within each meta-analysis and subsequently combined pooled estimates of treatment effect and statistical heterogeneity across meta-analyses using a random-effects model. Estimates were compared between instruments using a paired sign-rank test. Negative SMDs indicated a benefit of experimental interventions as compared with control. Results: We identified 5 meta-analyses including 24 large-scale trials with 13400 patients that reported on both WOMAC and VAS. The average SMD was −0.22 (95% CI −0.34 to −0.09) according to WOMAC and −0.32 (95% CI −0.46 to −0.18) according to VAS (p for difference = 0.038, see Table 1). Corresponding I2 estimates were 53% (95% CI 0% to Table 1 (P2B235). Standardized mean differences and I2 values presented per meta-analysis and pooled across meta-analyses for each outcome. WOMAC Intervention NSAIDS Food Supplements Opioids Viscosupplementation Acupuncture Overall VAS SMD I2 SMD I2 Total number of patients −0.31 −0.26 −0.35 −0.07 −0.01 −0.22 17% 89% 44% 34% 0% 53% −0.42 −0.50 −0.34 −0.03 −0.12 −0.32 80% 96% 71% 78% 83% 85% 9453 1271 1495 828 353 13400 WOMAC: WOMAC Pain subscale; VAS: Pain Overall measured on a visual analogue scale; SMD: standardized mean difference; NSAIDS: non-steroid anti-inflammatory drugs. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 134 80.7%) and 85% (95% CI 66.6% to 93.4%), respectively (p = 0.043). Conclusions: Differences in estimated SMDs between WOMAC and VAS were small on average, but WOMAC resulted in considerably lower heterogeneity between trials. Therefore, WOMAC should be given precedence over VAS when extracting data for meta-analysis of osteoarthritis trials using SMDs. P2B236 Methodological adequacy of articles published in two open-access Brazilian cardiology periodicals Macedo CR1 , Silva DL1 , Melnik T1 , Rachel R1 , Pedrosa A1 , Torloni R1 , Silva EMK1 , Atallah AN1 , Puga ME1 1 Brazil Cochrane Center, Br Background: The use of rigorous scientific methods has contributed towards developing scientific articles of excellent methodological quality. This has made it possible to promote their citation and increase the impact factor. Brazilian periodicals have had to adapt to certain quality standards demanded by these indexing organizations, such as the content and the number of original articles published in each issue. Objectives: This study aimed to evaluate the methodological adequacy of two Brazilian periodicals within the field of cardiology that are indexed in several databases and freely accessible through the Scientific Electronic Library Online (SciELO), and which are now indexed by the Web of Science (Institute for Scientific Information, ISI). Methods: All the published articles were evaluated according to merit assessment (content) and form assessment (performance). Results: Ninety-six percent of the articles analyzed presented study designs that were adequate for answering the objectives. Conclusions: These two Brazilian periodicals within the field of cardiology published methodologically adequate articles, since they followed the quality standards. Thus, these periodicals can be considered both for consultation and as vehicles for publishing future articles. For further analyses, it is essential to apply other indicators of scientific activity such as bibliometrics, which evaluate quantitative aspects of the production, dissemination and use of information, and scientometrics, which is also concerned with the development of science policies, within which it is often superimposed on bibliometrics. P2B237 Are we evaluating methodological quality in the same way? Comparison of tools used in general health research and physical therapy with the ‘Risk of bias’ tool Armijo-Olivo S1 , Ospina M2 , Rowe B3 , Hartling L1 1 Alberta Research Center for Health Evidence, Department of Pediatrics, University of Alberta, Canada; 2 School of Public Health, University of Alberta, Canada; 3 Faculty of Medicine and Dentistry, University of Alberta, Canada Background: Assessing the methodological quality of randomized controlled trials (RCTs) is crucial to understand how bias may affect study results and to accurately identify treatment effects. A wide variety of tools are available to evaluate RCT quality; however, it is unknown how these tools compare in terms of the items included. Objectives: To describe the frequency of individual items across quality assessment tools and to compare tools used in physical therapy (PT) to those used in general health research. Methods: We identified tools used to evaluate the methodological quality of RCTs in general health research and PT following a comprehensive search and using a systematic approach. We extracted the items included in each tool. We calculated the frequency of each item used across tools and compared these among tools used in general health research and PT. Comparisons were also made with the Cochrane Risk of Bias (RofB) tool. Results: Twenty-one tools were identified: 15 used in general health research and seven in PT. Ninety-five different items were used across tools. The most frequent items appearing in 15 of the general tools (67%) were: description of randomization, withdraws and dropouts, inclusion and exclusion criteria, and appropriate statistical analysis. In contrast, the most frequent items appearing in six out of seven tools used for PT (85.7%) were: baseline comparability, blinding of investigator/assessor, and use of intention-to-treat analysis. Prominent features of the RoB tool such as sequence generation and allocation concealment were included in four (57%) and five (71%) of the PT tools, respectively. Ten (63%) and six (40%) of the general health research tools included randomization and allocation concealment, respectively. Conclusions: There is extensive variation in the items included across quality assessment tools. It is critical that methodological features associated with bias are captured in tools used for PT trials. P2B238 Methodological quality of malaria RCTs conducted in Africa Lutje V1 , Gerritsen A2 , Siegfried N3 1 Cochrane Infectious Diseases Group, Liverpool, UK; 2 Department of Public Health, University of Venda, Thohoyandou, South Africa; 3 South African Cochrane Centre, Medical Research Council, Cape Town, South Africa Background: Good methodological quality is necessary to reduce risk of bias in randomized controlled trials (RCTs) and in meta-analyses. As part of a review of clinical and methodological characteristics of malaria RCTs conducted in Africa, we assessed the methodological quality of trials conducted after the publication of the original CONSORT statement in 1996. This is a novel analysis that can highlight training needs for clinicians conducting trials in potentially resource-limited settings. Objectives: To analyse the methodological quality of malaria RCTs conducted in Africa between 1997 and 2007. Methods: We run systematic searches for malaria RCTs in electronic databases (Medline, Table 1 (P2B238). Assessment of trial quality for malaria RCTs published 1997–2007. N = 60 trials. Sequence generation Allocation concealment Blinding Loss to follow up Adequate Not adequate Unclear Not done Not possible 35 14 23 49 3 21 36 27 11 1 10 8 2 Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 135 Embase, the Cochrane Library), and applied an African geographic search filter to identify trials conducted in Africa. Results were exported to the statistical package STATA 8 to obtain a random sample from the overall data set. We evaluated 60 trial reports published between 1997 and 2007 for risk of bias according to 4 domains (randomized sequence generation, allocation concealment, blinding, and loss to follow-up). Results: Sequence generation was considered adequate (as done by using a random numbers table or electronically generated) in 35 reports, but was not clearly reported in 21 trials (Table 1). Many RCTs did not mention methods of allocation concealment or blinding of participants or intervention providers. In contrast, loss to follow up was accounted for in most RCTs (49 out of 60). Conclusions: The quality of malaria trials’ reports was not consistent among the 4 domains analysed: a large proportion of RCTs had a high risk of bias for blinding and allocation concealment, whereas loss to follow-up was mostly well reported. Similar suboptimal reporting has been widely reported for RCTs in different healthcare areas, potentially affecting the validity of trial results and the estimates of treatment effects, and is not associated with trials conducted in resource-poor settings. P2B239 A systematic review on the efficacy of statins in animal model Pecoraro V1 , Moja L2 , Dall’Olmo L3 , Cappellini G4 , Liberati A1 , Garattini S4 1 Italian Cochrane Centre, Italy; 2 University of Milan, Italy; 3 I.O.V.-Istituto Oncologico Veneto- I.R.C.C.S, Italy; 4 Mario Negri Institute for Pharmacological Research, Italy Background: Statins are recognised as an effective treatment for lowering cholesterol levels and reducing the risk of cardiovascular diseases. They have been widely studied in basic and human research and there are no systematic reviews of statins across animal models. Objectives: We explored differences in the efficacy of statins in three popular animal models: mice, rats and rabbits, using the technique of systematic review (i.e. Cochrane methods). We also explored whether the reporting was accurate, and the risk of bias. Methods: We searched MEDLINE and EMBASE. Two independent reviewers assessed the eligibility, extracted study details and biochemical blood parameters, blood pressure, myocardial infarction and survival. Weighted and standard mean difference random effects meta-analysis was used to measure overall efficacy in specified species, strains and subgroups. Results: We analyzed 161 studies. Most papers did not use randomization (55%) or blinding (88%) and their reporting presented several shortcomings. Statins lowered total cholesterol in all species though with large differences in the effect size: −30% in rabbits, −20% in mice and −10% in rats. Many meta-analyses suffered substantial heterogeneity despite the postulated limited biological variability in single species and strains. Few studies considered strains at high risk of cardiovascular diseases even though mimic human cardiovascular diseases better or examined hard outcomes. A risk of pubblication bias our data also suggested. About 45% of publications identified report no significant findings on total cholesterol. Conclusions: Although statins showed substantial efficacy in animal models, the preclinical data on conditions mimicking the human pathologies for which the drugs are clinically indicated and utilized are often scarce, incomplete and at risk of bias. This study raises a Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. number of concerns, particularly about the reliability and quality of animal experiments that may drive future clinical trials in humans. P2B240 Shifting effects in three randomised controlled trials on music therapy in mental health Gold C1 , Erkkilä J2 , Crawford M3 1 Uni Research, Bergen, Norway; 2 University of Jyväskylä, Finland; 3 Imperial College London, UK Background: Randomised controlled trials (RCTs) are based on the assumption that the effects of interventions are constant across all participants. This may not always be true in complex interventions which are known to depend on provider’s expertise and experience and may also be vulnerable to context variables (such as whether therapy is delivered within a trial). Music therapy, with music experiences and a therapeutic relationship as the elusive key components, is a complex intervention with an emerging evidence base. Objectives: To investigate time trends in effect over the recruitment period in music therapy RCTs, which might indicate an impact of delivering therapy within a trial. Methods: We included recent (2006-present), methodologically rigorous RCTs on music therapy versus standard care for people with mental disorders (schizophrenia or depression) that used a parallel single-blind design to assess changes in a continuous scale of mental health (symptoms of schizophrenia or depression). Music therapy had to be delivered by experienced therapists. Individual patient data were examined graphically and statistically. Results: Three trials were included. In all trials, the comparative effects of the interventions (music therapy versus standard care) were not constant, but shifted over time. One trial suggested a? reverse effect in the first ten participants than in later participants. The other trials suggested a smaller effect in those first participants than in later ones. Whereas the trials overall suggested superiority of music therapy, this was not the case for the first participants recruited. Conclusions: For some complex interventions, it cannot be assumed that effects remain constant over the recruitment period of a trial. Delivering therapy within a trial may be an unsettling experience for experienced therapists, especially at the beginning of a trial. If replicated, our findings will have implications for the analysis and reporting of RCTs on complex interventions. P2B241 The quality of randomized clinical trials in surgery can be improved. A systematic review Verhagen AP1 , v Middelkoop M1 , Rubinstein SM2 , Jacobs WC3 , Ostelo RW2 , Peul W3 , Koes BW1 , v Tulder MW2 1 ErasmusMC, Netherlands; 2 EMGO-institute, Netherlands; 3 LUMC, Netherlands Background: The field of spine surgery undergoes rapid introduction of new surgical techniques. The effectiveness or efficacy of these new techniques needs to be evaluated in rigorous randomized clinical trials to support their use. Objectives: To evaluate the methodological quality in surgery trials of the cervical and lumbar spine to enhance awareness and endorsement of bias avoidance and Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 136 reporting recommendations of surgery trials. Methods: A risk of bias assessment was conducted on all included studies in systematic reviews of surgical interventions in patients with neck and low back pain by two reviewers independently using the assessment form of the Cochrane Back Review Group. A study was defined as having a low risk of bias when fulfilling six or more of the items. A descriptive analysis was performed. Results: In total, 90 RCTs were identified in three systematic reviews comparing different surgical interventions with conventional treatment or other surgical interventions. Of these studies only a few trials had a low risk of bias, and most of them were published between 2007–2009. The items that scored positive in most studies were adequate randomisation, acceptable drop-out rate and timing of outcome assessment. The most common items that were unclear or scored negatively included the concealment of allocation, blinding of patients, care provider or outcome assessor and the implementation of intention-to-treat analysis. Conclusions: The quality of RCTs on surgical interventions in the cervical and lumbar spine file has increased over time, but is still poor and shows room for improvement. Improvement can be reached by paying more attention to the randomisation procedure (concealment), co-interventions, preventing withdrawals and the analysis (intention to treat). P2B242 Assessing risk of bias in non-randomised studies and incorporating GRADE: Initial experience with a new Cochrane ‘Risk of bias’ tool under development MacLennan S1 , Imamura M1 , Dahm P2 , Neuberger M2 , Reeves B3 , MacLennan G4 , Omar MI1 , McClinton S5 , Griffiths L6 , N’Dow J7 1 Academic Urology Unit, University of Aberdeen, Aberdeen, UK; 2 Department of Urology, College of Medicine, University of Florida, Gainesville, Florida, USA; 3 Faculty of Medicine and Dentistry, University of Bristol, UK; 4 Health Services Research Unit, University of Aberdeen, UK; 5 Urology Department, NHS Grampian, Aberdeen Royal Infirmary, Aberdeen, UK; 6 Department of Cancer Studies and Molecular Medicine, University of Leicester, Clinical Sciences Unit, Leicester General Hospital, Leicester, UK; 7 Urology Department, NHS Grampian, Aberdeen Royal Infirmary, and Academic Urology Unit, University of Aberdeen, Aberdeen, UK Background: In instances where randomised controlled trials (RCT) are impossible or have not been conducted, clinical recommendations and decision-making must rely on other evidence. If systematic reviewers decide to include non-randomised studies (NRS), it is imperative to use a standard method to assess and communicate the risk of bias (RoB) in NRS. Objectives: To pilot a RoB tool for NRS and make it commensurate with GRADE. Methods: An extended version of the Cochrane RCT RoB tool was applied to NRS. This included an additional item on the risk of findings of an NRS being explained by confounding. Each pre-specified confounding factor was assessed on the precision of measurement, baseline imbalance, and quality of case-mix adjustment, on 5-point scales. Imbalance was judged by clinical consensus, while other items were assessed by two independent reviewers. Mean ‘adjustment’ scores per outcome across studies were used to determine the quality of evidence according to GRADE. The tool was applied to 33 NRS retrieved for a systematic review of surgical interventions for localised renal cancer. Results: The initial 5-point scale was unwieldy and lead to disagreement among reviewers. We created scoring guidelines and re-piloted. RoB scores were tabulated rather than aggregated to indicate where likely biases were located. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. All NRS were rated as either ‘low’ or ‘very low’ on GRADE; however, determining an appropriate cut-off required considerable judgement. Conclusions: Compared with RoB assessment in RCT, assessment of NRS was more difficult and increased required time and expertise resources. In areas where the quality of studies is known to be very low, the added time and complexity may make the assessment not worthwhile. Presentation of the large amount of information generated by this tool is challenging. Further research needs to strike a balance of making a ‘brief’ and ‘easy’ version while addressing complex methodological issues inherent in NRS. P2B243 Conflicting guidelines for screening mammography: Influence of author’s specialty and conflicts of interest Burda B1 , Holmer H2 , Ogden L2 , Fu R2 , Norris S2 1 Oregon Evidence-based Practice Center, Kaiser Permanente Center for Health Research, USA; 2 Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, USA Background: Financial and intellectual conflict of interest (COI) may explain conflicting recommendations in clinical practice guidelines (CPGs) for screening mammography. Objectives: To assess the financial and intellectual COI of the authors of CPGs and to examine the relationship of COI to recommendations for routine screening mammography in asymptomatic women 40 to 49 years of age who are at average risk of breast cancer. Methods: We searched the National Guideline Clearinghouse and MEDLINE for relevant CPGs published from 2005 to 2010. We documented the disclosures and specialties of the lead and secondary authors of these CPGs, and the publications of the lead authors, and examined their relationship to CPG recommendations. Results: Twelve CPGs were identified with 13 lead authors and 178 physician authors of various specialties. None of the four CPGs recommending non-routine screening had a radiologist member, while five of the eight CPGs recommending routine screening had at least one (P = 0.05). There was a trend that a CPG with a radiologist member was more likely to recommend routine screening, although the association was not significant (odds ratio [OR] 6.05, P = 0.14). The proportion of primary care physicians on CPGs recommending routine versus non-routine screening was significantly different (P = 0.01). The odds of a recommendation for routine screening were related to the number of recent publications on breast disease by the CPG lead author (OR 2.32 for each additional publication, P = 0.02). Conclusions: Recommendations for or against routine mammography screening in the target population may reflect the specialty and intellectual interests of the CPG authors. Our conclusions are limited by the observational nature of the data and by the small number of included CPGs. P2B244 Assessment of trial risk of bias among Cochrane reviews: A cross-sectional analysis Abou-Setta A1 , Dryden D1 , Hamm M1 , Moher D2 , Klassen T3 , Hartling L1 1 Alberta Research Centre for Health Evidence, Canada; 2 Ottawa Hospital Research Institute, Canada; 3 Manitoba Institute of Child Health, Canada Background: Risk of bias (RoB) of trials may affect the conclusions of systematic reviews. Since 2008, Cochrane reviews are expected to Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 137 include a detailed assessment of bias for included trials using the RoB tool. It is not clear how stringently this is followed, or how much potential bias is present across Cochrane reviews. Objectives: To assess the RoB among included trials in Cochrane reviews. Methods: We searched the Cochrane Database of Systematic Reviews (CDSR) for all active, completed reviews. Using a customized automated algorithm (Visual Basic for Applications) we extracted RoB assessments of included trials. An ‘unclear’ risk was imputed for missing assessments except where RoB was not assessed at all. Results: In April 2011, the CDSR contained 4,594 reviews. We excluded withdrawn reviews (n = 204) and overviews of reviews (n = 3). Further, 3,013 (66%) reviews were excluded for lack of data [RoB assessment not performed (n = 2,599); no trials included (n = 411); other quality assessment tool used (n = 3)]. In total, 1,374 reviews encompassing 17,220 trials were included in our analyses. Of these, 39% of trials had RoB judgments reported for all domains; remainder had variable missing assessments. Overall 3%, 53%, and 44% of trials were considered to be ‘low’, ‘unclear’, and ‘high’ risk, respectively. The rationale for the judgments was provided for 88% of the assessments. The number of trials classified as ‘high’ has significantly decreased over time, while trials classified as ‘low’ or ‘unclear’ risk have gradually increased (p < 0.0001). The domain most often assessed as ‘low’ risk was ‘incomplete outcome data’; ‘blinding’ was most often assessed as ‘high’ risk. Conclusions: RoB assessments for most trials included in Cochrane reviews are ‘unclear’, likely due to poor reporting at the trial level. Further, 97% of trials were assessed as ‘unclear’ or ‘high’ risk which raises questions regarding the sensitivity/ specificity of the RoB tool. P2B245 Does stratification analysis indicate the confounding factors in meta-analysis of observational studies? Naing C1 , Mak JW1 1 International Medical University, Kuala Lumpur, Malaysia Background: It has been suggested that any systematic review including meta-analysis of observational studies must consider the risk of bias in individual primary studies. To date, checklists for the reporting of observational epidemiological studies have been developed, but their usefulness is limited. Moreover, the design features of primary studies rather than a design label is important because the risk of bias is affected by the specific features of a study, not by a broad categorization of the approach taken. As such, listing potential confounding factors in the primary studies and stratification of participants into subgroups with respect to potential confounding factors is straightforward and valuable. Objectives: To analyse the effect estimates by stratification of the participants according to the potential confounding factors. Methods: As an illustration, we performed a meta-analysis of observational studies identifying the association between hepatitis C virus infection (HCV) and type 2 diabetes (T2D), following the Cochrane guideline for the non-randomised studies. We stratified the participants according to known risk factors age group, gender, and family history of diabetes, and performed subgroup analysis. Results: We identified observational studies addressing the association between HCV and T2D (k = 21, n = 294437). A positive and significant association between HCV and T2D was observed (odds ratio 1.71; 95% CI = 1.17–2.49). Patients of age >40 years (7.47; 95% CI = 5.93–9.42), obesity (0.32: Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. 95% CI = 5.93–9.42) and female sex (0.8: 95% CI = 0.66–0.97) were also significant factors. Conclusions: Observational epidemiological studies are generally poorly reported. The evaluation of methodological quality and risk of bias consistently across primary studies is difficult. The stratification by potential confounding factors in the interpretation of the findings of meta-analysis of observational studies could be valuable. P2B246 Unplanned crossover in randomised controlled trials: Consequences for efficacy and safety outcomes D’Amico R1 , Bonafede E1 , Balduzzi S1 , Longo G1 , Guarneri V1 , Piacentini F1 , Moja L2 , Liberati A1 1 University of Modena and Reggio-Emilia, Italy; 2 University of Milan, Italy Background: If a randomized controlled trial (RCT) shows results in favour of an intervention, investigators might surmise that the equipoise principle is no longer present. As a consequence, control patients may be offered to cross over to the experimental arm, even if it was unplanned. Objectives: We assessed: a) the prevalence of the unplanned crossover in oncology trials; b) how it is dealt when analysing data; and c) the potential magnitude and direction of bias. Methods: RCTs published between 2000 and 2011 in the NEJM, JNCI, Lancet, Lancet Oncology, JCO, Annals of Oncology, assessing the efficacy and safety of treatments in oncologic patients were searched. We explored the relationship between the effect and: a) the time when the crossover occurred; b) the fraction of patients who decided to cross over; and c) their clinical characteristics. The magnitude and direction of bias were investigated through simulations. Results: Out of 164 papers analyzed so far, in 11 RCTs investigators gave patients the opportunity to crossover to the experimental arm. The phenomenon is more frequent in the second half of the decade. All studies compared outcomes by using ITT and 20% used an additional censored analyses. Simulations suggest that ITT analyses dilute the treatment effect size for efficacy and safety, whereas the magnitude and direction of the bias caused by the censored analysis seem to be less predictable. The inverse probability of censoring weighted model was also used, but it is highly speculative since it is dependant on the characteristics of those patients who decided to cross over. Conclusions: While investigators justify the unplanned crossover of patients on ethical grounds, it can leave the scientific community with increased uncertainty of study results, and may be a condition to inflate the net benefit and risk of an intervention of clinical relevance. P2B247 Applicability of R-AMSTAR instrument to appraise systematic reviews Martı́nez-Zapata MJ1,2,3,4 , Rigau D1,2 , Selva A2 , Gich I2,3,4 , Bonfill X1,2,3,4 1 Iberoamerican Cochrane Centre, Barcelona, Spain; 2 Biomedical Research Institute Sant Pau (IIB Sant Pau), Barcelona, Spain; 3 CIBER of Epidemiology and Public Health (CIBERESP), Barcelona, Spain; 4 Universitat Autònoma de Barcelona, Spain Background: The science of research synthesis aims to systematically gather, examine and evaluate primary study reports that answer a clearly formulated question. However not all systematic reviews (SRs) reach the goal to minimize bias and some are furnished of Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 138 methodological flaws. The revised and validated AMSTAR (R-AMSTAR) instrument has been generated to quantify the SRs quality. Objectives: To describe the applicability and drawbacks of R-AMSTAR instrument. Methods: We did and overview on the effectiveness of acupuncture in obstetric-gynaecological disorders. A comprehensive literature search was performed in CENTRAL, CDSR, DARE, MEDLINE databases up to July 2010 and with no language restriction. Two reviewers independently performed the literature selection and appraised the retrieved SRs using the R-AMSTAR instrument. This instrument contains 11 questions (each with 3 or 5 items) rated from 1 to 4 (total score range from 11 to 44). Although there were not a previous consensus between reviewers, they were experienced in the critical appraisal of SRs. Each item and overall agreement was tested using the kappa coefficient. The intraclass correlation coefficient (ICC) was used to test the agreement of R-AMSTAR sum scores. Results: We identified 728 references and included 8 SRs. The median score obtained by R-AMSTAR tool was 32 (range 21–41). Kappa coefficient value of each item differed widely and overall agreement was low (K = 0.338). The agreement in the sum scores was ICC 0.506 (moderate agreement). A greater agreement existed in the questions about ‘a priori design provided’, ‘a duplicate study selection and data extraction’ and ‘the list of studies provided’. Conclusions: A rather low agreement was obtained with the use of the R-AMSTAR instrument. Some items are ambiguous and may not be a direct attribute of SR methodological quality. P2B248 How often do decision analyses of interventions agree with matching systematic reviews? Mhaskar R1 , Georgiev H1 , Wao H1 , Kumar A1 , Djulbegovic B1 1 University of South Florida, Center for Evidence based Medicine and Health Outcomes Research, USA Background: Systematic review (SR) of randomized controlled trials (RCT) is reckoned the gold standard for informing treatment choice. Decision analyses (DA) inform health care policy decisions in absence of RCTs or SR of RCTs. However, oversimplification of real world scenarios in DA can be problematic. We have previously shown that DA disagrees with RCTs in 50% of cases. However, it is not known how often the results of DA and SR of RCTs either agree or disagree. Objective: To compare the conclusions of DA and matching SR of RCTs. Methods: We searched PubMed up to 2008 for DAs comparing at least two interventions followed by SRs that matched the DAs based on patient population, intervention, control, and outcome criteria (PICO). From each DA and SR, we extracted data on PICO, conclusion, and impact of sensitivity analyses on the conclusion. Agreement between DA and SR was based on matching of respective conclusions. Results: From 42,704 retrieved DA citations, we found matching SR for 38 comparisons (Figure 1). We found 74% (28/38) agreement between the conclusions of the DAs and the SRs. The sensitivity analyses conducted in either DA or SR did not impact the agreement. Two DA design characteristics were significantly associated Figure 1 (P2B248). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 139 with agreement: use of single versus multiple data source (p = 0.048) and use of meta-analysis data (p = 0.040). Conclusions: This first study quantifying the correlation between the results of DA and SR of RCTs suggests a high level of agreement. Use of meta-analysis data and use of multiple sources of data appear to impact the agreement of conclusions between DA and SR of RCTs. P2B249 Epidemiology of the type of intention-to-treat reporting in meta-analyses Montedori A1 , Bonacini MI2 , Luchetta ML3 , Papini D4 , Rocchi RE1 , Orso M1 , Cozzolino F1 , Abraha I1 1 Regional Health Authority of Umbria, Italy; 2 Derriford Hospital, Plymouth, UK; 3 Azienda Sanitaria Locale 3, Foligno, Italy; 4 Regional Agency for Health of Emilia Romagna, Italy Background: Despite the intention-to-treat (ITT) principle is the recommended method to perform unbiased analyses in randomised trials, it is inadequately applied and participants are often improperly excluded. Recently, a modified intention-to-treat (mITT) analysis with inconsistent descriptions is increasingly being used in randomised trials. The reporting of mITT in trials is significantly associated with post-randomisation exclusions. However, little is known about the impact of mITT reporting trials in trials included in meta-analyses. Objectives: To estimate the prevalence of the type of ITT in meta-analyses and to assess the description of mITT reporting; and to determine differences in terms of reporting, methodological quality and effect size among the types of ITT. Methods: We searched MEDLINE for systematic reviews indexed from 2005 to 2009. Montori’s search strategy was used (MEDLINE[Title/Abstract]) OR (systematic[Title/Abstract] AND review[Title/Abstract] OR metaanalysis[Publication Type]). From each year we randomly selected 10% of the records. Only therapeutic meta-analyses that included at least one randomised trial were considered. Diagnostic, prognostic, or epidemiological studies were excluded. Within each meta-analysis trials were classified according to the type of ITT analyses used as follows: ITT, trials reporting the use of standard ITT analyses; mITT, trials reporting the use of ‘modified intention-to-treat’ analyses; or ‘no ITT’ trials not reporting the use of any ITT analyses. Trials reporting the use of ITT with descriptions or conditions different from the standard ITT definition were classified as mITT. Items for quality measures such as allocation concealment, blinding, and early stopping were considered. Results: We identified 200 meta-analyses out of 569 records published in 2005, 210 of 657 published in 2006, 190 of 804 published in 2007, 197 of 863 published in 2008, and 186 of 988 published in 2009. Conclusions: Definitive results will be presented at the Colloquium. P2B250 Using AMSTAR to assess the methodological quality of systematic reviews: An external validation study Parmelli E1 , Banzi R2 , Fernandez Del Rio MDP3 , Minozzi S3 , Moja L4 , Pecoraro V3 , Liberati A1 1 Italian Cochrane Centre and University of Modena and Reggio Emilia, Italy; 2 Mario Negri Institute for Pharmacological Research and Italian Cochrane Centre, Milan, Italy; 3 Italian Cochrane Centre, Mario Negri Institute for Pharmacological Research, Milan, Italy; 4 University of Milan, Italy Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: AMSTAR is an instrument for the evaluation of systematic reviews’ (SRs) methodological quality of conduct, a pre-requisite for the interpretation and application of SRs findings. Previous studies suggested that AMSTAR is reliable, valid and easy to use. A broader range of assessors on different types of SRs is needed. Objectives: To measure the reliability, construct validity and feasibility of AMSTAR on a sample of SRs in different medical fields. Methods: A convenient sample of SRs in different medical fields was independently assessed by two reviewers using: AMSTAR and the Overview of Quality Assessment Questionnaire (OQAQ). Reliability was assessed using the weighted Cohen’s kappa. Construct validity was investigated comparing the two instruments using Pearson’s Rank correlation coefficient. Time to complete scoring was also recorded. Results: Fifty-four SRs focusing on primary prevention of hypertension and colorectal cancer screening and treatment were assessed using both AMSTAR and OQAQ. Eleven out of 54 were Cochrane SRs. 78% of the SRs analysed included a meta-analysis. Preliminary data on half sample showed an overall score ranging from 0 to 11 (median: 6) for AMSTAR and from 0 to 9 (median: 6) for the OQAQ. The inter-observer agreement of the individual items on AMSTAR ranged from fair to very good with a kappa going from 0.20 to 0.92. Four items scored a kappa >0.75. Fair agreements related to items about methods to combine results and conflict of interest. The kappa for each individual item for OQAQ ranged from 0.25 to 0.81. The reliability of the total score was fair for both checklists: kappa scored 0.35 for AMSTAR and 0.32 for OQAQ. Final results will be presented at the Colloquium. Conclusions: This study represents a further validation of the AMSTAR involving a broader range of assessors and SRs, and could help to refine the instrument and disseminate its use. P2B251 Systematic evaluation of the methodology of randomized controlled trials of anticoagulation in patients with cancer Rada G1 , Schünemann H2 , Labedi N3 , Akl E3 1 Department of Internal Medicine, Evidence Based Health Care Program, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile; 2 Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada; 3 Department of Medicine, State University of New York at Buffalo, Buffalo, NY, USA Background: Biased randomized controlled trials (RCTs) mislead clinical practice and adversely affect patients’ outcomes. A number of methodological features can limit bias in RCTs. Objectives: To systematically assess the descriptive characteristics, the methodological quality and their time trends in RCTs included in six systematic reviews of anticoagulation in cancer patients. Methods: We conducted a comprehensive search, including several electronic databases (MEDLINE, EMBASE, ISI the Web of Science, and CENTRAL) until February 2010. We included RCTs in which the intervention and/or comparison consisted of: vitamin K antagonists, unfractionated heparin (UFH), low molecular weight heparin (LMWH), ximelagatran or fondaparinux. We used a standardized form to abstract study descriptive characteristics and rate methodological quality (blinding of patients, providers, outcome assessors and data analysts; intention to treat analysis; early stopping for benefit; and percentage follow-up). We performed descriptive analyses and assessed the association between the variables of interest and the year of publication categorized in 5-year intervals using chi-square test. Results: The title and abstract Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 140 screening of 3986 citations identified 69 potentially eligible RCTs, of which 2 were additional reports of RCTs and were excluded. 67 unique RCTs are represented in the sample, including 24071 participants. Adequate allocation concealment was 67%, patients’ blinding 40%, data collector’s blinding 40%, providers’ blinding 43%, outcome assessors’ blinding 71%, data analysts’ blinding 16%, intention to treat analysis 64%, and stopping early for benefit 3%. The mean follow-up rate was 96%. Results of association between the variables of interest and the year of publication will be presented at the Colloquium. Conclusions: RCTs of anticoagulation in cancer patients appear to use less rigorous outcome assessment methods and to have deficiencies in key methodological features. It is not clear whether this reflects a problem in the conduct or the reporting of these trials, or both. Trialists of anticoagulation in patients with cancer should design more rigorous trials and transparently report their methods and findings. P2B252 An online tool to assess the potential impact of missing outcome data on the estimates of treatment effect of trials Akl E1 , Agarwal N2 , Guyatt G3 1 Department of Medicine, State University of New York at Buffalo, NY, USA; 2 State University of New York at Buffalo, NY, USA; 3 Department of Clinical Epidemiology and Biostatistics, Hamilton, ON, Canada Background: Systematic reviewers commonly deal with RCTs for which relevant outcome data are missing. The Cochrane Handbook recommends reviewers to assess ‘how sensitive results are to reasonable changes in the assumptions that are made’. Objective: To develop an online tool to assess the potential impact of reasonable assumptions about missing data for binary outcomes on the estimates of treatment effect of RCTs. Methods: We developed the tool as an online interactive Java application. It conducts the sensitivity analysis at the RCT level. For each trial arm (i.e., the intervention and control), the user enters the number of participants randomized, the number of observed events, and the number of participants for whom outcome data is missing. For the RCT under consideration, the tool provides the effect estimate as relative risk with its 95% confidence interval based on each set of assumptions being tested. Results: The tool is available freely online at: http://apps.medinnovations.us/. It allows the testing of the following assumptions about the outcomes of participants for whom data is missing: (1) none had an event; (2) all had an event; (3) all in the treatment group had an event while none in the control group had one (worst case scenario); (4) none in the treatment group had an event while all in the control group had one (best case scenario); (5) the event incidence among participants with missing data (relative to observed participants) varies by ratios specified by the user for the intervention and control groups separately. It also provides the results for a complete case analysis (e.g., excluding participants for whom data I missing from the analysis). Conclusions: The tool allows systematic reviewers to conduct sensitivity analyses using the assumptions that are most plausible for the specific question of their review. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P2B253 Conflict of interest policies for clinical practice guideline organizations do not measure up to IOM standards Norris SL1 , Holmer HK1 , Ogden LA1 1 Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, USA Background: Conflict of interest (COI) is an important potential source of bias in the development of clinical practice guidelines (CPGs). A 2011 Institute of Medicine (IOM) report delineates standards for CPGs, including COI. Objectives: To describe the COI policies for organizations producing a large number of CPGs and to compare those policies to recommendations in the IOM report. Methods: We performed a cross-sectional study of organizations listing five or more CPGs within the National Guideline Clearinghouse between 1/1/09 and 11/10. We searched for COI policies in organizational websites and in recent CPGs. We abstracted the domains of each policy into a standardized template and compared them to recommendations 2.1–2.4 in the IOM report. Results: We identified 37 organizations that fulfilled inclusion criteria. 33% of organizations had a COI policy specific for CPGs; an additional 15% had a general COI policy encompassing research and professional behavior, and the remainder had no policy that we could identify. No organizational policy recommended that CPG funders should be disclosed and only 6% of organizations precluded industry funding. 17% of organizations considered COI in the selection of CPG panel members, and two organizations (6%) had a policy that prohibited the Chair from having a relevant COI. Only one organization indicated that a minority of members of a CPG panel could have a COI. Conclusions: Organizations producing large numbers of CPGs do not meet the standards of the 2011 IOM report and half do not have a specific policy. These organizations need to address this issue in order to provide trustworthy CPGs. P2B254 Reporting of conflicts of interest in Cochrane reviews of trials of pharmacological treatments Roseman M1 , Bero LA2 , Coyne JC3 , Lexchin J4 , Turner EH5 , Thombs BD1 1 McGill University and Lady Davis Institute for Medical Research, Jewish General Hospital, Canada; 2 University of California, San Francisco, USA; 3 University of Pennsylvania School of Medicine, USA, and University of Groningen, Netherlands; 4 University Health Network and University of Toronto, Canada; 5 Oregon Health and Science University and Portland Veterans Affairs Medical Center, USA Background: Reporting guidelines recommend that reports of trials of pharmacological treatments include disclosure of conflicts of interest (COIs) resulting from pharmaceutical industry funding and author-industry financial ties. A recent study (Roseman et al., JAMA, 2011) found that meta-analyses published in high-impact biomedical journals rarely include information on study funding and other COIs from included trials. The Cochrane Handbook specifies that trial funding sources be collected in all reviews, and it suggests that potential bias related to study sponsorship may be incorporated into risk of bias assessments. Objective: To investigate whether systematic reviews of pharmacological treatments published in the Cochrane Database of Systematic Reviews report COIs from included trials. Methods: We searched the Cochrane Database of Systematic Reviews using the MeSH Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 141 term ‘drug therapy’ to identify all systematic reviews of pharmacological interventions published in 2010, with content assessed as up-to-date in 2008 or later and including ≥1 RCT. Two investigators independently extracted data on whether Cochrane reviews reported study funding sources and other COIs from included trials and, if so, where in the review such information was reported. Results: Of 151 included Cochrane reviews, 30 (20%) reported trial funding information for all included studies; 16 (11%) reported for some, but not all, studies; and 105 (69%) did not report any information. Only 16 reviews (11%) provided any information on trial author financial ties or employment. Of 46 reviews that reported trial funding sources for at least some of the trials reviewed, 29 (63%) noted this in the context of risk of bias, either in the text and in ‘Risk of bias’ tables. Overall, however, the method of reporting COIs was inconsistent across reviews. Conclusion: Less than a third (31%) of Cochrane reviews of pharmacological treatments report information on study funding from included trials, and only 11% report on other potential sources of COIs. P2B255 The placebo response in neuropathic pain trials: It is not what you think Selph S1 , Carson S1 , McDonagh M1 1 Oregon Evidence-based Practice Center; Oregon Health & Science University; USA Background: The placebo response varies across neuropathic pain trials making it challenging to demonstrate a treatment effect and difficult to compare trials of different medications. Objectives: The primary objective was to determine the study-level characteristics which predict the placebo response in trials of painful diabetic neuropathy. A second objective was to identify characteristics that are associated with changes in relative effect, independent of the placebo response. Methods: A systematic review of the English-language literature was performed and fair-good quality, randomized, placebo-controlled trials measuring ≥50% pain relief from baseline as an outcome, were identified. Medications included the tricyclic antidepressants, the serotonin-norepinephrine reuptake inhibitors, gabapentin, pregabalin, and other anticonvulsants. Meta-regression techniques were employed to identify predictors of placebo response and predictors of relative effect while controlling for the placebo response. Covariates included: drug studied, publication year, trial duration, size and design, gender distribution, age, duration of diabetes and neuropathy, number of treatment groups, presence of a washout period, use of additional pain medications, and rate of patient recruitment. Results: Twenty-five trials of diabetic neuropathy were included. The best predictor of placebo rate was whether the study was conducted in the United States or Europe versus another country or region (p = 0.053). After adjusting for placebo response, predictors of relative risk were year of publication and length of trial. For example, assuming a 20% placebo rate, a 6-week trial published in 1997 is 1.7 times more likely to demonstrate pain relief versus placebo than a 12-week trial published in 2005. In contrast to previous research, neither baseline pain levels nor recruitment rate were found to predict placebo response. Conclusions: Although the placebo response in neuropathic pain trials is an important consideration for systematic reviewers, trial duration and publication year may be equally important. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P2B256 Concordance of reporting of primary outcomes in abstracts and full text of randomized clinical trials: Is there an outcome reporting bias? Ribic C1 , Lamontagne F2 , Johnston B3 , Briel M4 , You J1 , Sun X5 , Busse J6 , Bassler D7 , Vera C8 , Al-Shurafa M1 , Mills E9 , Guyatt G1 , Cook D1 , Akl E10 1 McMaster University, Canada; 2 University of Sherbrooke, Canada; 3 University of Alberta, Canada; 4 University Hospital Basel, Basel; 5 Sichuan University, China; 6 The Institute for Work and Health, Canada; 7 University Children’s Hospital, Germany; 8 Pontificia Universidad Catolica de Chile, Chile; 9 Centre for International Health and Human Rights, Canada; 10 State University of New York at Buffalo, USA Background: Reporting the results of pre-specified primary outcomes in the full text of randomized clinical trials (RCTs) but not in the associated abstract, or reporting primary outcomes in the abstract that are not reported in the full text, represents a potential source of publication bias. Objectives: To assess the concordance in reporting primary outcomes between abstracts and full text of RCTs published in top medical journals. Methods: We identified all intervention RCTs published from 2005 to 2007 in the NEJM, JAMA, Lancet, BMJ, or Annals of Internal Medicine. Ineligible trials were cross over and cluster designs. Teams of two reviewers appraised the abstract then, separately, the full text of each selected article using pretested, standardized forms. Abstractors were blinded to outcomes reported in full text when screening abstracts. We calculated the concordance of primary outcome reporting between abstracts and full texts. Results: Of 709 eligible RCTs, abstracts reported one primary outcome (55.1%, n = 391), multiple primary outcomes (25.6%, n = 182) or no primary outcome (19.2%, n = 136). Of abstracts reporting one primary outcome, 97.4% (n = 381) had concordant reporting in the full text. We found 2.6% (n = 10) of RCTs had discordant reporting of single primary outcomes in abstracts compared to full text. Of these, 6 trials reported a single primary outcome in the abstract and pre-specified multiple primary outcomes in the full text, while 4 trials reported a single primary outcome in the abstract and did not specify a primary outcome in the full text. Conclusions: In this representative heterogeneous sample of RCTs published in high-impact journals, there is a high concordance between single primary outcomes reported in abstracts and those specified in the full text. However, approximately one-fifth of the abstracts did not specify a primary outcome, potentially introducing bias in primary outcome reporting. P2B257 Conflict of interest among clinical practice guidelines authors Selph S1 , Holmer H2 , Ogden L2 , Norris S2 1 Oregon Evidence-based Practice Center; Oregon Health & Science University; USA; 2 Department of Medical Informatics and Clinical Epidemiology; Oregon Health & Science University; USA Background: Conflict of interest (COI) among clinical practice guideline (CPG) authors is an important potential source of bias for recommendations. A recent report of the Institute of Medicine (2009) suggested that updated descriptive information is needed on these conflicts. Objectives: To describe the extent and predictors of COI among CPG authors in the National Guideline Clearinghouse. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 142 Methods: A random sample of 250 CPGs was obtained on 11/22/2010. For each CPG, data were abstracted on characteristics of the CPG and author disclosures. Regression techniques were used to predict the presence of COI among authors. Results: Of 114 CPGs analyzed to date, over a third were developed and funded by medical specialty societies, 3% were funded by industry and 34% by government agencies. The median number of authors per CPG was 14; 21% of guidelines had ≥25 authors. The most common focus for CPGs was disease management, followed by diagnosis and therapy. Author disclosures were reported in 76% of CPGs. Of these, 30% reported one or more COIs. An average of two authors (range 0–17) disclosed any COI for each CPG, the most frequently reported disclosure was paid consultancy for industry. Although 87% of organizations provided a COI policy, 24% of guidelines reported no information on COIs. Journal publication was a significant predictor of a guideline reporting any authors with a COI (p < 0.0001). The year the CPG was released, the focus of the CPG, and the country originating the CPG were not significant predictors of COI reporting. Conclusions: Funding of CPGs by private specialty organizations is common, conflicts of interest among CPG authors is frequent, and a significant percentage of CPGs do not provide disclosures. To be trustworthy, efforts are needed to achieve accurate, relevant, and readily accessible author disclosures of conflicts of interest in all clinical practice guidelines. P2B258 Questionable early stopping: Case study of supplemental perioperative oxygen and the incidence of surgical site infection Tikkinen KAO1 , Garcia-Alamino JM2 , Devereaux PJ3 , Guyatt GH3 , Alonso-Coello P4 1 Dept. of Urology and Dept. of Clinical Epidemiology & Biostatistics, Helsinki University Central Hospital and McMaster University, Finland; 2 Dept of Primary Health Care, University of Oxford, UK; 3 Dept. of Clinical Epidemiology & Biostatistics, McMaster University, Canada; 4 Iberoamerican Cochrane Centre, Clinical Epidemiology and Public Health Department, Institute of Biomedical Research IIB Sant Pau, Spain Background: The potential impact of stopping studies early on estimates of effect in meta-analysis has been an area of controversy. We encountered this issue in a systematic review of supplemental oxygen in the prevention of surgical site infections (SSIs) site infections which are a common, potentially serious, and costly complication of surgery. Methods: We conducted a comprehensive search to identify all published and unpublished randomized controlled trials (RCTs) evaluating the effect of supplemental perioperative oxygen on SSIs. Two reviewers independently determined study eligibility and extracted data. Results: Seven RCTs that included a total of 4,544 patients met eligibility criteria. Studies suffered from moderate risk of bias. Three out of seven (43%) trials were stopped early: one for benefit (apparent 54% reduction in relative risk), one for Figure 1 (P2B258). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 143 harm (apparent doubling of risk), and one for futility. The mean number of SSIs was 129 (range 5–272) in the non-stopped and 32 (range 27–41) in the stopped trials. Confidence intervals around the pooled estimate from all trials included a 35% reduction, and a 23% increase, in relative risk (614 events; RR 0.89 95% CI 0.65 to 1.23), with substantial heterogeneity (I2 = 68%) (Figure 1). Differences in patients, interventions, outcome measurement, or risk of bias could not explain differences in study results. Conclusions: Our systematic review demonstrates how trials stopped early for benefit or harm tend to provide misleading results. Stopping early for any reason reduces the likelihood of accumulating sufficient evidence to provide definitive results. P2B259 The quality of trials in quality improvement: Systematic review of risk of bias in diabetes studies Ivers N1 , Tricco A2 , Turner L3 , Moher D4 , Grimshaw J5 1 Department of Family Medicine, Women’s College Hospital, University of Toronto, Canada; 2 Li Ka Shing Knowledge Institute of St Michael’s Hospital, Toronto, ON, Canada; 3 Ottawa Methods Centre, Ottawa Hospital Research Institute, Canada; 4 Department of Epidemiology and Community Medicine, Faculty of Medicine, University of Ottawa, Canada; 5 Clinical Epidemiology Program, Ottawa Hospital Research Institute, Canada Background: Of the 356 studies included in seven recent Cochrane reviews evaluating interventions to improve health care professional practices, 14.3% were deemed as having a high risk of bias. Despite repeated calls for reducing the risk of bias in trials, no previous reviews have assessed whether risk of bias in quality improvement (QI) trials has changed over time. Objectives: To assess for changes in risk of bias of QI trials over time. Methods: This was a secondary analysis of data from a systematic review of 142 trials of QI interventions for diabetes. Risk of Bias was abstracted using the Cochrane ‘Risk of bias’ tool. We also grouped studies such that those with high risk of bias in one of allocation sequence, concealment, blinding, or outcome reporting were deemed high risk of bias overall. Results: In the diabetes QI review, 25 trials were published before 2001, 50 from 2001–2005, and 68 from 2006–2010. There was no improvement over time for 8 of 9 Risk of Bias domains; only ‘similarity in baseline covariates’ showed a trend for improvement, with the proportion of trials at high risk of bias decreasing from 47% prior to 2001 to 13% in those published 2006–2010. Loss to follow up was a common source of bias in recent trials, with 23% of trials published 2006–2010 judged high risk of bias. Allocation sequence (58% unclear) and allocation concealment (56% unclear) were poorly reported. Overall, 14% of the trials were deemed high risk of bias; this has not improved over time. Conclusions: Although other methodological reviews of RCTs have shown improvement over time, this is not reflected in the QI literature. There is a need to reduce the risk of bias and improve the quality of reporting of QI trials so that stakeholders have adequate evidence for action. P2B260 Development of the McHarm: A tool evaluating validity of the collection and reporting of harms Santaguida P1 , Keshavarz H1 , MacQueen G2 , Levine M3 , Beyene J4 , Raina P1 1 Evidence-based Practice Centre, Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada; 2 Dept of Psychiatry, University of Calgary Regional Clinical Dept Head, Alberta Health Services, Calgary Health Zone Foothills Medical Centre, Calgary, Canada; 3 Dept. of Clinical Epidemiology & Biostatistics and Dept. of Medicine, McMaster University St. Joseph’s Healthcare Hamilton; 4 Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada Background: Although the assessment of harms is almost always included as an outcome in intervention and medical test studies, the manner of capturing, and reporting harms is significantly different than the outcomes of benefit. There is a need to consider criteria for separate assessment of validity specific to harms. Objectives: To develop criteria for assessing internal validity of collecting and reporting harms and assess the reliability and validity of the items. Methods: A systematic review of the literature to identify biases and items within existing quality assessment tools was undertaken; criteria related to validity were extracted. A Delphi Consensus exercise with 6 experts was undertaken to reduce the items. Following consensus, the criteria within the tool were pilot testing for phrasing and a help file was developed. Test-retest reliability was undertaken. Construct, and discriminant validity testing was undertaken. Results: From 5549 citations, 101 full text articles were screened and from this 70 studies had at least one item. A total of 415 items were generated from the review; after elimination of redundant items, 126 criteria were included in the Delphi consensus. Following two rounds of consensus and piloting, 15 criteria were included in the McHarm. Reliability testing showed acceptable levels (internal consistency all studies = 0.82; 95% CI 0.79 to 0.85, and intra-class correlation = 0.95, 95% CI 0.91 to 0.98). Construct validity with the Chou and Helfand scale was 0.66; discriminant validity with the Jadad scale was 0.22. Reliability and validity varied between surgical and pharmacological studies suggesting a potential influence by drug type. Conclusions: The McHarm contains 15 items related to assessing the internal validity of capturing and reporting harms in intervention studies and was shown to be reliable and valid. Diagnostic test accuracy review methods Day: 21 Oct (P2B261–P2B271) P2B261 Reporting of meta-analyses and systematic reviews of screening tests in China Tian HL1 , Tian JH1 , Yi K1 , Gou YN1 , Yang KH1 1 Evidence-Based Medicine Centre, Lanzhou University, China Background: As a developing country with a relative lack of medical resources, for patients in China it is very important that early screening and diagnosis is being carried out. Objective: To describe the current status and production to date of meta-analyses and systematic reviews of screening tests in China. Methods: We searched three main medical databases from 1978 to 2011: CNKI (China National Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 144 Knowledge Infrastructure), VIP (a Chinese full text database) and CBM (China Biomedicine Database). We used the following search strategy: (diagnostic accuracy OR screen OR sensitivity OR specificity) AND (systematic review OR meta-analysis). Results: We searched 1496 potentially relevant papers, nine studies met the inclusion criteria. The first meta-analysis of a screening test was reported in 2002. Five studies reported reference standard. For sensitivity and specificity analysis, the summary receiver operating characteristic (SROC) was the preferred method for analysis in eight studies, only two studies used QUADAS tool. Five studies reported the sensitivity analysis and heterogeneity analysis. Meta-disc was the most used statistical analysis software (44.4% of all studies). Cancer was the most common target disorder examined (44.4%). Only one study clearly reported financial conflicts of interest. Two studies were published in profesional evidence based medicine journals (Chinese journal of evidence based medicine, Chinese journal of evidence-based pediatric). Conclusion: The dissemination and production of the meta-analyses and systematic reviews of screening test research in China needs to be improved, especially in the methodological quality and conflict of interest reported. We would like to suggest that firstly: researchers should conduct systematic reviews of diagnostic test accuracy according to the Cochrane Screening and Diagnostic Tests Methods Group’s requirements regarding format as much as possible; secondly, as a non-native English-speaking country, more monographs and research about screening tests translated into Chinese is needed. We used the search terms ‘‘systematic reviews’’ or ‘‘meta-analysis’’ and ‘‘sensitivity’’ or ‘‘specificity’’ or ‘‘accuracy’’ or ‘‘diagnostic’’ in title and keywords to search the Chinese Biomedical Medicine database (CBM) (1979–2010), the Wanfang database (2001–2005), the Chinese National Knowledge Infrastructure (CNKI) (1994–2010) and the full database of China (VIP) (1989–2010). We excluded duplicate papers, papers not published in Chinese, conference abstracts, simple literature evaluation, and protocols. Results: There were 153 studies included. The number of DTA systematic reviews has increased over the last 10 years in China (Figure 1), but the uptake of new methods and statistical software lags behind. Pooling sensitivity and specificity or using the summary receiver operating characteristic (SROC) was the prior method in 75% of studies. The fixed or random effects models were used in reviews for 19% of studies. However, the bivariate random effects model and the hierarchical summary receiver operating characteristic (HSROC) model were not used in any of the included reviews. At the same time statistical software developments were limited. Meta-DiSc and Review Manager were used in 50% and 31% of the reviews respectively. STATA was only applied in 7% and SAS in 5% of the reviews. Other statistical software (SPSS, EXCEL, etc.) was used in 31% of the reviews. Conclusions: Even though the quantity of DTA systematic reviews has increased in the last decade in China, uptake of up to date statistical methods and software is still lagging behind. It is necessary to increase the use of up to date statistical methods and software in diagnostic systematic reviews in China. P2B262 Methodological developments and statistic software used in diagnostic systematic reviews in China P2B263 Epidemic characteristics and quality of meta-analysis or systematic reviews in screening tests Chong L1 , Sun R1 1 Evidence-Based Medicine Centre, Lanzhou University, China Background: With the introduction of Quality Assessment of Diagnostic Accuracy Studies (QUADAS), the production of diagnostic test accuracy (DTA) systematic reviews of diagnostics have further progressed in China. The quantity of reviews has been growing from 2001 to 2010. But the developments of use of methods and statistical software have only been assessed in small survey reviews so far. Objectives: To evaluate the status of methodological developments and statistical software in DTA reviews conducted in China. Methods: Figure 1 (P2B262). The number of diagnostic systematic reviews in last decade of China. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Tian HL1 , Gou YN1 , Yi K1 , Tian JH1 , Wang XJ1 , Yang KH1 1 Evidence-Based Medicine Centre, Lanzhou University, China Background: These years there were a number of developments in the methodology used in systematic reviews or meta-analysis of screening tests. But the epidemic characteristics and quality of meta-analysis or systematic reviews in screening tests has not been evaluated. Objectives: To explore the epidemic characteristics and quality of meta-analysis and systematic reviews in screening tests. Methods: Eight databases (including MEDLINE and EMBASE) were searched. Outcome analysis methods, methodological quality, statistical analysis software and conflicts of interest reported were evaluated. Results: The searches yielded 4,726 relevant articles. After application of inclusion criteria, 98 studies were included. Cancer was the most common category of target disorder (33% or studies), followed by depression (6%) and alcohol abuse disorders (6%). The main methods used for outcome analysis were sensitivity (96%; 94/98 of included studies), specificity (96%; 94/98), positive predictive value (57.1%; 56/98), negative predictive value (52%; 51/98), positive likelihood ratio (30.6%; 30/98), negative likelihood ratio (29.6%; 29/98), diagnostic odds ratio (32.7%; 32/98), pre-test odds (8.2%; 8/98), post-test odds (6.1%; 6/98), receiver operating characteristic curves (SROC: 8.2%; 8/98). Statistical analysis reported were: fixed-effects model (8.2%; 9/98 of included studies), random-effects model (15.7%; 19/98), publication bias (5.6%; 9/98), bayesian bivariate hierarchical mode (6.1%; 6/98), sensitivity analysis (22.4%; 22/98), heterogeneity analysis (35.7%; 35/98). The included reviews assessed methodological quality through QUADAS (30.6%; 30/98 of included studies), not reported (56.1%; 55/98), and others (13.3%; 13/98) (Figure 1), and Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 145 Figure 1 (P2B263). used the following statistical softwares: Meta-DiSc (19.4%; 19/98 of included studies), STATA (14.3%; 14/98), SAS (4.1%; 4/98), RevMan (6.1%, 6/98), others (13.3%; 13/98). Conclusions: The number of meta-analyses and systematic reviews on screening tests has increased over time (Figure 2). Sensitivity, specificity, positive predictive value and negative predictive value remain the preferred methods for analysis. Although quality assessment in meta-analyses has improved with the introduction of QUADAS in 2004, most screening test reviews still do not use it. P2B264 Addressing methodological challenges in evaluating diagnostic tests: Combining the GRADE approach and the RAND/UCLA Appropriateness Method to produce clinical recommendations Ballini L1 , Vignatelli L1 , Negro A1 , Maltoni S1 , Grilli R1 1 Agenzia Sanitaria e Sociale Regionale Emilia-Romagna – Italy Background: Developing recommendations for appropriate use of diagnostic tests presents serious challenges from defining the clinical rationale for the new test and its position in the current diagnostic-therapeutic pathways to dealing with the lack of evidence on impact of test on patient-important outcomes. Recent methodological breakthroughs (1,2) provide valuable insights to address these challenges. Objectives: develop a new methodology (transparent and encompassing clinical reasoning) which combines the GRADE approach for grading quality of evidence and strength of recommendations with the RAND method of consensus; test the methodology to produce recommendations on the use of Positron Emission Tomography (PET) in oncology. Methods: two multidisciplinary panels of experts were convened to produce recommendations on PET’s use in four types of cancer (breast, esophageal, lung, and colo-rectal cancer). Answerable research questions were developed by positioning and comparing PET against existing diagnostic tests (replacement, add-on, triage). Clinical rationale was based on capacity of new test to modify initial diagnosis, subsequent change in therapeutic approach and resulting clinical benefits for patients. Results from the systematic review of literature on PET’s diagnostic accuracy were analysed and presented using GRADE’s levels of evidence. Lack of direct evidence on patient-important outcomes was bypassed using GRADE’s logic for eliciting experts’ judgement on clinical consequences of testing positive and testing negative. Results of two rounds of votes on level of appropriateness were analysed using the RAND/UCLA Appropriateness Method. Results: a total of 34 a voting forms for each clinical question was generated via the above described methodology; recommendations for each cancer were produced by the panels in just two meetings; levels of agreement and disagreement were registered and reported. Conclusions: 37 experts engaged in producing clinical recommendations managed to work through the GRADE’s approach and delivered 34 recommendations in a very short period of time. References 1. Bossuyt PM, Irwig L, Craig J, Glasziou P. Comparative accuracy: assessing new tests against existing diagnostic pathways. BMJ. 2006; 332(7549):1089–92. 2. Schünemann HJ, Oxman AD, Brozek J, Glasziou P, Jaeschke R, Vist GE, Williams JW Jr, Kunz R, Craig J, Montori VM, Bossuyt P, Guyatt GH; GRADE Working Group. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ. 2008;336(7653):1106–10. P2B265 Multivariate meta-analysis of diagnostic test accuracy studies with multiple thresholds per study Riley R1 , Guha A1 , Biswas A2 , Smith-Bindman R3 , Deeks J1 1 University of Birmingham, UK; 2 Indian Statistical Institute, India; 3 University of California, USA Background: A typical meta-analysis of diagnostic test accuracy studies uses a single two by two table per study. A bivariate Figure 2 (P2B263). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 146 random-effects meta-analysis can synthesise these tables, utilising their binomial distribution within-studies. For tests on a continuous scale, each study’s two by two table corresponds to a threshold value dichotomising the test into ‘positive’ and ‘negative’ groups. Sometimes studies report multiple two by two tables, relating to multiple threshold values, that collectively follow a multinomial within-study distribution. However researchers often ignore this multinomial structure, and meta-analyse thresholds separately assuming a binomial dsitribution. Aims & Methods: Hamza 2009 meta-analysis studies with multiple thresholds results and account for their multinomial structure; they focus on complete data (all studies provide all thresholds) and estimating ROC curves. In this poster we evaluate their approach given missing data (some studies provide an incomplete set of thresholds) and focus on sensitivity and specificity estimates. We apply the method to 30 studies of endovaginal ultrasound for detecting endometrial disease, with up to 7 different thresholds per study and missing data. Results: We show that, compared to a separate meta-analysis of each threshold assuming a binomial distribution, the multinomial approach produces different summary estimates for each threshold (with sensitivity or specificity changed by up to 10%) and smaller standard errors (reduced by up to 45%). For example, for a threshold of 8 mm the summary sensitivity was 85% (binomial) and 79% (multinomial), with standard errors of logit-sensitivity of 0.17 (binomial) and 0.09 (multinomial). Conclusions: When studies provide test accuracy results for multiple thresholds, these thresholds should be jointly meta-analysed accounting for their multinomial structure; otherwise statistical precision is lost and clinical conclusions may be misleading. Reference 1. Hamza TH, Arends LR, van Houwelingen HC, Stijnen T. Multivariate random effects meta-analysis of diagnostic tests with multiple thresholds. BMC Medical Research Methodology 2009;9:73 P2B266 Overinterpretation of results in diagnostic test accuracy studies: evidence of ‘‘spin’’ Ochodo E1 , de Haan M1 , Reitsma J1 , Lotty H2 , Bossuyt P1 , Leeflang M1 1 Academic Medical Centre, Amsterdam, Netherlands; 2 Dutch Cochrane Centre, Netherlands Background: Overinterpretation of the results by authors in order to make the interventions look favourable is referred to as ‘spin’. This has been shown to be common in randomized trials but may also play a role in diagnostic accuracy studies. The clinical use of tests based on inflated conclusions could lead to unnecessary testing and may trigger physicians to make incorrect clinical decisions, compromising patient safety. Objectives: To explore the mechanisms behind ‘spin’ in diagnostic test accuracy studies and to estimate their prevalence in journals with a high impact factor. Methods: We searched MEDLINE for diagnostic test accuracy studies published between January and June 2010 in journals with an impact factor of 4 or higher. A random selection was evaluated with a structured data extraction form by two authors independently. Disagreement was resolved by consensus or by a third party. Results: Our search yielded 420 eligible articles of which we selected 85. The interrater agreements were 81% and 100% before and after discussion. Of the 74 articles included in the analysis, 66 (89%) contained a form of spin. The mechanisms behind spin included: not stating a prior study hypothesis (85%), Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. reporting positive test recommendations not in line with the test accuracy measures (12%), basing test recommendations on results from sub-group analysis only (12%), conclusions that differed from the aim of the study (12%), an overoptimistic abstract, with selective reporting of favourable results only, or stronger test recommendations compared to the main text (45%). Conclusions: Overinterpretation of results of diagnostic accuracy studies is common. We hope that highlighting the mechanisms behind spin will enable peer reviewers correctly sieve overoptimistic reports of diagnostic accuracy studies and encourage investigators to be more clear in designing, more transparent in reporting, and more stringent in interpreting test accuracy studies. P2B267 What measures of accuracy should we present in DTA reviews? Jameson CM1 , Whiting PF1 , Burke M1 , Ben-Shlomo Y1 , Sterne JAC1 1 School of Social and Community Medicine, University of Bristol, UK Background: Diagnostic test accuracy (DTA) systematic reviews are standard ways of summarising research evidence. DTA reviews may change clinical practice on an individual level, or their findings may be incorporated into guidelines to influence practice at the health service level. One factor that may affect a review’s potential to influence clinical practice, and the way clinicians subsequently use tests, is how clearly the results are presented and how easy they are to interpret. Objectives: To review available evidence regarding health professionals’ understanding of diagnostic information to inform decisions on how best to present the results of DTA reviews. Methods: MEDLINE, EMBASE and PsycINFO were searched until March 2011, experts were contacted, citation searches of key papers were conducted, and reference lists were screened. Primary studies were included that reported on any outcomes relating to health professionals’ understanding of diagnostic test accuracy which was summarised as sensitivity, specificity, likelihood ratios, predictive values, accuracy or ROC curves. Due to the heterogeneity between studies in terms of design, health professionals, and measures of accuracy investigated, a narrative synthesis was used to combine results. Results: To date 12 studies have been included; the review is ongoing. Initial results suggest that sensitivity and specificity are generally poorly understood by clinicians and are often confused with predictive values. Clinicians tend to overestimate the impact of a positive test result on the probability of disease and this overestimation increases with decreasing pre-test probabilities of disease. The most informative measures for clinicians may be estimates of the post-test probability of disease, which can be presented as a range corresponding to different pre-test probabilities. Conclusions: Conclusions will inform recommendations for presenting estimates of test accuracy in DTA reviews. P2B268 Discrepancy in relative test performance due to modelling strategy in comparative diagnostic meta-analysis: A case study Takwoingi Y1 , Abba K2 , Garner P2 , Deeks J1 1 University of Birmingham, UK; 2 University of Liverpool, UK Background: Hierarchical models for diagnostic test accuracy meta-analysis include study-specific random effects to account for Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 147 between-study heterogeneity. In a comparative meta-analysis, equality of variance parameters can be assumed for different tests whilst allowing other model parameters to depend on each test. Objectives: To demonstrate discrepancy in summary accuracy measures when differences in heterogeneity exist between studies of different tests and equal variances are assumed across tests. Methods: A Cochrane review on rapid diagnostic tests (RDTs) for Plasmodium falciparum malaria included an assessment of the relative accuracy of different types of tests. Type 1 and Type 4 tests were compared by including parameters in the hierarchical summary ROC (HSROC) model to allow each type to have a different threshold, accuracy and summary ROC shape. The impact on the variability of random effects of accuracy and threshold was also investigated, and separate variance parameters included if required based on likelihood ratio (LR) tests. Summary sensitivities and specificities were derived from the models. The analyses were done using SAS Proc NLMIXED. Results: Sixty-five study cohorts evaluated Type 1 tests and were more heterogeneous than the 16 that assessed Type 4 tests. Between models, little or no difference in specificities for either test type was observed (table 1). The sensitivity of Type 4 tests derived from the model with equal variances (model 1) did not reflect the data, and differed from that of the model with separate variances (model 2) by 5.5%. The difference in sensitivity between Type 1 and Type 4 tests was statistically significant (p < 0.001) in model 1 but not in model 2 (p = 0.20). Conclusions: Modelling assumptions can affect conclusions of a comparative meta-analysis. In this case, the more complex and appropriate assumptions led to more conservative differences. The effect of each test on model parameters should be investigated when feasible. P2B269 Can generalizability be considered in systematic reviews of diagnostic test accuracy? Scheibler F1 , Janssen I1 , Schröer-Günther M1 , Sauerland S1 1 Institute for Quality and Efficiency in Health Care, Germany Background: Most diagnostic tests consist of variable components, and very often the technical quality of the diagnostic device or the experience of medical staff play an important role for test accuracy. Therefore these factors should be considered carefully when results of studies of diagnostic test accuracy (DTA) are synthetized in systematic reviews (SRs). To our knowledge no systematic and validated method for extracting and assessing variations in the generalizability of diagnostic tests has been published so far (compare section 9.2.2 in the Cochrane Handbook on DTA). Objectives: To develop a multi-item checklist on the generalizability of DTA studies. Methods: Generalizability aspects were retrieved from the Cochrane Handbook, methodological papers on quality assessment of diagnostic studies and method sections of identified SRs in an overview of SRs on positron emission tomography. An extraction sheet was developed consisting of 7 items and a summary assessment item and tested in 7 SRs with 160 included primary DTA studies. Results: Due to the comprised format and the clear-cut items the instrument proved to be feasible for application within a systematic review. Limitations to be considered are: i) generalizability differs with different health care settings; ii) generalizability cannot be proven, but it is possible to find evidence for non-generalizability; iii) in several aspects there seems to be a trade-off between generalizability and internal validity (e.g. blinding); and iv) as only few studies were categorized as non-generalizable in our feasibility study, sensitivity analyzes did not show differences in diagnostic accuracy depending on generalizability. Conclusions: Aspects of generalizability should be assessed in a systematic and transparent way, separately from aspects of bias. We demonstrate the current state of our instrument and very much appreciate comments and suggestions for its improvement. P2B270 Methodological filters for the identification of delayed cross-sectional studies Noel-Storr A1 , Beecher D2 1 (CDCIG) University of Oxford, United Kingdom; Sclerosis Group, Italy 2 Cochrane Multiple Background: Up to now no published methodological filter designed for the retrieval of diagnostic test accuracy (DTA) studies has proved sensitive enough for use within a Cochrane DTA systematic review. A recent study tested the hypothesis that normal cross-sectional studies should be treated differently from delayed cross-sectional studies Table 1 (P2B268). Variance parameters, summary sensitivities and specificities for Type 1 and Type 4 RDTs from models with and without separate variances for the random effects. Variance parameters Type 1 RDTs Type 4 RDTs Summary estimates Type 1 RDTs Type 4 RDTs Ratio Model 1 Model 2 Common shape and equal variances Common shape and separate variances Accuracy Threshold Accuracy Threshold 2.73 2.73 0.88 0.88 2.53 1.69 0.84 0.67 Sensitivity Specificity Sensitivity Specificity 95.5 (93.9, 96.7) 86.0 (81.3 89.7) 0.90 (0.87, 0.94); p < 0.001 95.0 (93.1, 96.1) 98.8 (98.3, 99.2) 1.04 (1.03, 1.05); p < 0.001 94.8 (93.1, 96.1) 91.5 (84.7, 95.3) 0.96 (0.91, 1.02); p = 0.20 95.2 (93.2, 96.7) 98.7 (96.9, 99.5) 1.04 (1.02, 1.06); p < 0.001 Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 148 (longitudinal analyses). Objectives: i) further test the hypothesis that normal cross-sectional studies and delayed cross-sectional studies are generally described differently in the published literature, ii) refine an unpublished methodological filter designed for identification of delayed cross-sectional studies (this will be done in MEDLINE (Ovid)), and iii) develop an equivalent filter for use in PubMed. Methods: An unpublished filter will be further validated by expansion of the data set. Key elements of reports of potential studies for inclusion within Cochrane DTA reviews that focus on longitudinal diagnosis and prediction will be entered into textual analysis software so as to refine the existing unpublished filter. The filter will then be further tested by a large dataset of reports of potentially relevant studies from existing literature on methodological filters. Finally, a PubMed equivalent filter will be developed. Results: The results will show the new filter’s sensitivity, specificity, precision and accuracy and will be presented at the Cochrane Colloquium in Madrid, October 2011. Conclusions: DTA studies generally fall into two camps – normal cross-sectional studies (e.g. looking at the accuracy of a new test to see if someone is pregnant or not) and delayed cross-sectional studies (e.g. a new test to see whether someone will develop symptomatic dementia from a ‘pre-dementia’ state). The filters published so far treat these studies as one. The literature therefore condemns these filters based on potentially invalid criteria. found. Study characteristics that may be influential in one comparison, may have no influence at all in another comparisons. Knowledge translation and communicating evidence Day 22 Oct (P3A272–P3B345) P3A272 What kind of evidence can be found in incident reporting systems? Systematic evaluation of incident reports to identify risks using syringe pumps Rohe J1 , Hahnenkamp C1 , Dichtjar T2 , Sangiuno H. A1 , Thomeczek C1 , Schleppers A2 , Fishman L1 1 German Agency for Quality in Medicine (AQuMed/AEZQ); 2 Professional Association of German Anaesthesiologists, Germany Background: Incident reporting systems in medicine enable health care professionals to anonymously report patient safety incidents so that others can learn and patient safety can be strengthened. The German system, www.CIRS-AINS.de, is widely established in anesthesia and encompasses 1606 anesthesia-specific reports. Objectives: Learning opportunities of individual reports can be published in case reports or alerts, but also in systematic evaluation. The P2B271 method presented demonstrates the possibility to systematically analyze Adjusting for indirectness in comparative test accuracy incident reports. Using the example of syringe pumps, reports were meta-analyses evaluated to show the risks of their use. Methods: Titles of 1606 incident reports were searched by the term ‘syringe pump’ 1 2 2 1 Leeflang MMG , Di Nisio M , Rutjes AWS , Zwinderman AH , Bossuyt 1 (German ‘Perfusor’), 54 reports were identified. The reports were PMM 1 University of Amsterdam, Netherlands; 2 University G. d’Annunzio Foundation, classified according to the phase (A-D) of the medication process in which the incident originated. Within these phases 13 areas of risk Italy were identified. Results: Phase and area of risk. A. Prescribing: Background: The accuracy of a diagnostic test should be compared to Discontinue medication when indicated; B. Transcribing/Documenting: Communication of prescription; C. Dispensing: Preparation of syringe the accuracy of its alternatives. Direct comparisons of tests, in the same for pump; D. Administering: Programming/labelling of syringe pump, patients and against the same reference standard, offer the most valid Connection of lines, Lines intact, Power supply of syringe pump, Alarms study design, but are not always available. Comparative systematic reviews are therefore bound to rely on indirect comparisons. As the of syringe pump, Change of syringe, Syringe pump works correctly, Syringe pumps and magnetic resonance imaging, Mounting of syringe results from these comparisons can be biased, we investigated ways pump; E. Monitoring: Monitoring of vital signs. The 13 areas of to correct for indirectness. Methods: From a large systematic review risk encompassed one to eight reports and exhibited different aspects about the accuracy of D-Dimer testing for venous thromboembolism, we within the area of risk. Conclusions: Reports from incident reporting selected those comparisons between two assays that contained three systems are one possibility to make experiences of individuals available or more direct comparisons and four or more indirect comparisons for systematic learning. Because not all safety incidents are reported, or single assay studies. Each comparison was analyzed using the data is not representative. The method presented can help–through bivariate random effects meta-regression model with assay-type and summation of individual evidence—to systematically identify areas of directness as covariates in the model. In comparisons with a significant risk during the medication process or within other areas of health effect of directness on sensitivity or specificity, we included the following study features to correct for these differences: referral filter, care. The identified areas of risk can be used to develop strategies to diminish risk and strengthen patient safety. consecutive enrolment, time-interval, type of reference standard and verification. Results: Of the seventeen eligible comparisons, five showed a significant effect of direct comparisons on logit-sensitivity or P3A273 logit-specificity. A first analysis showed that in three of these analyses, How to obtain NNT from Cohen’s d: Comparison of two the addition of at least one of the study features decreased the impact methods of directness as a covariate. For example, in one analysis directness Furukawa TA1 , Leucht S2 had a significant effect on specificity, but addition of time-interval or 1 Kyoto University, Japan; 2 Technische Universität München, Germany reference standard removed this effect (from P = 0.048 to P = 0.22 resp. P = 0.12). Further analyses will be presented at the Colloquium. Conclusions: Adjusting the effect of directness for study features Background: Many indices of size of treatment effect (effect size: seems to be possible in some instances, but no systematic effects were ES) are found in the literature. The preferred one in evidence-based Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 149 medicine is the number needed to treat (NNT) while the most common is Cohen’s d, when the outcome is continuous. There is confusion about how to convert Cohen’s d into NNT. Methods: We conducted meta-analyses of individual patient data from 10 randomized controlled trials of second generation antipsychotics for schizophrenia (n = 4278) to produce Cohen’s d and NNTs for various definitions of response, using cutoffs of 10% through 90% reduction on the symptom severity scale. These actual NNTs were compared with NNTs calculated from Cohen’s d according to Kraemer’s method (Biological Psychiatry, 2006; 59:990–996) and Furukawa’s method (The Lancet, 1999; 353:1680). Results: NNTs from Kraemer’s method overlapped with the actual NNTs in 56%, while those based on Furukawa’s method fell within the observed ranges of NNTs in 97% of the examined instances. For various definitions of response corresponding with 10% through 70% symptom reduction where we observed a non-small number of responders, the degree of agreement for the former method was at a chance level (ANOVA ICC of 0.12, p = 0.22) but that for the latter method was ANOVA ICC of 0.86 (95%CI: 0.55 to 0.95, p < 0.01). Conclusions: Furukawa’s method allows more accurate prediction of NNTs from Cohen’s d. Kraemer’s method gives a wrong impression that NNT is constant for a given d even when the event rate differs. P3A274 Development of a methodological framework for systematic review-based evidence briefings Chambers D1 , Wilson P1 1 CRD, University of York, UK Background: As part of a larger research project, we are providing a knowledge translation service to National Health Service (NHS) decision-makers in the Leeds–Bradford area of England, translating existing sources of synthesised and quality-assessed evidence, primarily systematic reviews, into actionable messages. The service is particularly aimed at commissioners of healthcare services but in principle could cover a wide range of decisions in clinical effectiveness and service delivery and organisation. Methods: We developed a framework for production of evidence briefings based on a scoping review of existing resources (recently published in the Milbank Quarterly ) and our experience of evaluating the evidence base on models of service provision for young people with eating disorders. Where appropriate, we drew on the SUPPORT tools for evidence-informed policymaking. Results: The framework assumes that briefings will be produced in response to decision-makers’ requests and that the process will involve active collaboration between decision-makers and researchers. It covers development of the research question (including a checklist to be completed at an initial meeting); identification of relevant systematic reviews and economic evidence; relating the evidence to the local setting; and consideration of issues of health equity and implementation of any changes to practice or service delivery. Evaluation of use, usefulness, and impact is an important part of the process. Results and implications for further development and research will be presented at the Colloquium. Conclusions: Production of evidence briefings is labour-intensive but appears promising for increasing the impact of systematic reviews and promoting evidence-informed decision-making. Development of a standard approach, together with a more centralised provision of this type of service, could offer economies of scale and make use of national expertise to support the needs of decision-makers at the local level. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P3A275 Cochrane reviews: Beyond implications for research González U1 1 Research Unit for Evidence-based Dermatology, Hospital Plató, Barcelona, Spain Background: Cochrane systematic reviews are an ample source of suggestions and information for further health-care research. At present the section on ‘Implications for research’ specifically comments on the need and nature of further research that would be most desirable. Objective: To explore how Cochrane reviews can further help to make suggestions to assist researchers to plan and conduct useful clinical trials. Methods: Two papers of Cochrane reviews are analyzed (González U, Pinart M, Reveiz L, et al. Designing and reporting clinical trials on treatments for cutaneous leishmaniasis. Clin Infect Dis 2010;51:409–19; and Gonzalez U, Whitton M, Eleftheriadou V et al. Guidelines for designing and reporting clinical trials in vitiligo. Arch Dermatol – in press-). This is a new initiative for helping authors conduct high quality trials by elevating the importance of research gaps and providing specific research guidelines for new trials. Results: The structure of this new type of article is described and recommendations on optimum designs for RCT in every specific area based on the evidence synthesis are made. The first part includes the known sources of bias from existing randomised controlled trials such as selection bias, blinding assessment, attrition bias, and the PICO framework. The second part provides specific recommendations for designing and reporting the thematic trials and it is structured according to the study question, the study design and reporting details. Conclusions: Development of specific clinical research guidelines and recommendations based in Cochrane reviews could improve and inform future efforts for new thematic clinical trials and other initiatives as outcome research. This could be a major need in the production of the best evidence in all health areas as well as a unique type of work within the medical literature and the Cochrane Library. P3A276 Building a database of validated pediatric outcomes Adams D1 , Sivakumar L1 , Nasser H1 , Surrette S1 , Hartling L1 , Vohra S1 1 University of Alberta, Canada Background: Pediatric populations have increasingly been included in clinical research that relies on the availability and use of appropriate outcome measurement tools. Objectives: i) To develop an inventory of valid and reliable pediatric outcome measurement tools and ii) to identify gaps in outcomes reporting in publications of pediatric randomized controlled trials (RCTs). Methods: The top 6 general medicine journals and top 4 pediatric journals were searched for pediatric RCTs published since 2000. Two independent reviewers conducted screening and data extraction. Variables to be extracted included: journal type (general versus pediatric), population age, sample size, condition of interest, intervention, control, primary outcome, outcome measurement tools, and information on psychometric testing. Results: Searches identified 2229 unique references. Most (70%) were identified from pediatric journals, with ages ranging from 33 weeks gestation to 21 years. The most common condition studied was psychological disorders (n = 12). A single intervention was tested in 57% of trials and more than one intervention was tested in 22 trials (36%). Half (48%) reported one Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 150 primary outcome, while 31% did not identify a primary outcome, and 13% identified more than one. We identified 79 different scale or questionnaire-type measurement tools, from 31 studies, that measured outcomes in 21 different conditions. For 41% of these 79 tools, authors provided information on psychometric properties and included relevant citations in 28% for these properties. The most commonly reported properties were reliability (59%) and validity (41%). Conclusions: A wide variety of pediatric outcome measurement tools are in use by researchers. Psychometric properties of measurement tools are inconsistently reported in pediatric RCTs, thus it is unclear to readers if the tools are of high quality. Developing a comprehensive database of validated pediatric outcome measures may facilitate use of high quality pediatric research. P3A277 Patient values and preferences for decision making in antithrombotic therapy: A systematic review MacLean S1 , Mulla S1 , Jankowski M2 , Akl E3 , Vandvik P4 , Ebrahim S5 , McLeod S6 , Bhatnagar N7 , Guyatt G1 1 Department of Clinical Epidemiology and Biostatistics, McMaster University, Canada; 2 Department of Internal Medicine, Faculty of Medicine, Jagiellonian University Medical College, Krakow, Poland; 3 Department of Clinical Epidemiology and Biostatistics, McMaster University, Canada; Department of Medicine, State University of New York at Buffalo, NY, US; 4 Norwegian Knowledge Centre for the Health Services, and Department of Medicine Gjøvik, Innlandet Hospital Trust, Norway; 5 Department of Biostatistics and Clinical Epidemiology, McMaster University, Canada; 6 The Division of Emergency Medicine, Department of Medicine, Schulich School of Medicine & Dentistry, The University of Western Ontario; 7 Health Sciences Library, McMaster University Background: Development of clinical practice guidelines involves making trade-offs between desirable and undesirable consequences of alternative management strategies. Although the relative value of health states to patients should provide the basis for these trade-offs, few guidelines have systematically summarized the relevant evidence. Objectives: In preparation for the 9th iteration of the American College of Chest Physicians anti-thrombotic guidelines, we conducted a systematic review relating to values and preferences of patients considering anti-thrombotic therapy. Methods: We included studies examining patient preferences for alternative approaches to anti-thrombotic prophylaxis and studies that, in the context of anti-thrombotic prophylaxis or treatment, examined how patients value alternative health states and experiences with treatment. We conducted a systematic search and compiled structured summaries of the results. Steps in the process that involved judgment were conducted in duplicate. Results: We identified 48 eligible studies. Sixteen dealt with atrial fibrillation, 5 with venous thromboembolism, 4 with stroke or myocardial infarction prophylaxis, 6 with thrombolysis in acute stroke or myocardial infarction, and 17 with burden of anti-thrombotic treatment. Findings included: 1) Values and preferences for treatments and for health states consistently varied appreciably between individuals. 2) Factors that influence apparent preferences for health states include previous exposure to treatment, previous exposure to health states, and the occurrence of a health state as an adverse effect of treatment. 3) Varying methods of preference elicitation lead to different choices. 4) A reasonable trade-off to assume between stroke and bleeds would be a ratio of disutility of stroke to bleeds ranging from 2:1 to 3:1 Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. (although there are troubling inconsistencies across studies). 5) For most patients, warfarin therapy does not have important negative impacts on quality of life, although there exists a minority for whom it does. Conclusions: Systematic reviews such as ours should become a standard for clinical practice guidelines. P3A278 Publication of IPD meta-analyses in The Cochrane Database of Systematic Reviews: Potential barriers and opportunities Rydzewska L1 , Tierney J1 , Clarke M1 , Stewart L1 , Rovers M2 1 IPD Meta-analysis Methods Group, UK; 2 IPD Meta-analysis Methods Group, Netherlands Background: Individual participant data (IPD) systematic reviews are usually non-Cochrane reviews based on ‘raw data’ and are produced by collaboration of a group of original researchers. IPC are resource-intensive, but they can improve the quality of the data and analyses, providing more definitive findings than are possible with summary data, and are described as the ‘gold standard’ for systematic reviews. Therefore, conversion to Cochrane reviews is encouraged. Their collaborative nature with multiple authors prior to publication, however, may difficult compliance with certain procedures, style, and format recommendations for Cochrane reviews. Objectives: To assess whether members of the IPD Meta-analysis Methods Group (IPDMAMG) convert their IPD reviews to Cochrane reviews, and if not, to describe the reasons why. Methods: Experiences of a small group responsible for IPD reviews in cancer, who have published in CDSR following publication in other medical journals, were reviewed to collect members’ views. The survey across the IPDMAMG, was then extended. Results: The initial survey highlighted the following challenges: time taken to re-format reviews; obtaining permission from the original journal; obtaining approval from all authors for changes; providing new author declarations; and completing the risk of bias tool retrospectively. Experiences varied depending on each Cochrane Review Group’s (CRG’s) awareness of what IPD reviews involve and their individual procedures. Results from the full survey will be presented, with recommendations for facilitating the conversion process, relevant to both CRGs and IPD review authors. Conclusions: If more IPD reviews were converted to Cochrane reviews, this would increase the amount of high-quality evidence in CDSR and the impact of The Cochrane Collaboration’s output. It would reduce unnecessary duplication of effort by making use of work already done for the IPD review, with IPD authors benefitting from increased accessibility to their findings and more opportunities to update reviews as new data become available. P3A279 Strategies and measures to improve the quality of clinical trials in China Wu T1 , Li Y1 1 Chinese Cochrane Centre, Chinese Clinical Trial Registry, West China Hospital, Sihuan University Background: The number of published ‘randomized controlled trials (RCT)’, as reported by their authors, increased quickly in past years in China. Authors of this paper, found that only about 7% of those trials Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 151 were authentic RCTs and that only about 0.2% were submitted to an ethics review committee. Objective: Aiming to improve the quality of clinical trials in China, a series of strategies and measures have been developed and are being implemented by the authors. Methods: 1. Promoting transparency of clinical trials in three ways: prospective registration of the trial before recruting participants, making the trial process transparent and promoting reporting on the quality of the trial. 2. Improvement ethics review committees and increase submission to these bodies. Results: A clinical trials quality control system has been established and is functioning in China: 1. The Chinese Clinical Trial Registry (ChiCTR) was established in October, 2004 becoming a Primary Registry for WHO’s ICTRP in July, 2007. Registration in ChiCTR is free. Up to 30 March, 2011, more than 1200 trials have been registered. 2. The ‘Chinese Clinical Trial Registration and Publication Collaboration (ChiCTRPC)’ was initiated in June 2006 amongst ChiCTR and 48 key Chinese medical journals aiming to promote a policy that would make trial registration mandatory in China. 3. The Clinical Trial eCRF System was established to offer management of raw data of clinical trial on a voluntary basis. In this system the case records and the real-time results of the trials are sent to a central deposit, the raw data and calculated data, except the private information of the participants, can be publicly accessed. 4. The Chinese Ethics Committee for Registering Clinical Trials (ChiECRCT) was established in 2008 to review trial registration. 5. Cooperation with the CONSORT group and EQUATOR has been established to promote the use of the CONSORT guidelines in China. Conclusions: A comprehensive management system of clinical trials in China has been established by the authors. P3A280 Investigating determinants of experts’ judgement on appropriateness for diagnostic tests when using the ‘Grading of Recommendations Assessment, Development and Evaluation (GRADE)’ method for presenting evidence Background: Within a guideline production programme on appropriate use of Positron Emission Tomography (PET), a new methodology—combining the GRADE approach for grading quality of evidence and strength of recommendations with the RAND/UCLA Appropriateness Method—was developed and tested. Objectives: Evaluate whether and to what extent information provided to experts of multidisciplinary panels determined their decision on appropriateness of the diagnostic test. Methods: 37 experts, convened in two multidisciplinary panels with the mandate to define appropriate use of PET for 34 clinical indications in four types of cancer, were provided with a voting form for each clinical question containing a range of information (Table 1). Panellists individually rated the importance of outcomes and level of appropriateness. Statistical analyses included: correlation of each variable (sensitivity and specificity—favouring PET or comparator—level of evidence (LoE), pre-test probability, importance score for the four patient-important clinical outcomes) with rating of appropriateness of all panellists for all clinical questions; multiple linear regression modelling the relationship between the variables and appropriateness rating for all panellists; multiple logistic regression modelling the relationship between the variables and appropriateness rating for each panel. Preliminary Results: correlation coefficients of all variables with rating for appropriateness are reported in Table 2. Linear regression analysis showed that the ten variables accounted for 63.4% of variability of appropriateness rating; among them level of evidence explained 36.5% of variability. The remaining analyses will be completed in due time and final results will be presented. Conclusions: LoE seems to play a major role in explaining the rating of appropriateness. When the analysis is completed we expect to: better quantify the relationship of all chosen explanatory variables with the rating of appropriateness (dependent variable), establish to what extent panellists used the information provided to make judgements, and suggest explanation for would-be unpredicted correlations. Ballini L1 , Negro A1 , Vignatelli L1 , Maltoni S1 , Grilli R1 1 Agenzia Sanitaria e Sociale Regionale Emilia-Romagna, Italy Table 1 (P3A280). Information provided in voting form for each clinical question. clinical rationale in support of FDG-PET clinical effectiveness of therapeutic approach resulting from test results suggested role of FDG-PET in diagnostic pathway (replacement; triage; add-on) evidence of change in management – when available pre-test probability estimates of diagnostic accuracy for FDG-PET and comparator level of evidence list of patient-important clinical outcomes for patients testing true and false positive or true and false negative matrix of ‘‘natural frequencies’’ reporting absolute numbers for true and false positive and negative results per 100 patients estimates of impact on clinical outcomes – when available – and level of evidence Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. appropriateness Panel 2 appropriateness Panel 1 appropriateness All panellists 0.22227 p < 0.0059 N 152 0.56239 p < .0001 N 107 0.35925 p < .0001 N 259 True positive 0.32992 p < .0001 N 152 0.53341 p < .0001 N 107 0.39336 p < .0001 N 259 True negative 0.19891 p < 0.0137 N 153 0.62271 p < .0001 N 107 0.36084 p < .0001 N 260 False negative 0.23894 p < 0.0031 N 151 0.33718 p < 0.0004 N 107 0.26006 p < .0001 N 258 False positive 0.56209 p < .0001 N 171 0.57848 p < .0001 N 122 0.60577 p < .0001 N 293 Level of Evidence 0.59193 p < .0001 N 176 0.58954 p < .0001 N 135 0.58621 p < .0001 N 311 Favours PET −0.37956 p < .0001 N 176 −0.35319 p < .0001 N 135 −0.34348 p < .0001 N 311 Sensibility Favours comparator Table 2 (P3A280). Correlation coefficient of each variable with judgement of appropriateness (global and by panel). 0.00911 p < 0.9059 N 176 0.55358 p < .0001 N 135 0.25646 p < .0001 N 311 Favours PET 0.3093 p < .0001 N 176 −0.30828 p < 0.0006 N 135 0.04966 p < 0.397 N 311 Specificity Favours comparator 0.06368 p < 0.4342 N 153 0.39895 p < .0001 N 110 0.24723 p < .0001 N 263 Pre-test probabiliy 152 Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 153 P3A281 Evidence-based guidelines in a teaching primary care health service: The beginning of evidence-based medicine in the Amazonian region Andriolo RB1 , Silva BNG2 , Atallah N2 1 Universidade do Estado do Pará – Brazil; 2 Universidade Federal de São Paulo, Brazil Background: Evidence-based guidelines (E-BGs) are an important tool to conduct translational research and for improving health services’ quality and outcomes. Objectives: To share information about the first implementation of E-BGs in one teaching primary care health service located in the Amazonian region. Methods: The following actions were planned to achieve the objectives: i) mapping the health care and administrative teams; ii) evaluating the teams’ knowledge and perceptions of evidence-based medicine (EBM); iii) deliver an annual EBM course with topics on the Cochrane Collaboration; iv) mapping the diagnostic, preventive and therapeutic technologies most commonly used by each health specialty; v) hold consensus meetings involving internal and external health professionals, and the administrative body to prioritise themes, taking into account existing uncertainty, epidemiological relevance and costs; vi) preparing evidence-based medicine recommendations (EBMR); vii) holding a new consensus meeting to prepare E-BGs with multidisciplinary contributions to the EBMR documents, according to the local context; viii) comparing outcomes among patients when medical decisions were or were not based on the guidelines (Figure 1). Results: The first phase, already accomplished, shows that our health service is comprised of 50 administrative employees; 70 non-graduated health auxiliaries; and 112 graduate or post-graduate health professionals of different health areas and specialties including pharmacists, cardiologists, clinicians, otorhinolaryngologists, nurses, nutritionists, pediatricians and dentists Figure 1 (P3A281). and the service is responsible for more than 160,000 visits per year. Conclusions: It is expected that successful implementation of the E-BGs will improve health outcomes and be more economical in this public health service. Moreover, this pilot experience with translational research is intended to be reproduced in the Health Department of Pará State, which is responsible for 143 cities located in a region with more than 7 million people characterized by an extreme paucity of human and economic resources for health care. P3A282 Characteristics of studies included for HTA reports Wolff R1 , Ravaud P2 , Liberati A3 , Bonfill X4 , Antes G5 1 Kleijnen Systematic Reviews Ltd, UK/German Cochrane Centre, Germany; 2 French Cochrane Centre, France; 3 Italian Cochrane Centre, Italy; 4 Iberoamerican Cochrane Centre, Spain; 5 German Cochrane Centre, Germany Background: Clinical trials are the backbone of evidence syntheses, such as systematic reviews, guidelines, and health technology assessments (HTA). Accordingly, usage of a clinical trial in evidence syntheses can be used as proxy for quality and the practical value of the trial and hence for the transfer of knowledge. Characteristics of published studies were reported for various fields [1–6], specific journals [7, 8], and the ‘Clinical Trials’ database, formerly known as ‘Cochrane Central Register on Controlled Trials’ (CENTRAL) [9]. A recent paper assessed the countries of origin of studies used for Cochrane reviews [10]. However, only little is known on studies used for HTA reports. These reports have crucial impact on health spending of countries as they are used to inform decisions on reimbursement of drugs, medical devices, and health technologies. Objectives: To examine the characteristics of studies included for HTA reports. Methods: HTA reports by members of the International Network of Agencies for Health Technology Assessment (INAHTA) of four countries (France, Germany, Italy, Spain) will be included. Characteristics of studies (e.g. country of origin, year of publication, number of participants, type of HTA report, type of study, and information on funding, if available) will be extracted independently by two reviewers. Results will be presented quantitatively and adjusted for economic power and size of countries. Results/Conclusions: Preliminary results for Germany (based on 59 reports including 1099 studies) showed the USA as the leading contributor (27%), followed by the UK (7%) and Germany (5%). All countries are dependent on evidence from clinical trials generated in other countries. This is particularly true for continental European countries who contribute fewer trials to the global knowledge pool as the UK and USA do. We will present detailed results for all four countries and discuss possible implications of our findings. References 1. Soteriades ES, Rosmarakis ES, Paraschakis K, Falagas ME (2005) Research contribution of different world regions in the top 50 biomedical journals (1995–2002). Fed Am Soc Exp Biol 20:29–34. 2. de Jong JW, Schaper W (1996) The international rank order of clinical cardiology. Eur Heart J 17:35–42. 3. Rahman M, Haque TL, Fukui T (2005) Research articles published in clinical radiology journals: trend of contribution from different countries. Acad Radiol 12:825–829. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 154 4. Grossi F, Belvedere O, Rosso R (2003) Geography of clinical cancer research publications from 1995 to 1999. Eur J Cancer 39:106–111. 5. Lambers Heerspink HJ, Knol MJ, Tijssen RJW, van Leeuwen TN, Grobbee DE, et al. (2008) Is the randomized controlled drug trial in Europe lagging behind the USA? Br J Clin Pharmacol 66:774–780. 6. Rahman M, Fukui T (2003) Biomedical publication – global profile and trend. Pub Health 117:274–280. 7. Johnson CD, Green BN (2008) Trends in articles published over the past 20 years in the journal of chiropractic education: country of origin, academic affiliation, and data versus nondata studies. J Chiropr Educ 22:4–11. 8. Ozsunar Y, Unsal A, Akdilli A, Karaman C, Huisman TA, et al. (2001) Technology and archives in radiology research: a sampling analysis of articles published in the AJR and Radiology. American Journal of Roentgenology. Am J Roentgenol 177:1281–1284. 9. Gluud C, Nikolova D (2007) Likely country of origin in publications on randomised controlled trials and controlled clinical trials during the last 60 years. Trials 8. 10. Wolff RF, Reinders S, Barth M, Antes G (2011) Distribution of Country of Origin in Studies Used in Cochrane Reviews. PLoS ONE 6(4): e18798. doi:10.1371/journal.pone.0018798 P3A283 How many evidence based guidelines in China interventions for use within NHS mental health services research and 2. to present the results of a pilot study where that measure is used to rate ten different psychosocial interventions for psychosis. Methods: A literature search to identify papers assessing blocks and enablers of implementing interventions within NHS mental health services was conducted. Papers were thematically analysed to identify common themes which specifically related to features of the intervention. Vote counting was used to assess the frequency of each theme across the literature. Themes which were present in three or more papers were retained and turned into an item on the feasibility measure. Scoring for the measure was developed through a consultation process involving researchers and clinicians. The feasibility measure was piloted by two researchers assessing 10 different interventions for psychosis. Results: 95 barriers and facilitators were identified from 11 papers of which 39 themes were specifically related to the intervention and used in the vote counting stage. The final tool included 16 items organised into blocks and enablers. Each item was rated on a likert-type scale. Results of the pilot will be discussed including Inter-rater reliability. Conclusions: The feasibility measure is a novel approach to evaluating a particular intervention and may be used alongside efficacy and health economic evidence from systematic reviews. Rating the feasibility of an intervention during the evidence review stage may assist commissioners, policy makers, and guideline developers with their decision-making process. Yang K1 , Chen Y1 , Yao L2 , Liang H2 , Wang Z2 , Xiao X2 , Wang Q2 1 Evidence-Based Medicine Center of Lanzhou University, China; 2 School of Basic Medical Sciences of Lanzhou University, China P3A285 Logic model as a knowledge translation tool for complex interventions in systematic reviews Background: Little is known about quality and quantity of Chinese clinical guidelines. Objectives: To systematically review all of Chinese clinical guidelines. Methods: We searched for the term guideline in CNKI (China National Knowledge Infrastructure/Chinese Academic Journals full text Database), VIP and WANFANG (two Chinese fulltext databases), and CBM (China Biomedicine Database Disc). Two groups of review authors independently applied inclusion criteria, assessed trial quality, and extracted data. Results: From 1978 to 2010, 397 clinical guidelines were identified of which only 37 (9.3%) claimed that an evidence based approached was used in the process of development; 3(0.8%) provided search strategies and 20 (5%) provided the levels of evidence and the recommendation. Conclusions: There were very few evidence based clinical guidelines in China. Further assessment of all guidelines will be made using AGREE II tool. Augustincic Polec L1 , Ueffing E1 , Welch V1 , Tanjong Ghogomu E1 , Pardo Pardo J1 , Grabowsky M2 , Attaran A1 , Tugwell P3 1 Centre for Global Health, Institute for Population Health, University of Ottawa, Canada; 2 National Vaccine Program Office, Washington, USA; 3 Centre for Global Health, Institute for Population Health, Faculty of Medicine, University of Ottawa, Canada P3A284 Evaluating the feasibility of interventions for use within NHS mental health services Bird V1 , Leamy M1 , Le Boutillier C1 , Williams J1 , Slade M1 1 Institute of Psychiatry, King’s College London, UK Background: Implementation research is an emerging discipline within the field of mental health research. The blocks and enablers of successful implementation within the NHS are currently being researched. The feasibility of an intervention has been identified as one factor linked to its successful implementation. Objectives: 1. To describe the development of a feasibility measure to rate potential Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: Logic models are useful tools that are recommended for equity-oriented systematic reviews. They aim to explain how interventions work and how different factors interact. For this systematic review, a logic model was developed to facilitate the understanding of the complex interventions aiming at increasing the use of insecticide-treated bednets. Objectives: To develop a logic model to frame the research question and guide the systematic review process. Methods: A multi-stage revision process was conducted in which topic experts, stakeholders, and researchers with expertise in systematic reviews, knowledge translation, malaria, and health equity were involved and their feedback incorporated. This multidisciplinary approach ensured that the logic model was examined from different perspectives and that all essential components were included. Results: Initially, a research question was specified based on literature review and consulting stakeholders and malaria experts. Populations, interventions, and outcomes were identified. Classifying complex interventions was the most challenging part of developing the logic model. Building on existing research, interventions were classified in three major groups: interventions focusing on delivery mechanisms, interventions focusing on the proper use of the bednets such as education and peer monitoring, and interventions that combined the first two groups. Intervention categories were further subclassified Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 155 accounting for the channel through which the intervention was delivered, duration of the intervention, cost to the end user, choice of the intervention and accessibility, and sector of delivery. The logic model that was developed guided the systematic review process, and it enabled us to present our research in an understandable manner (Figure 1 and Figure 2). Conclusions: Logic models are useful tools for conceptualizing complex interventions. They can facilitate a systematic review process, and may be used in policy planning and field research. Figure 1 (P3A285). LOGIC MODEL: Strategies to increase the use of insecticide-treated bednets in households and vulnerable populations to reduce morbidity and mortality from malaria in endemic settings. Figure 2 (P3A285). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 156 P3A286 Systematic review of clinical practice guidelines and consensus statements about living kidney donation Tong A1 , de Bruyn J2 , Wong G3 , Chapman J3 , Craig J1 1 University of Sydney, Australia; 2 University of Amsterdam, Netherlands; 3 Westmead Hospital, Australia Background: Clinical practice guidelines (CPGs) and consensus statements provide a basis for implementing research evidence, clinical expertise and patient preferences. In living kidney donation, donors face the risk of acute surgical complications and psychosocial harm; and in practice, there is variability in donor screening and follow up. Objectives: To compare the quality, scope, content and consistency of international CPGs and consensus statements on the assessment and care of living kidney donors. Methods: We searched for CPGs and consensus statements for living kidney donation in electronic databases and guideline registries up to 21 February 2011. The methodological quality of guidelines was assessed using the Appraisal of Guidelines for Research and Education (AGREE) instrument. A descriptive textual synthesis was used to analyse and compare guideline recommendations and consensus statements. Results: Seven CPGs and one consensus statement were included. Three indicated the levels of evidence underpinning the evidence, only one graded their recommendations. Lack of explicitness about the methodology and development of CPGs/consensus statements was apparent. While a comprehensive range of relevant topics were covered, the scope varied considerably between each CPG/consensus statement. Across guidelines, there was slight variability in thresholds for clinical contraindications for live kidney donation. A major discrepancy concerned the acceptance of minor (under 18 years) kidney donors. The importance of informed voluntary consent, genuine altruistic motivation, adequate support, and psychological health and resilience were recognised but specific guidance or tools for conducting psychosocial assessments were lacking. Conclusions: Comprehensive and rigorously developed guidelines which include explicit links to high-quality evidence, graded recommendations, and specific tools for evaluating the medical and psychosocial well-being of living kidney donors may promote the uptake and implementation of evidence and CPGs in clinical practice. This can lead to better screening and care of living kidney donors to minimise acute surgical complications, mortality and psychosocial harms. P3A287 Chinese Cochrane Database of Systematic Reviews of Cochrane Library: A nationwide survey of utilization in Taiwan Chiu YW1 , Wen YH2 , Kuo KN3 1 Institute of Population Health Sciences, National Health Research Institutes, Taiwan; 2 Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Taiwan; 3 College of Medicine, Taipei Medical University, Taiwan Background: The Cochrane Library is the preeminent online database of systematic reviews. The National Health Research Institutes (NHRI) started to translate the abstracts of the Cochrane Database of Systematic Reviews into Chinese (CDSR) in 2007. The abstracts are being summarized in the Chinese Cochrane Database of Systematic Reviews (CCDSR), available at http://clc.nhri.org.tw/admin/clcmain1. aspx since 2008. Objectives: To investigate the behavior in the Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. usage of Chinese abstracts in CDSR. Methods: Online logs were obtained for August 2008 through December 2010. Results: A total of 3,549 abstracts, classified into 53 categories, are available. There was monthly increase in the usage of the CCDSR during 2008 ad 2009. Usage of the CCDSR increased also every year. The average usage rates per month are 325 hits in 2008, 636 hits in 2009, and 6290 hits in 2010. Overall, the most frequent accessed term was kidney, followed by pregnancy and delivery, newborn, and respiratory tract. The most frequently accessed month was April, followed by October. Conclusions: The Chinese abstracts in CCDSR provided by NHRI are helpful for users who prefer to read Chinese and even more so for users who can only read Chinese, such as patients and the public. P3A288 Uptake of Cochrane Acute Respiratory Infections reviews in primary care guidelines van Driel M1 , Thorning S2 , Dooley L2 , van der Wouden H3 1 Bond University, Australia; 2 Cochrane Acute Respiratory Infections Group, Australia; 3 Erasmus University Rotterdam, Netherlands Background: The Cochrane ARI group supports 120 reviews of which many are relevant for primary care. Evidence-based clinical practice guidelines draw upon systematic reviews to formulate recommendations for clinicians. It is unclear how available Cochrane ARI reviews have been cited and used in guidelines. Better knowledge of their uptake and the nature of recommendations derived from the reviews will provide better insight in gaps in the evidence and issues related to translating Cochrane reviews into clinical guidelines. Objectives: To assess how clinical practice guidelines refer to Cochrane ARI Systematic Reviews. Methods: Scopus is the largest abstract and citation database containing both peer-reviewed research literature and quality web sources. We used the citation tracker features of Scopus to identify guidelines that cite systematic reviews published by the Cochrane ARI group in topics that are relevant to primary care. In addition, we searched for nationally endorsed primary care guidelines through professional organisations’ websites in Europe, North America, Australia and New Zealand. We selected citations in clinical guidelines and analysed how the recommendations correspond with the conclusions of the review authors. Results: The project is currently underway and results will be available at the Cochrane Colloquium. Conclusions: Cochrane reviews in the area of acute respiratory infections in primary care are cited in clinical practice guidelines. Tracking and analysing their uptake can identify gaps in the evidence and inform reviewers how to improve transparency of reviews and relevance to primary care practice. P3A289 Scoring trials on the efficacy-effectiveness continuum: A systematic analysis Witt C1 , Manheimer E2 , Lüdtke R3 , Hammerschlag R4 , Lao L5 , Berman B2 1 Institute for Social Medicine and Epidemiology, Charité University Medical Center Berlin; 2 Cochrane Collaboration Complementary Medicine Field, University of Maryland School of Medicine; 3 Carstens Foundation, Essen; 4 Oregon College of Oriental Medicine; 5 University of Maryland School of Medicine Background: ‘Efficacy’ refers to the extent to which a specific intervention is beneficial under ideal conditions. Efficacy produces Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 157 results for an intervention under carefully controlled conditions chosen to maximize the likelihood of observing an effect if it exists. In contrast, ‘effectiveness’ is a measure of the extent to which an intervention, when deployed in the field in routine circumstances, does what it is intended to do for a specific population. To this end, effectiveness trials use eligibility criteria, treatment protocols, and outcomes that are close to usual care. For valid decision making in usual care, there is an urgent need for more evidence from Comparative Effectiveness Research (CER). PRECIS (Thorpe et al. CMAJ. 2009;180(10):E47-E57.) was mainly developed to guide the design of RCTs in 10 dimensions along the efficacy-effectiveness continuum. It is of major interest whether these dimensions can be applied to existing trials as a means of strengthening the evidence base for CER. Objectives: To assess the efficacy-effectiveness continuum of randomized studies using acupuncture for low back pain as an example. Methods: All English language RCTs that compared acupuncture with a conventional treatment control and had >30 acupuncture patients were analyzed by 5 raters using a PRECIS-derived-scale, before and after a consensus process. Results: 10 studies were evaluated with PRECIS (119 abstracts, 44 publications screened). The first rating showed large variance between raters and items (intraclass correlation 0.02–0.60). This was mainly due to missing information in the publications and to difficulties in operationalizing the scoring items. After the consensus discussions, the intraclass correlation improved to 0.20–1.00. Conclusions: To appraise the value of RCTs for CER, clearer operational criteria are needed, raters have to be trained in applying the criteria, and more detailed information is needed when reporting RCTs. P3A290 Interactive tools to teach the public to be critical consumers of information about health care: What’s out there? Krause LK1 , Schwartz L2 , Woloshin S2 , Oxman A3 1 Colorado School of Public Health, USA; 2 The Dartmouth Institute for Health Policy and Clinical Practice, USA; 3 Norwegian Knowledge Center for the Health Services, Norway Background: People are frequently exposed to claims about the effects of health care. To make sense of this information, the public—including consumers, journalists, and decision-makers—need to know how to question and interpret the information presented, and when to seek additional information. Objectives: To create an inventory of interactive tools that teach the public the necessary skills to become critical consumers of health information. Methods: We systematically performed web (Google) and database (Medline) searches, and solicited input from key informants to identify tools which teach the public how to interpret information about the effects of health care. We included tools that are interactive (i.e. allow for some kind of two-way transfer of information), open-access, free, and web-based or downloadable. Potentially eligible tools were screened and coded independently by 2 of the authors. Results: Of the 3,148 potential tools initially identified, 63 were interactive and web-based or downloadable, including 20 which were free and open access (1 from the Medline database search, 9 from key informants, 10 from the web search). Among these, 15 targeted medical professionals/students. Only 5 targeted the public (4 in English, 1 in Italian). Interactive elements of these tools included: Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. multimedia presentations (e.g. narrated PowerPoint presentation, cartoon animation), quizzes (users enter answers and receive feedback), and an annotated newspaper story. Skills covered included: study design, bias, risk, where and how to search, and how to appraise research evidence. Only 1 of the tools underwent formal evaluation for effectiveness (a non-randomized, controlled study). Conclusions: Few free open-access interactive tools are available to help people become critical consumers of information about the effects of health care. P3A291 Partnership to improve health research reporting in Spanish-speaking countries-especially Latin America and the Caribbean Villanueva E1 , Simera I2 , Cuervo LG1 , Altman D3 , Moher D4 , Reveiz L1 1 Policy and Research, Health Systems Based on Primary Health Care, Pan American Health Organization, WDC, USA. 2 Head of Programme Development EQUATOR, Centre for Statistics in Medicine, Oxford, UK; 3 Director, Centre for Statistics in Medicine, University of Oxford, UK; 4 Senior Scientist, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada Background: A large number of studies have critical deficiencies in health research reporting—this seriously compromises further use of new research findings. Effective dissemination and implementation of research findings are compromised when research is reported badly. Studies have shown that research publications frequently lack or distort crucial information, making the assessment of study validity and reliability difficult1 . This is a global problem, including Latin America2 . Objective: To raise reporting standards in Latin America and the Caribbean (LAC) by increasing awareness and implementation of a key collection of reporting guidelines available in Spanish and hosted in the EQUATOR website recently translated into Spanish. Methods: Using reporting guidelines, checklists that specify minimum information necessary for a complete and clear account of research methodology and findings, can substantially reduce deficiencies in research literature. The EQUATOR Network, recently translated into Spanish, is an international initiative that aims to increase the quality of health research reporting by promoting the use of reporting guidelines. EQUATOR provides online resources for authors, editors, peer reviewers, and guideline developers. Results: The EQUATOR portal hosts over 100 reporting guidelines, listed by study type, and other resources supporting responsible reporting. A growing number of journals refer their authors to this resource. The website use has been increasing every year but predominantly by the English speaking countries (United States, United Kingdom, and Canada). Making resources available in Spanish should increase understanding of the principles of good research reporting and improve the quality of Spanish scientific publications. EQUATOR supports the use of guidelines through training courses for editors, peer reviewers and researchers. The new partnership will allow development of education activities that address language specific needs. This collaboration should significantly improve dissemination and implementation of reporting guidelines in LAC and positively influence standards of research reporting and more efficient use of published research. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 158 P3A292 Evidence-based practice in first aid and prevention guidelines of Belgian Red Cross-Flanders Pauwels NS1 , Dieltjens T1 , Van de Velde S1 , De Buck E1 1 Belgian Red Cross-Flanders, Mechelen, Belgium Background: In the past, first aid handbooks for lay public were primarily based on experience and expert knowledge. The Belgian Red Cross-Flanders decided to use an evidence-based approach when revising its first aid programme. The curriculum consists of diverse topics related to first aid and prevention of sudden illness and trauma, issued in didactical handbooks for diverse target groups. Objectives: To provide evidence-based recommendations in a limited time span, 12 months, for 151 diverse topics. As a reference, the Scottish Intercollegiate Guideline Network (SIGN) defines 40 questions as highly elaborate and two year projects are common. Methods: To address these many topics in a timely and cost-conscious manner, a pragmatic approach to develop the guideline was used. To identify the best current evidence, reviewers performed literature reviews for each topic (one reviewer/topic). If an eligible guideline or systematic review was found, we formulated draft recommendations based on this evidence. If not, we searched controlled intervention studies. When no study was retained, the search continued for cohort and case-control studies. Databases searched included Guideline International Network Library, National Guideline Clearinghouse, Cochrane Library, BestBETs, Medline, and Embase. The quality of evidence was graded using GRADE. A multidisciplinary panel of 11 experts discussed draft recommendations while taking into account the evidence. Results: During a one year period, the reviewers devoted 2 full-time equivalents (FTE) to prepare the evidence summaries. No relevant evidence was found for 49 of the 151 topics and for the other topics, the available evidence was limited. Eight panel meetings (3-4h) were needed to finalise the recommendations. Conclusions: Despite the massive number of topics, the first aid curriculum was reviewed within a limited time frame. For the first time, the integral first aid curriculum of the Belgian Red Cross-Flanders systematically incorporates evidence and expert judgment. P3A293 How evidence-based are the pharmaceutical industry’s printed promotional material presented to physicians in three Latin American countries Ciapponi A1 , Reveiz L2 , Glujovsky D1 , Garcı́a Martı́ S1 , Rey Ares L1 , Bardach A1 , Ramı́rez Moreno L3 , Ferreira AM3 , Rubinstein F1 , Rada G4 1 IECS – Institute for Clinical Effectiveness and Health Policy. Argentina; 2 Andean Cochrane Branch of the Ibero-American Cochrane Center; 3 Clı́nica Reina Sofı́a, Bogotá Colombia; 4 Department of Internal Medicine. Faculty of Medicine, Pontificia Universidad Católica de Chile Background: A major marketing technique used by pharmaceutical companies is direct-to-physician marketing. Of these, one of the most used is the printed promotional material (PPM). Objectives: To determine if the references provided by printed promotional material (PPM) presented to physicians by pharmaceutical representatives support its statements and to assess the validity of the PPM against the best available evidence. Methods: A consecutive sample of all the PPM distributed by pharmaceutical representatives was collected Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. from January 2010 to March 2011, in a family medicine practice in Buenos Aires, Argentina; in a general medicine practice in Bogotá, Colombia; and in an internal medicine ambulatory practice in a public hospital in Santiago de Chile, Chile. Reprints and monographs were excluded. The number and accuracy of PPM’s statements supported by cited references; and the correctness of references cited in PPM according to the International Committee of Medical Journal Editors guidelines was evaluated. The retrievability of the references was also assessed. Adverse reactions/warnings about drug interactions and contraindications were collected. Relevant clinical information of the material was translated to PICO format questions to systematically search the best related evidence (systematic reviews [SR] or RCTs) in: 1) The Cochrane library 2) MEDLINE. We assessed the risk of bias (ROB) of identified SRs and RCTs using the AMSTAR instrument and the Cochrane Handbook respectively. PPM agreement with citation/best evidence was classified as ‘concordant’, ‘partially concordant’ or ‘discordant’. Pairs of independent reviewers graded the citations’ ROB and the quality of the evidence using the GRADE system. Results: See Table 1. Conclusions: -2/3 PPMs citations have serious flaws (High ROB, discordant or reference unavailable); -50% of statements are supported by the citation, and 30% are concordant with the best available evidence; Cochrane SR are rarely cited by PPMs. -Physicians should not trust in PPM and should demand they be regulated. P3A294 Strategies to reduce attrition in randomised trials: a methodology review Brueton V1 , Tierney J2 , Stenning S2 , Nazareth I1 , Meredith S2 , Harding S3 , Rait G1 1 MRC GPRF, UK; 2 MRC CTU, UK; 3 MRC SPHSU, UK Background: Attrition from randomised trials can introduce bias and reduce statistical power affecting the generalizability, validity, and reliability of results. Many strategies are used by researchers to reduce attrition such as motivating and engaging participants and adding sites to generate optimal data return or compliance to follow-up procedures. Objectives: To quantify the effect of strategies to reduce attrition from randomised trials in any health care setting. Methods: Included studies were randomised evaluations of strategies to reduce attrition embedded within randomised trials from all diseases and clinical settings. The following sources were searched for eligible studies: MEDLINE (1950 to present), EMBASE (1980 to present), PsycINFO (1806 to present), DARE (most recent issue), CENTRAL (most recent issue), CINAHL (1981 to present), C2-SPECTR (most recent date), and ERIC (1966-present), Cochrane Methodology Register, Current Controlled Trials metaRegister, WHO trials platform, Society for Clinical Trials (SCT) conference proceedings (1980–2010), and publication reference lists. A survey of all UK clinical trials units was conducted to identify ongoing or completed but unreported studies. Data from trials were pooled using the fixed effect model and heterogeneity explored to determine the level of effectiveness of the type of strategies used and the study design to reduce attribution. Results: From 19,281 abstracts 32 unique RCTs were identified from the following sources: MEDLINE, CENTRAL, CINAHL n = 9; SCT abstracts 1980–2010 n = 4; reference lists of relevant reviews n = 7; and of included trials n = 8 (7 duplicates); reference lists of excluded trials n = 1; word of mouth n = 4; survey of CTUs n = 6. Five databases returned no Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 159 Table 1 (P3A293). Citations and best evidence supporting Printed Promotional Material (PPM) statements. RCTs. Included RCTs evaluated 7 types of strategies to reduce attrition including: communication, questionnaire length, incentives, participant case management, visit frequency, behavioural, and methodological interventions. P3A295 Do the European clinical guidelines recommend evidence based interventions for drug addiction? A benchmark exercise Ferri M1 , Bo A1 , Allara E2 1 EMCDDA Lisbon; 2 University of Turin, Italy Background: An increasing body of evidence for interventions in the Drug Addiction field is becoming available due to the decennial activity of the Cochrane Group on Drugs and Alcohol. The European Monitoring Center for Drugs and Drug Addiction (EMCDDA) created an inventory of National Guidelines for the Treatment of Drug Addiction by collecting information on 30 European Countries. A preliminary analysis of the guidelines provided an overview of of the recommendations for treatment. It also offers the basis for further efforts aimed at promoting an evidence-based approach across Europe in this area of work. Objectives: To describe the existing guidelines for drug addiction treatment in 30 European countries in terms of target clients, interventions and to perform a benchmark exercise focused on 15 recommendations. Methods: Guidelines were collected through a two-round survey and an international experts meeting. The Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. EMCDDA’s network of national focal points performed a benchmark exercise comparing 15 recommendations from an evidence-based guideline (WHO 2009) with similar recommendations in their National Treatment Guidelines. Results: 141 national treatment guidelines from 27 out of 30 countries were identified and obtained in full text (availableat http://www.emcdda.europa.eu/best-practice). Almost half of them (66/141) are about opioid dependence and 55 have Methadone Maintenance as the main treatment. Overall recommendations are similar across guidelines such as the recommendation for an integrated package of long term substitution treatment combined with psychological support. Difference exists in the process for the development of the guidelines as some of them are evidence-based while others use experts consensus. Divergence exists with regard to the choice between Buprenorphine or Methadone for first-line treatment and the drugs dosages. Conclusions: The study is a collection and inventory of practice guidelines for the treatment of drug addiction. Further studies are needed to assess the methodological validity of the guidelines by the mean of standardized tools such as AGREEII. P3A296 Progress in time to update of Cochrane Systematic Reviews Ngamjarus C1 , Jaidee W2 , Laopaiboon M1 1 Faculty of Public Health, Khon Kaen University, Thailand; 2 Faculty of Public Health, Burapha University, Thailand Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 160 Background: Up-to–date evidence relevant to prevention and treatment of any health-related events are important information for healthcare decision-making and services. The Cochrane Collaboration policy is that Cochrane systematic reviews should be updated within two years. However, development of preventive and therapeutic interventions can changed and differed over timeSystematic reviews of the 52 Cochrane review groups may be updated at different time periods. Objectives: This paper will present time to updating systematic reviews of Cochrane review groups. Methods: The original cohort of Cochrane systematic reviews firstly published in the Cochrane Library between January 2000 and December 2005 were studied. Update versions of the cohort reviews until 2011 issue 4 were independently searched by two authors in the Cochrane Archie server and Cochrane Library. We defined ‘update’ following the suggestions in the Cochrane handbook. Time to update was measured from the date of current update version and the date of previous consecutive version. Disagreement between the two authors throughout the review was solved by consensus and discussed with the third author. The median survival time of updating and its 95% confidence interval (CI) was analyzed by using STATA software. Results and Conclusions: There were 623 Cochrane reviews from 47 review groups. Median survival time to update of the reviews was 1.8 years (95% CI 1.6, 1.9 years). Fifty five percent of them were lastly updated before 2 years. The top three groups that had large number of update reviews were Airways Group (56 reviews), Pregnancy and Childbirth Group (46 reviews), and Neonatal Group (40 reviews) with median survival time to update equal to 2.0 years (95% CI 1.4, 2.6 years), 1.6 years (95% CI 0.6, 2.5 years) and 1.7 years (95% CI 1.0, 2.5 years), respectively. P3A297 Bringing evidence into practice: Use of a rapid review methodology to formulate medical recommendations for the Red Cross Blood Service De Buck E1 , Van de Velde S1 , Dieltjens T1 , Vandekerckhove P1 1 Belgian Red Cross-Flanders Background: The Belgian Red Cross-Flanders is active both domestically and internationally in several fields: from blood supply to emergency aid. The central thread in our strategic plan is to have evidence-based program. In our field of work, no systematic reviews or evidence-based guidelines are available yet. A rapid review methodology was developed to promote the use of evidencebased in our field of work. Objectives: To inform about the application of evidence-based practice in an action-oriented organization. We illustrate our methodology with a specific question from the Blood Service, i.e. is it safe to use blood from hemochromatosis patients for blood donations. These patients frequently need bloodlettings for medical reasons. Methods: We used a rapid review methodology and not a full systematic review. The methods were based on published literature about this type of reviews. One reviewer reviewed a restricted number of using specific search strategy and criteria. Grey literature was excluded. GRADE was used to determine the quality of evidence. An expert panel considered the evidence and formulated recommendations for practice. Results: Six observational studies were relevant to our question. The strength of the body of evidence was low to very low. The available evidence indicates that blood from hemochromatosis patients, taken during maintenance Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. therapy, comply with quality requirements and is safe to be used for blood transfusion. Conclusions: To shift from an implementation and execution organization to one which actively sets the trend, the Belgian Red Cross introduced evidence-based practice in its activities. We gave the example of hemochromatosis, because it changed the blood donation procedures. A limitation of the rapid review is that the conclusions are tentative and may be subject to change once a systematic review is available. P3A298 Deficient reporting and interpretation of non-inferiority clinical trials in antiretroviral-naı̈ve HIV patients: A systematic review Hernandez AV1 , Pasupuleti V1 , Deshpande A1 , Collins JA2 , Vidal JE3 1 Cleveland Clinic, Ohio, USA; 2 Hospital Guillermo Almenara-EsSalud, Lima, Peru; 3 Sao Paulo University School of Medicine, Sao Paulo, Brazil Background: Non-inferiority (NI) randomized clinical trials (RCTs) commonly evaluate efficacy of new antiretroviral (ARV) drugs in ARV-naı̈ve HIV patients. Their reporting and interpretation have not been systematically evaluated. Objectives: To evaluate the reporting of NI RCTs of new ARV drugs in ARV-naı̈ve HIV patients according to the CONSORT statement; to evaluate the degree of misinterpretation of RCTs when NI was inconclusive or not established. Methods: Systematic review of NI RCTs evaluating drugs in ARV-naı̈ve HIV patients. Pubmed, The Web of Science, and Scopus were reviewed until March 2011. Selection and extraction was independently done by two reviewers. Key reporting information included: similarity to prior RCTs of the active comparator, description of method to determine the NI margin, use of confidence interval (CI) method to interpret the primary outcome, use of blinding, and use of intention-to-treat (ITT) vs. per protocol (PP) statistical analysis. When NI was inconclusive or not established, we evaluated whether authors highlighted NI and distracted readers with secondary results. Results: Fourteen RCTs were selected (range 71–3316 patients). None of the RCTs gave information about prior RCTs of the active comparator, and all used 2-sided CIs. All studies described the NI margin between 10% and 15%, but only 5 explained the method to determine it. Blinding was used in 4 studies, and the appropriate PP was the primary analysis in 4 studies. Five studies with NI inconclusive or not established highlighted NI or equivalence, and they distracted readers with positive secondary results in the abstract, results and conclusions. Conclusions: There is poor reporting and interpretation of NI RCTs done in ARV-naı̈ve HIV patients. Maximizing the reporting of the method of NI margin determination, the use of blinding and per-protocol analyses, and interpreting negative NI according to actual primary findings will improve the understanding and translation of results into clinical practice. P3A299 Setting priorities for comparative effectiveness research on the management of primary angle closure (PAC): A survey of Asia-Pacific clinicians Yu T1 , Li T1 , Puhan M1 , Dickersin K1 1 Johns Hopkins Bloomberg School of Public Health, Epidemiology Objective: To set priorities for new systematic reviews on the management of primary angle closure (PAC), using clinical practice Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 161 all) to 10 (highly important). To assess understanding, we first asked two knowledge assessment questions about the survey objective, and allowed retries if the response was not correct. Respondents were randomized to one of two questionnaires, which had different orders of the same questions. The survey and instructions were in English. Results: 175 agreed to participate in the survey, 132 responded (75.4% response rate), and 96 completed the questionnaire guidelines and a survey of clinicians from the Asia-Pacific region. Methods: We restated the American Academy of Ophthalmology’s Preferred Practice Patterns (AAO PPPs) recommendations for management of PAC into 42 answerable clinical questions. We asked registrants at the Asia-Pacific Joint Glaucoma Congress 2010 in Taipei to rate the importance of having an answer to each question for good patient care, using a Likert-type scale from 0 (not important at 9 Figure 1 (P3A299). Scatter plots of rating results for 42 clinical questions using questionnaires (I and II) with different orders of the same questions (a) Scatter plot of the mean ratings assigned in questionnaire I and questionnaire II for 42 clinical questions (with 45-degree line); the mean ratings are generally lower in questionnaire II compared to questionnaire I. (b) Satter plot of the rankings (ranked by mean ratings) in questionnaire I and questionnarire II for 42 clinical questions. The question with the highest mean rating is ranked 1. Numbers plotted on the figure refer to question numbers in questionnaire I. 8 Pearson correlation coefficient = 0.77 9 37 38 5 10 23 4 3 24 21 7 14 13 26 1642 34 33 35 36 12 32 6 15 41 19 11 1 6 7 39 1718 20 8 22 2 31 30 29 25 28 5 27 40 5 6 7 8 Mean of the ratings assigned in questionnaire I 9 9 1 (a) Spearman rank correlation coefficient = 0.73 38 5 10 6 23 4 11 24 21 3 14 16 16 21 26 26 31 12 36 17 25 27 40 41 1 22 2 42 34 33 15 35 11 8 13 41 37 41 6 19 7 36 32 39 18 20 31 30 29 28 36 (b) Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. 31 26 21 16 11 6 1 Ranking in questionnaire I Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 162 (54.9% usable response rate). For those who responded but did not complete, half stopped after the two knowledge assessment questions. Of 96 who completed, only 29 (30.2%) correctly answered the knowledge assessment questions on the first attempt. The distribution of importance ratings pooled from all questions was skewed to high importance (mean = 6.92; median = 8). The order of questions affected the importance ratings (Figure 1). We considered clinical questions to be of priority if at least 75% of respondents gave ratings above 5. Conclusions: Understanding the survey purpose and questions affected our response rate, and question order affected question importance rating. We identified a list of clinical questions on the management of PAC for which research should be prioritized. Where evidence does not exist, primary research and systematic reviews should be initiated to address these questions. P3A300 Course ‘Hot topics in pharmacotherapy’: A way to disseminate evidence Garjón Parra J1 , Erviti López J1 , Azparren Andı́a A1 , Gorricho Mendı́vil J1 , López Andrés A1 1 Servicio Navarro de Salud, Servicio Prestaciones Farmacéuticas, Spain Background: There is in dire need for up-to-date clinical information for doctors. However, medical education is provided to a great extent by pharmaceutical companies and hence there is a high risk of biased information. A new way to provide evidence-based education on pharmacotherapy to a wide audience of health professionals is needed. Objectives: To present controversial topics in drug therapy. To provide useful evidence-based education for patient care providers. To demonstrate the need for evidence-based pharmacotherapy and to get practitioners involved in evidence-based decision making. Methods: An annual two-day Drug Therapy Course organised by the Drug Information Unit of the Navarre Health Service in Spain took place in 2009. Five topics were addressed in each of the two scheduled sessions (10 topics total). Criteria for selection of themes: sharply defined, current controversy or new evidence, wide interest, variety. Features of the course:-Accessibility: Free, scheduled during off-duty days. Accredited for continuing medical education. -Peer teaching: Lecturers were professionals involved in day-to-day patient care or knowledgable about pharmacotherapy. Lecturers who were solely academics or researchers were not considered. Group interaction was promoted. Length or time: Short-duration conferences consisting of a 20-minute presentation followed by a 10-minute discussion. -Evaluation of each conference. Cost-consciousness: Lecturers were paid ¤90 for each presentation. Conference room was provided for free by the Government of Navarre. All educational materials were available in the web-site, www.bit.navarra.es. Independent: Free of sponsorship. Results: Participants: Year 2009: 140; Year 2010: 127; Year 2011: 170. Profession of attendees (Figure 1). Conferences global evaluation (122 responses): Bad: 1%; Fair: 14%; Good: 54%; Very good: 31%. Conclusions: It is feasible to provide education that is independent of pharmaceutical company sponsorship on pharmacotherapy to a wide audience of health professionals in a constrained resources setting. There is good acceptability of short conferences about new or controversial topics given by non-academic clinicians in an enviroment that facilitates participation. Figure 1 (P3A300). Profession of registered attendees. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 163 P3A301 Do systematic reviews within EPOC contain sufficient information about effect and enough information about the intervention to facilitate informed policy decision-making? Holte HH1 1 Norwegian Knowledge Centre for the Health Services, Oslo, Norway Background: Cochrane systematic reviews are intended to be part of the documentation that is evaluated prior to making a policy decision on how to improve local health services. Policy makers need to know if the intervention is effective relative to alternatives. Descriptions of the intervention need to be sufficiently detailed to evaluate if it is relevant and feasible to implement in a local setting. When reading an Effective Practice and Organisation of Care (EPOC) review, is there sufficient information to decide if it is relevant and feasible to implement the intervention that was reviewed in Norway’s local setting? Methods: All the new reviews and updated reviews published in the Cochrane Database of Systematic Reviews by the EPOC-group from January 2010 to January 2011 were examined. The information on the effect, the relevance, and the feasibility of the intervention were first considered by examining the title of the review, then the abstract. If it was still deemed to be potentially relevant and feasible, then full review was read. Results: Of the 28 reviews that were indentified, two reviews were specific to low and middle income countries and two reviews had no included studies. Sixteen reviews did not draw a conclusion for various reasons. Of the eight reviews reporting an effect of the intervention, none gave sufficient information that permitted me to think that the intervention could be implemented in a local setting in Norway. Discussion: The potential global applicability of Cochrane reviews is one of the positive aspects of Cochrane systematic reviews. EPOC reviews have special challenges because of the large variation in national organizations of health care, and the relatively larger impact of setting. Systematic reviews must include more information about the interventions and the settings to make it possible to consider implementation at local levels. P3A302 How and when to pool data in meta-analyses to make the results clinically meaningful Ringerike T1 , Klemp M1 1 Norwegian Knowledge Centre for the Health Services, Norway Background: Meta-analyses can be useful in deciding whether or not an intervention is feasible. However, it is subject to discussion which data should be included. An inclination to pool as much data as possible to get a sturdy estimate must be balanced against a possible increase in heterogeneity and difficulties in interpretation of the results within the clinical context. We highlight different choices which have to be made when comparing rivaroxaban to enoxaparin, drugs which are used as prevention of deep venous thromboembolism (DVT) in patients undergoing elective total hip or knee replacement surgery. Objective: The objective is to raise awareness regarding which choices have to be considered when pooling data and assessing what are their possible impact on the results and interpretation. Methods: A systematic literature search was performed. Data were extracted and pooled in different ways to give risk ratios (RR) and 95% confidence intervals (CI). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Results: An overview of the data for prevention of DVT is presented in Figure 1. Pooled data for hip replacement surgery, regardless of dose and treatment duration gained RR 0.46 (0.26–0.84). However, the approved dose of rivaroxaban is 10 mg and limiting the analysis to this gave RR 0.48 (0.23–0.98). Finally, the pooled effect estimate based on the doses and treatment durations representative of clinical use within EU countries is limited to one study with an RR 0.28 (0.16–0.49). On the other hand, instead of narrowing the focus one might want to expand to pool data regardless of hip or knee replacement therapy. Conclusions: Always adjust the meta-analysis according to the research question. Use of subgroups may facilitate getting an overview of the data, interpretation and extrapolation to similar research questions. P3A303 Long-term harm and memory impairment in electroconvulsive therapy treatment for depression: preliminary results from a systematic review using an integrative methodological approach Leiknes KA1 , Høie B1 , Jarosch-von Schweder L2 , Berg R1 , Smedslund G1 1 Norwegian Knowledge Centre for the Health Services, Norway; 2 Department of Research and Development, St. Olavs University Hospital, NTNU-Faculty of Medicine, Department of Neuroscience, Trondheim, Norway Background: A major concern in the use of electroconvulsive therapy (ECT) for depression is the potential long-term harm on cognitive (memory) function. Although evidence from controlled clinical trials suggest that memory impairment may not last over six months subjective reports of long-lasting memory loss have been discovered, with long-lasting memory loss ranging from 29% to 79%. Results from systematic reviews of trials with long-term within subjects, pre-ECT and post-ECT neuropsychological test data have thus been called upon. Objectives: To present preliminary results from included non-randomized (non-RCT) trials, concerning long-term (>6 months) cognitive functioning after ECT for depression, registered Cochrane review, using a novel, integrative methodological approach. Methods: A broad systematic literature search on ECT for depression was conducted. Medline, PsycINFO, Cinahl, SveMed+, Cochrane Central, CCDAN-CTR, Embase, British Nursing Index and Ovid Nursing databases were searched. All identified non-RCTs concerning ECT administered to adults (age over 15 and below 65 years, at baseline) for major depressive disorder (MDD) with a within subjects pre-ECT and post-ECT neuropsychological testing of cognitive (memory) function were full-texted screened. Mixed diagnostic populations (e.g. bipolar disorder, schizophrenia) and other disorders where MDD data were not separable were excluded. Results: Overall 132 non-RCT studies concerning ECT and cognitive functioning were screened and 68 were identified as containing neuropsychological tests to be examined. This yielded only 22 full text to be included. Main reasons for exclusions were mixed diagnostic study sample and old age (over 65 years), but also use of non-standardized cognitive instruments and insufficient data report. Diversity of neuropsychological test batteries for verbal, non-verbal, autobiographical and subjective memory makes meta-analyses and aggregation of the study data almost impossible. Conclusion: In spite of the seemingly abundance of non-RCT trials concerned with ECT for depression and cognitive functioning, long-term neuropsychological data for specifically ECT and MDD are still very sparse. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 164 Figure 1 (P3A302). P3A304 Use of indirect comparison methods in systematic reviews: A survey of Cochrane review authors’ views Abdelhamid A1 , Loke YK1 , Parekh S2 , Chen Y3 , Sutton A4 , Eastwood A5 , Holland R1 , Song F1 1 University of East Anglia, UK; 2 NETSCC- University of Southhampton, UK; 3 University of Birmingham, UK; 4 University of Leicester, UK; 5 University of York, UK Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: Many systematic reviews have evaluated multiple interventions for a particular condition, but such reviews often do not formally compare the relative effectiveness of interventions through adjusted indirect comparison (AIC) or network meta-analysis. The use of AIC has recently attracted growing interest. Objectives: To investigate the views and knowledge of Cochrane systematic review (CSR) authors regarding the use of AIC. Methods: An online survey was sent to 84 CSR corresponding authors of reviews we had identified as reviews in Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 165 which direct comparisons (DC) as well as AIC of interventions would have been possible. Authors were also sent results of DC and AIC compiled using data from their particular review. Results: Response rate was 57%. Respondents were academics (54%), clinicians (27%) or with equally shared roles (19%). When conducting the review in question, only one quarter of authors had known about AIC methods and have used them while over a third (35%) had knowledge of the methods but had not used them. Forty percent (40%) either did not know about AIC or had only heard about it. The majority of authors (76%) would, at least sometimes, consider evidence from AIC. Some authors expressed suspicion of the methods. Most authors (89%) felt they needed more training, especially in assessing the validity of AIC. Almost all authors felt that AIC could potentially be influenced by a huge number of effect modifiers. Some respondents accepted that use of AIC is needed as it may be the only source of information for relative effectiveness of competing interventions, provided that authors and readers are conscious of its limitations. Time commitment and resources needed were identified as important concerns for Cochrane reviewers. Conclusions: There is mostly an acceptance of the increasing demand for AIC as well as an urgent need to develop structured guidance and training for its use and interpretation. P3A305 Generation of quality of care indicators based on systematic reviews Osorio D1 , Aller M1 , Roqué M2 , Foradada C3 , Vives A4 , Rigau D2 , Bonfill X2 1 Institut d’Investigació Biomèdica Sant Pau (IIB Sant Pau), Spain; 2 Iberoamerican Cochrane Centre, Spain; 3 Consorci Sanitari Parc Taulı́, Spain; 4 Consorci Sanitari de Terrassa, Spain Background: Systematic and explicit methods to combine scientific evidence with professional consensus for assessing the appropriateness of care have been developed to a limited extent. Obstetric care during childbirth is suitable to further develop such methods because it is both a field with a high production of systematic reviews (SRs) and where many indicators currently used may be questionable or may need to be reassessed/reexamined. Objectives: To generate a set of indicators to assess the appropriateness of obstetric care related to childbirth based on evidence from SRs. Methods: We reviewed all the SRs from the Cochrane Pregnancy and Childbirth Group (Cochrane Library Issue 3, 2009) and searched for Clinical Evidence and main guidelines to identify further SRs. We selcted interventions supported by high quality evidence (of a benefit or harm) and from which a strong recommendation could be obtained using GRADE methodoloy. A group of experts partipated in the recommendation generation and provided feedback on the indicators’ content validity and reliability. Results: We identified 303 SRs. After excluding SRs about non-acute care, not delivered in hospital or without clear evidence (either in favour or against), we select 48 SRs that generated 21 strong recommendations based on high quality evidence. The group of experts considered valid and reliable a total of 18 quality indicators: eight referred to prepartum care, eight referred to delivery, one referred to postpartum and one referred to incomplete miscarriage. These quality indicators are currently being tested for their validity, reliability and feasibility in a pilot study. Conclusions: It is possible and feasible to develop evidence-based quality indicators to measure the appropriateness of care in Obstetrics. High-quality SRs that explicitly express support for or against an intervention are reliable and accessible tools for generating Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. indicators of quality of care, once considered to be the nuances of clinical practice, and help to make the indicators coherent. P3A306 Graphic display for effective reporting of evidence quality tables Khan K1 , Mignini L2 , Borowiack E3 , Ross C4 , Kowalska M3 , Zapalska A3 , Mol B5 , Meads C1 , Walczak J3 1 for Health Sciences, Barts and The London School of Medicine and Dentistry, Queen Mary University of London. UK; 2 Centro Rosarino de Estudios Perinatales, Rosario, Argentina; 3 Arcana Institute, Plk Dabka, Cracow, Poland; 4 Department of Obstetrics and Gynaecology, Radboud University Nijmegen Medical Centre, Nijmegen, Netherlands; 5 Department of Obstetrics and Gynaecology, Academic Medical Center, University of Amsterdam, Amsterdam, Netherlands Background: When generating guidelines, quality of evidence is frequently reported in tabulated form capturing several domains, e.g. study design, risk of bias, and heterogeneity. Increasingly this is done using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach. A more user-friendly format is needed. One example is a graphic display plotting the several domains used in evidence grading on equiangular spokes starting from the same point, the data length of each spoke proportional to the magnitude of the quality, which succinctly captures tabulated information. Objectives: To allow easy identification of deficiencies, outliers and similarities in evidence quality for individual and multiple comparisons and outcomes, paving the way for their routine use alongside tabulated information. Methods: We tabulated findings separately for tocolytic agents compared to placebo and those compared to beta agonists. For each comparison and outcome pair evidence was initially graded by the study design. We assigned all evidence a ‘high’ level of quality if it was based on randomised controlled trials. If any of the reasons below applied to the body of evidence for each comparison-outcome pair, the quality level was downgraded by one level (if the reason was classified as serious) or by two levels (if the reason was classified as very serious): bias, inconsistency, indirectness, imprecision. Results: Figures 1 and 2 provide a visually striking display of the strengths and weaknesses of the evidence across the spectrum of comparisons and outcomes which, when tabulated, is dry, cumbersome to read, and difficult to assimilate. Conclusions: There is considerable opportunity for confusion when moving from evidence syntheses to recommendations. The graphic approach we have developed can make the job of evidence assimilation for decision-making easier. When guideline statements are accompanied by these plots, this approach also has the potential for improving the credibility of the recommendations made as the strength of the evidence is more clearly displayed. P3A307 Mapping research gaps in interventions related to Millennium Development Goal (MDG) 5-A (maternal mortality) in the ‘Implication for Research’ section of Cochrane reviews Higgins V1 , Chapman E2 , Reveiz L2 , Cuervo LG2 1 International Heath Department, School of Nursing and Health Studies, Georgetown University, WDC, USA; 2 Public Policy and Research, Health Systems Based on Primary Health Care, Pan American Health Organization. WDC, USA Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 166 Figure 1 (P3A306). Graphic display of evidence quality on tocolytic efficacy compared to placebo or no treatment in randomized trials. Comparisons vs placebo or no treatment2 Magnesium Outcomes1 Atosiban3 Beta agonists Rank according to % of respondents rating critical or important 3 2 1 95% Perinatal morbidity6 0 3 2 1 95% Safety to mother7 0 3 91% Perinatal mortality 91% Stop birth before 34 weeks 85% Undelivered at 24 hours 82% Undelivered at 48 hours 55% Stop birth before 37 weeks Indomethacin NO Donor5 & Ca Blocker5 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 3 2 2 2 1 1 1 0 0 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 Design 3 Imprecision 2 1 Risk of Bias 0 Indirectness No data reported Inconsistency Each graph represents the quality domains shown on concentric spokes. For each of the spoke, the length represents the magnitude of quality ranging from very low at the centre of the plot to high at its maximum length. Concentric lines rippling out of the centre show quality increasing to low and then to moderate before reaching the maximum value of high. Evidence quality tabulated in appendix 2, tables A-D. 1 Importance of outcome colour coded for ease of display. Ranking of importance undertaken through formal survey of over 500 practitioners asking them to provide responses on a scale anchored between critical at one extreme and not at all important at the other. 2 Presented in alphabetical order except Ca Blocker which is presented along with Magnesium sulphate. 3 Atosiban trade name Tractocile®, which is now not protected by patent. 4 NO = Nitric oxide. 5 Ca = Calcium channel. 6 Based on neonatal admission to intensive care unit. 7 Based on maternal drug reaction requiring treatment cessation. Figure 2 (P3A306). Graphic display of evidence quality on tocolytic efficacy compared to Beta agonists in randomized trials. Comparisons vs Beta agonists2 Outcomes1 Rank according to % of respondents rating critical or important 95% Perinatal morbidity6 95% Safety to mother7 91% Perinatal mortality 91% Stop birth before 34 weeks 85% Undelivered at 24 hours 82% Undelivered at 48 hours 55% Stop birth before 37 weeks Atosiban3 Ca Blocker4 Indomethacin NO Donor5 Magnesium 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 Design 3 Imprecision 2 1 0 Indirectness Risk of Bias Inconsistency No data reported Each graph represents the quality domains shown on concentric spokes. For each of the spoke, the length represents the magnitude of quality ranging from very low at the centre of the plot to high at its maximum length. Concentric lines rippling out of the centre show quality increasing to low and then to moderate before reaching the maximum value of high. Evidence quality tabulated in appendix 2, tables E-I. 1 Importance of outcome colour coded for ease of display. Ranking of importance undertaken through formal survey of over 500 practitioners asking them to provide responses on a scale anchored between critical at one extreme and not at all important at the other. 2 Presented in alphabetical order except. 3 Atosiban trade name Tractocile®, which is now not protected by patent. 4 Ca = Calcium channel. 5 NO = Nitric oxide. 6 Based on neonatal admission to intensive care unit. 7 Based on maternal drug reaction requiring treatment cessation. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 167 Introduction: Achieving MDG-5A requires a concerted effort and political will for the implementation of strategies and cost-effective interventions based on scientific evidence to improve the health of women. Cochrane reviews could be useful to the identification of research priorities. Objective: To assist research sponsors and developers in achieving MDG-5A by mapping and prioritizing relevant research gaps identified in the ‘Implication for Research’ section of Cochrane reviews. Methods: Phase 1: We used an advanced search strategy in PUBMed limited to the Cochrane Database of Systematic reviews, and included reviews assessing interventions likely to reduce maternal mortality if they were published or updated during or after 2006. We evaluated the ‘Implication for Research’ section of each review and developed questions in ‘PICO’ (spell out) format according to what Cochrane reviewers presented, then categorized questions according to intervention type (Table 1). Phase 2: A group of experts will prioritize the list of identified questions by selecting and ranking each question based on specific criteria. We will verify that no new clinical trials address these questions, by identifying randomized clinical trials listed in PUBMED, the Cochrane Library, and the International Clinical Trials Registry Platform (ICTRP). We will provide a list of unanswered questions that remain relevant to the achievement of MDG5-A (maternal mortality). Results: We identified 695 references during phase 1 and included 203 reviews that met our inclusion criteria. More than 300 questions were structured and classified by intervention type (Table 1). An analysis of advanced phases of this study will be presented at the Cochrane Colloquium. Conclusion: By organizing and presenting pre-identified knowledge gaps, we will provide helpful information for research sponsors, funding and development agencies, and researchers interested in addressing MDG-5A. We expect this contribution to act as a catalyst for the development of the essential research needed for the reduction of maternal mortality. Table 1 (P3A307). PICO questions according to intervention. Number (%) of PICO questions (N = 312) Type of Intervention General 31 (9,9%) Pregnancy Prevention 20 (6.4%) Education/Training of Women 11 (3.5%) Specific 237 (76.0%) Post-Partum Hemorrhage 25 (8%) Abortion/Miscarriage 25 (8%) Hypertensive Disorders 27 (8.7%) Infection/Sepsis 11 (3.5%) Caesarean Section 16 (5.1%) Preterm birth 11 (3.5%) Other Direct Causes 65 (20.9%) Indirect Obstetric Causes 57 (18.3%) Health policy and systems 42 (13,5%) Governance 1 (0.3%) Financial 0 (0%) P3A308 A citation analysis of Cochrane systematic reviews in Chinese Science Citation Database Delivery 23 (7.5%) Implementation 18 (5.7%) Chen Y1 , Wei D2 , Li J2 , Xu H2 , Zhu S2 1 Evidence-Based Medicine Center of Lanzhou University, China; 2 School of Basic Medical Sciences of Lanzhou University, China No Questions Listed 2 (0.6%) Background: Although The Cochrane Database of Systematic Reviews has an Impact Factor of 5.653 and is ranked 11th out of 132 in the ISI category, ‘Medicine, General & Internal,’ in 2009, we know little about citation of Cochrane systematic reviews in Chinese articles. Objectives: To investigate the citation of Cochrane systematic reviews in Chinese articles. Methods: We searched the Chinese Science Citation Database (CSCD) using the term ‘Cochrane Database of Systematic Reviews’ from sources cited. Then we screened all relevant papers. Results: There were 1184 Cochrane Systematic Reviews cited by 1403 Chinese papers which were published in 191 medical journals. The total citations are 2005 and citation per paper is 1.7. The most cited paper is ‘Nicotine replacement therapy for smoking cessation’ (12 times). The top three citing journals are Chinese Journal of Evidence Based Medicine, Chinese Journal of Practical Gynecology and Obstetrics, and Chinese Journal of Clinical Rehabilitation, which have cited more than 500 CSRs. Conclusions: Citation per paper of The Cochrane Database of Systematic Reviews in China is only a third of the average number. It is necessary to further promote Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. the dissemination and utilization of Cochrane Database of Systematic Reviews in China. P3A309 A systematic review of the quality and types of instruments used to assess KTE implementation and impact Irvin E1 , Cole D2 , Keown K1 , Kramer D3 , Brenneman Gibson J1 , Kazman Kohn M4 , Mahood Q1 , Slack T1 , Amick III B1 , Phipps D5 , Garcia J3 , Morassaei S1 1 Institute for Work & Health, Canada; 2 University of Toronto, Canada; 3 University of Waterloo, Canada; 4 St Michael’s Hospital, Canada; 5 York University, Canada Background: Knowledge transfer and exchange (KTE) is a burgeoning practice at research organizations worldwide. However the effectiveness of KTE practices has not been routinely/consistently evaluated. The need for such evaluation has been voiced (Lavis et al, 2003; Eccles et al, 2005). One potential reason for the lack of evaluation is the paucity of tools and methods for evaluation across various disciplines. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 168 Figure 1 (P3A309). Flowchart of documents progressing through review steps. Objectives: To answer the following question: Are there reliable, valid and/or useful instruments to apply in the assessment of KTE implementation and its impact? Methods: A team of researchers and stakeholders followed a systematic review process developed by the Institute for Work & Health using a Cochrane-like approach and a descriptive synthesis. The basic review steps include: formulation of research question and search terms; decision-maker/stakeholder meetings; literature search; relevance review; quality appraisal; data extraction; evidence synthesis; decision-maker/stakeholder key message generation; and report preparation. Results: The search was conducted in MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, CAB Abstracts, LISA, Social Science Abstracts, and Business Source Premier. The search resulted in 9998 unique titles and abstracts (see Figure 1). We found 54 quantitative articles that described instruments for the evaluation of KTE approaches. Data extraction focused on instruments and measurement properties. Conclusions: Our synthesis of 54 quantitative studies revealed a variety of instruments used to evaluate KTE approaches. Overall, the description of the measurement properties was not consistent and, often, details of measurement properties were lacking or missing. However, a subset of articles, provided descriptions of instrument development and measurement properties and show promise as potentially useful tools in the evaluation of KTE approaches. Key messages from this review include the need to continue instrument development work and to clearly present the measurement properties of the instruments used. P3A310 The impact of systematic reviews on health care policy Background: Despite increasing emphasis on the use of research to inform decision making in health care there is concern that systematic reviews do not have the expected impact on policy and practice and more work may be needed to enable us to maximise their potential. Objectives: The aim of this study was to increase understanding of the likely impacts of systematic reviews on policy and identify factors that might facilitate their influence. Methods: Nine systematic reviews published between 1998 and 2008 and used established methods for the evaluation of research impact, including bibliometrics and documentary review, to examine whether or not these reviews had influenced policy development were reviewed. Data from these analyses were combined with an overview of the literature to identify factors that might increase impact. Results: The reviews had influenced the development of national and international policy, although much of the impact was at a ‘micro’ level in the form of guidelines for practice. There was considerable variation in the impact of the reviews. Reviews evaluating fluid resuscitation and road safety interventions showed the greatest evidence of impact, while a review of qualitative studies on barriers to fall prevention showed the least amount of impact. Differences might be explained by time since publication, type of question, importance to policy makers, the nature and strength of the evidence, the purpose of the review and the networks and strategies used for dissemination. Conclusions: Systematic reviewers should consider the desired impacts of their work early in the review process and detail appropriate strategies for increasing impact in the review protocol. This might include specifying methods to address applicability to particular contexts and devising active strategies for dissemination. Frances Bunn1 1 Injuries Group, UK Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 169 P3A312 Determining the role of patient values and preferences in the development and review of preference-sensitive practice guidelines MacLean S1 , Akl E2 , Johnston B1 , Guyatt G1 1 Department of Clinical Epidemiology and Biostatistics, McMaster University, Canada; 2 Department of Medicine, State University of New York at Buffalo, NY, USA Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada Background: While systematic summaries of the available evidence on benefits and harms are essential for the development of clinical practice guidelines, evidence alone is insufficient. Treatment recommendations involving a trade-off of desirable and undesirable consequences of alternative course of actions require value and preference judgments. Previous studies have identified a wide range of possible strategies for integrating patient values and preferences into guideline development. Objectives: To assess how clinical practice guidelines incorporate patient values and preferences into the processes of developing recommendations and stating them. Methods: We selected guidelines published between 2007 and 2009 which addressed five conditions for which incorporation of values and preferences would be important: breast cancer, prostate cancer, colorectal cancer, end of life care, and childbirth. We searched the National Guideline Clearinghouse, the Guidelines International Network, the Scottish Intercollegiate Guidelines Network, and the National Institute for Health and Clinical Excellence, using a combination of subject or condition headings, and keywords. One reviewer carried out the initial search, and two reviewers independently adjudicated the final inclusion. Results: In the sample of 60 eligible guidelines, we identified 1357 recommendation statements. Less than 5% of all recommendation statements were explicitly informed by evidence that pertained to patient values and preferences (.58%), or included recommendations in support of involving patients in clinical decision-making (4.9%). Of the 60 guidelines, 24 (40%) reported that patients were involved in guideline development and/or review. Twelve guidelines (20%) reported that patient advocacy group members, or members of patient charity organizations, were invited to participate, and 15 (25%) guidelines involved community members in the development and/or review of guidelines. Conclusion: Patient values and preferences are under-represented in guideline development. P3A313 A US hospital-based health technology assessment center to improve the quality, safety and cost-effectiveness of patient care Mitchell M1 , Williams K1 , Umscheid C1 1 Center for Evidence-based Practice, University of Pennsylvania Health System, USA Table 1 (P3A313). Reports requested by 2006–08 2009–11 Chief Medical Officers 12 9 Clinical Department 6 23 Purchasing Committee 12 15 Pharmacy & Therapeutics Comm. 4 14 Quality Improvement Groups 6 12 Nursing Administration 1 3 CDC (US government) 4 3 Administrative Department 1 3 Payer 1 0 Clinical Care Review Committee 0 1 Other 0 3 scientific evidence and applying it to decision making about purchasing, formularies, and clinical practice. CEP is staffed by two hospitalist co-directors trained in epidemiology, three HTA analysts, physician and nurse liaisons, a librarian, a health economist, and an administrator, totaling 5.5 full time equivalents. Results: Over 150 evidence reports have been completed to date, 44 in the most recent 12 months. Internal clients requesting reports include clinical departments, quality administrators and hospital committees. Topics have included processes of care like the use of heparin versus saline for catheter flushing; and high-cost and emerging technologies like telemedicine in critical care. Reports review existing guidelines and systematic reviews first, and review primary studies when previously published reviews do not offer sufficient evidence. Local utilization and cost data are incorporated so reports can be tailored to our medical center’s needs. CEP then works to implement findings, including integrating them into computerized clinical decision support, and measures their impact using administrative and clinical data. Evidence reviews are shared publicly through the National Guideline Clearinghouse, the HTA database, and peer-reviewed publications. CEP also offers education through workshops, a resident elective, and courses for medical and graduate students. In addition, CEP has developed collaborations with payors, government organizations, and private industry. Conclusion: An evidence-based practice center within an academic medical center can educate clinicians and support a culture of evidence-based decision-making. Purpose: Most existing centers for health technology assessment (HTA) are associated with payers or government agencies. They most frequently review and analyze emerging and costly technologies. But hospitals often have to make decisions about processes of care that impact not only cost, but also on quality and safety of patient care. Methods: Our academic medical center created a Center for Evidence-based Practice (CEP) in 2006 for the purpose of gathering Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 170 P3A314 Simple ‘quick’ searches to answer questions arising in clinical practice Jordan J1 , Stevenson K2 , Lewis R3 1 Arthritis Research UK Primary Care Centre, Keele University, UK; 2 University Hospital of North Staffordshire, UK; 3 Health Library, Univeristy of North Staffordshire, UK Background: Clinicians have limited time to search for literature to answer questions arising in day-to-day practice. Pre-appraised evidence summaries are more readily available via sources such as NHS Evidence and the TRIP Database. However, are these easy for clinicians to use and will they help clinicians to be able to find answers to their questions quickly? Objectives: The aim of this project was to assess the ability of simple searches to answer questions that arise in clinical practice and highlight some of the barriers that might be encountered. Methods: We selected clinical questions from Critically Appraised Topics (CATs) in Musculoskeletal Care produced by allied health professionals and researchers using traditional structured searches. Questions included three intervention effectiveness questions, one question on access to musculoskeletal primary care and a question on the effectiveness of outpatient appointment duration. Simple searches were carried out for each of these topics on Clinical Knowledge Summaries, NHS Evidence, PubMed Clinical Queries and the TRIP Database and compared to the results of the original CATs. Remember to spell out what these acronyms mean. Results: Only two of the intervention questions were answered by the simple search; one of these by a pre-appraised resource, Best Bets. No new literature was found that had not been found in the full CAT search. However, the full CAT search did retrieve literature not found in the simple search but this was based on lower quality evidence. Some resources searched produced a large volume of hits and literature had to be appraised and interpreted, which took additional time and skills. Conclusions: Evidence to answer clinicians’ questions about the effectiveness of interventions can sometimes be found from simple literature searches. However, this is not always a quick process and some judgement and appraisal skills are still required to select the most appropriate literature. Many questions remain unanswered about both simple and traditional searching methods. P3A315 A multi-faceted provider education program designed for consistent, evidence-based clinical practice change within an integrated care delivery system Okawa G1 1 Kaiser Permanente, USA Background: An ongoing challenge in medical education is demonstrating its ability to help implement evidence-based care, enable sustainable clinical practice change, and reduce unnecessary variations in care delivery. In 2009, an integrated care delivery system in the U.S. launched a multi-faceted organization-based provider education program strategically designed to improve the consistency and quality of care. Objectives: The primary objective was to design a provider education program integrating multiple strategies for guideline implementation and medical education to improve the quality, consistency, and sustainability of care. This required clarifying practice standards using the best available evidence, gaining agreement across clinical Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. departments, creating multi-faceted medical education interventions, and developing operational processes to optimize the care delivery system. Methods: Executive leadership identifies key clinical quality gaps and provides sponsorship for the planned interventions. Evidence-based methods are used to formulate clinical practice recommendations based on the best available evidence. Multi-faceted medical education is designed to communicate the key clinical recommendations. Operational processes, audit and feedback, clinical champions, and point-of-care decision support are created to sustain practice change. Objective measures are used to identify baseline performance and to monitor for process improvement over time. Be careful about switching from past tense to present tense. Results: Initial 6-month data shows improvement in several process measures following medical education interventions. Examples include a 4% improvement in the number of diabetic patients with a HBA1C of <9%, a 3.4% improvement in the number of patients achieving a target LDL <100, and a 14.4% improvement in the number of referred patients with a complete hematuria evaluation. Conclusions: Medical education, designed strategically, may be effective within a large integrated care delivery system in improving the consistency and quality of clinical care. P3A316 Effectiveness of quality improvement strategies in improving the management of diabetes: A systematic review and meta-regression Tricco A1 , Ivers N2 , Moher D3 , Turner L3 , Galipeau J3 , Halperin I4 , Vachon B3 , Ramsay T3 , Manns B5 , Tonelli M6 , Shojania K4 , Grimshaw J3 1 Li Ka Shing Knowledge Institute of St. Michael’s Hospital, Canada; 2 Women’s College Hospital, Canada; 3 Ottawa Hospital Research Institute, Canada; 4 University of Toronto, Canada; 5 University of Calgary, Canada; 6 University of Alberta, Canada Background: Diabetes mellitus has reached epidemic levels and quality of care gaps are persistent. Quality improvement (QI) strategies are commonly implemented at great cost, but the best approach to improve the provision of diabetes care remains uncertain. Objectives: To evaluate the effects of QI strategies on the management of diabetes. Methods: Systematic review of published and unpublished studies in MEDLINE (1966 to July 2010), Cochrane Effective Practice and Organisation of Care (EPOC) database (July 2010), and references of included trials. Randomized controlled trials (RCTs) examining 11 pre-defined QI strategies targeting healthcare professionals for managing adult outpatients with diabetes were included. RCTs lacking data (on glycemic control (HbA1c), blood pressure (BP), or write-out LDL cholesterol), RCTs focusing only on interventions at the patient level, and RCTs not written in English were excluded. Two reviewers independently extracted data and appraised risk of bias using the Cochrane EPOC tool. Results: Data from 145 RCTs and 20 companion reports were included. Compared with usual care, the QI strategies statistically improved HbA1c (117 studies, mean difference [MD] −0.4% [95% confidence interval {CI} −0.5 to −0.3%]), LDL (45 studies, MD −4.1 mg/dL [−5.8 to −2.5 mg/dL]), systolic BP (66 studies, MD −3.7 mmHg [−4.6 to −2.6 mmHg]), and diastolic BP (61 studies, MD −1.6 mmHg [−2.4 to −0.7 mmHg]). After adjusting for sample size and other factors, QI strategies with the most significant improvement included case management, team changes, and patient Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 171 education. Conclusions: Some QI strategies were more successful than others in improving outcomes. Clinicians and policy-makers can use these results to tailor the choice of intervention according to the desired outcome, available resources, and local healthcare context. P3A317 Exploring the experiences with a new policy for managing conflict of interest in guideline developers Neumann I1 , Akl E2 , Guyatt G3 1 Department of Medicine, Pontificia Universidad Catolica de Chile, Santiago, Chile; MScC Health Research Methodology, Department of Clinical Epidemiology and Biostatisitcs, McMaster University, Hamilton, Canada; 2 Department of Medicine, State University of New York at Buffalo, NY, USA; Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada; 3 Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada Background: The American College of Chest Physicians (ACCP) 9th edition of the Antithrombotic guidelines (AT9) has adopted a conflict of interest (COI) policy that gives the primary responsibility to methodologists, equally emphasises intellectual and financial conflicts of interest, and excludes conflicted individuals from decision-making regarding recommendations. A qualitative study found that prior to the implementation of the policy, the methodologists and content experts were uneasy regarding their respective roles. Also, they disagreed about the potential effect of the new strategy on the quality of the guideline. Objectives: To explore the experience of the methodologists and content experts with the implementation of the COI policy during AT9 guideline development process. Methods: We will conduct semi-structured interviews with both methodologists and content experts. We will ask about their experience with regards to: (1) giving primary responsibility to a methodologist; (2) emphasis on intellectual conflict of interest; (3) restrictions based on conflicts of interest; and (4) perceived impact on the quality of the guideline. Results: The project is underway and we will present results in October. Exploring the experience of the individuals affected by the new COI policy will help understand its feasibility, whether it achieved utilization of the insight of experts while ensuring that conflicts do not influence recommendations, and if there are refinement that should be considered for future guidelines. P3A318 Judiciary branch and scientific evidence: An analysis of judicial decisions on galsufase Marin dos Santos D1 , Atallah A1 , Dallari Jr. H1 , Barroso L1 1 Brazilian Cochrane Centre and Federal University of São Paulo, Brazil Background: The judiciary branch, in Brazil and other countries, has been an active actor in public health policies, especially regarding drug policies. In the present paper, we evaluate the judicial response to cases regarding Galsulfase, a recombinant form of human N-acetylgalactosamine 4-sulfatase, designed for the treatment of patients suffering from mucopolysaccharidosis type VI. Objectives: To identify the prevalence of scientifically grounded judicial decisions and the knowledge of Brazilian judges about evidence-based medicine. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Methods: We searched the electronic database of the Supreme Court (STF), the Superior Court of Justice (STJ) and all Federal Regional Courts (TRFs), looking for the keywords ‘naglazyme’ OR ‘galsulfase’. Results: We found 31 judgments, including collegial judgments (5; 16.12%) and monocratic decisions (26; 83.87%). Of these, 9 were excluded due to procedural reasons (29.03%) and 22 met the eligibility criteria. Out of them, 20 (90.09%) decisions determined the government to supply galsulfase. Only 1 decision (4.54%) considered it improper to deliver the drug due to lack of evidence about its effectiveness. And 1 decision (4.54%) determined the performance of forensic expertise. Out of the 22 decisions examined, 18 (81.81%) were not substantiated by scientific evidence. Other 2 (9.09%) referred to evidence-based medicine (EBM), but not considered it as a plea for the decision-making. Only 2 decisions (9.09%) considered EBM as the basis of the decision-making process. On the other hand, 7 decisions (31.81%) referred to medical expert opinion. Incompatible outcomes were associated to galsulfase in 7 decisions. Conclusions: The judiciary ignores scientific evidence as an aid to the decision-making process. In order to preserve the right to health and sustainability of the system, it is necessary that judicial decisions find ground on high-grade medical evidence. Therefore, it is imperative that the Judiciary branch becomes more pragmatic and less ideological. P3A319 High impact collaborations: Examples from the Cochrane HIV/AIDS group Horvath T1 , Kennedy GE1 , Rutherford GW1 1 Cochrane HIV/AIDS Group, University of California, San Francisco, USA Background: Policymakers need high-quality evidence to make appropriate decisions. United States (US) and international agencies have sought help from the Cochrane HIV/AIDS Review Group (CRG) in developing such evidence for guidelines and other policy statements. Objectives: To ensure that HIV/AIDS policy and guidelines are based on the best evidence. Methods: We review our experience with providing systematic reviews for state, national and international agencies and how they have been used. Results: Between 2001 and 2011 our group has been commissioned to conduct 42 systematic reviews for state, national and international agencies, including the US Centers for Disease Control and Prevention (7), the World Health Organization (31), the California Department of Health Services (3) and the US Council on Foreign Relations (1). Since 2008, we have conducted 22 reviews (including more than 70 GRADE profiles) for the World Health Organization. These evidence assessments have been used to develop worldwide guidelines for antiretroviral therapy of adults, adolescents and children, prevention of mother-to-child transmission, prevention of sexual transmission among men who have sex with men, prevention of sexual transmission among couples, and prevention, diagnosis and treatment of cryptococcal infection. Conclusions: For a decade, evidence generated by our CRG has helped guide regional, national and global HIV/AIDS guidelines. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 172 P3A320 Do Cochrane reviews inform the development of clinical practice guidelines? The case of hypertension Al-Ansary L1 , Tricco A2 , Adi Y3 , Al-Ghonaim M4 , Bawazeer G5 , Perrier L2 , Tashkandi M2 , Straus S6 1 Dept of Family and Community Medicine, College of Medicine, King Saud University, Saudi Arabia; 2 Li Ka Shing Knowledge Institute, University of Toronto, Canada; 3 Shaikh Bahamdan’s Research Chair for Evidence-Based Health Care and Knowledge Translation, King Saud University, Saudi Arabia; 4 Dept of Internal Medicine, College of Medicine, King Saud University, Saudi Arabia; 5 College of Pharmacy, King Saud University, Saudi Arabia; 6 Li Ka Shing Knowledge Institute, university of Toronto, Canada Background: The Institute of Medicine has redefined clinical practice guidelines (CPGs) as statements that include recommendations intended to optimize patient care that are informed by a systematic review (SR) of evidence. It is unclear whether commonly used CPGs for hypertension meet this standard despite the availability of many systematic reviews on the management of this condition. The Cochrane Library, the foremost ‘provider’ of SRs has over 35 reviews on the management of hypertension. Objectives: To determine the extent to which recently-developed hypertension CPGs are informed by Cochrane (or non-Cochrane) SRs. Methods: We searched MEDLINE, EMBASE, guidelines’ websites and Google for recently published (2006–2010) CPGs on the general management of hypertension in any practice setting and that were written in English or have an English abstract. Two reviewers assessed each guideline for quality using the AGREE tool and determined whether they were informed by available SRs. Results: Ten CPGs were identified; 8 included references. Seven CPGs cited a total of 59 non-Cochrane SRs and/or meta-analysis. Of these, 5 were cited by 3 CPGs and 10 by 2 CPGs. Only 4 CPGs referred to a total of 14 of the 35 Cochrane reviews (CRs); no CR was cited in more than one CPG. The country CPGs citing most were: Japan (8) and Malaysia (3). The relationship between the evidence generated by the reviews and the corresponding recommendation could not always be recognized. Conclusions: Despite the increased production of SRs in general and CRs in particular, few guidelines refer to available SRs. The reasons for lack of inclusion of SRs need to be explored. And the need for the Cochrane Collaboration to consider the practical means by which its knowledge can be translated clearly stands out. P3A321 Impact of an evidence based pediatrics electronic journal González MP1 , Buñuel JC1 , González de Dios J1 , Esparza MJ1 , Aizpurua P1 1 Spanish Association of Pediatrics, Spain Background: In order to give the best care to patients, health professionals need to integrate the highest quality scientific evidence with clinical expertise and the opinions of the patients. However they have to deal with a huge amount of publications. Evidencias en Pediatrı́a (EeP) (http://www.evidenciasenpediatria.es/) is a secondary publication of the Spanish Association of Paediatrics with the purpose of alerting clinicians of important advances in paediatric topics. EeP reviews 70 journals. The articles selected are summarised in abstracts and then commented by clinical experts. EeP has been included in the Cochrane Plus, and recently it has been supported by the Latin-American Association of Paediatrics (ALAPE). Objectives: To Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. describe the impact of the electronic journal EeP. Methods: Since March 2010, EeP has been in a new web site. The social bookmarking has been implemented by creating links to Facebook and Twitter pages of EeP. We have used Google Analytics to extract the most outstanding information on the impact and acceptance of the journal. Results: An amount of 142,377 visits have been received from April 2010 to May 2011, with 51,000 returning visitors (RV) (35.6%). RV meet the criteria of regular good clients of the site: more number of pages visited (3.12 pages/visit), more time on the site per visit (4.53 minutes) and smaller bounce rate (51,83%). The sources of traffic used were: referring sites (48.32%), search engines (28.2%)% (Google was the most used, 26.66%) and direct traffic (23.4%). The referring sites most used were the paediatric websites, followed by the social webs Facebook and Twitter. The majority of visitors come from Spanish-speaking countries, mainly Spain, Mexico and Argentina. Conclusions: EeP is the first electronic journal of evidence-based paediatrics in Spanish in the world. It receives a large number of total visits and specially of returning visitors, who come mainly from Spanish-speaking countries. P3A322 Do Cochrane reviews drive clinical decision-making at the point of care? Moja L1 , Banzi R2 , Cinquini M2 , Moschetti I2 , Pecoraro V2 , Tagliabue L1 , Liberati A3 1 University of Milan, Italy; 2 Mario Negri Institute, Milan, Italy; 3 University of Modena and Reggio-Emilia, Italy Background: ‘Without Cochrane Reviews, people making decisions are unlikely to be able to access and make full use of existing healthcare research’, the Cochrane Collaboration website reports. Although systematic reviews (SR) are increasingly popular as a form of convenient synthesis of evidence to support clinical decision-making, it is difficult to measure to what extent may back practitioners up in their daily practice. A proxy of this achievement is whether Cochrane Reviews are utilized at the point of care. Point-of-care summaries provide physicians with comprehensive, condensed evidence in easily digestible formats. Objectives: We compared Cochrane and non-Cochrane SR citations in point-of-care summaries. Our null hypothesis was that the citation hazard of a relevant sample of Cochrane and non-Cochrane reviews did not differ. Methods: We selected the top five point-of-care summaries for coverage of medical conditions, editorial quality and evidence-based methodology. As samples of relevant non-Cochrane SRs, we selected all the SRs signalled by two literature surveillance journals (ACP Journal Club and Evidence-Based Medicine Primary Care and Internal Medicine). For Cochrane SRs we selected those labelled as ‘Conclusion changed’ in the Cochrane Library. Parallel sampling ran from April to December 2009. We measured the occurrence and timing of Cochrane and non-Cochrane SR citation in point-of-care summaries from June 2009 to May 2010. We assessed the updating cumulative rate using Kaplan-Meier survival analyses. Cox model was used to calculate hazard ratios (HR) between the non- and Cochrane review citation. Results: The analysis included 128 SRs, 68 non-Cochrane (53%) and 60 Cochrane (47%). The difference between the nonand Cochrane review citation speed was not significant (HR 1.13, IC 95% 0.87 to 1.46, p = 0.36). Both were cited with similar patterns by point-of-care summaries. Conclusions: This finding belies down the criticism that Cochrane reviews are undervalued as a source of evidence for bedside clinical decision making. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 173 P3A323 Mr. Pickwick and his child went on a field trip and returned empty handed: the widening gap between non-randomised studies and evidence for circadian rhythm sleep disorders management in a paediatric population Khan S1 , Heussler H2 , McGuire T3 , Pache D3 , Dakin C2 , Cooper D2 , Norris R4 , Flenady V5 , Charles B1 1 School of Pharmacy, The University of Queensland, Brisbane, Australia; 2 Sleep and Respiratory Unit, Mater Children Hospital, Brisbane, Australia; 3 Pharmacy- Practice and Development, Mater Health Services, Brisbane, Australia; 4 Australian Centre for Paediatric Pharmacokinetics, Mater Children Hospital, Brisbane, Australia; 5 Translation Research into Practice (TRIP), Mater Medical Research Institute, Mater Health Services, Brisbane, Australia Background: Therapeutic protocols in the management of circadian rhythm sleep disorders (CRSDs) in the paediatric population are currently inadequate. There is a lack of randomised controlled trials (RCTs) and an ever increasing number of non-RCTs in this specialised niche. However, including information from non-randomised studies may adversely affect therapeutic recommendations. Such concerns have led the Cochrane collaboration to focus on evaluation of results from RCTs, to the virtual exclusion of other forms of evidence. Objectives: To determine the number of excluded studies on melatonin for the management of CRSDs in visually impaired children and to describe a general framework by which information from nonrandomised studies might be integrated into evidence. Methods: 1) A retrospective, observational study for determining the number of RCTs and non-RCTs/observational studies for melatonin in the management of CRSDs in visually impaired children through the Cochrane Collaboration [Cochrane Developmental, Psychosocial and Learning Problem Group (CDPLG)] was conducted. 2) A framework was developed to include non-randomised studies into evidence: (a) Conceptualisation of question under study (determining the internal and external validity; differentiating the efficacy and effectiveness trials); (b) Developing a conceptual model (based on CRSDs therapeutic outcome measures); (c) Recommendation of proposed for inclusion of non-RCTs / Observational studies. Results: A total of 9 studies of various designs were evaluated (3 case series; 3 non-RCTs; 2 randomised, but design-flawed studies and 1 N-of-1 trial). None met the Cochrane inclusion criteria. A framework for potential inclusion of such studies is provided. Conclusions: A null evidence report, based on inclusion criteria, runs the ethical risk of excluding potentially relevant practice-changing evidence. It thus unfairly evaluates the current practice as lacking a sufficient evidence basis. The proposed framework may assist to strengthen the quality and documentation of evidence from non-randomised studies in highly specialized therapeutic areas. P3A324 Including diverse study types in systematic reviews to address multiple questions about public health interventions: The case of a review of school effects on health Harden A1 , Bonell C2 , Jamal F1 , Wells H2 1 University of East London, Institute for Health and Human Development; 2 London school of hygiene and tropical medicine, department of social and environmental health research Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: There is increasing interest in addressing multiple questions in systematic reviews using a wider range of evidence, particularly in relation to broad and multi-faceted interventions such as those typically implemented within public health. Objectives: This study explores how diverse types of evidence including trials, process evaluations, theory, ‘non-intervention’ quantitative and qualitative studies can be combined using a review on the effects of schools and school environment interventions on health as a case example. Methods: Exhaustive searches were conducted and the results screened to identify relevant studies. Each study type underwent separate critical appraisal, data extraction and synthesis using specially developed tools tailored to each type of study. All five syntheses were integrated to draw conclusions and make recommendations. Results: Each type of study was used to address a different question within the review. Trials were used to estimate the effectiveness of interventions and process evaluations to address questions about intervention acceptability, feasibility and context. The quantitative non-intervention studies were used to examine the range of school-level factors which influence health and might form the focus of future intervention studies (these studies examined whether differences in health between schools could be explained by school level factors). The qualitative studies examined how students viewed the impact of schools on their health. Conclusions: This approach to combining diverse types of evidence in a single review proved especially valuable here as the interventions under study – those which aim to change the school environment – are at a relatively early stage of development. Current intervention studies have not addressed the full range of potential determinants of school health. Findings from the different study types were integrated to: build new theory for how schools affect health; suggest potential new avenues for intervention research; and refine logic models for interventions. P3A325 Measuring organizational readiness for knowledge translation in chronic care: A mixed methods systematic review on theories and instruments Gagnon M1 , Légaré F2 , Ouimet M3 , Estabrooks CA4 , Roch G1 , Labarthe J5 , Ghandour EK5 , Tremblay N5 , Grimshaw J6 1 Research Center of the Centre Hospitalier Universitaire de Québec, Québec, Canada and Faculty of Nursing, Université Laval, Québec, Canada; 2 Research Center of the Centre Hospitalier Universitaire de Québec, Québec, Canada and Department of Family Medicine, Université Laval, Québec, Canada; 3 Research Center of the Centre Hospitalier Universitaire de Québec, Québec, Canada and Department of Political Science, Université Laval, Québec, Canada; 4 Faculty of Nursing, University of Alberta, Edmonton, Alberta, Canada; 5 Research Center of the Centre Hospitalier Universitaire de Québec, Québec, Canada; 6 Ottawa Hospital Research Institute, Ottawa, Canada and Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada Background: Organizational readiness for change (ORC) assesses organizational members’ collective motivation and capability to implement change. However, the theorization of ORC lacks consensus and the available ORC instruments have shown limited validity and reliability. Objectives: We aim to develop an evidence-based, comprehensive, and valid instrument to measure organizational readiness (OR) for knowledge translation (KT) in chronic care. The existing evidence on ORC will be reviewed and synthesized as the basis for the development of a comprehensive, bilingual OR for KT instrument. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 174 Methods: Phase 1: We are conducting a mixed-methods systematic review on theories and instruments assessing ORC in health care. We will synthesize the findings in a conceptual map. A bibliography and a database of the ORC instruments will be prepared based on their psychometric appraisal. A pan-Canadian Delphi study will be carried out for the contextual assessment of these instruments. Phase 2: Individual interviews and focus groups will be conducted with key stakeholders for further development of the proposed instruments. A final bilingual (French/English) OR questionnaire will be tested in the field of chronic care to measure KT regarding the adoption of comprehensive, patient-centered and system-based chronic care models. Results: The initial review process retained 98 articles from a database of 2903 articles for full-text evaluation. After evaluation by two independent evaluators, 57 met the inclusion criteria and were critically appraised using quality criteria specific to quantitative, qualitative, and mixed-methods designs. Preliminary findings suggest a lack of consensus on the theoretical domains involved in ORC and limited evidence of ORC instruments’ validity. Conclusions: This study provides a comprehensive synthesis and aims at creating a consensus on the theoretical underpinnings and the instrumentation of ORC. The final product, a validated, comprehensive, bilingual instrument to assess ORC for KT will be useful for supporting the implementation of evidence-based chronic care practices. P3A326 Use of Cochrane Reviews in NICE Clinical Guidelines Alderson P1 , Tan T1 1 NICE, UK Background: It is routine practice when developing clinical guidelines for NICE to look for relevant Cochrane reviews. The NHS funds NICE and makes a substantial contribution to the funding of the Cochrane Collaboration. We wanted to see how many reviews have been used. Objectives: To collect citation rates for Cochrane reviews in NICE clinical guidelines and map cited reviews to guideline questions. Methods: We used free text searching for the word ’Cochrane’ in all NICE clinical guidelines published before March 2011 to identify citations of Cochrane reviews. We examined the use of the review as for background only or as a source of evidence for the guideline development group. We have also mapped, where possible, the questions addressed in NICE clinical guidelines, and are cross referencing this with the reviews cited. Results: NICE had published 116 clinical guidelines by the end of February 2011. Ten have been replaced by more up to date guidelines, leaving 106 guidelines. There were 731 citations of Cochrane reviews in the guidelines, ranging from 0 to 44 with a mean of almost 7. We judged that 23 of the citations were used purely as background information. 46 of the currently registered CRGs had reviews cited. We will also suggest how our mapping exercise can be used to identify where Cochrane reviews could have been better used, and as a source of information for CRGs when prioritising reviews. Conclusions: Cochrane reviews are widely cited in NICE clinical guidelines, emphasising the very important role of the reviews in helping to inform guideline recommendations. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P3A327 Developing a web-based international register of clinical prediction rules for primary care Keogh C1 , Wallace E1 , Fahey T1 1 Royal College of Surgeons in Ireland, Ireland Background: Clinical medicine is becoming increasingly interested in clinical prediction rules (CPRs) and their potential for improving patient care. However, identifying CPRs for primary care from electronic databases is difficult because several terms are used interchangeably to describe them. Objectives: The aim of the current work is to develop an international register of CPRs relevant to primary care to be distributed through the Cochrane Primary Health Care Field. Methods: The register is being developed by identifying relevant articles from various sources including: (1) the MEDLINE database [1980–2008] using an electronic search string, specifically developed and tested in-house; (2) other electronic resources (e.g. JAMA rational clinical examination series); and (3) contacting authors in the field for published and unpublished work. Relevant articles are classified according to a set of criteria including condition specific codes, level of evidence and quality assessment. Results: To date, the register consists of 300 articles that contain CPRs relevant to primary care, covering 16 different broad clinical areas. The largest number of articles was identified for cardiovascular conditions (n = 93) and the least identified for eye conditions (n = 1). These broad areas are further classified according to specific conditions. The register identified CPRs across all levels of evidence. The majority of articles were concerned with validating rules (narrow validation n = 99, broad validation n = 53). Few articles were concerned with formal impact analysis (n = 8). Assessment of the quality of the published articles produced mixed results. Conclusions: Although the register will need to be regularly updated, it offers many potential benefits for primary care practice. Classification of articles allows the clinician to objectively assess the utility of the CPR for use with patients. The register will also assist researchers to identify clinical areas in which CPRs have not yet been derived, as well as CPRs that have not undergone formal impact analysis. P3A328 Synthesis of recommendations for clinical use of biomarkers in oncology Trevisiol C1 , Pregno S2 , Rosin C3 , Fabricio ASC4 , Gion M3 1 Istituto Oncologico Veneto IOV – IRCCS, Padua, Italy; 2 Cochrane Italian Center, Italy; 3 Regional Center for Biomarkers, Department of Clinical Pathology, Azienda ULSS 12 Veneziana, Venice, Italy; 4 ABO Foundation/ Regional Center for Biomarkers, Department of Clinical Pathology, Azienda ULSS 12 Veneziana, Venice, Italy Background: the widely observation of inappropriate use of biomarkers in clinical practice and limited economic resources require strategies to translate the knowledge on their clinical utility and limitations to their users. Objectives: to produce synthesis of recommendations for clinical use of biomarkers in 17 neoplasias for all clinical spectrum together with AGREE critical appraisal, having the purpose to represents systematic comparison of guidelines addressing similar topic areas and facilitate their use by end-user in a independent and critical way. Methods: the project was developed in the last 2 years adopting a transparent, explicit, and systematic EBM Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 175 approach, involving a working group of 60 professional figures with expertises in different health and economic fields. The search of the guidelines was conducted following systematic methodological criteria and the selection was conducted by 2 independent reviewers using pre-settled criteria. Recommendations regarding biomarkers with the corresponding strength of evidence and recommendation rating schemes were extracted and recorded in tables and then synthesised into synoptical tables. The critical appraisal was conducted using the AGREE-I instrument by at least three independent professional figures from the working group. In addition, more in-depth original methodological tables were produced to describe the strengths and limitations of the body of selected evidence. Results: from over 3500 abstracts in the English and Italian language published in the last seven years (using MEDLINE and the principal guidelines databases) 52 EBM guidelines were selected and critically appraised between February and June 2010. The synoptical tables and the AGREE evaluation were published in a document in October 2010. Conclusions: the final document synthetises the available recommendations for each clinical spectrum and at the same time supplies a synthetic methodological quality appraisal of the guidelines. This approach allows the users to make critical and informed decisions on biomarkers use in clinical practice oncology. P3A329 Collaborative knowledge networks (CKN) for guideline development: A Kaiser Permanente and BMJ evidence centre case study Davino-Ramaya MD C1 , Brunnhuber MD K2 , Haynes MPH J1 , Simpkins MD C2 , Tom MS G1 , Minhas MB ChB R2 , Robbins MD C1 1 Kaiser Permanente USA; 2 British Medical Journal Evidence Center UK Background: As we experience one the greatest explosions in scientific knowledge of our time, guideline developers seek to explore creative and innovative strategies to address the challenges of resource constraints. Developing a short-term Collaborative Knowledge Network (CKN) provides the necessary foundation and serves as a catalyst for the development of a clinical practice guideline (CPG) through intellectual sharing, capacity expansion and quality enhancement. Objectives: Emphasizing effective knowledge translation and communicating evidence, we present a case study of the CKN between infectious disease experts, the Kaiser Permanente (KP-USA) guideline development team, and the evidence team at BMJ Evidence Centre (BMJEC-UK) during the development of a new sexually transmitted disease (STD) CPG. Methods: We describe the structure of the KP/BMJEC-CKN; the flow of information between key stakeholders; and the measures set in place to ensure high levels of quality, transparency, and rigor in communicating the requirements of the guideline development team, clinical knowledge of the guideline panel, and management of the large amount of data and information resulting from an elaborate evidence synthesis process. Results: The partnership provides insight into the benefits and challenges of an international cross-collaboration in evidence synthesis during the development of a HIV/STD CPG. It demonstrates the need for: – agreement on methodological approach including GRADE; – early identification/definition of roles; – on-going collaboration; multidirectional communication; – transparent, rigorous and regular documentation - aligning and setting clear timelines of evidence creation and guideline team work (problem formulation/clinical questions, evidence Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. summaries/systematic reviews); – use of agreed templates; – rapid feedback and query loops; – process sharing/evaluation. Conclusions: Collaborative Knowledge Networks for guideline development have the potential to increase quality and capacity of guideline developing organizations, to strengthen guideline methodology, to provide transparent and systematic processes while serving as a repository of evidence synthesis for global availability to develop localized clinical decisions. P3A330 Reporting guidelines for systematic review protocols Moher D1 , Shamseer L1 , Clarke M2 , Ghersi D3 , Liberati A4 , Petticrew M5 , Shekelle P6 , Stewart L7 1 Ottawa Hospital Research Institute, Canada; 2 Queen’s University Belfast, Ireland; 3 World Health Organization, Switzerland; 4 University of Modena, Italy; 5 London School of Hygiene & Tropical Medicine, UK; 6 RAND Corporation, USA; 7 Centre for Reviews and Dissemination, UK Background: Systematic review protocols are seldom published (with some exceptions, such as the protocols for Cochrane reviews). Furthermore, for systematic reviewers interested in publishing their protocols, there is limited guidance currently. Where protocols are accessible, they show what was – and was not -planned for the review which might not be clearly understandable in published reports. Objectives: To develop a guideline to aid authors when preparing and reporting protocols. This will extend the PRISMA Preferred Items for Reporting Systematic Reviews and Meta-Analyses (PRISMA) Statement, to provide guidance for protocols (PRISMAP). Methods: Development of PRISMAP will follow the process for reporting guideline development designed by the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) group. Potential checklist items were identified through the Delphi process and extensive consensus-building during the development of PROSPERO – an international register for systematic review protocols. The PROSPERO items will be debated and further refined into a checklist for the reporting of systematic review protocols during a June 2011 meeting of international experts, facilitated by the developers of PROSPERO and experts in the development of reporting guidelines. The draft guideline with accompanying checklist will be shared at the 2011 Cochrane Colloquium. Potential Impact: The availability of a tool to help Cochrane and other systematic reviewers create and report protocols will hopefully improve the quality of both protocols and the subsequent reviews. PRISMAP might also make it easier for readers and peer reviewers to identify selective reporting biases in systematic reviews. P3A331 An extrapolation checklist to answer ‘Can I use the results of this systematic review in a disadvantaged population?’ Ueffing E1 , Petticrew M2 , Welch V1 , Clarke M3 , Gruen R4 , Guyatt G5 , Kristjansson E1 , Mercer S6 , Pardo Pardo J1 , Tugwell P7 1 University of Ottawa, Canada; 2 London School of Hygiene and Tropical Medicine; 3 Queens University Belfast, Northern Ireland; 4 Monash University, Australia; 5 McMaster University, Canada; 6 CDC Guide to Community Preventive Services, USA; 7 Medicine & Epidemiology and Community Medicine, Faculty of Medicine, University of Ottawa, Canada Research Chair in Health Equity, Canada Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 176 Background: There is a need for guidance on how knowledge users can apply the results from Campbell and Cochrane reviews to disadvantaged groups. Knowledge users are under pressure to consider the effects of policies and programs on health equity. Objectives: To develop an extrapolation checklist to help decision-makers and practitioners decide when the results from systematic reviews are applicable to disadvantaged populations/settings. Methods: We developed an extrapolation checklist by drawing on items from existing checklists and consulting healthcare practitioners, researchers, consumers and policy-makers. We held a pilot session with Cochrane methodologists, review authors, and staff to test this extrapolation checklist with specific scenarios which asked them to apply results of Cochrane reviews to specific disadvantaged populations (e.g. women, people in low- and middle-income countries, the poor). We then refined the checklist and the scenarios. Results: We created 2 checklists: one for populations (to be used by policy-makers) and one for patients (to be used by practitioners). Feedback during a workshop and from one-on-one consultations suggested further refinement to operationalize the checklists as a decision aid, to take into account the relative and absolute effects, quality of evidence, values of the intended recipients of the intervention (population or patients), and other issues such as costs. Conclusions: Our work suggests that using a systematic review involves complex decisions which could benefit from structured decision support tools, similar to decision support tools developed for patients. P3A332 Systematic review of the impact of reporting guidelines on publication quality Moher D1 , Palepu A2 , Shamseer L1 , Turner L1 , Altman D3 , Hirst A3 , Hoey J4 , Schulze K5 , Simera I3 1 Ottawa Hospital Research Institute, Canada; 2 McMaster University, Canada; 3 Centre for Statistics in Medicine, UK; 4 University of Toronto, Canada; 5 Family Health International, USA Background: Reporting of health research is often inadequate. To this end, many reporting guidelines (RGs), aimed at improving the quality of health research reports, have been developed for reporting a wide variety of specific types of research. A RG typically consists of a checklist, flow diagram or explicit text to guide authors on what should be included in a research report. Despite their emergence, RGs are underused and published health research continues to be poorly reported. Synthesizing evidence of their impact on the quality of reporting will help identify the most effective guidelines and motivate their use. Objective: To systematically review and, where possible, synthesize evidence on the impact of RGs on the overall transparency of health research reporting. Methods: This systematic review will investigate the impact of 81 RGs identified by an earlier systematic review, as well as newly eligible guidelines identified by the EQUATOR Network. A search strategy has been designed to identify evaluations of each RG. We will explore characteristics of RGs and journals associated with their use. Analysis methods will follow those of a recently completed systematic review of evaluations of the CONSORT Statement – a reporting guideline for randomized trials. Preliminary findings for this review will be presented at the Cochrane Colloquium. Potential Impact: This project aims to provide evidence to help guide decision making for journal editors. Knowledge Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. translation activities will include a meeting of journal editors to help develop next steps for RG implementation based on the results of this review, since it is likely a prudent policy to endorse and adhere to those RGs for which there is some evidence on their effectiveness. Ultimately, this review will help increase the uptake of RGs by different user groups and contribute to the improvement in health research reporting. P3B333 Developing a typology of activities for linking research evidence to research use Tripney J1 , Kenny C1 , Gough D1 1 EPPI-Centre, SSRU, Institute of Education, London Background: One of the main objectives of using systematic methods for primary research synthesis is to inform policy and practice decision makers. However, the research on methods to increase the uptake of research by decision makers is at an early stage of development. The research is held back by a lack of theory and a lack of concrete measures of interventions and outcomes (see for example, Grimshaw 2004). Objectives: To develop a typology for activities that link research evidence to policy making, facilitating future studies on research utilization (as part of a European Commission funded study). Methods: A simple model of the evidence to policy system was developed based on previous work by Best (2010) Levin (2004), and Nutley et al (2003). A classification system for describing the content of activities and the mechanism by which they linked research to policy was developed from: (i) theories of evidence use; (ii) the system developed by Nutley et al (2003); (iii) a survey of such activities in Europe. Results: A typology of 27 activity types and 9 mechanisms that enabled the link between research evidence to policy was developed. Conclusions: The typology provides practical means for clarifying the nature of knowledge translation activities and for more focused and fruitful studies on research production and use. The typology will be further refined though application to further examples of research to policy linking activities Worldwide. P3B334 Agree instrument: A good instrument for family practice training Stein AT1 , Khan GS2 1 Grupo Hospitalar Conceição/Ufcspa/Ulbra, Brazil; Conceição/Ulbra, Brazil 2 Grupo Hospitalar Background: The Appraisal of Guidelines for Research & Evaluation (AGREE) Instrument was developed to address the issue of variability in guideline quality. Underutilization of evidence-based research is often described as a gap between ‘what is known’ and ‘what is currently done’ in practice settings. Objective: to introduce the AGREE instrument for family physicians in a residency programme. Methods: The Community Health Service of Conceição Hospital Group in Porto Alegre, Brazil has been training family physicians and other health professionals for over 20 years. There is a need to improve primary health care in Brazil. The standard of care in this service has been well established by various audit researches and several guidelines Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 177 have been produced by this group, although no evaluation has been performed with regards to the quality of these guidelines. There are 15 preceptors who have evaluated a Clinical Practice Guideline (CPG) which has been produced for the Brazilian Medical Association in liaison with the Brazilian Association of Family Practice on Management of Dementia in Primary Health Care. A workshop has been conducted to introduce the AGREE instrument and a questionnaire evaluating their perception on the importance of this instrument in the process of care. Results: Family physicians have found that the AGREE instrument has been very useful on the evaluation of the quality of CPG. They have identified that clinical practice guidelines when rigorously developed using a transparent process that combines scientific evidence, clinician experimental knowledge, and patient values are essential for a good standard of medical care. Conclusion: The process of using the AGREE instrument for family physicians has emphasized on establishing the process of knowledge translation and a good strategy for training of family physicians in Brazil. P3B335 Access to The Cochrane Library: Results of Canada’s National Pilot License Initiative Squires J1 , Grimshaw J2 , Schaafsma ME3 1 Ottawa Hospital Research Institute, Canada; 2 Ottawa Hospital Research Institute and The Canadian Cochrane Centre, Canada; 3 The Canadian Cochrane Centre, Canada Background: The Cochrane Library is one of the most reliable sources of health information about ‘what works’ in healthcare. Yet access in Canada is limited; approximately 10% of Canadians have access to The Cochrane Library. In response, a national pilot license was instituted that gave all Canadians ‘one-click’ access to The Cochrane Library for one year. Objective: To determine the impact of a national license on use of The Cochrane Library in Canada. Methods: We performed: (1) an interrupted time series study of use of The Cochrane Library in Canada and (2) a cross-sectional anonymous survey of users of the Library (n = 1192). Monthly use of The Cochrane Library (in terms of the number of: full-text downloads, access denials, abstracts viewed, and searches conducted) for the pilot period was compared with the expected numbers based on the 12 months preceding the pilot. The survey assessed characteristics of users of The Cochrane Library and their attitudes towards the Library. Results: Use of The Cochrane Library increased as a result of the pilot license. During the pilot, the number of full-text downloads and abstracts viewed were 65% (95% CI 34%, 96%) and 7% (95% CI 0%, 15%) higher than expected, respectively. The number of access denials and searches conducted decreased; there were 83% fewer denials (95% CI −97%, −70%) and 13% fewer searches (95% CI −20%, −6%) than expected. Attitudes towards the Library were positive with 88% and 91% of survey respondents reporting the Library as important to making good healthcare decisions and for all Canadians to have access to, respectively. Conclusions: Canadians want better access to high quality health knowledge. The provision of a national pilot license to The Cochrane Library in Canada significantly increased its use and, as a result, has the potential to positively impact the healthcare Canadians receive. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P3B336 Do value judgments play a role in evidence-based recommendations when information is based on proof of concept? Hutton B1 , Lee K1 , O’Grady T1 , Shukla V1 1 Canadian Agency for Drugs and Technologies in Health, Canada Background: The Canadian Expert Drug Advisory Committee (CEDAC) delivers evidence-informed formulary recommendations for drug reimbursement based on data from clinical systematic reviews and critiques of health economic information. Past work suggests that value judgments play a role in evidence interpretation and recommendations. Objectives: To explore the role of value judgments in CEDAC recommendations. Methods: CEDAC recommendations from 2009–2010 were reviewed. Recommendations were evaluated which had (1) a partial or full listing, where evidence consisted of only placebo controlled trials; and (2) a do not list recommendation, where evidence consisted of only active comparator trials. Information was gathered on factors potentially influencing the recommendation, such as: drug novelty (i.e. first drug for indication or first in drug class); indication risk and rarity; insufficiency of evidence; study quality; economic considerations; and harms concerns. Results: Nine partial or full listing recommendations were identified based on only placebo controlled trials, and 8 ‘do not list’ recommendations, based only on active comparator trials, were identified. Of the nine ‘list’ recommendations, factors were related to therapeutic need (3/9) and potential for cost savings (6/9). Reasons underlying the 8 ‘do not list’ recommendations related to cost/cost-effectiveness (8/8; increased cost or uncertainty around cost-effectiveness), insufficiency of evidence versus comparators (7/8; lack of studies versus key comparators, lack of meaningful improvements over other active therapies, inconsistency of findings across studies), and study quality (6/8; issues regarding blinding or withdrawals/missing data, appropriateness of non-inferiority margin, other design issues). For the ‘do not list’ category, recommendations were largely made in consideration of ≥2 criteria. Conclusions: The absence of active comparator trials may not be the determinant of a recommendation as value judgments related to the reliability of clinical trial data, therapeutic need, and other factors play a role in CEDAC recommendations. P3B337 Proof of value decisions from proof of concept data: A review of Canadian formulary review recommendations Shukla V1 , O’Grady T1 , Clifford T1 , Hutton B1 1 Canadian Agency for Drugs and Technologies in Health, Canada Background: Regulatory approval of drugs is often based on trials designed to demonstrate proof of concept. However, such studies are not typically designed to simultaneously demonstrate proof of value in a formulary recommendations setting. Objectives: To enumerate types of inadequate information in drug submissions which received a ‘do not list’ recommendation from the Canadian Expert Drug Advisory Committee (CEDAC). Methods: CEDAC ‘do not list’ recommendations from 2009–2010 were reviewed. Recommendations with cited reasons involving inadequate information were identified, and recorded. A summary of these reasons was compiled. Results: Of Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 178 Table 1 (P3B337). Types of Inadequate Information in CEDAC ‘‘do not list’’ recommendations (2009–10). Frequency as a Cited Factor (% of total # of recommendations) Submissions based on only placebo-controlled trials (n = 8) Submissions based on only active comparator trials (n = 6) Submissions based on a combination of placebo controlled and active comparator studies (n = 5) Lack of active comparator trials 5 (63%) 1 (17%) 3 (60%) Insufficient evidence of meaningful benefit vs placebo 2 (25%) 0 (0%) 2 (40%) Insufficient evidence of meaningful benefit vs active therapy 0 (0%) 5 (83%) 1 (20%) Failure to provide data for a key outcome measure 2 (25%) 1 (17%) 2 (40%) Use of a surrogate outcome considered to be limited value 3 (38%) 0 (0%) 0 (0%) Uncertainty of response durability over time 1 (13%) 0 (0%) 0 (0%) Limited study population generalizability 1 (13%) 0 (0%) 1 (20%) Cited Influential Criteria 26 ‘do not list’ recommendations, a total of 19 (73%) were associated with one or more forms of inadequacy of information. Amongst them, 10/19 (53%) were associated with inadequate evidence of a clinically meaningful benefit compared to either placebo or an active comparator, 9/19 (47%) were associated with a lack of trials versus any active comparator or a particular active comparator of interest, 5/19 (26%) failed to report data for a key outcome measure, 3/19 (16%) used a surrogate outcome considered to be of limited value, and 2/19 (11%) were associated with limited generalizability in terms of the study populations. Table 1 summarizes and highlights variations in the types of information inadequacies that were noted according to the type of evidence used in the submission. Conclusions: Proof of concept studies don’t always provide sufficient information to show proof of value, and represent a challenge for recommendation panels. In such situations, value judgments play an important role due to uncertainties regarding the information available. Carefully planned trial designs involving relevant comparators, evaluating outcomes relevant to the interests of regulators, physicians and patients, will increase the likelihood of a recommendation for formulary listing. P3B338 Lowering lipids enhances the effect of statins in preventing cardiovascular outcomes Contreras J1 , Schmid C1 , Alsheikh-Ali A2 1 Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, USA; 2 Institute of Cardiac Sciences, Sheikh Khalifa Medical City, UAE Background: Multiple randomized clinical trials have demonstrated that lowering low density lipoprotein (LDL) cholesterol reduces cardiovascular (CV) outcomes (death, myocardial infarction (MI), stroke). Meta-analyses to date have focused on treatment comparisons with respect to single outcomes, not addressing the potential competing risks among these correlated events to compare multiple treatments Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. across multiple outcomes. Objectives: To assess the effect modification of changes in LDL and other lipid measures on a network of treatment effects applied to a set of correlated outcome measures using multiple treatments meta-analysis (MTM). Methods: Data from 23 large randomized controlled trials comparing lipid lowering agents, statins, fibrates and conventional treatment at various doses and with different amounts of follow up were combined. Each trial reported some or all of 6 mutually exclusive outcomes: fatal or non-fatal MI, fatal or non-fatal stroke, or other fatal or non-fatal events. We applied a new Bayesian multinomial multiple treatments meta-regression model to estimate the effect of the different treatments on the different outcomes at different lipid levels. This model could account for the missing information on some outcomes in some trials. Results: When LDL cholesterol was reduced by 30 mg/dl, the average risk reduction was 26% for MI, 18% for stroke and 10% for death. The reductions were further reduced for each outcome by 5% for every additional 10 mg/dl. Smaller effects were found for other lipid measures. Reductions were greatest for high dose statins. Conclusions: Meta-analysis of large statin trials demonstrates that statins effectively reduce different CV events at different rates and that these rates depend on the amount of lipid reduction achieved in the trials and on the doses employed. P3B339 Resource se compared to evidence of effectiveness as an indirect measure of clinical quality and a basis for improvement Harris J1 1 The Permanente Medical Group, USA Background: Analysis of large health care claims databases indicate that many tests and treatments are done for indications not supported by evidence, or in quantities in excess of those supported by evidence. Findings of this sort, coupled with variance in care for groups of cases of Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 179 similar diagnoses, suggest that health care efficiency and effectiveness, key elements of quality, could be improved by adhering to practices supported by evidence of effectiveness. Objectives: 1. To review and evaluate systematic reviews of tests and treatments for low back pain. 2. To present examples of care in excess of evidentiary support for low back complaints. 2. To suggest means to reduce inappropriate and variable use of health care resources. Methods: We identified a number of systematic reviews and evidence-based guidelines for effective care of low back pain. We assessed them and collected recommendations and time frames for tests and treatments. We analyzed very large medical claims databases from the states of California, Texas and Ohio and compared the tests and treatments used per case to evidence of effectiveness. Results: There is good evidence for the effectiveness or lack thereof of imaging, pharmaceutical treatment, physical modalities, and procedures for low back pain, radiculopathy, and other common diagnoses. Our studies revealed testing and treatment far in excess of evidentiary support. Higher levels of treatment are associated with worse outcomes, including disability absence. We also found some evidence of improvement in these patterns following organizational or public policy interventions, but in some cases it was not sustained. Conclusion: Evidence-based best practices for back care are not uniformly applied. There is wide variation in practice patterns. Effective means to improve the consistency and appropriateness of practice are needed to improve outcomes. P3B340 A Canadian example of facilitating the uptake of syntheses into public health practice Docorby K1 , Tirilis D1 , Husson H1 , Robeson P1 , McCrae L1 , Greco L1 1 McMaster University, Canada Background: The Canadian Institutes of Health Research (CIHR) are mandated to ensure the results of the research it funds lead to improved health and health systems. However, current knowledge translation (KT) efforts may be insufficient to promote the uptake of these CIHR funded reviews into policy and practice in Canada. The Cochrane Collaboration, review groups and systematic review authors face similar challenges in facilitating the uptake of reviews into health policy and practice. Objectives: The purpose of this project is to identify high quality CIHR funded syntheses relevant to public health practice, and to promote the uptake of this evidence into policy and practice. Methods: Eleven methodologically rigorously completed CIHR funded reviews were identified with another 12 reviews expected to be completed during 2011. Short summaries highlighting what the evidence is and corresponding implications for policy and practice will be written for each review. The multi-pronged KT strategy to promote uptake of the evidence includes tailored, electronic messages, webcasts, and topic specific online discussion groups moderated by a knowledge broker. Results: We will evaluate these KT strategies by tracking how often the summaries and full text reviews are accessed at www.health-evidence.ca. Conclusions: At the time of the Cochrane Colloquium in October 2011, we will be nearing the completion of the KT strategy for this project and will be able to report on the extent to which we have been able to reach and engage the public health community in these CIHR funded syntheses. Lessons learned from this project will provide useful insight to those intent on promoting the uptake of Cochrane reviews. Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. P3B341 Bringing best evidence to front line clinicians Tejani A1 , Adlparvar C2 1 Fraser Health Pharmacy Services, Vancouver, Canada; 2 Therapeutics Initiative & Hypertension Review Group, Vancouver, Canada Background: Continuing education activities for clinicians are often not based on best evidence and have relied on pharmaceutical industry support, which can be problematic. The Therapeutics Initiative has developed a strategy to overcome these issues. Objectives: 1. To provide ethically sound source of knowledge based on systematic reviews that is not dependent on industry funding. 2. To establish a sustainable and collaborative knowledge exchange environment between physicians and pharmacists. 3. To develop tools to transfer evidence from systematic reviews into practice that accommodate different learning styles and different access to educational opportunities. Methods: We formed of a team of knowledge-users, academic researchers, and educators to develop a comprehensive multi-faceted continuing education strategy. We created learning environment that enables physicians and pharmacists to learn together. 3) developed the following educational tools that cater to the needs of different clinicians with different warning styles and different access to continuing education opportunities. • website with access to all educational tools • medical evidence podcasts • evidence summaries • case-based workshops • critical appraisal courses. We developed an educational strategy that is primarily grant funded and has no pharmaceutical industry input. Results: There were three educational events with 15 topics and over 250 practitioners in which over 90% of participants agreed; • the information was practical, • it was productive to work with colleague, • provided excellent information and • would attend again. Material for these events was used subsequently for over 50 presentations with smaller groups at hospitals in British Columbia (BC) Canada. There were over 20 presentations onteaching clinicians how to critically appraise the literature, four therapeutic letters which tere senw to over 14000 physicians and pharmacists in BC and over 5000 online users five podcast which elaborate on the systematic reviews and newsletters. Conclusions: The Therapeutics Initiative has been successful in providing best evidence to front-line clinicians in an ethically sound environment. P3B342 Evidence to support process improvement: What’s available, and how to proceed Harris J1 1 The Permanente Medical Group and the Care Management Institute, USA Background: Many processes of health care could be improved. Adherence to evidence-based best practices, and reduction in unwanted variation in care and prevention should significantly improve population health and better manage care provided through health care systems. Objectives: 1. To review the availability and quality of evidence that could support important process improvement (PI) efforts in clinical areas such as organ failure in sepsis, ventilator-acquired pneumonia, central line infections, acute coronary syndrome and stroke, and prevention of communicable disease, cancer, and cardiovascular disease. 2. To describe evidence and efforts to implement evidence-based practices consistently in a large health care delivery system. Methods: We searched the Cochrane Library, major data Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 180 bases, and guidelines for studies and systematic reviews on healthcare process improvement, preventive services, and PI implementation for preventive screening, immunizations, and care for sepsis, coronary disease, stroke, and pneumonia. We characterized the quality of the evidence and reviewed process improvement projects at Kaiser Permanente in these areas. Results: Evidence is generally observational for clinical process improvement, immunizations and screening. There was good evidence for the testing and treatment portions of process improvement protocols, but less so for the entire integrated process. Nevertheless, positive results were demonstrated in most areas. Conclusions: Trials are most often focused on specific treatments rather than processes of care. Processes occur over time, with a number of sequential or parallel components. The quality and availability of evidence to support health care processes varies significantly and can require chains of evidence, which are indirect by definition. Outcomes can include positive or adverse events, resource use, and operational efficiency as well as intermediate and final clinical states. Even when there is good evidence, how to have providers to do the right thing consistently remains an open question. P3B343 Cochrane Journal Club: Going from strength to strength? Curtis T1 1 Wiley-Blackwell, John Wiley & Sons, Chichester, UK Background: In October 2009, Cochrane and Wiley-Blackwell launched the Cochrane Journal Club (CJC) http://www.cochranejournal club.com/ to promote an increase in awareness of research, educate students to critique and use research findings, and to aid in the translation of research into practice. By the time of the Colloquium, the CJC will have been published monthly for almost two years. In order to continue to meet the needs of the users the CJC needs to reflect on the number and type of users and whether the initial objectives are still being met. Objectives: To analyze data for the first 18 months to find out who is using the Cochrane Journal Club in terms of numbers to the site, job type and country. To survey a random sample of users to find out whether CJC met their educational requirements. Methods: Since the first issue visitor-data, including country and job type, has been collected. Users are also invited to ‘join the club’ and register. It is a sample of these users who will be invited to provide information regarding educational use via an online survey. Results: Eighteen months of visitor data will be analysed and presented; early indications are that over 3,200 unique users visit the site each month, with just under 1000 users returning each month. To date, just over 4,200 have registered to join the Cochrane Journal Club and to receive email alerts about new journal club topics. Conclusion: Early indications are that the Cochrane Journal Club user-numbers have exceeded expectations, but through the online user survey we need to ensure that the content supplied meets and continues to meet the educational requirements of these users. P3B344 Who is listening? A qualitative study of Cochrane podcast users Stewart G1 , Mavergames C2 1 Wiley-Blackwell, Chichester, UK; 2 German Cochrane Centre, Freiburg, Germany Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: Podcasts from the Cochrane Library were first launched in January 2008. Since then, the number of listeners worldwide has increased substantially; more than 100,000 accesses per month were achieved in 2010. Previous quantitative studies have provided evidence to show that the Cochrane podcasts have grown in popularity but have not demonstrated why users are choosing to listen. A qualitative study will enable the podcasts to develop further and continue to serve the needs of the listeners. Methods: A user survey targeted toward existing users of the Cochrane podcasts will be conducted. Social networking outlets including Facebook and Twitter will be used in order to ask listeners a number of questions relating to their usage of the Cochrane podcasts. A small number of previous podcast authors will be asked to share their views on the current format. Results: The results from the survey will give a more detailed impression of the users of the Cochrane podcasts. The strengths and weaknesses of the current podcast format that the users identify will be used to improve the service for listeners. Conclusions: How the podcasts are produced and presented in the future will be influenced by the outcome of the user survey. P3B345 Cochrane review of school-based interventions for the prevention of human immunodeficiency virus (HIV) and sexually transmitted infections (STIs) in adolescents: Why biological outcomes are important Mason-Jones A1 , Mathews C2 , Kagee A3 , Lombard C4 1 Specialist Scientist, Health Systems Research Unit, South African Medical Research Council, Senior Lecturer, Adolescent Health Research Unit, University of Cape Town; 2 Associate Professor, Department of Family Medicine and Public Health, University of Cape Town, Chief Specialist Scientist, Health Systems Research Unit, Medical Research Council; 3 Department of Psychology, Stellenbosch University, South Africa; 4 Department of Biostatistics, South African Medical Research Council Background: Self-report outcome measures of sexual behaviour are the mainstay of most HIV/STI prevention trials amongst adolescents. All reviews to date have focussed on sexual risk behaviour as the primary outcome. This is partly due to the large sample sizes required to demonstrate a reduction in incidence of HIV or STIs. However, self-report measures are prone to recall and social desirability bias (Plummer 2004; Langhaug 2010) and the relationship between such outcomes and HIV or STIs is not known. Biological measures, such as incidence of HIV or STIs are the most convincing indicators of the success of prevention interventions. Objective: To summarise the effect of school-based HIV prevention interventions on the incidence and prevalence of HIV and STIs and on sexual risk behaviour among adolescents. Methods: We conducted a comprehensive search for randomised controlled trials, published between 1985 and 2010, evaluating school-based HIV infection prevention interventions in which the unit of randomisation was individual students, classrooms, schools or communities, and in which the outcome measures included HIV and/or other STI incidence or prevalence. Results: Three recent large randomised controlled trials of HIV prevention interventions among adolescents and youth in sub-Saharan Africa have included biological measures of STIs as their primary outcomes in addition to self-reported sexual risk behaviour (Cowan 2010; Ross 2007; Jewkes 2008). These Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 181 trials demonstrate that it is possible to include biological outcomes in adolescent HIV prevention trials. Conclusions: Future systematic reviews and individual intervention studies should include biological outcome measures. The development and validation of acceptable and valid biological measures should be prioritized and included, where feasible, in future studies and trials of interventions to reduce sexual health risk. Statistical methods Day 22 Oct (P3B346–P3B381) P3B346 Trial sequential analyses of meta-analyses from a Cochrane review of targeting intensive versus conventional glycaemic control in patients with type 2 diabetes Hemmingsen B1 , Lund S2 , Gluud C1 , Vaag A3 , Almdal T2 , Hemmingsen C1 , Wetterslev J1 1 Copenhagen Trial Unit, Centre of Clinical Intervention Research, Rigshospitalet, Copenhagen, Denmark; 2 Steno Diabetes Center, Gentofte, Denmark; 3 Department of Endocrinology, Rigshospitalet, Copenhagen, Denmark Background: Cumulative meta-analysis may increase the risk of type 1 and 2 errors due to sparse data and repetitive testing on accumulating data. To limit risk of random errors, meta-analyses may be analysed with trial sequential analysis (TSA) using heterogeneity-adjusted required information size to determine when convincing evidence is reached. Methods: Meta-analyses of the effects of targeting intensive versus conventional glycaemic control on mortality, non-fatal myocardial infarction, and severe hypoglycaemia in patients with type 2 diabetes were conducted. TSA was applied to limit the overall type 1 error to 5% and type 2 error to 20%. We used a heterogeneity-adjusted required information size calculated for a 10% relative risk reduction (RRR) of mortality and of non-fatal myocardial infarction and a 30% RRR of severe hypoglycaemia corresponding to numbers needed to treat (harm) of 50–100. Results: The relative risk reduction of all-cause mortality and non-fatal myocardial infarction were 1.02, 95% CI 0.91 to 1.13; I2 = 30% (Figure 1) and 0.85, 95% CI 0.76 to 0.95; I2 = 0% (Figure 2). TSA showed that a 10% RRR or more can be rejected for all-cause mortality (Figure 3). TSA showed lack of firm evidence for a benefit of targeting intensive glycemic control for non-fatal myocardial infarction. Only 27,958 participants (44%) of the heterogeneity-adjusted required information size of 63,446 have been accrued to detect a 10% RRR of non-fatal myocardial infarction (Figure 4). The RR of serious hypoglycaemia was 2.18, 95% Cl 1.47 to 3.23; I2 = 73% (Figure 5). TSA showed firm evidence for at least a 30% increase of severe hypoglycaemia with intensive glycaemic control (Figure 6). Conclusion: Intensive glycaemic control does not seem to reduce all-cause mortality, and if so then less than 10%. A 10% RRR of non-fatal myocardial infarction could not be confirmed. TSA confirmed that intensive glycaemic control increases the RR of severe hypoglycaemia with 30%. Figure 1 (P3B346). Figure 2 (P3B346). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 182 Figure 3 (P3B346). Figure 4 (P3B346). Figure 5 (P3B346). Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 183 Figure 6 (P3B346). P3B347 A comparison of subgroup analyses in grant applications and publications Boonacker CW1 , Hoes AW1 , van Liere-Visser K2 , Schilder AG3 , Rovers MM4 1 Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Netherlands; 2 ZonMw, Netherlands Organisation for Health Research and Development, The Hague; 3 Department of Otorhinolaryngology, Wilhelmina Children’s Hospital, University Medical Center Utrecht, Netherlands & Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Netherlands; 4 Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Netherlands & Department of Otorhinolaryngology, Wilhelmina Children’s Hospital, University Medical Center Utrecht, Netherlands Background: One of the most important recommendations of the available guidelines on studying and reporting subgroup analyses, is to pre-specify subgroups rather than define them post hoc. We therefore studied both grant proposals and their publications and compared the subgroup analyses that were pre-specified in the grant proposal to those that were finally published. Objectives: To compare grant applications and final publications regarding subgroup analyses. Methods: Grants awarded by the ‘Netherlands Organization for Health Research and Development’ from 2001, were studied. We analyzed whether or not projects mentioned subgroups in their grant application and related publications (i.e. the final report and scientific publications). The main outcome measure was the proportion of studies in which the publications were completely in agreement with the grant proposal, i.e., subgroups that were pre-specified in the grant proposal were reported and no new subgroup analyses were introduced in the publications. Of all individual subgroups that could be identified in the included projects, we analyzed if they were pre-specified or a post-hoc finding. Results: In 20 (25%) of the 79 included projects, publications were completely in agreement with the grant proposal. Of Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. the 149 pre-specified subgroups, 46 (31%) were reported in the final report or scientific publications, and 143 of the 189 (76%) reported subgroups were based on post-hoc findings. For 77% of the subgroup analyses in the publications, there was no mention whether these were pre-specified or post-hoc. Justification for subgroup analysis and methods to study subgroups were rarely reported. Conclusion: There is a large discrepancy between grant applications and final publications regarding subgroup analyses. Both non-reporting of pre-specified subgroup analyses and reporting of post-hoc subgroup analyses are common. P3B348 The impact of underpowered studies in meta-analyses reported by Cochrane Reviews Turner R1 , Bird S1 , Higgins J1 1 MRC Biostatistics Unit, UK Background: Most meta-analyses include data from one or more small studies, which would not themselves have power to detect an intervention effect. The relative influence of adequately powered and underpowered studies in published meta-analyses has not previously been explored. Objectives: To examine the amount of power available in studies included in meta-analyses reported by Cochrane reviews, and to investigate the impact of underpowered studies on meta-analysis results. Methods: Our analyses included 14,886 meta-analyses of binary outcomes extracted from 1,991 Cochrane reviews. For each study in each meta-analysis, we calculated the power available to detect a 30% relative risk reduction, based on mean study arm size and prevalence of the outcome. Associations between meta-analysis characteristics and power were examined. In a subset of the data set, meta-analyses were repeated with underpowered studies excluded. Results: In 81% of meta-analyses in this data set, all studies Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 184 included were underpowered (power < 50%) to detect a relative risk reduction of 30%. Only 10% of meta-analyses included at least two adequately powered studies (power ≥ 50%). The amount of power typically available varied across medical areas and outcome types. In meta-analyses including at least two adequately powered studies and at least one underpowered study, differences between results based on adequately powered studies alone and results based on all studies were small on average. The standard error of the intervention effect increased by a median of 12% (inter-quartile (IQ) range −1% to 35%) when underpowered studies were omitted. The between-study heterogeneity estimate decreased by a median of 27% (IQ range 100% decrease to 14% increase). Conclusions: In the majority of meta-analyses reported by Cochrane reviews, underpowered studies make up the entirety of the evidence. However, for topics in which at least two adequately powered studies are available, underpowered studies often contribute little information, which raises questions about scientific efficiency. P3B349 Quantifying, displaying and accounting for heterogeneity in the meta-analysis of RCTs using standard and generalised Q statistics Bowden J1 , Tierney J2 , Copas A2 , Burdett S2 1 MRC Biostatistics Unit; 2 MRC Clinical Trials Unit Background: Clinical researchers have often preferred to use a fixed effects model for the primary interpretation of a meta-analysis. Heterogeneity is usually assessed via the well known Q and I2 statistics, along with the random effects estimate they imply. In recent years, alternative methods for quantifying heterogeneity have been proposed, that are based on a ‘generalised’ Q statistic. Methods: We review 18 IPD meta-analyses of RCTs into treatments for cancer, in order to quantify the amount of heterogeneity present and also to discuss practical methods for explaining heterogeneity. Results: Differing results were obtained when the standard Q and I2 statistics were used to test for the presence of heterogeneity. The two meta-analyses with the largest amount of heterogeneity were investigated further, and on inspection the straightforward application of a random effects model was not deemed appropriate. Compared to the standard Q statistic, the generalised Q statistic provided a more accurate platform for estimating the amount of heterogeneity in the 18 meta-analyses. Conclusions: Explaining heterogeneity via the pre-specification of trial subgroups, graphical diagnostic tools and sensitivity analyses produced a more desirable outcome than an automatic application of the random effects model. Generalised Q statistic methods for quantifying and adjusting for heterogeneity should be incorporated as standard into statistical software. Software is provided to help achieve this aim. P3B350 Individual participant data meta-analysis of prognostic factor studies: State of the art? Riley R1 , Abo-Zaid G2 , Sauerbrei W3 1 University of Birmingham, UK; 2 University of Birmingham, UK; 3 University Medical Center Freiburg, Germany Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: Prognostic factors are associated with the risk of a subsequent outcome in people with a given disease or health condition. They have a broad array of uses; for example, they help define disease at diagnosis; they inform individual treatment strategies; and they inform the design and analysis of clinical trials. Studies to identify prognostic factors are abundant in the literature, and meta-analysis is needed to combine them. Methods: Meta-analysis using individual participant data (IPD), where the raw data are synthesised from multiple studies, has been championed as the gold-standard for synthesising prognostic factor studies. In this talk I will examine the feasibility and conduct of this approach, using a systematic review of currently published IPD meta-analyses of prognostic factors studies. Results: Forty-eight published IPD meta-analyses of prognostic factors were identified; only three were published before 2000 but thereafter a median of four articles exist per year. These show that having IPD is advantageous, allowing modelling assumptions to be checked; variables to be analysed on their continuous scale; non-linear relationships to be examined; and results to be adjusted for other variables. However, the IPD approach also raises many challenges, such as large cost and time required to obtain and clean IPD; unavailable IPD for some studies; different sets of prognostic factors in each study; and variability in study methods of measurement. Their conduct can also be improved; continuous variables are often categorised without reason and more sophisticated methods to meta-analyse them are required; and publication bias is rarely examined. Conclusions: IPD meta-analysis of prognostic factor studies is a huge improvement over an aggregate data meta-analysis approach. However, many practical, methodological and clinical problems still remain. The gold-standard is a prospectively planned meta-analysis using IPD. P3B351 The effect of the assessment goal on the interpretation of subgroup findings and the corresponding conclusions Bender R1 1 Department of Medical Biometry, Institute for Quality and Efficiency in Health Care, Germany Background: The interpretation of subgroup analyses in clinical research is still challenging. Beside statistical criteria such as preor post-specification, multiplicity, power, and the applied statistical technique, the reason for conducting subgroup analyses plays a role. Moreover, the interpretation of subgroup findings may be dependent on the context in which the corresponding results are assessed. Objectives: To discuss the effect of the assessment goal on the interpretation of subgroup analyses and the corresponding conclusions in the context of drug approval and reimbursement. Methods: By means of an example it is discussed and demonstrated that the different perspectives of drug approval and reimbursement can lead to different conclusions regarding subgroup analyses. Results: Noninferiority to an established drug is a plausible basis for licensing a safe new drug. However, reimbursement at higher costs requires an additional clinical benefit, which means that superior efficacy of the new drug compared to the established drug has to be demonstrated. In the situation of similar efficacy of the new and the established drug in one subgroup and superior efficacy of the new drug in a second subgroup, it would be adequate to approve the new drug for the whole population but restrict reimbursement to the second subgroup of patients. Conclusions: Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 185 Different assessment goals of different institutes or authorities may lead to different conclusions regarding subgroup analyses. The specific context of decisions has to be considered in the interpretation of the credibility and importance of subgroup findings. P3B352 Exploring treatment by covariate interactions in mixed treatment comparison meta-analysis: Individual patientlevel covariates versus aggregate trial-level covariates Donegan S1 , Williamson P1 , D’Alessandro U2 , Tudur-Smith C1 1 University of Liverpool, UK; 2 Prince Leopold Institute of Tropical Medicine, Belgium Background: Mixed treatment comparison (MTC) meta-analysis allows several treatment options to be compared simultaneously in a single analysis while utilising both direct and indirect evidence. Three key assumptions underlie the methodology: similarity; homogeneity; and consistency. Assessment of the assumptions is vital to ensure the results of the MTC meta-analysis are interpreted appropriately. A previously proposed assessment method is the inclusion of potential treatment effect modifying covariates in the MTC meta-analysis models. For conventional pair-wise meta-analysis, important benefits regarding the investigation of treatment by covariate interactions, gained from using individual patient data rather than aggregate data for meta-analysis, have previously been described. Objectives: We aim to compare individual patient data MTC meta-analysis models that include patient-level covariates with aggregate data MTC meta-analysis models that include study-level covariates. Methods: Two types of random effects MTC models for dichotomous outcomes were applied: models based on the individual patient data from the trials’ original datasets and models based on the ‘aggregate’ event rates from each trial. For each type of MTC model, three different model specifications were applied that made increasingly stronger assumptions regarding the treatment by covariate interactions. We compared the two types of models through application to a real dataset. To ensure that any differences in the results would be due to the model specifications, we used the real individual patient dataset to generate the aggregate data. Results: We found that the treatment effects and drug rankings from MTC models that include patient-level covariates and those from models that included study-level covariates differed. The inclusion of patient-level rather than study-level covariates produced more precise results. Conclusions: When assessing the similarity assumption of a MTC model, including patient-level covariates is more favourable than including study-level data when the covariate distributions vary within trials. P3B353 Is it reasonable to pool estimates of intervention effect estimated from different analytical methods for continuous outcomes? A meta-epidemiological study McKenzie JE1 , Deeks JJ2 1 School of Public Health and Preventive Medicine, Monash University, Australia; 2 Public Health, Epidemiology and Biostatistics, University of Birmingham, United Kingdom Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. Background: In randomised trials, continuous data are often measured before and after the intervention. Common analytical methods include analysing final values (FV), analysing change scores (CS) and analysis of covariance (ANCOVA). These approaches provide unbiased estimates of the same true underlying intervention effect. Theoretically, pooling effects from a mix of these methods should yield unbiased estimates of pooled intervention effect. However, the meta-analytic estimates may be biased if the trialists selectively report results. Objectives: To investigate if the pooled estimate of intervention effect and its standard error were modified by the analytical methods employed in the component trials. Methods: Fifty-four meta-analyses were selected from The Cochrane Database of Systematic Reviews (2004, Issue 4). Twenty met the inclusion criteria of at least four comparisons from parallel group trials with enough data available to extract at least one effect estimate, with a mix of analytical methods across the comparisons. Pooling effects using a mix FV and CS were compared with: pooling effects from i) one analytical method (either all FV or all CS), ii) a mix of FV, CS, and ANCOVA, iii) all ANCOVA. A ‘meta-meta-analysis’ was undertaken to investigate if the pooled effect was modified by the use of either FV or CS analyses. Results: Analyses are suggestive of selective reporting of analytical method by baseline imbalance. However, the impact of this on the random effects meta-analytical estimates was less clear. Inconsistency of effects (I2 ) varied: greater consistency was observed with approach i) and greater inconsistency with approach iii). Standard errors, p-values, and between trial heterogeneity were similar between the approaches. Conclusions: Pooling effects from a mix of analytical methods is appealing, however it may be problematic. The inclusion of ANCOVA estimates did not result in expected gains in terms of increased precision of the meta-analytic estimates. P3B354 IPD meta-analysis of time-to-event data: One-stage versus two-stage approaches to estimating the hazard ratio Bowden J1 , Tierney J2 , Simmonds M3 , Copas A2 , Higgins J1 1 MRC Biostatistics Unit; 2 MRC Clinical Trials Unit; 3 Queen Mary University of London Background: Meta-analyses of individual patient data (IPD) provide a strong and authoritative basis for evidence synthesis. IPD are particularly useful when the outcome of interest is the time to an event. Methodological developments now enable the meta-analysis of time-to-event IPD using a single model, allowing treatment effect and across-trial heterogeneity parameters to be estimated simultaneously. This differs from the standard approaches used with aggregate data, and also predominantly with IPD. Methods: Facilitated by a simulation study, we investigate what these new ‘one-stage’ random-effects models offer over standard ‘two-stage’ approaches. Results and Conclusions: We find that two-stage approaches represent a robust, reliable and easily implementable way to estimate treatment effects and account for heterogeneity. Nevertheless, one-stage models can be used to provide a deeper insight into the data. Software for fitting one-stage Cox models with random effects using REML methodology is made available, and its use demonstrated on an IPD meta-analysis assessing post operative radio therapy for patients with non-small cell lung cancer. Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 186 P3B355 The use of an ordered multinomial logit model in network analysis: An example in psoriatic arthritis Craig D1 , Epstein D2 , Yang H1 , Bojke L2 , Sculpher M2 , Woolacott N1 1 CRD, University of York. UK; 2 CHE, University of York, UK Background: Bayesian network meta-analyses (NMA) are an extension of classic meta-analysis, but where a meta-analysis includes only direct evidence, NMAs draw on both direct and indirect evidence. The lack of head-to-head trials for newer drugs has seen an increase in the use of NMAs. When appropriate trials are to be synthesised, model choice for NMA is extensive. Researchers should not be limited to commonly reported methods, but should select models which appropriately capture the relationship of the outcomes being synthesised. Objectives: To demonstrate the appropriate application of a hierarchical multinomial ordered logit model for a network meta-analysis in the context of a recent systematic review and decision model. Methods: A NMA was used to estimate the probability that patients have a Psoriasis Area and Severity Index (PASI) 50/75/90 response or American College of Rheumatology (ACR) 20/50/70 response. These outcomes represent percentage reduction, have an order and are clearly related. The models were implemented in WinBUGS. The hierarchical multinomial ordered logit model, allowed an estimation of the probability of achieving one level of response compared with another as a linear function of the explanatory variables. Two separate analyses were undertaken for the outcomes of PASI 50/75/90 response and ACR 20/50/70 response. We estimated the probability that patients have a PASI 50/75/90 or ACR20/50/70 response by means of a cumulative logistic model. All priors were non-informative. Alternative modelling scenarios/assumptions were assessed. Results and Conclusions: The models presented provided flexible methods to ensure that the relationship between the ordered outcomes was appropriately maintained and synthesised. In the absence of head-to-head trials, and outcomes that are clearly related, researchers need to look beyond the standard models in our toolkits and use methods that deal appropriately with the clinical outcome of interest. P3B356 Classification of methods for evaluation of treatment selection markers: A systematic review Tajik P1 , Mol BW2 , Bossuyt PM1 1 Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Center, University of Amsterdam; 2 Department of Obstetrics and Gynecology, Academic Medical Center, University of Amsterdam Background: There is an increasing interest in the evaluation of biomarkers useful for treatment selection. However, the literature on the methods for the evaluation of these markers is diverse and there is no consensus about their classification and nomenclature. Objectives: To systematically review and generate a classification of study methods to identify and/or validate treatment selection markers, and to standardize the nomenclature. Methods: We searched MEDLINE (1950 to Jan 2011), EMBASE (1980 to Jan 2011), Cochrane Methodology Register and MathSciNet (up to the Jan 2011), and handsearched the citations Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. of included studies. We included methodology reports that described one or more study design or analysis plans related to identification and validation of treatment selection markers. Results: Our search identified 1758 abstracts, among which 82 were found to be probably eligible. After full-text review, 36 studies were included. The following dimensions were identified for the classification of methods: trial phase, patient flow, randomization type, sample size requirements, timing, the number of evaluated markers, analysis plan, and summary statistics. For each study design advantages and limitations were described and the efficiency of methods were compared with each other. Using identified dimensions, all methods were classified and identical methods received a unique terminology. Conclusions: We propose a standard nomenclature and classification system for study designs of treatment selection marker evaluation. Using this classification system could help researchers in designing future studies and reduce reporting inconsistencies. P3B357 Adverse events of antiepileptic drugs, across indications: Can randomised controlled trial data from non-epilepsy indications be included in meta-analysis for AEDs used in epilepsy? Tudur-Smith C1 , Marson A1 1 University of Liverpool Background: Adverse Events (AEs) are reported in randomised controlled trials (RCTs) of epilepsy and non-epilepsy trials using the same antiepileptic drug (AED). Meta-analysis of these trials would increase study power. Objectives: To determine if AE outcomes from RCTs of AEDs across non-epilepsy indications (neuropathy or migraine) can be meta-analysed with data from epilepsy trials. Methods: We searched databases for RCTs meeting inclusion criteria. AEDs included were topiramate, gabapentin, valproate, oxcarbazepine, lacosamide, lamotrigine, carisbamate, zonisamide and pregabalin. Extracted data was analysed using RevMan V5.0. Common AEs analysed were; dizziness, ataxia, headache, fatigue, nausea, somnolence, withdrawals due to AE and any AE. Summary statistics of effect size were calculated using the Mantel-Haenszel method. Statistical heterogeneity was assessed using a random effects model. To test between indications we used a fixed effects model and calculated an I2 statistic. Results: We included 106 RCTs for analysis. Effect sizes varied with drug and outcome. When dizziness was analysed, test between indications showed no statistical heterogeneity (I2 = 0%) for gabapentin, topiramate, lacosamide and lamotrigine. However, heterogeneity was significant (I2 = 59%) for trials of oxcarbazepine. When somnolence was the AE outcome, heterogeneity was insignificant for oxcarbazepine (I2 = 8%), lacosamide (I2 = 0%) and topiramate (I2 = 0%), but significant for gabapentin (I2 = 56%) and lamotrigine (I2 = 60%). When nausea was analysed there was no heterogeneity (I2 = 0%) for lacosamide, oxcarbazepine and topiramate, but significant heterogeneity existed for gabapentin (I2 = 41%) and lamotrigine (I2 = 40%). In instances where there was significant heterogeneity, the size of relative risk was greater in the non-epilepsy indications. Conclusions: AEs of AEDs from non-epilepsy trials could be used in meta-analysis given the absence of statistical heterogeneity for some interventions and outcomes. Effect sizes were larger in the non-epilepsy indications overall. Further Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 187 meta-regression would unmask any dose effect on heterogeneity and effect size. P3B358 Exploring the effects of different treatment doses by ‘blurring’ the nodes of a treatment network: Application to a Cochrane overview for multiple sclerosis Del Giovane C1 , Mavridis D2 , Vacchi L3 , Filippini G3 , Salanti G2 1 University of Modena – Italian Cochrane Center and Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina Greece; 2 Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina Greece; 3 Neuroepidemiology Unit, Fondazione I.R.C.C.S. Istituto Neurologico Carlo Besta, Milano, Italy Background: Several immunotherapies exist for the management of multiple sclerosis and their relative efficacy can be evaluated via Multiple-Treatments Meta-analysis (MTM). Some agents are administered at different doses, but the impact of increasing the dose on delaying the disease progression is still unclear. Therefore it is uncertain whether different doses of the same agent can be considered to form a common ‘node’ in the treatment network. Objectives: a) to explore a series of MTM models where the uncertainty about the impact of dose is embedded in the definition of each node in the treatment network b) to evaluate the assumptions that dose effects are the same, randomly different but exchangeable or follow a monotonic pattern. Methods: We developed Bayesian models considering the different node definitions. We started with the network where each agent, whatever the dose, defines a node. We gradually ‘blurred’ the nodes so that different dose effects of the same agent are: random but exchangeable; isotonic or have a linear pattern around the agent’s mean effect. We ended up with the network where each different dose defines an independent node. Each model has different implications for the assumption of consistency of effect (either at agent level or at dose level). The models were compared using goodness of fit criteria; changes in heterogeneity and inconsistency were monitored. Results: Models for linear and independent dose effects had poor fit whereas exchangeable dose effects models had the best fit and lowest heterogeneity. This suggests that the dose effects of the same agent approximate the mean agent effect and hence a dose-response association is unlikely. Conclusions: The ‘node-blurring’ approach can be a useful exploratory tool when there is doubt whether similar interventions should be grouped under the same node in a network; a dilemma which is frequently encountered in MTM. P3B359 Meta-analysis of diagnostic studies: SROC modelling – the profile log-likelihood approach based upon the proportional hazard model Charoensawat S1 , Böhning D2 1 Udon Thani Rajabhat University, Thailand; 2 University of Reading, United Kingdom Background: The number of meta-analysis of diagnostic studies is increasing and models that deal with the summary receiver operating Copyright c2011 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd. characteristics (SROC) approach have become popular. These models have become complex and difficult to use which led us to develop a simple model, the proportional hazard model (PHM). Objective: To improve SROC modelling based upon PHM by finding the most efficient estimate of the diagnostic accuracy parameter θ. Methods: A simulation study was done to investigate the performance of the profile log-likelihood (PMLE), the adjusted profile log-likelihood (APMLE) and the iterative weighted least square (IWLS) approaches. The performances of approaches were found from score function and the Fisher information I(θ). The heterogeneity was considered and the APMLE with the random effect was developed. Maximum likelihood estimation was used to find the estimate of θ and tau2 , two parameters in the model. Also, the variance-covariance matrix was used to determine the inverse of I(θ) and I(tau2 ) and to find the estimate of θ and tau2 of I(θ). Meta-analysis of diagnostic studies of exercise echocardiography (ECHO) and exercise single photon emission compute tomography (SPECT) imaging in the diagnosis of coronary artery disease was exemplified under homogeneity and heterogeneity. Results and Conclusion: PMLE turned out to be an easy approach to eliminate nuisance parameter from the model but a normalizing constant was missing. APMLE has been suggested for correcting the shortcoming of the PMLE. In the simulation study, likelihoods behaved similarly, but the APMLE was the most efficient estimator. In our example of meta-analysis of diagnostic studies, θ of ECHO was smaller which meant it had higher diagnostic accuracy than the SPECT device, since the smaller the θ the higher the diagnostic power. p − values became non-significant when tau2 was incorporated. P3B360 How to improve the credibility of meta-analysis? How to avoid mistakes in choosing the statistical method of estimation? The Bayesian and traditional approaches Nikodem M1 , Walczak J1 , Borowiack E2 , Siedmiogrodzki K1 , Zapalska A2 , Khan K3 , Meads C4 , Mol B5 , Oude Rengerink K5 , Thangaratinam S3 , Zamora J6 1 CASPolska Association, Poland; 2 Arcana Institute, Poland; 3 Queen Mary, University of London, Great Britain; 4 Queen Mary, University of London; 5 Academic Medical Center Amsterdam, Netherlands; 6 Hospital Universitario Ramon y Cajal, Madrid, Spain Background: There are at least ten different statistical methods of meta-analysis. It can be surprising how different results of estimation can be obtained just by applying different method. Therefore, if we want to run the meta-analysis properly then we have to solve the crucial problem of choosing the most credible statistical method. This is especially important in the case of small sample size, low event rate or large discrepancy between event rates in groups of comparison. Objectives: To find out which methods are the most credible and precise for particular data type. Moreover to underline which methods should not be used to avoid substantial errors in estimation. Our main aim is to answer the question of how to deal with following problems: a. insufficient sample size, b. low event rate. Methods: Initially a systematic review on existing statistical methods was conducted. It was followed by an expanded analysis of credibility and precision of statistical methods. Ten statistical methods (including alternative methods such as Bayesian ones combining results of nonrandomized with randomized trials) were examined on different data sets. Overall Cochrane Database Syst Rev Suppl 3–233 (2011) DOI:10.1002/14651858.CD000003 188 Figure 1 (P3B360). not disappear completely, even in very large datasets. Conclusions: The authors conclude that the proposed method to incorporate previously published univariable predictor-outcome associations as evidence in new multivariable prediction analyses is superior to established approaches and is especially worthwhile when relatively limited individual patient data are available. P3B362 Aggregating published prediction models with individual patient data: A comparison of different approaches 124 sub-cases (different sample size, event rate, heterogeneity, etc.) were considered. Bias, confidence interval coverage and statistical power were the measures of credibility and precision. Results: Possible datasets were categorized and for particular categories the optimal methods are highlighted and warnings of possible mistakes are underlined. All results are presented in the form of guidelines. Conclusions: There is no universal method which is credible for every set of data, therefore the choice of method should be done carefully and depending on the data characteristics. For several cases, the Bayesian approach (with or without inclusion of non-randomized trials) was highly ranked, hence it should be taken under greater consideration. P3B361 Meta-analytical approaches for diagnostic and prognostic prediction research: Incorporating published univariable associations in multivariable analysis of individual patient data Debray T1 , Koffijberg H1 , Lu D2 , Vergouwe Y3 , Steyerberg EW2 , Moons KGM1 1 Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Netherlands; 2 Department of Public Health, Erasmus MC, Netherlands; 3 Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Netherlands – Department of Public Health, Erasmus MC, Netherlands Background: Diagnostic and prognostic literature is overwhelmed with studies reporting univariable associations. Currently, methods to incorporate such information in multivariable analysis of individual patient data (IPD) are