Download Basic study design influences the results of orthodontic clinical

Title Page
Basic study design influences the results of orthodontic clinical investigations
Spyridon N. Papageorgiou a,b,c*, Guilherme M. Xavierd, Martyn T. Cobourned
a
Department of Orthodontics, School of Dentistry, University of Bonn, 53111, Bonn, Germany
b
Department of Oral Technology, School of Dentistry, University of Bonn, 53111, Bonn,
Germany
c
d
Clinical Research Unit 208, University of Bonn, 53111, Bonn, Germany
Department of Orthodontics, King's College London Dental Institute, Floor 27, Guy's Hospital,
SE1 9RT, London, UK
*Corresponding author:
Spyridon N. Papageorgiou
Orthodontic Resident
Department of Orthodontics, School of Dentistry
University of Bonn
Welschnonnenstr. 17
53111 Bonn, Germany
Tel.: +49 (0)228-287 22449
Fax: +49 (0)228-287 22588
E-mail: [email protected]
Words in main text: 3468
1
Abstract
Objectives: Meta-analysis is the gold standard for synthesizing evidence on the effectiveness of
healthcare interventions. However, its validity is dependent upon the quality of included studies.
Here, we investigated whether basic study design (i.e. randomization and timing of data
collection) in orthodontic research influences the results of clinical trials.
Study Design and Setting: This meta-epidemiological study used unrestricted electronic and
manual searching for meta-analyses in orthodontics. Differences in standardized mean
differences (ΔSMD) between interventions and their 95% confidence intervals (CIs) were
calculated according to study design through random-effects meta-regression. Effects were then
pooled with random-effects meta-analyses.
Results: No difference was found between randomized and non-randomized trials (25 metaanalyses; ΔSMD=0.07; 95% CI=-0.21, 0.34; P=0.630). However, retrospective non-randomized trials
reported inflated treatment effects compared to prospective (40 meta-analyses; ΔSMD=-0.30; 95%
CI=-0.53, 0.06; P=0.018). No difference was found between randomized trials with adequate and
those with unclear/inadequate generation (25 meta-analyses; ΔSMD=0.01; 95% CI=-0.25, 0.26;
P=0.957). Finally, subgroup analyses indicated that the results of randomized and nonrandomized trials differed significantly according to scope of the trial (effectiveness or adverse
effects; P=0.005).
Conclusion: Caution is warranted when interpreting systematic reviews investigating clinical
orthodontic interventions when non-randomized and especially retrospective non-randomized
studies are included in the meta-analysis.
Words in abstract: 200
2
Keywords: orthodontics; meta-analysis; systematic review; randomized controlled trial;
prospective clinical study; retrospective clinical study
Running title: Basic study design in orthodontics
3
What is new?
Key findings

Evidence of bias was found between randomized and non-randomized orthodontic studies,
although the direction of bias varied between trials investigating the effectiveness or adverse
effects of orthodontic interventions.

Evidence of bias was found between retrospective non-randomized and prospective nonrandomized studies of orthodontic interventions
What this study adds to what was known
 The influence of basic study design on the results of clinical research, which has already been
identified in various biomedical fields, also exists in orthodontics.
What is the implication and what should change now?

Reporting of the study design in orthodontics is suboptimal and should be improved, as it can
influence study results.

If the inclusion of studies with different designs is judged appropriate in a meta-analysis, a
sensitivity analysis based on study design is warranted.
4
Manuscript text
1. Introduction
1.1. Background
Meta-analysis of clinical trials provides the best evidence for evaluating orthodontic
interventions, due to the increased statistical power and precision [1]. However, if the
methodological quality of these studies is suboptimal, then the results will be biased, even if the
meta-analysis is conducted to the highest standards (the so-called GIGO ‘garbage in-garbage out’
concept) [2].
Ideally, meta-analyses assessing the efficacy of orthodontic interventions would include
only well-conducted and appropriately-reported randomized controlled trials (RCTs), which are
seen as the epitome of clinical research [2]. Their principal advantage lies in the random
allocation of patients to different interventions, which minimizes selection bias [3]. However,
high quality clinical trials do not always exist and non-randomized controlled trials of
interventions (non-RCTs) are often included in systematic reviews and meta-analyses in
orthodontics [4,5], which can potentially affect their conclusions. Indeed, many widely-used
interventions in orthodontics are not adequately supported by clinical evidence, possibly because
orthodontics is a field where patient lives are rarely put at risk [6].
Currently, clinical trials reported in the orthodontic literature consist of only a modest
proportion of RCTs, whilst the rest are either prospective or retrospective non-RCTs [7-9]. The
majority of these are retrospective clinical trials, where data concerning the performance of an
intervention are extracted from archived patient files. Traditionally, these have provided much of
the evidence for clinical decision-making in orthodontics [10]. This can be attributed to (i) the
5
relative time-lag between the introduction of orthodontics as a specialty and the establishment of
evidence-based research methodologies; (ii) the fact that for some orthodontic questions, RCTs
can be unethical and (iii) personal preferences of researchers or research groups. Indeed,
criticisms have been leveled at the use of RCTs in orthodontics, characterizing them as unethical
and inappropriate in some cases [11]. The relative merits of RCTs in orthodontics have been
discussed in several contexts [12,13]; however, if orthodontics is to be considered a clinical
discipline with a sound scientific basis, conclusions should be based upon appropriate scientific
methodology.
Empirical evidence relating to the effect of study design characteristics on treatment effects
can be derived from meta-epidemiologic studies that integrate data from a collection of metaanalyses [14]. In this collection of meta-analyses, all component trials are classified according to
a specific study-level characteristic and then synthesized. As an example, it has been shown that
inadequacies in the generation of a randomization sequence, allocation concealment or blinding
in RCTs can lead to biased estimates [15-17]. Concerning interventional research in
orthodontics, empirical evidence is needed because it is important to know for questions that can
be answered through RCTs and non-RCTs, the extent of bias that is associated with each study
design. Also, because many orthodontic interventions will only ever be assessed from non-RCTs
because of their nature, it is important to know how biased they might be. To date, there is no
empirical evidence in relation to the impact of study design on treatment effect in orthodontics.
6
1.2. Aim
The aim of this meta-epidemiological study was to identify the extent of bias that might be
introduced in the results of clinical trials in orthodontics based on their basic study design.
2. Methods
2.1. Terminology and protocol
In this report we define as ‘systematic review’ a structured review with comprehensively planned
procedures of study identification, study selection, data extraction and methodological
assessment of included studies. We define ‘meta-analysis’ as the procedure of statistical
synthesis of the results of two or more studies. As such, meta-analysis is ideally conducted
within the framework of a systematic review and multiple meta-analyses can be included in a
systematic review. Primary studies (here, clinical trials) included in a systematic review or a
meta-analysis are termed ‘component trials’. We define non-RCTs as ‘any quantitative trial
estimating the effects of an intervention that does not use randomization to allocate units to
comparison groups’ [18]. Finally, the pooling of multiple meta-analyses according to a specific
factor (for example, study design) is termed ‘meta-epidemiological synthesis’.
In an attempt to make the nature of the component trials as transparent as possible, their
designs were categorized as follows: (1) tRCT: ‘true’ randomized controlled trial (adequate
random sequence generation method clearly-described according to Cochrane Collaboration
criteria); (2) uRCT: ‘unclear’ randomized controlled trial (an unclear random sequence
generation method); (3) qRCT: ‘quasi’ randomized controlled trial (an inadequate random
sequence generation method); (4) pCCT: prospective non-randomized controlled clinical trial;
7
(5) rCCT: retrospective non-randomized controlled clinical trial; and (6) Unclear: nonrandomized trial with unclear design (probably retrospective).
The protocol for this study was registered prospectively in the PROSPERO International
prospective register of systematic reviews (CRD42014013767) before any data extraction or
analysis.
2.2. Search and selection procedures
Study selection was based on a previously published CADMOS meta-epidemiological database
[19]. To construct this database, seven general open-access, regional or grey literature databases
were searched from inception to September 2012 for systematic reviews in the field of oral
medicine, including orthodontics. There was no language, publication year or publication status
restriction.
The
existing
database
was
updated
in
MEDLINE
through
PubMed
(http://www.ncbi.nlm.nih.gov/pubmed) on July 25th, 2014 with the strategy: (orthodon* OR
malocclusion) AND ("systematic review" OR "meta-analysis" OR "random-effects" OR "fixedeffect" OR "meta-regression"), limited to reviews, systematic reviews and meta-analyses.
Manual updates were conducted up to September 2014, using the same search strategy
(Appendix A).
2.3. Inclusion criteria
Eligible for this study were systematic reviews in orthodontics with at least one meta-analysis of
interventional studies with different design. Systematic reviews that included studies of a single
design (for example, all tRCTs) were excluded, as they would not contribute in the comparative
analyses among study designs. As a requirement, either the raw data or the calculated
8
Standardized Mean Difference (SMD) should be reported in the published report. All obtained
reports were screened by one author (SNP). MEDLINE was searched through PubMed in order
to assign unique identifiers to each component trial and systematic review. References not
indexed were manually assigned a unique identifier. Using the identifier, overlaps were checked
and duplicates were removed until there was no overlap between component trials.
2.4. Data extraction
A pre-defined form was used to extract the characteristics of included systematic
reviews/component trials, including the design of each included component trial. Multiple metaanalyses were extracted from a systematic review only when the component trials or their
outcomes differed. The design of component trials was categorized as tRCT, uRCT, qRCT,
pCCT, rCCT or Unclear according to the full-text. Data extraction and characterization of study
design were was performed independently by two authors (SNP and GMX) based on the full-text
of each review/component trial, as often misclassification of study designs in the orthodontic
literature has been reported [20]. In a handful of instances, where the full-text of a single
component trial could not be obtained, even after communication with the authors, this
component trial was excluded. In one instance, where no judgment about trial design could be
made, the third author (MTC) was consulted. Subgroup analyses were ignored, if an overall
pooled estimate of the subgroups was given. When the subgroups were not pooled together, data
were extracted from the largest subgroup. A preliminary calibration between the two authors
responsible for extraction (SNP and GMX) was conducted prior to the actual extraction
procedures until consensus was reached.
9
2.4. Analysis
2.4.1. Calculating effects within each meta-analysis
For both continuous and binary outcomes, raw data or calculated effect sizes were extracted for
each component trial (i.e. trial included in the original meta-analysis) and converted into SMDs,
recoded so that a negative SMD was beneficial (Appendix B).
Random-effects meta-regression was performed, fully incorporating heterogeneity betweentrials, to derive a ‘difference in SMDs’ (ΔSMD) and the standard error for each meta-analysis,
according to the component trial design. An iterative residual maximum likelihood (REML)
algorithm was used for the estimation of between-study variance, due to its performance [21],
and the Knapp-Hartung modification [22] was used for the calculation of the ΔSMDs, which
accounts for the uncertainty in the heterogeneity estimate [23]. For the ΔSMD of characteristic
[A] versus characteristic [B] a ΔSMD < 0 indicated that studies with characteristic [A] show
larger treatment effects than those with characteristic [B]. The magnitude for SMDs and ΔSMD
was assessed with the following guidelines (SMD of 0.2 = small effect; SMD of 0.5 = medium
effect; SMD of 0.8 = large effect) [24].
Three statistical comparisons were conducted: (1) RCTs versus non-RCTs; (2)
prospective non-RCTs versus retrospective non-RCTs; and (3) RCTs with an adequate random
sequence generation method versus RCTs with inadequate or unclear random sequence
generation method.
2.4.2. Meta-epidemiological synthesis among meta-analyses
10
The ΔSMDs among meta-analyses were pooled with the metan macro (random-effects model
based on the DerSimonian and Laird method) as no guidelines for meta-epidemiological
synthesis exist. Between-meta-analysis heterogeneity was assessed with the heterogeneity
parameter τ2, whilst between-meta-analysis inconsistency was quantified with the I² statistic,
defined as the proportion of total variability in the results explained by heterogeneity [25,26].
The 95% uncertainty intervals (similar to CIs) around the I2 were calculated [27] using the noncentral χ2 approximation of Q [28]. 95% predictive intervals were calculated for the ΔSMD,
which incorporate existing heterogeneity and provide a range of possible effects for a future
meta-analysis [29]. All analyses were run in Stata SE 10.0 (StataCorp, College Station, TX)
(Appendix B). A two-tailed P-value of 0.05 was considered significant for hypothesis-testing,
except for a 0.10 used for the test of heterogeneity and reporting biases [30].
2.4.3. Additional analyses
Mixed-effect subgroup analyses were performed to identify possible differences of study design
role according to (1) various fields of orthodontics; (2) outcome type (binary or continuous); (3)
research scope (effectiveness or adverse effects); and (4) nature of the outcome (subjective or
objective). Subjective outcomes included self-reported pain intensity and eating or speaking
difficulty. Indications of reporting bias were assessed with Egger’s linear regression test [31] if
10 or more meta-analyses were included in a meta-epidemiological synthesis.
2.4.4. Sensitivity analyses
Sensitivity analyses were performed by (1) comparing the results of fixed-effect and randomeffects models; (2) excluding trials with unclear description of methodology; (3) including only
11
the largest meta-analysis from each systematic review; and (4) including only the most precise
50% from the number of eligible meta-analyses (i.e. having the lowest standard error) for each
comparison.
3. Results
3.1. Study characteristics
Study selection
Following an initial screening of the pre-existing database and manual literature update, a total of
171 relevant systematic reviews were identified (Fig. 1). A total of 148 systematic reviews were
excluded after consideration, leaving 23 relevant reviews with 147 meta-analyses for inclusion.
After the addition of manually-identified reviews, 77 meta-analyses with 333 component trials
were included (Table 1 and Appendix C-D).
3.2. Main analyses
3.2.1. RCTs versus non-RCTs
A total of 25 meta-analyses included both RCTs and non-RCTs and could be pooled. On
average, RCTs showed minimally smaller treatment effects compared with non-RCTs (ΔSMD =
0.07; 95% CI = -0.21, 0.34; P = 0.630) (Table 2 and Fig.2).
3.2.2. Prospective non-RCTs versus retrospective non-RCTs
A total of 40 meta-analyses included both prospective and retrospective non-RCTs and could be
pooled. On average, retrospective non-RCTs showed inflated treatment effects compared with
prospective non-RCTs (ΔSMD = -0.30; 95% CI = -0.53, -0.06; P = 0.018) (Table 2 and Fig. 3).
12
The magnitude of the effect was small to medium, while moderate heterogeneity was found
among meta-analyses.
3.2.3. Adequate versus inadequate/unclear random sequence generation
A total of 25 meta-analyses included both RCTs with adequate random sequence generation and
RCTs with inadequate/unclear random sequence generation and could be pooled. On average,
RCTs with adequate random sequence generation showed almost the same treatment effects
compared with RCTs with inadequate/unclear random sequence generation (ΔSMD = 0.01; 95%
CI = -0.25, 0.26; P = 0.957) (Table 2 and Appendix E).
3.3. Additional analyses
According to the subgroup analyses (Table 3), considerable variation of ΔSMDs was found
among the various orthodontic fields with statistically significant differences (P between
subgroups = 0.027). Additionally, the scope of the meta-analysis had a significant modifying
effect on the results (P between subgroups = 0.005). When the effectiveness of an intervention
was studied, RCTs tended to show smaller treatment effects than non-RCTs (ΔSMD = 0.32; 95%
CI = 0.06, 0.58; P = 0.015). However, when adverse effects of interventions were studied, RCTs
tended to show greater treatment effects than non-RCTs (ΔSMD = -0.57; 95% CI = -1.05, -0.10;
P = 0.018) (Appendix F). Finally, no significant indications of reporting bias among metaanalyses could be found with Egger’s test for any of the three comparisons (Appendix G).
3.4. Sensitivity analyses
13
For all three meta-epidemiological comparisons the results of the sensitivity analyses were
similar to the original analyses (Appendix H), apart from some minor points. The difference
between prospective and retrospective non-RCTs was no longer significant in any of the
sensitivity analyses, but the effect’s direction was consistent. Therefore, this was attributed to
hampered precision following reduction in the sample size. The comparison of adequate versus
inadequate/unclear random sequence generation seemed to be the less robust of the analyses with
great variation of effect sizes between the original analysis (ΔSMD = 0.01) and the sensitivity
analysis (ΔSMDs of –0.03 to 0.13).
4. Discussion
4.1. Evidence and comparison to literature
As far as we are aware, this is the first empirical assessment of study design influence on
treatment effects in orthodontic clinical interventions. Despite the relatively restricted sample of
included meta-analyses, study design significantly influenced observed effects.
Based on the empirical evidence, the results from RCTs did not seem to differ
significantly from the results of non-RCTs, when pooling all eligible meta-analyses together (P =
0.630). Previous empirical studies of binary outcomes either confirm [32] or reject [33-36] a
significant difference between RCTs and non-RCTs. This difference is usually interpreted as an
overestimation in non-RCTs and not an underestimation in RCTs [37]. Almost all empirical
studies have concluded that RCTs and non-RCTs can sometimes differ substantially and that
systematic review authors should try to include RCTs whenever possible.
14
Retrospective non-RCTs showed significantly inflated treatment effects compared to
prospective non-RCTs. Similarly, a tendency for RCTs to agree more with prospective compared
to retrospective non-RCTs has been described [34]. This can be explained by the fact that
retrospective non-RCTs might be more prone to selection and observation bias than prospective
non-RCTs. Also, in retrospective non-RCTs there is a higher risk for confounding by indication,
meaning that the choice of intervention is determined by the severity of the disease [38]. In
orthodontics, this would translate to one treatment being allocated to a more compliant patient or
to a patient with a less severe malocclusion, where the prognosis is more favorable. In contrast, it
is possible that for a more difficult case, early cessation of treatment or a change of treatment
plan might also play a role. Furthermore, it is important to discriminate between ‘consecutivelyenlisted’ and ‘consecutively-finished’ patients, as this also entails the risk of overestimating the
effect of an intervention [39,40]. It is also possible that many retrospective trials are conducted
using data that have been collected for other purposes and therefore may not be as complete or
unbiased as one would wish [39].
In the present study, the difference between RCTs and non-RCTs varied between
effectiveness and adverse effects of interventions. It is accepted that RCTs cannot answer all
questions, as they are not always feasible, while they are mainly designed and powered to assess
the efficacy of interventions [18,40]. Estimations of a treatment’s adverse effects may be prone
to different biases than its efficacy [32]. RCTs may not be large enough or may not have
sufficiently large follow-up to identify some long-term harms [33,41-44]. Moreover,
generalizability of the RCTs’ results may be limited for various reasons [45]; for example, as
high-risk patients are often excluded from trials [32,33,46]. Our results might be considered as
supportive of this notion, as RCTs were more conservative than non-RCTs only regarding the
15
effectiveness of interventions. When adverse effects of interventions were considered, the results
of RCTs seemed to be inflated compared to the results of non-RCTs.
Conversely, it has been acknowledged for some time that non-RCTs are more prone to
bias than RCTs and therefore provide weaker evidence [47,48]. Risk of selection bias (baseline
differences between the patients in the two groups) is widely regarded as the principal difference
between RCTs and non-RCTs. Incorporating results of non-RCTs at high risk of bias in a metaanalysis runs the danger of producing a biased estimate with unwarranted precision [18,49].
Additionally, there are indications that publication bias might be more severe for non-RCTs than
for RCTs [50].
However, recent initiatives from the field of comparative research have highlighted the
need for trials that are more easily applicable to real-world settings [51] and for the incorporation
of non-RCTs in reviews assessing harmful effects [18]. Various statistical techniques have been
introduced to deal with selection bias in non-RCTs, although not always successfully [52,53].
Indeed, organizations such as the Cochrane Collaboration and the Campbell Collaboration have
decided to incorporate non-RCTs for assessing the harmful effects of interventions (Cochrane
since 2012 and Campbell since 2000).
Non-RCTs are invaluable for investigating interventions that cannot ethically be
randomized or for the assessment of long-term outcomes, rare events or adverse effects of
interventions. However, in cases where RCTs are not feasible or inappropriate, this does not
mean that retrospective non-RCTs are the only alternative. Well-conducted prospective nonRCTs, which guard against sources of bias, can provide complementary evidence and are not as
complex or expensive as RCTs. When including non-RCTs in systematic reviews, it should be
remembered that assessment of the risk of bias is more complicated for these types of trial than
16
for RCTs. It is advisable that systematic review teams include a person with methodological
expertise in assessing risk of bias and correctness of the analyses used, as well as awareness of
the topic area [54,55]. Finally, when incorporating non-RCTs in systematic reviews, it will
always make sense to explore potential sources of heterogeneity as well as adopt a randomeffects approach to acknowledge the unexplained heterogeneity [56,57]. Various methods have
also been suggested to inform the meta-analysis results about the extent of bias by empirically
based priors [58] or directly by mixed treatment comparison meta-analysis [59], but are not
widely used and may require specialized statistical expertise and software [60].
4.2. Strengths and limitations
The strengths of this study include the extensive literature search, which was not restricted to
orthodontic journals [61].
Also, misclassification of component trials was minimized, as the full texts of every component
trial in the meta-analyses were acquired and assessed first-hand. Furthermore random sequence
generation, was assessed based on the Cochrane risk-of-bias tool [62]. No overlap existed
between the included studies, whilst the ΔSMDs calculations took into account the betweenstudy heterogeneity [63]. Almost all meta-analyses agreed on the direction of the effect, the
magnitude of which was medium to large, meaning that the observed differences could have a
bearing on the clinical decision regarding this treatment. Finally, the results of the metaepidemiological analyses were relatively robust.
There are also some limitations to this study. Due to the inclusion criteria of this
empirical assessment, only a subsample of existing meta-analyses could be included. For
17
example, meta-analyses including studies of only one design were excluded, limiting the final
sample of eligible meta-analyses. Additionally, the use of design labels for trials as ‘prospective’
and ’retrospective‘ is ambiguous and both the degree of prospectiveness of a study and its risk of
bias can be variable [64]. Furthermore, the significance level of most results was marginally
significant (significant at the 5% level) and the 95% PIs, were not consistent, possibly
endangering our confidence in these estimates.
4.4. Conclusions
Existing evidence from orthodontic meta-analyses indicates that study design of interventional
clinical trials might have a bearing on estimated treatment effects. Based on existing empirical
evidence, intervention effects in orthodontic research seem to be inflated in non-RCTs compared
to RCTs, as well as in retrospective non-RCTs compared to prospective non-RCTs.
4.4. Recommendations for future research
Clear reporting of study design, preferably in the title and/or abstract, stating if randomization
took place and the prospective or retrospective nature of the study is essential. For clinical
questions where randomization is feasible, systematic reviews should preferably include RCTs.
If no RCTs are available or if authors decide to also include non-RCTs, a sensitivity analysis
based on study design is warranted. For clinical questions where randomization is unethical or
not feasible, systematic reviews should preferably include prospective non-RCTs over
retrospective non-RCTs. If both study designs are included, a sensitivity analysis is warranted.
18
Conclusions from systematic reviews based solely on non-RCTs, and especially retrospective
non-RCTs, should be viewed with caution.
19
References
1.
Sackett DL, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine–how to
practice and teach EBM. New York, NY: Churchill Livingstone; 1997.
2.
Egger M, Smith GD, Altman DG, editors. Systematic reviews in health care: meta-analysis
in context. 2nd edition. London, UK: BMJ Books; 2001.
3.
Kunz R, Vist G, Oxman AD. Randomisation to protect against selection bias in healthcare
trials. Cochrane Database Syst Rev 2007;MR000012.
4.
Papageorgiou SN, Papadopoulos MA, Athanasiou AE. Evaluation of methodology and
quality characteristics of systematic reviews in orthodontics. Orthod Craniofac Res
2011;14:116–37.
5.
Papageorgiou SN, Papadopoulos MA, Athanasiou AE. Reporting characteristics of metaanalyses in orthodontics: methodological assessment and statistical recommendations. Eur J
Orthod 2014;36:74–85.
6.
Amat P. What would you choose: evidence-based treatment or an exciting, risky alternative?
Am J Orthod Dentofacial Orthop 2007;132:724–5.
7.
Harrison JE. Clinical trials in orthodontics I: demographic details of clinical trials published
in three orthodontic journals between 1989 and 1998. J Orthod 2003;30:25–30.
8.
Gibson R, Harrison J. What are we reading? An analysis of the orthodontic literature 1999 to
2008. Am J Orthod Dentofacial Orthop 2011;139:e471–84.
9.
Pandis N, Polychronopoulou A, Madianos P, Makou M, Eliades T. Reporting of research
quality characteristics of studies published in 6 major clinical dental specialty journals. J
Evid Based Dent Pract 2011;11:75–83.
20
10. Proffit WR. Evidence and clinical decisions: Asking the right questions to obtain clinically
useful answers. Semin Orthod 2013;19:130–136.
11. Zuccati G, Clauser C, Giorgetti R. Randomized clinical trials in orthodontics: reality, dream
or nightmare? Am J Orthod Dentofacial Orthop 2009;136:634–7.
12. Meikle MC. Guest editorial: what do prospective randomized clinical trials tell us about the
treatment of class II malocclusions? A personal viewpoint. Eur J Orthod 2005;27:105–114.
13. Bondemark L, Ruff S. EJO Open Session 2013 A Debate Randomized controlled trial
(RCT):
The
gold
standard
or
unobtainable
fallacy.
Available:
http://www.oxfordjournals.org/our_journals/eortho/ejovideo.html. Accessed April 14, 2014.
14. Naylor CD. Meta-analysis and the meta-epidemiology of clinical research. BMJ
1997;315:617-9.
15. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of
methodological quality associated with estimates of treatment effects in controlled trials.
JAMA 1995;273:408–12.
16. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, et al. Empirical evidence of bias in
treatment effect estimates in controlled trials with different interventions and outcomes:
meta-epidemiological study. BMJ 2008;336:601–5.
17. Savović J, Jones H, Altman D, Harris R, Jűni P, et al. Influence of reported study design
characteristics on intervention effect estimates from randomised controlled trials: combined
analysis of meta-epidemiological studies. Health Technol Assess 2012;16:1–82.
18. Reeves BC, Higgins JPT, Ramsay C, Shea B, Tugwell P, et al. An introduction to
methodological issues when including non-randomised studies in systematic reviews on the
effects of interventions. Res Syn Meth 2013;4:1–11.
21
19. Papageorgiou SN, Antonoglou G, Tsiranidou E, Jepsen S, Jäger A. Bias and small-study
effects influence treatment effect estimates: a meta-epidemiological study in oral medicine. J
Clin Epidemiol 2014;67;984–92.
20. Koletsi D, Pandis N, Polychronopoulou A, Eliades T. What's in a title? An assessment of
whether randomized controlled trial in a title means that it is one. Am J Orthod Dentofacial
Orthop 2012;141:679–85.
21. Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of
methods. Stat Med 1999;18:2693-708
22. Knapp G, Hartung J. Improved tests for a random-effects meta-regression with a single
covariate. Stat Med 2003;22:2693–2710.
23. Higgins JP, Thompson SG. Controlling the risk of spurious findings from meta-regression.
Stat Med 2004;23:1663-82.
24 . Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. New York: Academic
Press; 1988.
25. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med
2002;21:1539–58.
26. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in metaanalyses. BMJ 2003;327:557–60.
27. Ioannidis JP, Patsopoulos NA, Evangelou E. Uncertainty in heterogeneity estimates in metaanalyses. BMJ 2007;335:914–6.
28. Orsini N, Bottai M, Higgins J, Buchan I. Heterogi: Stata module to quantify heterogeneity in
a meta-analysis. Available at http://econpapers.repec.org/RePEc:boc:bocode:s449201.
Accessed April 14, 2014.
22
29. Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects metaanalysis. J R Stat Soc Ser A Stat Soc 2009;172:137–159.
30. Ioannidis JP. Interpretation of tests of heterogeneity and bias in meta-analysis. J Eval Clin
Pract 2008;14:951–7.
31. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a
simple, graphical test. BMJ 1997;315:629–34.
32. Golder S, Loke YK, Bland M. Meta-analyses of adverse effects data derived from
randomised controlled trials as compared to observational studies: methodological overview.
PLoS Med 2011;8:e1001026.
33. Chou R, Helfand M. Challenges in systematic reviews that assess treatment harms. Ann
Intern Med 2005;142:1090–9.
34. Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI, et al. Comparison of evidence of
treatment effects in randomized and nonrandomized studies. JAMA 2001;286:821–30.
35. Tzoulaki I, Siontis KC, Ioannidis JP. Prognostic effect size of cardiovascular biomarkers in
datasets from observational studies versus randomised trials: meta-epidemiology study. BMJ
2011;343:d6829.
36. Jacobs WC, Kruyt MC, Moojen WA, Verbout AJ, Oner FC. No evidence for interventiondependent influence of methodological features on treatment effect. J Clin Epidemiol
2013;66:1347–55.e3.
37. Deeks JJ, Dinnes J, D'Amico R, Sowden AJ, Sakarovitch C, et al. Evaluating nonrandomised intervention studies. Health Technol Assess 2003;7:iii–x, 1–173.
38. Psaty BM, Koepsell TD, Lin D, Weiss NS, Siscovick DS, et al. Assessment and control for
confounding by indication in observational studies. J Am Geriatr Soc 1999;47:749–754.
23
39. Johnston LE Jr. Moving forward by looking back: 'retrospective' clinical studies. J Orthod
2002;29:221–6.
40. Flores-Mir C. Can we extract useful and scientifically sound information from retrospective
nonrandomized trials to be applied in orthodontic evidence-based practice treatments? Am J
Orthod Dentofacial Orthop 2007;131:707–8.
41. Papanikolaou PN, Ioannidis JP. Availability of large-scale evidence on specific harms from
systematic reviews of randomized trials. Am J Med 2004;117:582–589.
42. Vandenbroucke JP. When are observational studies as credible as randomised trials? Lancet
2004;363:1728–1731.
43. Papanikolaou PN, Christidi GD, Ioannidis JPA. Comparison of evidence on harms of
medical interventions in randomized and nonrandomized studies. CMAJ 2006;174:635–641.
44. Vandenbroucke JP. What is the best evidence for determining harms of medical treatment?
CMAJ 2006;174:645–646.
45. Zwarenstein M, Oxman A. Why are so few randomized trials useful, and what can we do
about it? J Clin Epidemiol 2006;59:1125–1126.
46. Hordijk-Trion M, Lenzen M, Wijns W, de Jaegere P, Simoons ML, et al. Patients enrolled in
coronary intervention trials are not representative of patients in clinical practice: results from
the Euro Heart Survey on Coronary Revascularization. Eur Heart J 2006;27:671–678.
47. O’Connor D, Green S, Higgins JPT. Chapter 5: Defining the review question and developing
criteria for including studies. In: Higgins JPT, Green S, editors. Cochrane Handbook for
Systematic Reviews of Intervention. Version 5.1.0 (updated March 2011). The Cochrane
Collaboration; 2011. Available at www.cochranehandbook.org. Accessed April 15, 2014.
24
48. Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, et al. GRADE guidelines: 4. Rating the
quality of evidence--study limitations (risk of bias). J Clin Epidemiol 2011;64:407–15.
49. Egger M, Schneider M, Davey Smith G. Spurious precision? Meta-analysis of observational
studies. BMJ 1998;316:140–4.
50. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research.
Lancet 1991;337:867–872.
51. Patsopoulos NA. A pragmatic view on pragmatic trials. Dialogues Clin Neurosci
2011;13:217–24.
52. Cepeda MS, Boston R, Farrar JT, Strom BL. Comparison of logistic regression versus
propensity score when the number of events is low and there are multiple confounders. Am J
Epidemiol 2003;158:280–287.
53. Lonjon G, Boutron I, Trinquart L, Ahmad N, Aim F, et al. Comparison of treatment effect
estimates from prospective nonrandomized studies with propensity score analysis and
randomized controlled trials of surgical procedures. Ann Surg 2014;259:18–25.
54. Higgins JP, Ramsay C, Reeves B, Deeks JJ, Shea B, et al. Issues relating to study design and
risk of bias when including non-randomized studies in systematic reviews on the effects of
interventions. Res Syn Methods 2013;4:12–25.
55. Papageorgiou SN. Meta-analysis for orthodontists: Part II - Is all that glitters gold? J Orthod
2014;41:327–36.
56. Valentine JC, Thompson SG. Issues relating to confounding and meta-analysis when
including non-randomized studies in systematic reviews on the effects of interventions. Res
Syn Meth 2013;4:26–35.
25
57. Papageorgiou SN. Meta-analysis for orthodontists: Part I - How to choose effect measure
and statistical model. J Orthod 2014;41:317–26.
58. Welton NJ, Ades AE, Carlin JB, Altman DG, Sterne JAC. Models for potentially biased
evidence in meta-analysis using empirically based priors. J R Stat Soc Ser A Stat Soc
2009;172:119-136.
59. Dias S, Welton NJ, Marinho VCC, Salanti G, Higgins JPT, Ades AE. Estimation and
adjustment of bias in randomized evidence by using mixed treatment comparison metaanalysis. J R Stat Soc Ser A Stat Soc 2010;173:613-629.
60. Hopewell S, Boutron I, Altman DG, Ravaud P. Incorporation of assessments of risk of bias
of primary studies in systematic reviews of randomised trials: a cross-sectional study. BMJ
Open 2013;3:e003342.
61. Mavropoulos A, Kiliaridis S. Orthodontic literature: an overview of the last 2 decades. Am J
Orthod Dentofacial Orthop 2003;124:30–40.
62. Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, et al. The Cochrane Collaboration's
tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928.
63. Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, et al. Statistical methods for assessing
the influence of study characteristics on treatment effects in ‘meta-epidemiological’
research. Stat Med 2002;21:1513–24.
64. Feinstein AR. Clinical epidemiology. The architecture of clinical research. Philadelphia, PA:
W. B. Saunders Company; 1985.
26
Acknowledgements
We thank Greg J. Huang and Anne-Marie Bollen (both from University of Washington, Seattle,
WA) for providing component studies included in their meta-analyses and Carlos Flores-Mir
(University of Alberta, Edmonton, Canada) for providing raw study data from their included
meta-analysis.
Changes from the PROSPERO protocol (CRD42014013767)
Minor changes from protocol:
(i) Types of study to be included: meta-analyses of only randomized trials no longer excluded, as
they were included in the comparison of “Adequate vs. inadequate/unclear random sequence
generation”.
(ii) Comparator(s)/controls / context: adopted a more detailed characterization of study designs
than the one that was initially included in the protocol.
(iii) Analyses of subgroups or subsets: comparisons originally planned for the sensitivity
analyses were moved to the main analyses of the paper (adequate vs. inadequate/unclear random
sequence generation).
27
Figures
Fig. 1. Flow diagram for the identification and selection of eligible systematic reviews.
28
Fig. 2. Forest plot for the comparison of RCTs versus non-RCTs. Due to data recoding, estimates
on the right side of the forest plot indicate that RCTs show smaller treatment effects than nonRCTs. ΔSMD = difference in standardized mean differences; RCT = randomized controlled
trial; non-RCT = non-randomized controlled trial.
29
Fig. 3. Forest plot for the comparison of retrospective versus prospective non-RCTs. Due to data
recoding, estimates on the left side of the forest plot indicate that retrospective non-RCTs show
larger treatment effects than prospective non-RCTs. ΔSMD = difference in standardized mean
differences; non-RCT = non-randomized controlled trial.
30
Tables
Table 1. Summary of the included meta-analyses’ characteristics
Systematic reviews
Total - n
Publication years - range
Meta-analyses
Total - n (%)
Field - n (%)
17
2009-2014
77 (100%)
Class II treatment (functional appliances)
Class III treatment (skeletal anchorage)
Cleft lip palate
Lingual appliances
Maxillary expansion
Orthodontic education
Self-ligating appliances
Skeletal anchorage (effects)
Skeletal anchorage (failure)
Tooth extraction
16 (21%)
3 (4%)
10 (13%)
3 (4%)
10 (13%)
2 (3%)
13 (17%)
5 (6%)
11 (14%)
4 (5%)
Binary
Continuous
17 (22%)
60 (78%)
Effectiveness
Adverse effects
50 (65%)
27 (35%)
Objective
Subjective
73 (95%)
4 (5%)
Outcome type - n (%)
Scope - n (%)
Outcome nature - n (%)
Component trials
Total - n (%)
Trials per meta-analysis - range
Trials per meta-analysis - median
Trial design - n (%)
333 (100%)
2-16
4
tRCT
49 (15%)
uRCT
38 (11%)
qRCT
18 (5%)
pCCT
89 (27%)
rCCT
109 (33%)
Unclear
30 (9%)
tRCT, randomized controlled trial (random sequence generation clearly-described and
adequate); uRCT, randomized controlled trial (unclear random sequence generation); qRCT,
quasi-randomized controlled trial (random sequence generation inadequate); pCCT,
prospective non-randomized controlled clinical trial; rCCT, retrospective non-randomized
controlled clinical trial; Unclear, non-randomized controlled clinical trial with unclear design.
32
Table 2. Results of the meta-epidemiological analyses
Comparison
1. Randomized vs.
non-randomized
Ref
pCCT
rCCT
Unclear
Exp
uRCT
qRCT
tRCT
rCCT
Unclear
MAs
25
Trials
(Ref/Exp)
116
(44/72)
Effect size (random-effects model )
ΔSMD (95% CI)
P-value
95% PI
Heterogeneity
I (95% CI)
τ2
0.07 (-0.21,0.34)
26% (0%,54%)
0.630
-0.70,0.83
2
0.116
2. Prospective vs.
190
pCCT
40
-0.30 (-0.53,-0.06)
-1.38,0.79
59% (35%,69%)
0.274
0.018
retrospective
(70/119)
3. Adequate vs.
inadequate/unclear
qRCT
tRCT
25
75 (37/38) 0.01 (-0.25,0.26)
0.957
-0.98,0.99
60% (32%,73%)
0.211
random sequence
uRCT
generation
Bold values indicate statistical significance at the 5% level.
Ref, reference; Exp, experimental; MA, meta-analysis; ΔSMD, difference in standardized mean differences; CI, confidence interval; PI,
predictive interval; pCCT, prospective non-randomized controlled clinical trial; rCCT, retrospective non-randomized controlled clinical
trial; Unclear, non-randomized controlled clinical trial with unclear design; uRCT, randomized controlled trial (unclear random
sequence generation); qRCT, quasi-randomized controlled trial (inadequate random sequence generation); tRCT, randomized controlled
trial (random sequence generation clearly-described and adequate).
33
Table 3. Results of the conducted subgroup analyses
1. Randomized vs. non-randomized
2. Prospective vs. retrospective
3. Adequate vs. inadequate/unclear random
sequence generation
MAs
ΔSMD (95% CI)
P-value
PSG
Subgroup
MAs
ΔSMD (95% CI)
P-value PSG
MAs
ΔSMD (95% CI)
P-value
PSG
Class II treatment (functional
10
0.29 (-0.05,0.64)
0.093
-0.36 (-0.94,0.22) 0.228
0.304
0.027 14
appliances)
Class III treatment (skeletal
2
0.40 (-0.53,1.33)
0.394
anchorage)
Cleft lip palate
10
0.13 (-0.42,0.68)
Lingual appliances
3
-0.07 (-1.30,1.16) 0.912
Maxillary expansion
3
-0.11 (-1.09,0.86) 0.822
7
0.17 (-0.19,0.54)
0.348
Orthodontic education
1
1.09 (0.41,1.77)
0.002
Orthodontic-periodontic
interactions
Self-ligating appliances
4
-0.18 (-0.75,0.39) 0.544
1
-0.36 (-0.79,0.08) 0.110
11
-0.04 (-0.31,0.24)
Skeletal anchorage-effects
3
-0.88 (-1.54,-0.22) 0.009
1
1.66 (0.13,3.20)
4
-0.18 (-0.99,0.63)
0.034
Skeletal anchorage-failure
1
-0.06 (-1.64,1.52) 0.941
11
-0.49 (-0.75,-0.24) <0.001
Teeth extraction
4
-0.20 (-0.79,0.38) 0.494
Binary
4
-0.05 (-0.94, 0.85) 0.914
0.807 11
-0.49 (-0.75,-0.24) <0.001
0.163
Continuous
21
0.07 (-0.23,0.37)
0.639
29
-0.18 (-0.50,0.14) 0.264
Effectiveness
17
0.32 (0.06,0.58)
-0.22 (-0.57,0.14) 0.231
0.408
19
0.06 (-0.26,0.37)
0.015
0.005 24
Adverse effects
8
-0.57 (-1.05,-0.10) 0.018
16
-0.40 (-0.63,-0.17) 0.001
6
-0.11 (-0.43,0.22)
Subjective
3
-0.07 (-1.30, 1.16) 0.912
0.841 1
-0.02 (-0.52,0.48)
Objective
22
0.07 (-0.22,0.36)
0.623
24
0.01 (-0.26,0.28)
Bold values indicate statistical significance at the 5% level.
MA, meta-analysis; ΔSMD, difference in standardized mean differences; CI, confidence interval; P SG, P-value for difference between subgroups.
34
-
0.862
0.640
0.794
0.663
0.731
0.524
0.939
0.944
0.591
0.963
Supplemenatry Information
Appendix A. Electronic databases and search strategy used in the original meta-epidemiological study (up to September 23rd, 2012) and in its update for this study (up to July
25th, 2014).
Electronic databases
Search strategy
Hits
#1 - ("systematic review" OR "meta-analysis")
80798
MEDLINE searched via PubMed (1950 – 23.09.2012)
www.ncbi.nlm.nih.gov/sites/entrez/
Scopus (1966 - 23.09.2012)
www.scopus.com
Cochrane Database of Systematic Reviews searched via The
Cochrane Library on 23.09.2012
www.thecochranelibrary.com
Thomson Reuters Web of Knowledge (1945 - 23.09.2012)
http://apps.webofknowledge.com
Bibliografia Brasileira de Odontologia (including LILACS)
searched on 23.09.2012
http://regional.bvsalud.org/php/index.php?lang=en
ADA Center for Evidence-Based Dentistry
http://ebd.ada.org/
PROSPERO
http://www.crd.york.ac.uk/prospero/index.asp
#2 - tooth OR teeth OR dentist* OR dental OR endodont* OR orthodont* pedodont* OR paedodont* OR
periodont* OR prosthodont*
#3 - (oral OR maxillofacial) AND (implant OR surg*)
#4 - (tooth OR teeth OR implant*) AND (prosth* OR restor* OR bridge* OR crown* OR denture*)
#5 - #2 OR #3 OR #4
#6 - #1 AND #5
Article Type: Systematic Reviews, Meta-Analysis
"systematic review" OR "meta-analysis"
Limit to Dentistry
tooth OR teeth OR dentist* OR dental OR endodont* OR orthodont* pedodont* OR paedodont* OR periodont*
OR prosthodont*
Limit to Cochrane Reviews & Other Reviews, Exclude Protocols
"Oral Health Group"
Limit to Cochrane Reviews & Other Reviews, Exclude Protocols
"systematic review" OR "meta-analysis"
Limit to Dentistry Oral Surgery Medicine
Limit to Review
("systematic review" OR "meta-analysis") AND (tooth OR teeth OR dentist* OR dental OR endodont* OR
orthodont* pedodont* OR paedodont* OR periodont* OR prosthodont* OR ((oral OR maxillofacial) AND
(implant OR surg*)) OR ((tooth OR teeth OR implant*) AND prosth*))
Type of Study: Systematic Reviews
92435
125845
118504
300055
1918
1702
1741
476
160
566
In Topic
584
Manually
-
Manually
-
("systematic review" OR "meta-analysis") AND (tooth OR teeth OR dentist* OR dental OR endodont* OR
Digital Dissertations searched via UMI/ProQuest on 23.09.2012 orthodont* pedodont* OR paedodont* OR periodont* OR prosthodont* OR ((oral OR maxillofacial) AND
(implant OR surg*)) OR ((tooth OR teeth OR implant*) AND prosth*))
http://proquest.umi.com/login
Source: Dissertations & Theses
Subject: Dentistry
439
Sum with overlap from original search
5668
MEDLINE searched via PubMed (2012 – 25.07.2014)
www.ncbi.nlm.nih.gov/sites/entrez/
(orthodon* OR malocclusion) AND ("systematic review" OR "meta-analysis" OR "random-effects" OR "fixedeffect" OR "meta-regression")
Limit to reviews, systematic reviews and meta-analyses
150
150
Sum with overlap from update only
35
Appendix B. Details and Statistical code used for the analysis in Stata SE 10.0 (StataCorp, College Station, TX)
A. Calculating effects within each meta-analysis
Firstly, the influence of each chosen basic study design was assessed within each eligible meta-analysis. In order for
a meta-analysis to be eligible one of the following data formats should be provided: (A1) raw continuous data
(sample, mean and standard deviation), (A2) already-calculated Standardized Mean Differences (SMDs) for
continuous data (SMD and the corresponding standard error), (A3) raw binary data (contingency 2 x 2 table) or (A4)
already-calculated odds ratios for binary data (together with their 95% confidence intervals).
SMD was chosen as the effect measure because it standardizes estimates by their variability and enables
overall synthesis [1]. For meta-analyses of continuous outcomes, raw data or SMDs were extracted for each
component trial (i.e. trial included in the original meta-analysis) and were recoded, so that a negative SMD was
beneficial. For binary outcomes, raw data or odds ratios were likewise extracted, recoded, and afterwards
transformed to SMDs [2] in order to enable synthesis of both continuous and binary outcomes together.
The SMD chosen for the analysis was Cohen’s d, defined as the difference between two means divided by
their pooled standard deviation:
d
x1  x2
s
, where s is the pooled standard deviation, defined as:
s
(n1  1) S12  (n2  1) S22
n1  n2  2
For the calculation of difference in standardized mean differences (ΔSMDs) from each included meta-analysis the
following formulas were used, according to the data provided:
A1. Raw continuous data:
metan exn exm exsd ctrn ctrm ctrsd, random rfdist by(design) nograph
metareg _ES design,wsse(_seES)
36
A2. Provided SMDs:
metan smd sesmd,random by(design) nograph
metareg smd design,wsse(sesmd)
A3. Raw binary data:
metan EVex NEVex EVctr NEVctr, random or log
gen smd=0.5513*_ES
gen sesmd=0.5513*_selogES
metan smd sesmd,random rfdist nograph by(design)
metareg smd design,wsse(sesmd)
A4. Provided ORs:
gen logOR = ln(OR)
gen logORlci = ln(ORlci)
gen logORuci = ln(ORuci)
gen selogOR = (logORuci-logOR)/1.96
gen smd=0.5513* logORlci
gen sesmd = 0.5513* selogOR
metan smd sesmd,random rfdist nograph by(design)
metareg smd design,wsse(sesmd)
B. Meta-epidemiological synthesis among meta-analyses
For the pooling of ΔSMDs across meta-analyses the following formulas were used
B1. Fixed-effect meta-analysis:
metan dsmd sedsmd, nograph
B2. Random-effects meta-analysis:
metan dsmd sedsmd, random rfdist nograph
heterogi [Q] [df],nc
B3. Reporting bias assessment:
metabias dsmd sedsmd,egger
B4. Subgroup analyses:
metan dsmd sedsmd, random rfdist by(factor) nograph
metareg dsmd factor,wsse(sedsmd)
Abbreviations
exn=sample (experimental group; continuous outcome)
exm=mean (experimental group; continuous outcome)
exsd=standard deviation (experimental group; continuous outcome)
ctrn=sample (control group; continuous outcome)
ctrm=mean (control group; continuous outcome)
ctrsd=standard deviation (control group; continuous outcome)
design=variable including the basic design of the component trials included in the meta-analyses
smd=standardized mean difference
sesmd=standard error of the smd
EVex=events in the experimental group (binary outcome)
NEVex=non-events in the experimental group (binary outcome)
EVctr=events in the control group (binary outcome)
NEVctr=non-events in the control group (binary outcome)
selogOR=standard error of the log odds ratio
37
logORuci=upper limit of the 95% CI for the log odds ratio
logOR=log odds ratio
dsmd=difference in the standardized mean differences
sedsmd=standard error for the dsmd
factor=variable with the subgrouping variable (field, adverse effects, binary, subjective,etc)
References to Appendix B
1. Cohen J. Statistical power analysis for the behavioral sciences. 2 nd ed. New York: Academic Press; 1988.
2. Chinn S. A simple method for converting an odds ratio to effect size for use in meta-analysis. Stat Med
2000;19:3127–31.
38
Appendix 3 C. Characteristics of the included meta-analyses (regarding all three meta-epidemiological comparisons)
Experimental
A/A Meta-analysis Field
Control group
Outcome
Binary
group
Computer-aided
Conventional
Al-Jewair
Orthodontic
Knowledge gain
1
orthodontic
orthodontic
No
2009 1rst
education
(pre/post tests)
learning
learning
Computer-aided
Conventional
Al-Jewair
Orthodontic
Knowledge gain
2
orthodontic
orthodontic
No
2009 2nd
education
(post tests)
learning
learning
Class II
Total
Al-Jewair
malocclusionControl
3
MARA appliance
mandibular unit
No
2014 1st
functional
(untreated)
length
appliances
Mandibular
Self-ligating
Self-ligating
Conventional
4
Chen 2010 2nd
incisor
No
brackets
brackets
brackets
alignment
Self-ligating
Self-ligating
Conventional
Time to remove
5
Chen 2010 5th
No
brackets
brackets
brackets
ligation module
Self-ligating
Self-ligating
Conventional
Time to engage
6
Chen 2010 6th
No
brackets
brackets
brackets
ligation module
Self-ligating
Self-ligating
Conventional
Intercanine
7
Chen 2010 9th
No
brackets
brackets
brackets
width
Chen 2010
Self-ligating
Self-ligating
Conventional
8
Intermolar width No
10th
brackets
brackets
brackets
Chen 2010
Self-ligating
Self-ligating
Conventional
Incisor
9
No
11th
brackets
brackets
brackets
inclination
Skeletal
Dalessandri
10
anchorage-failure Male patients
Female patients
Implant failure
Yes
2014 1rst
rates
Skeletal
Dalessandri
No site
11
anchorage-failure Site inflammation
Implant failure
Yes
2014 3rd
inflammation
rates
Skeletal
Dalessandri
12
anchorage-failure Maxilla
Mandible
Implant failure
Yes
2014 4th
rates
Skeletal
Dalessandri
Keratinized
Non-keratinized
13
anchorage-failure
Implant failure
Yes
2014 6th
gingiva
gingiva
rates
Dalessandri
Skeletal
14
Thin implants
Thick implants
Implant failure
Yes
2014 9th
anchorage-failure
39
Adverse
effects
Subjective
Studies*
Patients$
No
No
3
90
No
No
6
296
No
No
6
266
No
No
2
114
No
No
2
462
No
No
2
462
No
No
3
140
No
No
3
140
Yes
No
3
140
Yes
No
14
[4361]
Yes
No
5
[992]
Yes
No
14
[4136]
Yes
No
5
[1110]
Yes
No
8
[1969]
rates
Class II
malocclusionfunctional
appliances
Class II
malocclusionfunctional
appliances
Class II
malocclusionfunctional
appliances
Class II
malocclusionfunctional
appliances
Class II
malocclusionfunctional
appliances
Class II
malocclusionfunctional
appliances
15
Ehsani 2014
1st
16
Ehsani 2014
2nd
17
Ehsani 2014
3rd
18
Ehsani 2014
4th
19
Ehsani 2014
5th
20
Ehsani 2014
6th
21
Feng 2012 2nd
Class III-skeletal
anchorage
22
Feng 2012 3rd
Class III-skeletal
anchorage
23
Feng 2012 5th
Class III-skeletal
anchorage
24
Jambi 2013 1st
25
Jambi 2013
2nd
Distalization of
first molars
Distalization of
first molars
Twin Block
Control
(untreated)
SNA angle
No
No
No
5
303
Twin Block
Control
(untreated)
SNB angle
No
No
No
5
303
Twin Block
Control
(untreated)
Co-Gn distance
No
No
No
3
148
Twin Block
Control
(untreated)
Lower anterior
facial height
No
No
No
3
184
Twin Block
Control
(untreated)
U1-NL
No
No
No
3
184
Twin Block
Control
(untreated)
LI-ML
No
No
No
3
211
Skeletal-anchored
maxillary
protraction
Skeletal-anchored
maxillary
protraction
Skeletal-anchored
maxillary
protraction
Tooth-anchored
maxillary
protraction
Tooth-anchored
maxillary
protraction
Tooth-anchored
maxillary
protraction
Maxillary
advancement
No
No
No
3
135
Mandibular
plane angle
No
No
No
3
135
Maxillary molar
extrusion
No
No
No
2
66
Intraoral appliance
Headgear
No
No
No
4
150
Intraoral appliance
Headgear
No
Yes
No
4
150
Molar
distalization
Movement of
upper incisors
40
26
Li 2011 4th
Skeletal
anchorageeffects
Skeletal anchorage
27
Liu 2010 1st
Tooth extraction
Tooth extraction
28
Liu 2010 2nd
Tooth extraction
Tooth extraction
29
Liu 2010 3rd
Tooth extraction
Tooth extraction
30
Liu 2010 4th
Tooth extraction
Tooth extraction
31
Long 2013 1st
32
Long 2013 5th
33
Long 2013 6th
34
35
Pandis 2014
1st
Pandis 2014
2nd
36
Papadopoulos
2011 1st
37
Papadopoulos
2011 2nd
38
Papadopoulos
2012 5th
39
Papadopoulos
2012 6th
40
Papadopoulos
2012 7th
41
Papadopoulos
2012 8th
Lingual
appliances
Lingual
appliances
Lingual
appliances
Self-ligating
brackets
Self-ligating
brackets
Skeletal
anchorageeffects
Skeletal
anchorageeffects
Cleft lip
with/without
palate
Cleft lip
with/without
palate
Cleft lip
with/without
palate
Cleft lip
with/without
palate
No tooth
extraction
No tooth
extraction
No tooth
extraction
No tooth
extraction
Maxillary
incisor
inclination
Lower lip
prominence
Upper lip
prominence
Lower lip
thickness
Upper lip
thickness
Lingual appliances
Labial appliances
Pain intensity
Yes
Yes
Yes
4
201
Lingual appliances
Labial appliances
Eating difficulty
Yes
Yes
Yes
4
201
Lingual appliances
Labial appliances
Speaking
difficulty
Yes
Yes
Yes
3
154
Self-ligating
brackets
Self-ligating
brackets
Conventional
brackets
Conventional
brackets
Teeth alignment
No
No
No
5
277
Teeth alignment
No
No
No
2
123
Skeletal anchorage
Conventional
anchorage
Anchorage loss
No
Yes
No
10
206
Skeletal anchorage
Conventional
anchorage
Anchorage loss
rate
No
Yes
No
6
153
Presurgical infant
orthopedics
Control
(untreated)
Speech
development
No
No
No
2
74
Presurgical infant
orthopedics
Control
(untreated)
SNA angle
No
No
No
2
329
Presurgical infant
orthopedics
Control
(untreated)
SNB angle
No
No
No
2
299
Presurgical infant
orthopedics
Control
(untreated)
SN-MP angle
No
No
No
2
151
Headgear
41
No
Yes
No
2
77
No
Yes
No
5
356
No
Yes
No
3
136
No
Yes
No
3
136
No
Yes
No
3
136
42
Papadopoulos
2012 12th
43
Papadopoulos
2012 15th
44
Papadopoulos
2012 17th
45
Papadopoulos
2012 18th
46
Papadopoulos
2012 19th
47
Papadopoulos
2012 22nd
48
Papageorgiou
2012 1st
49
Papageorgiou
2012 2nd
50
Papageorgiou
2012 3rd
51
Papageorgiou
2012 4th
52
Papageorgiou
2012 5th
53
Papageorgiou
2012 7th
54
55
Papageorgiou
2014 2nd
Papageorgiou
2014 3rd
Cleft lip
with/without
palate
Cleft lip
with/without
palate
Cleft lip
with/without
palate
Cleft lip
with/without
palate
Cleft lip
with/without
palate
Cleft lip
with/without
palate
Skeletal
anchorage-failure
rates
Skeletal
anchorage-failure
rates
Skeletal
anchorage-failure
rates
Skeletal
anchorage-failure
rates
Skeletal
anchorage-failure
rates
Skeletal
anchorage-failure
rates
Self-ligating
brackets
Self-ligating
brackets
Presurgical infant
orthopedics
Control
(untreated)
Maxillary arch
depth
No
No
No
2
60
Presurgical infant
orthopedics
Control
(untreated)
Maxillary arch
width I
No
No
No
2
52
Presurgical infant
orthopedics
Control
(untreated)
Maxillary arch
width II
No
No
No
2
65
Presurgical infant
orthopedics
Control
(untreated)
Maxillary arch
width III
No
No
No
2
65
Presurgical infant
orthopedics
Control
(untreated)
Maxillary arch
form I
No
No
No
2
65
Presurgical infant
orthopedics
Control
(untreated)
Maxillary arch
form II
No
No
No
2
62
Female patients
Male patients
Implant failure
Yes
Yes
No
9
[1724]
Adolescent patients
Adult patients
Implant failure
Yes
Yes
No
5
[1124]
Right mouth side
Left mouth side
Implant failure
Yes
Yes
No
8
[1295]
Maxilla
Mandible
Implant failure
Yes
Yes
No
16
[2315]
Cortical bone
thickness<1mm
Cortical bone
thickness>1mm
Implant failure
Yes
Yes
No
2
[296]
No root contact
Root contact
Implant failure
Yes
Yes
No
4
[604]
Self-ligating
brackets
Self-ligating
brackets
Conventional
brackets
Conventional
brackets
Tooth alignment
No
No
No
5
281
Intercanine
width
No
No
No
5
284
42
56
Papageorgiou
2014 4th
Self-ligating
brackets
Self-ligating
brackets
Conventional
brackets
Intermolar width
No
No
No
5
284
57
Papageorgiou
2014 8th
Self-ligating
brackets
Self-ligating
brackets
Conventional
brackets
Mandibular
incisor
inclination
No
Yes
No
3
174
58
Papageorgiou
2014 9th
Self-ligating
brackets
Conventional
brackets
Pain intensity
No
Yes
Yes
5
266
59
Perinetti 2014
1st
Fixed funtional
appliances
Control
(untreated)
Total
mandibular
length
No
No
No
5
325
60
Perinetti 2014
2nd
Fixed funtional
appliances
Control
(untreated)
Composite
mandibular
length
No
No
No
4
228
61
Perinetti 2014
3rd
Fixed funtional
appliances with
fixed appliances
Control
(untreated)
Total
mandibular
length (pubertal
patients)
No
No
No
6
352
62
Perinetti 2014
5th
Fixed funtional
appliances with
fixed appliances
Control
(untreated)
Composite
mandibular
length
No
No
No
2
89
63
Yang 2014 1st
Fränkel appliance
Control
(untreated)
SNA angle
No
No
No
5
234
64
Yang 2014
2nd
Fränkel appliance
Control
(untreated)
SNB angle
No
No
No
5
234
65
Yang 2014 3rd
Fränkel appliance
Control
(untreated)
MPA angle
No
No
No
4
136
66
Yang 2014 4th
Self-ligating
brackets
Class II
malocclusionfunctional
appliances
Class II
malocclusionfunctional
appliances
Class II
malocclusionfunctional
appliances
Class II
malocclusionfunctional
appliances
Class II
malocclusionfunctional
appliances
Class II
malocclusionfunctional
appliances
Class II
malocclusionfunctional
appliances
Class II
malocclusionfunctional
appliances
Fränkel appliance
Control
(untreated)
ANB angle
No
No
No
5
234
43
67
Yang 2014 5th
Class II
malocclusionfunctional
appliances
Maxillary
expansion
Maxillary
expansion
Maxillary
expansion
Fränkel appliance
Control
(untreated)
Slow maxillary
expansion
Rapid maxillary
expansion
Rapid maxillary
expansion
Control
(untreated)
Control
(untreated)
Slow maxillary
expansion
Overjet
No
No
No
4
163
Maxillary
No
No
No
4
220
intermolar width
Maxillary
69
Zhou 2014 3rd
No
No
No
6
479
intermolar width
Maxillary
70
Zhou 2014 6th
No
No
No
3
104
intermolar width
Maxillary
Maxillary
Slow maxillary
Control
71
Zhou 2014 9th
intercanine
No
No
No
3
112
expansion
expansion
(untreated)
width
Zhou 2014
Maxillary
Slow maxillary
Control
Mandibular
72
No
No
No
4
220
11th
expansion
expansion
(untreated)
intermolar width
Maxillary
Zhou 2014
Maxillary
Rapid maxillary
Control
73
intercanine
No
No
No
6
479
13th
expansion
expansion
(untreated)
width
Maxillary
Zhou 2014
Maxillary
Rapid maxillary
Control
74
interpremolar
No
No
No
6
479
16th
expansion
expansion
(untreated)
width
Zhou 2014
Maxillary
Rapid maxillary
Control
Mandibular
75
No
No
No
5
429
19th
expansion
expansion
(untreated)
intermolar width
Maxillary
Zhou 2014
Maxillary
Rapid maxillary
Slow maxillary
76
intercanine
No
No
No
3
104
22nd
expansion
expansion
expansion
width
Zhou 2014
Maxillary
Rapid maxillary
Slow maxillary
Mandibular
77
No
No
No
2
84
28th
expansion
expansion
expansion
intermolar width
*Number refers to the trials/trial arms included in the original meta-analysis. Trials might have been omitted from the meta-epidemiological comparison for overlap or
eligibility reasons.
$Numbers in parentheses indicate number of implants and not number of patients. Multiple implants might have been inserted per patient.
68
Zhou 2014 1st
44
Appendix 4 D. Included trials in the selected meta-analyses with the judgments about their design (citations
available upon request).
TrialID*
First author/Year
Design Description (quotation from published report)
Subjects with a maxillary bilateral crossbite were selected and two treatment
9699403
Akkaya 1998
pCCT
groups each with 12 patients were constructed.
The conditions for patient enrollment, based on personal choice, could be
assimilated to a random allocation of patients. Each practitioner enrolled 28
19524823
Bacceti 2009
pCCT
patients consecutively for treatment with his respective specific 1-phase
orthodontic protocol. One practitioner performed HG1FA therapy with Class
II elastics, and the other treated 28 patients with the BH 1 FA protocol.
Two groups of 20 adolescents (5 boys and 15 girls in each group), one treated
9674676
Bondemark 1998
pCCT
and one untreated, participated in the study and were followed longtudinally
for 5 years.
None of the patients presented with severe craniofacial anomalies and all were
9088600
Bravo 1997
pCCT
to be treated with Edgewise appliances.
After three months of treatment, each patient completed a seven-part survey
15747820
Caniklioglu 2005
pCCT
with 12 questions…
Success of therapy at the end of the observation period was not a determinant
20578848
Cevidanes 2010
pCCT
factor for selection of patients because this sample was collected
prospectively.
The aim of this prospective study was to investigate the complications and
14982362
Cheng 2004
pCCT
failures of orthodontic mini-implants in a series of consecutive patients.
...all patients signed an informed consent form before participating in the
23876947
Chiqueto 2013
pCCT
study.
Inclusion criteria were as follows:...no previous orthodontic treatment.
19892288
El-Beialy 2009
pCCT
The experiment ended 6 months after loading the miniscrew.
The patients examined were part of a prospective clinical investigation, the
16679203
Geran 2006
pCCT
Michigan Expansion Study, of mixed dentition patients who underwent RME
in a private faculty practice.
The purpose of this prospective clinical trial, therefore, was to investigate the
22423185
Ghislanzoni 2013
pCCT
role of timing in the treatment of Class II malocclusion with MARA and fixed
appliances with respect to Class II untreated control data.
In this prospective study nine patients with a mean age of 17.4 years (range
17524619
Hedayati 2007
pCCT
15.5–19 years) were randomly selected from patients referred to the
Orthodontic Department
Patients received a questionnaire to assess their opinion regarding the
18384412
Justens 2008
pCCT
treatment…
They volunteered to receive orthodontic treatment with maxillary miniimplants and agreed to have a CBCT scan after mini-implant placement. They
20816295
Kim 2010 b
pCCT
also consented to have the removal torque measured with a digital device after
treatment.
None of them had a severe craniofacial anomaly, and all were to be treated
12142899
Kocadereli 2002
pCCT
with edgewise appliances.
This prospective controlled study investigated the net effects of the Twin
9457025
Lund 1998
pCCT
Block functional appliance taking into account the effects of normal growth in
an untreated control group.
In this prospective clinical study we investigated the factors of importance for
17580417
Luzi 2007
pCCT
the failure rate of immediately loaded orthodontic mini-implants.
Treated group. The treated sample analyzed in this study (112 subjects, 61
females and 51 males) was part of a long-term prospective study on
12940553
McNamara Jr 2003
pCCT
consecutively treated patients who had undergone Haas-type RME and
nonextraction edgewise appliance therapy in a single orthodontic practice.
Eighteen patients (5 males and 13 females; mean age 23.8; minimum 10.7 and
20413451
Miyazawa 2010
pCCT
maximum 45.5 years) who required skeletal anchorage for orthodontic
therapy were included in this prospective study.
16441792
Motoyoshi 2006
pCCT
The data presented here are from subjects who gave their consent to be part of
45
17974113
Motoyoshi 2007 a
pCCT
17521887
Motoyoshi 2007b
pCCT
16905065
O'Grady 2006
pCCT
6961781
Pancherz 1982
pCCT
22640678
Phelan 2012
pCCT
R_987654321
Rosenberg 2008
pCCT
9082855
Sandikcioglu 1997
pCCT
21536207
Sar 2011
pCCT
21750242
Shalish 2012
pCCT
21536211
Suzuki 2011
pCCT
12401054
Thomson 2002
pCCT
17346597
Turnbull 2007
pCCT
19577145
Viwattanatipa 2009
pCCT
R_987654309
Waheed-Ul-Hameed
2002
pCCT
17348892
Wiechmann 2007
pCCT
20018798
Wu 2010
pCCT
14717690
Aly 2004
qRCT
18193961
Chaddad 2008
qRCT
15259572
Lohmander 2004
qRCT
11709597
Lowe 2001
qRCT
the study before surgical placement of the mini-implant.
This study used only the diagnostic materials of patients who consented to the
placement of mini-implants in cooperation with this study.
The data presented here are from patients who gave their consent to be part of
the study before placement of the mini-implant.
The patients examined were part of the MES, a prospective clinical
investigation of mixed-dentition patients who had undergone RME.
The control subjects were followed on a parallel basis with the treated
subjects during a time period of 6 months…
This prospective clinical study was based on the records of 34 consecutively
treated patients from the private practice of the third author (R.H.).
..Hence students would not be penalized for agreeing to use an educational
tool that they might not benefit from.
..A detailed answer sheet was developed prior to administration of the test so
as to standardize grading.
Patients with unilateral or bilateral posterior crossbites in the mixed dentition
were studied. They were divided into three groups of 10 patients in each
group.
The aim of this prospective clinical study was to evaluate the skeletal,
dentoalveolar, and soft tissue effects of maxillary protraction with miniplates
compared with conventional facemask therapy and an untreated Class III
control group.
Consecutive patients were recruited prospectively from the orthodontic clinic
in the Hebrew University-Hadassah School of Dental Medicine and from two
private clinics.
In this study, direct analysis was performed of MIT and MRT values of
predrilling and self-drilling miniscrew implants that were used as skeletal
anchorage in orthodontic patients.
This investigation used epidemiological data from a longstanding prospective
observational study to systematically evaluate the equity, efficacy,
effectiveness, and safety of orthodontic treatment
Introduction: In this prospective clinical study, we assessed the relative speed
of archwire changes, comparing self-ligating brackets with conventional
elastomeric ligation methods, and further assessed this in relation to the stage
of orthodontic treatment represented by different wire sizes and types.
No patients withdrew from our study.
A total of 20 functional class-3 cases were chosen from a general clinic
intake. 10 of the 20 cases formed the treatment group while 10 untreated
patients were taken as a control group....The patients were observed over a
period of one year.
The aim of this prospective study was therefore to investigate the clinical
outcome of orthodontic micro-implants by a prospective study in a series of
consecutive patients.
Sixty adult patients treated in the Orthodontic Department, Prince Philip
Dental Hospital, Hong Kong, over a 3 month period were included in this agematched case–control prospective longitudinal study.
Students were alphabetically assigned to each group. They could not
themselves select in what way to study the subject matter.
Ten healthy patients, ages 13 to 65 years, whose treatment plan included the
use of temporary anchorage devices (TADs), were included in the study...The
two mini-implant systems were alternately placed until a minimum of 15
mini-implants were placed for both systems
The first ten children (five girls and five boys) were treated with infant
orthopaedics (IO-group), whereas the following ten children (two girls and
eight boys) were not treated with infant ortopaedics (no-IO-group).
Eighty-five 3rd year undergraduate dental students allocated by pseudo-
46
16429868
17693371
19959610
Miles 2005
Pandis 2007
Pandis 2010
qRCT
qRCT
qRCT
6951655
Peat 1982
qRCT
22214390
Al-Jewair 2012
rCCT
12938839
Allais 2003
rCCT
19651342
Antoszewska 2009
rCCT
15014405
Baik 2004
rCCT
10730669
Bowman 2000
rCCT
17868386
Chen 2007
rCCT
18983323
Chen 2008
rCCT
10474101
Erdinc 1999
rCCT
21299410
Franchi 2011
rCCT
21299408
Ghislanzoni 2011
rCCT
10833001
Handelman 2000
rCCT
11683811
Harradine 2001
rCCT
16257988
Isik 2005
rCCT
12923511
Janson 2003
rCCT
15827701
Kalavritinos 2005
rCCT
17465652
Kucukkeles 2007
rCCT
17208101
Kuroda 2007
rCCT
17448389
Kuroda 2007
rCCT
randomisation to two groups.
Consecutive eligible patients were assigned alternately to one of two groups.
Personal communication with author.
Personal communication with author.
The decision to use pre-surgical oral orthopedics depended mainly on
traveling distance. If traveling involved several hundred miles then no early
treatment was carried out, although other cases simply slipped through the
early screening.
A retrospective study was conducted using lateral cephalograms in habitual
occlusion of adolescent patients who received treatment for their Class II
skeletal malocclusions using the MARA or the AdvanSync functional
appliances.
A retrospective case–control study based on the analysis of study-casts and
intra-oral slides of 300 adult patients was carried out
Our aims in this retrospective study were to investigate the success rates of
MIs,…
The criteria for sample selection were as follows: … (5) no appliances used
other than the FR III, and (6) good cooperation during the treatment period
(the patients wore the appliance for at least 14 hours per day).
A sample of 120 Caucasian orthodontic patients (70 extraction and 50
nonextraction, Table 1) was randomly selected from the senior author’s
treatment files.
Objectives: The aim of this retrospective study was to assess systematically
the case distribution among three types of mini-implants and …
Objectives: The aim of this retrospective study was to evaluate systematically
the potential factors that influence failure rates of temporary anchorage
devices (TADs) used for orthodontic anchorage.
In this retrospective study, altogether 37 cases with posterior crossbites
forming two treatment groups and one control group were treated at the
Department of Orthodontics, Istanbul University, Faculty of Dental Medicine.
A sample of 32 subjects with Class II division 1 malocclusion ... was treated
consecutively at a single private practice by one of the authors.
A sample of 27 subjects was selected from the files of the University of
Michigan Growth Study (12 subjects) and of the Denver Child Growth Study
(15 subjects).
From a parent sample of 62 Class II division 1 subjects treated consecutively
with the MARA appliance, 23 subjects were selected according to the
following inclusion criteria
Inclusion in the study required that the subject be 18 years or older at the time
of the pretreatment records and that records from before and after appliance
therapy were available.
Design – A retrospective study of two groups.
Pre- and post-treatment orthodontic models of 84 patients comprised the
subject matter of this retrospective study (Table 1).
Because this was a retrospective study, the information regarding hygiene
status during treatment was obtained from the clinical charts
The aim of this retrospective clinical study was to evaluate dental arch,
skeletal, dentoalveolar, and soft tissue profile changes following treatment of
Class III malocclusion by means of the Function Regulator (FR-3) appliance.
The sample consisted of the records from 45 growing patients (22 boys, 23
girls) exhibiting skeletal Class II malocclusion characterized by mandibular
retrognathism.
Seventy-five patients, 116 titanium screws of 2 types, and 38 miniplates were
retrospectively examined.
Seventy-five patients, 116 titanium screws of 2 types, and 38 miniplates were
retrospectively examined.
47
7625394
Ladner 1995
rCCT
20152674
Lee 2010
rCCT
18929269
Levin 2008
rCCT
18405816
Lim 2008
rCCT
19651354
Lim 2009
rCCT
20926556
Manni 2011
rCCT
9674675
Mills 1998
rCCT
14560266
Miyawaki 2003
rCCT
18193973
Moon 2008
rCCT
20620833
Moon 2010
rCCT
3181296
Ogaard 1988
rCCT
20691348
Ong 2010
rCCT
22432591
Pangrazio 2012
rCCT
12637901
Pangrazio-Kulbersh
2003
rCCT
18405824
Park 2008
rCCT
18617111
Phatouros 2008
rCCT
W_000080316600375
Ribeiral 1999
rCCT
22084789
Sharma 2011
rCCT
21674183
Shundo 2012
rCCT
19852635
Siara-Olds 2010
rCCT
15947523
Sidlauskas 2005
rCCT
10587592
Toth 1999
rCCT
A retrospective study of dental and maxillary skeletal changes occurring
during a period of orthodontic treatment was made from pretreatment and
posttreatment dental casts.
One hundred forty-one orthodontic patients (treated from October 1, 2000, to
November 29, 2007) were included in this survival study. Oral hygiene status
was determined by reviewing orthodontic records and intraoral photos
subjectively.
The aim of this retrospective controlled investigation was to analyze the shortterm and long-term skeletal and dentoalveolar treatment outcomes of Function
Regulator 3 (FR-3) therapy.
Fifty premolar extraction and 50 nonextraction Korean patients were selected
from the files of Chonnam National University Hospital in Gwangju, Korea.
One hundred fifty-four consecutive patients (47male, 107 female; mean age,
21.9 years; SD, 8.3 years) who had miniscrews placed as orthodontic
anchorage were included in this retrospective study
In this retrospective study, conducted in a private practice, the records of 300
miniscrews inserted in 132 consecutive patients (80 females, 60.6 per cent) by
the same surgeon were evaluated.
Pretreatment and posttreatment cephalometric records of 28 consecutively
treated patients with Class II malocclusions were evaluated and compared
with an age- and sex-matched sample of untreated Class II control subjects.
The clinical features and treatment progress for 1 year were retrospectively
examined for the 51 subjects.
Materials and Methods: Four hundred eighty OMI placed in 209 orthodontic
patients were examined retroactively.
The samples in this retrospective study consisted of 778 OMIs
Ninety-eight individuals were examined of whom 51 (28 girls and 23 boys),
had received orthodontic treatment. They had been treated for different forms
of malocclusion in private practices.
Patient records were included if they satisfied the following inclusion
criteria:...(3) intraoral photos and study models were available at pretreatment
(T0), 10 weeks (T1), and 20 weeks (T2) postbonding; …
This retrospective cephalometric study examined 30 consecutively treated
patients involving 12 boys with a mean age of 11.9 years
The following criteria were established for the sample:...(6) the appliance was
not removed prematurely due to breakage.
Sixteen nongrowing patients (14 women, 2 men; ages, 22.5 4.8 years) who
had been treated orthodontically for bialveolar protrusion were selected.
The purpose of this retrospective study was to estimate the area change of the
palate after rapid maxillary expansion (RME) in the early mixed dentition
stage by using a 3-dimensional (3D) helical computed tomography (CT)
scanning technique.
This study aimed at analyzing the periodontal state of 53 orthodontic treated
patients (finished cases for at least two years), whose ages ranged from 17 to
26 years.
The aim of this retrospective study was to find factors related to the clinical
success of micro-implants in Asian patients.
The subjects in the treatment and control groups were selected retrospectively
In this retrospective long-term investigation, the treatment groups were
chosen strictly based upon the appliance used for the correction of the Class II
malocclusion and not upon their treatment responses.
Cephalometric analysis of skeletal and dentoalveolar facial structures of 34
Class II Divison 1 patients treated with Twin-block appliance was performed
using the same reference system before and after treatment.
This retrospective cephalometric study compares the treatment effects
produced in 40 patients treated…
48
19615569
Wu 2009
rCCT
16679208
Xu 2006
rCCT
18984393
Yao 2008
rCCT
21357655
Baysal 2013
tRCT
16279817
Bondemark 2005
tRCT
19123718
Fleming 2009
tRCT
19409342
Fleming 2009 a
tRCT
19732667
Fleming 2009 b
tRCT
21195256
Godoy 2011
tRCT
23075062
Khattab 2012
tRCT
12498603
Konst 2003
tRCT
19602104
Liu 2009
tRCT
18540016
Ma 2008
tRCT
21889063
Pandis 2011
tRCT
18538237
Petren 2008
tRCT
11695749
Prahl 2001
tRCT
19651344
Pringle 2009
tRCT
18929262
Scott 2008 a
tRCT
18339656
Scott 2008 b
tRCT
18617099
Upadhyah 2008a
tRCT
18298220
17124564
R_987654320
10194289
19087583
20122433
Baek 2008
Berens 2006
Cha 2011
Franchi 1999
Jiang 2008
Kim 2010 a
Unclear
Unclear
Unclear
Unclear
Unclear
Unclear
From January 2001 to December 2006, 166 patients (35 male patients and 131
female patients; mean age, 26.5 8.9 years) who had received mini-implants
(total number, 414) for orthodontic anchorage at the Section of Orthodontics
and Pediatric Dentistry, Taipei Veterans General Hospital (Taipei, Taiwan)
were enrolled in this study
Records of 39 borderline patients treated at the Faculty Clinic of the Peking
University Orthodontic Department were evaluated retrospectively by 5
associate professors.
The subjects in this retrospective study included 47 adults (4 men, 43
women).
Randomization was made at the start of the study with preprepared random
number tables with block stratification on gender.
A restricted randomization method was used in blocks of 10 to ensure that
equal numbers of patients were allocated to each of the two treatment groups.
An unstratified subject allocation sequence was generated by a computer
program; random numbers were generated and assignment was concealed
from the clinician until the time of the appointment at which the appliance
was to be placed.
An unstratified subject allocation sequence was made by using a computergenerated randomization program.
An unstratified subject allocation sequence was generated using an electronic
randomization program.
For randomization, numbers were randomly drawn from a plastic bag.
They were divided into two groups; patient assignment was based on
computer-generated random numbers.
Babies with complete UCLP without soft tissue bands were recruited within 2
weeks after birth and randomly assigned to one of two groups by means of
computerized balancing with regard to alveolar cleft width and birth weight.
After informed consent, they were randomly assigned to two groups with the
aid of a table of random numbers:
The subjects were randomly divided (RandA1.0 Software, Planta Medical
Technology and Development Co. Ltd, Beijing, China) into two equal groups,
one anchored by intraoral micro-implants and one by extraoral headgear
Fifty patients were randomized to either a conventional or a self-ligating
appliance. The statistical software package was used by the first author, and
the user-written ralloc command was implemented to generate the random
allocation sequence.
4 opaque envelopes were prepared with 20 sealed notes in each (5 notes for
each group).
A computerised balanced allocation method was used in order to reduce
imbalance on relevant prognostic factors between groups.
Simple randomization was done with a computer-generated list of random
numbers.
The subjects were randomly allocated for treatment with either Damon3 selfligating brackets or Synthesis (Ormco) preadjusted edgewise brackets by
using a restricted random number table to ensure equivalence of numbers in
each group.
Randomization was carried out using a table of random numbers.
A restricted randomization method was used in blocks of 10 to ensure that
equal numbers of patients were allocated to each treatment group.
No description
No description
No description
No description
No description
No description
49
This study used only the data for patients who consented to placement of the
miniimplants and agreed to participate in the study.
16849067
Park 2006
Unclear No description
19472896
Wang 2009
Unclear No description
19649577
Wilmes 2009
Unclear No description
The patients were randomly divided into two groups: group 1 (seven females
and eight males; mean age 15.0 ± 3.4 years), were treated with a pendulum
20231213
Acar 2010
uRCT
appliance supported with a K-loop buccally, while subjects in group 2 (10
females and 5 males; mean age 14.2 ± 2.9 years) were treated with CHG.
A comparative study consisting of 14 patients (all females) randomized into 2
20386216
Basha 2010
uRCT
groups.
Students were randomly allocated either to the lecture group or the computer
9459032
Clark 1997
uRCT
group.
This was a controlled clinical trial on the effects of 2 approaches for Class II
17628251
De Oliveira 2007
uRCT
correction, with randomization in the assignment of the 2 treatment regimens.
The first 58 patients who had been on the waiting list for the longest period
9825553
Illing 1998
uRCT
were randomly allocated to one of three groups involving treatment with
either a Bass appliance, a Bionator, or a Twin Block appliance.
All 56 sophomore dental students volunteered to participate and were
3519716
Irvine 1986
uRCT
randomly assigned to instruction either by means of the computer or via the
lecture method
R_987654310
Jiang 2009
uRCT
[patients were randomly allocated to one of two groups]
The 103 members of the second-year dental class were randomly assigned
12056770
Komolpis 2002
uRCT
into two study groups, conventional and web-based.
6388628
Luffingham 1984a
uRCT
Students were allocated randomly into ten groups
20575195
Miles 2010
uRCT
The subjects were randomly allocated to one of two groups.
Eight other children [Hotz(-) group] were selected at random who did not
8734726
Mishima 1996
uRCT
receive the appliance.
The anterior teeth of 10 randomly selected cases were retraced by anchorage
R_987654312
Shi 2008
uRCT
of miniscrews (Xi’an Zhongbang) with diameter of 1.5 mm and a length of 8
mm.
The subjects were randomly allocated to either a group treated with an intra21696108
Toy 2011
uRCT
oral Pendulum appliance with a midline expansion screw (PEN) or a group
treated with a Ricketts-type cervical headgear (CHG).
8198080
Ulgen 1994
uRCT
The patients were divided randomly into treatment and control groups.
19061808
Upadhyah 2008b
uRCT
Then the subjects were randomly divided into 2 groups before treatment
R_987654311
Uzdil 2008
uRCT
[randomly allocated in the four groups]
Twenty-nine patients (10 males and 19 females), between 14 and 30 years of
age (mean 20.7 years) who met the inclusion criteria, were invited to
21478298
Wahab 2012
uRCT
participate and were randomly allocated to be treated using either SLB or
CLB.
*TrialIDs without lettering indicate PubMed Identifiers; TrialIDs starting with W_ indicate Web of Knowledge unique
identifiers; TrialIDs starting with R_ indicate random unique IDs generated for non-indexed component trials.
pCCT, prospective clinical controlled trial; qRCT, quasi-randomized controlled trial; rCCT, retrospective clinical controlled trial;
tRCT, randomized controlled trial (clear and adequate description of random sequence generation method); Unclear, clinical trial
with unclear design; uRCT, randomized controlled trial (unclear random sequence generation method).
18963818
Motoyoshi 2009
Unclear
50
Appednix E. Forest plot for the comparison of RCTs with adequate random sequence generation and RCTs with inadequate/unclear
random sequence generation. Due to data recoding, estimates on the left side of the forest plot indicate that RCTs with
inadequate/unclear random sequence generation show larger treatment effects than RCTs with adequate random sequence generation.
ΔSMD = difference in standardized mean differences; RCT = randomized controlled trial.
.
51
Appendix F. Forest plot for the comparison of RCTs versus non-RCTs with subgroups according to the scope of component trials.
Due to data recoding, estimates on the right side of the forest plot indicate that RCTs show smaller treatment effects than non-RCTs.
ΔSMD = difference in standardized mean differences; RCT = randomized controlled trial; non-RCT = non-randomized controlled
trial.
52
Appendix G. Results of Egger’s test for small-study effects.
Reporting bias (Egger's test)
Comparison
Intercept (95% CI)
P-value
1. Randomized vs. non-randomized
-0.91 (-2.16,0.35)
0.148
2. Prospective vs. retrospective
3. Adequate vs. inadequate/unclear
random sequence generation
CI, confidence interval.
-0.78 (-1.78,0.23)
0.126
0.17 (-1.48,1.82)
0.830
53
Appendix H. Results of the sensitivity analyses
Original analysis
Fixed-effect model
ΔSMD
ΔSMD (95%
P-value
P-value
(95% CI)
CI)
1. Randomized vs.
non-randomized
(n=25)
0.07 (0.21,0.34)
0.630
2. Prospective vs.
-0.30 (0.018
retrospective (n=40) 0.53,-0.06)
0.12 (0.10,0.35)
Study design definition
ΔSMD (95%
P-value
CI)
0.286
-0.19 (-0.33,0.008
0.05)
-
Excluded Unclear
from retrospective;
dropped 5 MAs
-0.28 (0.63,0.08)
0.131
Largest MA from each SR
ΔSMD (95% PCI)
value
Most precise 50%
ΔSMD (95%
P-value
CI)
Dropped 14 0.19 (MAs
0.16,0.53)
0.298
Dropped 12
MAs
0.12 (0.22,0.45)
0.496
Dropped 29 -0.11 (MAs
0.45,0.24)
0.538
Dropped 20
MAs
-0.18 (0.40,0.05)
0.120
3. Adequate vs.
inadequate/unclear 0.01 (0.00 (Excluded uRCTs;
0.13 (Dropped 19 0.01 (Dropped 12 -0.03 (0.957
0.997
0.290
0.931
0.745
random sequence
0.25,0.26)
0.15,0.15)
dropped 16 MAs
0.11,0.36)
MAs
0.27,0.30)
MAs
0.24,0.17)
generation (n=25)
Bold values indicate statistical significance at the 5% level.
ΔSMD, difference in standardized mean differences; CI, confidence interval; MA, meta-analysis; Unclear, non-randomized controlled clinical trial with unclear study design; uRCT,
randomized controlled trial (unclear random sequence generation).
54