Download [S2] Assessment of specific center effects after adjustment

J Rüschoff et al
Supplementary Material
[S1] Sensitivity analyses
Sensitivity analyses were conducted based on several different patient sets and
several modifications of the multiple logistic regression model.
The initial analysis included all 16,528 samples which had a positive or negative
HER2 status in the dataset. The application of the stepwise multiple logistic
regression model identified six covariates with a significant effect on the probability
of a positive HER2 status; these were the five covariates used in the final model plus
type of carcinoma (invasive versus in situ), which had a P value of < 0.0001. Type of
carcinoma showed a statistically significantly higher positivity rate for patients with in
situ type of carcinoma compared with patients with invasive carcinoma. This full
patient set was used to show that it was appropriate to model age as a simple linear
covariate. In these analyses, as in the final analyses, missing values in the nominal
covariates were consistently modeled as an additional level denoted as “unknown”.
A sensitivity analysis was performed, where patients were included only if they had
measurements available for the five covariates, with the exception of nodal status. It
was concluded that both approaches lead to consistent results; for further analyses
we selected the first approach, which did not require excluding patients because of
missing covariates.
A further 631 patients with ductal carcinoma in situ (DCIS) or unknown type of
carcinoma were excluded, so that only patients with invasive carcinoma were
included in subsequent analyses. After exclusion of DCIS patients, type of carcinoma
had only one informative level left and was thus excluded from the model.
1
J Rüschoff et al
Two sensitivity analyses were conducted based on this analysis set of 15,721
patients. The first analysis showed that the potential additional covariate of sample
origin did not have a statistically significant effect on HER2-positivity and thus was
not included in the model. The second sensitivity analysis showed that excluding the
covariate nodal status from the model did not change the effects and relative
importance of the other variables.
In the third sensitivity analysis, we rebuilt our model using only resection samples
(n = 4,486). Information on nodal status was available for approximately 80% of
resection samples compared with approximately 19% of biopsy samples. As more
information on nodal status was available, the total effect this covariate had on the
model increased slightly (from 0.05 to 0.10). However, an additional center effect
was not identified.
Due to its clinical importance, nodal status was retained in the model with the
intention of ensuring a more complete data collection for this parameter in the future.
[S2] Assessment of specific center effects after adjustment for covariates
S2.1 Descriptive approach
For the individual centers, a predicted HER2-positivity rate was estimated using the
following two steps: 1) For each patient, the probability of HER2-positivity was
estimated as a function of the five identified patient- or tumor-related characteristics
using the model described earlier; 2) The predicted positivity rate at each center was
then estimated as the mean positivity rate across the patients at the centers.
The documented positivity rate at the individual centers was estimated as the relative
frequency of HER2-positive samples; 99% confidence intervals were calculated for
the documented positivity rates based on the binomial distribution. The difference
2
J Rüschoff et al
between the documented and predicted HER2-positivity rate of a center was
considered as a center effect that cannot be explained by covariates.
Figure 5 was generated to allow a comparison of documented and predicted HER2positivity rates across the centers, sorted according to their predicted positivity rate.
S2.2 Model-based approach
For this approach the multiple logistic regression model was extended to include a
nominal center variable with one level for each of the 48 individual centers and one
level for the pooled center. The overall P value of the included center effect was
small (P = 4.17e–16); the receiver operating characteristic area under the curve of
the extended model was area under the curve = 0.750 which meant an increase of
0.015 compared with the model without center effect, which had an area under the
curve of 0.735. The covariance-adjusted P value of each center level in the extended
model was interpreted as a measure for the statistical evidence of a specific effect of
this center that cannot be explained by the patient characteristics. As there was no
intention of testing the effect of the pooled center statistically because of lack of
potential for interpretation, 48 statistical tests of center effects were planned and
conducted. To adjust for the resulting multiplicity issue, the Bonferroni-Holm
procedure was used. With the Bonferroni-Holm procedure, the effect of the center
with the smallest P value was tested on the usual level of alpha = 0.05 after
multiplication of this P value by 48; when this Bonferroni-Holm-adjusted P value was
< 0.05, then the second smallest P value was multiplied by 47 (= 48–1) and again
compared with the alpha level of 0.05. This procedure was continued until the first
adjusted P value was ≥ 0.05; if this occurred, no further statistical significances were
declared on this level. The Bonferroni-Holm procedure is well known to control the
family-wise error rate on the specified level in the strong sense; also, we assumed
3
J Rüschoff et al
that the Bonferroni-Holm procedure is not too conservative here, because the
different centers in this study can be considered as statistically independent. Despite
this control of error rate we interpret P values mainly in the sense of descriptive data
analysis.
[S3] Correlation of center effect with center properties
S3.1 Influence of type of center on center effect
Overall, 48 centers contributed samples to this study. Of these, the three types of
center were based either at a university or university clinic (“university hospitals”; 11
centers [22.9%]), located at other hospitals (“other hospitals”; 19 centers [39.6%]) or
run by pathologists in private practice (“private practice”; 18 centers [37.5%]).
A significant center effect or a trend toward a center effect was detected for the
groups “other hospitals” and “private practice”. In the group “private practice” (n =
18), two centers had a statistically significant center effect on HER2-positivity rate,
while a further three showed a trend toward a center effect. In the group “other
hospitals” (n = 19), one center showed a significant center effect. Further analysis
indicated that these center effects were independent of the number of samples
documented per center.
Based on the P values, there was a trend toward an increasing center effect in the
“private practice” group compared with the “other hospitals” group, particularly when
compared with centers described as “university hospitals” (P = 0.0378).
S3.2 Correlation of center effect with number of HER2 diagnostic tests per annum
Of the 48 centers which provided samples in this study, 41 centers (85.4%) specified
the number of breast cancer cases diagnosed on an annual basis. Overall, no
4
J Rüschoff et al
statistically significant correlation was found between center effect and the numbers
of cases diagnosed per year (P = 0.7159).
Analysis 3.3: Influence of consecutive versus selected documentation on center
effect
Overall, 32 of the 48 centers (67.7%) which contributed samples to this analysis
provided all of their samples from routine diagnostics within a specified time period.
Two centers (4.2%) confirmed selection of samples provided, while 14 centers
(29.2%) did not provide information regarding sample selection. Based on this
information, we did not detect any statistically significant association between
significance of center effect and whether there was information on whether samples
were unselected or selected (P = 0.238).
S3.4 Influence of sample retesting for other pathologies on center effect
Overall, 11 of the 48 centers (22.9%) with documented cases were asked to provide
the results of sample retesting or a “second opinion”; 23 centers (47.9%) stated that
they did not document such cases and 14 centers (29.2%) did not provide
information on this question. Based on the information provided, there was no
correlation between documentation of “second opinion” cases and significance of
center effect (P = 0.5677).
S3.5 Influence of extent of immunohistochemistry/in situ hybridization testing
automation on center effect
In this study, 23 of the 48 centers (47.9%) used fully automated systems for
immunohistochemistry and in situ hybridization. Nine centers (18.8%) used semiautomated systems and two centers (4.2%) used manual systems. Fourteen centers
(29.2%) did not provide information on the extent of automation. Overall, no
5
J Rüschoff et al
statistically significant differences were observed when comparing the significance of
center effect with the amount of automation in immunohistochemistry/in situ
hybridization testing (P = 0.364).
6
J Rüschoff et al
Supplementary Table S1. Patient/tumor characteristics considered in statistical
analyses
Patient/tumor characteristics
Modeling type
Age
Continuous
Nominal; 3+1 (= unknown)
Histologic grade
levels
Hormone receptor status
Nominal; 4+1 levels
Nodal status
Nominal; 4+1 levels
Nominal; 2+2 levels (others
Histologic subtype (lobular versus ductal)
and unknown)
Type of carcinoma (invasive versus in situ)
Nominal; 2+1 levels
Sample origin (primary tumor, metastatic site
Nominal; 5+1 levels
[primary, secondary], locally recurrent)
Method of sample retrieval (biopsy, resection)
Nominal; 2+1 levels
Age was modeled as a simple continuous variable; extended modeling of age as a
quadratic function or a several knot spline did not provide a significant additional
contribution. Missing values were coded as the separate level “unknown” in each of
the nominal influential variables; therefore, it was not necessary to exclude any
patients for this reason. Sample origin was included initially in the model but then
excluded from the main model because it did not contribute significantly after
adjustment for the other variables (P = 0.7484; P = 0.91 after manual restructuring of
its levels). In a final modeling step, method of sample retrieval was excluded from
the model because of a P value of 0.467.
7
J Rüschoff et al
Supplementary Table S2. Assessment of HER2-positivity by immunohistochemistry
and in situ hybridization
HER2-positivity assessment
Immunohistochemistry 3+
Immunohistochemistry 2+ and in situ
n (%)
1,740 (79.1)
313 (14.2)
hybridization-positive
in situ hybridization-positive only
111 (5.0)
Borderline in situ hybridization-positive
12 (0.5)
N/A
24 (1.1)
Total
2,200 (100.0)
N/A, supportive information not available.
8
J Rüschoff et al
Supplementary Table S3. Assessment of individual study centers based on
estimated center effect size and P values
Center N
28
206
Center
Cov adj.
Cov + B-Holm
effect %
P value
adj. P value
12.4%
1.31e–07
6.28e–06
Assessment P values
Cov + B-Holm adj.
P value < 0.05
32
640
–5.8%
1.74e–05
0.0008
Cov + B-Holm adj.
P value < 0.05
4
276
6.5%
4.19e–04
0.0193
Cov + B-Holm adj.
P value < 0.05
17
172
8.0%
0.0020
0.0920
Cov + B-Holm adj.
P value < 0.2
1
741
–3.7%
0.0023
0.1021
Cov + B-Holm adj.
P value < 0.2
35
661
3.5%
0.0027
0.1177
Cov + B-Holm adj.
P value < 0.2
14
712
3.9%
0.0051
0.2158
Cov adj. P value <
0.05
23
132
–8.2%
0.0067
0.2750
Cov adj. P value <
0.05
27
316
–5.0%
0.0100
0.4013
Cov adj. P value <
0.05
9
J Rüschoff et al
7
352
4.8%
0.0122
0.4748
Cov adj. P value <
0.05
60
126
–5.9%
0.0356
1.0
Cov adj. P value <
0.05
41
332
–3.4%
0.0458
1.0
Cov adj. P value <
0.05
2
171
5.4%
0.0485
1.0
Cov adj. P value <
0.05
11
401
–3.1%
0.0502
1.0
Cov adj. P value < 0.2
55
67
–8.5%
0.0519
1.0
Cov adj. P value < 0.2
50
113
6.5%
0.0602
1.0
Cov adj. P value < 0.2
30
125
4.7%
0.0619
1.0
Cov adj. P value < 0.2
15
306
3.3%
0.0853
1.0
Cov adj. P value < 0.2
58
313
–3.2%
0.0861
1.0
Cov adj. P value < 0.2
6
580
2.2%
0.0926
1.0
Cov adj. P value < 0.2
18
317
–2.9%
0.1176
1.0
Cov adj. P value < 0.2
5
90
–4.7%
0.1588
1.0
Cov adj. P value < 0.2
10
381
–2.2%
0.1843
1.0
Cov adj. P value < 0.2
37
33
8.9%
0.1891
1.0
Cov adj. P value < 0.2
31
63
5.5%
0.2347
1.0
Inconspicuous
46
10
J Rüschoff et al
Cov adj., covariate-adjusted; Cov + B-Holm adj., covariate- and Bonferroni-Holmadjusted.
We considered a Bonferroni-Holm-adjusted P value < 0.05 as the level 1 signal of a
conspicuous unexplained center effect, which occurred for the first three centers in
the above list; for these centers, this P value signal concurred with a relatively high
estimated center effect. A Bonferroni-Holm-adjusted P value < 0.2 is still considered
a signal of a conspicuous unexplained center effect, which seems to be confirmed by
the relatively large center effect of Center 17, but not so much by the moderate
center effects of Centers 1 and 35 where the P value signal seems to be mainly
related to the high numbers of patients at these two centers. Two weaker P valuebased signals (covariate-adjusted P value [not adjusted for multiplicity] < 0.05 and
< 0.2) were considered in addition, but here additional considerations are needed for
the assessment of the center effect. Centers 14 and 23 were identified as not
statistically significant after Bonferroni-Holm adjustment, but with a 99% confidence
intervals of the documented HER2-positivity rate that does not include the predicted
rate of these centers. The remaining 23 centers in general had relatively small center
effects combined with covariate-adjusted P values > 0.2 (and Bonferroni-Holmadjusted P values = 1); Center 31 was the only center effect of 5.5% ≥ 5%. All these
centers were considered as inconspicuous. For a final assessment, the results of
both the descriptive and the model-based approach should be considered.
11
J Rüschoff et al
Supplementary Table S4. Likelihood ratio test of the effect of covariates (including
center effect) on HER2-positivity and covariate importance in predicting HER2positivity
Variable
Likelihood ratio test of
Covariate importance in
effect of independent
predicting HER2-positivity
variables on
HER2-positivitya
Main effecta
Total effectb
1.85e–79
0.335
0.394
1.16e–47
0.226
0.249
4.17e–16
0.159
0.177
9.15e–27
0.116
0.130
Age
2.86e–17
0.071
0.078
Nodal status
7.067e–6
0.042
0.044
(P value)
Histologic grading
Hormone
receptor status
Center effect
Histologic
subtype (lobular
versus ductal)
a
The main effect reflects the relative contribution of that factor alone.
b The
total effect reflects the relative contribution of that factor both alone and in
combination with other factors.
The center effect ranks third in variable importance both in main and total effect; it
ranks at fifth place only based on the lowness of the P value because of its relatively
high number of degrees of freedom (df = 48).
12
J Rüschoff et al
Supplementary Table S5. Predicted HER2-positivity rates for covariate factor levels
ER, estrogen receptor; HER2, human epidermal growth factor receptor 2; HR,
hormone receptor; PgR, progesterone receptor.
13
J Rüschoff et al
Supplementary appendix
The following investigators belong to the NIU HER2 Study Group and provided
information on samples, along with clinical and diagnostic data for the analyses:
Ambrosius C-A. Institut für Pathologie Hilden, Hilden; Arndt N. Städtisches Klinikum
Dessau, Dessau; Baldus SE. Institut für Pathologie, Zytologie & Molekularpathologie,
Bergisch-Gladbach; Berger I. Institut für Pathologie Klinikum Kassel, Kassel;
Berghäuser K-H. Institut für Pathologie Thüringer Kliniken, Saalfeld; Bittmann I.
Medizinisches Versorgungszentrum am Diakoniekrankenhaus, Rotenburg; Blasius
S. Institut für Pathologie am Klinikum Hanau, Hanau; Bollmann R. Institut für
Pathologie, Bonn; Brockmann M. and Schildgen O. Klinik der Stadt Köln, gGmbH,
Cologne; Bürrig K-F. Pathologie Hildesheim, Hildesheim; Chmelar C. Pathologisches
Institut Recklinghausen, Recklinghausen; Decker T. Institut für Pathologie DietrichBonhoeffer-Klinikum, Neubrandenburg; Diergardt U. Pathologisches Institut am AKH,
Hagen; Dykgers A. Pathologisches Institut Dortmund, Dortmund; Engers R. Zentrum
für Pathologie Neuss & Pathologische Gemeinschaft Engers und Donner, Neuss;
Falk S. Gemeinschaftspraxis für Pathologie Frankfurt am Main, Frankfurt am Main;
Friedrich K. Institut für Pathologie des Universitätsklinikums Carl Gustav Carus an
der Technischen Universität, Dresden; Gabriele D. Institut für Pathologie
Kreiskliniken Reutlingen GmbH, Reutlingen; Gaiser T. Pathologisches Institut
Universitätsklinikum Mannheim, Mannheim; Gassel AM. Institut für Pathologie
Evangelisches Krankenhaus Oberhausen, Oberhausen; Gerharz C-D. Institut für
Pathologie Evangelisches Krankenhaus Bethesda, Duisburg; Haedicke W. Institut für
Pathologie am Evangelischen Waldkrankenhaus, Berlin-Spandau; Haroske G.
Krankenhaus Dresden Friedrichstadt & Städtisches Klinikum, Friedrichstadt;
Hartmann A. Pathologisches Institut Universitätsklinikum Erlangen, Erlangen;
14
J Rüschoff et al
Hellwig K. Universitätsklinikum Magdeburg GmbH, Magdeburg; Hinze R. Institut für
Pathologie HELIOS Kliniken Schwerin, Schwerin; Junker K. Institut für Pathologie
Klinikum Bremen Mitte, Bremen; Kasper U. Gemeinschaftspraxis im Medical-Center
am Clemens Hospital, Münster; Klauschen F. Charité Universitätsmedizin Berlin,
Berlin; Klosterhalfen B. Institut und Praxisgemeinschaft für Pathologie Krankenhaus
Düren, Dueren; Knöß P. Institut für Pathologie Bad Kreuznach, Bad Kreuznach;
Kreipe H. Medizinische Hochschule Hannover, Hannover; Kristiansen G. Institut für
Pathologie Universitätsklinikum Bonn, Bonn; Krüger S. Klinikum Magdeburg GmbH
Abt. Pathologie, Magdeburg; Kunze A. Gemeinschaftspraxis für Pathologie Bad
Berka, Bad Berka; Lüders P. Gemeinschaftspraxis für Pathologie Stendal, Stendal;
Neuber A. Institut für Pathologie, Dermatohistologie, Zytologie und
Molekularpathologie, Wesel; Niemann P. Gemeinschaftspraxis für Pathologie,
Hamm; Niendorf A. Pathologie Hamburg-West, Hamburg; Orth C. Institut für
Pathologie SRH Waldklinikum, Gera; Ott G. Robert Bosch Krankenhaus, Stuttgart;
Philippou S. Institut für Pathologie und Zytologie an der Augusta-Kranken-Anstalt,
Bochum; Richter-Sadocco B. Institut für Pathologie Hannover Zentrum, Hannover;
Röcken C. Institut für Pathologie Universitätsklinikum Schleswig-Holstein, SchleswigHolstein; Rüschoff J. Institut für Pathologie Nordhessen, Kassel; Sauter G.
Universitätsklinikum Hamburg-Eppendorf, Hamburg-Eppendorf; Schmitz J. Institut
für Pathologie Grevenbroich, Grevenbroich; Sen Gupta R. St. Agnes-Hospital
Bocholt-Rhede, Bocholt; Sinn P. Pathologisches Institut Universitätsklinikum
Heidelberg, Heidelberg; Staebler A. Institut für Pathologie und Neuropathologie
Universitätsklinikum Tübingen, Tübingen; Tennstedt-Schenk C. Gemeinschaftspraxis
für Pathologie Mühlhausen, Mühlhausen; Thorns C. Institut für Pathologie
Universitätsklinikum Lübeck, Lübeck; Tiemann K. Institut für Hämatopathologie,
15
J Rüschoff et al
Hamburg; Turzynski A. Pathoplan GbR, Lübeck; Vollmer E. Medizinisches
Versorgungszentrum Borstel, Borstel; Weimann R. Institut für Pathologie Klinikum
Saarbrücken, Saarbrücken; Wittersheim M. Institut für Pathologie
Universitätsklinikum Cologne, Cologne.
16