Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
J Rüschoff et al Supplementary Material [S1] Sensitivity analyses Sensitivity analyses were conducted based on several different patient sets and several modifications of the multiple logistic regression model. The initial analysis included all 16,528 samples which had a positive or negative HER2 status in the dataset. The application of the stepwise multiple logistic regression model identified six covariates with a significant effect on the probability of a positive HER2 status; these were the five covariates used in the final model plus type of carcinoma (invasive versus in situ), which had a P value of < 0.0001. Type of carcinoma showed a statistically significantly higher positivity rate for patients with in situ type of carcinoma compared with patients with invasive carcinoma. This full patient set was used to show that it was appropriate to model age as a simple linear covariate. In these analyses, as in the final analyses, missing values in the nominal covariates were consistently modeled as an additional level denoted as “unknown”. A sensitivity analysis was performed, where patients were included only if they had measurements available for the five covariates, with the exception of nodal status. It was concluded that both approaches lead to consistent results; for further analyses we selected the first approach, which did not require excluding patients because of missing covariates. A further 631 patients with ductal carcinoma in situ (DCIS) or unknown type of carcinoma were excluded, so that only patients with invasive carcinoma were included in subsequent analyses. After exclusion of DCIS patients, type of carcinoma had only one informative level left and was thus excluded from the model. 1 J Rüschoff et al Two sensitivity analyses were conducted based on this analysis set of 15,721 patients. The first analysis showed that the potential additional covariate of sample origin did not have a statistically significant effect on HER2-positivity and thus was not included in the model. The second sensitivity analysis showed that excluding the covariate nodal status from the model did not change the effects and relative importance of the other variables. In the third sensitivity analysis, we rebuilt our model using only resection samples (n = 4,486). Information on nodal status was available for approximately 80% of resection samples compared with approximately 19% of biopsy samples. As more information on nodal status was available, the total effect this covariate had on the model increased slightly (from 0.05 to 0.10). However, an additional center effect was not identified. Due to its clinical importance, nodal status was retained in the model with the intention of ensuring a more complete data collection for this parameter in the future. [S2] Assessment of specific center effects after adjustment for covariates S2.1 Descriptive approach For the individual centers, a predicted HER2-positivity rate was estimated using the following two steps: 1) For each patient, the probability of HER2-positivity was estimated as a function of the five identified patient- or tumor-related characteristics using the model described earlier; 2) The predicted positivity rate at each center was then estimated as the mean positivity rate across the patients at the centers. The documented positivity rate at the individual centers was estimated as the relative frequency of HER2-positive samples; 99% confidence intervals were calculated for the documented positivity rates based on the binomial distribution. The difference 2 J Rüschoff et al between the documented and predicted HER2-positivity rate of a center was considered as a center effect that cannot be explained by covariates. Figure 5 was generated to allow a comparison of documented and predicted HER2positivity rates across the centers, sorted according to their predicted positivity rate. S2.2 Model-based approach For this approach the multiple logistic regression model was extended to include a nominal center variable with one level for each of the 48 individual centers and one level for the pooled center. The overall P value of the included center effect was small (P = 4.17e–16); the receiver operating characteristic area under the curve of the extended model was area under the curve = 0.750 which meant an increase of 0.015 compared with the model without center effect, which had an area under the curve of 0.735. The covariance-adjusted P value of each center level in the extended model was interpreted as a measure for the statistical evidence of a specific effect of this center that cannot be explained by the patient characteristics. As there was no intention of testing the effect of the pooled center statistically because of lack of potential for interpretation, 48 statistical tests of center effects were planned and conducted. To adjust for the resulting multiplicity issue, the Bonferroni-Holm procedure was used. With the Bonferroni-Holm procedure, the effect of the center with the smallest P value was tested on the usual level of alpha = 0.05 after multiplication of this P value by 48; when this Bonferroni-Holm-adjusted P value was < 0.05, then the second smallest P value was multiplied by 47 (= 48–1) and again compared with the alpha level of 0.05. This procedure was continued until the first adjusted P value was ≥ 0.05; if this occurred, no further statistical significances were declared on this level. The Bonferroni-Holm procedure is well known to control the family-wise error rate on the specified level in the strong sense; also, we assumed 3 J Rüschoff et al that the Bonferroni-Holm procedure is not too conservative here, because the different centers in this study can be considered as statistically independent. Despite this control of error rate we interpret P values mainly in the sense of descriptive data analysis. [S3] Correlation of center effect with center properties S3.1 Influence of type of center on center effect Overall, 48 centers contributed samples to this study. Of these, the three types of center were based either at a university or university clinic (“university hospitals”; 11 centers [22.9%]), located at other hospitals (“other hospitals”; 19 centers [39.6%]) or run by pathologists in private practice (“private practice”; 18 centers [37.5%]). A significant center effect or a trend toward a center effect was detected for the groups “other hospitals” and “private practice”. In the group “private practice” (n = 18), two centers had a statistically significant center effect on HER2-positivity rate, while a further three showed a trend toward a center effect. In the group “other hospitals” (n = 19), one center showed a significant center effect. Further analysis indicated that these center effects were independent of the number of samples documented per center. Based on the P values, there was a trend toward an increasing center effect in the “private practice” group compared with the “other hospitals” group, particularly when compared with centers described as “university hospitals” (P = 0.0378). S3.2 Correlation of center effect with number of HER2 diagnostic tests per annum Of the 48 centers which provided samples in this study, 41 centers (85.4%) specified the number of breast cancer cases diagnosed on an annual basis. Overall, no 4 J Rüschoff et al statistically significant correlation was found between center effect and the numbers of cases diagnosed per year (P = 0.7159). Analysis 3.3: Influence of consecutive versus selected documentation on center effect Overall, 32 of the 48 centers (67.7%) which contributed samples to this analysis provided all of their samples from routine diagnostics within a specified time period. Two centers (4.2%) confirmed selection of samples provided, while 14 centers (29.2%) did not provide information regarding sample selection. Based on this information, we did not detect any statistically significant association between significance of center effect and whether there was information on whether samples were unselected or selected (P = 0.238). S3.4 Influence of sample retesting for other pathologies on center effect Overall, 11 of the 48 centers (22.9%) with documented cases were asked to provide the results of sample retesting or a “second opinion”; 23 centers (47.9%) stated that they did not document such cases and 14 centers (29.2%) did not provide information on this question. Based on the information provided, there was no correlation between documentation of “second opinion” cases and significance of center effect (P = 0.5677). S3.5 Influence of extent of immunohistochemistry/in situ hybridization testing automation on center effect In this study, 23 of the 48 centers (47.9%) used fully automated systems for immunohistochemistry and in situ hybridization. Nine centers (18.8%) used semiautomated systems and two centers (4.2%) used manual systems. Fourteen centers (29.2%) did not provide information on the extent of automation. Overall, no 5 J Rüschoff et al statistically significant differences were observed when comparing the significance of center effect with the amount of automation in immunohistochemistry/in situ hybridization testing (P = 0.364). 6 J Rüschoff et al Supplementary Table S1. Patient/tumor characteristics considered in statistical analyses Patient/tumor characteristics Modeling type Age Continuous Nominal; 3+1 (= unknown) Histologic grade levels Hormone receptor status Nominal; 4+1 levels Nodal status Nominal; 4+1 levels Nominal; 2+2 levels (others Histologic subtype (lobular versus ductal) and unknown) Type of carcinoma (invasive versus in situ) Nominal; 2+1 levels Sample origin (primary tumor, metastatic site Nominal; 5+1 levels [primary, secondary], locally recurrent) Method of sample retrieval (biopsy, resection) Nominal; 2+1 levels Age was modeled as a simple continuous variable; extended modeling of age as a quadratic function or a several knot spline did not provide a significant additional contribution. Missing values were coded as the separate level “unknown” in each of the nominal influential variables; therefore, it was not necessary to exclude any patients for this reason. Sample origin was included initially in the model but then excluded from the main model because it did not contribute significantly after adjustment for the other variables (P = 0.7484; P = 0.91 after manual restructuring of its levels). In a final modeling step, method of sample retrieval was excluded from the model because of a P value of 0.467. 7 J Rüschoff et al Supplementary Table S2. Assessment of HER2-positivity by immunohistochemistry and in situ hybridization HER2-positivity assessment Immunohistochemistry 3+ Immunohistochemistry 2+ and in situ n (%) 1,740 (79.1) 313 (14.2) hybridization-positive in situ hybridization-positive only 111 (5.0) Borderline in situ hybridization-positive 12 (0.5) N/A 24 (1.1) Total 2,200 (100.0) N/A, supportive information not available. 8 J Rüschoff et al Supplementary Table S3. Assessment of individual study centers based on estimated center effect size and P values Center N 28 206 Center Cov adj. Cov + B-Holm effect % P value adj. P value 12.4% 1.31e–07 6.28e–06 Assessment P values Cov + B-Holm adj. P value < 0.05 32 640 –5.8% 1.74e–05 0.0008 Cov + B-Holm adj. P value < 0.05 4 276 6.5% 4.19e–04 0.0193 Cov + B-Holm adj. P value < 0.05 17 172 8.0% 0.0020 0.0920 Cov + B-Holm adj. P value < 0.2 1 741 –3.7% 0.0023 0.1021 Cov + B-Holm adj. P value < 0.2 35 661 3.5% 0.0027 0.1177 Cov + B-Holm adj. P value < 0.2 14 712 3.9% 0.0051 0.2158 Cov adj. P value < 0.05 23 132 –8.2% 0.0067 0.2750 Cov adj. P value < 0.05 27 316 –5.0% 0.0100 0.4013 Cov adj. P value < 0.05 9 J Rüschoff et al 7 352 4.8% 0.0122 0.4748 Cov adj. P value < 0.05 60 126 –5.9% 0.0356 1.0 Cov adj. P value < 0.05 41 332 –3.4% 0.0458 1.0 Cov adj. P value < 0.05 2 171 5.4% 0.0485 1.0 Cov adj. P value < 0.05 11 401 –3.1% 0.0502 1.0 Cov adj. P value < 0.2 55 67 –8.5% 0.0519 1.0 Cov adj. P value < 0.2 50 113 6.5% 0.0602 1.0 Cov adj. P value < 0.2 30 125 4.7% 0.0619 1.0 Cov adj. P value < 0.2 15 306 3.3% 0.0853 1.0 Cov adj. P value < 0.2 58 313 –3.2% 0.0861 1.0 Cov adj. P value < 0.2 6 580 2.2% 0.0926 1.0 Cov adj. P value < 0.2 18 317 –2.9% 0.1176 1.0 Cov adj. P value < 0.2 5 90 –4.7% 0.1588 1.0 Cov adj. P value < 0.2 10 381 –2.2% 0.1843 1.0 Cov adj. P value < 0.2 37 33 8.9% 0.1891 1.0 Cov adj. P value < 0.2 31 63 5.5% 0.2347 1.0 Inconspicuous 46 10 J Rüschoff et al Cov adj., covariate-adjusted; Cov + B-Holm adj., covariate- and Bonferroni-Holmadjusted. We considered a Bonferroni-Holm-adjusted P value < 0.05 as the level 1 signal of a conspicuous unexplained center effect, which occurred for the first three centers in the above list; for these centers, this P value signal concurred with a relatively high estimated center effect. A Bonferroni-Holm-adjusted P value < 0.2 is still considered a signal of a conspicuous unexplained center effect, which seems to be confirmed by the relatively large center effect of Center 17, but not so much by the moderate center effects of Centers 1 and 35 where the P value signal seems to be mainly related to the high numbers of patients at these two centers. Two weaker P valuebased signals (covariate-adjusted P value [not adjusted for multiplicity] < 0.05 and < 0.2) were considered in addition, but here additional considerations are needed for the assessment of the center effect. Centers 14 and 23 were identified as not statistically significant after Bonferroni-Holm adjustment, but with a 99% confidence intervals of the documented HER2-positivity rate that does not include the predicted rate of these centers. The remaining 23 centers in general had relatively small center effects combined with covariate-adjusted P values > 0.2 (and Bonferroni-Holmadjusted P values = 1); Center 31 was the only center effect of 5.5% ≥ 5%. All these centers were considered as inconspicuous. For a final assessment, the results of both the descriptive and the model-based approach should be considered. 11 J Rüschoff et al Supplementary Table S4. Likelihood ratio test of the effect of covariates (including center effect) on HER2-positivity and covariate importance in predicting HER2positivity Variable Likelihood ratio test of Covariate importance in effect of independent predicting HER2-positivity variables on HER2-positivitya Main effecta Total effectb 1.85e–79 0.335 0.394 1.16e–47 0.226 0.249 4.17e–16 0.159 0.177 9.15e–27 0.116 0.130 Age 2.86e–17 0.071 0.078 Nodal status 7.067e–6 0.042 0.044 (P value) Histologic grading Hormone receptor status Center effect Histologic subtype (lobular versus ductal) a The main effect reflects the relative contribution of that factor alone. b The total effect reflects the relative contribution of that factor both alone and in combination with other factors. The center effect ranks third in variable importance both in main and total effect; it ranks at fifth place only based on the lowness of the P value because of its relatively high number of degrees of freedom (df = 48). 12 J Rüschoff et al Supplementary Table S5. Predicted HER2-positivity rates for covariate factor levels ER, estrogen receptor; HER2, human epidermal growth factor receptor 2; HR, hormone receptor; PgR, progesterone receptor. 13 J Rüschoff et al Supplementary appendix The following investigators belong to the NIU HER2 Study Group and provided information on samples, along with clinical and diagnostic data for the analyses: Ambrosius C-A. Institut für Pathologie Hilden, Hilden; Arndt N. Städtisches Klinikum Dessau, Dessau; Baldus SE. Institut für Pathologie, Zytologie & Molekularpathologie, Bergisch-Gladbach; Berger I. Institut für Pathologie Klinikum Kassel, Kassel; Berghäuser K-H. Institut für Pathologie Thüringer Kliniken, Saalfeld; Bittmann I. Medizinisches Versorgungszentrum am Diakoniekrankenhaus, Rotenburg; Blasius S. Institut für Pathologie am Klinikum Hanau, Hanau; Bollmann R. Institut für Pathologie, Bonn; Brockmann M. and Schildgen O. Klinik der Stadt Köln, gGmbH, Cologne; Bürrig K-F. Pathologie Hildesheim, Hildesheim; Chmelar C. Pathologisches Institut Recklinghausen, Recklinghausen; Decker T. Institut für Pathologie DietrichBonhoeffer-Klinikum, Neubrandenburg; Diergardt U. Pathologisches Institut am AKH, Hagen; Dykgers A. Pathologisches Institut Dortmund, Dortmund; Engers R. Zentrum für Pathologie Neuss & Pathologische Gemeinschaft Engers und Donner, Neuss; Falk S. Gemeinschaftspraxis für Pathologie Frankfurt am Main, Frankfurt am Main; Friedrich K. Institut für Pathologie des Universitätsklinikums Carl Gustav Carus an der Technischen Universität, Dresden; Gabriele D. Institut für Pathologie Kreiskliniken Reutlingen GmbH, Reutlingen; Gaiser T. Pathologisches Institut Universitätsklinikum Mannheim, Mannheim; Gassel AM. Institut für Pathologie Evangelisches Krankenhaus Oberhausen, Oberhausen; Gerharz C-D. Institut für Pathologie Evangelisches Krankenhaus Bethesda, Duisburg; Haedicke W. Institut für Pathologie am Evangelischen Waldkrankenhaus, Berlin-Spandau; Haroske G. Krankenhaus Dresden Friedrichstadt & Städtisches Klinikum, Friedrichstadt; Hartmann A. Pathologisches Institut Universitätsklinikum Erlangen, Erlangen; 14 J Rüschoff et al Hellwig K. Universitätsklinikum Magdeburg GmbH, Magdeburg; Hinze R. Institut für Pathologie HELIOS Kliniken Schwerin, Schwerin; Junker K. Institut für Pathologie Klinikum Bremen Mitte, Bremen; Kasper U. Gemeinschaftspraxis im Medical-Center am Clemens Hospital, Münster; Klauschen F. Charité Universitätsmedizin Berlin, Berlin; Klosterhalfen B. Institut und Praxisgemeinschaft für Pathologie Krankenhaus Düren, Dueren; Knöß P. Institut für Pathologie Bad Kreuznach, Bad Kreuznach; Kreipe H. Medizinische Hochschule Hannover, Hannover; Kristiansen G. Institut für Pathologie Universitätsklinikum Bonn, Bonn; Krüger S. Klinikum Magdeburg GmbH Abt. Pathologie, Magdeburg; Kunze A. Gemeinschaftspraxis für Pathologie Bad Berka, Bad Berka; Lüders P. Gemeinschaftspraxis für Pathologie Stendal, Stendal; Neuber A. Institut für Pathologie, Dermatohistologie, Zytologie und Molekularpathologie, Wesel; Niemann P. Gemeinschaftspraxis für Pathologie, Hamm; Niendorf A. Pathologie Hamburg-West, Hamburg; Orth C. Institut für Pathologie SRH Waldklinikum, Gera; Ott G. Robert Bosch Krankenhaus, Stuttgart; Philippou S. Institut für Pathologie und Zytologie an der Augusta-Kranken-Anstalt, Bochum; Richter-Sadocco B. Institut für Pathologie Hannover Zentrum, Hannover; Röcken C. Institut für Pathologie Universitätsklinikum Schleswig-Holstein, SchleswigHolstein; Rüschoff J. Institut für Pathologie Nordhessen, Kassel; Sauter G. Universitätsklinikum Hamburg-Eppendorf, Hamburg-Eppendorf; Schmitz J. Institut für Pathologie Grevenbroich, Grevenbroich; Sen Gupta R. St. Agnes-Hospital Bocholt-Rhede, Bocholt; Sinn P. Pathologisches Institut Universitätsklinikum Heidelberg, Heidelberg; Staebler A. Institut für Pathologie und Neuropathologie Universitätsklinikum Tübingen, Tübingen; Tennstedt-Schenk C. Gemeinschaftspraxis für Pathologie Mühlhausen, Mühlhausen; Thorns C. Institut für Pathologie Universitätsklinikum Lübeck, Lübeck; Tiemann K. Institut für Hämatopathologie, 15 J Rüschoff et al Hamburg; Turzynski A. Pathoplan GbR, Lübeck; Vollmer E. Medizinisches Versorgungszentrum Borstel, Borstel; Weimann R. Institut für Pathologie Klinikum Saarbrücken, Saarbrücken; Wittersheim M. Institut für Pathologie Universitätsklinikum Cologne, Cologne. 16