Download How Do Statisticians Perceive Statistics Journals?

General
How Do Statisticians Perceive Statistics Journals?
Vasilis THEOHARAKIS and Mary SKORDIA
Since researchers and academic institutions are increasingly
evaluated based on their publication record in peer reviewed
journals, it is crucial to assess how the statistics community
perceives statistics journals. This study presents four subjective
quality metrics of statistics journals as expressed by different
segments of statisticians. Based on a worldwide sample of 2,190
statisticians, our Ž ndings indicate that the research interest and
geographic origin of the researcher have a signiŽ cant impact on
journal perceptions, which are highly correlated with a journal’s
total number of citations.
KEY WORDS:
Journal rankings; Statistics research.
1.
INTRODUCTION
The recognition and development of an academic institution
depends heavily on its faculty’s publication record in prestigious journals (Lane, Ray, and Glennon 1990). As a result, an
increased emphasis is placed on publishing in refereed journals and promotion criteria rest heavily on the faculty’s publication record (Gibbons 1990). In fact, not only is the publication record one of the criteria for selecting Fellows at the
American Statistical Association (Bailar 1988), but it is also
used to measure the productivity of countries and institutions
for their contributions to statistics (Genest 1997). Genest measured institution and country research productivity based on the
number of articles, number of authors, and page counts in 16
international journals publishing in statistical theory. Since he
believed the selection of these journals to be “subjective and
far from comprehensive,” a study that systematically identiŽ es
the relevant journals would facilitate such studies. The need for
identifying relevant journals was also demonstrated by Baltagi
(1999) in his article on the ranking of individuals and institutions in applied econometrics. To demonstrate impact, Baltagi
used page counts and citations of relevant articles from 15 journals, but could not control for journal quality since no journal quality measure was available. Although citation reports do
provide an aggregate measure of a journal’s impact, the perVasilis Theoharakis is Assistant Professor (E-mail: [email protected]),
and Mary Skordia is Research Associate, Athens Laboratory of Business Administration (ALBA), Athinas & Areos 2A, Vouliagmeni 166 71, Athens, Greece.
This article would not have been in place without the support of colleagues who
spent many hours in providing direct input and feedback. The authors thank the
editor and associate editor for their very constructive comments that signiŽ cantly
improved this article.
®
c 2003 American Statistical Association DOI: 10.1198/0003130031414
ceptions of statisticians with different research interests may
vary. In fact, in the UK where funds to universities are disbursed
based on the Research Assessment Exercise (RAE), impact factors or citation indices are not used to assess research output in
journals. Instead, the assessment of the RAE panel for statistics is based on the “perceived editorial standards of journals”
(http://www.hero.ac.uk/rae/criteria/crit24.htm).
Despite the fact that the assessment of journals is a crucial issue for the research community, it is a surprise that the statistics
community’s perceptions have not been systematically examined. We therefore pose the following questions: What are the
most popular journals in the Ž eld of statistics? Since promotion
decisions frequently depend on the number of publications in
top tier journals, how do statisticians classify journals in tiers?
Besides one’s perception about a journal’s standing, how useful
do researchers Ž nd a particular journal? Do statisticians from
different research or geographic areas or with a different type
of employment value journals differently? How do the subjective perceptions of journal quality relate to the more objective
journal citation measures? By addressing these questions, this
study seeks to assist: (1) authors in their search for a research
outlet, (2) departments in promotion and tenure decisions, and
(3) journal editors, by providing them a view of their journal’s
standing. We should note that while we examine the perceptual journal rankings, there is a substantial overlap in the quality
of individual articles that appear in journals of vastly different
reputation.
2.
SURVEY INSTRUMENT AND METHODOLOGY
Since we sought to examine journal perceptions over a broad
sample, we located four publicly available membership directories of statisticians (American Statistical Association, Institute of MathematicalStatistics, InternationalStatisticalInstitute,
and an online listing of UK-based academic statisticians, found
at http://www.swan.ac.uk/statistics/das/). Due to the pervasive
use of the Internet among statisticians, we developed an online
survey. Our questionnaire requested from participants to place
statistics journals in rank order and at the same time provide
demographic information. The demographic variables were selected in order to be used as segmentation variables that could
provide answers to the questions raised earlier. Therefore, participants were asked to rank up to ten statistics journals that they
considered as top tier (most rigorous, prestigious, and important) and up to ten additional journals that they considered as
second tier. In addition, respondents were asked to list up to ten
journals that they considered to be most useful in their work. A
list of 110 statistics journals was available on pull-down menus
(Appendix), but respondents could also Ž ll in any other journal
The American Statistician, May 2003, Vol. 57, No. 2
115
follows:
Table 1. Respondents’ ProŽ le
Highest academic degree
Doctorate
Masters
Bachelor’s
Other
No answer
TOTAL
1734
355
40
24
37
2190
Type of employment
Faculty member
Government employee
Researcher/clinician at a health/medical facility
Manufacturing industry employee
Private consultant
Service industry employee
Retired
Actuary
Other
No answer
TOTAL
1234
185
152
121
101
43
32
3
179
140
2190
Geographical location
North America
Europe
Asia
Latin America
Australia/New Zealand
Africa
No answer
TOTAL
1495
412
149
57
37
23
17
2190
title they wished. From the directories identiŽ ed, we collected
the E-mail addresses of 12,053 statisticians and proceeded by
sending an E-mail invitation to them for completing our online
questionnaire (the questionnaire and full set of tables are available at www.alba.edu.gr/survey). The survey was pretested on
a sample of 30 statisticians and minor alternations were made.
Two weeks after the initial E-mail invitation,an E-mail reminder
was sent to individuals who had not responded.
In total, we received 2,190 usable responses (521 from the second wave) with a usable response rate of 18.2%. No signiŽ cant
differences in the ranking of journals were found between Ž rst
and second wave respondents, that is, those who responded to
the reminder E-mail, which may indicate that our sample does
not suffer from nonresponse bias. However, statisticians that
do not believe in ranking journals, may have not participated.
Nearly two-thirds of our respondents are from North America,
more than half are faculty members, and nearly 80% of our respondents hold a doctorate (Table 1). Sixty seven percent of our
respondents replied that their institution uses the number and/or
character of journal publications for personnel decisions.
2.1 Measures of Perceived Quality
Previous studies on the ranking of journals in other disciplines
have reported Familiarity and Average Rank Position as measures of perceived quality (Luke and Doke 1987; Hult, Neese,
and Bashaw 1997). We measure the Familiarity of a journal by
the percentage of respondents who placed the journal among
their top 20 (%Top20), whereas Average Rank Position (ARP)
refers to the mean of the ranking positions given by respondents who chose to rank the particular journal and is deŽ ned as
116
General
ARPi =
P20
j= 1
P20
Rij ¤ j
j= 1 R ij
(1 µ ARPi µ 20);
(1)
where i denotes the journal and Rij is the number of times journal i has been ranked in the jth position. Thus, a lower ARP
denotes a higher perceived journal importance. In addition, we
report the percentage of respondents who included the journal
in their top ten with respect to the total number of respondents
(%Top10) and journal Usefulness that corresponds to the percentage of respondents who listed the journal among the ones
most useful in their work. But one has to be careful when ranking
journals on any single measure of perceived quality. For example, if journal A is ranked by 100 respondents who all place it in
the 1st Ž rst position and journal B is ranked by 101 respondents
who all place it in the 20th position, then journal A would be
ranked lower if journals were ranked based on Familiarity. In
order to minimize such problems, we considered multiple quality measures when performing the ranking of journals, by using
a weighted Index of familiarity and rank (Theoharakis and Hirst
2002) that is deŽ ned as follows:
P20
j= 1 R ij ¤ (21 ¡ j)
Indexi = 100 ¤
20 ¤ n
21 ¡ ARPi
¤ %Top20i
= 100 ¤
20
(0 µ Indexi µ 100);
(2)
where i denotes the journal and Rij is the number of times
the journal i has been ranked in the jth position and n is the
number of respondents in the sample. Thus, the Index assigns
to the jth position a decreasing weight of (21 ¡ j)=20, with the
Ž rst rank position carrying a weight of 20/20 and the last (20th)
position a weight of 1/20. We also extend the original Index
by Theoharakis and Hirst (2002) to indicate its connection with
ARP and %Top20. As we list the journals based on this Index, we
present each individual measure and suggest that readers should
examine each journal individually across the metrics presented.
3.
RESULTS
We present journals based on the weighted Index of familiarity and rank for our worldwide sample and the two largest
regional samples (Table 2). Although the correlations between
our perceptual metrics (%Top10, %Top20, ARP, and Usefulness) are high (Table 3), the correlations of each one of these
metrics with our Index are even higher; the only exception is
the correlation of ARP with %Top10 that is about the same with
the correlation of ARP with Index (this is not a surprise since
%Top10 depends on rank position). This indicates that our Index
is indeed a representative measure to conduct our ranking.
The Journal of the American Statistical Association (JASA),
Biometrika (Bka), The Annals of Statistics (AoS), the Journal
of the Royal Statistical Society, Series B (JRSB), and Biometrics (Bcs) are highly perceived across all quality metrics by
our worldwide and regional samples. These journals are closely
followed by Technometrics (Tech), the Journal of the Royal Statistical Society, Series A (JRSA), Statistics in Medicine (SMed),
the Annals of Probability (AoP), and The American Statistician
The American Statistician, May 2003, Vol. 57, No. 2
117
JASA
Bka
AoS
JRSB
Bcs
Tech
AmSt
JRSA
SMed
AoP
ApSt
SSci
Ecnt
JMA
CSTM
CJS
JSPI
SJS
AnAP
Ssin
JCGS
Bsts
ISR
Bern
AJE
Psyc
SPL
AISM
Chnc
AAP
CSDA
SnkA
JAS
JAP
TStt
PTRF
JQT
JTSA
JEcn
CSSC
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
4.84
4.94
5.61
5.12
5.62
5.99
5.98
6.12
6.12
6.12
6.22
6.30
6.68
6.53
6.70
7.34
6.85
7.57
8.01
7.98
8.21
9.61
10.54
10.00
10.92
12.32
11.01
13.70
18.98
17.13
20.47
24.22
20.57
28.07
42.00
29.79
50.46
56.61
51.74
74.00
2.79
5.07
5.30
4.25
6.39
5.98
4.75
5.43
4.57
5.80
5.80
3.61
5.16
5.71
6.80
6.71
5.75
6.62
6.80
5.98
7.49
10.00
8.49
8.45
6.30
11.74
7.95
14.98
18.04
18.58
21.96
26.71
21.42
23.56
45.57
32.37
59.91
65.48
58.22
80.73
11.51
8.77
9.32
11.64
9.54
11.14
13.70
12.83
13.84
13.11
11.19
13.61
15.98
12.65
13.42
13.74
12.28
16.03
15.89
15.21
17.40
15.66
22.97
22.69
25.84
25.07
23.33
22.01
32.37
29.32
29.68
38.77
32.42
47.72
58.77
47.58
64.84
72.69
64.52
85.30
4.21
5.04
8.05
5.30
4.52
4.94
6.65
5.51
7.64
2.55
4.47
7.06
7.74
4.26
4.47
4.16
10.29
6.18
8.05
9.30
4.99
6.44
8.73
5.61
4.73
8.52
9.61
7.22
15.22
17.97
7.64
8.88
26.91
33.92
38.13
20.31
32.00
40.57
26.70
58.65
Rank Journal INDEX % Top10 % Top20 Usefulness
Worldwide (n = 2,190)
12.60
9.72
8.97
12.20
9.22
10.25
12.27
11.47
12.15
11.66
9.88
11.74
12.64
10.67
11.01
10.32
9.84
11.56
10.92
10.51
11.57
8.73
11.82
12.18
12.55
11.17
11.56
8.55
9.28
9.31
7.21
8.51
8.31
9.23
6.71
8.48
5.44
5.43
4.96
3.65
ARP
TStt
JEcn
SPL
JAP
Bern
JBES
AAP
CCT
CSSC
AISM
CSDA
ISR
JAS
SnkA
JQT
Psyc
SJS
AJE
Ssin
Chnc
JCGS
AnAP
JSPI
Bsts
JMA
CJS
CSTM
Ecnt
SSci
ApSt
AoP
JRSA
SMed
AmSt
Bcs
Tech
JRSB
Bka
AoS
JASA
4.36
4.37
4.61
4.41
4.67
4.90
4.89
4.93
5.37
5.32
5.41
5.49
6.10
5.82
6.84
7.16
6.94
7.28
8.44
7.86
8.50
8.58
8.88
8.64
9.98
11.07
10.74
12.22
17.29
16.88
18.04
23.40
21.80
31.59
43.19
31.70
46.42
54.59
48.21
76.64
3.55
4.41
2.94
4.41
4.15
4.62
4.55
4.48
2.94
4.62
3.88
4.48
5.55
5.69
6.56
7.36
5.48
6.09
7.96
4.55
7.36
8.83
7.22
6.29
9.36
6.69
7.36
13.38
19.00
15.99
19.20
26.22
22.88
26.89
47.42
34.92
55.99
63.48
54.72
82.68
10.17
7.69
12.31
8.36
9.23
10.17
8.70
9.16
12.71
10.03
12.37
12.17
12.37
12.51
10.90
14.25
16.52
12.91
17.26
16.32
16.72
13.71
19.20
16.19
20.54
25.69
23.08
19.60
29.10
29.03
26.29
36.79
33.85
52.31
58.60
48.96
60.00
69.70
60.27
86.56
3.99
4.37
4.14
2.92
2.38
5.45
2.69
9.21
4.68
2.76
6.14
4.14
4.53
2.30
9.67
4.45
3.84
11.28
5.30
8.44
8.67
5.07
7.14
10.44
6.52
4.60
8.75
5.37
17.88
12.89
5.91
6.75
29.32
38.53
40.45
21.49
27.78
38.99
23.33
61.17
Journal INDEX % Top10 % Top20 Usefulness
North America (n = 1,495)
12.43
9.63
13.51
10.45
10.88
11.36
9.76
10.23
12.55
10.40
12.25
11.98
11.14
11.69
8.45
10.95
12.60
9.72
11.22
11.37
10.84
8.49
11.75
10.33
11.28
12.38
11.69
8.53
9.11
9.37
7.28
8.28
8.12
8.92
6.26
8.05
5.53
5.33
5.00
3.29
ARP
Table 2. Journal Rankings Based on Geographical Location
SC
SnkA
Ssin
AJE
SMMR
JEcn
JAS
Psyc
Bsts
AISM
JTSA
JCGS
CSTM
CSDA
JAP
AAP
SPA
PTRF
AnAP
CJS
SPL
TStt
JSPI
ISR
Bern
JMA
Ecnt
SSci
SJS
AmSt
SMed
AoP
Tech
ApSt
Bcs
JRSA
AoS
JRSB
Bka
JASA
Journal
4.85
4.94
5.18
5.18
5.32
5.92
5.72
6.00
6.53
6.21
7.00
7.32
8.14
7.77
8.40
8.47
8.42
9.83
10.47
9.84
10.69
12.34
13.13
12.86
14.67
16.44
15.39
17.26
19.24
18.03
20.39
23.85
22.99
25.81
36.83
29.50
55.95
60.28
58.82
65.22
4.13
3.64
4.13
4.85
4.85
6.80
5.10
6.07
5.10
4.85
4.85
5.58
5.58
5.34
8.25
7.52
8.98
11.65
11.17
5.34
8.98
9.22
9.95
12.86
14.32
15.05
16.75
18.20
17.96
14.56
20.87
26.21
23.54
26.46
38.59
32.28
61.17
68.20
66.99
73.06
10.68
11.41
12.38
8.74
9.22
9.95
12.62
11.41
13.59
14.56
16.26
15.05
17.23
17.96
15.53
15.78
13.35
15.53
17.96
24.03
23.30
26.70
27.91
24.03
25.49
32.52
24.51
29.37
40.29
33.74
33.50
34.47
41.26
40.53
54.13
47.82
70.15
75.97
76.21
78.88
6.02
1.83
1.83
8.38
6.02
5.76
6.81
4.71
7.59
4.19
7.59
6.54
7.07
9.42
7.59
8.38
9.16
8.12
7.85
4.71
12.83
13.09
10.47
10.21
8.90
10.21
10.47
14.40
10.99
20.42
24.08
11.26
14.14
22.77
32.20
16.23
31.15
41.88
38.74
51.83
11.91
12.34
12.63
9.14
9.47
9.10
11.94
10.49
11.39
12.47
12.39
11.27
11.55
12.35
10.19
10.26
8.38
8.34
9.34
12.81
11.82
11.75
11.59
10.29
9.49
10.89
8.45
9.25
11.45
10.31
8.83
7.16
9.86
8.26
7.39
8.66
5.05
5.13
5.56
4.46
INDEX % Top10 % Top20 Usefulness ARP
Europe (n = 412)
Table 3. Perceived Measure Correlations
INDEX
%Top10
%Top20
Usefulness
ARP
%Top10
%Top20
0.966
0.936
¡0.802
0.941
¡0.784
0.996
0.982
0.951
¡0.799
Usefulness
¡0.770
(AmSt) which is the fourth most useful journal worldwide. In
order to have some idea if a difference in the Index values between journals is signiŽ cant, we calculated the standard errors
for each journal’s Index. For our worldwide sample, the average
standard error for journals with an Index greater than 13 ranges
from .61 to .89 (mean of .75) while for those with an Index less
than 13 ranges from .32 to .53 (mean of .42).
Some signiŽ cant differences in journal rankings appear between North American and European statisticians. On one hand,
North American statisticians rank higher the Journal of Computational and Graphical Statistics (JCGS), Communications
in Statistics, Part A–Theory and Methods (CSTM), Canadian
Journal of Statistics (CJS), Biostatistics (Bsts), Psychometrica
(Psyc), Statistica Sinica (Ssin), the American Journal of Epidemiology (AJE), and Sankhya, Series A (SnkA). On the other
hand, Europeans perceive highly the Scandinavian Journal of
Statistics (SJS), Bernoulli (Bern), the International Statistical
Review (ISR), The Statistician (TStt), Statistics and Probability
Letters (SPL), Probability Theory and Related Fields (PTRF),
Advances in Applied Probability (AAP), Stochastic Processes
and their Applications (SPA), Journal of Applied Probability
(JAP), and Journal of Time Series Analysis (JTSA). These differences in ranking can be explained either by a geographical bias,
that is, Americans favor American journals and Europeans favor
European journals, or by differences in the research interests of
the two populationsin our sample. A higher percentage of European researchers are interested in mathematical statistics, Bayes
methods, probability, and stochastic processes, with respect to
the corresponding percentage of American researchers. Thus,
European researchers tend to favor journals related to
Table 4. Journal Rankings Based on Employment Type
Academics with
PhD (n = 1149)
Remaining statisticians
(n = 856)
Government (n = 185)
Rank
Journal
INDEX
% Top10
Journal
INDEX
% Top10
Journal
INDEX
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
JASA
Bka
AoS
JRSB
Bcs
Tech
AoP
JRSA
AmSt
SSci
ApSt
Ecnt
SMed
JMA
JSPI
SJS
CJS
CSTM
AnAP
Bern
Ssin
SPL
JCGS
PTRF
JAP
AAP
AISM
Bsts
ISR
Psyc
JTSA
JEcn
SnkA
CSDA
SPA
JBES
CSSC
AJE
JAS
TStt
77.16
64.88
64.14
60.21
42.05
30.30
26.24
23.74
22.25
20.54
19.33
18.52
17.21
17.18
14.74
13.92
13.91
13.55
13.12
10.84
10.27
9.95
9.77
9.33
8.80
8.60
8.58
8.48
8.09
7.39
7.25
7.15
7.01
6.98
6.89
5.91
5.40
5.28
5.13
4.94
84.77
74.85
72.67
71.37
44.56
32.72
28.55
25.33
17.49
22.37
18.19
20.63
17.06
16.10
11.58
11.84
7.57
9.57
14.10
10.10
9.31
8.18
8.18
10.70
9.14
8.01
7.48
6.61
6.53
7.22
5.74
7.22
6.70
4.79
7.40
5.48
2.87
3.74
3.92
3.74
JASA
Bka
JRSB
Bcs
AoS
AmSt
JRSA
SMed
Tech
ApSt
JOS
SSci
SrvM
AoP
Ecnt
ISR
Chnc
CSTM
AJE
CJS
Bsts
JBES
JAS
JSPI
CSDA
TStt
Ssin
SJS
AnAP
CSSC
JMA
JCGS
JEcn
Psyc
SMMR
IJE
JABE
SnkB
AoE
CCT
77.86
51.68
42.30
39.38
37.73
36.89
30.08
20.27
20.00
19.95
16.43
15.27
14.27
11.89
11.84
11.54
9.89
9.89
9.84
9.73
9.35
7.46
6.84
5.76
5.70
5.57
5.46
5.30
5.24
4.92
4.86
4.59
4.46
4.43
3.92
3.76
3.76
3.70
3.68
3.68
84.32
60.00
51.89
45.41
42.16
32.43
35.68
21.62
21.62
16.76
15.68
15.68
14.59
12.43
11.89
10.81
4.86
5.95
9.73
7.03
7.03
8.65
7.57
3.78
4.86
4.32
4.32
3.78
4.86
3.24
4.32
4.32
4.86
3.78
3.78
3.78
2.70
3.24
2.70
3.24
JASA
Bka
Bcs
JRSB
AoS
AmSt
Tech
SMed
JRSA
ApSt
AoP
SSci
JQT
AJE
Chnc
CSTM
Ecnt
TStt
JMA
JAS
CJS
Bsts
JCGS
Psyc
Ssin
ISR
JSPI
CCT
AnAP
SJS
SnkA
CSDA
JBS
AISM
SMMR
AAP
CSSC
BJnl
SnkB
QE
68.94
46.57
42.51
39.13
38.11
33.98
31.21
25.13
23.59
18.30
14.58
12.97
8.50
8.32
8.30
7.84
7.64
7.46
7.39
7.28
7.16
7.01
6.38
6.27
6.02
6.02
5.94
5.86
5.83
5.77
5.47
5.06
4.94
4.60
4.25
4.10
4.06
3.95
3.93
3.77
118
General
% Top10
74.53
54.09
46.96
46.26
42.29
29.79
34.23
27.22
26.64
18.11
15.19
14.14
8.29
7.59
5.49
6.19
8.06
6.19
7.48
7.01
4.44
4.91
5.49
6.89
5.72
5.84
5.37
5.26
5.61
4.91
5.26
4.21
4.32
3.97
3.74
3.86
2.57
2.57
4.09
3.62
Table 5. Journal Rankings of Ph.D. Academics Based on Research Area
Biometrics/biostatistics
(n = 207)
Remaining Ph.D.
academics (n = 880)
Econometrics (n = 62)
Rank
Journal
INDEX
% Top10
Journal
INDEX
% Top10
Journal
INDEX
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
JASA
Bcs
Bka
JRSB
AoS
SMed
Tech
JRSA
AmSt
ApSt
SSci
Bsts
SJS
CSTM
AJE
CJS
Ecnt
JSPI
AoP
JCGS
CCT
Ssin
CSDA
JMA
ISR
SMMR
LDA
CSSC
JAS
BJnl
Psyc
SPL
JSCS
TStt
JABE
Envr
AnAP
ANZJ
Chnc
IJE
85.22
75.14
74.52
68.02
51.43
48.74
31.74
31.30
26.98
26.76
25.68
25.14
14.64
14.44
14.37
14.08
13.45
11.79
10.53
10.10
9.66
9.42
9.35
9.06
8.96
6.86
6.71
6.69
6.28
6.16
5.31
4.93
4.78
4.78
4.23
3.94
3.67
3.67
3.26
3.07
95.65
85.99
86.47
83.09
61.35
52.17
34.30
34.30
21.74
27.05
30.43
22.22
13.04
8.21
10.14
9.18
14.01
8.70
9.66
7.73
8.21
7.73
7.73
6.76
7.25
4.83
5.31
1.93
5.80
1.45
3.86
3.86
3.86
3.38
3.38
2.90
2.90
1.45
0.97
1.93
JASA
Ecnt
AoS
Bka
JEcn
JRSB
JBES
ET
JRSA
RES
JTSA
AoP
JAE
AmSt
Tech
Bcs
JMA
ER
CSTM
ApSt
IJF
SJS
SSci
JFor
SPL
CJS
Psyc
SnkA
EcxJ
ISR
SPA
JSPI
CSSC
SnkB
AISM
CSDA
JNS
JAP
SN
JCGS
84.11
70.73
69.44
65.89
58.55
54.92
39.44
37.90
27.82
24.52
22.74
22.26
20.73
19.68
18.39
16.21
14.44
14.11
11.45
9.44
8.31
8.06
7.82
6.85
6.29
6.21
5.97
5.40
5.16
5.16
5.08
4.84
4.76
4.44
4.19
4.19
4.19
3.39
3.39
3.23
91.94
82.26
77.42
79.03
64.52
69.35
40.32
40.32
33.87
24.19
19.35
25.81
19.35
12.90
20.97
9.68
11.29
9.68
4.84
1.61
6.45
8.06
8.06
4.84
4.84
1.61
6.45
4.84
0.00
3.23
3.23
3.23
4.84
4.84
3.23
3.23
1.61
3.23
3.23
3.23
JASA
AoS
Bka
JRSB
Bcs
Tech
AoP
AmSt
JMA
JRSA
CJS
SSci
JSPI
SJS
ApSt
CSTM
SPL
Ecnt
AnAP
Bern
Ssin
SMed
JCGS
AISM
PTRF
JAP
AAP
ISR
SnkA
JTSA
CSDA
Psyc
SPA
SnkB
CSSC
TStt
JAS
JNS
ANZJ
Chnc
91.64
84.84
84.48
78.87
60.02
52.86
44.63
42.60
39.97
39.38
37.23
37.23
35.68
34.72
34.48
29.71
28.52
27.80
26.97
24.94
24.70
23.51
21.36
21.00
20.88
19.93
19.45
19.21
18.85
18.73
16.59
16.59
15.63
14.68
13.60
13.24
12.89
12.89
12.53
11.69
Mathematics, Probability and Stochastic Processes (e.g., Bern,
SPL, PTRF, SPA, JAP).
A close examination of these rankings reveals that although
some journals are ranked lower they appear to have relatively
good conditional rank (low ARP). This suggests that a subset
of respondents hold a very different view than the whole group.
In particular, this behavior appears to arise in the case of probability and econometrics journals, which is consistent with previous Ž ndings where a signiŽ cant cultural difference was found
between probabilists and statisticians (Genest 1999). This observation prompts us to examine segments of our sample with
research areas that might favor probability and econometrics
journals.
3.1 Journal Rankings Based on Employment
Besides differences in perspective due to the geographical location of respondents, we examine the ranking of the two largest
groups based on the type of employment. In particular, we focus
on academics with a doctorate and government employees (Ta-
% Top10
81.70
75.00
71.82
68.75
37.27
33.18
33.18
16.82
18.64
22.61
7.61
21.48
12.84
11.82
17.27
10.23
9.43
17.84
17.61
12.73
10.23
9.89
8.64
9.09
13.86
11.48
9.77
6.59
7.95
5.91
4.20
8.07
9.43
3.98
2.95
4.09
3.75
2.84
2.50
2.39
ble 4). Since academics with a Ph.D. constitute more than half
of our worldwide sample, their rankings demonstrate similarities with our worldwide sample. Government employees on the
other hand, demonstrate a higher appreciation towards the Journal of OfŽ cial Statistics (JOS) and Survey Methodology(SrvM).
This is not as a surprise since the largest portion of statisticians
interested in ofŽ cial statistics and survey methodology are government employees.
3.2 Journal Rankings Based on Research Area
The previous rankings based on geography and type of employment indicated that the underlying research areas of respondents did in uence the overall ranking. We therefore proceed by
examining the ranking of two subgroups of Ph.D. academics
based on their self-reported research areas (Table 5). Ph.D. academiciansinterested in Biometrics/Biostatisticsrank higherBiometrics (Bcs), Biometrika (Bka), Statistics in Medicine (SMed),
Biostatistics (Bsts), American Journal of Epidemiology (AJE),
and Controlled Clinical Trials (CCT). Moreover, researchers in
this area Ž nd Biometrics more useful to their work than the
The American Statistician, May 2003, Vol. 57, No. 2
119
Table 6. Journal Rankings Based on Research Area (all respondents)
Applications of statistics
(n = 273)
Mathematical statistics
(n = 169)
Bayes methods (n = 93)
Rank
Journal
INDEX
% Top10
Journal
INDEX
% Top10
Journal
INDEX
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
JASA
Bka
JRSB
AoS
Bcs
Tech
AmSt
JRSA
ApSt
SSci
SMed
AoP
Ecnt
CSTM
JQT
Psyc
JAS
Chnc
TStt
AnAP
JSPI
JMA
CJS
JCGS
ISR
CSDA
Bsts
SJS
SnkA
Ssin
AAP
CSSC
AISM
BJnl
SnkB
AJE
SPL
JAP
QE
SMMR
74.93
52.38
45.66
40.59
38.81
37.55
36.98
27.11
24.29
18.90
16.25
14.67
11.61
11.28
11.10
10.40
9.73
9.71
9.14
8.66
8.06
7.73
7.66
7.05
6.41
6.14
6.10
5.46
5.27
5.13
5.05
4.96
4.87
4.65
4.62
4.14
3.97
3.66
3.46
3.42
80.59
60.81
54.58
45.42
43.96
40.66
34.80
30.04
23.08
19.78
16.48
15.75
12.82
9.52
10.62
10.99
9.16
6.23
7.69
8.06
6.96
7.69
3.66
5.49
5.49
5.49
3.66
3.30
5.86
4.76
5.49
3.30
5.49
2.56
4.76
2.56
3.30
3.66
2.56
2.93
AoS
JASA
Bka
JRSB
AoP
JMA
Bcs
JSPI
AISM
Bern
SJS
SSci
Tech
Ecnt
SPL
CJS
AmSt
AnAP
PTRF
CSTM
JRSA
SnkA
Ssin
JAP
ApSt
AAP
SPA
SMed
ISR
TPA
ANZJ
SnkB
JTSA
JCGS
JNS
Mtka
JAS
CSDA
CSSC
Stat
85.92
74.62
66.33
59.29
34.70
34.35
30.74
29.53
24.26
24.08
22.57
21.45
21.39
21.30
20.12
18.61
17.37
15.98
15.80
15.56
14.82
13.34
13.14
13.08
12.19
9.29
8.28
8.25
8.17
7.81
6.89
6.86
6.78
6.48
6.21
5.18
4.73
4.41
4.35
4.29
92.90
85.80
74.56
71.01
38.46
32.54
27.22
27.22
23.08
23.08
21.89
23.08
21.89
21.89
17.16
10.65
12.43
14.79
17.16
13.61
13.02
13.02
12.43
13.02
10.06
8.28
9.47
7.10
7.10
7.10
5.33
6.51
3.55
4.73
4.73
4.73
2.96
2.37
2.96
3.55
JASA
JRSB
Bka
AoS
Bcs
SSci
Tech
ApSt
CJS
JCGS
AoP
JSPI
JRSA
SMed
Ecnt
AmSt
SJS
Ssin
JMA
SnkA
AnAP
TStt
Bern
JTSA
Bsts
SnkB
JEcn
CSTM
SPL
ISR
JBES
SC
AISM
JAP
Psyc
JABE
AAP
CSDA
JFor
JAS
91.67
74.68
73.66
68.71
41.08
32.26
30.11
22.69
19.84
19.46
18.06
16.94
16.72
15.32
15.16
14.52
14.41
14.03
14.03
10.81
8.28
8.17
8.06
8.06
7.63
7.15
6.83
6.77
6.67
6.29
5.91
5.27
4.73
4.03
3.92
3.76
3.55
3.49
3.39
3.33
Journal of the American Statistical Association (JASA). On the
other hand, Ph.D. academicians interested in econometrics naturally appreciate Econometrica (Ecnt), that they place in the
second position and Ž nd as useful as JASA, but Ž nd the Journal of Econometrics (Jecn) as the most useful journal. This
group demonstrates a preference for the Journal of Business
and Economics Statistics (JBES), Journal of Time Series Analysis (JTSA), Review of Economics & Statistics (RES), the International Journal of Forecasting (IJF), and the Econometrics
Journal (EcxJ).
Further, we examine the journal perceptions of several groups
of researchers based on their research area and independentlyof
their employment type (Table 6). We Ž nd that researchers involved in the applications of statistics perceive more highly the
Journal of Quality Technology (JQT), Psychometrika (Psyc),
the Journal of Applied Statistics (JAS), The Statistician (TStt),
and Quality Engineering (QE). Mathematical statisticians rank
120
General
% Top10
95.70
86.02
84.95
74.19
40.86
32.26
30.11
21.51
8.60
13.98
19.35
10.75
13.98
12.90
18.28
12.90
12.90
9.68
16.13
9.68
8.60
5.38
7.53
5.38
4.30
3.23
6.45
3.23
3.23
5.38
3.23
3.23
3.23
4.30
3.23
1.08
2.15
2.15
3.23
3.23
Annals of Statistics (AoS) in the Ž rst place across all metrics,
seem to prefer theoretical journals, and highly value the Journal
of Multivariate Analysis (JMA), Journal of Statistical Planning
and Inference (JSPI), Annals of the Institute of Statistical Mathematics (AISM), Bernoulli (Bern), and Probability Theory and
Related Fields (PTRF).
Bayes methods statisticians have become heavily computationally oriented using Markov chain Monte Carlo analyses and
rank higher the Journal of Computationaland Graphical Statistics (JCGS), the leading computational statistics journal, along
with Sankhya Series A and Series B (SnkA and SnkB) and the
Journal of Time Series Analysis (JTSA). Researchers interested
in survey methodology rank higher the Survey Methodology
(SrvM), that they Ž nd to be the most useful journal, the Journal
of OfŽ cial Statistics (JOS), Journal of Business and Economics
Statistics (JBES), and The Statistician (TStt).
Table 6. (continued) Journal Rankings Based on Research Area (all respondents)
Survey methodology
(n = 86)
Probability (n = 63)
Stochastic processes (n = 56)
Rank
Journal
INDEX
% Top10
Journal
INDEX
% Top10
Journal
INDEX
% Top10
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
JASA
Bka
JRSB
AmSt
SrvM
AoS
JOS
JRSA
Bcs
ISR
Tech
Chnc
CJS
ApSt
SSci
Ecnt
AoP
SnkB
CSTM
SMed
AJE
SJS
JBES
TStt
SnkA
JSPI
Ssin
JMA
ANZJ
JAS
Bsts
CSSC
AISM
Psyc
AnAP
JABE
JSCS
JCGS
JFor
AoE
78.49
50.23
46.45
40.52
38.31
33.43
33.08
31.34
28.55
20.29
17.79
13.37
12.79
12.56
12.50
9.83
9.19
8.55
7.56
7.56
7.27
7.03
6.34
6.05
5.81
5.29
5.00
4.94
4.42
4.19
4.01
3.90
3.72
3.72
2.50
2.44
2.44
2.38
1.98
1.98
83.72
59.30
55.81
36.05
39.53
39.53
31.40
38.37
29.07
19.77
18.60
9.30
8.14
9.30
9.30
11.63
9.30
8.14
4.65
8.14
5.81
3.49
5.81
5.81
4.65
2.33
3.49
4.65
2.33
4.65
3.49
1.16
4.65
3.49
2.33
1.16
2.33
1.16
2.33
1.16
AoP
AoS
PTRF
AnAP
SPA
AAP
JAP
JASA
JTP
Bka
Bern
AIHP
EJP
JMA
TPA
SPL
JRSB
SSci
ECP
JRSA
JSPI
RSA
CJS
Bcs
SnkA
MPRF
SSR
AmSt
Tech
CSTM
AISM
Ecnt
PEIS
JTSA
TPMS
MF
Ssin
JAS
ISR
FS
80.48
58.65
51.51
48.97
38.65
36.98
31.83
30.08
29.52
28.10
27.14
26.98
22.78
22.78
20.16
19.68
16.19
12.38
12.30
11.83
11.35
10.16
9.37
7.86
7.86
7.30
6.90
6.51
6.51
6.43
6.35
5.56
5.00
4.13
3.97
3.89
3.81
3.57
3.49
3.17
88.89
73.02
68.25
71.43
60.32
60.32
50.79
44.44
44.44
44.44
42.86
47.62
36.51
42.86
36.51
36.51
28.57
20.63
23.81
25.40
22.22
17.46
23.81
19.05
14.29
19.05
14.29
17.46
11.11
15.87
12.70
11.11
7.94
9.52
7.94
7.94
7.94
7.94
7.94
9.52
AoS
AoP
JASA
Bka
AnAP
JRSB
SPA
AAP
JAP
PTRF
JRSA
Bern
Bcs
SPL
TPA
AIHP
JMA
SJS
CSTM
JTSA
JTP
Tech
CJS
ApSt
SSR
SM
ASMD
SAA
AISM
AmSt
SSci
Envr
Ssin
SA
JSPI
Ecnt
MPRF
TPMS
FS
SnkA
77.59
67.32
48.66
47.05
44.02
43.04
42.59
36.34
35.27
34.64
29.55
27.50
22.68
21.07
16.34
15.71
15.45
15.09
14.91
11.34
11.16
10.45
8.21
7.95
7.68
7.68
7.32
7.14
6.79
6.70
6.52
5.18
5.18
5.09
5.00
4.91
4.82
4.64
4.55
4.46
87.50
78.57
53.57
51.79
51.79
50.00
50.00
37.50
39.29
42.86
28.57
26.79
19.64
21.43
16.07
14.29
14.29
12.50
12.50
8.93
10.71
10.71
1.79
3.57
7.14
3.57
3.57
5.36
5.36
3.57
7.14
5.36
3.57
5.36
3.57
3.57
1.79
3.57
3.57
1.79
Our examination of the worldwide rankings indicates that
probability journals appear to be appreciated by an underlying
segment of the population. More speciŽ cally, when examining
statisticians interested in probability and statisticians interested
in stochastic processes, the Annals of Probability (AoP) and The
Annals of Statistics (AoS) share the leading positions. Although
differences exist between these two groups, in general we Ž nd
JASA ranked noticeably lower and not as useful while Probability Theory and Related Fields (PTRF), Annals of Applied Probability (AnAP), Stochastic Processes and Applications (SPA),
Advanced Applied Probability (AAP), the Journal of Theoretical Probability (JTP), and the Statistics and Probability Letters
(SPL) are ranked higher.
3.3 Perceptions Versus Objective Criteria of Journal
Ranking
Apart from examining the perceptions about statistics journals, we also considered the relationship between worldwide
journal perceptions and journal citations. We therefore calculated the correlationsof our worldwide sample Index with the To-
tal Cites and Impact Factor (Total Cites adjusted for the number
of articles publishedin the two previousyears) of the 54 statistics
journals included in the 2001 ISI Journal Citation Reports. The
correlation between Total Cites and Index is .84, whereas the
correlation between Impact Factor and Index is .56. Although
the Impact Factor appears to be more meaningful as it was developed to eliminate the bias towards large journals over small
ones and older versus new ones, our results indicate that journal perceptions are more closely related to the total number of
citations. One explanation might indeed be that perceptions are
in uenced by a journal’s volume of publication rather than its
impact. But before arriving at such a conclusion and given that
we observe signiŽ cant differences in journal perceptions among
segments of statisticians, one may reconsider the use of citation
measures provided by ISI; these measures do not focus on citations found in the statistics literature or the speciŽ c research area
within statistics, but include any citation source available. This
concern has been raised by other disciplines that have addressed
the issue by conducting citation studies using as the source for
The American Statistician, May 2003, Vol. 57, No. 2
121
citations the most relevant journals of the discipline rather than
all available journals in a database (Alexander and Mabry 1994).
4.
CONCLUSIONS
The results of this survey shed some light on the perceptions
of statistics journals worldwide. Findings indicate that although
a leading set of journals is highly perceived, journal perceptions differ considerably depending on the statistician’s speciŽ c
geographical origin, research interests and employment type.
In addition, while some journals may not be perceived as top
tier journals overall, they are indeed useful to researchers from
speciŽ c research areas. Furthermore, we found that subjective
journal perceptions are more related with a journal’s total number of citations rather than its citations impact factor. This is an
issue that needs to be further examined by performing a citation
analysis using as the source of citations only journals relevant
to the particular area of statistics.
APPENDIX: JOURNALS
Acronym
Journal name
AAP
AJE
AmSt
AIHP
AISM
AnAP
AoE
AoP
AoS
ASMD
ApSt
ANZJ
Bern
BJnl
Bcs
Bka
Bsts
CJS
Chnc
CSSC
CSTM
CSDA
CmpS
CCT
DMKD
DS
ER
ET
Ecnt
EcxJ
ECP
EJP
EES
Envr
ESPS
FS
IJE
Adv Applied Probability
Amer J of Epidemiology
The American Statistician
Ann de l’Institut Henri Poincare
Ann Inst Statistical Mathematics
Ann of Applied Probability
Ann of Epidemiology
Ann of Probability
Ann of Statistics
Appl Stoch Models & Data Anal
Applied Statistics
Austr & New Zeal J of Statistics
Bernoulli
Biometrical Journal
Biometrics
Biometrika
Biostatistics
Canadian J of Statistics
Chance
Commun Statist - Simul Comput
Commun Statist - Theor Meth
Comput Statistics & Data Anal
Computational Statistics
Controlled Clinical Trials
Data Mining Knowl & Discovery
Decision Sciences
Econometric Reviews
Econometric Theory
Econometrica
Econometrics Journal
Electronic Commun Probability
Electronic J of Probability
Environmental & Ecological Statist
Environmetrics
ESAIM: Probability & Statistics
Finance & Stochastics
Intl J of Epidemiology
122
General
IJF
ISR
JABE
JBES
JASA
JAE
JAP
JAS
JBS
JChm
JCls
JCGS
JEcn
JEBS
JFor
JKSS
JMA
JNS
JOS
JQT
JRSA
JRSB
JSCS
JSPI
JSS
JSE
JTP
JTSA
LDA
MPRF
MF
MCAP
Mtka
NAAJ
PEIS
PTRF
Psyc
QE
QP
ROSE
RSA
RES
RA
SnkA
SnkB
SAJ
SJS
SA
SJSC
SASJ
SISP
SMMR
SN
Ssin
StaM
SP
SSci
Stat
SC
SD
SPL
SMed
SAA
SM
SPA
SSR
SrvM
Intl J of Forecasting
Intl Statistical Review
J Agric Biol & Environ Statistics
J Bus & Economic Statistics
J of Amer Statistical Assoc
J of Applied Econometrics
J of Applied Probability
J of Applied Statistics
J of Biopharm Statistics
J of Chemometrics
J of ClassiŽ cation
J of Comput & Graph Statistics
J of Econometrics
J of Educ & Behav Statistics
J of Forecasting
J of Korean Statist Soc
J of Multivariate Analysis
J of Nonparametric Statistics
J of OfŽ cial Statistics
J of Quality Technology
J of Royal Statist Soc, Ser A
J of Royal Statist Soc, Ser B
J of Statist Comput & Simul
J of Statist Planning & Inference
J of Statistical Software
J of Statistics Education
J of Theoretical Probability
J of Time Series Analysis
Lifetime Data Analysis
Markov Proc & Related Fields
Mathematical Finance
Meth & Comput in Appl Probability
Metrika
North Amer Actuarial J
Prob in Enginrg & Informational Sci
Probability Theory & Rel Fields
Psychometrika
Quality Engineering
Quality Progress
Random Opratrs & Stoch Equns
Random Structures & Algorithms
Review of Econ & Statistics
Risk Analysis
Sankhya, Series A
Sankhya, Series B
Scandinavian Actuarial J
Scandinavian J of Statistics
Sequential Analysis
SIAM J ScientiŽ c Computing
South African Statistical J
Statist Inference for Stoch Proc
Statist Methods in Medical Rsrch
Statistica Neerlandica
Statistica Sinica
Statistical Modelling
Statistical Papers
Statistical Science
Statistics
Statistics & Computing
Statistics & Decisions
Statistics & Probability Letters
Statistics in Medicine
Stochastic Analysis & Applic
Stochastic Models
Stochastic Proc & Applications
Stochastics & Stochs Reprts
Survey Methodology
TS
Tech
Test
TStt
TPA
TPMS
Teaching Statistics
Technometrics
Test
The Statistician
Theory Probab & Applications
Theory Probab & Math Statistics
[Received September 2002. Revised January 2003.]
REFERENCES
Alexander, J. C., and Mabry, R. H. (1994), “Relative SigniŽ cance of Journals,
Authors, and Articles Cited in Financial Research,” Journal of Finance, 49,
697–712.
Baltagi, B. H. (1999), “Applied Econometrics Rankings: 1989–1995,” Journal
of Applied Econometrics, 14, 423–441.
Bailar, B. (1988), “Statistical Practice and Research: The Essential Interactions,”
Journal of the American Statistical Association, 83, 1–8.
Genest, C. (1997), “Statistics on Statistics: Measuring Research Productivity
by Journal Publications between 1985 and 1995,” The Canadian Journal of
Statistics, 25, 427–443.
(1999), “Probability and Statistics: A Tale of Two Worlds?,” The Canadian Journal of Statistics, 27, 421–444.
Gibbons, J. (1990), “U.S. institutional Representation on Editorial Boards of
U.S. Statistics Journals,” The American Statistician, 44, 210–213.
Hult, G. T., Neese, W. T., and Bashaw, R. E. (1997), “Faculty Perceptions of
Marketing Journals,” Journal of Marketing Education, 19, 37–52.
Lane, J., Ray, R., and Glennon, D. (1990), “Work ProŽ les of Research Statisticians,” The American Statistician, 44, 9–13.
Luke, R. H., and Doke, E. R. (1987), “Marketing Journal Hierarchies: Faculty
Perceptions, 1986–87,” Journal of the Academy of Marketing Science, 15, 1,
74–77.
Theoharakis, V., and Hirst, A. (2002), “Perceptual Differences of Marketing
Journals: A Worldwide Perspective,” Marketing Letters, 13, 389–402.
The American Statistician, May 2003, Vol. 57, No. 2
123