Download 3 - Lab Times

Editorial
3-2012
Lab Times
page 3
The Nastiest Enemies
Lurking Within Your Experiment...
... are probably the “false positives”. Apparently, you can’t
Pickrell’s concerns proved to be well-founded. In March, Scipush hard enough when choosing the most stringent conditions
ence published three independent comments on the “bombshell”and employing a maximum of independent controls in your expaper, one of them co-authored by Pickrell himself (Science vol.
periment, merely to reduce the risk of catching those annoying
335 (6074): 1302). Each of them strongly confirmed the sceptics
“data bugs” to the lowest possible level. And this, not least, for the
by concluding from their own analyses that at least 90 percent of
sake of your own protection. Just imagine how extremely embarthe reported DNA-mRNA mismatches do constitute false positive
rassing it would be, if your colleagues were later to prove that, in
artifacts. And Joe Pickrell added, “The comments [...] indicate
your latest paper, you were badly fooled by false positive artifacts.
that the analyses done by Li et al. are based on technical artifacts,
Take, for example, that study on RNA editing published last
and do not provide evidence for interesting biology. My opinion is
summer by Mingyao Li and colleagues from the Universities of
that the Li et al. study should have been outright retracted.”
Pennsylvania and Chapel Hill
An unfortunate exception, after
(Science vol. 333(6038): 53-8).
all? Obviously not! Just two weeks lat“Widespread RNA and DNA Seer, an article appeared in PLoS Genetquence Differences in the Huics entitled “Critical Evaluation of Imman transcriptome” was its title
printed Gene Expression by RNA-Seq:
and, at that time, it really made
A New Perspective” (PLoS Genet 8(3):
a major impact, far beyond only
e1002600). This paper made a clean
the RNA community.
sweep with two earlier publications
In his blog genomes unzipped,
in Science (vol 329: 643-8 and 682-5)
for instance, Harvard human gewhich, by following a similar pattern,
neticist Joe Pickrell retrospechad proclaimed an obviously far too
Nasty “false positives”
tively confirmed, “In light of
high total number of imprinted mamin Monty Python’s movie classic “Life of Brian”.
what we know about RNA editmalian genes. To make it short, those
ing, Li et al. was a bombshell.” And in an earlier post he had altwo studies concluded that mammalian genomes contain more
ready made clear, “If these observations are correct, they reprethan 1,300 imprinted genes, thereby contrasting sharply with the
sent a fundamental change in how we view the process of gene
previous and widely accepted estimate of less than 200.
regulation.”
As in the RNA editing example, the authors of the PLoS GenetThis “fundamental change”, in a nutshell, was based on the
ics paper relentlessly uncovered methodological and statistical
authors’ claim that in the protein-coding regions of human messhortcomings in the two studies – finally cutting back the number
senger RNA (mRNA) they had identified more than 10,000 seof imprinted genes to close to the original estimate:
quence mismatches when compared to the corresponding genom“Further analysis and pyrosequencing-based validation reic DNA. Furthermore, Li et al. described that they did not only obvealed that the vast majority of the novel reported imprinted loci
serve the two known RNA editing types of changing adenine for
are false-positives explained by technical and biological variation
inosine and cytosine for uracil but rather “all 12 possible categoof the experimental approach. [...] The results emphasize the imries of discordances”. Thus, the striking conclusion from their reportance of independent validation and suggest that the number
sults was the completely unexpected high frequency and pluraliof imprinted genes is much closer to the initial estimates.” So the
ty, at which RNA editing obviously occurs.
authors write, whereas a comment in PLoS Genetics on the obviUnexpected, to such an extent, however, that a couple of inous lapse just summarises that “[…] the limitations of the novel
siders had remained sceptical right from the start. Joe Pickrell,
technology may not have been fully appreciated”.
for example, started a blog essay on his concerns: “ [...] in this
So what’s the moral of the two examples? Controls, controls,
post I am going to point out a couple of technical issues that,
controls,... and, wherever possible, independent evaluation of
if not properly taken into account, have the potential to cause
the obtained results. Otherwise, those ever-lurking false positives
a large number of false positives in this type of data. The main
might all too easily turn some huge “bombshell” into a “blind
point can be summarized like this: RNA editing involves the proshell”. And, as you all know, the laugh is always on the loser.
duction of two different RNA and/or protein sequences from a
single DNA sequence. To infer RNA editing from the presence of
two different RNA and/or protein sequences, then, one must be
very sure that they derive from the same DNA sequence, rather
than from two different copies of the DNA (due to, for example,
paralogues or copy number variants).”
Moreover, Pickrell in this context explained a couple of additional potential problems that might easily lead to an erroneous
detection of splice junctions, where Li et al. had described to have
found a particularly high rate of sequence mismatch.