Download Benchmarking the Multiplex Circulating and Cellular miRNA Assays

Benchmarking the Multiplex Circulating
and Cellular miRNA Assays using
Abcam’s Firefly™ particle technology
Technical note
Abstract
The full utility of miRNA profiling has yet to be realized largely due to the limited throughput of existing validation
technologies, errors introduced during sample purification and insufficient methods used for analytical normalization
and quality control. We introduce Abcam’s Firefly™ particle technology with assays that enable researchers to rapidly
and cost-effectively evaluate miRNA biomarker signatures on standard flow cytometers. In order to accommodate
studies in solid tissues and circulating biofluids, we developed the Multiplex Cellular miRNA Assay and Multiplex
Circulating miRNA Assay, respectively. We demonstrate that these assays provide miRNA profiles that correlate well
with currently available miRNA profiling technology and show excellent reproducibility even with diminishing RNA
input. We have integrated bioinformatics tools into our assays to effectively interpret experimental data, help identify
candidate signatures and produce publication-quality figures.
Background
Although single-marker diagnostic indicators provide the
most straightforward assessment of disease, they are
rarely able to provide a reliable and complete biological
snapshot necessary to characterize diseases (Pepe et
al., 2001). Multiplexed approaches that test for many
markers simultaneously have been shown to be more
sensitive and accurate in detecting and staging complex
diseases like cancer, even surpassing the performance
of today’s current practice (Cao et al., 2011; Shimizu et
al., 2012).
Conversely, RNA obtained from biofluids is quite scarce,
requiring platforms with enhanced sensitivity. Circulating
miRNAs represent ideal biomarkers for several reasons:
(1) miRNAs are stable in the bloodstream and in
collected samples (Chen et al., 2008; Grasedieck et
al., 2012), (2) blood is easily accessible, (3) circulating
tissue-specific miRNAs can be measured to assess
tissue damage (Laterza et al., 2009) and (4) there is
significant evidence that various cancers have distinct
miRNA profiles (Mitchell et al., 2008).
While most multiplexed biomarker signatures today
are based on genes or proteins, there is a significant
push to use microRNAs (miRNAs) as independent or
supplemental biomarkers (Iorio & Croce, 2012). These
regulatory molecules have been implicated in diabetes,
Alzheimer’s disease, heart disease, and cancers
including leukemia, brain, colon, liver and lung cancers
(Calin et al., 2005, Cummins & Velculescu, 2006;
Esquela-Kerscher & Slack, 2006; Jay et al., 2007; Volinia
et al., 2006; Wu et al., 2007).
Most multiplexing technologies for miRNA detection
are designed for either an in-depth analysis of a few
samples (e.g. sequencing) or a few targets over many
samples (e.g. manual qRT-PCR). However, because
the number of samples required to make statisticallysound observations in validation studies increases
dramatically with biomarker number (Olkin & Finn,
1995), appropriately designed validation studies require
profiling of 10–100 miRNAs across hundreds or even
thousands of samples.
Biomarker signatures based on miRNAs can be derived
from solid tissues or circulating biofluids. Tissue-based
samples yield an abundance of RNA (typically 1–10 μg),
which is suitable for most miRNA profiling platforms.
Several experimental design and pre-analytical factors
can lead to error when performing miRNA studies,
including sample collection, RNA purification, data
normalization and identification of confounding factors
(Kroh et al., 2010). Specifically, it is well known that
methods currently used for miRNA purification can have
profound effects on miRNA profile, yield and throughput,
and all create the potential for user error. The sequencespecific bias introduced during RNA purification has
already led to erroneous conclusions in at least one
important miRNA study (Kim et al., 2012). Ideally,
miRNAs would be detected directly from clinical samples
with no RNA purification.
Another critical consideration in the development of a
biomarker signature is the use of robust normalization
and quality control methods. Lack of attention to
normalization can lead to false positives, false negatives
and biased assay results. Because single-target
normalization has been shown to be problematic
(Vandesompele et al., 2002), a normalization method
using multiple endogenous features and robust analytical
methods could greatly increase concordance of miRNA
profiling across studies.
Overview of Firefly particle technology
The Firefly particle technology is designed to meet
the needs of miRNA biomarker validation, providing a
combination of performance, multiplexing, throughput,
cost and ease of use not available from existing
technologies. The platform offers:
•Parallel processing of 96 samples
•Multiplexed detection of up to 70 targets (68 miRNA
and 2 internal controls) in every sample
•Streamlined analysis with a full-service software suite
•Use of hydrogel particles, which provides 75x
increase in hybridization sensitivity and two extra
logs of dynamic range when compared with the
leading bead-based system
To accommodate the large sample numbers required for
appropriately-powered validation studies, Firefly particle
technology enables the high-throughput analysis of
hundreds of samples over 10–100 miRNA targets (Figure
1A). Furthermore, in order to meet the range of input
requirements seen across the biomarker development
field, we have developed the Multiplex Cellular miRNA
Assay to be used when RNA is abundant, and the
Multiplex Circulating miRNA Assay for highly-sensitive
miRNA profiling in biofluids or when extracted RNA
samples are limiting (Figure 1B).
•Ability to detect miRNAs directly from crude
sample digests
•Readout on existing instruments without the
purchase of new equipment
A
B
Required Input
Firefly technology
RT-qPCR
1,000
Microarrays
# Samples
Sequencing
Circulating Assay
Cellular Assay
100
10
1
1
10
100
1,000
#miRNAs per Sample
10,000
0.1
1
10
100
1,000
Total RNA (ng)
Figure 1: Position of Firefly particle technology in the miRNA profiling landscape. (a) Capability of different miRNA
profiling technologies at handling multiple miRNAs across numerous samples, (b) required input RNA amounts
suggested for major platforms.
The flexibility of Firefly particle technology allows
complete customization and analysis of a broad
range of small RNAs, whether annotated in miRBase
or not. Importantly, the assays are not susceptible to
PCR inhibitors, and have been shown to work well
with modified RNAs, including small interfering RNAs
(siRNAs) and plant miRNAs.
The Multiplex Circulating miRNA Assay is intended
for applications where RNA input is extremely scarce,
as is the case with circulating biofluids. This assay is
optimized for use in crude digests of biofluids including
plasma, serum, exosomes and milk. This allows
researchers to completely eliminate workflow challenges
and target biases seen in RNA purification protocols.
The Multiplex Cellular miRNA Assay is designed for
simplicity and robustness. With only three primary
steps, the workflow is extremely streamlined; it is fully
automatable and does not require reverse transcription,
amplification or other complicated molecular biology
steps typical of library preparation for next generation
sequencing. This assay is ideal when the starting
material is derived from tissue, cell culture or any other
medium where RNA is fairly abundant.
Cross-platform concordance
miRNA profiling platforms exhibit platform-specific
effects with respect to sensitivity, specificity and
sequence-dependent detection efficiency (Mestdagh et
al., 2014). While direct comparison of platforms across
miRNAs in a single sample rarely provides strongly
correlating data, it is more important that the differential
expression of miRNAs across samples is accurately
captured. To assess agreement with currently available
miRNA profiling technology, we compared the miRNA
profiles obtained using Firefly particle technology with
those obtained using sequencing (Illumina MiSeq) and
qPCR (TaqMan Low Density Array, TLDA, Figure 2A).
As miRNAs are differentially expressed across tissue
types, we used total RNA isolated from three diverse
human tissues, profiling miRNAs known to vary greatly
in expression across them. Commercially available
total RNA isolated from brain, lung and liver were used
as an input to the assays. Multiplex Circulating and
Cellular miRNA Assays were performed in-house, while
sequencing was performed by Genewiz and qPCR
analysis was performed by the Shannon McCormack
Molecular Diagnostics Laboratory at Dana Farber Cancer
Institute. Input amounts for the Multiplex Cellular miRNA
Assay, Multiplex Circulating miRNA Assay, qPCR and
sequencing assays were 1 µg, 5 ng, 60 ng and 1 µg,
respectively.
For comparison of miRNA profiles, qPCR signals were first
transformed to linear space. For each platform, signals
were geometric-mean normalized across targets, then
for each target across the three samples. This approach
is used to account for differences in RNA input amounts
and overall analytical signal magnitude. Using these
normalized data, cross-platform comparison was qualified
using Pearson correlation (Figure 2B). Overall, platforms
showed good concordance yielding correlations above
88% in all cases, with the Multiplex miRNA Assays giving
correlations consistently above 90%. Interestingly, while
the Multiplex Cellular and Circulating miRNA Assays
showed the highest correlation (Figure 2C), qPCR and
sequencing showed the lowest across this target set
(Figure 2D).
These data suggest that for differential expression
analysis, the Multiplex Cellular and Circulating miRNA
Assays compare very well with both sequencing and
qPCR-based approaches. Similar results were obtained
with microarray-based approaches (data not shown).
Additionally, the high level of agreement between
the Multiplex Cellular and Circulating miRNA Assays
demonstrates excellent intra-platform concordance.
A
0 1 2 3
Value
Cellular Assay
BRAIN
Circulating Assay
qPCR
Sequencing
Cellular Assay
LUNG
Circulating Assay
qPCR
Sequencing
Cellular Assay
LIVER
Circulating Assay
qPCR
Sequencing
Cellular
Assay
Cellular
Assay
Circulating
Assay
C
qPCR
Sequencing
100%
Circulating
Assay
95%
100%
qPCR
92%
90%
100%
Sequencing
90%
92%
88%
Circulating Assay vs.
Cellular Assay
100%
D
Brain
Lung
Liver
Cellular Assay
Sequencing vs. qPCR
Sequencing
Cross-platform correlation
Circulating Assay
B
Brain
Lung
Liver
qPCR
Figure 2. Cross-platform comparison of miRNA profiling assays. (a) Heatmap of normalized expressions across
Multiplex Cellular miRNA Assay, Multiplex Circulating miRNA Assay, TaqMan TLDA and Illumina MiSeq assays. (b)
Pearson correlations between normalized platform expression profiles, averaged across three sample types. (c)
Correlation plots of Multiplex Cellular miRNA Assay versus Multiplex Circulating miRNA Assay. (d) Correlation plots
showing sequencing versus qPCR.
Assay reproducibility and sensitivity
For an assay to be suitable for biomarker validation and
potential clinical use, it must be robust and reproducible.
Specifically, it is important to understand the detectable
fold-change of a target if differential expression is
to be used as a decision criterion. To assess assay
reproducibility, we performed replicate analysis on a
single sample and quantified the number of targets
measured within a given fold change. This approach is
consistent with that used in the miRNA quality control
study by Mestdagh et al. (2014).
We used 5 ng each of total RNA from brain, lung and
liver as input for duplicate analysis with the Multiplex
Cellular miRNA Assay, Multiplex Circulating miRNA
Assay and qPCR (TLDA). Data were analyzed for
expression differences across replicates; the fraction of
miRNA targets showing smaller deviation than a given
fold-difference were plotted against that fold-difference
(Figure 3A). Platforms with very good reproducibility
yield a cumulative target fraction that quickly approaches
a value of 1 (ideal curve shown).
Overall, the three platforms show good reproducibility in
this study. While 90% of the targets assayed show less
than 0.5-fold difference across duplicates for qPCR, the
same number show less than 0.4-fold for the Multiplex
Circulating Assay and less than 0.2-fold for the Multiplex
Cellular Assay. These data suggest that the Multiplex
Cellular Assay is capable of detecting fairly minute
differential expression of most targets.
The reproducibility of a platform across input amounts
demonstrates assay robustness and indicates dynamic
range. Using a 1:1:1 mixture of brain, lung and liver total
RNA, we assessed intra-platform reproducibility across a
broad range of input amounts. In all cases, we observed
Pearson correlations >94% for all input amounts
(Figure 3B).
This experiment also demonstrates the sensitivity of the
Firefly Circulating miRNA Assay. While most microarray
platforms use micrograms of RNA, and most qPCR
platforms require nanogram inputs, we show the robust
detect of miRNAs with as little as 39 picograms. The
ability to profile multiple miRNAs in a single well without
splitting the sample makes Firefly particle technology
ideal for low-input assays.
Replicate fold-difference analysis
A
1
Cumulative Fraction
0.8
0.6
Ideal Curve
0.4
qPCR
Cellular Assay
0.2
Circulating Assay
0
0
0.2
0.4
0.6
0.8
1
Relative Expression Difference (log2)
B Correlation with input amount
1000 ng 333 ng
111 ng
37 ng
1000 ng
100%
333 ng
99%
100%
111 ng
97%
99%
100%
37 ng
94%
97%
99%
100%
156 pg
39 pg
2500 pg 625 pg
Cellular Assay
2500 pg
100%
625 pg
100%
100%
156 pg
98%
99%
100%
39 pg
96%
97%
95%
Circulating Assay
100%
Figure 3. Reproducibility of Multiplex Circulating and Cellular miRNA Assays. (a) Fold-change analysis for duplicate
measurements of brain, lung, liver total RNA mix. (b) Pearson correlation values for the Multiplex Cellular miRNA Assay
and Multiplex Circulating miRNA Assay across total RNA input amounts.
Integrated analysis capabilities
Proper data interpretation is critical to obtain reliable
results. The Firefly Analysis Workbench is a software
suite that provides an efficient means of visualization,
normalization and data analysis behind a simple
point-and-click user interface. Our software includes
many advanced features, comparable to packages such
as Exiqon Genex, Affymetrix Expression Console or
Agilent GeneSpring.
In addition to standard visualization features
including barplots and scatter plots, the Firefly
Analysis Workbench allows for heat map analysis with
expression in linear, log or fold-change space, along
with hierarchical clustering of samples and/or miRNAs
(Figure 4). Data preparation for plotting includes blank
and negative control signal subtraction (i.e. background
signal), as well as normalization either by hand-selected
probes or by probes automatically selected according to
several algorithms, including GeNorm.
Samples can be assigned experimental variables
(e.g. disease state, tissue type, etc.) and grouped
for subsequent analysis of variance. Probes that are
differentially expressed between groups are ranked by
their statistical significance and displayed on a volcano
plot, which provides a rapid indicator of the miRNAs that
show sizeable fold-change with high significance
(Figure 4, right). Other displays include scatter plots
comparing signatures across samples, and box and
whisker plots showing relative miRNA expression across
groups (Figure 4, right).
Variable Analysis
Plate Preview
Heatmap View
Sample/Target View
Barplot View
Sample Correlation
Figure 4: Software features included in the Firefly Analysis Workbench. From the main screen (top left), the software
allows visualization by heatmap, barplot and sample correlation. Statistical analysis includes ANOVA with p-value
calculation and box-and-whisker plots.
The Analysis Workbench can be used for the
interpretation of one or many experiments. Multiple
experimental plates can be loaded simultaneously, and
any combination of samples can be mixed and matched
into a single experimental dataset, even if they contain
dissimilar targets. Overall, the software is well-equipped
to handle the experimental scope of large miRNA
validation studies.
Conclusion
Firefly particle technology enables the multiplexed, highthroughput validation of miRNA biomarkers with readout
on standard flow cytometers. The Multiplex Cellular
miRNA Assay and Multiplex Circulating miRNA Assay
span the entire range of miRNA profiling needs. These
assays have strong correlation to other miRNA profiling
platforms with comparable or improved reproducibility in
comparison to qPCR. Our bioinformatics tools allow for
streamlined data analysis and generation of publicationready figures in minutes. When used together, the
Multiplex Circulating and Cellular miRNA Assays and
Firefly Analysis Workbench provide a fast and easy-touse platform for miRNA signature validation.
Cited References
Calin GA, Ferracin M, Cimmino A, Di Leva G, Shimizu M, Wojcik
Grasedieck S, Schöler N, Bommer M, Niess JH, Tumani H,
SE, Iorio MV, Visone R, Sever NI, Fabbri M, Iuliano R, Palumbo
Rouhi A, Bloehdorn J, Liebisch P, Mertens D, Döhner H, Buske
T, Pichiorri F, Roldo C, Garzon R, Sevignani C, Rassenti L, Alder
C, Langer C, Kuchenbauer F (2012). Impact of serum storage
H, Volinia S, Liu C, Kipps TJ, Negrini M, Croce CM (2005). A
conditions on microRNA stability. Leukemia 26, 2414–16.
microRNA signature associated with prognosis and progression
Iorio MV, Croce CM (2012). MicroRNA dysregulation in cancer:
in chronic lymphocytic leukemia. N Engl J Med 353, 1793–1801.
diagnostics, monitoring and therapeutics. A comprehensive
review. EMBO Mol Med 4, 143–59.
Cao DL, Ye DW, Zhang H-L, Zhu Y, Wang Y-X, Yao X-D (2011). A
multiplex model of combining gene-based, protein-based, and
Jay C, Nemunaitis J, Chen P, Fulgham P, Tong AW (2007).
metabolite-based with positive and negative markers in urine for
miRNA profiling for diagnosis and prognosis of human cancer.
the early diagnosis of prostate cancer. Prostate 71, 700–10.
DNA Cell Biol 26, 293–300.
Chen X, Ba Y, Ma L, Cai X, Yin Y, Wang K, Guo J, Zhang Y,
Kim Y-K, Yeo J, Kim B, Ha M, Kim VN (2012). Short structured
Chen J, Guo X, Li Q, Li X, Wang W, Zhang Y, Wang J, Jiang
RNAs with low GC content are selectively lost during extraction
X, Xiang Y, Xu C, Zheng P, Zhang J, Li R, Zhang H, Shang X,
from a small number of cells. Mol Cell 46, 893–5.
Gong T, Ning G, Wang J, Zen K, Zhang J, Zhang C-Y (2008).
Characterization of microRNAs in serum: a novel class of
Kroh EM, Parkin RK, Mitchell PS, Tewari M (2010). Analysis of
biomarkers for diagnosis of cancer and other diseases. Cell
circulating microRNA biomarkers in plasma and serum using
Res 18, 997–1006.
quantitative reverse transcription-PCR (qRT-PCR). Methods 50,
298–301.
Cummins JM, Velculescu VE (2006). Implications of micro-RNA
profiling for cancer diagnosis. Oncogene 25, 6220–7.
Laterza OF, Lim L, Garrett-Engele PW, Vlasakova K, Muniappa
N, Tanaka WK, Johnson JM, Sina JF, Fare TL, Sistare FD,
Esquela-Kerscher A, Slack FJ (2006). Oncomirs - microRNAs
Glaab WE (2009). Plasma microRNAs as sensitive and specific
with a role in cancer. Nat Rev Cancer 6, 259–69.
biomarkers of tissue injury. Clin Chem 55, 1977–83.
Mestdagh P, Hartmann N, Baeriswyl L, Andreasen D, Bernard
Shimizu Y, Takeuchi H, Sakakura Y, Saikawa Y, Nakahara T,
N, Chen C, Cheo D, D’Andrade P, DeMayo M, Dennis L,
Mukai M, Kitajima M, Kitagawa Y (2012). Molecular detection
Derveaux S, Feng Y, Fulmer-Smentek S, Gerstmayer B,
of sentinel node micrometastases in patients with clinical
Gouffon J, Grimley C, Lader E, Lee KY, Luo S, Mouritzen
N0 gastric carcinoma with real-time multiplex reverse
P, Narayanan A, Patel S, Peiffer S, Rüberg S, Schroth G,
transcription-polymerase chain reaction assay. Ann Surg
Schuster D, Shaffer JM, Shelton EJ, Silveria S, Ulmanella
Oncol 19, 469–77.
U, Veeramachaneni V, Staedtler F, Peters T, Guettouche T,
Vandesompele J (2014). Evaluation of quantitative miRNA
Vandesompele J, De Preter K, Pattyn F, Poppe B, Van Roy N,
expression platforms in the microRNA quality control (miRQC)
De Paepe A, Speleman F (2002). Accurate normalization of
study. Nat Methods 11, 809–15.
real-time quantitative RT-PCR data by geometric averaging of
multiple internal control genes. Genome Biol 3, research0034.
Mitchell PS, Parkin RK, Kroh EM, Fritz BR, Wyman SK,
Volinia S, Calin GA, Liu C, Ambs S, Cimmino A, Petrocca F,
Pogosova-Agadjanyan EL, Peterson A, Noteboom J, O’Briant
Visone R, Iorio M, Roldo C, Ferracin M, Prueitt RL, Yanaihara
KC, Allen A, Lin DW, Urban N, Drescher CW, Knudsen BS,
N, Lanza G, Scarpa A, Vecchione A, Negrini M, Harris CC,
Stirewalt DL, Gentleman R, Vessella RL, Nelson PS, Martin
Croce CM (2006). A microRNA expression signature of human
DB, Tewari M (2008). Circulating microRNAs as stable blood-
solid tumors defines cancer gene targets. Proc Natl Acad Sci
based markers for cancer detection. Proc Natl Acad Sci USA
USA 103, 2257–61.
105, 10513–8.
Wu W, Sun M, Zou G-M, Chen J (2007). MicroRNA and cancer:
Olkin I, Finn JD (1995). Correlations redux. Psychol Bull 118,
155–64.
Pepe MS, Etzioni R, Feng Z, Potter JD, Thompson ML,
Thornquist M, Winget M, Yasui Y (2001). Phases of Biomarker
Development for Early Detection of Cancer. JNCI J Natl
Cancer Inst 93, 1054–61.
Discover more abcam.com
Current status and prospective. Int J Cancer 120, 953–60.