Download Manginelli.Angela

Comparison of Performance of Laboratory Equipment
1
Manginelli ,
2
Hoyer ,
3
Brestrich
Dr. Angela A.
Dr. Walter
Dr. Gordon
1) R&D Statistics, 2) CMC Statistical Science, 3) Clinical Laboratory Science
Emil-von-Behring-Straße 76 D-35041 Marburg, Germany.
Abstract
The Three Liquid Handlers
• Within the validation of a new highly automated assay, the Clinical
Laboratory Science tested several clinical samples in order to
evaluate precision and linearity of the assay.
• The equipment utilized for the testing includes three liquid handlers
(highly automated pipetting robots).
• The statistical evaluation of the data revealed a significantly poorer
performance of one liquid handler compared to the other two.
• To detect the root cause of the issue, an intensive work including
complete dismantling of the machine with help of a technician of the
vendor was conducted.
• A tiny difference in the programming code of the liquid handlers was
detected.
• JMP 11.0.0 was used for the statistical evaluation before and after
the adjustment of the setting of the liquid handlers.
Maja
Bonnie
Kriemhild
Comparison of Performance of Laboratory Equipment
1
Manginelli ,
2
Hoyer ,
3
Brestrich
Dr. Angela A.
Dr. Walter
Dr. Gordon
1) R&D Statistics, 2) CMC Statistical Science, 3) Clinical Laboratory Science
Emil-von-Behring-Straße 76 D-35041 Marburg, Germany.
How The Assay Works – Serial Dilution Scheme
Automated
pipetting
384 Well –
Micro titer plate
Transfer 14 µL
Bacteria
Step 1
Sample +
Buffer Mix
1:2
28 µL
Predilution
Buffer
14 µL
Buffer
14 µL
Buffer
14 µL
Transfer 14 µL
Step 2
Mix
1:2
Mix
1:4
Buffer
Buffer (diluent)
384 Well – Micro titer plate
Buffer
Transfer 14 µL
Step 3
....
Mix
1:2
....
Mix
1:4
....
Mix
1:8
....
Buffer
....
• After incubation time, the number of surviving bacteria is
determined by means of measurement of fluorescence intensity.
Sample 1
Sample 1  1:2
Sample 1  1:4
Sample 2
Sample 2  1:2
Sample 2  1:4
Normalized values
Human
complement
Serum of
vaccinated
donors
Predilution and Titer Determination
Inflection Point
Log(Dilution)
• Serial dilution is used to determine titer.
• The bactericidal antibody titer is the reciprocal dilution
of the serum at which 50% of bacteria are killed.
Comparison of Performance of Laboratory Equipment
1
Manginelli ,
2
Hoyer ,
3
Brestrich
Dr. Angela A.
Dr. Walter
Dr. Gordon
1) R&D Statistics, 2) CMC Statistical Science, 3) Clinical Laboratory Science
Emil-von-Behring-Straße 76 D-35041 Marburg, Germany.
Dilutional Linearity
• Samples are tested on three liquid handlers which
are expected to deliver identical results.
Results
Error bars:
95% confidence
interval of the mean
• Relative accuracy of the assay is evaluated by
means of dilutional linearity.
Liquid Handler
Bonnie
Kriemhild
Maja
• Dilutional linearity is assessed by computing
Recovery Rate (RR): ratio between the geometric
mean (GM) of the two replicates at one
predilution and the GM at previous predilution.
• 20 samples (serum)  40 RRs on each robot (one
for 1:2 and one for 1:4 predilution).
Mean RR [%]
90
98
97
Std Dev RR
0.28
0.10
0.12
• Lower RR and higher variability on Bonnie.
• Difference in RRs was significant (GLM) when removing the
outlier circled in red (p = 0.0102).
Statistician
The reaction of the team
Lab Manager
Lab Operator
Comparison of Performance of Laboratory Equipment
1
Manginelli ,
2
Hoyer ,
3
Brestrich
Dr. Angela A.
Dr. Walter
Dr. Gordon
1) R&D Statistics, 2) CMC Statistical Science, 3) Clinical Laboratory Science
Emil-von-Behring-Straße 76 D-35041 Marburg, Germany.
Investigation and Solution
Statistical Confirmation
• Lab operator and technician of the vendor were locked for three weeks in the
lab to investigate the issue.
• They compared the setting of the three liquid handlers ...
• ... and found out that the pipet of Bonnie was set to a different mixing speed
compared to the other two:
~16 µl/sec vs ~24 µl/sec
• The setting was adjusted and the experiment was repeated:
24 Samples: undiluted, 1:2 and 1:4 prediluted
2 Replicates (to be averaged) for each predilution on each robot
 18 measurement for each sample, 6 per robot
2 RRs for each sample on each robot (for predilutions 1:2 and 1:4)
Conclusion
• Stay calm, keep trying!
• Never underestimate the importance of small things!
• Luckily with JMP it is possible to write very nice and useful scripts!
Add Image
Comparison of Performance of Laboratory Equipment
1
Manginelli ,
2
Hoyer ,
3
Brestrich
Dr. Angela A.
Dr. Walter
Dr. Gordon
1) R&D Statistics, 2) CMC Statistical Science, 3) Clinical Laboratory Science
Emil-von-Behring-Straße 76 D-35041 Marburg, Germany.
JMP Platform/Script Used
• Graph Builder and ANOVA model
4. Log function
• Script to do the analysis BEFORE and AFTER the issue:
1. Import xlsx file
5. Lag function to create RR
2. Stack and adjust column names
3. Munger function