Download PPT

Introduction to Regulatory
Statistical Principles
Peter A. Lachenbruch
Oregon State University
College of Public Health and Human
Sciences (Ret.)
1
Conclusion

“There can be few areas where the discipline of
statistics is conducted with greater discipline.
Pharmaceutical statisticians may be engaged in
work that may sometimes involve routine
calculations, but the regular application of
statistical principles to produce high quality
experiments, data and analysis promotes a
professionalism that can itself be a source of
satisfaction.”
Stephen Senn (2000 – The Statistician)
1.2
Research vs. Development

Planned development vs. Exploratory
analyses
 Implications
regarding significance tests
 Journal Publication or Licensing
 Understanding a biological process or
demonstrating a therapeutic benefit
Assess efficacy and Safety
Submit data for review by regulators
1.3
The Cast of Characters

Sponsor – the entity that will market the
product, be responsible for writing the marketing
application (NDA, BLA etc.)



Usually have many skills - Biology, Biochemistry,
Medicine, Regulatory Affairs, Biostatistics, etc.
These people have roles that change over time:
biologists in pre-clinical times, biostatistics in clinical
phases and pre-clinical, etc.
May be drug company, university, government
agency, etc.
1.4
Characters (2)

Investigators – the scientists who conduct the
trials. Must demonstrate their qualifications.
May be physicians, biochemists, etc.
Employees of sponsor or contracted to sponsor
(e.g., a University scientist), CRO (contract
research organization)

May also be independent scientists – e.g., many
statisticians serve as consultants to drug companies
1.5
Characters (3)

Regulatory agency – in US the FDA plays this
role. Has 6 centers: CBER, CDER, CDRH,
CVM, CFSAN, NCTR. Most of what we will talk
of here is related to the first three (Biologics,
Drugs and Devices)

Requirements are governed by the authorizing laws
and regulations that have evolved over many years.
See the Code of Federal Regulations (CFR)
1.6
Two trials needed



For many years, the FDA required two “adequate and
well-controlled” trials that showed clinical benefit to
license a drug or biologic. The sole exception was
vaccines which usually were studied in large (n>5000)
trials.
Recently, the agency has adopted a more flexible
approach, especially in rare diseases and conditions in
which obtaining enough patients for two trials would be
difficult.
If you wish to license a product with one trial, you should
contact the agency and discuss the case .
1.7
ICH

The International Conference on Harmonization
(ICH) is an effort to require the same standard
of evidence for licensure in various regions of
the world.


Joint by US, Europe, and Japan regulatory agencies
and by the pharmaceutical industry organizations in
those regions.
You can find links to these at www.ich.org
1.8
Phases of Drug Development

Pre-clinical:






Prior to the first human study, the sponsor must show
evidence that the product does not have apparent
unacceptable risks for humans
Done in animals: rats, mice
Carcinogenicity, teratogenicity
Stability, shelf-life, potency, contaminant detection
Studies of Maximum Tolerated Dose, route of
administration, frequency of administration (also done
in phase 1 studies
ICH S series (safety) and Q series (quality)
1.9
Phase 1 Studies

Must have a valid Investigational New Drug
authorization – allows sponsor to conduct
studies in human populations




Find MTD
Show a therapeutic response in a wide enough dose
range that the product can be used safely
Primarily safety studies. May be in non-diseased
population or a diseased population. Depends on the
drug.
Often not randomized
1.10
Phase 2 Studies

Learn more about proper dose, schedule of
administration, route of administration (oral,
subcutaneous, intravenous)


Vaccines are frequently single dose, sometimes a
booster dose is given
Therapeutic drugs may be given multiple times



Some products are given by multiple routes (at physician
discretion)
Pain medications usually given until relief is obtained, so
concern about cumulative dose
Chronic medication – e.g., cholesterol lowering medication,
cardiovascular meds, diabetes medication may be given for
long periods – lifetime. Must monitor for problems.
1.11
Phase 2, continued

Must pre-specify response/outcome




Can’t collect many outcomes and select one that
seems to work well. This is exploratory analysis and
is never accepted by regulators.
Analysis plan must be specified
Report to regulatory agency of results, including
safety, efficacy, other data.
It is wise to study several doses, routes of
administration, and schedules of administration
 Sometimes there is information from products of a
similar class that will assist in these studies
1.12
Phase 3 Studies

Must show superiority of the product




Two trials usually needed
Prefer there to be two different populations; e.g.
tertiary care and primary care centers
Need reports for each trial; often detailed information
on patients is given.
The statistical analysis plan (SAP) must be specified
and shouldn’t be changed unless there is new
evidence of changes in the endpoints. Changes after
the data have been locked are highly suspicious.
1.13
Working with Regulators
(with reference to FDA)



Goal of both sponsor and FDA is to bring safe,
effective products to market
Sponsors want to present their product in the
most favorable light (sometimes downplaying
the bad things)
Regulators (FDA especially) are cautious and
don’t want to approve a product that is
ineffective or has a poor balance of benefit and
risk.
1.14
Regulatory work (2)

Note that regulatory agencies are not monoliths:



Advice may differ based on which center, division, or
branch you deal with; sometimes reviewers will affect
the advice
Advice you get may depend on what the favored
research paradigm is, approach, custom of
division/branch – e.g., how missing values are treated
Note that you can appeal the decision if you differ
from the reviewer
1.15
Regulatory work (3)

Evidence standard:

Two “convincing” trials – p-value below 0.05
 What if the two convincing trials happen after 10
attempts?



Can use a negative binomial to suggest this isn’t
persuasive. I find the probability of 2 successes by the
10th trial is 0.0186 under the null hypothesis
If p(success ) is 0.8, the probability of 2 successful trials
by 4 trials is 0.97
Usually need to show superiority, rather than noninferiority although safer products can be persuasive
1.16
Regulatory work (4)

Single trial can be accepted if



A very large trial such as a vaccine trial (tens of
thousands of observations)
A very serious and rare disease is being studied
Speak to the regulators! Many sponsors are
reluctant to discuss their concerns with the FDA
since they seem to fear that if they tell some
problem, the FDA will focus on it.

This is a recipe for disasters. It’s better to avoid
problems in interpretation early than have a fight later
on.
1.17
Some suggestions

Know your reviewers




Perspective
Listen to any advice they offer and make counterproposals if appropriate – reviewers will listen. They
may not agree
Reviewers may change during your study – it’s
important to have a record of any agreements; don’t
rely on memory.
Don’t rely on biological arguments – clinical trials
may contradict the biological argument
1.18
Types of Paperwork

IND or IDE

The application to conduct a clinical trial. No trial can
proceed without one. Can be amended





Outline studies to be conducted, modifications to ongoing
studies, where the study will be conducted, when it will be,
etc.
What drugs, dosage, schedules, route of administration will
be examined, etc.
Specify statistical analysis plan at an early amendment
Non-standard analyses can be proposed – should be justified
Any IND can be amended – studies are not cast in stone.
1.19
Paperwork (2)

One-arm studies (no control) are usually
unacceptable
FDA has seen too many trials that have the single
arm later shown to be inferior to a control arm
Non-randomized studies are notorious for having bias –
the investigator knows what drug is given, rates the
patients more highly than in a controlled trial.


1.20
Paperwork (3)

A famous study (Moertel) extracted results from many
oncology studies and found the non-randomized ones
always showed high response rates, while the
randomized ones were much poorer. Some studies
of this type have been accepted when the population
is small and the natural history of the disease is well
known.
1.21
Paperwork (4)

Make use of ICH guidelines and FDA guidelines.



These are not mandatory, but represent ‘best current thinking’ in
their areas. Alternatives can be proposed.
Most important for clinical trials are E3, E9, E10
Generalizing to a population


Clinical trials are conducted on a small subset of a population.
In order for a product to be approved for marketing, FDA and
other agencies want to be convinced that the results apply to the
broader population. For this reason, sponsors either use broad
entry criteria, or have multiple studies on different populations
This also includes patients or subjects of different ages, different
physical conditions, or different hospitals.
1.22
Paperwork (5):
Marketing applications

NDA and BLA are applications to market a new
product. NDAs are used by CDER (center for
drugs) and BLAs are used by CBER (center for
biologics)


The BLA requires close scrutiny of the manufacturing
process and facilities, because of the greater
variability of biological products.
The application will include complete reports on all
studies including patient listings, analysis according
to the original statistical analysis protocol (SAP) as
well as any exploratory analyses.
1.23
Paperwork (6):
Marketing Applications

Full safety analyses:


Listing of all adverse events
 By organ system
 By frequency
Since neither FDA nor sponsors can define what
events will occur a priori, such analyses tend to be
exploratory. Some direction may be found in other
drugs of the same class (e.g., a new statin or bet
blocker, a new vaccine, etc.)
1.24
Regulator’s needs/wants




Well-written – good grammar, concise, wellorganized
Correct – relevant tables, proper statistics (e.g.
no women in prostate cancer denominators,
etc.); no adults in tables referring to pediatric
applications
Proper number of digits (no spurious precision)
Data set noted as source for each table/graph or
analysis
1.25
Regulator’s needs (2)


Justify any statements – data should
demonstrate this.
Be open and honest about the data


If there are problems, discuss them – show the warts
as well as the beauty.
Do not lie about the data – if an analysis is
determined after the data has been locked, it can be
treated as exploratory. FDA will usually not accept
‘discovered hypotheses.’
1.26
Research paper vs. Marketing
Application


In research, the goal is to have a good paper
published in a good journal. The author(s) may
contact the editor to ascertain interest in the
topic. The paper itself will rarely contain the
data and the analysis will not be reproduced by
referees.
For a marketing application, the sponsor will
have been working with the regulatory agency
for years and will submit all data. The analysis
will be reproduced by the reviewers.
1.27
Research vs. Marketing (2)


For a marketing application, FDA and sponsor
may have agreed on some points. These
generally aren’t binding on either party. It is
similar to the journal – in either case, the journal
may decide it isn’t appropriate for the journal;
the regulatory agency may decide it isn’t
adequate for licensure if the data don’t show
convincing evidence of efficacy
The ultimate goal in regulation is to write a clear,
good label for the product that the prescriber
and public can understand
1.28
Reasons a marketing application
may fail to be approved



The application is not reviewable – data aren’t
adequate, application is incomplete (sections
missing)
Data do not show evidence of efficacy
There are safety concerns even if efficacy is
demonstrated – in one case I know of, there
were serious safety concerns but a blinded
review of the data indicated that these were not
the problem the investigators thought they were.
1.29
Options if Marketing Application
is not Approved







Respond to criticisms
Appeal to Office Director or Center Director
Appeal to FDA Ombudsman
Have advocacy group place pressure on FDA
Have congressman put pressure
The last two are rarely effective since science
usually has left the debate
Conduct a new study
1.30
Understanding FDA language






“Must” is directive – no options to do otherwise
“No” means no
“Recommend” is not directive – you should
discuss your plans in detail
“Should” is strong, but less directive than must.
Generally, FDA will not comment on something
without reviewing the data
Show clinical benefit usually means they are not
interested in means.
1.31
FDA language(2)

Time issues:



In reporting, define what you mean.


What happens at k weeks is different from what
happens by k weeks or within k weeks
If confused, ask!
“clinically meaningful improvement” must be carefully
defined and not rely on a physician’s impression
Be careful about interpreting FDA words:
wanting a sensitivity analysis means FDA wants
to be sure small changes in assumptions don’t
affect the conclusions.
1.32
FDA language (3)

FDA will often look at subgroups to ensure the
findings are robust. This is not doing lots of
subgroup analyses to find something – it is more
looking at various groups to ensure a finding is
real.

In one study, a sponsor had found an effect, but FDA
did some subgroup analysis and found 3 of 4 sites
had no effect and one had a major effect. Subsequent
investigation found that the study coordinator for the
site in question had unblinded the randomization and
gave the treatment to patients she thought would
have a better chance of benefit.
1.33
Interacting with Regulators



Don’t ignore advice – the regulators have likely
seen similar products and know pitfalls
Make your submissions clear – ensure your
analysis answers the questions fully. If they
differ from advice, explain why
Ensure your data are complete and consistent


Have a plan for dealing with missing values
Always present the pre-specified analyses and
any others you have done.

Analyses not pre-specified will be considered
exploratory
1.34
Interacting with Regulators (2)

In one example, I had been working on a 2 part
model theory and a sponsor had a study that I
thought was appropriate for the application.



I suggested it and the regulatory affairs person said
“well do it!”
I responded that the sponsor should check it out to
see if it was appropriate.
A week or two later, the statisticians had run a small
simulation study and decided that a Mantel-Haenszel
test would work better.
1.35
Interactions with Regulators (3)

Communicate with regulators

Some sponsors ban such communications unless a
regulatory affairs representative is present or on the
phone – can result in delays
 Often the regulatory affairs person does not
understand the statistical details being discussed
 This is a message for the regulatory affairs people
– don’t delay work to maintain your control – trust
your statisticians and other staff.
1.36
Interactions with Regulators (4)

Working within a system


Concurrent documentation of procedures and studies
 Include data cleaning and editing procedures
 Data analysis methods and formulas
SOPs for lab tests and all measurements of patients
 Normal ranges for key variables
 How various labs were standardized
1.37
Interactions with Regulators (5)

Definitions:







Treatments – including dose, schedule, route
Population being studied
Null and alternative hypotheses
Tests to be conducted
Size of tests
Power of tests at alternatives
Guidances and Regulations, ICH
1.38
Contents of IND

For a phase 3 trial




Design of study – often involves two group
comparisons; superiority vs. non-inferiority
Inclusion and exclusion criteria
Outcomes (pre-specified!), how measured, how
standardized across investigators
Covariables (not too many – remember label must be
fairly general) – phase 3 is NOT the time for variable
selection
 The covariates should be predictive of outcome,
not of imbalance.
1.39
IND (2)


Sample size computation – often it’s good to
give several alternatives for different size, power
and effect sizes. Give details
Distinguish between non-inferiority and
superiority trials/analysis
1.40
IND (3)


Assumptions involved in analysis
How will missing values be handled (maybe have several
methods)
 I prefer not using LOCF
 Multiple imputation
 Model based methods – likelihood, mixed models, GEE
 Details of randomization: within site, balancing
(minimization)
 Proposed analysis – equations, references, justification
of effect size based on pilot studies, the literature, etc.
1.41
IND (5)

Modifications of a study



It’s almost always done as details change – e.g.,
recruiting is slow, so inclusion criteria may be
broadened (age, disease inclusion, extending
duration of recruitment)
SAP can be changed as long as data are still blinded
to sponsor.
Changing endpoints is possible early on, but after
data are complete, changing becomes suspect even if
sponsor swears they haven’t peeked.
1.42
IND (5)

Documenting work- YES




Data cleaning and editing steps – if programs are used,
give them – regulator may not examine, but can be useful
Acceptable ranges for covariables and endpoints –
 Handling if out of range (go to site and check?)
 Handling inconsistencies in data
Software used – some differences due to different
algorithms – e.g. SAS offers several definitions of
percentiles, Stata does not
Distinguish between exploratory and confirmatory –
generally not helpful to have p-values with exploratory
analyses
1.43
Meeting the Regulators

Prepare a briefing book describing issues to be
discussed


Don’t bring up new results at the meeting as the FDA
will say to submit the information and they will
respond. Extreme example of a consultant
presenting new, un-reviewed data at an advisory
committee meeting.
FDA will answer questions, often close to the meeting
date making it difficult to reply to their concerns
1.44
Meetings (2)

Useless question:



Does the FDA agree that these results support
licensure?
Better to ask “what additional analyses are needed to
support licensure?”
More helpful discussions related to schedule of
submissions, etc.
1.45
Meetings (3)

Type A meetings – when drug development is
stalled and sponsor can’t proceed without FDA
input – e.g., clinical hold (need to know what is
needed to remove the hold), or to resolve a
dispute between sponsor and FDA, or for a
Special Protocol Assessment.

FDA will schedule a type A meeting within 30 days of
receipt of a written request
1.46
Meetings (4)

Type B meetings occur at natural junctures
during the course of drug development – e.g.
pre-IND, end of phase 1, end of phase 2-pre
phase 3, pre-NDA/BLA


Pose clear questions based on interaction with FDA,
problems observed during study
Type C meetings are all other meetings
regarding the development and review of a
product.
1.47
Meetings (5)

Regulators want to approve products but only on
the basis of convincing data.



Sponsors will discuss issues before the meeting
(dress rehearsals)
The FDA review team will also discus issues in each
of the areas: safety, efficacy, quality
Note that FDA reviewers are privy to information from
other studies in the same class and comments may
be related to problems observed in other products.
The FDA can’t disclose specifics, but sponsors should
be aware that issues mentioned may be relevant to
them.
1.48
A Few Statistical Issues

Recent developments and methodological
research of importance to FDA


Incorporating Bayesian methods into drug approval –
informativeness of priors. CDRH has accepted these
for years.
Genomics – large number of variables, relatively few
patients. What standards should be used for
approval? License a process or a product? Seems to
suggest using theory rather than clinical data
1.49
A Few Statistical Issues (2)





Selecting the margin for non-inferiority studies (I’m
not sure where this is at present)
Robust methods for complex statistical models (e.g.,
hierarchical models for non-normal data)
Follow on Biologics
Biologics equivalent to generics – since Biologics are
much more variable than drugs, many tough issues
Adaptive Methods
1.50
Conclusion

“There can be few areas where the discipline of
statistics is conducted with greater discipline.
Pharmaceutical statisticians may be engaged in
work that may sometimes involve routine
calculations, but the regular application of
statistical principles to produce high quality
experiments, data and analysis promotes a
professionalism that can itself be a source of
satisfaction.”
Stephen Senn (2000 – The Statistician)
1.51