Download Astro2010 Expedited Concepts: Ground Overview

Aerospace Support to the Decadal
Study Large Mission Concept Teams
Debra L. Emmons, PhD.
Principal Director
NASA Science & Technology Programs
24-25 April 2017
Presentation to NASA Astrophysics Advisory Committee (APAC)
© The Aerospace Corporation 2017
Agenda
•
Cost and Technical Evaluation (CATE) Background
•
Support to the Decadal Study Large Mission Concept Teams
•
Considerations for Design Trades
2
What is a CATE?: Cost and Technical Evaluation
•
CATE developed by NAS/Aerospace for recent Decadal Surveys
– Previous Decadal Surveys had no process to validate advocate mission costs
– US Congress required National Academies to use independently validated costs
– CATE estimates needed to reflect historical project growth
• CATE estimates needed to reflect realistic NASA/ESA cost sharing
• Realistic CATE estimates needed for future budget analysis & decisions
•
CATE process differs from typical ICE and process for TMC evaluation
– Begins with typical Independent Cost Estimate, ICE
– Adds three types of cost threats, where appropriate:
• Schedule, design (mass & power growth) and launch vehicle
•
CATE is used for future consideration with respect to NASA budgets
– Used to evaluate science value versus budget availability
• Sometimes used to re-assess Decadal recommended concept descopes
– Incorporates typical growth based on the historical record and design maturity
• It is more conservative than an ICE of a “specific” concept presented
3
Aerospace is the Custodian for the NAS CATE Process
•
Requires independent analysis
– Reconciliation with Project teams is recommended, where appropriate
– However, NAS committees are concerned with maintaining confidentiality
•
Requires consistency across diverse concepts
– CATE process is flexible to handle differences in design maturity
– CATE has been used for Astro2010, Planetary and Heliophysics
•
•
•
4
Will be used for Earth Science Decadal in 2016
May be used for Astro2020 and beyond
Stay true to the NAS process
– Advocate teams and NASA HQ do have special requests
– This often can be handled, but the CATE generated S-curve represents the
cost risk assessment
– There is a “T” in CATE and committees and decision makers need a
consistent technical risk assessment
General Limitations of Assessment
•
Technical risk assessment
– Limited to top-level maturity and risk discussions
• Not meant to be a Proposal Evaluation level of effort
•
Cost and schedule assessment
– Meant for high-level budgetary estimates
•
Often includes a profile in real year dollars
– It is understood that the CATE is likely to be higher than advocate estimate
•
Decision makers consider the range in the two estimates
– When appropriate, reconciliation with the project occurs
•
•
Typically when CATE is being presented to NASA HQ
Does not occur when under direct evaluation by an NAS committee
– Design growth threat is typically the biggest disconnect with project teams
•
•
5
Project often defends specific concept being presented
Advocate estimate may not adequately factor in “future” modifications and “growth”
Technical Risk & Maturity Assessment Approach
•
Identify key risks to achieving required performance
– Highlight significant deviations from current state of the art performance
– Trace performance risk to science mission impact
– Evaluate potential of planned risk mitigation efforts
•
Assess technical maturity risk liens on cost and schedule
– Assess claimed TRL level of key technologies
– Apply mass and power growth contingencies consistent with maturity
• Mass growth allowance could result in launch vehicle cost threat
– Late technology maturation steps identified as schedule risks
– Complex system integration issues identified as schedule risks
Low
6
Medium
Low
Medium
Medium
High
High
Project X Top Technical Risks and Concerns
Project X Technical Risk Rating is Medium
•
•
•
7
Medium new development, mostly in the engineering implementation
– Increase in detector array size
– Migration from FPGAs to ASICs
– Modernization of heritage instrument control unit
Mass margins and power margins are aggressive and launch mass
margin is very sensitive to changes in dry mass
– Concept design is closer than recommended to Atlas V 551 capacity
limit and the system is very sensitive to changes in mass
– Several mass liens against concept design
Time critical mission operations contributes to medium operational
risk
– Fault management for autonomous mode requires further definition
– Sampling operations and hardware need further definition
CATE Cost Estimating Approach Overview
Technology Development Cost Factors
Estimated Cost (FY09$M)
$100
$80
Adjusted Analogy
Model
$60
Aerospace
Project
$40
Total Cost Multiplier
$120
4.5
100%
4
90%
3.5
80%
Cumulative Probability
$140
3
2.5
2
1.5
1
0.5
$20
Project
Aero
MICM
IRS
STIS
ACS
0
WFPC2
60%
70th Percentile
50%
Initial Cost
Reserves
Estimate
40%
30%
20%
0
$0
Sum of
Most-likely
Costs
70%
1
2
3
4
5
6
10%
7
0%
$1,000
Starting TRL Level
$1,200
$1,400
$1,600
$1,800
$2,000
Estimated Cost (FY09$M)
Estimate Instruments &
Spacecraft
Estimate Other
Elements
Estimate Cost Reserves
Multiple analogies and models
Based on historical data
Based on probabilistic cost
risk analysis
80%
Kepler
Spitzer
% Growth or Recommended Contingency
70%
GALEX
WBS Element
Swift
60%
GLAST
Chandra
44
$
44
Other Guidelines
$
$
45
251
$
$
98
308
Wrap factors from Kepler, Spitzer, Chandra, JWST, LRO, GLAST
MICM, analogies (Spitzer, Kepler, WISE, IRS, STIS, ACS, WFPC2)
Other Guidelines
Spacecraft
$
174
$
243
NAFCOM, analogies to Kepler, Spitzer, SDO, LRO
Other Guidelines
Pre-launch Ground and Science
$
98
$
Phase E Costs and EPO
$
160
$
183 MO costs from HST, Spitzer, Chandra. DA passed-through.
Total Reserves
$
172
$
309
Launch Vehicle/Services
$
Total Mission Cost Without Threats $
161
1,105
$
$
30%
20%
10%
Schedule Threats
$
Mass and Power Cont. Threats
0%
100%
Pre-Phase A
Phase A Start
Phase B Start
PDR
CDR
Percent Likilihood of Completing on Schedule (%)
Estimate Mass and Power
Contingency Threat
80%
Cumulative Prob. Dist.
SIM-Lite Plan Value
60%
100
7/30/15
LRD (Goal)
54%
50%
89
40%
30%
Wrap factors from Spitzer, Chandra, and JWST
70th Percentile from cost risk Analysis
154 Atlas 511 assumed. Costs from Flagship Mission Studies
1,432
82 9 months at project burn rates
Total Mission Cost With Threats
$
1,105
$
76 Additional 288 kg and 186 Watts
1,590
Estimate Schedule
Threat
Integrate Results & Level
Across Concepts
Based on ISE results and
project burn rates
Cross-check with CoBRA
20%
10%
Re-run estimate with
Aerospace contingencies
0%
40
50
60
70
80
90
100
110
Schedule Duration: Phase B to Launch (Months)
8
$
92
Pass-through
LV Threats
Launch
90%
70%
Basis of Aerospace Estimate
$
Mission PM/SE/MA
Instruments
Aerospace Guideline
40%
Project Aerospace
Estimate Estimate
Phase A
Average
50%
120
130
140
Example Hardware Estimate Results
$140
Notional Results
$120
Estimated Cost (FY15$M)
$100
$80
Adjusted Analogy
$60
Aerospace
$40
$20
$0
9
Model
Project
Cost Risk Process Overview
Used to estimate reserves
Multiple Cost Estimates for Each WBS Element
Triangular Distribution of Possible Element Cost
$25
Adjusted Analogy
Model
Estimate
$15
H
DMF: Design
Maturity Factor
added to high
estimate to
capture worst case
DMF
M
Probability
Cost (FY09$M)
$20
L
$10
$5
$0
L
M
H
Lower Most-Likely
Limit
Cost
Upper
Limit
Example Total Cost Distribution
Total Distribution is a Combination of
Cost Distributions for WBS Elements
100%
90%
80%
Cumulative Probability
WBS Element 1
+
WBS Element 2
+.
.
.
+
Total Cost Probability Distribution
70%
60%
70th Percentile
50%
Initial Cost
Reserves
Estimate
40%
30%
20%
10%
0%
$1,000
WBS Element N
Sum of
Most-likely
Costs
$1,200
$1,400
$1,600
Estimated Cost (FY09$M)
$1,800
$2,000
Design Growth and Launch Vehicle Threats
•
All CATE estimates based on project team inputs
– Wide range in the maturity of the designs
– Some responses are essentially concept descriptions
– Others already have had significant investment maturing the design and required
technology
•
Need to ensure that immature projects didn’t have an unfair
advantage
– Apply higher mass and power contingencies for less mature projects
• Mass and power drive cost estimates from both analogies and models
– Use project-supplied contingencies for estimate without threats
•
•
Aerospace-applied contingencies to develop “Design Growth” cost
threat
Add cost of moving to next larger launch vehicle as “Launch Vehicle”
cost threat
– If mass contingency results in less than 10% launch vehicle mass margin
11
Payload Mass Contingency Values for Threat Estimate
80%
Spitzer
FERMI
% Growth or Recommended Contingency
70%
60%
60%
Kepler
Used for
Astrophysics CATEs
Chandra
Swift
GALEX
Average
CATE Guideline
50%
45%
Other Guidelines
40%
Actual Past Mission
Contingency
30%
30%
20%
10%
0%
Pre-Phase A
Phase A
Phase B Start
PDR
CDR
Launch
CATE contingency values are an extrapolation of historical
Astrophysics mission data
12
Example Cost Estimate Table
Notional Results
WBS Element
Project
Estimate
CATE
Estimate
Basis of Aerospace Estimate
Phase A
$
44
$
44
Pass-through
Mission PM/SE/MA
$
45
$
98
Wrap factors from analogous projects
Instruments
Spacecraft
$
$
251
174
$
$
308
243
Pre-launch Ground and Science
$
98
$
92
Phase E Costs and EPO
$
160
$
183
Total Reserves
$
220
$
250 70th Percentile from cost risk Analysis
Launch Vehicle/Services
$
161
$
154
Total Mission Cost Without Threats $
1,137
$
1,373
Instrument models and analogies
Bus models and analogies
Wrap factors from analogous projects
MO costs from analogous projects. DA passed-through.
Cost for proposed LV
Schedule Threats
$
32 Potential slip from ISE multiplied by project burn rates
Mass and Power Cont. Threats
$
48 Cost re-estimated using Aerospace contingencies
LV Threats
Total Mission Cost With Threats
$
$
13
$
1,137
1,453
Cost difference to move to larger LV
CATE Conclusions
•
CATE is a consistent and robust process to estimate missions for
prioritization within a budget constraint profile
– Analogies and parametric models are used
– Instruments and spacecraft are key drivers and often are under
estimated by the projects
– Process uses a statistical approach to capture appropriate reserves
• Concept maturity
• Technology readiness
– Process uses historical data to address likely cost threats
• Design growth
• Schedule
• Potential change in launch vehicle
•
Results are suitable for prioritization and long-range planning
•
CATEs will be incorporated into future NRC Decadal Surveys
14
Objectives for Support to Large Mission Concept Teams
•
Prime objective
– Achieve a better understanding of technical, cost-risk trades and the
impacts on the large concepts
•
Approach for achieving the prime objective
– Help study teams to understand cost and risk implications of their
mission architecture choices
– Help study teams to better distinguish between areas for deeper
engineering and areas where rules of thumb can suffice
– Aerospace Corporation to provide independent guidance and
suggestions to the study teams for consideration in reducing the
technical and cost risk of the engineering elements.
– Allow Aerospace to better understand mission concepts, technology
requirements and technology maturity as the studies progress, without
commenting on the science merits and without creating a conflict of
interest situation with National Academies
15
NASA Guidance to the Astrophysics Large Mission Concept
Teams
•
•
•
•
•
•
16
Every team will have an ‘allowance’ of 400 work hours plus $4.5K of travel
support from The Aerospace Corporation for their mission concept studies
– This support is from March 8, 2017 through December 31, 2018
– This includes Phase 1 and 2 support only (in-scope items from Task statement on
next slide)
– This allocation does not include the costing and full CATE-like support (this is
Phase 3 support for APD) and is not within scope of this effort
Each team can prioritize the in-scope Phase 2 (next slide) support from
Aerospace
Teams have the option of using their own study funds to augment this support
Phase 1 of the original task, which included education to the teams on the
CATE process is considered complete.
A CATE will be accomplished for each mission concept for the Astrophysics
Division, but will be performed after the final reports are submitted to the
Division and is not included in this effort; CATE is considered Phase 3
support
On the next page is a menu of tasks that are in scope that can be prioritized by
each team
NASA Guidance – In Scope Tasks
•
Phase 2 – Trade Study Support
– Feedback on concepts and mission architecture choices
• Includes consultation of IDL/MDL runs
• Identification of risks
– Feedback on technology roadmaps
• Includes identification of gaps and risks
– Support of F2F meetings as determined by team leadership
– Review interim and final reports
Note: Additional tasks that are not included in the above menu,
but were submitted in the team’s request for support, can be
conducted using these allocated hours, if the teams end up not
using all of the allocation. However these must be negotiated
with the DSMT before proceeding
17
Aerospace Support Provided in Phase I
•
•
•
•
•
Cost and Risk Coaching (at Key decision Points and/or to inform
key trades)
Independent Technology readiness level assessment
Margin Guidance
Discuss the CATE at the PAL and November F2F meetings
F2F meetings and presentations
– Pause & Learn Meeting at HQ – October 20th (Bob Bitten/Angie Bukley)
– Aerospace Briefing on CATE process -- November F2F
• Lynx (X-Ray Surveyor) F2F - Washington DC on 14-15 November
• OST F2F – Boulder, CO on 2-3 November
• LUVOIR and HabEx F2Fs the week of 09-11 November in New
Haven, CT
•
18
On-call support to the Instrument and Mission Design Lab runs
Considerations for Trades & Costing
•
Launch Vehicle
– May want to look at trade of maximizing science on existing
launch vehicles as well as trades beyond current state of the art
vehicles
•
On-orbit Servicing
– Development cost of including provisions for on-orbit servicing
will be part of cost of mission
– Cost of future operational servicing would be outside of budget
•
International Partners
– Missions will be costed as a complete project and elements will
be subtracted as international partnership is acknowledged
19