Download Software Engineering Process Group Skills Rick Hefner

Process Improvement
in the
Aerospace Industry
Caltech Aerospace Seminar
13 April 2009
Rick Hefner, Ph.D.
Director, Process Management
Northrop Grumman Corporation
[email protected]
Process Improvement Workshop
Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
– Agile
• Northrop Grumman Approach
2
Process Improvement Workshop
NDIA Top 5 Systems Engineering Issues
(2003)
• Lack of awareness of the importance, value, timing,
accountability, and organizational structure of SE on
programs
• Adequate, qualified resources are generally not available
within Government and industry for allocation on major
programs
• Insufficient SE tools and environments to effectively execute
SE on programs
• Requirements definition, development and management is
not applied consistently and effectively
• Poor initial program formulation
3
http://www.ndia.org/Content/ContentGroups/Divisions1/Systems_Engineering/PDFs18/
Modeling_Committee_PDFs/February2003_top_5_issues.pdf
Process Improvement Workshop
NDIA Top 5 Systems Engineering Issues
(2006)
• Key systems engineering practices known to be effective are not
consistently applied across all phases of the program life cycle.
• Insufficient systems engineering is applied early in the program life
cycle, compromising the foundation for initial requirements and
architecture development.
• Requirements are not always well-managed, including the effective
translation from capabilities statements into executable requirements to
achieve successful acquisition programs.
• The quantity and quality of systems engineering expertise is insufficient
to meet the demands of the government and the defense industry.
• Collaborative environments, including SE tools, are inadequate to
effectively execute SE at the joint capability, system of systems (SoS),
and system levels.
4
Systems Engineering Update, NDIA Top 5 Issues Workshop. July 26, 2006. Briefing by
Mr. Robert Skalamera
Process Improvement Workshop
Summary - Evolution of Top SE Issues
2003
Lack of awareness
of SE importance
Lack of adequate,
qualified resources
Insufficient SE
tools and
environments
Inconsistent
requirements
definition
Poor initial program
formulation
5
2006
Inconsistent SE
practices across all
life cycle phases
Insufficient quantity
and quality of SE
expertise
Inadequate tools
and collaborative
environments
Requirements not
well managed or
translated
Insufficient SE
early in the life
cycle
Process Improvement Workshop
Complex systems,
systems of
systems
Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
– Agile
• Northrop Grumman Approach
6
Process Improvement Workshop
Two Complimentary Approaches
to Process Improvement
Data-Driven (e.g., Six Sigma)
Model-Driven (e.g., CMMI)
• Clarify what your customer
wants (Voice of Customer)
• Determine the industry best
practice
– Critical to Quality (CTQs)
– Benchmarking, models
• Determine what your
processes can do (Voice of
Process)
• Compare your current
practices to the model
– Statistical Process Control
– Appraisal, education
• Identify and prioritize
improvement opportunities
• Identify and prioritize
improvement opportunities
– Causal analysis of data
– Implementation
– Institutionalization
• Look for ways to optimize the
processes
7
• Determine where your
customers/competitors are
going (Voice of Business)
Process Improvement Workshop
– Design for Six Sigma
The Frameworks Quagmire
Sarah A. Sheard, Software Productivity Consortium
8
http://stsc.hill.af.mil/crosstalk/1997/sep/frameworks.asp
Process Improvement Workshop
Heritage of Standards for Systems
Engineering
2002
2002
ISO/IEC
15504
ISO/IEC
19760
(FDIS)
(PDTR)
1998
1994
EIA / IS
632
1994
1974
1969
Mil-Std499
Mil-Std499A
Mil-Std499B
(Not Released)
EIA
632
(Full Std)
1994
(Trial Use)
ISO/IEC
15288
(FDIS)
1998
(Interim Standard)
IEEE
1220
2002
1998
IEEE
1220
(Full Std)
EIA/IS
731
SE CM
(Interim Standard)
2002
CMMISE/SW/IPPD
Legend
Supersedes
Source for
9
Standards for Systems Engineering, Jerry Lake, 2002
Process Improvement Workshop
Breadth and Depth of Key SE Standards
Process
description
High level
practices
ISO/IEC 15288
EIA/ANSI 632
IEEE 1220
Level of detail
System life
Detailed
practices
Conceptualize
Input to 632/1220
Develop
Transition to
Operation
Operate,
Maintain,
or Enhance
Replace
or Dismantle
• ISO/IEC 15288 - Common framework for describing the life cycle of systems
• EIA/ANSI 632 - Integrated set of fundamental processes to aid a developer in
the engineering or re-engineering of a system
• IEEE 1220 - Standard for managing systems engineering
10
Draft Report ISO Study Group May 2, 2000
Process Improvement Workshop
SEI’s Strategic Classification Taxonomy
Summary of the SEI approach of harmonizing multiple models, by Jeannine Siviy and Pat Kirwan, 2008
PrIME Workshop, http://www.sei.cmu.edu/prime/hardquestionsoutput.html
11
Process Improvement Workshop
Problem/Solution Space
CMMI
• Benchmark with community;
demonstrate process
capability to customers
• Learn new practices
• Establish infrastructure
• Encourage consistency across
projects
Lean Six Sigma
Agile
• Prioritize improvements by
business value
• Root cause analysis
• Statistical analysis
• Identify/implement focused
improvements
• Improve quality, cost, timeto-market
• Reduce waste
12
Process Improvement Workshop
Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
– Agile
• Northrop Grumman Approach
13
Process Improvement Workshop
What is the Capability Maturity Model
Integrated?
• The CMMI is a collection of industry best-practices for engineering,
services, acquisition, project management, support, and process
management
– Developed under the sponsorship of DoD
– Consistent with DoD and commercial standards
• CMMI for Development - used by engineering organizations
– Version 1.0 released in 2000, v1.1 in Mar 2002, v1.2 (CMMI-DEV) in Aug 06
• CMMI for Acquisition - used by buyers (e.g., govt. agencies)
– Version 1.0 released in 2007
• CMMI for Services - used by service providers (e.g., help desk)
– Under development
14
Process Improvement Workshop
The Basic Building Blocks of CMMI –
22 Process Areas
Implemented by
the organization
Implemented by
each project
Project
Management
– Project Planning
– Project Monitoring
and Control
– Supplier
Agreement
Management
– Integrated Project
Management)
– Risk Management
– Quantitative
Project
Management
15
Engineering
Support
– Requirements
Development
– Requirements
Management
– Technical Solution
– Product
Integration
– Verification
– Validation
– Configuration
Management
– Process and
Product Quality
Assurance
– Measurement and
Analysis
– Decision Analysis
and Resolution
– Causal Analysis
and Resolution
Process Improvement Workshop
Process
Management
– Organizational
Process Focus
– Organizational
Process Definition
– Organizational
Training
– Organizational
Process
Performance
– Organizational
Innovation and
Deployment
Expected Practices Provide Guidance
for Implementation & Institutionalization
Project Planning - Implementation
SG 1 Establish Estimates
SP 1.1 Estimate the Scope of the
Project
SP 1.2 Establish Estimates of Work
Product and Task Attributes
SP 1.3 Define Project Life Cycle
SP 1.4 Determine Estimates of Effort
and Cost
SG 2 Develop a Project Plan
SP 2.1 Establish the Budget and
Schedule
SP 2.2 Identify Project Risks
SP 2.3 Plan for Data Management
SP 2.4 Plan for Project Resources
SP 2.5 Plan for Needed Knowledge and
Skills
SP 2.6 Plan Stakeholder Involvement
SP 2.7 Establish the Project Plan
SG 3 Obtain Commitment to the Plan
SP 3.1 Review Plans that Affect the
Project
SP 3.2 Reconcile Work and Resource
Levels
SP 3.3 Obtain Plan Commitment
16
Project Planning - Institutionalization
GG 2 Institutionalize a Managed Process
GP 2.1 Establish an Organizational
Policy
GP 2.2 Plan the Process
GP 2.3 Provide Resources
GP 2.4 Assign Responsibility
GP 2.5 Train People
GP 2.6 Manage Configurations
GP 2.7 Identify and Involve Relevant
Stakeholders
GP 2.8 Monitor and Control the Process
GP 2.9 Objectively Evaluate Adherence
GP 2.10 Review Status with Higher
Level Management
GG 3 Institutionalize a Defined Process
GP 3.1 Establish a Defined Process
GP 3.2 Collect Improvement Information
Process Improvement Workshop
Practice Ratings for the
Organization/Projects
17
Process Improvement Workshop
How is the CMMI Used for Process
Improvement?
IDEAL
Model
www.sei.cmu.edu/ideal/
18
Process Improvement Workshop
Organizational Infrastructure Required for
CMMI Level 3
Process Group
Training Program
Measurement Repositories
Predictive Modeling
Best-Practice Libraries
Audits & Appraisals
Defects per component
Policies, Processes,
Templates & Tools
Process Improvement
Communications
25
20
15
UCL
10
_
X
5
0
1
11
21
31
41
51
61
71
Component #
Developing and maintaining mature processes requires
significant time and investment in infrastructure
19
Process Improvement Workshop
Benefits
• The typical benefits are:
–
–
–
–
–
Reduced cost
Faster schedules
Greater productivity
Higher quality
Increased customer satisfaction
• Benefits are cited in many forums
– DoD DACS website: www.thedacs.com/databases/roi/
– “Demonstrating the Impact and Benefits of CMMI: An Update and
Preliminary Results,” Software Engineering Institute, CMU/SEI-2003SR-009, Oct 2003
20
Process Improvement Workshop
Typical CMMI Benefits Cited in Literature
• Reduced costs
• Greater Productivity
– 33% decrease in the average cost
to fix a defect (Boeing)
– 20% reduction in unit software
costs (Lockheed Martin)
– 25-30% increase in productivity
within 3 years (Lockheed Martin,
Harris, Siemens)
• Higher Quality
• Faster Schedules
– 50% reduction in release
turnaround time (Boeing)
– 60% reduction in re-work
following test (Boeing)
21
– 50% reduction of software
defects (Lockheed Martin)
• Customer Satisfaction
– 55% increase in award fees
(Lockheed Martin)
Process Improvement Workshop
Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
– Agile
• Northrop Grumman Approach
22
Process Improvement Workshop
What is Lean Six Sigma (LSS)?
• Lean Six Sigma is a
powerful approach to
improving the work we do
• LSS improvement projects
are performed by teams
• Teams use a set of tools
and techniques to
understand problems and
find solutions
23
Process Improvement Workshop
+
• Lean Six Sigma integrates
tools and techniques from
two proven process
improvement methods
What is Six Sigma?
• Six Sigma is a management philosophy based on meeting
business objectives by reducing variation
– A disciplined, data-driven methodology for decision making and
process improvement
• To increase process performance, you have to decrease
variation
Too early
Too late
Defects
Too late
Defects
Delivery Time
Spread of variation
too wide compared to
specifications
24
Too early
Reduce
variation
Delivery Time
Spread of variation
narrow compared to
specifications
Process Improvement Workshop
• Greater
predictability in
the process
• Less waste and
rework, which
lowers costs
• Products and
services that
perform better
and last longer
• Happier
customers
A General Purpose Problem-Solving
Methodology: DMAIC
Problem or goal statement (Y)
Define
Measure
• Refine problem & goal
statements.
• Define project scope &
boundaries.
25
Analyze
Improve
Control
• An improvement journey to achieve goals and
resolve problems by discovering and understanding
relationships between process inputs and outputs,
such as
Y = f(defect profile, yield)
= f(review rate, method, complexity……)
Process Improvement Workshop
DMAIC Roadmap
Define
Measure
Analyze
Control
Define
project
scope
Identify
needed
data
Explore
data
Identify
possible
solutions
Define
control
method
Establish
formal
project
Obtain
data set
Characterize
process &
problem
Select
solution
Implement
Evaluate
data quality
Summarize
& baseline
data
Update
improvement
project scope
& scale
Document
Implement
(pilot as
needed)
Evaluate
Phase Exit Review
[Hallowell-Siviy 05]
26
Improve
Process Improvement Workshop
DMAIC Toolkit
Define
Benchmark
Contract/Charter
Kano Model
Voice of the
Customer
Voice of the
Business
Quality Function
Deployment
Measure
GQIM and
Indicator
Templates
Data Collection
Methods
Measurement
System
Evaluation
Analyze
Improve
Control
Cause & Effect
Diagrams/ Matrix
Design of
Experiments
Statistical
Controls:
Failure Modes &
Effects Analysis
Modeling
Control
Charts
Statistical
Inference
Tolerancing
Reliability
Analysis
Root Cause
Analysis,
including 5 Whys
Hypothesis Test
ANOVA
Time Series
methods
Robust Design
Systems
Thinking
Decision & Risk
Analysis
PSM Perform
Analysis Model
Non-Statistical
Controls:
Procedural
adherence
Performance
Mgmt
Preventive
measures
27
Process Improvement Workshop
Design for Six Sigma (e.g., DMADV)
Define
Define
project
scope
Establish
formal
project
Measure
Identify
customers
Research
VOC
Benchmark
Analyze
Explore
data
Design
solution
Predict
performance
Quantify
CTQs
28
Process Improvement Workshop
Design
Verify
Develop
detailed
design
Evaluate
pilot
Refine
predicted
performance
Develop
pilot
Scale-up
design
Document
What is Lean?
• Series of tools and techniques refined by Toyota and called
the “Toyota Production System”
– Called “Lean” by Womack, Jones and Roos in The Machine That
Changed the World
• Focused on increasing efficiency by eliminating non-value
added process steps and wasteful practices
• Being adopted world-wide by both manufacturing and
transactional based organizations
• Utilizes tools like “Value Stream Mapping,” “Just in Time”
and “Kaizen”
LEAN FOCUS: ELIMINATE WASTE AND REDUCE CYCLE TIME
29
Process Improvement Workshop
Wastes in Production
CORRECTION
•
WAITING
Repair or
Rework
Any non-work time
waiting for tools,
supplies, parts, etc..
PROCESSING
Doing more work than
is necessary
MOTION
Any wasted motion
to pick up parts or
stack parts. Also
wasted walking
Types
of
Waste
INVENTORY
OVERPRODUCTION
Producing more
than is needed
before it is needed
CONVEYANCE
Maintaining excess
inventory of raw mat’ls, Wasted effort to transport
materials, parts, or
parts in process, or
finished goods into or
finished goods.
out of storage, or
between
processes.
30
Process Improvement Workshop
The Hidden Process
Returns
Supplier
Input
Process
Inspect
and/or
Sign
Output
Not
sure
Second
look
Reprocess
Scrap
No
Re-route
OK
Reprocess
31
Customer
Process Improvement Workshop
Return or
Reprocess
Return
To
process
The
Hidden
Process
Organizational Adoption:
Roles & Responsibilities
• Champions – Facilitate the leadership,
implementation, and deployment
• Sponsors – Provide resources
• Process Owners – Responsible for the processes being
improved
• Master Black Belts – Serve as mentors for Black Belts
• Black Belts – Lead major Six Sigma projects
– Typically requires 4 weeks of training
• Green Belts – Lead minor Six Sigma teams, or serve on
improvement teams under a Black Belt
– Typically requires 2 weeks of training
32
Process Improvement Workshop
A Typical Lean Six Sigma Project
in Aerospace
The organization notes that systems integration has been problematic
on past projects (budget/schedule overruns)
A Six Sigma team is formed to scope the problem, collect data from
past projects, and determine the root cause(s)
The team’s analysis of the historical data indicates that ineffective peer
reviews are leaving significant errors to be found in test
Procedures and criteria for better peer reviews are written, using best
practices from past projects
A pilot project uses the new peer review procedures and criteria, and
collects data to verify they solve the problem
The organization’s standard process and training is modified to
incorporate the procedures and criteria, to prevent similar problems on
future projects
33
Process Improvement Workshop
Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
– Agile
• Northrop Grumman Approach
34
Process Improvement Workshop
The Manifesto for
Agile Software Development
“We are uncovering better ways of developing software by doing it
and helping others do it. Through this work we have come to value:
Individuals and interactions
over
processes and tools
Working software
over
comprehensive documentation
Customer collaboration
over
contract negotiation
Responding to change
over
following a plan
That is, while there is value in the items on the right, we value the
items on the left more.”
- Kent Beck et al
35
Process Improvement Workshop
Definition of Agile
• Agile is an iterative and incremental (evolutionary)
approach to software development which is performed in a
highly collaborative manner with "just enough" ceremony
that produces high quality software which meets the
changing needs of its stakeholders.
http://www.agilemodeling.com/essays/agileSoftwareDevelopment.htm
36
Process Improvement Workshop
What is “Agility”?
• Effective (rapid and adaptive) response to change
• Effective communication among all stakeholders
• Drawing the customer onto the team
• Organizing a team so that it is in control of the work
performed
Yielding …
• Rapid, incremental delivery of software
37
Process Improvement Workshop
An Agile Process
• Is driven by customer descriptions of what is required
(scenarios)
• Recognizes that plans are short-lived
• Develops software iteratively with a heavy emphasis on
construction activities
• Delivers multiple ‘software increments’
• Adapts as changes occur
38
Process Improvement Workshop
Agile is Supported by Many Methodologies
Extreme
Programming
(XP)
• Based on values of
simplicity,
communication,
feedback, courage,
and respect
• Start with simple
solution, add
complexity through
refactoring
• Frequent feedback
through unit,
integration, and
acceptance testing
• 4 dev. Phases:
coding, testing,
listening, designing
39
SCRUM
Crystal
• Small teams of 6-8
people
• “Backlog” defined
requirements that will
be addressed in each
Sprint
• Daily 15 min.Scrum
meeting to discuss
work for the day
• Divide projects into 30
day “Sprints”
• Review conducted at
end of each Sprint to
review progress and
revise backlog
• Frequent delivery
• Reflective
improvement
• Close communication
w/personal safety
• Access to expert
users
• Automated testing
• Frequent integration
• Configuration
management
Agile Unified
Process
Adaptive
• Repeating Speculate,
Collaborate, Learn
cycles
• Provides for
continuous learning
and adaptation to
changing project state
• Simplified version of
RUP – reduced
number of disciplines
• Consists of 4 RUP
phases (Inception,
Elaboration,
Construction,
Transition)
Process Improvement Workshop
Dynamic
Systems
Development
Method
(DSDM)
• 3 primary phases:
Pre-Project, Project
Life-Cuc;e , PostProject
• Project Life-Cycle
consists of Feasibility
Study, Business
Study, Functional
Model Iteration,
Design/Build Iteration,
and Implementation
Feature Driven
Dev.
• More value on design
then the “code is the
design”
• Model-driven
• Develop feature list
• Plan, Design, Build by
Feature
Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
– Agile
• Northrop Grumman Approach
40
Process Improvement Workshop
Northrop Grumman Approach:
Mission Success Requires Multiple Approaches
Risk Management
Systems Engineering
Independent Reviews
& Cost Estimates
Dashboards for
Enterprise-Wide
Measurement
Program
Effectiveness
Training, Tools, &
Templates
Process
Effectiveness
Mission
Assurance &
Enterprise
Excellence
Communications &
Best-Practice Sharing
Operations
Effectiveness
CMMI Level 5 for
Software, Systems, and
Services
ISO 9001 and AS-9100
Certification
41
Six Sigma
Process Improvement Workshop
Robust Governance Model
(Policies, Processes,
Procedures)
Northrop Grumman Approach:
Institutionalizing Our Improvements
We systematically analyze quality and
process data and trends to determine
how to improve our processes
We improve our process assets
based on internal and external best
practices
Disposition
Analysis
ISO/AS9100
Findings
CMMI Appraisal
Findings
• Systems/
Software
Engineering
Process Group
Customer
Comments
• QMS Working
Group
Independent
Audits
Lessons Learned
& Metrics
Tools
42
Increasing
program
efficiency
• Program
Management
Advisory Board
Industry
Standards
Internal
Best Practices
Configuration
Control Board
Information
Deployed
to
programs
Policy
Process
Six Sigma
Projects
Procedures
External
Best Practices
Checklists and
Guides
Templates and
Examples
eToolkit
StartIt!
msCAS
My MS Portal
Process Improvement Workshop
PAL
Workbench
PCDB
Northrop Grumman Approach:
Lessons Learned
Based on over 20 Northrop Grumman CMMI Level 5 and Lean Six Sigma organizations
• Multiple improvement initiatives helps encourage a change in behavior
as opposed to “achieving a level”
– Reinforces that change (improvement) is a way of life
• The real ROI comes in institutionalizing local improvements across the
wider organization
– CMMI establishes the needed mechanisms
• CMMI and Lean Six Sigma compliment each other
– CMMI can yield behaviors without benefits
– Lean Six Sigma improvements based solely on data may miss innovative
improvements (assumes a local optimum)
• Training over half the staff as Lean Six Sigma Green Belts has resulted in
a change of language and culture
– Voice of Customer, data-driven decisions, causal analysis, etc.
– Better to understand/use tools in everyday work than to adopt the “religion”
43
Process Improvement Workshop