Download ICSE04 - Department of Computer Science

The Impact of Software Reuse and
Incremental Development on the Quality
of Large Systems
Parastoo Mohagheghi
Dept. Computer and Information Science (IDI)
University of Science and Technology (NTNU)
Trondheim, Norway
[email protected]
Doctoral thesis presentation, 21 September 2004
1
Parastoo Mohagheghi- 21 Sept.2004
Motivation of the study
• Software organizations (and researchers) have always been
looking for effective strategies to develop software faster,
better and cheaper:
– OO analysis, CASE tools, formal methods, software reuse, agile
methods etc. are proposed, but there is no “silver bullet”.
– There is an absence of models and theories that can help to reason
about the software without building it.
– Rapid evolution of technologies does not allow proper assessment.
– Collecting context-independent knowledge or representing context
is difficult.
– Software development is inherently complex, especially for large
systems.
• There is a lack of empirical studies in industry on
technologies that can answer what they are good at, and
not so good at.
• The INCO- INcremental and COmponent-based Software
Development project (2001 to 2004) tries to advance the
state-of-the-art and practice for these technologies.
2
Parastoo Mohagheghi- 21 Sept.2004
Definitions
• “Software reuse is the systematic practice of developing
software from a stock of building blocks..” [Morisio,2002].
– Reusable assets can take many forms: Component libraries, COTS
and OSS components, or entire software architectures and
components in a product family approach.
– A reusable component in this study is a component that is reused in
more than one product.
– Two distinct activities: development for and with reuse.
• Component-Based Development (CDB) covers all the
aspects of developing (reusable, executable?) software
components and assembling system from them.
Components adhere to a component model.
• Incremental development is covering (or discovering)
requirements gradually and developing systems in
increments that accumulate functionality. Iterative
development is recursive application of development
activities and recursive elaboration of artifacts.
3
Parastoo Mohagheghi- 21 Sept.2004
Company background
• Ericsson has developed two large-scale telecom systems
(470 KSLOC or 1000 KSLOC in equivalent C each) that
share software architecture, components and a component
model, development environment and software process.
• Systems are developed incrementally (5 releases of one
system and 2 of the other). In periods, over 200 developers
in different countries were working with design and test.
• Lots of data has been gathered in 5 years in different
repositories. Data from several releases are analyzed in this
study. It covers trouble reports, change requests, effort,
size of components, measures from internal measurement
programs and inspections.
• Personnel turnover has traditionally been small, but
software development in Norway stopped in mid-2002.
4
Parastoo Mohagheghi- 21 Sept.2004
Overview of the Ericsson packetswitched core network- GPRS
MSC/
VLR
SMS-GMSC
HLR SMS-IWMSC
EIR
Other
networks
IP
network
SGSN
Backbone
network
GGSN
BSC/
RNC
Packet-switched
core network
Other
SGSN
BGW
5
Parastoo Mohagheghi- 21 Sept.2004
Evolution of the GSN software architecture
and the software process model
The original architecture
The hierarchical architecture
Rev. &
Re-eng.
GPRS for WCDMA
GPRS for GSM
3’ GPRS
4 for GSM
1’
GPRS for GSM
1
4
2
5
3
6’
6
Wireless Packet Platform (WPP)
5’
2
8
7
Wireless Packet Platform (WPP)
Adaptation of RUP
Initial Process
6
Parastoo Mohagheghi- 21 Sept.2004
Adapted RUP
Application-specific
layer
Business-specific
layer
Common services Reused
Layer (including
component framework)
Characteristics of the system
• The approach to initiate a product family has been an
extractive one and involved high risks regarding cost,
quality, training, coordination and management support.
• Quality requirements: Performance, availability, scalability
and maintainability (or evolvability) are of great
importance. There are no direct user interfaces.
• Large systems face challenges in all phases of
development that small systems do not: difficulties in
iteration planning, complexity of design, integration and
test and maintenance costs. Large systems are designed to
be long-lived.
• Components are developed in-house by different Ericsson
organizations.
7
Parastoo Mohagheghi- 21 Sept.2004
Research questions
• RQ1. Why is a reuse program initiated and how is it
implemented?
• RQ2. What is the impact of software reuse, CBD and
incremental development on the quality? The impact of
development approaches on product quality metrics and on
project attributes such as schedule or effort are sought.
• RQ3. How to improve the practice of incremental
development of product families in some aspects?
• Research questions are further refined in studies.
– Some research questions and hypotheses were identified from
earlier work (top-down style) -> replication in new context and
confirmatory studies.
– Other questions and hypotheses are results of exploratory work on
available data and practices in the industry, in a bottom-up style ->
explorative and descriptive studies.
8
Parastoo Mohagheghi- 21 Sept.2004
Research methods
• Quantitative studies by exploring (‘data mining’) company
databases, studying distributions and assessing hypotheses.
• Qualitative data collected from web pages, textual
documents, discussions with personnel and own
experience on the process and practices are combined with
results from quantitative studies.
• Several case studies (post-mortem), a small survey and a
controlled experiment are performed.
• The rationale for mix of methods has been:
– The impact of introducing reuse or incremental development is
widespread.
– It is useful to take benefit of all available data.
– Triangulation of data/methods.
– Answering questions that are not possible to answer otherwise.
9
Parastoo Mohagheghi- 21 Sept.2004
Overview of all studies
Phase 1
P1
P7
Phase 2
Combining results in Phase 3
C1
P8 Study of
Trouble Reports
2003
Study of
Reuse Practice
2002
C6b
P11 Developing Data
Mining Method
2004
C1
C2
P10
Study of
Change Requests
2003
Experiment on
Inspection
P4
2002
P2 P9
P6
Survey
2002
Study of Software
Process & RUP
2001-2002
March 2001
10
P12
C3
P11 Assessing the Impact of
Dev. Approaches &
Identifying Metrics
2003-2004
Study of
Effort
2003-2004
Developing Effort C3
Estimation Method
2003-2004
P13
Prototype
Study of MDA
P3
2003
Quantitative study
Qualitative study
C4
P5
C6a
C5
July 2004
P
Paper
C
Contribution
Parastoo Mohagheghi- 21 Sept.2004
Input
Contributions and research questions
Group
Contribution
Reuse
C1. Empirical verification of reuse benefits.
RQ1 RQ2 RQ3
√
√
Increme C2. Increased understanding of the origin and
ntal dev. type of changes.
Increme C3. Developing an effort estimation method
ntal dev.
√
Both
C4. Identifying metrics for a combination of
reuse and incremental development.
√
Method
C5. Developing a data mining method.
√
Reuse
C6a. Adaptation of the Rational Unified
Process for reuse.
Increme C6b. Improving techniques for incremental
ntal dev. inspection of UML models.
11
Parastoo Mohagheghi- 21 Sept.2004
√
√
√
Contributions in software reuse
• C1. Empirical verification of reuse benefits (RQ2):
– Quantitative studies of Trouble Reports (TRs) and Change
Requests (CRs). 13 000 TRs and 169 CRs were analyzed.
– Reused components have significantly lower defect-density than
non-reused ones, and are more stable between releases (less
modified). Data from 3 releases.
– There is no difference in change-proneness between reused and
non-reused components (#CRs/KLOC). Data from 4 releases.
– First large-scale industrial study on reuse benefits.
• C6a. Adaptation of the Rational Unified Process (RUP)
regarding reuse (RQ1, RQ3):
– Proposing changes in workflows and activities for development for
and with reuse.
– No empirical studies have been found on evaluating or adapting
RUP in this aspect. Proposals are generic and may be reused in
other adaptations of RUP.
12
Parastoo Mohagheghi- 21 Sept.2004
Reuse and defect-density, release 3
Defect-density of blocks
Mean
Median
Variance
#TRs/KLOC, Reused
1.32
0.76
1.70
#TRs/KLOC, Non-Reused
3.01
2.44
4.39
#TRs/MKLOC, Reused
3.50
1.78
21.26
#TRs/MKLOC, Non-Reused
5.69
3.73
21.76
•H0: Reused components have the same defect-density as
non-reused ones. Rejected.
•Reused components have lower defect density, but they have
proportionally more defects with severity A (highest severity).
•Reused components have proportionally less defects
after delivery -> more reliable.
13
Parastoo Mohagheghi- 21 Sept.2004
Size vs. defect-density
8
7
#TRs/KLOC
6
5
Reused
4
3
Nonreused
2
1
0
0
5
10
15
25
20
30
35
40
45
KLOC
H0: There is no relation between defect-density and component
size for all/reused/non-reused components. Not rejected.
14
Parastoo Mohagheghi- 21 Sept.2004
Contributions in incremental development
• C2. Increased understanding of the origin and type of
changes in requirements or artifacts in incremental
development (RQ2):
– A quantitative analysis of CRs showed that perfective changes
(enhancements and optimizations) are most common. Of these,
non-functional changes to improve “quality attributes” are more
frequent than pure functional changes.
– Most CRs (66%) are issued by the organization itself (and not
external actors). Only 34% of CRS are adaptive or customerinitiated.
– Hypotheses are grounded in data. Generalization needs further
study.
Perfective/
Functional
CRs
15
Perfective/
Quality Attr.
40
Parastoo Mohagheghi- 21 Sept.2004
74
Adaptive
35
Preventive
30
Other
(cost)
8
Contributions in incr. dev.- cont.
• C3. Developing an effort estimation method using use
case specifications and effort distribution in different
phases of incremental software development (RQ3):
– The method was adapted for incremental changes in (large) use
cases. It also includes a factor to estimate effort for building on a
previous release.
– The impact of effort-breakdown profile on the estimation results is
discussed.
– Method adapted to the context and scaled up.
• C6b. Improving techniques for incremental inspection of
UML models (RQ3):
– A small experiment (first in industry) was performed, comparing
company’s inspection technique with the new and adjusted OORTs
(Object-Oriented Reading Techniques). The results showed no
difference in cost-efficiency, but difference in the types of detected
defects.
– Useful for improvement of OORTs and adapting these to the
context.
16
Parastoo Mohagheghi- 21 Sept.2004
Contributions in software reuse and incr.
dev., and research methods
• C4. Identifying metrics for a combination of reuse of
software components and incremental development
(RQ3):
– Other literature discusses metrics for CBD. However, these metrics
should be adapted for incremental development and for reuse.
• C5. A data mining method for exploring industrial data
repositories (RQ3):
– A data mining method is developed that is based on experience
from the quantitative studies. It combines top-down theory search
with bottom-up hypotheses generation.
– Challenges in integrating the results of several studies with one
another are classified into physical and conceptual ones.
17
Parastoo Mohagheghi- 21 Sept.2004
Assessing development approaches
Development Approaches
Practices
Product/process Quality
Requirement Modification +/-
Incremental &
Iterative
Development
Planning Precision
+/Incremental Delivery
Success
Solution Modification
Needed Effort
Software Reuse
Incremental Planning
Component Stability
+
Incremental Integration
Component-Based
Development
+
Reusable Artifacts/
Components
Leads to
18
Parastoo Mohagheghi- 21 Sept.2004
+
Reduced Defect-Density
Changeability
The Impact of development approaches
and context on practices
Reuse
Incremental dev.
CBD
Large
systems
Changeproneness
and
evolvabilit
y.
Increased change
rate,
Earlier releases
not evolved.
Component
granularity is
large for
CRs.
QA are
improved
Effort
ROI? No
estimation metrics.
Software from a
previous release
is evolved.
Incr. changes in
requirements,
Large CM.
No data for
effort spent
on
components.
Large CM
and testing
effort.
Data
collection
Data should be
collected for
increments.
Different
granularity.
No common
repository.
Software
evolution
19
Parastoo Mohagheghi- 21 Sept.2004
incrementally.
Long-lived.
Answers to research questions
• RQ1. Why a reuse program is initiated and how is it
implemented?
– A product family is initiated because of the similarity between
requirements of the emerging system and an existing system.
Software architecture is evolved.
– A lightweight or extractive approach to product family adoption
was chosen. Management support, common goals and common
infrastructure, and experienced personnel have been critical for the
success of reuse. The software process model is not adapted to the
same degree.
• RQ2. What is the impact of software reuse, CBD and
incremental development on the quality?
– Lower defect density and higher stability of reused components.
– Product is getting more change-prone (in %of accepted CRs and
reduced requirement stability). Quality is improved incrementally,
and functionality is both enhanced and improved. Earlier releases
are not evolved.
– High share of integration and testing effort.
20
Parastoo Mohagheghi- 21 Sept.2004
Answers to research questions- cont.
• RQ3. How to combine the qualitative and quantitative
results to improve the practice in some aspects?
– The effort estimation method, metrics for a combination of
development approaches, the research method for data mining, and
proposals for adapting RUP.
• Relation to INCO goals:
– G1. Advancing the state-of-the-art of software engineering. Better
understanding of approaches is achieved. Contributions C1, C2
and C4.
– G2. Advancing the state-of-the-practice in software-intensive
industry and for own students. Some feedback is given to Ericsson.
Contributions C3 to C6 are reusable.
– G4. Disseminating and exchanging the knowledge gained. Most
results are published. 11 students were involved in the studies.
Teaching in courses.
21
Parastoo Mohagheghi- 21 Sept.2004
Discussion
• Case study research:
– Studies are performed in a real context. Context-independent
knowledge is rare in SE and generalization is difficult.
– Affiliation in the company provided access to data (revelatory
case).
– The system is a critical case for verifying reuse benefits (extractive
approach, only two products and high risks).
– The size of the system allows evaluating techniques for large-scale
development.
– In some cases, an example of industrial practice, e.g. RUP.
• Incremental development is combined with reuse of
software in multiple products and CBD.
22
– All aspects of software development need adaptation; e.g.
estimation method, software process model, data collection
routines etc.
Parastoo Mohagheghi- 21 Sept.2004
Conclusions
1. Describing different aspects of software
development; i.e. the power of example.
–
The work also identified areas for improvement like
the process model and the data collection routines.
2. Verifying existing theories and assessing existing
methods in new contexts; i.e. the power of
replication:
–
–
–
23
Reuse benefits in a large industrial system,
Estimation method and inspection techniques in a
large project and with incremental development,
RUP in the context of reuse.
Parastoo Mohagheghi- 21 Sept.2004
Conclusions- cont.
•
Generating new theories, hypotheses or methods
by analyzing data from new perspectives (as in
grounded theory) or combining the results of
several studies; i.e. the power of generalization:
–
The origin of change requests and the distribution
over functional and quality attributes,
– The distribution of effort over development phases for
a large project with incremental development,
– The impact of development approaches on quality
attributes,
– Metrics for a combination of development approaches,
– A data mining method based on experience.
24
Parastoo Mohagheghi- 21 Sept.2004
Why such studies are important for
research community and industry?
• Methods should be tested in real context and for
development in large.
• Companies are increasingly using mainstream
methods, tools and programming languages.
Results are of interest to a larger community.
• The studies analyzed data that was hardly studied
before: evaluation of data collection routines and
metrics for the company, in addition to concrete
results.
25
Parastoo Mohagheghi- 21 Sept.2004
Future work
•
•
•
•
•
•
26
Study of RUP adaptations.
Empirical studies on software evolution.
Study of effort distribution.
Estimation of incremental projects
Use cases for effort estimation.
Relevant metrics for incremental development of
large systems.
Parastoo Mohagheghi- 21 Sept.2004
Acknowledgements
• Thanks to INCO, Simula and NTNU for financial
support.
• Thanks to my supervisor, Prof. Reidar Conradi,
and colleagues at INCO and IDI.
• Thanks to Ericsson for the permission to perform
studies and to publish the results.
27
Parastoo Mohagheghi- 21 Sept.2004