Download Software Configuration Management

Using UML, Patterns, and Java
Object-Oriented Software Engineering
Build Management
Outline of the Lecture
 Purpose of Software Configuration Management
(SCM)
 Some Terminology
 Software Configuration Management Activities
 Software Configuration Management Plan
• Build Management
• Continous Integration
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
2
The Problem
• The transition from source code to the
executable application contains many
mechanical steps:
• Settings required paths and libraries
• Compiling source code
• Copying source files (e.g. images, sound files, start
scripts)
• Setting of file permissions (e.g. to executable)
• Packaging of the application (e.g. zip, tar, dmg)
• Executing these steps manually is timeconsuming and the chance of introducing
failures is high
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
3
Requirements for Build Management
• Large and distributed software projects need to
provide a development infrastructure with an
integrated build management that supports:
•
•
•
•
•
Regular builds from the master directory
Automated execution of tests
E-mail notification
Determination of code metrics
Automated publishing of the applications and test
results (e.g. to a website)
• Tools for Build Management:
• Unix’s Make
• Ant
• Maven
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
4
Activities in Build Management
• The transition from source code to the
executable application contains many
mechanical (boring) activities:
• Settings required paths and libraries
• Compiling source code
• Copying source files (e.g. images, sound files, start
scripts)
• Setting of file permissions (e.g. to executable)
• Packaging of the application (e.g. zip, tar, dmg)
• Executing these steps manually is timeconsuming and the chance of introducing
failures is high
• Automating these steps has its origins in Unix
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
5
Development of Build Management
• 1950:
• Execution of a sequence of commands
• Job command language scripts
• 1970s: make
• Unix tool to support builds with makefiles
• 1990s: Ant
• Open source platform independent build tool
• 2000s: Maven
• Project management and integrated build tool
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
6
Example: Build Management in Unix
• The Unix command make executes compilation of
large programs according to a set of dependency
rules
• Dependency rule
• Tree structure of prerequisites
• Time stamps of files
Example of a dependency rule:
main.obj : main.c
gcc -c main.c
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
7
Example Makefile
project.exe : main.obj io.obj
tlink main.obj, io.obj, project.exe
main.obj : main.c
gcc -c main.c
io.obj : io.c
gcc - c io.c
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
8
Make
• Make is as powerful as the existing commands it
executes
• However, make has its problems:
• Complex shell scripts are hard to maintain and hard to
debug
• Different shell commands are needed on different
platforms
• Firs step towards platform independence: Ant
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
9
Ant, a Platform Independent Build
Management Tool
• Build tool based on Java, http://ant.apache.org
• Tasks get executed by invoking Java classes,
not shell scripts or shell commands
• Ant allows the execution of tasks such as:
•
•
•
•
•
compilation
execution
file system operations (e.g. copy, move)
archiving
deployment
• Ant has become a standard in the Java Open
Source community
• Ant build files are platform independent
• Ant’s build file is an XML document
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
10
General Structure of Ant Build Files
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
11
Examples of Ant Tasks
Each Ant Task is a Java application.
Custom tasks can be implemented in Java to support
custom needs.
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
12
Construction of Ant Build Files
• The default name for a Ant build file is build.xml
• The xml root element must be the ‘project’
element
• The ‘default’ attribute of the project element is
required and specifies the default target to use
• Targets contain zero or more AntTasks
• The ‘name’ attribute is required
• AntTasks are the smallest units of the build
process
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
13
Ant Build File Example
<project default="hello">
<target name="hello">
<echo message="Hello, World"/>
</target>
</project>
Execution
of build file:
Bernd Bruegge & Allen H. Dutoit
% ant
Buildfile: build.xml
hello: [echo] Hello, World
BUILD SUCCESSFUL
Total time: 2 seconds
Object-Oriented Software Engineering: Using UML, Patterns, and Java
14
Maven Overview
• A Java project management and integration
build tool, http://maven.apache.org/
• Maven supports the same functionality as Ant
• Ant is actually integrated into Maven
• Maven’s extended functionality
• Automatically generates a development website
containing
• Project organization information
• Project development state reports
• Based on the concept of a central project object
model (POM)
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
15
Maven Capabilities
• Maven is able to
• Receive developer artifacts from the programmers’
SCM directory
• Compile and package the source code
• Download dependent artifacts from another (external)
repository
• Generate Javadoc websites
• Generate developer and code metrics
• Generate websites containing status information to
track issues
• Deploy the compiled source code and the generated
website to a webserver
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
16
Modeling a Project in Maven
Project Object Model (POM)
Project name, id, description, version number
Website URL
Issue tracking URL
Mailing list URLs
SCM directory URL
Developer information
License information
Dependencies
• Name, version, location of a Java library from jar file
• Project directory layout
• Specifies source code and unit test code directories
•
•
•
•
•
•
•
•
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
17
Modeling a Project in Maven (UML)
Project Object Model
Project Descriptor
name
currentVersion
organization
inceptionYear
description
url
Build Descriptor
sourceDirectory
resources
Developer Descriptor
name
id
organization
roles
Bernd Bruegge & Allen H. Dutoit
Repository Descriptor
connection
developerConnection
url
Object-Oriented Software Engineering: Using UML, Patterns, and Java
18
Example:
Generating ARENA’s Development Website
• ARENA uses a SCM master
directory at
oose.globalse.org(*)
• ARENA depends on the
Java library servlet.jar
available in the Ibiblio
repository
• The ARENA website is
located on the server
unicase.in.tum.de
• Maven compiles the
ARENA sources in the
programmer’s directory of
the developer
(*)
unicase.in.tum.de:Server
Fictive server address
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
19
Modeling a Project in Maven (UML)
Project Object Model
Project Descriptor
name
currentVersion
organization
inceptionYear
description
url
Build Descriptor
sourceDirectory
resources
Developer Descriptor
name
id
organization
roles
Bernd Bruegge & Allen H. Dutoit
Repository Descriptor
connection
developerConnection
url
Object-Oriented Software Engineering: Using UML, Patterns, and Java
20
Project Descriptor for ARENA (in project.xml)
<project>
<!-- a unique name for this project -->
<id>arena</id>
<!-- a short but descriptive name for the project -->
<name>ARENA</name>
<currentVersion>0.9</currentVersion>
<!-- details about organization who 'owns' the project -->
<organization>
<name>Bernd Bruegge &amp; Allen H. Dutoit</name>
<url>http://wwwbruegge.in.tum.de/</url>
<logo>/images/ase_logo.gif</logo>
</organization>
<inceptionYear>2004</inceptionYear>
<logo>/images/arena.gif</logo>
<description>ARENA</description>
<!-- the project home page -->
<url>http://unicase.in.tum.de/arena</url>
<siteAddress>unicase.in.tum.de
</siteAddress>
<siteDirectory>/www/arena</siteDirectory>
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
21
The ARENA Website (Generated by Maven)
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
22
Team Members and Roles in project.xml
<developers>
<developer>
<name>Allen Dutoit</name>
<id>dutoit</id>
<organization>Technische Universität München
</organization>
<roles>
<role>Architect</role> <role>Developer</role>
</roles>
</developer>
<developer>
<name>Michael Nagel</name>
<id>nagel</id>
<organization>Technische Universität München
</organization>
<roles> <role>Developer</role>
</roles>
</developer>
<developer>
<name>Timo Wolf</name>
<id>wolft</id>
<organization>Technische Universität München
</organization>
<roles> <role>Developer</role>
</roles>
</developer>
</developers>
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
23
Generated Website contains a Description
of the Team
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
24
Software Configuration Management
Specification in project.xml
• The repository tag enables Maven to
• map the user ids used by the SCM tool to the
developer names listed in the Maven team site
• identify developers activities
• identify source code changes made by the developers
<repository>
<connection>
scm|svn|http|//oose.globalse.org/svn/trunk/examples/arena
</connection>
<developerConnection>
scm|svn|http|//oose.globalse.org/svn
</developerConnection>
<url>
http://oose.globalse.org/svn/trunk/examples/arena/
</url>
</repository>
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
25
Generated Website: Developer Activities
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
26
Generated Website: Change Log
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
27
Build Management in the project.xml
The Build tag specifies the programmer’s directory
containing the source code to be used for the
build
<build>
<sourceDirectory>src</sourceDirectory>
<resources>
<resource>
<directory>src</directory>
<includes>
<include>**/*.properties</include>
<include>**/*.gif</include>
</includes>
</resource>
</resources>
</build>
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
28
Result: Website contains the current source
code of ARENA
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
29
Outline of the Lecture
 Purpose of Software Configuration Management
(SCM)
 Some Terminology
 Software Configuration Management Activities
 Outline of a Software Configuration
Management Plan
 Build Management
• Continous Integration
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
30
Typical Problems and Risks in Software
Projects
• Lack of project visibility
• Nobody knows what is going on
• Late discovery of faults
• The later a fault is found, the more expensive it is to fix
it.
• Lack of deployable software
• “It worked on my machine”
• Also, horizontal integration testing strategies don’t
focus on an early running system. It is often created in
the last moment
• Low quality software
• Managers and developers make last-minute decisions to
deliver a system developed with the above problems.
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
31
Continuous Integration Motivation
• Risk #1: The later integration occurs in a
project, the bigger is the risk that unexpected
faults occur
• Risk #2: The higher the complexity of the
software system, the more difficult it is to
integrate its components
• Continous integration addresses these risks by
building as early as possible and frequently
• Additional Advantages:
• There is always an executable version of the system
• Team members have a good overview of the project
status
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
32
Definition Continuous Integration
Continuous Integration: A software
development method where members of a
team integrate their work frequently, usually
each person integrates at least daily, leading
to multiple integrations per day.
Each integration is verified by an automated
build including the execution of tests to detect
integration errors as quickly as possible.
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
33
Continuous Integration can regularly
answers these Questions
• Do all the software components work together?
• How much code is covered by automated tests?
• Where all tests successful after the latest
change?
• What is my code complexity?
• Is the team adhering to coding standards?
• Where there any problems with the last
deployment?
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
34
Modeling a Continuous Integration System
• Functions:
•
•
•
•
•
•
•
Set up scheduling strategy (poll, event-based)
Detect change
Execute build script when change has been detected
Run unit test cases
Generate project status metrics
Visualize status of the projects
Move successful builds into Software repository
• Components (Subsystems)
• Master Directory: Version control (IEEE: Controlled Library).
• Builder Subsystem: Execute build script when a change has been
detected
• Continuous Integration (CI) Server
• Management Subsystem: Visualize project status via Webbrowser
• Notification Subsystem: Publishes results of the build via different
channels (E-Mail Client, RSS Feed)
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
35
Analysis: Functional Model for Continuous
Integration
Simple Exercise: Develop
the functional model!
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
36
Analysis: Functional Model for Continuous
Integration
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
37
Design: Deployment Diagram of a
Continuous Integration System
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
38
Design of a Continuous Integration
System
• Development Node
•
•
•
•
Manage Programmer’s Directory (IEEE: Dynamic Library)
Software Configuration Management client
Integrated Development Environment
Run build script locally using the Builder Subsystem
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
39
Design of a Continuous Integration System
(Deployment Diagram)
• Software Configuration Management Node
• Contains the Master Directory (IEEE: Controlled Library)
• Runs the Software Configuration Management Server
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
40
Design of a Continuous Integration
System (Deployment Diagram)
• Integration Build Node
•
•
•
•
Creates & maintains the Software Repository (IEEE: static library)
SCM Client interacts with SCM Node if (when) a change has occurred
Runs build globally using Builder when a change has been detected
Notification component publishes results of the build
• uses different channels such as E-Mail or RSS Feed
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
41
Design of a Continuous Integration
System (Deployment Diagram)
• Management Node
• Visualize build results
• Visualize project metrics
• Receive notification about build status
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
42
Examples of Available
Continous Integration Systems
•
•
•
•
•
Cockpit
CruiseControl and CruiseControl.NET
Anthill
Continuum
Hudson
Used in our next
Exercise and Homework
List of continuous integration tools:
http://confluence.public.thoughtworks.org/display/
CC/CI+Feature+Matrix
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
43
CruiseControl: Subsystem Overview (not in
UML!)
Source: http://cruisecontrol.sourceforge.net/overview.html
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
44
Cruise Control: The Build Loop
• The daemon process checks for source code
changes in the projects
• If a change has occurred, the subsystem
executes the following actions:
• Obtain all the build artifacts (described in
“config.xml”)
• Possibly via the project.xml file (if Maven is used)
• Execute the Build file
• Write the results into a log file (“xml logs”)
• Send notifications to subscribers.
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
45
Cruise Control: Reporting Module and
Dashboard
• Reporting: A Java Server Page (JSP) based web
page
• Reads the XML log file
• Gives access to the build artifacts
• Shows build & test results
• Dashboard: An Ajax based web page
• Gives overview of projects on the cruise control server
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
46
CruiseControl Directory Layout I
Build artifacts
Main
configuration
file
Ant binaries
Scripts for starting the
cruisecontrol server
Dashboard
configuration file
Log files
Projects to be
build
Bernd Bruegge & Allen H. Dutoit
Files needed
for operation
Object-Oriented Software Engineering: Using UML, Patterns, and Java
47
CruiseControl Directory Layout II
Project folders
Timestamp named
folders for build artifacts
Project artifacts of a
specific build
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
48
CruiseControl Webpages
http://localhost:8080/cruisecontrol/
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
49
CruiseControl Webpages
http://localhost:8080/dashboard/
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
50
Getting Started with CruiseControl
• Download CruiseControl binary distribution
• http://cruisecontrol.sourceforge.net/download.html
• Unzip downloaded archive
• Run cruisecontrol.sh (.bat)
• Watch the example project being built
• http://localhost:8080/cruisecontrol
• Modify config.xml to include your own project
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
51
Ant References
http://ant.apache.org
http://codefeed.com/tutorial/ant_intro.html
http://blog.ideoplex.com/software/java/
http://www.iseran.com/Java/ant/tutorial/ant_tu
torial.html
• http://www.javaworld.com/javaworld/jw-102000/jw-1020-ant.html
•
•
•
•
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
52
Maven References
• http://maven.apache.org/
• http://www.sandcastsoftware.com/articlesandtu
torials/index.html
• http://www106.ibm.com/developerworks/java/library/jmaven/
• http://www.jpox.org/docs/1_1/tutorials/maven.
html
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
53
Further Reading
[Duvall 2007]
• Paul M. Duvall, Steve Matyas and Andrew Glover:
Continuous Integration. Improving Software Quality
and Reducing Risk, Addison Wesley, Upper Saddle
River, NJ, 2007
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
54
Backup Slides
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
55
Continous Integration Exercise
• Goal: Being able to set up and manage continuous integration
• Team-based organization: The exercise is team-based
• We recommend a team size of 5 people (4-6)
• Choose a team leader
• Equipment:
• Bring at least 2 laptops per team to the class room
• Preparations (Mandatory, by midnight)
• E-Mail the team member names to [email protected]
• Set up Eclipse on at least one of the team laptops
• Check out and study the Bumpers project using the information
available on the exercise portal :
https://wwwbruegge.informatik.tumuenchen.de/twiki/bin/view/Lehrstuhl/POMSS09Exercises
• Additional details
• Will be distributed at the beginning of the exercise session.
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
56
Deployment in Continous Integration
Exercise
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
57
Examples of concrete Ant Tasks
Each Ant Task is a Java application.
Custom tasks can be implemented in Java to support
custom needs.
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
58
Another Ant Example
<project name="Asteroids" default=”compile" basedir=".">
<target name="compile" >
<mkdir dir="classes"/>
<javac srcdir="src" destdir="classes” />
<copy todir="classes/org/globalse/oose/asteroids">
<fileset dir="src/org/globalse/oose/asteroids">
<include name="**/*.gif"/>
<include name="**/*.wav"/>
<exclude name="**/*.java"/>
</fileset>
</copy>
</target>
<target name="jar" depends="compile">
<jar destfile=”Asteroids.jar” basedir=”classes” />
</target>
</project>
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
59
Another Ant Example (2)
% ant compile
Buildfile: build.xml
compile:
[mkdir] Created dir: /Users/wolft/Asteroids/classes
[javac] Compiling 12 source files to
/Users/wolft/Asteroids/classes
[copy] Copying 6 files to
/Users/wolft/Asteroids/classes/org/globalse/oose/asteroids
BUILD SUCCESSFUL
Total time: 6 seconds
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
60
Another Ant Example (3)
% ant jar
Buildfile: build.xml
compile:
jar:
[jar] Building jar: /Users/wolft/Asteroids/Asteroids.jar
BUILD SUCCESSFUL
Total time: 2 seconds
Bernd Bruegge & Allen H. Dutoit
Object-Oriented Software Engineering: Using UML, Patterns, and Java
61