Download Document

GRUPPO TELECOM ITALIA
ETSI TTCN-3 User Conference and Model Based Testing Workshop
11-14 June, 2012 – Bangalore, INDIA
MONET (MObile NEtwork Tester)
Telecom Italia experience
Telecom Italia LAB / T.TG.TL.AP
SERAZIO Luca
Index
►
►
►
►
►
►
►
►
►
►
►
Testing Lab and Telecom Italia’s Test Plant
Testing areas and objectives
Testing automation
Testing Methodology
The MoNeT solution
Test Scenarios
A test scenario
Monet history
System framework
Economics
Next steps
Telecom Italia LAB / T.TG.TL.AP
SERAZIO Luca
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Telecom Italia Group
Telecom Italia offers technological infrastructures and platforms in which voice
and data are converted into advanced telecommunications services - as well as
the latest ICT and Media solutions. These tools cater for the Group's as well as
the country's growth.
Our presence and role in the Italian market
Key numbers - First Half 2011
» 31.3 million mobile lines
» 9.1 million broadband accesses, of which 1.9 million
wholesale customers
» 15 million retail network connections to the fixed network
in Italy, of which 7.2 million retail broadband accesses
» 3.65% daily average audience share by La7
» 49.3 page views and 4.1 million single visitors to Virgilio
(at March 31, 2011)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
3
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Telecom Italia in the world
Telecom Italia in the world
Italy, Brazil and Argentina are the strategic markets where the
Telecom Italia Group operates.
The leadership of our Group at national level originates from an
exhaustive offer aimed at transforming traditional communication
into digital communication: activities carried out through
companies operating on the domestic market range from land to
mobile telephony, internet, research and development, office
products and services, television and entertainment, media and
ICT.
The Group is present in Brazil with TIM Brazil, one of the most
important players because of the quantity of mobile lines. In
Argentina the Group operates through Telecom Argentina in the
sectors of land and mobile lines, internet and data transmission.
Telecom Argentina also offers mobile telephony services in
Paraguay.
The Telecom Italia Group is also active in other parts of Europe,
North America, Africa and Asia through Sparkle, Lan Nautilus and
Med Nautilus, Telecom Italia’s local operational companies.
The Group can be accessed throughout the world by contacting
Telecom Italia and Telecom Italia Sparkle’s representative offices.
Our presence and role in Latin America
Abroad Group's focus is on Latin America. It operates in Argentina and Paraguay,
providing fixed and mobile telephony services and internet through Telecom Argentina
group. In Brazil, TIM Brasil remains one of the major players.
Key numbers - First Half 2011
Brazil » 55.5 million mobile lines » 25.5% of market share
Argentina
» 4.1 million fixed lines » 1.5 million broaband accesses » 17.4 million mobile
customers
Paraguay
» 2 million mobile lines
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
4
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Testing Lab
Testing Lab is responsible of the maintenance and the management of test
laboratories such as the Mobile Network Lab used for the validation of mobile
network platforms and architectures, terminals and services.
A testing phase for each service development step
Services development phases are paired with testing in order to reduce the
The service development steps are paired with corresponding testing phases, in order
impact and the probability
faults in
older
phases
of the on
process.
to ensure a smoothof
development
process
and to
prevent malfunctions
the
Testing Labs
23 June 2010
•
launched services
Service Development
Service
Requirements
Definition
Service
Usability
Verification
RFI and
RFQ process
Credibility
Tests
Network
Solution
Development
Network
Integration
Solution
Tuning
PreDeployment
Service
Launch
Lab Tests in
stand-alone
configuration
E2E Lab
Tests
Friendly
Users Trial
Tests
in field
Tests by
Troubleshooting
Testing
Testing Labs involvment
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
5
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Telecom Italia’s test plant: an overview
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
6
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Telecom Italia’s test plant:
the numbers of the core network
►
2 HLRs
►
5 MSCs
►
5 MGWs
►
3 SGSNs
►
3 GGSNs
►
►
Several service nodes in continuous configuration and adaptaion to services
(SMSC, TGDS, INS, IVR, …)
Interconnecting IP backbone
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
7
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Testing areas and objectives
►
Verification and validation of:
►
New releases, patches and data
►
New network elements and new hardware
►
New network architectures
►
Regression testing
►
Network analysis under traffic conditions
►
Toll Ticketing validation
►
►
Test plant chains control using an end-to-end approach oriented to the final
user
Testing remotization
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
8
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Testing automation
Test automation is the use of software to control the execution of tests, the
comparison of actual outcomes to predicted outcomes, the setting up of test
preconditions, and other test control and test reporting functions. Commonly,
test automation involves automating a manual process already in place that
uses a formalized testing process.
Automation benefits
►
Reduction of time and costs for test execution and verdict evaluation
►
Avoiding repetitive manual activity which is prone to error
►
Saving costs related to human experts and test bed occupation
►
Reduction of time to market for services, products and network
architectures
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
9
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Testing Methodology
The testing methodology applied is End-to-End testing with protocol
conformance checks at the main SUT interfaces which is aimed to ensure the
delivery of service to the final user with the correct communication between
network elements.
SUT
USER
A
EUT
A
EUT
B
USER
B
Interface conformance checks
E2E functionality
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
10
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
The MoNeT solution
►
►
►
MoNeT (MObile NEtwork Tester) is an automatic tool for the testing of the
Mobile CS Core Network
The tool is connected directly to the mobile core Network by network
interfaces: A interface for GSM and Iu-CS interface for UMTS and acts over
these interface with the same signalling generated by an end user over the
radio interface
The platform offers, from a web based remote GUI:
►
A tool for the automatic execution of test lists and test scenarios
►
A virtual mobile terminal with all its functionalities
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
11
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
The MoNeT solution: test plant deployment
S
U
T
.
MONET 2G
MONET 3G
MONET
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
12
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Test Scenarios
►
►
Basic call
Supplementary Services
►
►
►
►
►
►
►
►
►
►
►
►
►
►
►
►
►
►
►
CFB Call Forwarding on Busy
CFU Call Forwarding Unconditional
CFNRy, Call Forwarding no Reply
CFNRc, Call Forwarding no Reachable
CW Call Waiting
CH Call Holding
MPTY Multi Party Service
CLIP Calling Line Identification Presentation
CLIR Calling Line Identification Restriction
COLP Connected Line Identification Presentation
COLR Connected Line Identification Restriction
Call Barring
Emergency call
Narrow Band and Wide Band call
SMS
Unsuccesful scenarios
Multi toll ticketing
Intelligent Network Services (TIM custom VAS services: TCA, TIT, RIF, I&B, 2in1, PN, Premium rate,
PagoIO, SOS PayForMe),
Hand over (3G-2G, 3G-3G, 2G-3G, 2G-2G)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
13
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
A test scenario: Call Forwarding on Busy - qualitative
User A
User B
User D
User C
Mobile
Core Network
CFB activation, destination USER D
Preamble
Auxiliary call from B to A to establish busy state
Active call
Call from C to A forwarded to D
Active call
Test
End calls
Postamble
CFB deactivation
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
14
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
A test scenario: Basic call - signalling
User A
Mobile Call Network
User B
REGISTER (Deactivate SS - CF and CW)
RELEASE COMPLETE
Preâmbulo
REGISTER (Deactivate SS - CF and CW)
RELEASE COMPLETE
SETUP (Speech, CdPN = B)
CALL PROCEEDING
SETUP
CALL CONFIRM
ALERTING
ALERTING
CONNECT
CONNECT ACKNOWLEDGE
Test
CONNECT
CONNECT ACKNOWLEDGE
Active call
DISCONNECT
MONET emulates the behaviour of
the users at the mobile core network
interfaces (A or Iu-CS) enabling for
each user to:
►
Perform protocol procedures
(including settimg of test preconditions)
►
set the value of each field of sent
signalling messages at Call
Control level (e.g. Bearer
Capability, CdPN, …)
►
check (or ignore) each field of
received signalling messages at
call control level (e.g. Release
cause,received Bearer
Capability, CgPN, …)
RELEASE
REL COMP
DISCONNECT
RELEASE
REL COMP
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
These features enable MoNeT to:
►
Verify the correctness of the
service and of the signalling
procedures
►
Produce the logs, the traces and
the verdict of each test
15
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Monet history
►
►
2001: feasibility study
2002: platform development and integration with scenarios from Eurescom
P613 project (TTCN-2)
►
2003: debugging of call scenarios
►
2004-5: continuous extension of call scenarios with TIM custom serveces
►
2006-2008: project frozen for the realization of Mobile Network Labs in Turin
►
2009: extension of call scenarios
►
2010: extension of call scenarios with Hand-over procedures
►
2011: feasibility study for migration to TTCN-3
►
2012: migration to TTCN-3
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
16
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
System hardware framework
The MoNeT is composed of:
►
Linux server with:
►
A Interfaces
CS 2G/3G Core
Network
IP
WEB Client
Scheda STM1
►
signalling board for
interconnection with 2G
network (A interface)
IP
SS7 Interf
►
web server for remote
control
PSTN Intef.
►
Control Unit
WEB Server
►
PSTN
MONET
PSTN modem for calls
involving PSTN lines
Ixia Catapult for
interconnection with 3G
network (Iu-CS interface)
Iu-CS
Interfaces
Ixia
DCT2000
A generic web client
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
17
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
System software framework
Control Plane
TestList
Interfaccia
User
Interface
utente
Test Script
Language
GUI
Report
Test Manager
TTCN tool
Database
Virtual
Terminale
Terminal
Virtuale
User Plane
TTCN
Test Suites
WEB Server
Tests (ETS)
GSM-A Interface Manager
Terminale
Virtual
Virtuale
Terminal
PSTN Interface Manager
IU Interface Manager
Linux Based PC
Simulated
Protocol layers
(CC/SS/SMS, MM)
GSM-A Interface Driver
PSTN Interface Driver
IU Interface Driver (ddriver)
Adaptation and custom functions
GSM-A Interface
Interfaces
PSTN Interface
IU Interface
Interfaces
Emulated
Protocol layers
(RANAP, SCCP)
Emulated
Protocol layers
(ALCAP, STC)
Emulated Protocol layers
(MTP3b, SSCF, SSCOP, AAL5, ATM)
Emulated
Protocol layers
(Iu-UP, AAL2, ATM)
STM-1 Board
Physical Layer
Iu-CS Interface
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
18
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Migration from TTCN-2 to TTCN-3
TTCN-2 Source Code Translation
•30 kLoC
•more than 60 test cases
•more than 130 test functions
Migration of External Functionality
•Test Suite Operations
•Test Suite Operations Procedures
Test Adapter Implementation
•CP Implementation
•PCO Implementation
•Codec Implementation
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
19
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
TTCN-2 Source Code Translation and Functionality
Migration
►
►
Automated translation of
►
Extern Functionality
►
test types
►
TSOs
►
test data
►
TSOPs
►
test behavior
Because of a good quality (TTCN-2
standard compliance) of the TTCN-2
►
nearly no manual efforts
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
►
Translated to
►
TTCN-3 Predefined functions
►
TTCN-3 Functions
►
External functions that have to
implemented in the platform
adapter
20
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Test adapter implementation (I)
Communication Implementation
Coordination Point / CP
►
►
No need for implementation.
Provided by TTCN-3 tools
Point of Control and
Observation / PCO
►
Repackaging
Cleary separated at runtime
Repackaging
Functionality
Functionality
OS spec.
handling
OS spec.
handling
TTCN-2
specific
integration
TTCN-3 Port
Impl. / SA
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
21
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Test Adapter Implementation (II)
Codec Implementation
►
ASN.1 Based Codecs
►
►
No effort as TTCN-3 tool support all of the ASN.1 requirements and encoding rules,
built-in
Non-ASN.1 Based Codecs
►
►
Decision is to reimplement the non-ASN.1 codecs
►
Codec generation approach suitable
►
Low-efforts, high-performance
►
Low-risk
Only component that really has to be fully touched!
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
22
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
TTCN-3 based MoNet Architecture
Test System User
Test System User
GUI
WEB Server
CMD Interface
CMD Interface
CMD Interface
TE CMD Interface
TE CMD Interface
CMD
Line Interface
TE
CH
SA
Plugins
CH
SA
Plugins
CH
SA
Plugins
SA
CH
SA
Plugins
Plugins
SA
CH
SA
Plugins
Plugins
SA
SA
Plugins
SA
Plugins
SA
Plugins
Plugins
SA
SA
Plugins
Plugins
SA
SA
Plugins
PA
Plugins
SA
Plugins
Plugins
PA
SA
Plugins
Plugins
PA
CD
SA
Plugins
Plugins
PA
CD
Plugins
PA
CD
Plugins
PA
CD
Plugins
CD
TE
CH
CMD Line
CD
CH
TM / TL
TE
PA
Plugins
SA
Plugins
SA
Plugins
SA
Plugins
TE
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
CD
TE
23
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Economics: an application revenue
►
Context:
►
Regression test for FOA CNCS8.0
►
Activity duration: 4 months
►
Test cases for each session: 470
►
►
Rate of usage: several daily execution on each testing chain (6 testing
chains combining 3 MSCs and 2 HLRs) after every new software
installation or correction
Overall usage: over 200 session which means 200x6x470>550K tests
(about 5000 tests/day)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
24
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Economics:
the BEP between automated and manual testing
Ipothesys
Manaul Testing
Automated Testing
Hardware (terminals, PC, Interfaces, ...)
10 [KEuro]
230 [KEuro]
Automatic testing platform development
~30 [min]
3 Person Years
~5 [sec]
Test Execution Time (user selection, execution, analysis)
Testing environment setting
Working days/year
Number of tests
Expert people cost/year
~0,2 Person Years
214
500
~60 [KEuro]
GSM Network Regression Testing - Costs
800
700
600
KEuro
500
Total Cost
(manual)
400
300
BEP
200
Total Cost
(automated)
100
51
41
31
21
11
1
0
Test list executions
The BEP (Break Even Point) is reached with a limitated number of executions of the test list.
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
25
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Economics:
the effort required for automated and manual testing
GSM Network Regression Testing - Total Time
300
250
Person Years
200
150
Test execution Time
(manual)
100
Test execution Time
(automated)
50
0
1
201
401
601
Test List Executions
801
1001
1201
During the FOA CNCS8.0 activity, the numer of test executed to validate software and
corrections is so high that it shouldn’t be possible to apply the methodology with a manual
approach.
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
26
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
Next steps
►
CS scenarios extension
►
PS scenarios
►
User plane handling and check
►
Interfaces migration over IP
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
27
ETSI TTCN-3 User Conference and Model Based Testing Workshop
MONET (MObile NEtwork Tester)
QUESTIONS?
THANK YOU FOR YOUR ATTENTION!
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
28