Download OPNET Simulation of Self-organizing Restorable

From Proc. OPNETWORKS '98 Conference (CD-ROM), Washington, D.C., April 24-25 1998, paper 04
OPNET Simulation of Self-organizing Restorable SONET Mesh Transport Networks
Demetrios Stamatelakis, Wayne D. Grover
TRLabs c/o Dept. of Electrical and Computer Engineering
University of Alberta, Edmonton, Alberta, Canada
configured cycles, called p-cycles, amongst the previously unconnected spare links (e.g., STS-1s or STS-3s)
of a mesh-restorable network [2,3,4]. The advantage in
speed is that only two DCS nodes have any real-time
cross-connection workload for any given failure. Also,
the end nodes only have to effect end-node traffic-substitution connections to ports which are identified by the
planning process, before failure.
The Cycle Preconfiguration Concept
Figure 1 illustrates the ways in which an individual pcycle may be used for restoration. In Fig. 1(a) an example
of a p-cycle is shown. In (b), a span on the cycle breaks
and the surviving arc of the cycle is used for restoration.
In (c) and (d), however, we see how the p-cycle can also
be accessed for restoration of working paths that are not
on the cycle. In fact, cases (c) and (d) are the more advantageous in general, because two restoration paths are
obtained from each p-cycle for such failures. Further
inspection of the example in Fig. 1 shows in fact that this
particular p-cycle provides a single restoration path for
nine span failures and two restoration paths for each of
six other span failures (the latter are for each of those
spans which are not on the cycle, but straddle it). These
paths are immediately available and do not require any
time to calculate or form.
Abstract
Today’s digital fiber-optic telecommunication transport
networks carry prodigious quantities of information: the
equivalent of millions of phone conversations can be carried on a small fiber optic bundle. However, such capacity comes at a price: if not dealt with very promptly, a
cable cut can have significant societal and economic consequences. We have devised a novel approach to the network restoration problem; prior to a failure network
redundant capacity is preconfigured, using a distributed
design protocol, so that any failure can immediately be
restored. This method can realize restorable designs with
low capacity redundancy and potentially very rapid restoration. The protocol executes in a peer-to-peer manner in
the network’s nodes eliminating the need for a centralized design mechanism. The protocol is simulated using
OPNET modeler and a platform independent Java animation tool is used for realtime replay of the OPNET generated trace data.
1. Introduction
A mesh restorable network is composed of nodes, formed
with Digital Crossconnect Systems (DCS), and node-tonode spans which transport digital multiplexed signals.
A DCS permits the formation of connections between
individual pairs of digital multiplexed signals (e.g. DCS3, STS-1, or STS-n) in a highly configurable manner. A
mesh restorable network routes working traffic between
node pairs using working paths formed by the DCS.
Conventionally, a mesh network becomes restorable by
distributing unconfigured spare capacity among the network spans. In the event of a span failure, the severed
working traffic is rerouted with dynamically constructed
alternate paths formed from the undedicated spare capacity. The pathset is calculated dynamically in response to
each failure and differs substantially between failures.
The pattern of crossconnections, for any node, varies significantly from failure to failure. The dynamic nature of
the restoration pathsets is both an advantage and a disadvantage. Because of it, the restoration algorithm can
make the most efficient use of the spare capacity to
restore a specific failure; this is why mesh network
require relatively little spare capacity compared to other
restoration methods. However, time is required to
dynamically calculate and form a restoration pathset so a
conventional mesh network may not respond to a failure
quickly enough for some applications [1].
With this motivation, we have developed a strategy for
network operation that should make it possible to achieve
much faster restoration times while retaining the desirable capacity efficiency of the mesh-restorable alternative. The method is based on the formation of pre-
a) A p-cycle, X
c) A span off the p-cycle
fails, p-cycle X contributes
two paths
b) A span on the cycle fails, pcycle X contributes one restoration path
d) A span off the p-cycle
fails, p-cycle X contributes
two paths
FIGURE 1. Use of a p-cycle in restoration
1
From Proc. OPNETWORKS '98 Conference (CD-ROM), Washington, D.C., April 24-25 1998, paper 04
A surprising recent finding is that with p-cycle restoration the spare capacity design takes little or no additional
capacity than conventional mesh restoration [2,3].
This adds considerable practical motivation to pursue this
approach further. In this regard one of the key issues to
realizing a practical restoration scheme is the complexity
of computing and deploying an always-current spare
capacity plan to support the restoration mechanism. One
way to compute the optimal p-cycle spare capacity and
configuration plan is via a centralized Integer Program
(IP) solution (see [2,3].) Centralized computation and
control is, however, considered a significant practical
burden, with the ever present risk of the centralized
design configuration being out of date with the real network state. This is why we have sought a way in which
the p-cycle configuration of the network can be made
autonomous or self-organizing.
Outline of paper
We proceed as follows: The rest of Section 1 defines
some needed terminology. Section 2 then addresses the
desire to avoid dependence on centralized control for
deployment and maintenance of a the p-cycle state for a
network. This is done through a distributed self-planning
protocol, which is the main contribution of the paper.
Section 3 describes how this protocol was investigated
using OPNET simulation.
Definitions
We use link to denote an individual bidirectional digital
carrier signal between adjacent nodes. In general the link
is whatever unit of capacity the DCS manipulates for
transport management and restoration. For instance, a
DS-3 or STS-1. A span is the set of all working and spare
links in parallel between adjacent nodes, whether on one
or several OC-n systems in parallel. A useful path is a
path segment contained in a p-cycle which is related to a
failure span in a manner that allows it to contribute to restoration of a given failure.
many outgoing statelets but each outgoing statelet can
have only one precursor.
The DCPC involves nodes acting in either of 2 roles, that
of a Cycler node or a Tandem node. The Cycler sources,
and later receives, parts of the statelet broadcast pattern it
initiates. Each node adopts this role in a round-robin
fashion. While in this role it is temporarily in charge of
the cycle-exploration process within the network as a
whole. When not in the Cycler role, each node plays a
Tandem-node role which mediates the statelet broadcast
competition.
To give an overview, the DCPC first allows each node to
explore the network for p-cycle candidates that are discoverable by it. After completion of its role as Cycler, it
hands off to the next node in order by a “next-node handoff” flood-notification. After all nodes have assumed the
role of the Cycler once, each “posts” its best found cycle
in a distributed network-wide comparison of results. The
competition flood expands through the network as each
node locally relays the statelet with the best cycle metric,
or asserts its own best if and while the latter is still superior to anything else it has received notice of yet. Eventually, the globally best cycle candidate dominates
everywhere. Upon thus learning of the winning candidate, the node which discovered this p-cycle, triggers its
formation as a p-cycle. All nodes on the p-cycle update
their local tables of restoration switching pre-plans to
exploit the new p-cycle. The whole process then repeats
without any central control, adding one p-cycle per iteration until a complete deployment of near-optimal pcycles is built. Thereafter, it continually adapts the pcycle set to changes in the working capacity layer.
3. OPNET implementation of the DCPC
The DCPC algorithm is a highly parallel and asynchronous process. There is no centralized control point; the
distributed interactions of spatially separate nodes determine the final design. Although the rules which are
applied locally at each node are relatively simple, the global interactions and behaviors of many concurrent
instances of the DCPC rules are not always obvious.
OPNET modeler was an enabling tool to develop the correct global behavior of this protocol.
One of the more interesting parts of this simulation is that
the DCPC is a state-based protocol; it operates on the
basis of state information which is embedded on each
link using statelets. OPNET, however, is packet driven.
It was required that packets be used to simulate state
based messaging. This was done using packets to transmit the statelet information, and registers to receive and
store the statelet information.
The OPNET model is constructed at three design levels:
the network, node, and process level. Each level is discussed in the upcoming sections.
A. OPNET Design at the Network Level
Figure 2 contains a network level view of one of the networks for which the DCPC was simulated using OPNET.
2. Self-organization of the p-cycle state
Here we give an brief overview of the self-organizing
strategy we have developed for the autonomous deployment and continual adaptation of the preconfiguration
state. A more detailed description is given in [3]. The
Distributed Cycle PreConfiguration (DCPC) protocol is
based on distributed interaction through statelets. A statelet is embedded on each spare link and contains a number
of state fields. Each logical link, as viewed by a node
attached to it, has an incoming statelet and outgoing
statelet. An incoming statelet arrives at a node on a link,
and originates from the adjacent node connected through
the link.
Each outgoing statelet has an incoming statelet which
forms its precursor. An incoming statelet is a precursor to
an outgoing statelet if the incoming statelet was cause,
under the protocol rules, for the creation of the outgoing
statelet. One incoming statelet can be the precursor for
2
From Proc. OPNETWORKS '98 Conference (CD-ROM), Washington, D.C., April 24-25 1998, paper 04
Each node is connected to other nodes according to the
topology of the network being simulated. Although not
obvious in the figure, each span is allocated a certain
number of working and spare links. Each simulated network also has an artificial Tracer node whose role it is to
trace the actions of the other nodes and to print summary
information at the end of the simulation run.
FIGURE 3. Node level view of the OPNET model
The Xpt_Controller maintains a map of any crossconnections commanded by the DCPC. Basically, this module’s
only job is to make and unmake crossconnects, at the
command of the DCPC, and maintain a record of the
crossconnect matrix state.
C. OPNET Design at the Process Level
There is a Process view corresponding to each of the
modules discussed in the previous section. Due to space
constraints only the process view of the DCPC module
will be discussed. The function of the DCPC module is
to implement the statelet broadcasting rules of the DCPC
protocol. Figure 4 shows the DCPC process state diagram.
The DCPC protocol effects the following stages: exploration of the best p-cycle candidate by each node, competition among each node to discover who found the best
cycle candidate, and construction of the best cycle by the
winner of the competition.
The exploration stage has each network node, in turn,
assume the Cycler role. All statelets originate at a Cycler
node. After initiation of statelet broadcast the Cycler
node samples for incoming statelets. Because a cycle
tends to contain more useful paths as it grows in size during the sampling interval, the score of a prospective
cycle will generally improve with time under the actions
of the Tandem nodes. The Cycler node therefore waits to
observe the evolution of the best cycle possible. The
Cycler maintains a record of the received statelet with the
best score, and corresponding p-cycle. When the sampling time runs out, the Cycler node records the p-cycle
candidate with the best score. It then suspends all statelet broadcasts, terminates its role as the Cycler, and emits
a Cycler hand-off flood (a statelet with op-code “handoff” and the node name, on one link of each span.) The
hand-off flood is relayed (once only by all nodes, without
link persistence, with one copy in each span). When node
n hears “hand-off flood, n-1” it knows that it is its turn to
become Cycler.
After a round of cycler action by every node, the last
node in the sequence knows all nodes are ready to take
part in a network wide comparison of results. The purpose is to find the one globally best p-cycle candidate.
This process is triggered by the initiation of a global
FIGURE 2. Network level view of the OPNET model
B. OPNET Design at the Node Level
Figure 3 contains a node level view of one the network
nodes, in this case a degree 3 node. The network node
contains 5 user designed modules: Port card modules,
Delay modules, a Poll_Timer module, a Xpt_Controller,
and a DCPC module.
The DCPC module encapsulates all of the statelet processing rules of the DCPC. The network wide interaction
of the DCPC modules is what determines the final pcycle design. More detail follows at the Process level.
The Port modules terminate a span. They receive and
transmit the statelets which are the medium of communication for the DCPC. The information for a newly
received statelet is stored within a register corresponding
to the link on which the statelet arrived and a receive
interrupt is raised. Similarly, when transmitting the statelet information is written into a port’s transmit register
(by an other module such as the DCPC) and the port is
sent a transmit interrupt.
The Delay module receives statelet events from its corresponding Port module and delays the transmission of the
packet by a time equal to the sum of the insertion delay
and the propagation delay. Insertion delay is based on a
SONET 64 kb/s line-overhead byte and the propagation
delay is calculated at 70% of the speed of light.
The Poll_Timer module periodically polls the node’s port
cards for incoming statelets and forwards these to the
DCPC module. The Poll_Timer also tracks the number
of statelets which are processed by the DCPC module in
the current polling interval, and uses this to introduce a
time delay due to computation. It also relays any statelet
broadcast transmissions, required by the DCPC module,
to the Port modules subject to the estimated computational delay.
3
From Proc. OPNETWORKS '98 Conference (CD-ROM), Washington, D.C., April 24-25 1998, paper 04
broadcasts at the Cycler, which initiates the p-cycle candidate exploration. It also sets up a countdown timer
which limits the Cycler to a predetermined sampling
interval. The Sample state is unforced and represents the
Cycler’s “low” state as the Cycler pauses in this state
while waiting for events to process. The Eval state is
where the Cycler processes newly arrived statelets and
evaluates the cycles they represent. The arrival of cycles
with superior scores is recorded here. The CycDone state
is entered when the sampling interval lapses and the
Cycler role is released. If the node is the last node in the
Cycler role assumption sequence then a competition
flood is initiated; otherwise, the initialization of the
Cycler hand-off flood takes place.
The Tandem node logic for the processing of cycle exploration statelets is found in the Prc_Sig and Updt_Tx
forced states. Prc_Sig processes newly arrived incoming
statelets according to the rules of the DCPC statelet
broadcast rules, while Updt_Tx evaluates all incoming
statelets which are currently present on the node and
updates the outgoing statelets so they are consistent with
the DCPC rules. The DCPC rules are structured to favor
the rebroadcast of high scoring incoming statelets which
can loop back to the Cycler to form closed cycles. The
scoring mechanism is designed so that those statelets
which represent a route through the network which has a
high potential to loop back to the Cycler node and to
form a good p-cycle are given a high score. A detailed
description of the DCPC broadcast rules is given in [3].
Between the processing of statelets the node settles into
the unforced Idle state which represents the low state for
this part of the state diagram.
The logic for the construction of a p-cycle is contained in
the StrtCycle, Wait, Finish, and ContCycle states. The
node which starts the construction of the cycle does this
in the StrtCycle state. It then settles in the unforced Wait
state. Intermediate construction nodes receive and process the cycle construction request in the ContCycle state
where they build their portion of the cycle and then relay
the construction to the next node along the cycle. When
the cycle construction loops back to the initiating node it
completes construction in the Finish state. This state is
also where the next iteration of cycle exploration and
construction is initiated.
FIGURE 4. Process State Diagram of the DCPC
module in the OPNET model
competition flood by the last node in the sequence of
cycler phases. The initiating node broadcasts a single
statelet, containing the node’s name and its best cycle’s
score, on each span. When adjacent nodes receive such a
statelet they compare the received best score to their local
best score and relay the better of the two into all spans,
along with the name of the node who is reporting the better cycle. If scores are equal, precedence is based on ordinal rank of the node names involved. Rapidly, only the
single best score is present everywhere, and the node
which found this candidate proceeds to initiate its construction.
The node associated with the best candidate cycle scans
for the first unoccupied spare link on the span joining it
to the first node in the route field (which contains a
record of the cycle’s route through the network) of the
cycle to be constructed. It sends the route vector on to
that node. Each other node along the route makes the
specified spare-link cross-connection and forwards the
route vector on to the next node in the prescribed cycle.
Eventually this process returns to the node which initiated the cycle construction allowing it to form the final
cross-connection, closing the cycle. All spare links used
by the p-cycle are removed from eligibility in further
cycle constructions.
Upon completion of the cycle, the node which began the
construction either initiates a Cycler hand-off flood commanding the first node to assume the role of the Cycler,
or, if it happens to be the first node, it directly assumes
the role of the Cycler node. This starts the next iteration
of cycle construction. p-Cycles are constructed iteratively until no more cycles are feasible.
Process states
The Cycler logic is contained in the following states in
Figure 4: InitCycler, Eval, Sample, and CycDone. The
InitCycler state primarily sets up the outgoing statelet
4. Results
The OPNET DCPC simulation was run in 5 test networks
which were optimally provisioned to be fully restorable
to any span failure using p-cycle restoration with an Integer Linear Program (IP) p-cycle design formulation [2,3].
The test cases, therefore, represent operation with the
minimum of spare capacity.
The performance of the DCPC in these minimal capacity
networks is in Table 1. The figure of merit is restorability
which is the percentage of working links over all span
failures which can be restored. Two types of restorability
are given in the table: p-cycle is the restorability when
using only p-cycle restoration, and 2-step is the
4
From Proc. OPNETWORKS '98 Conference (CD-ROM), Washington, D.C., April 24-25 1998, paper 04
restorability when first using all useful p-cycle restoration paths and then topping the effort up with conventional restoration within the remaining spare capacity.
The restorability results are quite good considering the
DCPC is a database-free self-organizing process being
run in a minimally spared network.
Network
p-cycle
Restorability(%)
2-step Restorability
(%)
Net1
100
100
Net2
100
100
Net3
90.94
97.16
Net4
89.16
97.68
Net5
83.75
95.44
OPNET’s statistic gathering facilities were not used in
this simulation, as our main concern was in the performance of the p-cycle design which resulted at the end of
a simulation run and not in the statistics of the run itself.
To complement the OPNET simulation, we have also
developed a Java based animation tool which is used to
visualize the simulation trace files. The op-codes which
the animator interprets describe generic network processes; they are not specific to the visualization of DCPC
processes. For example, an op-code would be of the
form “turn node 5 bright blue” and not “show node 5
becoming the Cycler.” This permits the animation tool to
be used to visualize other concepts being worked on
within our group; this is convenient as the utility may be
used among multiple projects, eliminating the need to
custom design an animation tool for each project. Also,
as it is written in Java, animations can be conveniently
played back on either PC or Unix systems. Figure 6 contains a screen capture of the animator.
TABLE 1: Performance in minimal-spare test networks
Cycle Score (numpaths / hopcount)
Figure 5 is a simulation trace portraying overall operation
of the DCPC process, using Net 1 for its simplicity of
illustration. The plot shows the score of the best p-cycle
candidate seen by any cycler, versus simulated real time.
Six main regions appear in the plot, corresponding to the
5 p-cycles formed by DCPC for Net 1, plus a sixth iteration to realize the halting criterion (the existence of no
suitable p-cycle candidate). Each of the larger regions
correspond to the all-nodes cycler search, comparison,
and construction of a single p-cycle. Inside each region,
there are 10 individual cycler node explorations and nextnode hand-off floods (Net 1 has 10 nodes).
Global Operation of the DCPC Protocol
4
3.5
3
FIGURE 6. Screen Capture of the Java-based
Animator
2.5
2
1.5
References
1
1. SR-NWT-002514: “Digital cross-connect systems in
transport network survivability,” Bellcore, 1, January
1993.
0.5
0
0
1
2
3
4
5
6
2. D. Stamatelakis, Theory and Algorithms for Preconfiguration of Spare Capacity in Mesh Restorable Networks, M.Sc.Thesis, University of Alberta, Spring
1997.
7
Time (s)
FIGURE 5. Best cycle score vs. time in Net1
3. W. D. Grover, D. Stamatelakis, “Cycle-Oriented Distributed Preconfiguration: Ring-like Speed with Meshlike Capacity for Self-planning Network Restoration,”
Proc. ICC’98, 1998.
5. Concluding Discussion
The simulation described in this paper is perhaps quite
different from many OPNET simulations by virtue of its
parallelism and the state-based interaction model for this
application. The OPNET modeler tool was extremely
useful in simulation and development of the DCPC protocol. The alternative would have been the design and
programming of a discrete event simulation from scratch.
4. D. Stamatelakis, W.D. Grover, "Distributed Preconfiguration of Spare Capacity in Closed Paths for Network
Restoration," U.S. Patent Pending, July 11, 1997.
5