Download Cognitive Packet Networks - TERENA Networking Conference 2001

Toward Networks with
Cognitive Packets
Erol Gelenbe Ricardo Lent
Esin Seref
Zhiguang Xu
School of Electrical Engineering & Computer
Science
University of Central Florida
Orlando, FL 32816
Overview








Introduction to CPN
Cognitive Packet Networks
Packet Routing Using Neural Networks
– Random Neural Networks & Learning
– Random Neural Networks with Reinforcement
Learning
Simulation (LFRNN, RNN RL, & Bang-Bang)
Cognitive Adaptive Routing
Analytical Model Predictions using G-Networks
CPN Test-Beds
Future Work
Problem Definition
Packet networks have capabilities for flow
control, error control and routing
Cognitive Packet Networks (CPN)
concentrate intelligent capabilities for routing
in the packets rather than in the nodes and
protocols
For the time being, we assume that error and
flow control in CPN are handled using current
techniques
Introduction to CPN

A packet switching network where packets route
themselves
Packets are assigned goals before entering the
network
Packets learn to achieve their goals
Learning is performed by sharing information
between packets
Packets sharing same goals can be grouped into

Packets do not rely on nodes for routing




classes
OSI Layers






TCP/IP is a layered protocol stack
Application handles particular
applications, Presentation handles
compression and encryption of data,
Session controls establishment,
management, termination of sessions
Transport provides flow of data
Network handles the transmission of
packets in the network
Data-link is responsible for the
interaction of the device driver in the
operating system and the network
card in the machine
Physical defines electrical and
mechanical specifications
Application
Presentation
Session
Transport
Network
Data Link
Physical
Transport Layer (TCP)



Connection-oriented: client and server applications
establish connection before data exchange
Reliable: sends checksum on data and header, takes
care of the order of arriving packets, takes care of
duplicate packets, sends acknowledgments,
maintains timer
Flow-control: receiving side allows the sending side
to send as much as it can buffer
Network Layer (IP)



Unreliable: best effort service, no guarantees to get
to the destination
Connectionless: no state information about
successive datagrams, each handled independently
Routing: done on a hop-by-hop basis according to
the routing tables, required information is kept in the
IP header (Routing table information comes from the
routing protocols)
Current Routing Protocols

Distance-vector Protocols
– Routing tables updated according to a vector of distances
(hop counts)
– Regular and triggered updates
– Whole network topology is distributed
– No information about the network topology
Drawbacks:
– Regular updates consume bandwidth
– Takes too long to stabilize after failure
– Using only hop count omits other variables
Current Routing Protocols

Link-state Protocols
– Each router tests the status of its link to its neighbors and
sends this information to its other neighbors
– Converges faster than distance-vector protocols
– Each router can be assigned a cost based on
throughput,reliability, round-trip time, etc
– Load balancing possible
– Multicasting reduces load on systems not participating
Drawbacks:
– No policy based routing
– Needs more memory and CPU power
CPN and CP
CPN: Packet switching networks in which
intelligent capabilities for routing are
concentrated in packets rather than in the
nodes and protocols
CP: Packets in CPNs which have the route
themselves
Structure of CPs
Identifier
Field
DATA
Cognitive
Map
Executable
Code
•Identifier Field: The unique identifier of a packet
•Data Field: Payload
•Cognitive Map: S-D information, the packet’s view of
the state of the network, etc
•Executable Code: Code used to update the CM and the
decision algorithm
Nodes in a CPN



Storage area for CPs:
Input and Output Buffers
Mailboxes are used to
exchange data between
CPs
The Node executes the
code for each CP in the
input buffer
Update of CM by Node
Mailbox
(MB)
Cognitive
Map
(CM)


Code
Updated
Cognitive
Map
Packet arrives at the input buffer of a node
The code of each CP in the input buffer is
executed
– The CP retrieves relevant information from the
mailbox
– The packet’s CM is updated
– Some information is moved from the CP to the MB
– The CP is moved to an output buffer
Cognitive Packet Classes
A Class is a set of packets having the same
QoS requirements, sets of internal
states, control rules, input-output
signals
NODE
(X,Y)
S
S
D
D
NODE
(X+1,Y)
S
Class
(S,D,L)
D
S
D
NODE
(X,Y+1)
Random Neural Networks

A spiked RNN with a mathematical structure similar
to a queuing network (Gelenbe in Neural Computation 93,
IEEE Trans. on NN 99 etc.)

Probability that the neuron is excited:


where i   q j w ji   i
j


and
qi 
i
ri  i
i   q j wji  i
j
wij is the rate at which neuron i sends excitation
spikes to neuron j when i is excited
wij is the rate at which neuron i sends inhibition
spikes to neuron j when i is excited
Learning Feed-Forward RNN



Input to each neuron is
represented as a pair of
excitatory and inhibitory
signals
Each connection is
represented as a pair of
weights
Output of the system is
the q of the output
layer neurons
i
i
LFF RNN Equations

Tl

Normalized
training set
D
l 1
 0.9 *
Dl  Dl 1
k
k
 0.9 *
Dl
Dl 1
k
Parameters adjusted to minimize the error function
Ek

Tl
otherwise
Network parameters are computed from the training
set
W   {w (i, j )}
W   {w (i, j )}
Dl
k

,
Weight updates
1

2
n
a
i 1
i
( qi  yik ) 2 , ai  0
n
wk (u, v)  wk 1 (u, v)    ai (qik  yik )[qi / w(u, v)]k
i 1
LFF RNN Equations

Where
Q / w (u, v)  Q / w (u, v)W    (u, v)qu
Q / w (u, v)  Q / w (u, v)W    (u, v)qu
Q  (q1 ,..., qn )
W  {[ w (i, j )  w (i, j )q j ] /  ( j )}
 1 /  (i )


 i   1 /  (i)
0

if u=i,v!=i
if u!=i,v=i
otherwise
 (1  qi ) /  (i )


 1 /  (i)

i  

 qi /  (i)
0

if u=i,v=i
if u=i,v!=i
if u!=i,v=i
otherwise
RNN with Reinforcement Learning



Fixed input values
Each connection is
represented as a
pair of negative and
positive weights
Output of the
system is the largest
q of all the neurons
i
j
RNN RL Algorithm

Decision threshold
Tl  aTl  1  (1  a) Rl , R  G

Compute Tl 1
If Tl 1  Rl
w  (i, j )  w  (i, j )  Rl
w  (i, k )  w  (i, k ) 
Rl
w (i, k )  w (i, k ) 
,k  j
n2
w  (i, j )  w  (i, j )  Rl

1
Rl
, k
n  2

otherwise

j
RNN RL Algorithm

Re-normalize all weights by calculating
n
ri*   [ w (i, m)  w (i, m)]
1
and
w  (i , j )  w  (i , j ) 
ri
ri*
ri
w (i , j )  w (i , j )  *
ri


Simulation






100 nodes
Link speeds normalized 1
Arrivals are Poisson
Buffer sizes are unlimited
Packet loss simulated
probabilistically,
intentionally high in some
areas
Goal is to minimize a
weighted combination of
delay W and loss L
G  W  L
Bang-Bang to Minimize
Average Delay
Bang-Bang to Minimize Avg
Delay + Loss
RNN Reinforcement Learning to
Minimize Avg Delay + Loss
LFF RNN Control to
Minimize Avg Delay + Loss
Bang-Bang & Shortest Path
in the Network Without Loss
Cognitive Adaptive Routing (CAR)

There are three types of packets:
– The packets sent into the network by the source with small
rate  which travel intelligently to their destination (SMART
or Cognitive Packets) –
– A SMART packet which reaches its Destination generates an
ACK back to the source; ACKs travel back along the same
(reverse) route as the packet


Reception of an ACK at the Source may trigger the
transmission of a DUMB packet along the route which
the older packet had taken
ACKs may not be necessary for each packet
Cognitive Adaptive Routing (CAR)
The ACK coming back from Destination
D deposits Delay Information
W(n,O,D) when it passes through Node
n, entering it from Node O
 W(n,O,D) is used to Update the CAR
Learning Algorithm (eg Reinforcement
Learning) either by the Smart Packet, or
by the Node itself

Analyzing CAR with G-Networks

G-Networks are Queuing Networks with Three Types
of Customers:
– Normal Customers
– Negative Customers: They Destroy Normal Customers
– Triggers: They Move Normal Customers to Another Queue





Multiple Class Models Allowed
G-Networks Have Product Form … They Can be
Solved Analytically and then Numerically
Work of Gelenbe (89, 91, 93, 98) and Many Others
Key Papers Mainly in Journal of Applied Probability
G-Networks Can Model “Worst-Case CAR” and
“Best-Case CAR”
G-Network Equations

Non-Linear Equations for the Queue Utilizations
q
S
S, R , D
q
0
S, D 

q iS, R ,D
q DS, R ,D μ DS, R ,D  q 0S, D  q 0S, D μ 0S, D  PS, R ,D

μ SS, R ,D
μ 0S, D
0S, D
  (q DS, R ,D μ DS, R ,D )
R
q j S, R ,D μ j S, R ,D

μ iS, R ,D
- queue S is Source, queue D is Destination
-  S, R, D  is a route from S to D with queue i and queue j as
components and j is the predecessor of i
- ACKs will trigger packets from queue 0, which constantly feeds queue S
with small rate 0<S,D>
Analytical Solution

Calculate the average No of packets in each queue:
N
i
S, R , D 
N
i

q iS, R , D 
1
q
R
S

 N
S
N S, D 
N 
0
R

N
D
i
S, R , D 
D
q0
S, D 
1  q0
S, D 
i

 N
i

i
S, R , D 
S
0
S, D 
D
Calculate the average transit delay for a packet
W
N
 Δ 0S,D
S
D
Cognitive Adaptive Routing
Performance
All Packets
Are Smart
but Not Always
Successful
Fewer Packets
Are Smart:
Dumb Packets
Follow Successful
Smart Packets
Cognitive Adaptive Routing
Performance
CAR Performance as a Function of
Smart Packet Injection Rate
Test-beds for CPN
Goals
 Performance evaluation and capability enhancements
 Demonstration of applications
Requirements
 Explicit separation and clear borderline between the
different protocol layers
 Standardized interfaces both between the application
layer and the CPN layer, and between the CPN layer and
the data transport network
 Compatibility with a wide range of computing platforms
 Interchangeability of the data transport networking layer
A CPN-VN (Virtual Network)
 Standard
Linux on Intel-PCs
 New CPN Code Instead of IP
CPN Test-Bed
Different Selected
Logical
Connectivities
Different Selected
Physical
Connectivities
CPN-VP Test-Bed
Current Test-Bed Connectivity
Special Purpose
CPN-Router Test-Bed
Special-Purpose CPN-Router
MPC860sar
PowerPC
CPU
Core
10Mbps
Ethernet
Transceiver
Ethernet
MAC
SCC1
ATM
SAR
SCC4
Memory
Controller
RAM
Memory
32K
Flash
ROM
16K
ATM
Transceiver
155Mbps
ATM
Transceiver
155Mbps
ATM
Transceiver
155Mbps
Smart & Dumb Packets
Identifier
Routing
Data + Code
SMART
or
DUMB
Packet
Indicator
Payload
Cognitive Map
and
Executable Code
Packet Contents
Mail Boxes in CPN-Routers

The Mailbox (MB) is a LRU stack where information is
stored: delaysweights for different directions
associated with certain <S,D>, ...

Packets (esp. ACKs) deposit information in the MB
CPN-Router Software
State Diagram of CPN-Router
PDU Arrived (either from higher layer or
from peer entity)
PDU Assembled
Complete Route
Available
No Route
Available
Incomplete Route
Available
Update Route
Complete /
Incomplete Route
Consult MB
Update CM
No Route Found
PDU
Ready
Proceed at Random
Send
Work Accomplished
CPN: packet network architecture in which routing
is carried under Packet Control -Cognitive Adaptive Routing: Smart or Cognitive
Packets Control the Routing of the Dumb Packets
Cognitive Packet Routing is Designed Using Random
Neural Network based Reinforcement Learning,
Feed-Forward Learning, Bang-Bang Control
Analytical Modeling and Simulation Results are
Presented
Test-Bed architectures are being implemented
Work to be Done: I
Run and Measure Significant Applications
(eg File Transfer, Video-Conferencing, Voice,
Web Browsing …)
Evaluate Network Adaptation under Time-Varying
Workloads
Deploy a “Large” Test-Bed
Examine Implications with respect to TCP
Consider CAR as a Joint Routing & Flow Control
Scheme
Work to be Done: II
Can CAR be used to enhance IP ?
Can CPN run in conjunction with IP ?
Can Generalized CAR be a way of
selecting paths in BGP ?