Download Mixed SpaceWire - SpaceFibre Networks

Mixed SpaceWire - SpaceFibre Networks
Martin Suess(1, Steve Parkes(2
(1European Space Agency,
(2University of Dundee
E-mail: martin.suess at esa.int,
sparkes at computing.dundee.ac.uk
2-4 November 2008
2nd International SpaceWire Conference
in Nara, Japan
Slide : 1
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Overview
•
•
•
•
•
•
•
Introduction
SpaceWire – SpaceFibre comparison
SpaceFibre Virtual Channels
Priorities and Group Adaptive Routing
Mixed Network Examples
SpaceFibre Outlook
Conclusion
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 2
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Introduction
• SpaceWire supports bi-directional traffic of up to 200Mbit/sec
over a distance up to 10m.
• SpaceFibre shall be capable to improve to both figures by at
least a factor of 10:
– Data rates ≥ 2.5Gbit/sec
– Distance ≥ 100m
– Provide additional features like galvanic isolation
• This requires a number of modifications at different levels of the
protocol stack.
• The aim is to maintain compatibility between SpaceWire and
SpaceFibre on Packet and Network level.
• In the following the solutions implemented in the SpaceFibre
breadboard are presented
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 3
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Physical & Signal Level - Optical
• SpaceWire uses cables with 4 twisted pairs with nine-pin micro
miniature D-type connectors.
• SpaceFibre uses two optical fibres as medium:
– The Draka MaxCap 300 radhard graded-index multimode
fibre has been selected after testing
– A cable protecting the fibre was designed based on
expanded polytetrafluoethylene
– Diamond AVIM connectors where already qualified for space
• Electro optical transceivers produce 850-nm laser light with a
power of 3dBm peak
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 4
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Diamond AVIM connectors and electro optical transceiver breadboard
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 5
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Physical & Signal Level - Electrical
• In addition an electrical version for short distances forseen.
• Prototype used 4 coaxial cables with SMA connectors.
• The electrical interface of the transceivers use Current Mode
Logic (CML).
• CML is directly used as signal level in the electrical version.
• Tolerance to common mode voltage differences can be
improved by blocking capacitors.
• More investigations are needed before physical & signal level
definition of electrical SpaceFibre.
AC coupling of a CML transmitter and receiver
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 6
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Character Level - 8B10B Encoding
• SpaceWire defines data and 4 control characters
– FCT, EOP, EEP, ESC
– The combination of ESC with FCT and Data Characters
defines the Null control code and the Time-Codes
• SpaceFibre characters are based on 8B10B encoding
– DC balanced data signal plus 12 special characters for
control functions
– Three of these special characters are comma characters
– Comma characters contain a unique sequence of ones and
zeroes that are used for character alignment
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 7
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Character Level - Ordered Sets
• Ordered Sets are a sequence of 4 characters starting with a
comma character (K28.5)
• The second character defines the type of ordered set
• The last two characters can carry additional information
• Ordered Sets greatly extend the possibility to embed control
information in the data steam
• Several types of ordered sets are defined for SpaceFibre:
– Link-level, power management, reset, flow control, faming
and user ordered sets
– User ordered sets are used to propagate time codes and
interrupts though the network
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 8
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Exchange Level - Flow Control & Framing
• SpaceWire uses flow control to prevent receive buffer over flow.
– Each FCT indicates 8 Bytes of free buffer space.
• SpaceFibre maintains the concept of flow control.
– Granularity of flow control is increased due to higher
bandwidth.
– Each FCT ordered set controls the flow of one frame of
maximum 255 data words.
• A frame starts with a Start of Frame ordered set and ends with
an End of Frame ordered set.
• The End of Frame ordered set contains the 16 bit CRC of the
frame for error detection.
• SpaceWire packets are segmented into frames and
reassembled at link level.
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 9
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Virtual Channels in SpaceFibre
• Flow control and start of frame ordered sets contain the virtual
channel number.
• With separate frame buffers the virtual channel data flow is
logically separated while using the same medium.
• Congestion in one virtual channel does not influence the traffic
in the other virtual channels.
• A SpaceWire packet in one virtual channel can pass a packet in
another virtual channel.
• Priorities can be used to control the access of a virtual channel
to the physical medium so that the higher priority channel has
always direct access.
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 10
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Virtual Channels in SpaceFibre
VC1
Buffer
Virtual Link 1
VC1
Buffer
VC2
Buffer
Virtual Link 2
VC2
Buffer
SpaceFibre
VC3
Buffer
Virtual Link 3
VC3
Buffer
VC4
Buffer
Virtual Link 4
VC4
Buffer
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 11
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Number of Virtual Channels in a SpaceFibre Link
• Maximum number of virtual channels is 256.
• In practice less will be used to limit the number of buffers
needed.
• The individual virtual channels must be accessible without
blockage or bottle neck.
• In a SpaceWire router the total number of ports for path
addressing is limited to 31.
• The SpaceWire standard allows to use two consecutive address
bytes for path addressing in large routers.
– The first path address byte indicates the SpaceFibre link.
– The second path address byte indicates the virtual channel
number.
• Beyond this some of the virtual channels could be accessed by
logical addressing only.
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 12
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
SpaceWire - SpaceFibre Router
VC 6
SpaceFibre
1,6
VC 5
1,5
VC 4
1,4
VC 3
1,3
VC 2
1,2
0
1,1
Internal
Configuration
Port 0
VC 1
SpaceFibre Port 1
Crossbar
Switch
4
SpW
SpaceWire
Port 5
5
SpW
SpaceWire
Port 6
6
SpW
SpaceWire
Port 7
7
SpW
SpaceWire
Port 8
SpW
SpaceWire
Port 9
9
SpW
SpaceWire
Port 10
10
8
3,5
3,4
3,3
3,2
3,1
VC 6
VC 5
VC 4
VC 3
VC 2
VC 1
12
External
Port 12
3,6
11
External
Port 11
2,1
VC 1
2,2
VC 2
2,3
VC 3
2,4
VC 4
2,5
VC 5
2,6
VC 6
SpaceFibre Port 2
SpW
SpaceWire
Port 4
Router example with:
• 3 SpaceFibre links
with 6 virtual
channels each
• 7 SpaceWire links
• 2 External ports
SpaceFibre
• Non blocking crossbar
switch provides direct
access to every virtual
channel
SpaceFibre
SpaceFibre Port 3
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 13
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Virtual Channel Priorities & Group Adaptive Routing
• Each virtual channel can provide the full bandwidth of the
SpaceFibre link.
• If two or more virtual channels request a bandwidth higher than
the full bandwidth of the link some arbitration is required.
• Priorities can be assigned to virtual channels:
– The virtual channel with higher priority is allowed to send the
next frame.
– Round robin arbitration is performed between virtual
channels of the same priority.
• User ordered sets for time-code and interrupt distribution have
priority and are sent in the middle of the frame currently
transmitted.
• SpaceWire-RT protocol should be used to provide Quality of
Service beyond priorities.
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 14
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Group Adaptive Routing
• If packets are routed through the same virtual channel the
access is arbitrated by the router.
• Routers can provide group adaptive routing among virtual
channels with the same priority:
– Two packets to the same logical address can use parallel
virtual channels.
– The receiving side can then decide which should be
processed first.
– Available overall bandwidth is not increased.
• Routers can provide group adaptive routing among several
SpaceFibre links.
– This can be used to increase the available bandwidth.
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 15
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Network Example – Single SpaceFibre Link
High Data Rate
Instrument
Electronics
Mass Memory
SpaceFibre
Sp
a
e
ce Fibr
ceFibr
a
e
Sp
High Data Rate
Instrument
Electronics
redundant
2-4 November 2008
Mass Memory
redundancy
Modulator &
Downlink Unit
Modulator &
Downlink Unit
redundant
SpaceFibre
2nd International SpaceWire
Conference in Nara, Japan
Slide : 16
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
SpaceFibre as Backbone
Unit 3
Unit 4
Unit 8
Unit 6
Unit 7
SpW
SpW
SpW
SpW
SpW
SpW
SpW
SpaceWire,
SpaceFibre
Routing Switch
2nd International SpaceWire
Conference in Nara, Japan
SpaceFibre
SpaceFibre
SpW
SpaceFibre
SpaceWire,
SpaceFibre
Routing Switch
SpW
SpW
SpaceFibre
SpW
2-4 November 2008
SpW
SpW
SpW
SpW
SpaceWire,
SpaceFibre
Routing Switch
SpW
SpW
SpW
SpW
SpaceWire,
SpaceFibre
Routing Switch
Unit 9
SpW
Unit 1
Unit 5
SpaceWire,
SpaceFibre
Routing Switch
SpW
Unit 2
SpaceWire,
SpaceFibre
Routing Switch
Slide : 17
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Mixed Networks
Inst 3
Unit 2
SpW
SpW
Inst 2
SpW
Unit 3
Unit 1
SpW
SpW
SpW
Inst 1
SpW
SpaceWire,
SpaceFibre
Routing Switch
SpaceWire,
SpaceFibre
Routing Switch
SpaceFibre
SpaceFibre
SpaceFibre
SpaceWire,
SpaceFibre
Routing Switch
SpaceFibre
SpaceFib.
SpaceFibre
SpW
SpW
High Data Rate
Instrument
Electronics
redundand
SpaceFibre
SpaceWire,
SpaceFibre
Routing Switch
SpW
High Data Rate
Instrument
Electronics
SpW
SpW
SpaceFibre
SpaceFibre
Mass
Memory
Mass
Memory
redundancy
Modulator &
Downlink Unit
Modulator &
Downlink Unit
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 18
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
SpaceFibre Breadboarding
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 19
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
SpaceFibre Breadboarding
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 20
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
SpaceFibre Outlook
• A first SpaceFibre prototype covering all levels has been
implemented and tested.
• A first outline specification has been published and discussed in
the frame of the SpaceFibre working group.
• The experience gained will be consolidated and used for the
development of a SpaceFibre demonstrator.
• The SpaceFibre Demonstrator activity will target:
– Development of a SpaceFibre IP core,
– Test and validation of IP core using existing prototype,
– SpaceFibre Demonstrator implementation based on Actel
FPGA and Wizard Link SerDes,
– Preparation of a SpaceFibre specification as input for the
standardisation process.
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 21
MIXED SPACEWIRE - SPACEFIBRE NETWORKS
Conclusion
• The different levels of SpaceFibre have been compared with
SpaceWire.
• SpaceWire and SpaceFibre is intended to be fully compatible on
Packet and Network level.
• This allows the easy implementation of mixed SpaceWire –
SpaceFibre networks.
• Some examples of those networks have been presented.
• This compatibility is seen as essential feature of SpaceFibre.
• Experience has been gained with a prototype implementation.
• As next step a demonstrator will be developed based on space
qualified components.
• Standardisation in ECSS is envisioned in corporation with the
other space agencies.
2-4 November 2008
2nd International SpaceWire
Conference in Nara, Japan
Slide : 22