Download Timing requirements/diff between MBH and FSI network

Differences between
Financial and Telecom
Network Environment
Kamatchi Gopalakrishnan
Distinguished Engineer
Agenda
Network
Time-sync
Telecom
versus
Financial
Network
Time-sync
Profile
comparison
Summary
1
Time-sync basics and requirements
Synchronization overview
Frequency
Synchronization
Time Synchronization
Phase
Synchronization
TA=1/fA
A
A
01:00:00
01:00:10
01:00:00
01:00:10
TA=1/fA
A
t
TB=1/fB
B
t
fA=fB
B
TB=1/fB
B
fA=fB
fA=fB
Aligning clocks with
respect to frequency
Aligning clocks with
respect to phase
Aligning clocks with respect
to time.
FDD application
TDD application
FSI application
Synchronization methods and options
GNSS (GPS, GLONAS,
Galileo, BeiDou)
- Jamming, Spoofing, RF factors
- Expensive
- Line of site issue
NTP
- Good for milliseconds
- Not good for high precision
SONET/SDH, Sync-E
- Not for phase or time sync
- Every node must support
PTP
- High precision – Phase,
Frequency and Time
Telecom versus FSI network timing requirements
Telecom network
Frequency and/or phase sync
UTC traceability not a must.
Arbitrary time source is acceptable
Ranges from 100nsec to order of
microseconds – depends on MBH
applications
Operational requirements
FSI network
Time of day (ToD) sync
UTC traceability is must (ESMA) or
NIST traceability in US (SEC)
Must be less than 100usec from
UTC (ESMA)
Regulatory requirements
2
Timing in Telecom versus Financial network
Telecom versus FSI network model comparison
Telecom network model
Multiple administrative domain
FSI network model
(Mostly) Single or few cases multiple
administrative domain
Network services and transports not in Controlled network services and
control of single admin
transports
No direct leverage to upgrade network Leverage to upgrade/modify network
segments
segment
Downtime causes financial impact
Downtime causes both financial and
regulatory impact
Timing is operational requirement
Timing is a regulatory requirement
Telecom – Multi Operator network
Operator-3
Operator-2
Operator-1
GNSS
Access
Network
Backhaul Network
End to End 1.5 microsecond
Core GM
Mobile Core
Network
Mobile Phone roaming between cell-sites
BS1
BS2
BS2 drifts outside 50ppb window
+/-50 ppb
BS2
Mobile cannot lock to BS2, call dropped
+/-50 ppb
BS1
Phase (TDD) and Frequency (FDD) sync
Frequency Division Duplex
Time Division Duplex
Cell tower
Send to handset
at frequency F1
Cell tower
Time: 12:00.01
Send to handset
at frequency F1
Handset
Time: 12:00.05
Send to tower at
frequency F1
Handset
Send to tower at
frequency F2
F1
Tower
F2
Handset
F1
Tower
Hand
Tower
Hand
Tower
Time
Time
Requires: accurate frequency
Requires: accurate frequency & phase
Mobile application phase sync requirements
(Frequency : 16 ppb/ 50 ppb)
Application
CDMA2000
Phase
+/-3 us to +/- 10 us
LTE-TDD
LTE MBMS (LTE-FDD and LTE-TDD)
+/- 1.5 us (< 3KM cell radius)
+/- 5 us (> 3KM cell radius)
+/- 10 us
LTE-A CoMP
LTE-A eICIC
+/- 0.5 us to +/- 1.5 us
+/- 1.5 us to 5 us
E911 and Locating services
Small cells
+/- 0.1 us
+/- 3 us (1 to 5 us) with 100 to 250 ppb
Financial Network – Single Operator Domain
• 10 / 100 thousand servers.
• Boundary Clocks to serve
group of servers
•
EXCHANGE
CUSTOMERS
EXCHANGE
MARKET DATA FEED
HFT SERVERS
Compute Cluster
Interconnect
BACK END
COMPUTE
CLUSTER
MONITORING
APPLICATIONS
Synchronization flow model
Gateway
routers
Leaf
Switches
Spine
Switches
Time Legality
• How do you prove that something
happened before (or after) a certain
time?
• How do you correlate events across a
large/global network?
Time-sync requirements in FSI (MiFID-2/ESMA)
• RTS 25 of Regulatory and Implementing standards – annex 1
• Adopted regulatory requirements
 Reference Time – UTC traceable
 Compliance with maximum divergence requirements
 Level of accuracy for operator of trading venue
Gateway-to-Gateway
latency
Maximum divergence
from UTC
Time stamp Granularity
> 1 millisecond
=< 1 millisecond
1 millisecond
100 microseconds
1 millisecond or better
1 microsecond or better
Telecom
3
Financial/Ente
rprise
Profile comparison
What are profiles?
• Timing profiles are subset of requirements derived from IEEE1588
specification.
• Targeted for particular application to achieve required synchronization
goals in most reliable and interoperable way.
• Different Timing profiles:




Telecom profile – Mobile Backhaul applications
Enterprise profile – Enterprise/DC and financial application
802.1AS/AVB profile – Audio/Video in bridged network
SMPTE-2059-2 profile – IP based video broadcasting network
Telecom versus Enterprise profile
Telecom Profile
Enterprise Profile
•
•
•
•
•
•
•
•
•
•
PTP over Ethernet Multicast
Sync-E + PTP combined mode
Phase accuracy in microseconds
Fixed PTP packet rates
Alternate Best Master Clock
Algorithm (ABMCA)
• No Unicast Negotiation
PTP over IPv4 or IPv6 Multicast
Plain PTP mode
Time of day (less than 100us UTC)
Configurable packet rates
1588 default Best Master Clock
Algorithm
• Unicast negotiation allowed
4
Summary
Summary - Precision Timing Challenges
•
•
•
•
•
Packet Delay Variation
Scaling - Number of PTP clients support
Number of hops between GM and End Slave nodes
Precision Performance Monitoring
Overlay versus Inline synchronization flow
Thank you
Backup slides
Case1 - Accuracy in Hi-FREQ algorithmic trading
Trade Execution
Servers
Market Data
Creation Servers
Market Feed
Generator Servers
Trade Execution Market Data Creation
T2
T3
Timestamp
Timestamp
T1
Market Feed
Timestamp
2
3
4
5
6
7
1
With NTP precision, often T3 <
T2 < T1 – ie, market data is sent
before it is created and even
The
market
data
feed
reaches
creation
the
The
trade
market
execution
feed
generator
data
before
trade
has
been
When
athe
trade
executes,
a is sent
The
market
data
is
sent
to the
servers
customers’
generate
algorithmic
a timestamp
servers
to
servers
the
market
timestamp
data
the
creation
settled!
Algorithms
arefeed
confused
timestamp
is generated
by the
feed
generator
servers
when
with
the
new
three
market
timestamps
data is
servers
which
they
send
out
leading
to lost business
trade execution
server and
created
embedded
angry customers for the
exchange
With PTP, exchanges can achieve better
precision (1 us or less) that will let them
fix this problem
T3
Market Feed
Timestamp
Algorithmic
Trading Servers
T2
Market
Data
T1
Trade Data
Algorithmic Trading
Algorithmic
Servers
Trading Servers
Case-2: End To End Latency Analysis
• In HFT, latency is king
• Different latency components
• PTP based timing to correlate
measurements across nodes
Market data feed
BGP/OSPF
IP Multicast
HFT servers
HFT servers
Case 3: logging for regulatory reasons
Customer Trade
Requests
Buy
request
arrives
from
Customer
X for IBM
options at
Buy
request
arrives
from
Customer
With
NTP
precision,
it might
thatYAfor
T3
< T1options
< T2,
soatthis
Trade
is executed
(“filled”)
bybeserver
atIBM
time
T2
– log
server
A –– timestamped
timestamped with
with T1, record
record sent to
to the
the logging
logging
server
is
how Cthe
like
whenT3,
read aftersent
the fact:
12
4
record
sentlog
to looks
logging
server
3 server
server
Market data feed
BGP/OSPF
IP Multicast
T1
T2
Customer
T3 X
A
B
Buy
IBM
@250
Customer
Fill IBM
Y @250
Buy IBM @251
C
HFT Algorithmic Trading
Servers
Logging Server
Raises regulatory concerns around fair trading
Precise timing provided by 1588 is needed to achieve accuracy in logging operations to
alleviate regulatory problems
MiFID-2/ MiFIR and ESMA
• MiFID – Markets in Financial Instrument Directive
• MiFIR – Markets in Financial Instrument Regulation
• ESMA – European Security Market Authority
•
•
Regulatory Technical Standard (RTS)
Implementing Technical Standard (ITS)
• Imposes:
 Fairer, safer and more efficient markets
 Greater transparency
 Stronger investor Protection