Download moreiraMiniWorkshop - Indico

GBT Project Status
Paulo Moreira
Mini Workshop of the Joint ATLAS CMS Opto Working Group
4th – 5th March 2010, CERN
http://cern.ch/proj-gbt
GBT – People
Luiz Amaral – CERN, Switzerland
Mohsine Menouni – CPPM, France
Sophie Baron – CERN, Switzerland
Alessandro Marchioro – CERN, Switzerland
Sandro Bonacini – CERN, Switzerland
Frederic Marin – CPPM, France
Jean-Pierre Cachemiche – CPPM, France
Stefano Meroli – INFN, Italy
Bruno Checcucci – INFN, Italy
Paulo Moreira – CERN, Switzerland
Jorgen Christiansen – CERN, Switzerland
Christian Paillard – CERN, Switzerland
Ozgur Cobanoglu – CERN, Switzerland
Nataly Pico – SMU, USA
Federico Faccio – CERN, Switzerland
Antonio Ranieri – INFN, Italy
Philippe Farthouat – CERN, Switzerland
Giuseppe De Robertis – INFN, Italy
Tim Fedorov – SMU, USA
Angelo Rivetti – INFN, Italy
Rui Francisco – CERN, Switzerland
Sergio Silva – CERN, Switzerland
Alessandro Gabrielli – INFN, Italy
Csaba Soos – CERN, Switzerland
Tullio Grassi – University of Maryland
Filipe Sousa – CERN, Switzerland
Ping Gui – SMU, USA
Jan Troska – CERN, Switzerland
Paul Hartin – SMU, USA
Francois Vasey – CERN, Switzerland
Kostas Kloukinas – CERN, Switzerland
Ken Wyllie – CERN, Switzerland
Gianni Mazza – INFN, Italy
Bryan Yu – SMU, USA
http://cern.ch/proj-gbt
[email protected]
2
Outline

Radiation Hard Optical Link Architecture
•
GBTX-TO-FRONTEND:
•
•
•

The GBT Chipset:
•
•
•

GBT Link Bandwidth
Parallel Modes
E-Link Modes
Gigabit Laser Driver (GBLD)
Gigabit Trans-Impedance Amplifier (GBTIA)
Gigabit – Serializer/De-serializer (GBT-SERDES)
E-Links
•
•
•
E-Port
SLVS Transceiver Macrocell
Scalable Low-Voltage Signalling (SLVS)

GBT on FPGAs

GBT Specifications

GBT project schedule
http://cern.ch/proj-gbt
[email protected]
3
Radiation Hard Optical Link Architecture
Defined in the “DG White Paper”
Radiation Hard Optical Link:


“Work Package 3-1”
•
Objective:
•
Deliverable:
•
Duration:
•
•
•
•
•
•
Development of an high speed bidirectional
radiation hard optical link
•
Tested and qualified radiation hard optical link
4 years (2008 – 2011)
Versatile link project:

GBT project:
•
•
•
•
On-Detector
Custom Electronics & Packaging
Radiation Hard
http://cern.ch/proj-gbt
Opto-electronics
Radiation hardness
Functionality testing
Packaging
[email protected]
ASIC design
Verification
Functionality testing
Packaging
Off-Detector
Commercial Off-The-Shelf (COTS)
Custom Protocol
4
GBT Link Bandwidth

Bandwidth:
•
•

Dedicated channels:
•
•




•
•
Efficiency: 73%
To be compared with 8B/10B: 80%
•
Link is symmetrical:
Down-link highly flexible:
•
•
Scrambler
No bandwidth penalty
Forward Error Correction and Frame
Synchronization
Link is bidirectional
•
Link control: 80 Mb/s
Data/Slow control channel: 80 Mb/s
DC balance:
•
•

User: 3.36 Gb/s
Line: 4.8 Gb/s

Can convey unique data to each frontend device
that it is serving
“Soft” architecture managed at the control room
level
Other schemes would require dedicated
topologies that will be difficult to accommodate
on a generic ASIC like the GBTX
Line code and frame structure compatible with
modern FPGAs
no error correction capability
http://cern.ch/proj-gbt
[email protected]
5
GBTX-TO-FRONTEND: Parallel Modes

P-Bus Mode:
•
•
•
•

B-Bus Mode:
•
•

Simple parallel interface
40-bit wide bus
Bidirectional
Double Data Rate (DDR)
A byte-bus mode is also available
Up to five independent buses can be used
simultaneously
Electrical levels:
•
SLVS electrical level:
•
•
•
•
100 W termination
400 mV differential
200 mV common mode
ILOAD = ± 2 mA
JEDEC standard, JESD8-13
Scalable Low-Voltage Signalling for 400 mV (SLVS-400)
http://www.jedec.org/download/search/JESD8-13.pdf
http://cern.ch/proj-gbt
[email protected]
6
GBTX-TO-FRONTEND: E-Link Modes

GBT/Frontend interface:
•
•
•
•

Programmable data rate:
•
•
•



Type
Data Rate
OFF
Power off
-
B-Bus
parallel
80 MB/s
Up to 5 Bytes (DDR)
P-Bus
parallel
80 MW/s
One 40-bit word (DDR)
2×
serial
80 Mb/s
Up to 40 serial links
4×
serial
160 Mb/s
Up to 20 serial links
8×
serial
320 Mb/s
Up to 10 serial links
8×
serial-lanes
> 320 Mb/s
http://cern.ch/proj-gbt
Notes
Electrically
“Protocol”
Package (preliminary):
•
•
[email protected]
Three pairs: DOUT/DIN/CLK
SLVS
E-Links will be handled by E-ports:
•
•

80 Mb/s
E-Link:
•
•

To achieve > 320 Mb/s
Two or more e-links can be grouped
forming a “lane”
Slow control channel:
•
Mode
Independently in five groups
Independently for up/down links
80 Mb/s, 160 Mb/s and 320 Mb/s
Lanes:
•
•
SEU tolerant
Electrical links (e-link)
Serial
Bidirectional
Up to 40 links
BGA: 361 – PINS
16 mm x 16 mm, 0.8 mm pitch
7
The GBT Chipset

•
•
•
•


Radiation tolerant chipset:
GBTIA: Transimpedance optical receiver
GBLD: Laser driver
GBTX: Data and Timing Transceiver
GBT-SCA: Slow control ASIC
Supports:
•
•
•
•
•

Data readout
TTC
Slow control and monitoring links.
Radiation tolerance:
•
•
Bidirectional data transmission
Bandwidth:
•
•
The target applications are:
Total dose
Single Event Upsets
Line rate: 4.8 Gb/s
Effective: 3.36 Gb/s
GBTIA
Data<119:0>
Clock<7:0>
GBTX
GBLD
Frontend
GBT-SCA
Electronics
Control<N:0>
http://cern.ch/proj-gbt
[email protected]
8
GBLD
Main specs:
•
Bit rate 5 Gb/s (min)
•
Modulation:
•
•
•
Laser modulation current: 2 to 12 mA
•
Laser bias: 2 to 43 mA
•
“Equalization”
•
•

current sink
Single-ended/differential
Pre-emphasis/de-emphasis
Independently programmable for rising/falling
edges
•
Supply voltage: 2.5 V
•
Die size: 2 mm × 2 mm
•
I2C programming interface
Packaging:
•
Part of the versatile link project
Engineers :
•
Gianni Mazza – INFN, Italy
•
•
Ping Gui – SMU, USA
Angelo Rivetti – INFN, Italy
•
Ken Wyllie – CERN, Switzerland
Status:
•
•
Chip fabricated and tested
A re-spin was necessary:
•
Chip submitted for fabrication: 16-Feb-2010
http://cern.ch/proj-gbt
[email protected]
9
GBLD – Test Results

GBL tests:
•
•

The problem was identified:
•
•

Chip is “functional” however the
bandwidth falls short of specifications!
Fortunately the pre-emphasis circuit is
working fine, allowing to partially recover
the bandwidth!
2.5 Gb/s
Parasitic capacitance evaluation
Layout symmetry
New version:
•
•
•
•
Submitted for fabrication: 16-Feb-2010
Pre-driver load resistance halved
Cascode transistors bulks connected to a
reference voltage
Metal stack : LM → DM
•
•
Voltage regulator:
•
•
•
Single layer aluminum inductors (higher
series resistance but lower parasitic
capacitance)
Single supply voltage
PW modulation circuit removed
Pre-emphasis added on both outputs
•
Full symmetry
•
I2C controller:
•
•
Package: QFN 24-pin 4 mm x 4 mm
Design database : CDB → OA
•
5 Gb/s
ARM std cells → IBM std cells
http://cern.ch/proj-gbt
[email protected]
10
GBTIA
Main specs:
•
•
•
•
•
•
•
•
Bit rate 5 Gb/s (min)
Sensitivity: 20 μA P-P (10-12 BER)
Total jitter: < 40 ps P-P
Input overload: 1.6 mA (max)
Dark current: 0 to 1 mA
Supply voltage: 2.5 V
Power consumption: 250 mW
Die size: 0.75 mm × 1.25 mm
Packaging:
•
Part of the versatile link project
Engineers :
•
•
Ping Gui – SMU, USA
Mohsine Menouni – CPPM, France
Status:
•
Chip fabricated and tested
•
•
Chip fully meets specifications!
Radiation tolerance proven!
•
Work has started to encapsulate the GBTIA +
PIN-diode in a TO Package
•
(Versatile link project)
http://cern.ch/proj-gbt
[email protected]
11
GBTIA – Test Results
-6 dBm
-18 dBm (specs: -17 dBm)
BER versus Total dose
1E-04
pre-rad
10M
1E-06
200M
BER
1E-08
1E-10
1E-12
1E-14
-21
-20
-19
-18
-17
Optical input level (dBm)
http://cern.ch/proj-gbt
[email protected]
12
GBT-SERDES
Serial
input
120
DES
Frame
Aligner
120
Switch
FEC
Decoder
120
120
Switch
De-scrambler
Header decoder
120
120
Switch
Parallel
Out/
BERT
dOut [29:0]
rxDataValid
rxClock40
rxClock160
ClkOut3
120
120
Phase
Shifter
120
ClkOut2
ClkOut1
ClkOut0
RX: 40 MHz & 160 MHz
Clock
reference
Clock
Generator
rxRdy
txRdy
TX: 40 MHz & 160 MHz
Control
Logic
I2C
JTAG
AUX[n:0]
RST
dIn [29:0]
Serial
out
120
SER
120
Switch
Full custom
Engineers:
Ozgur Cobanoglu - CERN, Switzerland
Federico Faccio - CERN, Switzerland
Rui Francisco – CERN, Switzerland
Ping Gui – SMU, USA
Alessandro Marchioro - CERN, Switzerland
Paulo Moreira - CERN, Switzerland
Christian Paillard - CERN, Switzerland
Ken Wyllie - CERN, Switzerland
http://cern.ch/proj-gbt
FEC
Encoder
120
120
Switch
Scrambler
Header encoder
Data path
Clocks
120
Switch
120
Parallel
In/
PRBS
txDataValid
txClock40
txClock160
Full
PROMPT
custom
Control bus
 Submitted for fabrication: 26-Nov-2009
 Currently being packaged.
[email protected]
13
GBT-SERDES
Slow Control & Monitoring ASIC: GBT - SCA
GBT-SCA Main specs:
•
Dedicated to slow control functions
•
Interfaces with the GBTX using a dedicated
E-link port
•
Communicates with the control room using
a protocol carried (transparently) by the
GBT
•
Implements multiple protocol busses and
functions:
•
•
I2C, JTAG, Single-wire, parallel-port,
etc…
Implements environment monitoring
functions:
•
•
Temperature sensing
Multi-channel ADC
Engineers:
•
•
•
•
•
•
Alessandro Gabrielli – INFN, Italy
Kostas Kloukinas – CERN, Switzerland
Alessandro Marchioro – CERN, Switzerland
Antonio Ranieri – INFN, Italy
Giuseppe De Robertis – INFN, Italy
Filipe Sousa – CERN, Switzerland
Status
•
Specification work undergoing:
•
1st Draft already available
•
RTL design undergoing
•
Tape-out: 2010
http://cern.ch/proj-gbt
[email protected]
15
E-Links: e-port
The FE interfaces with the GBTX through an
e-port
No protocol added on data stream
The e-port handles:
•
•
•
•
•
•
The physical interface;
No overhead
•
No frame/word alignment
•
Fixed latency
Two protocols are available as possible additional
MAC wrappers:
The multiple data rates;
The lanes (for bandwidth > 320 Mb/s)

Line coding:
Clock recovery (if required)
AC coupling (if required)

The user application does not have to care about
the frame formats in full detail:
•
•
It s done through a well defined interface!
An E-Link Port Adaptor (EPA) “macro” will be
available for integration in the front-end ASICs
7B/8B
•
Balanced, fixed latency, suitable for trigger commands links
•
RTL code under development
High-Level Data Link Control (HDLC)
•
Packet oriented data, high bandwidth efficiency (~96%)
•
Non-fixed latency, suitable for slow control and data links
•
RTL code ready
To be specified
•
Lanes support
FE ASIC
cset[3:0]
Engineers:
•
•
Sandro Bonacini – CERN, Switzerland
Kostas Kloukinas – CERN, Switzerland
To/From
GBT
shift
regster
e-port PHY
tx_sd
tx dat[7:0]
tx register
register
(negedge)
rx_clk
rx_clk40
clock
divider
register
(negedge)
rx_sd
rx dat[7:0]
rx register
shift
regster
rx_off
http://cern.ch/proj-gbt
[email protected]
16
SLVS Transceiver

•
•
Power Supply: 1.2V to 1.5V
Power Dissipation:
•
•

Electrical Specifications
Receiver
150uW @ 320Mbs, 1.2V supply
<1uW @ power down
Transmitter
•
•
Electrical Specifications
Power Supply: 1.2V to 1.5 V
Power Dissipation:
•
•
3.1mW @ 320Mbs, 1.2 V supply
<10uW @ power down
Programmable Output Current

Engineer
•
Sandro Bonacini – CERN, Switzerland
Status:
•
Chip currently under testing
http://cern.ch/proj-gbt
[email protected]
17
SLVS Tests
 SLVS (Scalable Low Voltage Standard)

•
•
 JEDEC standard: JESD8-13
 Main features:

 2 mA Differential max
 Line impedance: 100 Ohm
 Common mode ref voltage: 0.2V
5
2
Xilinx S3E
board
2
2
media
2
SLVS-RT
2
2
media

2
SLVS-RT
2
Xilinx S3E
board
1 driver
1 receiver
Various types of transmission media
tested:
•
•
•
 Signal: +- 200 mV
5
Tests on SLVS-RT chip
Kapton
PCB
Ethernet cable
Test equipment
•
•
at 320Mbps
Bidirectional link
FPGAs perform pseudo-random data
generation and checking
140 mV
min swing
20cm kapton
< 1.00E-13
3cm UTP
< 1.00E-13
1m PCB microstrip < 1.00E-13
2m PCB microstrip
3.20E-12
2m PCB stripline
1.05E-08
5m ethernet
* 2.37E-12
200 mV
half swing
< 1.00E-13 <
<
< 1.00E-13 <
9.00E-13
1.00E-12
* 1.60E-12
400 mV
nominal
1.00E-13
1.00E-13
1.00E-13
8.00E-13
8.00E-13
(*) PRELIMINARY
http://cern.ch/proj-gbt
[email protected]
18
GBT on FPGAs

GBT-SERDES successfully implemented in
FPGAs:
•

Scrambler/ Descrambler + Encoder/ Decoder
+ Serializer/CDR
FPGA Tested:
•
•

XILINX Virtex-4FX
ALTERA StratixII GX
Ongoing work:
•
Optimization of use of resources
•
Fixed and “deterministic” latency
•

Detailed implementation is device dependent
Firmware:
•
•
“Starter Kit” is now available for
download.
Available for:
•
•
Altera + opto TRx - 4.8 Gb/s
StratixIIGx and Virtex5FXT
Available soon for:
•

Xilinx - 4.8 Gb/s
StratixIVGx
Engineers:
•
•
•
•
Sophie Baron – CERN, Switzerland
Jean-Pierre Cachemiche – CPPM, France
Frederic Marin– CPPM, France
Csaba Soos – CERN, Switzerland
http://cern.ch/proj-gbt
[email protected]
19
GBT Specifications
Link specification group:

Formed in 2008

Members:
•
Electronics coordinators of:
•
ALICE, ATLAS, CMS and LHCb
•
Five members of the Radiation Hard Optical Link (RHOL) project

“Mandate”:
•
Identify the “GBT” needs of each experiment for the SLHC upgrade
•
Discuss the specification documents (before they are distributed within the collaborations).

Meetings:
•
1st meeting (CERN – 2008/04/17)
•
ALICE, ATLAS, CMS and LHCb electronics coordinators presented outlooks of their requirements for SLHC.
nd
•
2 meeting (CERN – 2008/11/14)
•
GBT system proposal was presented to the electronics coordinators.
rd
•
3 meeting (CERN – 2009/05/05)
•
Link specification feedback
Documents:

Share point web site created (2008):
•
http://cern.ch/proj-gbt

Specification documents:
•
GBTX specifications (V1.0, January 2009)
•
GBTIA specifications (V1.7, May 2008)
•
GBLD specifications (V2.0, July 2008)
•
GBT-SCA specifications (V1.5, June 2008)
•
E-Port IP 7B8B specifications (V0.1, December 2008)
•
E-Port IP Core specifications (V0.2, January 2009)
•
E-Port IP HDLC specs (V0.2, January 2009)
http://cern.ch/proj-gbt
[email protected]
20
Project Schedule

2008
•
Design and prototyping of performance critical building blocks:
•
•
First tests of optoelectronics components
•
•
•

Proceed with the link specification meetings
General link specification
Design/prototype/test of basic serializer/de-serializer (GBT-SERDES) chip
•
•
GBT-SERDES (“Tape-out” 9th of November)
Design/prototype/test of optoelectronics packaging
•
GBTIA + PIN on TO CAN
2010
•
•
•
•
•

SEU tests on PIN receivers
2009
•

GBTIA, GBLD, Serializer, De-Serializer, Phase Shifter
!
GBLD re-spin
GBT-SERDES testing
Detailed link specification document
Full prototype of optoelectronics packaging
Prototype of “complete” GBTX chip
2011
•
•
•
?
Extensive test and qualification of full link prototypes
System demonstrator (s) with use of full link
Schedule of the final production version is strongly dependent on the evolution of the LHC
upgrade schedule
http://cern.ch/proj-gbt
[email protected]
21
Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides

Additional slides
http://cern.ch/proj-gbt
[email protected]
22
GBT Chipset – Power Consumption
GBT ASICS Power Consumption breakdown
GBTIA
GBLD
GBTX
Summary
GBTIA
GBLD
GBTX
Total (max)
Power [mW]
123.7
585.9
1092.3
1801.9
GBTX – Power Consumption
GBTX Power consumption breakdown
SER
CDR
PHASE SHIFTER
LOGIC
E-link input data buffers (40)
E-link de-serializers (10)
E-link output data buffers (40)
E-link clock buffers (40)
E-Link serializers (10)
GBTX Circuit
SER
CDR
PHASE SHIFTER
LOGIC
E-link input data buffers (40)
E-link de-serializers (10)
E-link output data buffers (40)
E-link clock buffers (40)
E-Link serializers (10)
Total (max)
Supply [V]
1.5
1.5
1.5
1.5
1.5
1.5
1.5
1.5
1.5
1.5
Current [mA] Power [mW]
202.1
303.2
194.8
292.2
73.3
110.0
43.3
65.0
10.0
15.0
2.3
3.5
100.0
150.0
100.0
150.0
2.3
3.5
728.2
1092.3
Comments
Simulation
Simulation
Simulation
Synthesis (12 mW leakage + 53 mW dynamic)
Simulation (SLVS receiver)
Estimated
Simulation (SLVS driver with maximum current settings)
Simulation (SLVS driver with maximum current settings)
Estimated
GBLD – Power Consumption
GBLD Power consumption breakdown
Input stage
Pulse width control
Pre-driver
Output driver (2)
Delay line
Emphasis pulse generator (2)
Emphasis driver
Laser bias
DACs
Bandgap
Voltage regulator
GBLD Circuit
Input stage
Pulse width control
Pre-driver
Output driver (2)
Delay line
Emphasis pulse generator (2)
Emphasis driver
Laser bias
DACs
Bandgap
Voltage regulator
Total (max)
Typical: edge-emitting diode
Typical: VCSEL
Supply [V]
1.5
1.5
1.5
2.5
1.5
1.5
2.5
2.5
1.5
1.5
2.5
2.5
2.5
2.5
Current [mA] Power [mW]
11.5
17.3
24.0
36.0
10.0
15.0
48.0
120.0
30.0
45.0
26.0
39.0
24.0
60.0
43.0
107.5
13.7
20.6
0.2
0.2
4.0
10.0
234.4
585.9
234.4
585.9
153.4
383.4
Comments
Simulation
Simulation
Simulation
Simulation (Half of the figure when used as a VCSEL driver)
Simulation
Simulation (Half of the figure when used as a VCSEL driver)
Simulation (can be powered off)
Simulation (can be powered off)
Simulation
Simulation
Estimated (not included in the 1st prototype)
Total power assumes an internal regulator and the chip fully powered from 2.5V
Ibias = 43 mA, Imod = 24 mA
Ibias = 7 mA, Imod = 8 mA
GTIA – Power Consumption
GBTIA Power consumption breakdown
Pre-amp
Post-amp
Offset compensation
Bias
Output buffer
Voltage regulator
Carrier - detection
GBTIA Circuit
Pre-amp
Post-amp
Offset compensation
Bias
Output buffer
Voltage regulator
Carrier - detection
Total
Supply [V]
2.0
2.0
2.0
2.0
2.0
2.5
2.0
2.5
Current [mA] Power [mW]
14.0
28.0
19.0
38.0
0.2
0.5
3.0
6.0
11.0
22.0
2.0
5.0
0.3
0.5
49.5
123.7
Comments
Simulation
Simulation
Simulation
Simulation
Simulation
Estimated (not included in the 1st prototype)
Estimated (not included in the 1st prototype)
Total power assumes an internal regulator and the chip fully powered from 2.5V
Forward Error Correction (FEC)
High rates of Single Event Upsets (SEU) are
expected for SLHC links:
•
•
Experimental results confirmed that for Errorrates below 10-12 Error Correction is
mandatory!
•

Upsets lasting for multiple bit periods have been
observed
Proposed code:
•
•
•
•
Interleaved Reed-Solomon double error
correction
4-bit symbols (RS(15,11))
Interleaving: 2
Error correction capability:
•
•
•
Code efficiency: 88/120 = 73%
Line speed: 4.80 Gb/s
Coding/decoding latency: one 25 ns cycle
•

BER

Particle “detection” by Photodiodes used in optical
receivers.
SEUs on PIN-receivers, Laser-drivers and SERDES
circuits
2 Interleaving × 2 RS = 4 symbols  16-bits
GBT frame efficiency: 70%
•
•
•
A line code is always required for DC balance and
synchronization
For comparison, the Gigabit Ethernet frame
efficiency is 80% (at the physical level)
At a small penalty (10%, when compared with
the Gigabit Ethernet) the GBT protocol will offer
the benefits of Error Detection and Correction
BER

A. Pacheco, J. Troska, L. Amaral, S. Dris, D. Ricci, C. Sigaud, F. Vasey, P. Vichoudis,
"Single-Event Upsets in Photoreceivers for Multi-Gb/s Data Transmission,”
Nuclear Science, IEEE Transactions on , vol.56, no.4, pp.1978-1986, Aug. 2009
http://cern.ch/proj-gbt
[email protected]
27