Download hcal_tridas_sept_03 - UMD Physics

Tridas Status
Drew Baden
University of Maryland
September 2003
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
1
HTR Status
• Rev 3 – 30 boards made in
March 2003
– Board production changes:
• New assembler, in-house Xray, DFM review, QC
• Gold plated (Rev 1 was whitetin) for better QC
TTC
mezzanine
VME
Stiffeners
Dual-LC O-to-E
– Changes to HTR:
• Change from FBGA (1.00mm)
to BGA (1.27mm)
• Added stiffeners
• Moved all SLB/TPG output to
front-panel daughterboards
• Modified Rx refclk scheme (the
usual TTC/refclk clocking
concerns)
Deserializers
SLBs
(6)
Xilinx
XC2V3000-4
– Full 48 channel capability
• Rev 1 in 2002 was “half HTR”
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
2
TTC receiver - TTCumd
• General purpose TTC receiver
board (TTCumd)
Princeton Fanout
Card
– TTCrx ASIC and associated
PMC connectors
• Will be used to receive TTC
signal by HTR, DCC, and clock
fanout boards
• No signal receivers.
– Copper/fiber receivers must be
on the motherboard
– Signal driven through TTC
connectors
• Tested successfully by
Maryland, Princeton, BU groups
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
3
Clocks and Synchronization
• Clocking considerations can be divided into 2 parts:
– Deserializers REFCLK: stability critical (80MHz frame rate)
• Stability: must have a very low jitter – 40ps pkpk spec
• Frequency: TI TLK2501 spec is 100ppm (8kHz) to lock
– Measured ~350ppm (30kHz) needed to establish link
» LHC variation expected to be few kHz
– Once link is established, just needs to be stable (it’s a REFCLK!!!)
• Phase: relationship to LHC clock totally irrelevant
– Phase critical clock for pipeline synchronization
• Must be in phase with LHC clock
• Jitter spec is less stringent
– For FPGA sequential logic
– For SLB transmission 80ps pkpk
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
4
HCAL Clock Fanout
• HTR clocks provided by a single 9U VME board
– Chris Tully/Jeremy Mans from Princeton
– Has fiber TTC input
• Signals fanned out over Cat6 twisted pair:
– TTC stream
• To be used by each HTR and by DCC to decode commands & L1A
– BC0
• To be used by SLBs to synchronize TPGs
– “40MHz” clock
• To be used by FPGA and SLBs to maintain pipeline
– Comes from QPLL
– “80MHz” clean clock
• To be used for deserializer REFCLK
– Comes from QPLL
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
5
Clock Distribution
HTR
HTR
HTR
TTC fiber
O/E
TTC
TTC
TTC
Brdcst<7:0>,
BrcstStr, L1A
TTC
..
BC0
TTCrx
..
CLK40
Princeton
Fanout
Board
CMS/HCAL/TriDas. Sept, 2003
..
FPGA
QPLL
TTCrx
..
..
Cat6E
or Cat7
Cable
BC0
..
..
..
CLK40
distribution
to 6 SLBs
and to 2 Xilinx
CLK80
CLK80
Test
..
Points for
..
RxCLK
and RxBC0
80.18
MHz
HCAL TriDAS
to Ref_CLK of
SERDES
(TLK2501)
6
May Testbeam Setup
• Lack of a QPLL – we had to improvise:
– Front-end used commercial Cypress PLL
• Beats GOL 100ps pkpk jitter spec in the lab
• Terry Shaw measured sufficiently small jitter on backplane
– Fanout card
• No clean 80MHz REFCLK, so provided 2 alternatives:
– 2xLHC clock from crystal oscillator
– High quality clock from HP signal and pulse generators
– Jumper selectable on mezzanine cards
• No clean 40MHz system clock
– Just used 40MHz output from TTCrx chip anyway
– Worked well enough
» The system clock doesn’t need a low jitter spec, and Xilinx has a DLL to clean it up
– Much struggling with the fiber links, synchronization issues, etc.
• Almost all issues
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
7
May 2003 Testbeam
• What we struggled with and learned about
– Fiber links
• FE Tx clock critical. No QPLL so we used backup plan.
• Great deal of experience gained here.
– TTC
• Had a few minor issues, discover how to best synch L1A with pipeline
– DAQ
• Production DCC logic board
– Able to operate SLINK64 after BU group discovered bug in documentation
– Logic in place for inline error checking, data integrity, etc
• XDAQ implementation experience
– Calorimeter
• Took useful calorimeter data!
– HB, HE, HO, HF
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
8
May Testbeam Experience
• Fiber synchronization
– FE
• With crystal oscillator on FE, virtually no troubles
• TTC 40MHz clock cleaned up by Cypress “roboclock” chip (Cy7B993)
– Many synch errors, varied from fiber-to-fiber, hard to maintain link on all fibers
over ~few hours time period
– During synchronized beam running, sent reset between spills to ensure link.
This seemed to work ok. Exploring using rescynch in abort gap…
– Fanout card
• Fanout from onboard ~80MHz crystal oscillator for REFCLK
• Fanout TTC 40MHz clock for system clock
– HTR
• 40MHz system clock scheme worked
– Some firmware changes necessary to synchronize with L1A
• TLK2501 link circuitry always enabled.
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
9
Fiber Links (cont)
• Tests at Maryland using TI eval board and Stratos transmitter
– Link exceedingly stable (see slides below)
• Current plan
– Study FE → HTR link at FNAL
• FNAL test stand is setup, link tests underway, more to come this fall
– Investigate noise characteristics of H2 environment
• H2 is clearly different than FNAL, Maryland (and BU) experience
– Review of HTR and Fanout card
• Will learn what we need to do from the above
• Best guess
– All tests in US indicate solid link, but experience in H2 disagree
– Probably a linear combination of:
• TTC clock jitter (most likely)
• Some kind of new noise component (less likely – FE works with crystal oscillators)
• Bottom line:
– We have a system that works in our labs (BER<10-14) but…
• H2 environment? Need more studies during the next 6-12 months
• Lack of working QPLL. Looking forward to getting correct QPLL/crytals
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
10
Optical Attenuation and BER
• “Typical setup”
– VECSEL transmitter, coupled to fiber via LC connector
• Not locked, but fixed in place
– Fiber to LC to 8-way MTP male on HTR front panel
– Single fiber to LC connector for connection to STRATOS receiver
• Output power:
– VECSEL advertised to put out 500mW (-3dBm)
• Terry Shaw measured 570mW for a particular VECSEL
– UMD uses STRATOS LC transmitter
•
•
•
•
Advertised output 100-400mW (-4 to -10dBM)
Measured to be 90mW for a particular STRATOS
About 6dB below what we will use in CMS
Working on FE emulator now using GOL+VECSEL…
• Attenuations measured:
– At each LC connector, 10 – 50% (0.5 to 1.5 dBm)
– At MTP connector, same thing (.75dB advertised)
– Fibers are about ¼ dBM per 100m
MTP (8-way)
VECSEL
HTR
FE
Stratos
LC
LC
LC
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
11
Optical Power (cont)
• What do we need at the receiver to
maintain link?
Points with error bars are worst case BER: <1 error
– Did a series of measurements with
known attenuator
– Varied attenuation, looked at:
• BER
• TTL “signal detect” (SD) signal provided
by Stratos part
Measured ~5k errors in 10sec
– Found:
• SD signal goes away when power is
below about 2mW
See next slide
– Measured 1.5mW but accuracy of meter
is probably ±.2mW
• BER climbs very fast right at this
shoulder
• NB: achieved BER<10-15 with
multiple fibers in parallel with
crystals
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
12
Link Loss Observation
• Ran a test at 7.4mW (-21dBm), found 44 errors in 18 hours
• Simultaneously triggered scope on VCC to transmitter and
crate
• Found that the scope had triggered on transmitter voltage
but not crate
– Voltage spike - 1-2V oscillation with ~50ns rise time
– Crate was isolated through UPS
• These errors were due to a spike on the A/C line from
equipment being power cycled, noise in building, etc.
• Moral: noise in the power/grounds will be our nightmare
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
13
Optical Attenuation (cont)
• Input power required to maintain link:
– Measured failure for power < ~2mW (33dBm)
• Power output by VECSEL:
– 500mW output
• Divide by 2 for digital averaging
• Gives 250mW (-6dBm) output at source
• Expected Attenuations
– Maximum of 8 couplings until the signal
gets to the Stratos receiver on the HTR
VECSEL
Operating
• 8x(0.5 - 1.5)dBm = (4 – 12)dBm
– Add another ~1dBM due to fibers
• Total power at inputs to HTRs:
– -6dBm – (4-13)dBm = -10 to -19 dBm
– FNAL measured/calculated 7.3dB
• Operating would be -13dBm
• We should have at least 10dB margin
– Probably more like 15dB
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
14
Longitudinal Separation Attenuation
• MTP connector ends are spring
loaded into adapter
• Measured attenuation as a
function of the separation
– Separation should be ~0 if keys
and adapters are working well
– This should not be an issue for us
(famous last words….)
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
15
Changes to HTR for Rev4
• Moved 2 LC’s down to giver more
clearance for fibers
– Upper rear of card
• Spread out routing of differential
pairs for 6 SLB and 2 FPGA
system clocks
• Removed hot swapping circuits
– Worry about noise, decided not to
require HTR to be hot swappable
• Front-panel changes
– Rotary switch, LEDs, eliminate
REFCLK mux via jumper, etc
• Miscellaneous changes
– Fixed what was found to be wrong with
Rev3 board, add test points, other
minor stuff
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
16
Project Timeline
2005
2004
2003
“Magnet” Tests (no HCAL)
H2 Testbeam
“Vertical Slice” (~March)
HB, HE, and ME plus Level 1/TPG
Fiber synchronization
Check in alcove…compare environments
Firmware
TPG firmware has to be ready. Fall 03 task.
Boards
Must have Rev4. How many TBD.
CMS Magnet Integration System Test
Boards
Will have Rev4.
Fiber synchronization
Check in H2…compare environments
Boards
Will have Rev4. How many TBD.
FE Commissioning (~Feb)
Firmware
TB 03 firmware should be adequate
Boards
Can use current crop of Rev3
CMS/HCAL/TriDas. Sept, 2003
To do:
Commission QPLL, global clocking.
Level 1 Trigger: SLB, TPG, latency…
HCAL TriDAS
17
HTR Firmware
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
18
TPG Path
• Still under development
– The following is already coded/simulated but not tested in HTR
• Things to do (not necessarily in order)
– Linearize QIE data to 10 bits
• With .5GeV resolution gives 512 GeV max
– Apply BCID filter
• Probably will sum over 2 buckets and assign based on high/low patter
around the bucket that has the max energy
–
–
–
–
Sum or divide depending on HB, HE, or HF
Extract a muon window for the “feature bit”
Apply logic to eliminate false muons from shower leakage, etc.
Compress and send to SLB
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
19
TPG Path Schematic
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
20
TODO - TPG
• HTR production can begin after:
– SLB/HTR connectivity check
• Logic analyzer card on SLB site
already shows connectivity
• Need to plug in real SLBs to check that
they can coincide
– SLB/HTR/Wisconsin check
• Check that we can maintain link
• Will bring the setup from Princeton to
Maryland this month and do the tests.
Measure BER, etc.
• Data validation
– We propose to build a 6U VME board
with sites for the Wisconsin Vitesse
receiver boards
– Will fifo data and read out over VME
– Use this to check data validity, try
different TPG tests, etc.
– Plan to have this board ready in a few
months. Will use this for the early
slice tests to test TPG output.
CMS/HCAL/TriDas. Sept, 2003
Clock Input
HCAL TriDAS
FPGA
21
TODO (cont)
• Firmware
– Focus on L1 latency optimization
• Measure full latency, scheme for random TLK latency, how to meet L1
latency budget, etc.
• This will be the main activity from now until we go into production.
– Bells and whistles for error reporting/recover
• Meeting next week in Princeton to finalize what we learned in 2003
• More to be learned in 2004
• HTR boards
– Make Rev4 this fall, go into production early 04
• ASAP given schedule
• Depends on results from above
CMS/HCAL/TriDas. Sept, 2003
HCAL TriDAS
22