Download FECC-III Integration Issues

SLAC
Stanford Linear Accelerator Center
FECC-III
INTEGRATION
ISSUES
Eric J. Siskind
June 12, 2003
SLAC
Project Goals
• Replace Multibus-I micro hardware with
commercial-off-the-shelf personal
computers.
• Replace special purpose accelerator
networks (SLCnet, KISnet, PNET) with a
single commercial-off-the-shelf TCP/IP
network such as switched gigabit Ethernet.
• Develop personal computer CAMAC
interface to replace MBCD/MBCD-II.
E. Siskind, June 12, 2003
2
SLAC
New CAMAC Features
• CAMAC interface differs from
MBCD/MBCD-II in four major ways:
– Two board design to support remote I/O;
– Real-time preemptive hardware with recovery;
– Maximizes CAMAC utilization when user
software environment in PC is multi-threaded;
– Large (i. e. comparable to existing micro) user
programming environment within interface.
E. Siskind, June 12, 2003
3
SLAC
Hardware Architecture
• Hardware consists of two circuit boards connected
by up to 10 km of single mode fiber optics:
– PCIL (“PCI Link”) board plugs into PCI option slot in
PC;
– FECC (“Front-End CAMAC Controller”) is now a
stand-alone chassis with a single-width CAMAC
module to access PDU triggers.
• PC moves from micro’s current location to MCC
computer room, leaving behind only FECC.
E. Siskind, June 12, 2003
4
SLAC
PCIL Features
• Basically an intelligent special-purpose Network
Interface Card.
• Processor with DMA PCI block transfer master
and DMA fiber optic link transmitter/receiver.
• Makes no assumptions about nature of PC’s realtime operating system.
• Semi-static assembly language PROM code with
downloaded updates.
• First generation hardware doubles as Alpha
SLCnet interface.
E. Siskind, June 12, 2003
5
SLAC
PCIL Code Functions
• Supports PC FECC message passing.
• Supports FECC remote read/write access to PC
memory with optional PC interrupt.
• Performs function-specific scatter/gather DMA
to/from PC memory for all classes of operations,
avoiding unnecessary buffer copying by PC CPU.
• Forwards trigger pattern from PCI registers to
FECC (only in 1st generation; pattern forwarding
in hardware in 2nd generation).
E. Siskind, June 12, 2003
6
SLAC
FECC Features
• Logical successor to MBCD/MBCD-II.
• Accesses host memory via point-to-point fiber
optic link instead of parallel I/O bus.
• 1st generation supports MBCD-II’s four strings of
SLAC serial CAMAC crate controllers (5 MHz,
bit serial, half duplex).
• 2nd generation adds support for four strings of
IEEE standard crate controllers (5 MHz, byte
serial, full duplex) and dual Bitbus strings.
E. Siskind, June 12, 2003
7
SLAC
CAMAC Interrupt
Recovery & Control
• If a CAMAC block transfer is interrupted,
hardware can repeat the previous write operation
(address pointer load?) with updated write data
before resuming the transfer.
• Hardware can prevent an interrupt between
CAMAC cycles transferring two 16-bit halves of a
single 32-bit word.
• Hardware can prevent an interrupt of a particular
block transfer entirely (needed because of a
current PIOP feature).
E. Siskind, June 12, 2003
8
SLAC
System Performance
• Many PC jobs now can have simultaneously
outstanding long CAMAC packages.
• All packages are sent to FECC firmware ASAP,
and parsed into their individual (block transfer)
operations in a single crate (“packets”).
• Packets are immediately queued to hardware for
the cable accessing the target crate.
• Cable operation begins as soon as any outstanding
package has a packet requiring access to a crate on
that cable.
• Parallelism now achieved over multiple packages.
E. Siskind, June 12, 2003
9
SLAC
PCIL2 Intelligence
• Analog Devices ADSP-21060 “SHARC” 40
MHz 32-bit DSP on 2nd generation PCIL.
• 40k x 48 bit on-chip program memory; 64k
x 32 bit on-chip data memory.
• 256k x 48 bit external program/data
memory.
• 256k x 64 bit external DMA I/O data
memory.
E. Siskind, June 12, 2003
10
SLAC
FECC3 Intelligence
• IBM PPC405 250 MHz 32-bit PowerPC
embedded in Xilinx FPGA on 3rd
generation FECC.
• 16k x 64 bit on-chip program memory; 16k
x 32 bit on-chip data memory.
• 2048k x 64 bit external program/data
memory with DMA link/CAMAC
hardware.
E. Siskind, June 12, 2003
11
SLAC
User FECC Programming
• Real-time executive plus MBCD/Bitbus emulation
uses ~15% of on-chip program memory, ~25% of
on-chip data memory, and ~5% of off-chip RAM.
• Remaining minimum of 15+ megabytes of
memory on 250 MHz 32-bit processor exceeds
capacities of existing 386/486 CPU boards.
• Move existing 360 Hz interrupt driven code from
micro to CAMAC interface to minimize CAMAC
access delay.
• C run-time environment with dynamic heap
allocation; full environment and stack preserved
over context switch.
E. Siskind, June 12, 2003
12
SLAC
nd
2
Generation Hardware
• PCIL2 uses 64 bit/66 MHz PCI slot.
• Single specialized fiber optic link for PCIL2
FECC3 communication.
• 125 megabyte/second full duplex link, but
9/19 used for hardware overhead.
• All peripherals FPGA-based DMA except
for Bitbus (programmed I/O).
• Hardware based on one large FPGA per
board, plus SHARC on PCIL2.
E. Siskind, June 12, 2003
13
SLAC
nd
2
Generation Status
• PCIL2 now debugged and tested.
• Two PCIL2 boards built in 2001; three more
in 2002.
• FECC3 design in completed in 2002; FPGA
delivery currently 29 August (5 month
delay).
• Initial build of two FECC3 boards in 2003;
three more after debugging.
E. Siskind, June 12, 2003
14
SLAC
Bitbus Master
• Existing master based on MCS-51 2 MHz 8 bit
microcontroller with SDLC/HDLC serial link.
• 8044BEM is an MCS-51 with mask-programmed
firmware (iDCX-51 RTE plus Bitbus application).
• FECC3 has dual-channel programmed I/O
hardware (one PowerPC FIFO read or write per 32
bits) to send a packet to one slave and receive that
slave’s response.
• Poll list, counters, etc. maintained by new
PowerPC code in FECC3.
• Enhanced real-time support via reserved high
priority buffers and multiple prioritized queues.
E. Siskind, June 12, 2003
15
SLAC
Schedule
• Migration of 360 Hz processing in progress
(TEG).
• All optimizations not necessary at early
stage.
• Trying to deploy MPG plus one ordinary
PC-micro in development system in
January, 2004.
• Hoping to replace MPG plus one ordinary
micro in summer of 2004.
E. Siskind, June 12, 2003
16
SLAC
Simplified Hardware
Configuration
• No PCPC fast feedback (KISnet equivalent)
network traffic.
• Use 2nd generation PCIL-FECC hardware:
– Include IEEE CAMAC and Bitbus;
– Need PC with 64 bit/66 MHz PCI slot.
• MPG  PC (PNET equivalent) pattern broadcast
on dedicated point-to-point Ethernet link .
• PC-based MPG (MP10/MP11) broadcasts pattern
to existing micros via FECC3 BITbus interface
reconfigured as a PNET transmitter; talks to
SP00/SP01 and BIC via PC  VME reflective
memory links.
E. Siskind, June 12, 2003
17
SLAC
Integration Projects
• Build, boot, and debug iRMX-III in off-theshelf personal computers. Provide minimal
FECC debugging support.
• Make real-time micro  micro data
communications streams coexist with
slower back-end  front-end traffic on
switched TCP/IP network (deferred).
• Move 360 Hz interrupt processing from
micro to FECC (MPG + pattern consumer).
E. Siskind, June 12, 2003
18
SLAC
Integration Projects II
• Change control of 360 Hz interrupt processing
from shared memory to network link model.
• Integrate PCIL-FECC CAMAC/Bitbus emulation.
• Add support for real-time CAMAC/Bitbus;
Modify fast feedback actuator to specify real-time
priority for CAMAC/Bitbus operations.
• Modify slower micro jobs to send larger packages
to CAMAC interface to maximize parallelism.
• Modify klystron job to protect PIOP CAMAC
operations from interrupts.
E. Siskind, June 12, 2003
19
SLAC
Integration Projects III
• Support logical to physical micro mapping.
• Support PC access to file server delivering
current versions of PCIL and FECC
executable images appropriate to different
types of micros.
E. Siskind, June 12, 2003
20