* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download OIF CEI-56G Application Note
Survey
Document related concepts
Electrification wikipedia , lookup
Alternating current wikipedia , lookup
Power over Ethernet wikipedia , lookup
Electrical engineering wikipedia , lookup
Mains electricity wikipedia , lookup
Microprocessor wikipedia , lookup
Immunity-aware programming wikipedia , lookup
Electronic engineering wikipedia , lookup
Pulse-width modulation wikipedia , lookup
Power engineering wikipedia , lookup
Thermal copper pillar bump wikipedia , lookup
Telecommunications engineering wikipedia , lookup
Transcript
OIFCEI-56GApplicationNote CommonElectricalInterfaceat56Gb/s Abstract: TheOIFiswellalonginthedevelopmentoftheCEI-56Gsuiteofhighspeedinterconnect ImplementationAgreements(IA)s.TheOIFisdevelopingIAsfornine56Gclauses, spanningfivereachesandthreemodulationtechniques. TheOIFhasalonglegacyofdevelopingIAsforserialelectricalinterfaces,withtheCEI familybeingtheprincipalindustrydefinitionforSerDesforthe6Gb/s,11Gb/sand28 Gb/sgenerations.InterfacesthatleveragetheCEIfamilyofspecificationshavebeenbuilt inmostofthemajorASICflowsandFPGAdevices.CEIhasbeenadoptedoradaptedinto multipleinterfacesbyIEEE802.3,InfiniBandandFibreChannel. AbouttheOIF: TheOIFfacilitatesthedevelopmentanddeploymentofinteroperablenetworkingsolutions andservices.MemberscollaboratetodriveImplementationAgreements(IAs)and interoperabilitydemonstrationstoaccelerateandmaximizemarketadoptionofadvanced internetworkingtechnologies.OIFworkappliestoopticalandelectricalinterconnects, opticalcomponentandnetworkprocessingtechnologies,andtonetworkcontroland operationsincludingsoftwaredefinednetworksandnetworkfunctionvirtualization.The OIFactivelysupportsandextendstheworkofnationalandinternationalstandardsbodies. Launchedin1998,theOIFistheonlyindustrygroupunitingrepresentativesfromacross thespectrumofnetworking,includingmanyoftheworld’sleadingserviceproviders, systemvendors,componentmanufacturers,softwareandtestingvendors.Informationon theOIFcanbefoundathttp://www.oiforum.com. Foradditionalinformationcontact: TheOpticalInternetworkingForum,48377FremontBlvd,Suite117, Fremont,CA94538USA [email protected] OFC2016 OpticalInternetworkingForum 1 TableofContents Abstract:.......................................................................................................................................................................1 AbouttheOIF:.......................................................................................................................................................1 Glossary†.....................................................................................................................................................................4 TheOIFCEILegacy..................................................................................................................................................7 Introduction................................................................................................................................................................8 MotivationforCEI-56G..........................................................................................................................................9 CEI-56GChallenges...............................................................................................................................................12 DieArea.................................................................................................................................................................12 ChipPowerDissipation..................................................................................................................................12 PackagePowerDissipation...........................................................................................................................12 ChipPowerWiring............................................................................................................................................12 SystemPowerDissipation–Mid-planes.................................................................................................12 SystemPowerDissipation–Mid-boardoptics.....................................................................................13 I/ODensitiesonChipsandConnectors...................................................................................................13 ChannelMaterialsandCharacteristics....................................................................................................13 ChannelReach....................................................................................................................................................14 Cables.....................................................................................................................................................................14 Latency...................................................................................................................................................................14 Summaryofchallenges...................................................................................................................................14 ModulationTechniques.......................................................................................................................................16 OFC2016 OpticalInternetworkingForum 2 NRZ..........................................................................................................................................................................16 ENRZ.......................................................................................................................................................................16 PAM-4.....................................................................................................................................................................17 InterconnectReachesandApplicationSpaces..........................................................................................19 USR-DietoDieInterconnectWithinAPackage.................................................................................20 USR-DietoOpticalEnginewithinaPackage.......................................................................................21 XSR-ChiptoNearbyOpticalEngine.........................................................................................................21 XSR-CPUtoCPU...............................................................................................................................................22 XSR–DSPArrays...............................................................................................................................................22 XSR-CPUtoMemoryStack..........................................................................................................................22 VSR-ChiptoModule.......................................................................................................................................22 MR-ChiptoChipwithinaPCBA................................................................................................................23 LR–ChiptoChipacrossaBackplane/Midplane.................................................................................23 LR–ChiptoChipacrossaCable.................................................................................................................24 InteroperabilityPoints........................................................................................................................................24 DieBump...............................................................................................................................................................24 PackageBall.........................................................................................................................................................24 ModuleInterconnect........................................................................................................................................24 Summary....................................................................................................................................................................25 TableofFigures Figure1InterconnectApplicationSpaces....................................................................................................8 Figure2ScalingofGates,Bumps,PinsandI/O(Ref.Xilinx,I/OLineadded)............................10 Figure3InterconnectChallenges..................................................................................................................11 OFC2016 OpticalInternetworkingForum 3 Figure4TypicalNRZEyeDiagramandStatisticalEyePlot...............................................................16 Figure5ENRZTypicalStatisticalEyes........................................................................................................17 Figure6TypicalPAM-4Eyes...........................................................................................................................18 Figure7CEI-56GReaches.................................................................................................................................19 Figure8UseofCEI-56GReaches...................................................................................................................20 Figure9USR-DietoDie....................................................................................................................................20 Figure10USR-DietoOpticalEngine..........................................................................................................21 Figure11XSRChiptoOpticalEngine..........................................................................................................21 Figure13MR-ChiptoChip..............................................................................................................................23 Figure14LRChiptoChipacrossabackplane..........................................................................................23 Figure15PackageBumpInteroperabilityPoints...................................................................................24 Figure16ChiptoModuleInteroperabilityPoint....................................................................................25 Glossary† 2.5D:Referstoatypeofdie-to-dieintegrationviaasiliconinterposerhavingthroughsiliconvias(TSVs)connectingitstopandbottommetallayers 3D:Referstoathree-dimensional(3D)integrateddeviceinwhichtwoormorelayersof activeelectroniccomponents(e.g.,integratedcircuitdies)areintegratedverticallyintoa singlecircuitwherethrough-siliconvias(TSVs)arecommonlyusedfordie-to-die connection. ApplicationSpaces:Portionsofequipmentornetworkarchitecturethatcouldbenefit fromhavingadefinedsetofinterconnectionparameters. ASIC:Anapplication-specificintegratedcircuitisanintegratedcircuit(IC)customizedfora particularuse,ratherthanintendedforgeneral-purposeuse. BCH:Bose,Ray-Chaudhuri,Hocquenghemforwarderrorcorrection(FEC)codesareaclass ofadvancedcyclicerror-correctingcodesthatareconstructedusingfinitefields. OFC2016 OpticalInternetworkingForum 4 BER:BitErrorRatioisthenumberofbiterrorsdividedbythetotalnumberoftransferred bitsduringastudiedtimeinterval. BGA:BallGridArray,apackagetype CDR:Clockanddatarecovery,ablockinareceiverthatrecreatesaclockorclocksina receiverbylookingatthestatisticaledgeinformationinthereceiveddataandthenuses thatclockorclockstosamplethereceiveddata. CEI:CommonElectricalInterface,anOIFImplementationAgreementcontainingclauses definingelectricalinterfacespecifications. ClockForwarding:Aclockcanbeforwardedinparallelwiththedatatoallowa transceivertoavoidusingCDRinordertosavepower.ThisistypicallydoneinUSR applications. EMB:Effectivemodalbandwidth,seeTIA-492AAAD. ENRZ:EnsembleNonReturntoZero,amulti-wirecodeinwhich3bitsaremodulatedwith theHadamardTransformontofourwires. FEC:Forwarderrorcorrectiongivesareceivertheabilitytocorrecterrorswithoutneeding areversechanneltorequestretransmissionofdata. FPGA:FieldProgrammableGateArray–areprogrammablelogicdevicethatoftenincludes SerDes. FR4:Agradedesignationassignedtoglass-reinforcedepoxyprintedcircuitboards(PCB). Gb/s:Gigabitspersecond.Thestatedthroughputordatarateofaportorpieceof equipment.Gb/sis1x109bitspersecond. GBd:Thebaudrateistheactualnumberofelectricaltransitionspersecond,alsocalled symbolrate.OneGigaBaudis1x109symbolspersecond. IA:ImplementationAgreements,whattheOIFnamestheirdefinedinterfacespecifications. IC:IntegratedCircuit I/O:InputOutput,acommonnamefordescribingaportorportsonequipment ISI:Inter-SymbolInterference,theeyeclosurecausedbytheenergyremainingonthe channelfrompreviousunitintervals,whichtypicallyimpactsthehorizontaleyeopening. ISI-Ratio:Theratioofthelargestcodeeyetothesmallestcodeeye,whichisasimple metrictoevaluatetheimpactofISIonhorizontaleyeclosureforagivencode. OFC2016 OpticalInternetworkingForum 5 LR:LongReach–longenoughtoreachachiplocatedacrossabackplane MCM:Multichipmodule,aspecializedelectronicpackagewheremultipleintegrated circuits(ICs),semiconductordiesorotherdiscretecomponentsarepackagedontoa unifyingsubstrate,facilitatingtheiruseasasinglecomponent(asthoughalargerIC). Mid-boardoptics:anopticaltransceiverthatismountedonaPCBAawayfromthePCBA edge,closetoaswitchASICtoreducetheamountofPCBAtracelossbetweenanASICand theopticaltransceiver.Thisisincontrasttothecommonpracticetodayoflocatingoptical transceiversatthePCBAedge. MUX/DEMUX:Multiplex/demultiplex,amultiplexer(ormux)isadevicethatselectsone ofseveralanalogordigitalinputsignalsandforwardstheselectedinputintoasingleline, Conversely,ademultiplexer(ordemux)isadevicetakingasingleinputsignalandselecting oneofmanydata-output-lines,whichisconnectedtothesingleinput. MR:MediumReach–longenoughtoreachachiplocatedacrossaPCBAincludingona daughter-card NRZ:NonReturntoZero,abinarycodeinwhich1sarerepresentedbyonesignificant condition(usuallyapositivevoltage)and0sarerepresentedbysomeothersignificant condition(usuallyanegativevoltage),withnootherneutralorrestcondition. PAM:Pulseamplitudemodulation,aformofsignalmodulationwherethemessage informationisencodedintheamplitudeofaseriesofsignalpulses. PAM-4:Pulseamplitudemodulation-4isatwo-bitmodulationthatwilltaketwobitsata timeandwillmapthesignalamplitudetooneoffourpossiblelevels. PCBA:Printedcircuitboard(PCB)assembly,anassemblyofelectricalcomponentsbuilton arigidglass-reinforcedepoxybasedboard. RS:ReedSolomonFECcoding,thisisatypeofblockcode.Blockcodesworkonfixed-size blocks(packets)ofbitsorsymbolsofpredeterminedsize.Itcandetectandcorrectmultiple randomandbursterrors. SerDes:Serializer/Deserializer Tb/s:Terabitspersecond.Thestatedthroughputordatarateofaportorpieceof equipment.Tb/sis1x1012bitspersecond USR:Ultra-ShortReach–justlongenoughtoreachanotherdiewithinthesamepackage VSR:Very-ShortReach–longenoughtoreachatransceiverinafront-panelopticalmodule OFC2016 OpticalInternetworkingForum 6 XSR:eXtra-ShortReach–longenoughtoreachanearbydeviceormid-boardoptical module †Somedefinitionsincludecontentfromwww.wikipedia.com TheOIFCEILegacy TheOIFhasalonglegacyofdevelopingIAsforhighspeedelectricalinterfaces.TheCEI familyhasbeentheprincipalindustrydefinitionforSerDesforthe6Gb/s,11Gb/sand28 Gb/sgenerations.TheOIFSxIinterfacewasCEI’simmediatepredecessorandservedthe 3Gb/sgeneration.TheOIFSPI/SFI3and4electricalinterfacesservedthe800Mb/sand 1.6Gb/sgenerations.Thislonghistoryof7generationsandsixdoublingsgivestheOIFa uniqueviewpointontheneedsoftheindustryfortodayandtomorrow. InterfacesthatsupporttheCEIfamilyofspecificationshavebeenbuiltinmostofthemajor ASICflowsandFPGAdevices.CEIhasbeenadopted,adapted,orinfluencedthe developmentofmultipleinterfacesincludingIEEE802.3,InfiniBandandFibreChannel, SATA,SAS,RapidIOandHyperTransportinterfacesaswellasnumerousproprietary protocols. Name Rateperpair Year Adopted,AdaptedorInfluenced CEI-56G 56Gb/s 2016 Thefutureisbright CEI-28G 28Gb/s 2011 InfiniBandEDR,32GFC,SATA3.2, 100GBASE-KR4and100GBASECR4,CAUI4,SAS-4 CEI-11G 11 2008 InfiniBandQDR,10GBASE-KR, 10GFC,16GFC,SAS-3,RapidIOv3 CEI-6G 6 2004 4GFC,8GFC,InfiniBandDDR,SATA 3.0,SAS-2,RapidIOv2, HyperTransport3.1 OFC2016 OpticalInternetworkingForum 7 SxI5 3.125 1.6 2SPI4,SFI4 Introduction SPI3,SFI3 0.800 2.1 Purpose 2002-3 Interlaken,FC2G,InfiniBandSDR, XAUI,10GBASE-KX4,10GBASECX4,SATA2.0,SAS-1,RapidIOv1 2001-2 SPI-4.2,HyperTransport1.03 2000 (fromPL3) The OIF Next Generation Interconnect Framework identifies application spaces for next generation systems and identifies areas for future work by the OIF and other standards bodies. The scope of this document explores Next Generation (NG) Interconnects that are limited to data center or intra-office applications which are generally less than 2km from both an electrical and optical perspective. Virtual Platforms in the cloud is an example of just one of the applications that will take advantage of NG Interconnect technology to achieve higher data bandwidths in a smaller footprint with better energy efficiency. Introduction Figure 1 Interconnect Application Spaces Figure1InterconnectApplicationSpaces As shown in Figure 1, interconnection interfaces in a typical system are needed for chip-tochip within a module, chip to chip within a PCBA (printed circuit board assembly), between two PCBAs over a backplane/midplane, or between two chassis’. These interfaces may be unidirectional or bi-directional, optical or electrical, and may support a range of data rates. OFC2016 OpticalInternetworkingForum For each application space, the IAs that follow from this framework should identify requirements to support interoperability across the various application spaces for optical and electrical links. They may include, but not be limited to: 8 AsshowninFigure1,interconnectioninterfacesinatypicalsystemareneededforchip-tochipwithinamodule,chiptochipwithinaPCBA(printedcircuitboardassembly),between twoPCBAsoverabackplane/midplane,orbetweentwochassis’.Theseinterfacesmaybe unidirectionalorbi-directional,opticalorelectrical,andmaysupportarangeofdatarates. Considerationsfortheselinksmayinclude: • • • • • • • • • • • • • • Cost LinkPerformance PowerConsumption Channellossbudgets ChoiceofPCBmaterial Modulationtechniqueandsignallevels Numberoflanes,channelconfigurationandgeneralcharacteristics Referenceclockjitter ForwardedClockorCDR Latency Connectorperformance Reliability Size OperatingTemperature MotivationforCEI-56G Nextgenerationsystemsarebeingdrivenbytheneedtohandletheincreasingvolumeof datatraffic.Atthesametime,thenextgenerationsystemsareconstrainedbylimitson powerconsumption,bylimitsonthesizeofasystem,andbytheneedtoprovideacost effectivesolution. Theseneedsdrivenextgenerationsystemstoeverincreasingcommunicationport densities.Theincreaseddensityleadstosmallersurfaceareasavailabletodissipatethe heatgeneratedandthereforerequiresdecreasedpowerconsumptionforaport. Theindustrycurrentlyhaselectricalinterfacesfor10Gb/s(OIF’sCEI-11G),25Gb/s& 28Gb/s(OIF’sCEI-25/28G)inproduction,andworkiscomingalongon56Gb/s(OIF’sCEI56G).However,channelsproducedwithcopperPCBtracesareseverelybandwidthlimited, andbecauseofthisitisincreasinglydifficulttoachievethesamelinkdistancesusing highersignalingrates. ImprovementsinICintegration,whichhasbeenrelentlesslydrivenbyMoore’sLawover pastdecades,haveenabledhigherdensitiesoflogicgatesatescalatinghigherclockratesto OFC2016 OpticalInternetworkingForum 9 beusedinICdesigns.Thesetrendshaveallowedtheindustrytodeployincreasinglymore complexcommunicationsystemsateachgenerationtomeettheinfrastructureneeds. However,asonedigsdeeper,thefutureappearstobechallenging.Manydifferent technologiesneedtoconvergetoimprovethethroughput.ThesecomplexICshave increasinggatecounts,butthepowerdissipationpergateandI/Ospeedhavenolonger scaledatthesamerateasthegatecount.Inaddition,thenumbersofelectricalconnections (bumpsandpackagepins)arealsonotscalingatthesamerate-leadingtoapower, capacityandportcountgapforthenextgenerationinterconnectinterfaces. Figure2ScalingofGates,Bumps,PinsandI/O(Ref.Xilinx,I/OLineadded) Thepredominantinterconnectchallengestoovercomearepresentedbelowinasolution spacediagram,andarediscussedingreaterdetailinsubsequentsub-sections. OFC2016 OpticalInternetworkingForum 10 Figure3InterconnectChallenges OFC2016 OpticalInternetworkingForum 11 CEI-56GChallenges DieArea SerDesimplementingCEI-56GareexpectedtobelargerthanSerDessupportingCEI-28G interfacesbecauseoftheneedformoreadvancedequalization.Additionallythesizeofthe analogportionoftheinterfacesdonotscaleaswellasthedigitalportions. Thisareasituationhasrenderedsomepreviouschiparchitecturesunuseable.Thishasled toatrendofSerDes“out-boarding”whereanUltra-ShortReach(USR)oreXtraShortReach (XSR)interfaceisputonthemainASICandthelongerreachSerDesareputinthesame packageornearby. ChipPowerDissipation SerDesimplementingCEI-56GareexpectedtobehigherpowerthanSerDessupporting CEI-28Ginterfacesbecauseoftheneedformoreadvancedequalization.Additionallythe powerdissipationoftheanalogportionoftheinterfacesdonotscaleaswellasthedigital portions. ThishasfurtheredthetrendmentionedaboveofSerDes“out-boarding”whereanUltraShortReach(USR)interfaceisputonthemainASICandthelongerreachSerDesareputin thesamepackage. PackagePowerDissipation Whenpackagepoweristhekeyconstraint,theuseofanXSRinterfacecanhelpmovesome ofthepowerdissipationoutsideofthemainASICandontootherchipsnearbyonthePCBA. ChipPowerWiring Lowervoltagepowersourcesareusedwithsmallertechnologynodes.Theneteffectof higherintegrationoflowvoltagesemiconductorsisasignificantincreaseintotaldevice current,whichrequireshighcurrentsourcepowersupplieswhichmustbecontrolled withinseveralmVtolerances,whichinturnrequiresadditionalpowerpinsperdeviceto accommodatethehighelectricalcurrents. SystemPowerDissipation–Mid-planes Becausesystemcoolingcanescalatebeyondphysicallimits,multiplepowersaving strategiesmaybeneeded. OFC2016 OpticalInternetworkingForum 12 SincetheI/Opowerisrelatedtothedistancethattheelectricalsignalstravelforagiven channel’sproperties,reducingthedistancethattheelectricalsignalmustbedrivencan reducepowerdissipation.Onetechniquethathasbeenusedistousemid-planes. Withthisphysicalarchitecture,aswitchboardismountedhorizontallybehindthe verticallymountedfrontboards.Withthisarchitecture,theelectricallinksareshorterand onlyhavetotraverseoneconnector. SystemPowerDissipation–Mid-boardoptics Insomecases,itmaymakesensetointegratetheopticsclosetotheASICpackage,thereby minimizingtheneedtodriveelectricalsignalsfar.Thesearesometimescalledoptical enginesandthetechniqueisreferredtoasusingmid-boardoptics. I/ODensitiesonChipsandConnectors ThemaximumnumberofusefulI/Osforhighspeedseriallinksperdeviceisnotonly limitedbytheavailablepackagetechnologyitself,butalsobytheabilitytoroutethedevice onthePCBA. Inordertomaintainsignalintegrityforahighspeedseriallinkdesign,itisrequiredtobe abletorouteadifferentialpairbetweentwopackageballswhenescapingfromtheinner ballrowsofaballgridarray(BGA),anditthereforemaybecomemorecostlytouse packageswithaballpitchbelow1.0mm. Inaddition,foreveryballrowfromtheedgeofthepackageonwhichdifferentialpairshave beenplaced,aseparatecircuitpackagelayerhastobeused.Differentialpairsintheouter4 rowsoftheBGArequires4signallayersonthePCBA,whilethe6outerrowswouldrequire 6layers,andthusthePCBlayerstackgrowswitheveryinnerBGArowtoberouted. ChannelMaterialsandCharacteristics Linkapplicationsarecharacterizedbythesupportedlossbudgetandsignalimpairments. Lossisdeterminedbylinklength,boardmaterials,back-drilling,viatypeusage,the numberofconnectors,connectortypes,andpackagematerials.Increasesinsignalingrate causemorefrequency-dependentlossandthusalowerSignaltoNoiseRatio(SNR). Thedeviationofthelossfromasmoothcurveisalsoimportantastheripplesinaloss curvearenoteasilyequalizable.ThisiscalledInsertionLossDeviation(ILD). Ingreenfieldapplications,advancedmaterialsforthechannel(PCBandconnector)can resultinandecreasedlossandcanthussupporteitheranincreasedelectricaldatarateora decreasedamountofrequiredequalization. OFC2016 OpticalInternetworkingForum 13 ChannelReach Incommunicationsystemsusuallythefrontplateareaisoccupiedbypluggablemodules forinter-systemcommunications,whiletheintra-systemtrafficbetweenthePCBAsis connectedoverthebackplane/midplane.Tosupportreasonablesystemdimensions, systembackplane/midplaneconnectionstypicallyneedtobridgedistancesofupto70–100 cm.Insomemorerecentapplications,alternativechannelarchitecturesreplacethe backplanewithdirectplugorthogonal(DPO)structureswhereI/Olinecardsonthefront sideoftheequipmentdirectlyplugintoswitchfabriclinecardsontherearsideofthe equipment,thusdirectlyconnectingthelinecardswithoutthelossofthebackplane betweenthem Theintroductionofrepeaterdevicesintothedatapathcanincreasetheeffectivereachat theexpenseofpowerandboardarea. Cables Twinax,micro-coaxand/orflexcircuitsarealsoanoptiontoreducelossandextendreach forthehighspeedlinks,particularlyinsystemswithfewerhighspeedlinks.Insome circumstances,thiscanallowthemainPCBAtobebuiltfromlowcostmaterial.Inanother casecableassembliesarenowbeingintegratedontobackplaneconnectorandusedinlieu ofPCBbackplanesduetotheirabilitytoreducelossandextendreach.Theuseofcables bringstheirownshareofchallenges. Latency CPU-to-CPUandCPU-to-MemoryapplicationsofCEIareoftenintolerantoflatency.This oftenprecludestheuseofheavyweightFECintheseapplications.Thecharacteristic methodthatisintolerantoflatencyistheuseofcredit-basedflowcontrolwhereareceiver grantsthetransmittertherighttosendacertainamountofdata.Onlyamoderateamount ofbufferingistypicallyprovidedinthesesystems,mostofteninmorecostlySRAM.Mostof themajorCPUtoCPUprotocolsusecredit-basedflowcontrol. Summaryofchallenges Astimeproceeds,ICswillbecomefasteranddenser.Tocopewiththeissueofinterconnect capacityanddensityoffuturesystems,photonicinterconnectswillbecomeanevenmore importantconnectiontechnology. Theimplementationof56Gb/sInterconnectstechnologyposesseveralchallenges especiallyinrelationto:diearea,chippowerdissipation,packagepowerdissipation, systempowerdissipation,limitedI/Odensity,channelloss,channelreach,andlatency. Highlightedweretheside-effectsofsomesolutions,whicharearesultofthecomplexinterOFC2016 OpticalInternetworkingForum 14 dependenciesof:higherintegration,complexmodulationschemes,chipbreak-outand routing,signalconditioning,thermal&powerissues,andpackagefootprint. OFC2016 OpticalInternetworkingForum 15 ModulationTechniques AdvancedmodulationtechniquescanbeusedtolowertheNyquistfrequencyrequirementfora linkusingabandwidthlimitedelectricalchannel.ThesetechniquesrequireahigherSNRfor equivalentBER.Theprovisionofchannelswithmorecorrelationstomeasureagainstcanalsohelp. Forwarderrorcorrection(FEC)canincreasetheeffectiveSNRandthereforethesupportedloss budgetattheexpenseofhigherpowerdissipation,complexity,andlatency.Bandwidthlimitations andincreasesinsignalingrateresultinimpairmentsthatmaybecompensatedbyusing equalizationtechniques,whichthemselvesalsoimpactpowerconsumptionandcomplexity. NRZ NRZ(NonReturntoZero)isthemodulationtechniqueusedbymostcurrentspeedlinks.It requiresachannelwithtwocorrelatedconductors.Itusestwosignallevelsandconveysonebit perbaud.Tosupport56Gb/sperpair,NRZrunsatasymbolrateof56GBaudandhasaNyquist frequencyof28GHz.IthasanISI-Ratioofone,meaningtheratioofthelargesteyetothesmallest eyeisalwaysthesame. At56Gb/s,NRZisusefulinapplicationswherethechannel’sfrequencydependentlossisnot excessive.Thatlosshasbeenfoundtobeexcessiveinmanylongreachapplications. in a there ulation aches. hnical Figure4TypicalNRZEyeDiagramandStatisticalEyePlot The use of two level signaling (0 and 1) provides a 9dB SNR advantage over PAM4 (4 levels). This provides a ENRZ general rule of thumb that the insertion loss of a given channel measured at the PAM4 nyquist frequency should be EnsembleNRZ(ENRZ)requiresachannelwithfourcorrelatedconductors.Usefulchannelscanbe 9dB less than the insertion loss at the NRZ Nyquist constructedwithtwocorrelatedlooselycoupledpairs.Itusesfoursignallevelsandconveysthree frequency (2x the PAM4 Nyquist frequency) before PAM4 bitsperbaudoverthefourconductorsbymodulatingthethreesub-channelscollectivelyusingthe should be considered. Using this rule of thumb on the 28G Hadamardmatrix.ItTosupporta56Gb/sperpaireffectiverate,ENRZrunsatasymbolrateof VSR insertion loss curves extended to 56G (See Figure X) shows NRZ to be the best choice. OFC2016 Why PAM4? OpticalInternetworkingForum As Common Electrical Interfaces have increased in data rates system vendors have always requested support to the same printed circuit board trace distances. This has 16 Comparison Channel for opening analysis Eye comparison Diagrams ach! ning vs. modulation technique 37.3GBaudandhasaNyquistfrequencyof18.6GHz.IthasanISI-Ratioofone,meaningtheratioof thelargesteyetothesmallesteyeisalwaysthesame.ENRZcanoftenbeusedwithoutForward ErrorCorrection,evenonsomelongreachchannels. Ata56Gb/seffectiverate,ENRZisusefulinlongerreachapplications,particularlywhentheuseof FECisnotdesired. wires for each throughput per-wire st similar, ENRZ(purple) @ 18.66 GHz ENRZ is Nyquist al ISI-ratio = 1! Figure5ENRZTypicalStatisticalEyes reasonable frequency! PAM-4balanced eye shape PAM-4requiresachannelwithtwocorrelatedconductors.Itusesfoursignallevelsandconveys twobitsperbaud.Tosupport56Gb/sperpair,PAM-4runsatasymbolrateof28GBaudandhasa Nyquistfrequencyof14GHz.IthasanIntersymbolInterferenceRatio(ISI-Ratio)ofthree,meaning thelargestcodeeyeisthreetimesaslargeasthesmallestcodeeye.Ithasaverticalimpairmentof Source: TE Connectivity 9dBascomparedtoNRZandhasahorizontalimpairmentduetoitshighISI-Ratio.PAM-4is usuallypairedwithequalizationandForwardErrorCorrection. 25 At56Gb/sPAM-4isusefulinapplicationswherethechannel’sfrequencydependentlosswould otherwisebeexcessive. OFC2016 OpticalInternetworkingForum 17 Demo 1: CEI-56G VSR NRZ QSFP channel demo This demonstration shows a working VSR channel using Test equipment from Tektronix to generate a 50Gbps NRZ signal which is sent thru commercially availableQSFP connectors on test boards from Multilane and Yamaichi. (See Figure ?)The receiver is a commercially available NRZ receiver from Credo. The BER of the system is shown on a PC using the BER tester in the Credo receiver. Figure6TypicalPAM-4Eyes PAM modulation has been used for many years as the solution of higher data rates over Category 3 and 5 twisted pair wiring for Ethernet. Figure Y shows an example of a 28G VSR compliant channel that would benefit from the use of PAM4 modulation to double the data rate with no changes in PCB materials. Note that the loss difference at the NRZ Nyquist frequency vs the PAM4 Nyquist is well beyond 9dB. frequency Demo 2: CEI-56G-VSR PAM4 QSFP Compliance board The ability to transmit and receive PAM4 signals across a commercially available connectors is shown in Figure 14 using test equipment from Tektronix, Multilane and Anritsu. One demonstration uses a Tektronix PAM4 signal generator to transmit PAM4 formatted data over a QSFP connector using test boards from Molex and TE Connectivity. The PAM4 receiver is provided by Multilane. An additional demonstration uses an Anritsu PAM4 data generator sending signals through a CFP connector using test boards from Yamaichi and Multilane. The PAM4 receiver is provided by Tektronix. The CEI 56G Technology Showcase In an effort to highlight the progress of the OIF in the CEI 56G projects a technology showcase was developed with the intent of demonstrating technical feasibility of the different reaches. There are 6 demonstrations from various members of the OIF. OFC2016 OpticalInternetworkingForum 18 InterconnectReachesandApplicationSpaces ThefiveCEI-56Greachescanbesummarizedinthefollowingtable: Figure7CEI-56GReaches Theuseofthesereachesforinterconnectapplicationspacescanbebrokendownas follows. OFC2016 OpticalInternetworkingForum 19 Figure8UseofCEI-56GReaches USR-DietoDieInterconnectWithinAPackage Figure9USR-DietoDie Itmaybenecessarytousemultipledieswithinamulti-chipmodule(MCM)toachievethe powerobjectives.Theseco-packagedsolutionscancommunicatewithlesspowersincethe substrateprovidesahighqualitycommunicationchannel. Thecommunicationchannelislessthan10mm.Thisshortelectricallinkmayallowfora muchsimplerinterfaceandrequirelesspowerthananexistingstandardelectrical interface.Forexample,equalizationisunlikelytobeneededanditmaybepossibleto assumesuchshortlinksaresynchronous(singlereferenceclockgoingtoallchips), removingtheneedforafrequencytrackingCDR. OFC2016 OpticalInternetworkingForum 20 Atypicaluseforthisistooff-boardtheSerDesfromaswitchASIC. ClockforwardingcanbeusedintheplaceofCDRforUSRlinks. USR-DietoOpticalEnginewithinaPackage Figure10USR-DietoOpticalEngine Itmaybenecessarytouseadieandanopticalenginewithinamulti-chipmodule(MCM)to achievetheindustry’sobjectives.Theseco-packagedsolutionscancommunicatewithlow powersincethesubstrateprovidesahighqualitycommunicationchannel. Thecommunicationchannelwouldtypicallybelessthan10mm.Thisshortelectricallink mayallowforamuchsimplerinterfaceandrequirelesspowerthananexistingstandard electricalinterface. XSR-ChiptoNearbyOpticalEngine Figure11XSRChiptoOpticalEngine Itmaybeusefultoplaceanopticalinterfaceveryclosetothehostchip.Someoptical devicescannotsitwithinahostMCMduetoheatrestrictionsoftheopticalcomponents.In thiscase,ashortelectricallinkoflessthan50mmisanticipated.Theshortreachofthis channelallowspowertobesaved.Theseopticalmodulesareoftencalledmid-board optics. OFC2016 OpticalInternetworkingForum 21 XSR-CPUtoCPU CPUscanbeconnectedviaashortconnectionoverhighrateSerDes.SomeCPUsusea coherencyprotocoltomakethedifferentprocessorsappeartobeinthesamecoherency domain. Itisgenerallynotpossibletouseheavy-weightForwardErrorCorrectioninCPU applicationsduetothelatencyrequirementsoftheseinterfaces,whichtypicallyusecreditbasedflowcontrol. XSR–DSPArrays DSPsaresometimesconnectedinarraystoprocesshighrateinformationsuchasfroma RADARoraLIDAR.Anautonomousvehicleisanexampleofsuchanapplication. Sometimestheinterfacesareunidirectionalandothertimes,bidirectional.Bothare typicallylatencysensitivesimilartoCPUsaseventheunidirectionalapplicationsoften havecomplex,timesensitivecontrolflows. XSR-CPUtoMemoryStack CPUscanbeconnectedviaashortconnectiontoamemorystack.Aclassicwayto accomplishthisistohaveadevicemadewithalogicprocessthatformsthebaselayer devicefortheactuallymemorydies. Itisnotpossibletouseheavy-weightForwardErrorCorrectioninthisapplicationdueto thelatencyrequirementsofmemoryinterfaces.CPUthreadsorstatemachinesareoften waitingfortheresultsofthememoryaccess.Memorycacheshelp,butdonoteliminatethe significantperformancehitsthatCPUsseewhenlatencyisadded. VSR-ChiptoModule Itiscommoninmoderncommunicationsystemstosupportpluggablemodulesatthefront faceplateoftheequipment.Thisfacilitateslowcostinitialdeploymentoftheequipmentif someportsareleftunpopulated.Apayasyougopolicyisthenuseduntiltheentire faceplateispopulatedwithpluggablemodules.Theelectricallinkusedtoconnectthese pluggablemodulescanextendtobeyond30cm.Athigherdataratesthischallengesthe abilityofthehostchiptodrivetheselongtracelengthswithinthepowerconstraintsof largeswitchchips.Placingretimingdevicesinsidethepluggablemoduleprovidessupport forlongerhosttracesbuttheinclusionofcomplexequalizationfeaturescanoverburden thelimitedpowerbudgetsofthepluggablemodule.Advancedmodulationformats(suchas PAMorDMTschemes),ForwardErrorCorrection(FEC)andequalizationfeaturesareall possiblesolutionsforthechiptomoduleinterconnect. OFC2016 OpticalInternetworkingForum 22 Onetechniquethathasbeenusedistoengineertheelectricallinkstohavealowernative BERthanthatoftheopticallink.Inthisway,asingleFECcanbeusedtoprotectaCEI-56G linkateachendandanopticallinkinthemiddle. MR-ChiptoChipwithinaPCBA Figure12MR-ChiptoChip AninterconnectioninterfacemaybeneededbetweentwochipsonthesamePCBAorona daughtercardorshortermid-plane.Bydefinition,thisinterfaceisrelativelyshortranging upto50cm.Thisinterfacecouldincludeasingleconnector. LR–ChiptoChipacrossaBackplane/Midplane Figure13LRChiptoChipacrossabackplane Thisinterfacecommunicatesbetweentwocardsacrossabackplane/midplanewithina chassisandislessthan1mwithupto2connectors. FECmaybearequirementtomeettheBER–howeverthechoiceoftheFECmustbe consideredcarefullytoaddressbothlatencyandpowerconcerns.PossibleFEC implementationsincludeRSorBCH. OFC2016 OpticalInternetworkingForum 23 LR–ChiptoChipacrossaCable WhilenotanexplicitgoaloftheCEI-56Gproject,cabledversionsofCEI-28Ghavebeen usedtosupportbothbackplaneandchassistochassisinterconnects. InteroperabilityPoints CEI-56Gdefinesthreedifferentinteroperabilitypointsforthevariousreaches.Theseare thepointsatwhichcompliancethetheOIFIAscanbedeterminedbymeasuringthe propertiesofanimplementationandcomparingthoseresultstotheparametersintheIA. DieBump CEI-56G-USRdefinesinteroperabilityatthediebumpofaflip-chipsemiconductordevice PackageBall CEI-56G-XSR,CEI-56G-MRandCEI-56G-LRalldefinesinteroperabilityatthepackageball. Figure14PackageBumpInteroperabilityPoints OIFchosepackageballsastheinteroperabilitypointsforthesethreespecificationsbecause theentirechannelistypicallytheresponsibilityofthesamecompany.XSRdoesnothavea connectorandthereforenootherinteroperabilitypointisavailable.ForMRandLR,which dohaveconnectors,thesystemdesignerstillownstheentirechannelandthereforeitdoes notmakesensetoconstrainhowthesystemdesignerallocatesthelinkbudget. ModuleInterconnect CEI-56G-VSRdefinesinteroperabilityatthemoduleinterconnect. OFC2016 OpticalInternetworkingForum 24 Figure15ChiptoModuleInteroperabilityPoint Hostcomplianceboards(HCB)andmodulecomplianceboards(MCB)aredefinedto measurecompliance. Summary TheCEI-56Gfamilyofclausesdefinesasetofinterfacesthatcanbeusedtosolvemosthigh speedinterconnectneeds.Fiveseparatereachesandthreeseparatemodulation techniquescansupporttheneedsofmostapplications. OFC2016 OpticalInternetworkingForum 25