Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Dynamically Provisioned Networks as a Substrate for Science David Foster CERN Objectives • To explain in a high-level way why dynamic circuits are needed to serve demanding scientific users. • To put the activities into an overall context of global research networking and show the future directions. David Foster, CERN Environment • Science projects are global enterprises o Megascience: LHC, ITER, LoFAR, JIVE, SKA ……. o ESFRI projects funded under FP7: • http://ec.europa.eu/research/infrastructures/index_en.cfm?pg=ri_pro jects_fp7 o Increasing coordination on an increasingly global scale. • Computing for science is increasingly distributed o Clouds and Grids • People are mobile but lifestyle choice is important o Work anywhere, anytime. o “Think Globally but act Locally” • Open access to information is empowering o “Bring Science to the Scientists” - Bring the best minds to the problem. David Foster, CERN Situation • As perceived by the user community, networks are a victim of their own success o Expected to be transparent. “Networking is not a problem” o Assumed to be infinite and free (or nearly so). • Data volumes are increasing o LHC creates 6-7 PB raw data per year , all 4 experiments together. o CERN generates in total 15 PB of data per year, all 4 experiments together. o The raw data, 6-7PB/year, is distributed and there are more than 120PB/year of data products that are created and stored world-wide. o Other science project collaborations will generate equivalent or more data. David Foster, CERN Characterization of the User Marketplace David Foster, CERN Cees de Laat http://ext.delaat.net/talks/cdl-2005-02-13.pdf Issues With Demanding Users • There are more and more of them. • The swamping of IP infrastructures with traffic from “well connected sites” o Occurs when the capability of a site are approaching that of the routed IP network. o Looks like a “denial of service” to the other users. • Solution 1: Build a bigger routed IP network. o A big investment to solve a problem for relatively few users. o All domains in any end-end path must do the same. o Only temporary, new users will come with bigger requirements. • Solution 2: Give the sites “what they need when they need it”. o May be considered as “Just in time provisioning” o Has led to the circuit approach. David Foster, CERN How Circuits are Used • For efficiently using resources: o Long term or static circuits if the number of sites is small (~10) o Dynamic circuit provisioning for a community that is manageable and has continual needs (~100 sites) o Dynamic circuit provisioning for a large community that has occasional needs (periodic data transfer) • For traffic management: o Separates flows from the general IP infrastructure. David Foster, CERN LHC: Case Study • LHC tier-1 sites are connected at 10G in a semi-mesh. o LHCOPN • Tier-2 sites (originally) needed 1Gbps to realistically be “part of” the grid community. o A dedicated circuit to India enabled and effectively empowered the TIFR. • Now, tier-2 sites are increasingly connected at 10G to be able to dynamically access all data products wherever they are (remember the 120PB/year?) o But connected to where? The association between Tier-1 and Tier-2 has “disappeared”. o All sites must be able to access all sites, so IP is the best fit! • Probably so at 1Gbps/site, but not at 10Gbps/site. • A new approach was needed o LHCONE: http://lhcone.net David Foster, CERN What is LHCONE? • A sociology o Has helped to raise awareness of end site needs. o T2’s in Europe are requesting increasing capacity from the NREN’s • A process o Transatlantic bandwidth review of all R&E circuits o Discussions on really how to make a cross domain network. o Process is perhaps more important than outcome in delivering collaboration and focus • as long as the outcome works! • An architecture o Use of open exchanges to empower networks and users o Use of software for managing network capacity through dynamic provisioning. • OpenFlow is the flavour of the month • A model o The internet-2 OS3E is an open exchange architecture to support all sciences. Experience with LHCONE will be important. David Foster, CERN Open Exchanges • • A growing consensus on the way forward o o I2 members meeting discussion with Bill St Arnaud. A paper in preparation on a “definition” o They promote customised bilateral relationships by the exchange point owner not interfering • “lightweight” rules for connecting, link policies controlled by the link owners. They have no specific commercial allegiance • So-called “carrier-neutral” Users of the exchange point (can be NREN’s or end users) like them because they remain in control and directly manage the relationships. • no third-party involvement They provide the possibility to create diverse solutions by working with different partners. • risk management They do not impose technical decisions, so everyone can go at their own speed. • Avoids “lowest common denominator” solutions They allow for organic growth and new entrants are welcome both as exchange operators and connected parties • Avoids single point of failure in the system as a whole Optical exchanges permit provisioning of circuits of different transport protocols to exchange traffic. • http://www.broadnets.org/2004/workshop-papers/Gridnets/DijkstraF.pdf They allow for activities to follow means and ambition. • They can be “pay as you go” and not “subscription based” Why is there so much interest? o o o o o o o David Foster, CERN To Be Resolved • Costs incurred to connect to an open exchange. o o Depends on the exchange operator and the “last mile” provider. Will be born by the end-site connecting. o o Currently born by the exchange owners, but is this scalable? Might be also the responsibility of the science community but they are classically not structured to fund central network resources. • Needs to be more awareness that networks are not free inside the science communities. • Costs incurred to interconnect open exchanges • Management of multi-domain circuit based infrastructures o o o o Is a hot topic and has been for some time. • Buzzword heaven: Oscars, ION, Dragon, DRAC, Federica, OpenFlow, DICE, NSI … Many software solutions are used and under development. • But some are moving much faster than others. We need some consolidation. Some collaboration activities underway to develop domain interworking. • But we need a more open and inclusive process. Operations and management processes are still to be agreed. • No real process addressing this at present. David Foster, CERN Summary • Science needs are increasing and diversifying rapidly o Big growth in large international projects in all areas needing global high bandwidth connectivity. o The pressure is to always seek Open, Neutral and Diverse solutions. • The best service at the best price. • Circuit based approaches are inevitable o They address the needs of the network providers to serve the high-end users with resource efficiency and manageability. o The downward pressure on bandwidth costs from commercial operators make them increasingly cost effective. • Open Exchanges are inevitable o Because of the compelling combination of sociological, business and technical rationale. o We are not a strict hierarchy of users, nren’s, operators at world or european level and perhaps becoming less so as time goes on. David Foster, CERN