Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
ITS BUSINESS CASE High Speed Network Links for CSIRO Observatories Last Updated: 29-Jun-17 Version: 0.5 Role Representative Business Owner Ray Norris ATNF Marsfield Ron Beresford Chris Phillips Neil Killeen Vince McIntyre Tasso Tzioumis ATNF Parkes John Reynolds ATNF Narrabri Bob Sault CTIP Marsfield Shaun Amy Michael Homsey ICT Centre Marsfield Terry Percival IT Services Ron Baxter John Morrissey Rob Thomsett AARNet Keith Burston NSW RNO Alan Cowie ITS BUSINESS CASE Network Links to Observatories Change Record Date Author Version Change Reference 25-Mar-04 Ron Baxter 0.1 No previous document. 31-Mar-04 Ron Baxter 0.2 Some risks and costs added, distributed for Planning Meeting on April 2 23-Apr-04 Ron Baxter 0.3 Incorporating outcomes from Planning Meeting 28-May-04 John Morrissey 0.4 Revised sections 6.1 and 6.4 Chris Phillips 0.5 ATNF revisions Tasso Tzioumis Table of Contents Change Record ..................................................................................................................... 2 Table of Contents .................................................................................................................. 2 1. Executive Summary ..................................................................................................... 3 2. Project Introduction ..................................................................................................... 3 2.1 Background ........................................................................................................................... 3 2.2 Overview ............................................................................................................................... 4 2.3 3. Project Success ...................................................................................................................... 4 Project Scope and Objectives ........................................................................................ 5 3.1 Key Objectives & Outcomes ..................................................................................................... 5 3.2 Out of Scope .......................................................................................................................... 6 4. Constraints ................................................................................................................. 6 5. Related Projects .......................................................................................................... 6 6. Key Activities and Deliverables ...................................................................................... 7 6.1 “Lighting up” the Regional Network ........................................................................................... 7 6.2 Fibre Links to CSIRO Sites ....................................................................................................... 8 6.3 Production Network – Connect Site LANs ................................................................................... 9 6.4 Research Network – for high speed data transfer ...................................................................... 10 6.5 Key Milestones ..................................................................................................................... 11 7. Risks ........................................................................................................................ 11 7.1 Project Risks ........................................................................................................................ 11 7.2 Business Risks ..................................................................................................................... 12 8. Benefits/Value Analysis .............................................................................................. 12 8.1 Position CSIRO as a World Leader in VLBI ................................................................................ 12 8.2 Developing a Data Grid ......................................................................................................... 13 8.3 Intensify the Pulsar Timing Partnership ................................................................................... 14 8.4 Efficient Service Delivery ....................................................................................................... 14 Last updated: 29-Jun-17 v0.5 2 of 23 ITS BUSINESS CASE 8.5 8.6 9. Network Links to Observatories Research Network: Enhance CSIRO’s Reputation ...................................................................... 15 Returns and Benefits ............................................................................................................. 15 Cost Analysis............................................................................................................. 15 9.1 10. Costs- Future Value .................................................................... Error! Bookmark not defined. Staff Impact/Change Management ............................................................................... 16 Attachment A: Related Project: Link Cotton CRC to ATNF Narrabri ............................................ 17 Attachment B –Notes on Reduced Costs and Benefits of Network Links to Observatories .............. 18 Attachment C - Quantifying the gains from increased VLBI data rates & sensitivity...................... 21 1. Executive Summary This project will build on the Australian government’s investment in the Australian Research and Education Network (AREN) to provide fibre network connections to the three CSIRO observatories and to the Tidbinbilla Tracking Station. It is motivated by CSIRO’s goals to enhance our global reputation and to intensify partnerships. A new dedicated research network will connect the observatories to selected partner sites, and will use SX Transport to connect internationally. The benefits will include an increased return on investment from CSIRO’s observatories, an early demonstration of the benefits of grid computing, and opportunities to lift CSIRO’s profile both with government and internationally. For CSIRO’s production network, the local networks of the observatories will connect to the CSIRO Wide Area Network at gigabit speeds. The benefits will include a bigger improvement from One-IT Service delivery, better data integrity through automated off-site backups and better collaboration facilities that will be used more widely. The capital cost is about $2.5 million, and the running costs will be about $500K annually. This annual cost is similar to the cost of leasing 2 megabit links commercially, but will deliver more than 1,000 times the bandwidth. 2. Project Introduction 2.1 Background In the CSIRO Strategic Plan for 2003-2007, the second goal is Delivering world-class science – enhancing our global reputation for science excellence. An objective under this goal is 2.4 – Help Australia play a leadership role in major international science facilities such as the SKA. The Square Kilometre Array (SKA) will be an array of radio telescope antennas that will be at least 100 times more sensitive than current arrays, and may be located in Australia. This year CSIRO has an opportunity to demonstrate this leadership by building on the infrastructure of the Australian Research and Education Network (AREN) which is being funded by the Australian Government. This will enable high- Last updated: 29-Jun-17 v0.5 3 of 23 ITS BUSINESS CASE Network Links to Observatories speed interconnection of the three observatories of ATNF and the Tidbinbilla Tracking Station, and will enhance their capabilities. The third goal of the CSIRO Strategic Plan is Partnering for Community Impact with objectives 3.1 Focus and intensify collaboration with universities, CRCs and other agencies 3.4 Partner with other agencies to advance Australia’s global development contributions This project will facilitate ATNF partnerships with Swinburne University, University of Tasmania and Australian National University for the projects discussed here; and will assist the other ATNF partners in Australia and internationally. Currently, Wide Area Network links to the ATNF telescopes are provided by leased phone lines with speeds as shown in the table. Telescope Location Nearby town Leased Line Speed Connects to ATNF Parkes Observatory Parkes 512K Radiophysics Laboratory, Marsfield ATNF Mopra Observatory, (near Coonabarabran) Coonabarabran 128K ATNF Paul Wild Observatory, Narrabri ATNF Paul Wild Observatory, Narrabri Narrabri 512K Radiophysics Laboratory. Marsfield Tidbinbilla Deep Space Tracking Station Canberra 128K ITS Canberra Because these links are slow, two of these sites must operate email, file servers and backup facilities locally. 2.2 Overview As part of AREN, AARNet is implementing the third generation for the national backbone (AARNet3) as described in these briefing slides. AARNet3 will use the NextGen network which was acquired by Leighton Contractors in December 2003. The AARNet3 NextGen fibre will be used as the backbone to interconnect the observatories. Scientific drivers for high speed network connections to the telescopes include Very Long Baseline Interferometry (VLBI), the Virtual Observatory, and Pulsar Timing. 2.3 Project Success Successful completion of this Project will be measured as follows: Success Measure Last updated: 29-Jun-17 Relativity v0.5 [Key Reports] 4 of 23 ITS BUSINESS CASE 1 Network Links to Observatories 2 3 4 5 6 7 Satisfied Client Group/s 8 9 10 X Meet all the project's objectives/requirements X Meet the agreed budget X Deliver the key product/service on time X Add value to the organisation X Meet quality requirements X Sense of professional satisfaction for the Team X 3. Project Scope and Objectives 3.1 Key Objectives & Outcomes 3.1.1 An Advanced Regional Network for AARNet The objective is to collaborate with AARNet on the implementation of a regional network design that can deliver strategic requirements for research and education in regional locations. In this collaboration, CSIRO will assist with RFT, review network architecture, provide environment for equipment on CSIRO sites, and install tail-end equipment at sites. The outcomes will be 1. an impetus to research and education in regional Australia through advanced networking 2. a new competitive force in the regional network market. 3.1.2 A Research Network for High Speed Transfer of Observed Data The objective is to design and implement a separate network with dedicated circuits interconnecting the three CSIRO observatories, Tidbinbilla tracking Station, and the Southern Cross Trans-Pacific Testbed (SX Transport). The outcome will be a network that can provide dedicated high-speed data paths for VLBI within 3 years (at least 10 Gb, possibly 40 Gb) that will stimulate scientific advances in projects such as eVLBI. 3.1.3 Production Network – Gigabit Connections to Regional Sites The objective is to provide gigabit connections from the local networks at Parkes, Mopra and the Narrabri Compact Array to the AARNet shared regional network backbone and thus to the CSIRO WAN, and to also provide dial-up ISDN links as a backup. The outcomes will be transfer of observed data in “near real time”, and consolidated IT management for these sites under the One-IT Service Delivery model. Last updated: 29-Jun-17 v0.5 5 of 23 ITS BUSINESS CASE 3.2 Network Links to Observatories Out of Scope Objectives out of scope for this Project include: Related Project Objectives (Is Not) Stakeholder/RP Connection to Mt Pleasant (near Hobart) telescope for eVLBI Future project with ATNF as stakeholder Connection to Ceduna telescope for eVLBI Future project with ATNF as stakeholder Research network connection to Swinburne Univ for eVLBI and Pulsar Timing Steven Tingay, Swinburne Uni Grangenet Demonstrate high-speed networks as an enabling technology for educational and social projects in regional Australia (Dubbo) Terry Percival, CeNTIE Network connections for Siding Spring and Mt Stromlo telescopes ANU Implement off-site backups over the network Data Storage Management Project Consolidate Exchange mail servers ITS Operations Service Delivery Efficiencies from connection of Cotton CRC to ATNF Narrabri ITS Networks project, ITS Operations 4. Constraints Publicity in 2004 – sufficient progress to enable significant announcements by Aug/Sep 2004 is highly desirable. Significant VLBI achievements by 2006 will assist CSIRO’s case for SKA. 5. Related Projects Other projects that this Project is critically dependent upon are: Project Relationship/Dependencies Services Involved One-IT Service Delivery Improved service delivery over gigabit connections is a key benefit Enterprise service desk, desktop support, server management Data Storage Management Project Provision of off-site backups over the network is a key benefit Enterprise Data Storage & Management Last updated: 29-Jun-17 v0.5 6 of 23 ITS BUSINESS CASE ATNF eVLBI Network Links to Observatories Real-time correlation of highspeed data streams is a key objective Development of high-speed correlator 6. Key Activities and Deliverables The east coast section of the NextGen network is a self-healing loop, and the western side of the loop passes close to the ATNF observatories. Currently the NextGen network has one pair of fibres in operation, and AARNet will “light up” a second pair to provide these regional links. A third pair could be used later for regional network links other than research and education. Figure 1 - The NextGen Network 6.1 “Lighting up” the Regional Network At present the NextGen regional loop consists of “dark” fibre and as such requires the installation of equipment every 80km to activate or “light” the network. It is hoped that the base costs of activating the network will be provided by a request for additional AREN funding. It will cost between 8 and 10 million dollars to light the base network. If this funding is not provided the member institutions will be required to jointly fund the lighting of the network. This would double the costs of the whole project and is therefore a major project risk. This is an AARNet activity and CSIRO will assist with planning, design and implementation. The second stage in activating the network requires the installation of Wave Division Multiplexing equipment on the second pair of NextGen fibres. Two options exist: 1. Coarse Wave Division Multiplexing (CWDM), which can provide about 8 circuits with data-speeds of 1 Gb. This equipment is relatively inexpensive. 2. Dense Wave Division Multiplexing (DWDM), which can provide up to 80 circuits and offering bandwidths greater than 1 Gb – such as 10 Gb or 40Gb The VLBI project wants to use 10 Gb or 40 Gb links by 2007, so CSIRO is keen to see DWDM deployed on this network. The cost of DWDM is about 40% higher, so the contributions of all the AARNet members using this network will be higher. Although the base costs of DWDM are higher the equipment lifecycle Last updated: 29-Jun-17 v0.5 7 of 23 ITS BUSINESS CASE Network Links to Observatories is also significantly higher at greater than 10 years compared to 3-4 years for CWDM. Early estimates of CSIRO’s startup costs for a DWDM network are between $1.6 million and $2.4 million with the annual cost between $300K and $500K. This startup cost includes a provision for a single 1 Gb connection to each site on the network. 6.2 Fibre Links to CSIRO Sites NDC has surveyed the sites and provided indicative estimates of costs for the tail-end fibre connections to the CSIRO sites. The maps show both the closest point on the cable, and the closest Controlled Environment Vault (CEV). (To see the full detail of these maps, zoom in and view in colour to see the blue paths.) 6.2.1 The Parkes Observatory Figure 2 - Connection to Parkes 6.2.2 The Mopra Observatory The Siding Spring Observatory is operated by ANU and houses a number of optical telescopes. ANU will contribute so that this link will extend beyond ATNF-Mopra to Siding Spring. Last updated: 29-Jun-17 v0.5 8 of 23 ITS BUSINESS CASE Network Links to Observatories Figure 3 - Connection to Mopra 6.2.3 The Paul Wild Observatory at Narrabri This link also passes the Cotton CRC (CSIRO and NSW Department of Agriculture) and the Wheat Research Institute (Sydney University). Connection of the Cotton CRC is related project with a low added cost – see Attachment A. Figure 4 - Connections at Narrabri 6.3 Production Network – Connect Site LANs One component of AARNet3 will be a shared network backbone provided on the second pair of NextGen fibres. This network will provide regional links for Last updated: 29-Jun-17 v0.5 9 of 23 ITS BUSINESS CASE Network Links to Observatories Universities and CSIRO in NSW. This will be shared by AARNet members, with each member providing the tail links. Preliminary estimates indicate a total budget of between $900K and $1.4 million is required for these tail circuits. ANU, Siding Springs ATNF Mopra (Coonabarab ran) ATNF Narrabri Cotton Research Narrabri ATNF Parkes AARNet Regional Network Sydney Access Node, UTS Figure 5 - Production Network Tail Links If additional circuits are required on the network above those provided in the base costs listed in section 6.1 CSIRO will be required to spend $40K per link for the installation of interfaces at each end of the link. 6.4 Research Network – for high speed data transfer The AARNet3 Regional Network may use Coarse Wave Division Multiplexing (CWDM) or Dense Wave Division Multiplexing (DWDM) to provide the backbone as discussed above. The research network will be built from dedicated wavelengths. ATNF Narrabri ATNF Mopra ATNF Parkes ATNF Epping SX Transport Tidbinbilla This Project Swinburne Univ Ceduna Last updated: 29-Jun-17 Related (future) Projects Mt Pleasant, Tas v0.5 10 of 23 ITS BUSINESS CASE Network Links to Observatories Figure 6 - Research Network - a possible design When the DWDM network needs to be upgraded for additional bandwidth over the AARNet3 Regional Network the cost per 10 Gigabit link is about $72K for distance under 600 Km, and $144K for longer distances. The design above shows 2 shorter links (Mopra and Parkes) and one longer link (Epping). The project has the option of configuring 10Gb circuits on day one if required. The Tidbinbilla link is more complex as it involves connecting via the Canberra based ICON network and obtaining bandwidth on the AARNet3 metropolitan backbone. The proposed Canberra-Tidbinbilla interconnect passes Mt Stromlo. The cost to CSIRO for the Tidbinbilla link is dependant on some negotiations with Telstra for access to spare fibre pairs on an existing cable to the site. If these negotiations are unsuccessful we estimate that it will cost approximately $400K to install a new cable between the site and the Canberra ICON network. Future links will include Mount Pleasant Radio Telescope (near Hobart), the Ceduna Radio Telescope,the Swinburne University Centre for Astrophysics and Supercomputing, international connections (e.g. Japan, USA) and possibly to SKA pathfinder projects in WA. 6.5 Key Milestones Key milestones for this Project are: Event/Milestone Deliverable Date AARNet secures AREN funding Agreement with Department of Education, Science and Training May-04 AARNet completes RFT for equipment Supplier selected and equipment costs known May-04 AARNet secures member sign-offs Agreement from participating universities and CSIRO Jun-04 AARNet provides physical connections to CSIRO sites Connections into active equipment in site computer rooms Aug-04 Production network operational with gigabit links Cutover from existing leased lines to gigabit backbone Aug-04 Research network links (first phase) operational Dedicated links operate at 1 Gb Nov-04 7. Risks 7.1 Project Risks Ability to Deliver Networks to Required Specifications The technology being deployed is well tested, and the suppliers well known. Our network designs are not pushing specifications to their limits. So the risk that we will not be able to deliver the specified network performance is low. Ability of ATNF, ITS and AARNet Staff to Deliver Last updated: 29-Jun-17 v0.5 11 of 23 ITS BUSINESS CASE Network Links to Observatories The teams in the three groups are well qualified and highly motivated, so this is also a low risk. Stakeholder Commitment Commitment from ATNF scientists and ATNF partners (Swinburne, University of Tasmania, ANU, …) is high. AARNet is motivated to deliver good outcomes in response to Federal Government funding. However, it is not clear yet if the AARNet regional members will all be in a position to support the initiative promptly, so this risk is assessed as moderate. Summary This project is not a high risk project. 7.2 Business Risks Level of AREN Funding The current expectation is that the Department of Education, Science (DEST) and Training will fund $5 million, and the total cost of equipment required is around $10 million. In Costs Analysis, we present costs to CSIRO based on this expectation. Any substantial decrease in DEST funding would put the project at risk. This risk is assessed as low. Financial commitment from AARNet members The project depends on all the participating universities providing a share of the costs, so any financial difficulties of members could present a project risk. This risk is assessed as medium. Loss of Reputation if unable to deliver The Department of Education, Science is supporting the project with the expectation that it will lead to scientific advances. If CSIRO is not able to demonstrate these advances, our reputation as a leading science agency would suffer. This risk is assessed as low. Summary The business risks are not high. 8. Benefits/Value Analysis 8.1 Position CSIRO as a World Leader in VLBI VLBI (Very Long Baseline Interferometry) currently requires data to be recorded at each telescope and then transported to a central point for correlation. For eVLBI, data will be recorded at each telescope, and at the same time will be transferred across the network to be correlated in real time (or near real time in the early stages) at a central location. The eVLBI project will include the ATNF telescopes and the Tidbinbilla tracking station near Canberra initially. Other research groups working on real-time correlation for VLBI are aiming at data rates of 1-2 Gb/sec. CSIRO aims to operate with data rates of at least 2 x 8 Gb/sec within 3 years and this would provide the world's best VLBI sensitivities. This will open new avenues of astronomical research and has the potential to revolutionise the VLBI field and lead to significant new discoveries. Last updated: 29-Jun-17 v0.5 12 of 23 ITS BUSINESS CASE Network Links to Observatories Attachment B and Attachment C provide details of the costs and benefits summarised below. 8.1.1 Current Costs and Outputs Costs for the VLBI project are approximately: o 5% usage of equipment valued at $240 million (ATNF and Univ of Tas) o 2% usage of equipment valued at $100 million (Tidbinbilla) o Exclusive use of correlation equipment and tapes valued at $1 million o $500,000 per year in operating costs including staff for observing Annual output about 8 papers/year (= 5% of ATNF papers (156 in 2003)) 8.1.2 With the First Stage of Research Network When the research network is in place with dedicated 1 Gb links, the sensitivity will be increased 2 to 3 times. The benefits will be: o the science achievable is enhanced by the increased sensitivity which enables study of fainter astronomical objects o greater sensitivity and ease of use which will attract more astronomers o automated data transfer will save 20 person weeks per year These benefits will effectively double the value of the ATNF infrastructure utilised by VLBI (currently 5% of $240 million). 8.1.3 With the Final Stage of the Research Network When the research links are operating at 10 Gb or better, the sensitivity will be increased 11 times. This will: o open up new avenues of research by greatly increasing the number of astronomical objects for study. This has the potential for many significant discoveries in high resolution radio astronomy. o enhance and facilitate access to VLBI observations for the whole astronomical community and it is expected to bring many new users to VLBI. Thus VLBI in Australia will cease to be a niche technique and will join the mainstream of radio astronomy observing. o increase the proportion of VLBI time by increasing the scientific capabilities and attracting new users. o provide savings in operating costs by effectively eliminating the specific VLBI costs of the current network and integrating operations with the ATCA at Narrabri. These benefits will provide ATNF with infrastructure that has an effective value of $2.7 billion if the increases in sensitivity were achieved by constructing larger antennas. The 5% utilised by VLBI will effectively increase in value from $10M to $135M. 8.2 Developing a Data Grid The Australian Virtual Observatory aims to provide an interface to data archives from Australian Telescope Compact Array. This project, together with other Last updated: 29-Jun-17 v0.5 13 of 23 ITS BUSINESS CASE Network Links to Observatories international projects, will facilitate data mining projects by astronomers here and internationally. The ICT Centre is running a case study for data grids that use a computer cluster to manage about 2 terabytes of observed data. This cluster is in Canberra and data is physically transported. Gigabit connections will allow the data to be transferred across the network and the collection will probably be moved to somewhere on the Research Network – perhaps Sydney. 8.3 Intensify the Pulsar Timing Partnership The Pulsar Timing Array is a collaborative effort between the ATNF, Swinburne University of Technology and international collaborators. It also uses computer clusters – currently a small cluster at Parkes, but the aim is to use a more powerful cluster at Swinburne. High speed links will allow transport in near real time, rather than shipping disk packs. 8.4 Efficient Service Delivery This project will improve service delivery on the local networks at Parkes and the Narrabri Compact Array. A Related Project will provide a similar improvement for the local network at Cotton Research. One-IT will re-organize service delivery to use a 3 tier model with only tier-2 being on-site, and the impact of this re-organisation will be greater on sites with gigabit connections. To illustrate, consider some services provided over slow links compared to high-speed links, as shown in the table. Service Slow speed site Gigabit Site Desktop support Most tasks by local support staff Many tasks solved by central support PC software builds Performed locally using CDs Done using downloads from a central site, and can be policy driven and automated. Management of servers for files, DHCP, DNS, etc Done locally, need to maintain expertise for a range of servers at each site, so the expertise available is medium level Done remotely, using highly experienced staff. Also saves on training costs. Off-site backups Tapes written locally and physically transported to another site Backups can be over the network to another data centre Video conference support Need to use ISDN, require expertise locally, bridging is complex Only require IP on-site, technical support can mostly be provided by remote specialists Last updated: 29-Jun-17 v0.5 14 of 23 ITS BUSINESS CASE Network Links to Observatories The outcome is that part of the time of local IT support staff is freed up from the generic IT tasks and can be re-directed to science-linked requirements of projects at that site. 8.5 Research Network: Enhance CSIRO’s Reputation The Government is funding AREN with $42.5 million. This project is one potential high profile outcome that can demonstrate benefits from the government investment. If CSIRO is able to use this project to enhance its reputation in the eyes of government and the international research community, it should improve prospects for future government funding to CSIRO. 8.6 Returns and Benefits Key Project Objectives Output Measure/TYPE (IRACIS) A Regional Network for AARNet in NSW Fibre links to CSIRO sites IS A Research Network to connect observatories Dedicated high-speed links for observed data IR, IS A Production network with gigabit links One IT Service Delivery AC, IS Outcome(s) Measure/TYPE (IRACIS) Accountable Stakeholder Enhanced value of observatory infrastructure IR ATNF Improved efficiency for consolidated service delivery AC One-IT 9. Cost Analysis 9.1 Infrastructure Costs CWDM Option The CSIRO initial cost would be between $1 million and $1.5 million with annual maintenance between $150,000 and $350,000. DWDM Option The CSIRO initial cost would be between $1.7 and $2.5, with annual maintenance between $300K and $500K Last updated: 29-Jun-17 v0.5 15 of 23 ITS BUSINESS CASE 9.2 Network Links to Observatories Staff Costs Time and travel for ITS Networks project staff - $150K 10. Staff Impact/Change Management There will be changes to the way ATNF scientific staff time is used during observations for VLBI. There will be changes to the job descriptions of IT support staff at Parkes, ATNF Narrabri and the Cotton CRC. These changes will come as part of the One-IT project, but the One-IT changes will have more impact on sites with gigabit connectivity (compared to sites on slow links). Last updated: 29-Jun-17 v0.5 16 of 23 ITS BUSINESS CASE Network Links to Observatories Attachment A: Related Project: Link Cotton CRC to ATNF Narrabri The AARNet connection at Narrabri will be into the computer room at the ATNF site. The fibre laid from the AARNet backbone to ATNF will have at least 12 pairs, so that Myall Vale can be connected to ATNF using one of the spare pairs. The Myall Vale site will then be configures as two or more VLANs of the local network based at ATNF. The cost to provide 1. tail-end fibre from the road into Myall Vale 2. active equipment at each end of the ATNF-Myall Vale link will be $50,000. The saving from discontinuing the current leased line to Myall Vale will be $30,000 per year. This project will enhance collaboration of the Cotton CRC and its 11 core participants around Australia. Last updated: 29-Jun-17 v0.5 17 of 23 ITS BUSINESS CASE Network Links to Observatories Attachment B –Reduced Costs and Increased Benefits of Fast Network Links to Radio Astronomy Observatories A. Tzioumis 20/4/2004 This note provides estimates of current costs and attempts to quantify the gains that will result from fast network links. The most significant aspect is the greatly enhanced performance of the network and the consequent enabling of new science. Position CSIRO as a World Leader in e-VLBI Other research groups working on real-time correlation for VLBI are aiming at data rates of 1-2 Gb/sec. CSIRO aims to operate with data rates of at least 2 x 8 Gb/sec within 3 years and this would provide the world's best VLBI sensitivities. This will open new avenues of astronomical research and has the potential to revolutionise the VLBI field and lead to significant new discoveries. An interim 1 Gb/s network would be useful scientifically and a technologically necessary step towards the higher data rates. The ATNF would be able to utilise shared Gb/s links but dedicated multiple 1 Gb/s links would provide much better opportunities for significant scientific improvements. Current Costs and Outputs Costs for the VLBI project at ATNF and partners in Australia. Values are approximate and rounded off. The components are o ~5% of time on ATNF telescopes, total value of about $200M (ATCA $100M, Parkes $80M, Mopra $20M) o + 5% of time on UTas antennas, valued at $40M (Hobart $20M, Ceduna $20M) o + 2% of Tidbinbilla time, total value of ~$100M Special correlation equipment (correlator, PTs, tapes) valued at $1M ~5% of cost of operating & maintaining the telescopes and general ATNF facilities. ATNF operations budget is ~$10M (out of total of $20M). Hence, prorata operating cost of VLBI is about $0.5M per year. This includes VLBI specific components i.e. o Staff for operations support and correlation at ATNF ~ 1 FTE o Support for VLBI observations at ATNF of at least 20-25 person weeks/year (23 people/antenna x 3 ATNF antennas x 3x1-week periods/year) + Operating costs at UTas & Tidbinbilla. Annual output about 8 papers/year (= 5% of ATNF papers (156 in 2003)) With Shared Gigabit Links When all the telescopes and correlator facilities have gigabit connections via a shared backbone, there are a number of options to use the network to: streamline current operations, with real productivity gains achieve a modest increase in network sensitivity and performance. To achieve any of these gains a substantial proportion of the network links will need to be devoted to e-VLBI during observing sessions. Last updated: 29-Jun-17 v0.5 18 of 23 ITS BUSINESS CASE Network Links to Observatories However, it should be noted that shared 1 Gb/s links will only approach the performance already achieved with disk-based VLBI systems. To improve on that the ATNF could use multiple dedicated 1 Gb/s links if these became available. The advantages will be: A. The ability to transfer the current data rates (128 Mb/s) in real time to the correlator facility in Sydney would greatly benefit operations, i.e. save most of the 20-25 person weeks/year needed to support observations. 2-3 people could run most of the network remotely, and some of this support could also come from current correlator operations support. Up to 20 person weeks/year may be saved. provide near real-time results, which will greatly enhance the reliability of the VLBI network and provide the ability to react to transient events attract new astronomers to VLBI by greatly simplifying the effort currently needed. B. Increase the data rates (2, 4 or 8 times) to increase VLBI sensitivity (Attach. C). 256Mb/s are available now at the telescopes and could be handled with a little effort at the Sydney correlator. Sensitivity increase ~40%. 512 Mb/s are available but need a modest effort to interface to the network. Will have to be processed by the Swinburne super-computer by storing on disks and correlating more slowly. Sensitivity increased x 2. 1 Gb/s may be achievable for special projects by duplicating some equipment at the telescopes. Again only the Swinburne supercomputer could handle these rates. Sensitivity increased ~ x3. The increases in data rates and sensitivity substantially enhance the achievable science. The largest impact is increasing the numbers of fainter astronomical objects that could be studied. These are relatively moderate for the 2-3 times increase in sensitivity offered by this network and comparable to what is achievable with disk systems. The value of these increases could be quantified by comparisons with equivalent increases in observing time or antenna size. Details are given in Attachment C. In summary, the faster network rates are effectively equivalent to: utilising 10-40% of the network while only using 5% of time increased productivity by "saving" equivalent costs of $1-4M/year increasing the effective value of the ATNF facilities from $200M to $260-500M. Thus the 5% VLBI time effectively increases in value from $10M to $12-25M. With Dedicated multiple 10 Gb/s Links When the new correlator and the dedicated 2 x 8 Gb/s data paths are in use (in about 3 years), the correlation will be done in real time, and the sensitivity will be enhanced about 10 times. This will: open up new avenues of research by greatly increasing the number of astronomical objects for study. This has the potential for many significant discoveries in high resolution radio astronomy. provide savings in operating costs by effectively eliminating the specific VLBI costs of the current network and integrating operations with the ATCA at Narrabri. enhance and facilitate access to VLBI observations for the whole astronomical community and it is expected to bring many new users to VLBI. Thus VLBI in Australia will cease to be a niche technique and will join the mainstream of radio astronomy observing. increase the proportion of VLBI time by increasing the scientific capabilities and attracting new users. generate equivalent savings by greatly increasing the productivity of the telescopes (see Attachment C for details). In summary these are effectively equivalent to: Last updated: 29-Jun-17 v0.5 19 of 23 ITS BUSINESS CASE o o o Network Links to Observatories the 5% of time is increased 128-fold! 1-week then will be equivalent to 2.5 years of continuous observing on the current systems. Or 1 day on the new system would be equivalent to the combined rates of the previous 6 years on the current VLBI system! equivalent operational budget of $64M at current data rates equivalent to having telescopes worth ~$2.7B The 5% VLBI time then effectively increases from ~$10M to ~$135M. The development of the networks, new correlators and the e-VLBI operation are all key technologies for the SKA. Thus these developments will facilitate demonstration of SKA capabilities in Australia and enhance Australia's prospects to host the ~US$1B SKA. Last updated: 29-Jun-17 v0.5 20 of 23 ITS BUSINESS CASE Network Links to Observatories Attachment C - Quantifying the gains from increased VLBI data rates & sensitivity A. Tzioumis 20/4/2004 Introduction It is difficult to accurately quantify the gains that may be achieved by increasing the data recording rates The approaches here often oversimplify the situation and are meant only as a guide. They give an idea of the cost of achieving similar increases by other available means. Alternatively, they provide an indicative dollar value of the proposed improvements in data rates. Some concepts Radio telescopes record extremely faint signals, many millions of times below the radio noise level. Hence, high sensitivity and special correlation techniques are used to detect the signal from the general radio noise. The sensitivity of a radio telescope or array, is determined primarily by: The size of the antenna. Sensitivity is proportional to the dish Area (A), which 2 is determined by the dish Diameter (D). Area A D The sensitivity of the receivers, as measured by the system Temperature (Tsys). State-of-the-art receivers achieve Tsys = 20K. Sensitivity is proportional to 2 1 Tsys . The Bandwidth (B) of the receiving system in MHz or GHz. This determines the output data rate in Mb/s or Gb/s. Usually, output data rates are 4 x B. The integration time T i.e. the time that raw data is appropriately averaged together to increase the signal-to-noise-ratio (SNR) of the data. Sensitivity is proportional to B T The sensitivity of the telescope provides a measure of the weakest radio signals that can be detected. The vast majority of astronomical objects emit very weak radio signals. Hence, sensitivity increases open to study a much greater population and wider range of astronomical objects. This enhanced "parameter space" of study most often provides the most exciting discoveries in astronomy. Sensitivity enhancement The sensitivity of a radio telescope or array can be increased by: increasing the size of antenna. This can be very expensive as the cost of an 2.7 antenna scales as D (See Thompson, Moran, and Swenson, p. 163). decreasing the system temperature Tsys. Improvements are made in this area all the time but changing receiver hardware can also be expensive. increasing the Bandwidth B, and hence the data rates. This is the area that fast network speeds have an impact. increasing the integration time T. This is also very expensive as telescopes are expensive to operate. In some cases like VLBI, the utility of longer integration is often limited by other technical factors like phase stability and coherence. The improved sensitivity resulting from increases in Bandwidth (made possible by higher network rates) can be quantified at a simple level by comparing with the equivalent costs of changing other parameters in the system like Time (T) and antenna Area (A). Last updated: 29-Jun-17 v0.5 21 of 23 ITS BUSINESS CASE Network Links to Observatories A. Comparison with Time T As sensitivity is proportional to B T , any increase in Bandwidth B while Time remains constant is directly equivalent to keeping B constant but increasing Time by the same amount. For the proposed e-VLBI systems the Bandwidth increases have the same effest as the data rate increases. Hence: The 1 Gb/s shared network proposed will allow increases in data rates from the current 128 Mb/s to 256 or 512 Mb/s or even up to 1 Gb/s. Thus data rates increase by factors of 2-8. This is equivalent to increasing the time of operation 2-8 times. As VLBI uses 5% of time on CSIRO facilities valued at $200M, increases by factors of 2-8 provide that many times increase in productivity of these assets. It may be thought of as equivalent to utilising 10-40% of the facilities while only really using 5% of the time. Equivalently, the 5% of VLBI time can be considered to cost 5% of the ATNF operating budget of $10M/year i.e. $0.5M/year. An effective time increase of 2-8 times can then be thought to provide equivalent productivity increases in saved operating costs of $14M/year! It would cost that much if one could buy the extra time on the telescopes. The 2x8 Gb/s e-VLBI network that would be made possible with the advanced 10 Gb/s network links, achieves an 128-fold increase in data rates or equivalently in bandwidth. Translating these to Time gives a 128-fold increase in effective time utilisation. Thus the 5% on the $200M facilities becomes 640% i.e. it is equivalent to using full-time 6 such facilities or using the facilities exclusively for VLBI for 6 years! Similarly, the $0.5M/year pro-rata operational budget in ATNF for VLBI becomes an equivalent $64M saving! CAUTION: These need to be treated carefully, as in many cases the increases in bandwidth cannot actually be substituted by increases in time. The above figures are intended to give a feel for equivalent costs. B. Comparison with telescope Area A Sensitivity is also directly proportional to the size of the antennas and the costs of these are relatively well known. So, the increases in sensitivity due to the increase data rates and hence Bandwidth can also be translated to equivalent increases in telescope collecting area. Increases in Bandwidth increase sensitivity as increase in Area also scales as the Diameter increase of B and hence the equivalent (increase _ in _ B) . The telescope Area A scales as D 2 . Thus an increase in B would correspond to a Diameter D 1 4 (increase _ in _ B) . However, it has been well established that antenna building costs scale as Thus antenna costs would scale as (increase _ in _ B) 1 ( 2.7 ) 4 D 2.7 . . The ATNF antennas comprise 7 x 22m antennas (ATCA and Mopra) and the 64m antenna at Parkes. Mechanical construction costs of these are about $100M ($4m for each 22m antenna, $70M for Parkes). The 1 Gb/s shared network proposed will allow increases in data rates from the current 128 Mb/s to 256 or 512 Mb/s or even up to 1 Gb/s. Thus data rates increase by factors of 2-8. This is equivalent to increasing the area of the antenna by 1.4-2.8 times and the antenna cost by 1.6-4 times. Effectively, the Last updated: 29-Jun-17 v0.5 22 of 23 ITS BUSINESS CASE Network Links to Observatories equivalent value of the ATNF facilities increases from $200M to $260-500M. The 5% VLBI share increases accordingly (from $10M to $12-25M). The 2 x 8 GB/s e-VLBI network that would be made possible with the advanced 10 Gb/s network links, achieves an 128-fold increase in data rates or equivalently in bandwidth. The increase in sensitivity or equivalently in area is 11 times and the effective antenna cost increase is 26 times! Hence, the equivalent value of the ATNF facilities increases to $2.7B!! The 5% VLBI time effectively increases in value from $10M to $135M. CAUTION: Again, these numbers are meant only to demonstrate that increases in data rates achieve increases in telescope sensitivities that could be achieved by building bigger telescopes costing many $10Ms or even $100Ms. Comparison with other planned facilities The only planned high data rates facility is the e-MERLIN in the UK. It is a similar instrument but covers shorter distances from 5-200km and has less overall collecting area. It will use dedicated fibre links for rates up to 24 Gb/s but the effective Bandwidth will be the same as the ATNF proposal. The extra bits are to be used for RFI mitigation which is a lesser problem in Australia. However, we can achieve similar RFI mitigation effects by increasing our rates as our correlator can handle a phenomenal 128 Gb/s from each telescope! This project has been funded as an upgrade of the existing MERLIN radio-linked instrument and will cost about A$20M. (Details in http://www.merlin.ac.uk/e-merlin/) Summary The proposed increases in data rates for VLBI that will be provided by the new network connections will provide very significant increases in observational capacity. These can be roughly quantified by comparison with equivalent increases that could be achieved by increasing observational time or the size of the telescopes. In both cases, the equivalent savings in operational costs or the equivalent increase in the value of the facility are many millions of dollars. However, the main impact of the increased sensitivity is the opening of new areas of study in radio astronomy and the huge scientific impact that these can have. Last updated: 29-Jun-17 v0.5 23 of 23