Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Natural computing wikipedia , lookup
Mathematical optimization wikipedia , lookup
Computational complexity theory wikipedia , lookup
Multiple-criteria decision analysis wikipedia , lookup
Inverse problem wikipedia , lookup
Stream processing wikipedia , lookup
Lateral computing wikipedia , lookup
Distributed Programming CA107 Topics in Computing Series Martin Crane Karl Podesta The Basics….. • What is a Distributed System (DS)? • How does it differ from a Parallel Computer (MPP)? – differences become fuzzy…now called Supercomputers or High Performance Computers (HPC) • Supercomputers and Supermodels: – – – – both expensive both hard to deal with/prone to tantrums both look glamorous but... Both spend lots of time doing tedious tasks for others: • mostly matrix-vector products for Supercomputers • being live mannequins for Supermodels Why High Performance Computing? • Solve larger and larger scientific problems – advanced product design – economic analysis – weather prediction/ climate modelling • Store and process huge amount of data – data mining and knowledge discovery – image processing, multi-media information – internet information storage and search (eg GOOGLE) Different Supercomputers (MPPs) in Your Neighbourhood • Single Instruction, Multiple Data (SIMD) – as seen on PlayStation 2 – very useful for processing large arrays eg a(i) = b(i) + c(i)*d(i) {as are found in games} • Multiple Instruction, Multiple Data (MIMD) – as seen in Deep Blue • But these are dinosaurs - we want something more flexible Problems with Traditional Supercomputer (ie MPP) • Expensive – – – – Very high starting cost ($10,000s per node) Expensive software High maintenance cost Costly to upgrade • Vendor dependent – lots of companies have come and gone (datacube, Connection Machines etc.) So, real/poor people cannot do HPC! PC Cluster: a poor-man’s supercomputer! • built from high-end PCs and high-speed comms network • supports standard parallel programming based on message-passing model (MPI language) • cheap (16 node cluster can cost less than $10k) Cluster Diagram Here DCU CA Cluster Resources • “John the Baptist” Cluster – – – – built by Redbrick using old CA machines 24 individual 450MHz machines connected by a fast ethernet switch harbinger of better things…. • “The one that is to come”…… – – – – 24 SMP machines each with 2 GHz plus loadsa memory! arrives about Xmas time, appropriately enough What are the issues in HPC? • Communication Vs Computation – size/ nature of problem – interconnect speed/ processor speed • Fault tolerance – quality of hardware – nature of problem • Load balancing – nature of problem/ quality of programmer – even an easy problem can be made difficult & slow by a bad implementation Influence of Nature of Problem on Speed • What is speed? – speed up is better: Time on 1 node/ Time on n nodes • Speed-up and Problems – very good: embarrassingly parallel problems – fair to middling: regular and synchronous problems • a bit of cross-talk between nodes – bad: irregular/ asynchronous problems • lots of cross-talk between nodes