Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Cracking of wireless networks wikipedia , lookup
Piggybacking (Internet access) wikipedia , lookup
Computer security wikipedia , lookup
Zero-configuration networking wikipedia , lookup
Recursive InterNetwork Architecture (RINA) wikipedia , lookup
Distributed firewall wikipedia , lookup
Service-oriented architecture implementation framework wikipedia , lookup
Introduction to Distributed System and Computing From Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edition 5, © Pearson Education 2012 Minqi Zhou @ ECNU Information Office :数学馆东111 Tel:021-62235022 Email: [email protected] 2 Overview This course focuses on the issues fundamental to the understanding of distributed systems. Text and reference books Distributed Systems - Concept and Design, G. Coulouris, J, Dollimore, and T. Kindberg, Addison-Wesley, 5th Edition, 2012. Distributed Systems – Principles and Paradigms, A. Tanenbaum and M. van Steen, Prentice Hall, 2nd Edition 2008. Course Webpage: http://dase.ecnu.edu.cn/mqzhou/Teaching.html Check Regularly Tentative Grading Class Projects Final Reports Open : : : : 25 % 40% 25 % 10% 4 Building a Distributed System “The number of people who know how to build really solid distributed systems…is about ten” Scott Shenker, Professor at UC Berkeley The point: it’s hard to build a solid distributed system. So, why is it hard?...but first of all… 5 Why distributed computing? Resource sharing/Economics: distributed systems allow the pooling of resources, including CPU cycles, data storage, input/output devices, and services. Scalability: Increasing demands and users can be easily addressed by adding more resources and the system can still run effectively. Reliability: a distributed system allow replication of resources and/or services, thus reducing service outage due to failures. The affordability of computers and availability of network access: The Internet has become a universal platform for distributed computing. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 5 © Pearson Education 2012 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Topics Defining Distributed Systems Resource sharing and the Web Design Challenges of Distributed Systems CHARACTERIZATION OF DISTRIBUTED SYSTEMS Defining Distributed Systems Various definition of distributed systems have been given in the literature, for example: A collection of logically related data that is distributed over different processing nodes of computer network. Definition above does not provide all characteristics of distributed systems. 8 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Defining Distributed Systems It is difficult to find a definition that provides all characteristics of distributed systems. Another way is a definition that gives a loose characterization for distributed systems such as: A distributed system is a collection of independent computers that appear to the users of the system as a single computer. With any definition, sharing of resources is a main motivation for constructing distributed systems. CHARACTERIZATION OF DISTRIBUTED SYSTEMS Defining Distributed Systems In this course, we define distributed systems more precisely as : A distributed system is one in which hardware or software components located at networked computers communicate and coordinate their actions only by message passing. Definition above covers the entire range of distributed systems in which networked computers can usefully be deployed. 10 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Defining Distributed Systems Networks of computers are everywhere! Examples of networks of computers are: Mobile phone networks Corporate networks Factory networks Campus networks Home networks In-car networks On board networks in aero planes and trains 11 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Defining Distributed Systems Our definition of distributed systems has the following significant consequences: Concurrency Tasks carry out independently No global clock Tasks coordinate their actions by exchanging messages 12 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Defining Distributed Systems Independent Failures Faults in the network result in the isolation of the computers that are connected to it. Each component of the system can fail independently, leaving the others still running. 13 CHARACTERIZATION OF DISTRIBUTED SYSTEMS General Examples of Distributed Systems Internet Intranets Mobile networks 14 CHARACTERIZATION OF DISTRIBUTED SYSTEMS General Examples of Distributed Systems The Internet The Internet is a vast interconnected collection of computer networks of many different types. Multimedia services are available in the Internet enabling users to access audio and video data including music, radio, TV channels, phone, and video conferencing. (Figure 1) 15 CHARACTERIZATION OF DISTRIBUTED SYSTEMS General Examples of Distributed Systems intranet % ISP % % % backbone satellite link desktop computer: server: network link: Figure 1. A typical portion of the Internet 16 CHARACTERIZATION OF DISTRIBUTED SYSTEMS General Examples of Distributed Systems Intranet An intranet is a portion of the Internet that is separately administered and has a boundary that can be configured to enforce local security policies. (Figure 2) 17 CHARACTERIZATION OF DISTRIBUTED SYSTEMS General Examples of Distributed Systems email s erv er Desktop computers print and other s erv ers Web server Loc al area netw ork email s erv er File s erv er print other s erv ers the res t of the Internet router/f irew all Figure 2. A typical Interanet 18 CHARACTERIZATION OF DISTRIBUTED SYSTEMS General Examples of Distributed Systems Mobile networks Technological advances in device miniatorization and wireless networking have led increasingly to the integration of small and portable computing devices into distributed systems. These devices include: Laptop computers 19 CHARACTERIZATION OF DISTRIBUTED SYSTEMS General Examples of Distributed Systems Handheld devices • • • • • Personal digital assistants(PDAs) Mobile phones Pagers Video cameras Digital cameras 20 CHARACTERIZATION OF DISTRIBUTED SYSTEMS General Examples of Distributed Systems Wearable devices Smart watches with functionality similar to a PDA Devices embedded in appliances Washing machines Hi-fi systems Cars Refrigerators (Figure 3) 21 CHARACTERIZATION OF DISTRIBUTED SYSTEMS General Examples of Distributed Systems Internet Host intranet WAP gatew ay Wireless LAN Mobile phone Laptop Printer Camera Home intranet Host site Figure 3. Portable and handheld devices in a distributed system. 22 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Other Examples of Distributed Systems Cluster A type of parallel or distributed processing system, which consists of a collection of interconnected stand-alone computers cooperatively working together as a single, integrated computing resource. The computers may be standard per uniprocessor or multiprocessor. A cluster can be used for providing highly scalable services such as search engines provide for users all over the Internet. 23 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Other Examples of Distributed Systems Grid A type of parallel and distributed system that enables the sharing, selection, and aggregation of geographically distributed autonomous resources dynamically at runtime depending on their availability, capability, performance, cost, and users' quality-ofservice requirements. 24 Examples of Distributed Systems Web Search: major growth industry in the last decade. 10 billion per month for global number of searches. Complex task for searching a big database with 63 billion pages. (e.g. Google distributed infrastructure, file system, storage, lock service, parallel computing) Massively multiplayer online games: Large number of people interact through the Internet with a virtual world. Challenges include fast response time, real-time propagation of events. Financial trading: provides real-time access to a wide range of information sources such as current share prices and trends, economic and political development news. Challenges include how to deliver events reliably and in a timely manner to very large numbers of clients. Peer-to-Peer system Trend in Distributed Systems Pervasive networking and modern Internet: Wifi, WiMax, Bluetooth, the third-generation of mobile phone networks. The result is that networking has become a pervasive resource and devices can be connected at any time and in any place. Mobile and ubiquitous computing: small and portable computing devices are integrated into the distributed system such as laptop, handheld devices( PDA, cell phone, camera etc). Distributed multimedia systems: it support a range of media types such as audio, video in a distributed system. So desktop can access live television, file libraries, music libraries, telephone IP phone (Skype) in distributed system. QoS issue Distributed Computing as a utility: A number of companies provide the computing, storage and application resources as a commodity or utility to users. So users do not need to maintain local IT. Pay as use. Analogy between distributed resources and other utilities such a water or electricity. CHARACTERIZATION OF DISTRIBUTED SYSTEMS Design Challenges of Distributed Systems Designers of distributed systems need to take the following challenges into account: Heterogeneity Heterogeneous components must be able to interoperate. Openness Interfaces should allow components to be added or replaced. Security The system should only be used in the way intended. 27 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Design Challenges of Distributed Systems Scalability System should work efficiently with an increasing number of users. System performance should increase with inclusion of additional resources. Failure handling Failure of a component (partial failure) should not result in failure of the whole system. 28 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Design Challenges of Distributed Systems Transparency Distribution should be hidden from the user as much as possible. 29 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Heterogeneity Heterogeneous components that must be able to interoperate, apply to all of the following: Networks Hardware architectures Operating systems Programming languages 30 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Heterogeneity Examples that mask differences in network, operating systems, hardware and software to provide heterogeneity are Middleware Internet protocols Mobile code 31 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Heterogeneity Middleware Middleware applies to a software layer. Middleware provides a programming abstraction. Middleware masks the heterogeneity of the underlying networks, hardware, operating systems and programming languages. The Common Object Request Broker (CORBA) is a middleware example. 32 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Heterogeneity Mobile code Mobile code is the code that can be sent from one computer to another and run at the destination. Java applets are the example of mobile codes. 33 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Heterogeneity Virtual machine Virtual machine provides a way of making code executable on any hardware. 34 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Openness Distributed systems must be extensible. Openness of a computer system is the characteristic that determines whether the system can be extended and re-implemented in various ways. 35 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Openness The first step in openness is publishing the documentation of software components and interfaces of the components to make them available to software developers. 36 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Security Security of a computer system is the characteristic that the resources are accessible to authorized users and used in the way they are intended. Security for information resources has three components: Confidentiality Protection against disclosure to unauthorized individual. 37 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Security Integrity Protection against alteration or corruption. Availability Protection against interference with the means to access the resources. 38 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Security Security Mechanisms are: Encryption Authentication Authorization 39 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Security challenges Denial of service attacks Denial of service attacks is an attempt to make a computer resource unavailable to its intended users. Security of mobile code Mobile code needs to be handled with care. E.g. receiving an executable program as an electronic mail attachment to display an interesting picture but in reality it may access local resources, or perhaps be part of a denial of service attack. 40 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Scalability Scalable distributed systems operate effectively and efficiently at many different scales, ranging from a small Intranet to the Internet. Scalable distributed systems remain effective when there is a significant increase in the number of resources and the number of users. 41 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Scalability Challenges of designing scalable distributed systems are: Controlling the cost of physical resources Cost should linearly increase with the system size. Controlling the performance loss For example, in hierarchically structured data, search performance loss due to data growth should not be beyond O(log n), where n is the size of data. 42 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Scalability Preventing software resources running out An example is the numbers used as Internet addresses (IP)(32 bit->128-bit) Y2K like problem. Avoiding performance bottlenecks Using decentralized algorithms to avoid having performance bottlenecks. Caching and replication in Web are examples of providing scalability. 43 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Failure handling Failures in distributed systems are partial, that is some components fail while others continue to function. Techniques for dealing with failures: Detecting failures E.g. Checksums Masking failures E.g. Retransmission of corrupt messages E.g. File redundancy 44 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Failure handling Tolerating failures E.g. Exception handling E.g. Timeouts Recovery from Failure E.g. Rollback mechanisms Redundancy E.g. Redundant components 45 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Concurrency With concurrency, services and applications can be shared by clients in a distributed system. For an object to be safe in a concurrent environment, its operations must be synchronized in such a way that its data remains consistent. 46 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Concurrency Concurrency can be achieved by standard techniques such as semaphores, which are used in most operating systems. 47 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Transparency Transparency is defined as the hiding of the separation of components in a distributed systems from the user and the application programmer. With transparency the system is perceived as a whole rather than a collection of independent components. 48 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Transparency Forms of transparencies: Access transparency Enables local and remote resources to be accessed using identical operations. Location transparency Enables resources to be accessed without knowledge of their physical or network location (for example, which building or IP address). 49 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Transparency Concurrency transparency Enables several processes to operate concurrently using shared resources without interference between them. Replication transparency Enables multiple instances of resources to be used to increase reliability and performance without knowledge of the replicas by users or application programmers. 50 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Transparency Failure transparency Enables the concealment of faults, allowing users and application programs to complete their tasks despite the failure of hardware or software components. Mobility transparency Allows the movement of resources and clients within a system without affecting the operation of users or programs. 51 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Transparency Performance transparency Allows the system to be reconfigured to improve performance as loads vary. Scaling transparency Allows the system and applications to expand in scale without change to the system structure or the application algorithms. 52 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Transparency The two most important transparencies are access and location transparency referred to together as network transparency. Presence or absence of network transparency most strongly affects the utilization of distributed resources. 53 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Resource sharing and the Web The World Wide Web The World Wide Web is an evolving system for publishing and accessing resources and services across the Internet. (Figure 4) 54 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Resource sharing and the Web www.google.com Web servers www.cdk3.net http://www.google.com/search?q=Buyya Browsers http://www.cdk3.net/ www.w3c.org File system of www.w3c.org http://www.w3c.org/Protocols/Activity.html Protocols Activity.html Figure 4. Web servers and web browsers. 55 CHARACTERIZATION OF DISTRIBUTED SYSTEMS Question 1.10 The INFO service manages a potentially very large set of resources, each of which can be accessed by users throughout the Internet by means of a key (a string name). Discuss an approach to the design of the names of the resources that achieves the minimum loss of performance as the number of resources in the service increases. Suggest how the INFO service can be implemented so as to avoid performance bottlenecks when the number of users becomes very large 56 Figure 1.5 Computers in the Internet Computers Date 1979, Dec. 1989, July 1999, July 2003, Jan. Web servers 188 0 130,000 56,218,000 171,638,297 0 5,560,866 35,424,956 Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Centralized vs. Distributed Computing terminal mainframe computer workstation network link network host centralized computing distributed computing Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Evolution of paradigms Client-server: Socket API, remote method invocation Distributed objects Object broker: CORBA Network service: Jini Object space: JavaSpaces Mobile agents Message oriented middleware (MOM): Java Message Service Collaborative applications Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Operating Systems Basics Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Process and program A process consists of an executing program, its current values, state information, and the resources used by the operating system to manage its execution. A program is an artifact constructed by a software developer; a process is a dynamic entity which exists only when a program is run. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Process State Transition Diagram terminated start queued exit dispatch ready event completion running waiting for event blocked Simplifed finite state diagram for a process's lifetime Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Java processes There are three types of Java program: applications, applets, and servlets, all are written as a class. A Java application program has a main method, and is run as an independent(standalone) process. An applet does not have a main method, and is run using a browser or the appletviewer. A servlet does not have a main method, and is run in the context of a web server. A Java program is compiled into bytecode, a universal object code. When run, the bytecode is interpreted by the Java Virtual Machine (JVM). Three Types of Java programs Applications a program whose byte code can be run on any system which has a Java Virtual Machine. An application may be standalone (monolithic) or distributed (if it interacts with another process). Applets A program whose byte code is downloaded from a remote machine and is run in the browser’s Java Virtual Machine. Servlets A program whose byte code resides on a remote machine and is run at the request of an HTTP client (a browser). Three Types of Java programs A standalone Java application is run on a local machine computer Java object Java Virtual Machine An applet is an object downloaded (transferred) from a remote machine, then run on a local machine. an applet Java object Java Virtual Machine A servlet is an object that runs on a remote machine and interacts with a local program using a request-response protocol a servlet request response a process A sample Java application /******************************************************* * A sample of a simple Java application. 1/8/02 * M. Liu ******************************************************/ import java.io.*; class MyProgram{ public static void main(String[ ] args) throws IOException{ BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); String theName; System.out.println("What is your name?"); theName = keyboard.readLine( ); System.out.print("Hello " + theName ); System.out.println(" - welcome to CSC369.\n"); } // end main } //end class A Sample Java Applet /*************************************************** * A sample of a simple applet. * M. Liu 1/8/02 ***************************************************/ import java.applet.Applet; import java.awt.*; public class MyApplet extends Applet{ public void paint(Graphics g){ setBackground(Color.blue); Font Claude = new Font("Arial", Font.BOLD, 40); g.setFont(Claude); g.setColor(Color.yellow); g.drawString("Hello World!", 100, 100); } // end paint } //end class <!-- A web page which, when browsed, will run > <!-- the MyApplet applet> <!-- M. Liu 1/8/02> <title>SampleApplet</title> <hr> <applet code="MyApplet.class" width=500 height=500> </applet> <hr> <a href="Hello.java">The source.</a> A Sample Java Servlet /******************************************************* * A sample of a simple Java servlet. * M. Liu 1/8/02 ******************************************************/ import java.io.*; import java.text.*; import java.util.*; import javax.servlet.*; import javax.servlet.http.*; public class MyServlet extends HttpServlet { public void doGet (HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { PrintWriter out; String title = "MyServlet Output"; // set content type and other response header // fields first response.setContentType("text/html"); // then write the data of the response out = response.getWriter(); out.println("<HTML><HEAD><TITLE>"); out.println(title); out.println("</TITLE></HEAD><BODY>"); out.println("<H1>" + title + "</H1>"); out.println("<P>Hello World!"); out.println("</BODY></HTML>"); out.close(); } //end doGet } //end class Concurrent Processing On modern day operating systems, multiple processes appear to be executing concurrently on a machine by timesharing resources. Processes P1 P2 P3 P4 time Timesharing of a resource Concurrent processing within a process It is often useful for a process to have parallel threads of execution, each of which timeshare the system resources in much the same way as concurrent processes. A parent process may spawn child processes. A process may spawn child threads a process parent process main thread child thread 1 child thread 2 child processes Concurrent processing within a process Java threads The Java Virtual Machine allows an application to have multiple threads of execution running concurrently. Java provides a Thread class: public class Thread extends Object implements Runnable When a Java Virtual Machine starts up, there is usually a single thread (which typically calls the method named main of some designated class). The Java Virtual Machine continues to execute threads until either of the following occurs: The exit method of class Runtime has been called and the security manager has permitted the exit operation to take place. All threads have terminated, either by returning from the call to the run method or by throwing an exception that propagates beyond the run method. Using a subclass of the Thread class Using a class that implements the Runnable interface Create a class that is a subclass of the Thread class Declare a class to be a subclass of Thread. This subclass should override the run method of class Thread. An instance of the subclass can then be allocated and started: import SomeThread; public class RunThreads { public static void main (String[] args) { SomeThread p1 = new SomeThread(1); p1.start(); SomeThread p2 = new SomeThread(2); p2.start(); SomeThread p3 = new SomeThread(3); p3.start(); } }// end class RunThreads public class SomeThread extends Thread { int myID; SomeThread(int id) { this.myID = id; } public void run( ) { int i; for (i = 1; i < 11; i++) System.out.println ("Thread"+myID + ": " + i); } } //end class SomeThread Create a class that implements the Runnable interface The other way to create a thread is to declare a class that implements the Runnable interface. That class then implements the run method. An instance of the class can then be allocated, passed as an argument when creating Thread, and started. public class RunThreads2 { public static void main (String[] args) { Thread p1 = new Thread(new SomeThread2(1)); p1.start(); Thread p2 = new Thread(new SomeThread2(2)); p2.start(); Thread p3 = new Thread(new SomeThread2(3)); p3.start(); } } class SomeThread2 implements Runnable { int myID; SomeThread2(int id) { this.myID = id; } public void run() { int i; for (i = 1; i < 11; i++) System.out.println ("Thread"+myID + ": " + i); } } //end class SomeThread Thread-safe Programming When two threads independently access and update the same data object, such as a counter, as part of their code, the updating needs to be synchronized. (See next slide.) Because the threads are executed concurrently, it is possible for one of the updates to be overwritten by the other due to the sequencing of the two sets of machine instructions executed on behalf of the two threads. To protect against the possibility, a synchronized method can be used to provide mutual exclusion. Race Condition time fetch value in counter and load into a register fetch value in counter and load into a register increment value in register fetch value in counter and load into a register store value in register to counter increment value in register fetch value in counter and load into a register increment value in register increment value in register store value in register to counter store value in register to counter store value in register to counter This execution results in the value 2 in the counter This execution results in the value 1 in the counter instruction executed in concurrent process or thread 1 instruction executed in concurrent process or thread 2 Synchronized method in a thread class SomeThread3 implements Runnable { static int count=0; SomeThread3() { super(); } public void run() { update( ); } static public synchronized void update( ){ int myCount = count; myCount++; count = myCount; System.out.println("count="+count+ "; thread count=" + Thread.activeCount( )); } } Network Basics Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Network resources Network resources are resources available to the participants of a distributed computing community. Network resources include hardware such as computers and equipment, and software such as processes, email mailboxes, files, web documents. An important class of network resources is network services such as the World Wide Web and file transfer (FTP), which are provided by specific processes running on computers. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Network standards and protocols On public networks such as the Internet, it is necessary for a common set of rules to be specified for the exchange of data. Such rules, called protocols, specify such matters as the formatting and semantics of data, flow control, error correction. Software can share data over the network using network software which supports a common set of protocols. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Protocols In the context of communications, a protocol is a set of rules that must be observed by the participants. In communications involving computers, protocols must be formally defined and precisely implemented. For each protocol, there must be rules that specify the followings: How is the data exchanged encoded? How are events (sending , receiving) synchronized so that the participants can send and receive in a coordinated order? The specification of a protocol does not dictate how the rules are to be implemented. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 The network architecture Network hardware transfers electronic signals,which represent a bit stream, between two devices. Modern day network applications require an application programming interface (API) which masks the underlying complexities of data transmission. A layered network architecture allows the functionalities needed to mask the complexities to be provided incrementally, layer by layer. Actual implementation of the functionalities may not be clearly divided by layer. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Protocol layers in the ISO Open Systems Interconnection (OSI) model Message received Message sent Layers Application Presentation Session Transport Networ k Data link Physical Sender Communication medium Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Recipient OSI protocol summary Layer Application Presentation Session Transport Network Data link Physical Description Protocols that are designed to meet the communication requirements of specific applications, often defining the interface to a service. Protocols at this level transmit data in a network representation that is independent of the representations used in individual computers, which may differ. Encryption is also performed in this layer, if required. At this level reliability and adaptation are performed, such as detection of failures and automatic recovery. This is the lowest level at which messages (rather than packets) are handled. Messages are addressed to communication ports attached to processes, Protocols in this layer may be connection-oriented or connectionless. Transfers data packets between computers in a specific network. In a WAN or an internetwork this involves the generation of a route passing through routers. In a single LAN no routing is required. Responsible for transmission of packets between nodes that are directly connected by a physical link. In a WAN transmission is between pairs of routers or between routers and hosts. In a LAN it is between any pair of hosts. The circuits and hardware that drive the network. It transmits sequences of binary data by analogue signalling, using amplitude or frequency modulation of electrical signals (on cable circuits), light signals (on fibre optic circuits) or other electromagnetic signals (on radio and microwave circuits). Examples HTTP, FTP , SMTP, CORBA IIOP Secure Sockets (SSL),CORBA Data Rep. TCP, UDP IP, ATM virtual circuits Ethernet MAC, ATM cell transfer, PPP Ethernet base- band signalling, ISDN Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 3.12 TCP/IP layers The Transmission Control Protocol/Internet Protocol suite is a set of network protocols which supports a four-layer network architecture. The Internet layer implements the Internet Protocol, which provides the functionalities for allowing data to be transmitted between any two hosts on the Internet. The Transport layer delivers the transmitted data to a specific process running on an Internet host. The Application layer supports the programming interface used for building a program. Message Layers Application Messages (UDP) or Streams (TCP) Transport UDP or TCP packets Internet IP datagrams Network interface Network-specific frames Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Underlying network Figure 3.14 The programmer's conceptual view of a TCP/IP Internet Application Application TCP UDP IP Socket programming in UDP and TCP. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Identification of Network Resources One of the key challenges in distributed computing is the unique identification of resources available on the network, such as email mailboxes, and web documents. Addressing an Internet Host Addressing a process running on a host Email Addresses Addressing web contents: URL Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 The Internet Topology The internet consists of an hierarchy of networks, interconnected via a network backbone. Each network has a unique network address. Computers, or hosts, are connected to a network. Each host has a unique ID within its network. Each process running on a host is associated with zero or more ports. A port is a logical entity for data transmission. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Network Basics 1. Addressing an Internet Host Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 3.15 Internet address structure, showing field sizes in bits 7 Class A: Class B: 0 24 Network ID 1 0 Host ID 14 16 Network ID Host ID 21 Class C : 1 1 0 8 Network ID Host ID 28 Class D (multicast): 1 1 1 0 Multicast address 27 Class E (reserved): 1 1 1 1 0 unused Internet routing scheme developed in the 1970s. Class A addresses are the largest, but there are few of them. Class Cs are the smallest, but they are numerous. Classes D and E are also defined, but not used in normal operation. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 3.16 Decimal representation of Internet addresses octet 1 octet 2 Network ID 192 to 223 Range of addresses Host ID Class A: 1 to 126 0 to 255 (0, 127 Network ID reserved) Class B: 128 to 191 0 to 255 Class C: octet 3 Network ID 0 to 255 0 to 255 1.0.0.0 to 126.255.255.255 0 to 255 0 to 255 128.0.0.0 to 191.255.255.255 0 to 255 Host ID 1 to 254 0 to 255 Host ID 192.0.0.0 to 223.255.255.255 Multicast address Class D (multicast): 224 to 239 0 to 255 0 to 255 1 to 254 224.0.0.0 to 239.255.255.255 Class E (reserved): 240 to 255 0 to 255 0 to 255 1 to 254 240.0.0.0 to 255.255.255.255 Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 The Internet addressing scheme Subdividing the host portion of an Internet address: byte 0 class B address 10 byte 1 network address byte 2 byte 3 host portion A class A/C address space can also be similarly subdivided.. Which portion of the host address is used for the subnet identification is determined by a subnet mask. subnet address local host address Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Example Example: Suppose the dotted-decimal notation for a particular Internet address is129.65.24.50. The 32-bit binary expansion of the notation is as follows: 129.65.24.50 10000001 01000001 00011000 00110010 is a Class B Since the leading bit sequence is 10, the address address. Within the class, the network portion is identified by the remaining bits in the first two bytes, that is, 1000000101000001, and the host portion is the values in the last two bytes, or 0001100000110010. so that we would say that this particular address is at network 129.65 and then at host address 24.50 on that network. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 The Internet Address Scheme For human readability, Internet addresses are written in a dotted decimal notation: nnn.nnn.nnn.nnn, where each nnn group is a decimal value in the range of 0 through 255 # Internet host table (found in /etc/hosts file) 127.0.0.1 localhost 129.65.242.5 falcon.csc.calpoly.edu falcon loghost 129.65.241.9 falcon-srv.csc.calpoly.edu falcon-srv 129.65.242.4 hornet.csc.calpoly.edu hornet 129.65.241.8 hornet-srv.csc.calpoly.edu hornet-srv 129.65.54.9 onion.csc.calpoly.edu onion 129.65.241.3 hercules.csc.calpoly.edu hercules Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 The Domain Name System (DNS) For user friendliness, each Internet address is mapped to a symbolic name, using the DNS, in the format of: <computer-name>.<subdomain hierarchy>.<organization>.<sector name>{.<country code>} root e.g., www.csc.calpoly.edu.us top-level domain com edu gov net org mil country code in the U.S. Top-level domain name has to be applied for. Subdomain hierachy and names are assigned by the organization. organization ... ... subdomain host name 5/5/2017 94 The Domain Name System For network applications, a domain name must be mapped to its corresponding Internet address. Processes known as domain name system servers provide the mapping service, based on a distributed database of the mapping scheme. The mapping service is offered by thousands of DNS servers on the Internet, each responsible for a portion of the name space, called a zone. The servers that have access to the DNS information (zone file) for a zone is said to have authority for that zone. Top-level Domain Names .com: For commercial entities, which anyone, anywhere in the world, can register. .net : Originally designated for organizations directly involved in Internet operations. It is increasingly being used by businesses when the desired name under "com" is already registered by another organization. Today anyone can register a name in the Net domain. .org: For miscellaneous organizations, including non-profits. .edu: For four-year accredited institutions of higher learning. .gov: For US Federal Government entities .mil: For US military Country Codes :For individual countries based on the International Standards Organization. For example, ca for Canada, and jp for Japan. Example Another example: Given the address 224.0.0.1, one can expand it as follows: 224.0.0.1 1110000 00000000 00000000 00000001 The binary prefix of 1110 signifies that this is class D, or multicast, address. Data packets sent to this address should therefore be delivered to the multicast group 0000000000000000000000000001. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Network Basics 2. Addressing a process running on a host Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Addressing a process running on a host host A ... host B ... process port Each host has 65536 ports. The Internet Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Well Known Ports Each Internet host has 216 (65,535) logical ports. Each port is identified by a number between 1 and 65535, and can be allocated to a particular process. Port numbers beween 1 and 1023 are reserved for processes which provide well-known services such as finger, FTP, HTTP, and email. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Well Known Ports Assignment of some well-known ports Protocol Port Service echo 7 IPC testing daytime 13 provides the current date and time ftp 21 file transfer protocol telnet 23 remote, command-line terminal session smtp 25 simple mail transfer protocol time 37 provides a standard time finger 79 provides information about a user http 80 web server RMI Registry 1099 registry for Remote Method Invocation special web server 8080 web server which supports servlets, JSP, or ASP Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Choose a port For our programming exercises: when a port is needed, choose a random number above the well known ports: 1,024- 65,535. If you are providing a network service for the community, then arrange to have a port assigned to and reserved for your service. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Network Basics 3. Addressing a Web Document Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 The Uniform Resource Identifier (URI) Resources to be shared on a network need to be uniquely identifiable. On the Internet, a URI is a character string which allows a resource to be located. There are two types of URIs: URL (Uniform Resource Locator) points to a specific resource at a specific location URN (Uniform Resource Name) points to a specific resource at a nonspecific location. “A URN is like a person's name, while a URL is like their street address. The URN defines something's identity, while the URL provides a method for finding something. Essentially, "what" vs. "where".” (from Wiki) Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 URL A URL has the format of: protocol://host address[:port]/directory path/file name#section A sample URL: http://www.csc.calpoly.edu:8080/~mliu/CSC369/hw.html # hw1 section name file name host name protocol of server directory path port number of server process Other protocols that can appear in a URL are: file ftp gopher news telnet WAIS Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005