Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Deep packet inspection wikipedia , lookup
Network tap wikipedia , lookup
Internet protocol suite wikipedia , lookup
Cracking of wireless networks wikipedia , lookup
Airborne Networking wikipedia , lookup
Recursive InterNetwork Architecture (RINA) wikipedia , lookup
Distributed firewall wikipedia , lookup
Service-oriented architecture implementation framework wikipedia , lookup
Programming systems for distributed applications Seif Haridi KTH/SICS 1 What are the properties of global distributed systems? • A distributed system is set of processes, linked by a network • No global information, no global time • Unpredictable communication delays • Concurrency and nondeterminism • Large probability of localized faults • Easy access by unauthorized users 2 Additional Properties of the Internet • A global network that is partitioned into several protection domains (Firewalls) • Private sub networks with multiple reassignment of IP addresses across networks • Dynamic reassignment of IP addresses -- ISP’s reuse a pool of IP addresses among customers 3 Middleware for distributed global applications • The middleware abstracts the complexity of the underlying system and provides the services of the OS to the application programmer. • The middleware should support the distribution structure, scalability, openness, failure handling and security issues. • Provides transparency (network and location) as much as possible or as desirable. The applications The middleware The operating system The Network 4 The middleware as a programming system for distributed applications • Design a programming system from the start that is suitable for distributed applications (Oz language and its system Mozart) • Extend an existing programming system with libraries to support distributed computing (JAVA) • Provide an distribution layer that is language independent (CORBA), this layer might be needed anyway for communication with foreign software. 5 Programming system for distributed applications • The programming language by design provides abstractions necessary for distributed applications: – Concurrency and various communication abstraction – Mobility of code (or more generally closures) and other entities – Mechanisms for security at the language level -- the programming language by construction support all the concept needed for allowing arbitrary security levels (no holes) – Notion of sited resources, how to plug and unplug resources – Notion of a distributed/mobile component (for mobility of applications) – Dynamic connectivity, transfer of entities and modification of various applications at runtime – Abstraction of the network transport media 6 Programming system for distributed applications • The programming system (runtime system) by design provides mechanisms to support: – Network transparency – Well defined and extended distributed behavior for all language entities -- part of network awareness. – Mechanisms for guaranteeing security on untrusted sites (fake implementations) – Mechanism for limiting resource (memory and processor time) consumption by foreign computations at runtime – Network layer that supports location transparency (mobile applications) (multiple) IP independent addressing – Configurable and scalable network layer (multiple protocols, TCP, TTCP, Reliable UDP, …) – Dynamic connectivity, fault/ connectivity detection – Firewall enabled 7 The programming system (runtime system) Network Transport Level Raw byte sequence Network layer Structured Messages Memory management layer Messages between distributed nodes Protocol layer Messages between threads and nodes Extended engine } } } Shared computation space Distributed garbage collection Distribution protocols Local execution is here 8 The issues in distributed programming Distribution Security Openness Functionality Resource Control Fault tolerance Scalability 9 The issues in distributed programming Distribution Security Openness Functionality Resource Control Fault tolerance Scalability Part of problem Interaction 10 The issues in distributed programming Distribution Distribution Security Security Functionality Openness Openness Functionality Resource Control Resource Control Fault tolerance Scalability Fault tolerance Scalability Part of problem Interaction 11 Network transparency • It means: – If you develop an application on a single machine, you can distribute the entities to different sites without changing the logical behavior (functionality/semantics) of the application – If you connect to independent applications together they will logically behave as if they were running on a single machine. • It does not means that you loose control over your applications • However network transparency breaks when faults (network partitioning) occurs. Therefore awareness is also needed. • The role of fault detection is to reflect abstracted failure to allow the programmer to develop fault tolerance mechanisms. 12 Application Logic Graphic Entities BS Board Server DB WM CM Display Broadcaster WM CM CM - Client Manager WM- Window Manager 13 Transparent distribution Graphic Entities BS Board Server DB WM CM Display Broadcaster WM CM CM - Client Manager WM- Window Manager 14 Network-transparency and scalability Distributed servers Services Connector User User User 15 Programming systems for distributed applications • The programming language by design provides abstractions necessary for distributed applications: – Concurrency and various communication abstractions – Mobility of code (or more generally closures) and other entities – Mechanisms for security at the language level -- the programming language by construction support all the concept needed for allowing arbitrary security levels (no holes) – Notion of sited resources, how to plug and unplug resources – Notion of a distributed/mobile component (for mobility of applications) – Dynamic connectivity, transfer of entities and modification of various applications at runtime – Abstraction of the network transport media 16 Language Entities Concurrency and asynchronous communication abstractions are important to hide latency in distributed applications Cheap light weight threads are necessary combined with dataflow synchronization to overlap computation and communication ports, threads single-assignment variable cells objects T1 : m(X) Object T0: …X… X=Value functors, components records, procedures, classes T0 time 17 Language Entities Concurrency and asynchronous communication abstractions are ports, threads important to hide latency in distributed applications single-assignment variable Just use Ports for asynchronous communication, no thread creation cells objects T0 : send m(X) to P T0: …X… receive m invoke Object X=Value functors, components records, procedures, classes T0 time 18 JAVA’s remote method invocation • JAVA’s basic mechanism for communication is extending the sequential method invocation into invocation on remote objects • It is possible to create a thread, but it is both expensive and synchronization with the main thread is awkard • A future abstraction has to programmed explicitly combined with the call-back in a separate thread T0 : invoke m(X) to object Object T0: …X… X=Value T0 time 19 Language Entities Mobility of code (or more generally closures) and other entities Every entity in the language is a runtime value (access by reference not by a string) In particular procedures and classes are values that can be sent in a message, or given as a parameter to remote invocation No name collision at remote site C variable send m(.) to O ports, threads single-assignment variable cells objects functors, components records, procedures, classes 20 Distributed Lexical Scoping/Mobility Site 2 C Site 1 C variable variable E1 E1 E2 E2 send m(.) to site 2 E1 E1 E2 E2 21 Advantages of distributed lexical scoping • Distribution transparency under procedure/class/object mobility – programs can be tested and understood on one site, and behave the same when distributed on different sites • Security, given correct implementation: – a procedure transferred to a site will not accidentally acquire unauthorized access to entities residing on the site. – Can easily implement capability based security, allowing subset of possible operations on an entity • Example a procedure referring to a file object, only allowing reads 22 Programming system for distributed applications • The programming system (runtime system) by design provides mechanisms to support: – Network transparency – Well defined and extended distributed behavior for all language entities -- part of network awareness. – Mechanisms for guaranteeing security on untrusted sites (fake implementations) – Mechanism for limiting resource (memory and processor time) consumption by foreign computations at runtime – Network layer that supports location transparency (mobile applications) (multiple) IP independent addressing – Configurable and scalable network layer (multiple protocols, TCP, TTCP, Reliable UDP, …) – Dynamic connectivity, fault/ connectivity detection – Firewall enabled 23 Distributed behavior Stateless Replication Single assignment Eager elimination Stateful Localization Private Eager record, procedure, class Lazy object-record single assign. variable Mobile cell, object-state Stationary port, thread resources 24 System support • Object could not be copied by value as in JAVA because this will loose transperancy • Object could be either stationary, or mobile with a consistency protocol that is fault tolerant. Mozart support both. 25 Resource Security with distributed components New Module Manager requires Open.file Imported to site Open I do not trust you You can only read/write in TEMP 26 Conclusions • I have shown that making a programming language distributed (Internet ready!) is not just a taking a centralized language and extending it with a distribution layer. • The design should take distribution into account from scratch. • I have just touched upon the subject, the topic I did not cover are just as important. • Look at the mozart demo, where you would also see the distributed component architecture where new functionality can be added on the fly and make available to all participants in a Mozart instant messenger (ICQ like) Internet service. 27