Download Introduction

Distributed Processing
Chapter 1 : Introduction
1
Pusan National University
STEM
Problem


There are n nodes, each of which has a value. A node
wants to know the maximum value among the n nodes.
Centralized Approach:



A server maintains the values of n nodes and each node
reports its value to the server.
Then the query node sends a message to ask the maximum
value to the server, which will answer to the query.
Distributed Approach:


Each node communicates with its 6 nearest neighbor nodes to
inform its value.
Then the query node eventually finds the maximum value by
exchanging information with its neighbor nodes.
2
Pusan National University
STEM
Discussion

Question 1: Find the algorithm for distributed approach.

Question 2: Compare the performance


In terms of the number of communications
Question 3: Make a comparison table for the two
approaches
3
Pusan National University
STEM
Definition of a Distributed System

Distributed system :
1) A collection of (scalability)
2) independent computers that (heterogeneity)
3) appears to its users as a single coherent system
(transparency)

Distributed System versus Parallel System


Separated Operating System vs. Single Operating System
Message Passing vs. Shared Memory
4
Pusan National University
STEM
Why Distributed System ?


Performance
Incremental Growth (Scalability)



Fault Tolerance



1 single mainframe of price W
N small machines of price W/N
1 single mainframe : critical weak point
Failure of a machine : replacement by other machines
Geographical Distribution and Availability

Flexible configuration


e.g. 1 Disk server, 3 Computing servers, 1 Graphic server, etc.
Geographical availability
5
Pusan National University
STEM
Distributed System
- Scalibility and Heterogeneity
1.1
A distributed system organized as middleware.
 Heterogeneity and Scalability
6
Pusan National University
STEM
Distributed System - Transparency
Different forms of transparency in a distributed system.
Transparency
Description
Access
Hide differences in data representation and how a resource is accessed
Location
Hide where a resource is located
Migration
Hide that a resource may move to another location
Relocation
Hide that a resource may be moved to another location while in use
Replication
Hide that a resource may be shared by several competitive users
Concurrency
Hide that a resource may be shared by several competitive users
Failure
Hide the failure and recovery of a resource
Persistence
Hide whether a (software) resource is in memory or on disk
7
Pusan National University
STEM
Distributed System : Heterogeneity
Application Program or Client
Driver for A
Server A
Driver for B
Driver for C
Server B
Server C
Client has to be provided with one different driver for each server
8
Pusan National University
STEM
Distributed System : Heterogeneity and
Object-Oriented Approach
Application Program or Client
Predefined interface
Server A
Server B
Server C
Wrapping with predefined interface
Encapsulation : Object-Oriented Approach
9
Pusan National University
STEM
Hardware Concepts : Multiprocessor
1.6
10
Pusan National University
STEM
Multiprocessors (1)

A bus-based multiprocessor.
1.7
11
Pusan National University
STEM
Multiprocessors (2)
(a) A crossbar switch
(b) An omega switching network
1.8
12
Pusan National University
STEM
Homogeneous Multicomputer Systems
(a) Grid
(b) Hypercube
13
Pusan National University
STEM
Software Concepts
System
Description
Main Goal
DOS
Tightly-coupled operating system for multiprocessors and homogeneous multicomputers
Hide and manage
hardware resources
NOS
Loosely-coupled operating system for heterogeneous
multicomputers (LAN and WAN)
Offer local services
to remote clients
Middleware
Additional layer atop of NOS implementing generalpurpose services
Provide distribution
transparency

An overview of



DOS (Distributed Operating Systems)
NOS (Network Operating Systems)
Middleware
14
Pusan National University
STEM
Issues in System Design






Transparency
Flexibility
Reliability
Performance
Scalability
Interoperability
15
Pusan National University
STEM
Transparency

Hiding physical details about








Location
Migration
Duplication
Relocation
Concurrency
Parallelism
Location
Access
16
Pusan National University
STEM
Flexibility

Should be easy to modify functionality and architecture

To provide with Configurability, Avalability and
Autonomy

Micro-Kernel vs. Monolithic Kernel


Monolithic Kernel : Provides all functionalities of OS.
example. UNIX
Micro-Kernel


Minimal subset of OS + what users want
Example

Kernel Watch
17
Pusan National University
STEM
Reliability

Important Goal of Distributed System




Reliability
Security
Fault-Tolerance
Failure Probability P


Should be P = P1·P2·P3 … ·Pn
But often P = P1+ P2+ P3 … + Pn in reality
18
Pusan National University
STEM
Performance and Scalability


Improve performance by parallelism
Throughput T


Ideally should be T = T·n when n is the number of sites
In reality T < T·n

Due to some Bottleneck
Throughput
Number
of sites
??
19
Pusan National University
STEM
Granularity of Parallelism

Unit of Task


Fine-Granularity




Fine-Granularity vs. Coarse Granularity
Large number of small tasks
Need a large amount of inter-task communication
Not good for distributed system (good for Parallel system)
Coarse-Granularity



Small number of big tasks
Only small amount of inter-task communication
Good for distributed system
20
Pusan National University
STEM
Interoperability

Easy to collaborate with other systems in run-time


Compatibility, Portability
How to achieve Interoperability


Well-Defined API set
Standardization
21
Pusan National University
STEM
Hardware Concepts : Multiprocessor
1.6
22
Pusan National University
STEM
Multiprocessors (1)

A bus-based multiprocessor.
1.7
23
Pusan National University
STEM
Multiprocessors (2)
(a) A crossbar switch
(b) An omega switching network
1.8
24
Pusan National University
STEM
Homogeneous Multicomputer Systems
(a) Grid
(b) Hypercube
25