Download File-System Design and Implementation

OS Design and Structure
 OS vs. Kernel
 Operating System Services (Shared/Protected)
Services: CPU execution/scheduling, memory,
I/O, file management, etc.
Interface (GUI, commands)
system calls or library calls
System programs (invoking system calls)
Utilities, compilers, editors, GUI, Shell
Misc. tools
 Operating System Design and Implementations
(hardware, types of systems?)
User and System Goals
User: easy to use, reliable, fast
System: easy to implement/maintain, flexible, reliable, efficient
Policy and Mechanism Separation:
Max.flexibility
Mechanism: How, timer struct, priority field
Policy: What will be done, time quantum value, priority value/range
 Layered Approach (most common)
Unix
System programs
Kernel mode (CPU/memory/process/device management, protection, etc)
Windows (Chap 22.9 slide)
System programs
Kernel mode (executive and kernel layers)
Hardware abstraction layer to different hardware platforms
 Kernel Architecture
Monolithic Kernel (Unix, Windows…)
Single address space – direct call communications, fast
Problems - maintenance, flexibility, expandability, reliability, secure
Microkernel (Mach, Chorus, etc)
Basic abstraction (mechanisms) in kernel – reliable, secure, extensibility
Problems –slow due to IPC (user/kernel) separate address space
Dynamically loadable kernel modules - Hybrid
Microkernel approach and plugin – single address space
Concerns:
Kernel size due to module management
Complexity of kernel bootstrap
Choices: how often modules to be used, kernel built for many architectures?
 Virtual Machines
IBM mainframes, VM
Multiple execution environments (OS)with protection
OS development
VMware (as in the text and HW assignment)
Processes
 Process Concept
Program in execution (text, data, stack, heap)
PCB, context switch
Process states (ready, blocked, running)
 Process Scheduling
Job/Ready/Device Queues
Long/medium/short term scheduling
 Operations on Processes
Creation/terminate/abort
 IPC
Shared memory
Threads within a process – naturally shared
Separate processes – shared memory mapping
(shmget, shmat, shmdt, shmctl system calls)
Message passing (mailbox, ports)
Various message system calls (thru kernel)
(msgget, msgctl, msgsend, msgrcv system calls)
Threads
 Single and Multithreaded Processes
 Benefits:
Sharing, concurrency, Multiple CPUs, Scalability
 User/Kernel Threads
POSIX Pthreads, Java Threads
Windows, Linux threads (tasks), Fibers
 Multithreading Models
Manty-to-One, One-to-One, Many-to-Many
POSIX Pthreads, Java Threads
 Two-Level Model (similar to M:M)
Allow: user thread to be bounded to kernel thread
 Pthread and Java Thread Example
 Threading Issues:fork, exit, signal (process)
 Scheduler Activations
 Lightweight Process
CPU Scheduling
 CPU and Device Scheduling
Job queue, CPU Ready queue, Device Queue
 CPU Scheduling Basics
CPU utilization with multiprogramming
CPU and I/O cycle
Long, Medium, and short Term scheduling
 CPU Scheduler and Dispatcher
CPU scheduling occurs when a process state
Run to Wait, Terminate (non-preemptive)
Run to Ready, Wait to Ready (Preemptive)
Dispatcher (latency):
switching context,
Switch/restart User program
 Optimization Principle
CPU utilization, throughput, turnaround time,
Waiting time, response time
 CPU Short-term Scheduling Algorithms
FCFS
SJF (optimal): Minimum average waiting time
Shortest-Remaining-Time First
Priority
RR (Quantum Size? Context Switch overhead)
Multi-Level Queue (Feedback)
 OS Scheduling Algorithm Examples
Solaris Dispatch Table, Solaris Scheduling
Windows Scheduling (Priority Classes)
Linux Scheduling (Active and Expired Queue)
 I/O-bound favored over CPU-bound
 Priority-Inversion Problem and Priority
Inheritance Solution
Disk Scheduling
 Cylinders, Tracks, Sectors
 Seek Time, Rotational Delay, Transfer Time
 Disk Arm Movement (Seek Time) – Mechanical
to be minimized
 Elevator Algorithm
SCAN, C-SCAN, LOOK, C-LOOK,
 Algorithm Selection
SSTF is common and natural
Elevator algorithm – heavy load systems
Can be influenced by file-allocation method
Process (Thread) Synchronization
and Deadlocks
 WHY?
Performance
Communication and Parallelism
Coordination (mutual exclusion, cooperation)
 Mutual Exclusion
Race Condition (Ex: Producer/Consumer)
Instruction Level, Preemption
 Critical-Section Problem (common variables)
(Entry/Exit section, remainder section)
 Solution to CS Problem
Mutually Exclusive
Progress
(can’t delayed indefinitely if no other in CS)
Bounded Waiting
(bounded wait for other entering times)
 Some Software Solutions (difficulties)
.....Examples…
Peterson’s Solution (turn and flag)
(Two Processes? All Conditions Met?)
 Hardware/Architecture Solution
Uniprocessors – Disable interrupts? Scalable?
Atomic hardware instructions (TS, SWAP, CSWAP)
EX: Solution using TS, Bounded Wait (text)
 Critical-section using locks
Implemented using atomic instructions
Busy-waiting, Non-busy-waiting
 Semaphores
What is?
P/V,
Down/Up,
“Wait”/”Signal”
General Synchronization Tool
Mutual Exclusion and Process Cooperation
Implemented by lower level primitives
 Binary/Counting Semaphores
 Potential Deadlock/Starvation - Semaphores
 Bounded Buffer Problem – Producer/Consumer
Examples…
Binary/Counter semaphores used?
 Readers and Writers Problem
Examples, Readers favored?
Binary/Counter semaphores used?
 Condition Variables, Monitors
 Pthread Examples
Mutex locks: pthread_mutex_init/lock/unlock
Condition Variables: pthread_cond_init/signal/wait
Read-Write Locks: pthread_rwlock_init/rdlock/wrlock
 Solaris, Windows, Linux Examples
All Synchronization tools are used
Solaris: Adaptive mutex
Windows: Spinlocks (kernel),
Dispatcher objects (Executive)
Linux: Sequence Locks (later)
 DeadLocks
Four Necessary conditions (mutex, hold-wait, preemption, circle-wait)
Detection and recovery (cycle, kill/abort)
Prevention: violate any of the necessary conditions
Avoidance: safe state, Banker’s algorithm
Virtual Memory Implementation
 Memory Allocation and Relocation Problems
Fragmentation (external) and Dynamic Management
 Paging and Page Tables for Mapping
Memory Effective Access Time
TLB to speedup, hit ratio
Why multilevel page tables? (hierarchical)
Internal Fragmentation
 Page Faults and Handling
 Segmentation and paging
MULTICS (Silberschatz OSC – Slide 8.52)
Intel Pentium (Silberschatz OSC – Slide 8.54)
 Design Issues for paging Systems
Working Set Model, Pre-paging
TLB Reach (Translation Look-aside Buffer)
Replacement Algorithms (LRU, Second-Chance, Clock)
Program Structures
Local versus Global Allocation Policies
Page Size Selection (table space, internal fragmentation. I/O)
 Operating System Examples
Windows XP
clustering, working-set trimming
Solaris
Clock algorithm
Parameters and Scan Rate: lotsfree, desfree, minfree (swap)
File-System Design and Implementation
 Virtual File Systems (Layered File System)
 In-Memory File System Structure (Silberscharz: Slide 11.8)
Open and Read
 Design Criteria of Allocation Methods
Contiguous Allocation
Simple, Random Access, Fragmentation, Files can not grow
Linked Allocation (FAT, Silberschatz OSC: Slide 11.20)
Simple, No Random Access, No Fragmentation
Indexed Allocation
Random Access, No Fragmentation, Extra space form index table
Combined Scheme (Unix, Silberschatz OSC: Slide 11.27)
Performance
File is usually accessed sequentially and file is small ? Contiguous
File is usually accessed sequentially and file is large? Linked
File is usually accessed randomly and file is large? Indexed
 Implementation of Free-Space management
Bit Vector
Extra space for bit maps, easy to get contiguous file
Protect bit map on disk
Access in memory – Consistency Problem
Linked List
No waste of space, can not get contiguous space easily
Protect pointer to free list
 Efficiency and Performance of File-system Design
Efficiency Issues (disk space)
Disk allocation and directory algorithms
Types of data kept in file’s directory entry
Performance Issues
Even after the basic file-system algorithms have been selected,
We can still improve performance in several ways:
Disk Cache (memory for frequently used blocks)
Free-behind and Read-ahead to optimize sequential access
Memory as virtual disk or RAM disk