Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Intended Schedule 0 1 2 3 4 5 VIII. Input/Output 6 7 8 9 10 11 12 ✏ Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 1 Date 20.04. 27.04. 04.05. 11.05. 18.05. 25.05. 01.06. 08.06. 15.06. 22.06. 29.06. 06.07. 13.07. 20.07. 27.07. 12.10. Lecture Introduction to Operating Systems Systems Programming using C (File Subsystem) Systems Programming using C (Process Control) Process Scheduling Process Synchronization Inter Process Communication Pfingstmontag Deadlocks Memory Management Hand out Submission Course registration 1. Assignment 2. Assignment 1. Assignment 3. Assignment 2. Assignment 4. Assignment 3. Assignment 5. Assignment 4. Assignment 6. Assignment 5. Assignment 7. Assignment 6. Assignment 8. Assignment 7. Assignment 9. Assignment 8. Assignment 10. Assignment 9. Assignment 10. Assignment Input / Output Filesystems Special subject: XQuery your Filesystem Wrap up session First examination date Second examination date Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 2 I/O Hardware Control Unit I/O Hardware Different Peripherals Mass Storage Devices Processing Unit Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Input / Output (I/O) 3 Main Memory Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Glatz ©2005 4 I/O Hardware A Typical PC Bus Structure Incredible variety of I/O devices Common concepts Port Bus (daisy chain or shared direct access) Controller (host adapter) I/O instructions control devices Devices have addresses, used by Direct I/O instructions Memory-mapped I/O Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 5 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 6 Device I/O Port Locations on PCs (partial) CPU Device Interaction Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 7 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 8 CPU CPU↔Device Interaction by Polling Device Interaction CPU determines state of device by repeatedly checking busy flag. • Principal Choices • Programmed I/O (Polling) • CPU waits (busy wait) for peripheral ... prepares data in data-out register (for write operation) ... sets command register to “write” ... sets command-ready bit Once controller sees command-ready bit • Interrupt-Driven I/O • Peripheral signals „ready“ to CPU by (hardware) interrupt • Direct Memory Access (DMA) ... sets busy bit ... sees command=write and reads data-in register ... performs write-I/O ... clears command-ready, error & busy bits Busy bit • needs special DMA controller to execute I/O independently from CPU Error bit • DMA controller signals „ready“ to CPU via (hardware) interrupt Command register Data-in, data-out registers ja NB: Sind mehrere Zeich senden, so wiederholt si Ablauf für jedes weitere bereit? nein Ende kap61-3_Ein-Ausgabe.fm Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 9 Silberschatz, Galvin and Gagne ©2005 10 Interrupt-Driven I/O Cycle CPU↔Device Interaction by Interrupts Betriebssysteme CPU Interrupt-request line (HW wire) Checked by CPU after each instruction 1 Zeichen senden 1 Zeichen senden Busy-wait cycle to wait for I/O from device Example © Glatz: Serial I/F Statusregister lesen Command-ready bit Triggered by I/O devices Beispiel: serielle Schnittstelle 6.2.1 Programmgesteuerte E/A Interface • Steal cycles from CPU to transfer data from/to main memory w/o CPU interaction Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 B Erkennen einer Unterbrechungsanforderung Zwei grundsätzliche Fälle: (1) Externes Hardware-Signal (2) Unterbrechung als direkte Folge der Instruktionsausführung ■ CPU-instruction cycle © Glatz ■ Interrupt handler receives interrupts Initialisierung loop Maskable to ignore or delay some interrupts FETCH (Instruktion holen) EXECUTE (Instruktion ausführen) Interrupt vector to dispatch interrupt to correct handler Based on priorities Some nonmaskable Using interrupt vector (and chaining) INTERRUPT CHECK kap61-3_Ein-Ausgabe.fm Folie 9 Interrupt mechanism can also used for Exceptions Virtual memory paging System calls, ... Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 11 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 12 Direct Memory Access Intel Pentium Processor Event-Vector Table Used to avoid programmed I/O (PIO, byte-wise data transfer between memory and devices) for large data movement Requires DMA controller Bypasses CPU to transfer data directly between I/O device and memory During DMA access to memory, CPU cannot access memory (“cycle stealing”) Signals completion by a CPU interrupt DMA may be implemented into real or into virtual memory Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 13 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 14 Betriebssysteme 6.2.3 Ein-/Ausgabe mittels DMA (Direct Memory Access) DMA: More details Six Step Process to Perform DMA Transfer DMA-Betriebsarten Cycle Stealing in Single vs. BlockBlocktransfer Transfer Einzeltransfer DMA-Anforderung DMA-Anforderung DMAZyklen DMAZyklen Betriebssysteme ProzessorPrinzipaufbau eines DMA-Kontrollers ProzessorZyklen Beispiel für indirekten Transfer: Zeit Zyklen Zeit DMA: Direct vs. Indirect Data Transfer kap61-3_Ein-Ausgabe.fm (A) Programmierung DMA-Kontr. Hauptspeicher DMAKontroller Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 15 CPU Peripheriegerät (B) Direkter Transfer Hauptspeicher DMAKontroller CPU Peripheriegerät Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 kap61-3_Ein-Ausgabe.fm Folie 24 (C) Indirekter Transfer Hauptspeicher DMAKontroller CPU Peripheriegerät Glatz ©2005 Folie 27 16 Application I/O Interface I/O system calls encapsulate device behaviors in generic classes Device-driver layer hides differences among I/O controllers from kernel Application I/O-Interface Devices vary in many dimensions Character-stream or block Sequential or random-access Sharable or dedicated Speed of operation read-write, read only, or write only Treiberhierarchie Betriebssystemteil © Glatz Anwenderprozess Geräteverwaltung (I/O manager) logischer Treiber Software Klassentreiber (class driver) physischer Treiber Gerätetreiber (device/port driver) Peripheriekontroller (interface HW) Hardware Gerät (device) Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 17 A Kernel I/O Structure Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 18 Characteristics of I/O Devices Silberschatz, Galvin and Gagne ©2005 19 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 20 Block and Character Devices Network Devices Block devices include disk drives Commands include read, write, seek Raw I/O or file-system access Memory-mapped file access possible Varying enough from block and character to have own interface Unix and Windows NT/9x/2000 include socket interface Separates network protocol from network operation Includes select functionality Character devices include keyboards, mice, serial ports Commands include get, put Libraries layered on top allow line editing Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Approaches vary widely (pipes, FIFOs, streams, queues, mailboxes) Silberschatz, Galvin and Gagne ©2005 21 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Clocks and Timers Silberschatz, Galvin and Gagne ©2005 22 Blocking and Nonblocking I/O Blocking - process suspended until I/O completed Provide current time, elapsed time, timer Programmable interval timer used for timings, periodic interrupts ioctl (on UNIX) covers odd aspects of I/O such as clocks and timers Easy to use and understand Insufficient for some needs Nonblocking - I/O call returns as much as available User interface, data copy (buffered I/O) Implemented via multi-threading Returns quickly with count of bytes read or written Asynchronous - process runs while I/O executes Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 23 Difficult to use I/O subsystem signals process when I/O completed Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 24 Two I/O Methods Kernel I/O Subsystem Scheduling — which I/O request to serve next? Some I/O request ordering via per-device queue Some OSs try fairness Buffering — store data in memory while transferring between devices To cope with device speed mismatch To cope with device transfer size mismatch To maintain “copy semantics” Caching — fast memory holding copy of data Always just a copy Key to performance Spooling — hold output for a device Synchronous Asynchronous If device can serve only one request at a time, e.g., Printing Device reservation — provides exclusive access to a device Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 25 System calls for allocation and deallocation Watch out for deadlock! Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 I/O a major factor in system performance: Determine device holding file Demands CPU to execute device driver, kernel I/O code Translate name to device representation Context switches due to interrupts Physically read data from disk into buffer Data copying Make data available to requesting process Network traffic especially stressful Return control to process Software Anwenderprozess 1 Anwenderprozess 2 Anwenderprozess n © Glatz Improving Performance: Hardware Treiber A Kontroller A Gerät A Treiber B Kontroller B Gerät B Reduce number of context switches Reduce data copying in memory between application and device Reduce frequency of interrupts by using large transfers, smart controllers, polling Use DMA to increase concurrency (offload CPU) Move processing into device controllers (offload CPU & bus) Balance CPU, memory, bus, and I/O performance for highest throughput Geräteverwaltung (I/O manager) Treiber X 26 Performance Transforming I/O Requests to Hardware Operations Consider reading a file from disk for a process: Silberschatz, Galvin and Gagne ©2005 Kontroller X Gerät X Treiber-Schnittstelle (driver interface) Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Overload Silberschatz, Galvin and Gagne ©2005 27 in one of these will leave others idle! Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 28 Mass Storage Overview • Secondary Storage • Magnetic disks provide bulk of secondary storage of modern computers • Disks can be removable • Drives attached to computer via I/O bus (e.h., EIDE,ATA, SATA, USB, FC, SCSI, ...) Mass Storage • Host controller (in computer) talks to disk controller (in device) via this bus • Tertiary Storage • Low cost is defining characteristic • Typically uses removable media • Considered „off-line“ storage; robot-machinery can turn it into“near-line“ • Magnetic tapes, floppy, CD, DVD, ... Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 29 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Moving-head Disk Machanism 30 Disk Structure Disk drives are addressed as large 1-dimensional arrays of logical blocks, where the logical block is the smallest unit of transfer. The 1-dimensional array of logical blocks is mapped into the sectors of the disk sequentially. Sector 0 is the first sector of the first track on the outermost cylinder. Mapping proceeds in order through that track, then the rest of the tracks in that cylinder, and then through the rest of the cylinders from outermost to innermost. © Glatz Seite 0 Sektor 8 7 23 Sektor 7 Seite 1 Sektor 1 Zylinder 0 0 Zylinder 1 16 Zylinder 2 32 6 22 5 33 17 1 18 2 21 Sektor 6 20 Disk access time = queue waiting + seek + rotational delay + transfer 4 Sektor 5 Sektor 2 Sektor 3 15 31 8 24 40 Sektor 7 14 30 Sektor 6 13 41 10 Sektor 3 28 12 Sektor 5 Sektor 2 25 9 26 29 19 3 Sektor 4 Sektor 1 Zylinder 0 Zylinder 1 Zylinder 2 Sektor 8 27 11 Sektor 4 Nummerierung: Zylinder 0.. (zero based), Seiten 0.. (zero based), Sektoren 1.. (one based) Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 31 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 32 Disk Attachment Network-Attached Storage Network-attached storage (NAS) is storage made available over a Host-attached storage accessed through I/O ports talking to I/O busses network rather than over a local connection (such as a bus) SCSI itself is a bus, up to 16 devices on one cable, SCSI initiator NFS and CIFS are common protocols requests operation and SCSI targets perform tasks Implemented via remote procedure calls (RPCs) between host and Each target can have up to 8 logical units (disks attached to device controller storage New iSCSI protocol uses IP network to carry the SCSI protocol FC is high-speed serial architecture Can be switched fabric with 24-bit address space – the basis of storage area networks (SANs) in which many hosts attach to many storage units Can be arbitrated loop (FC-AL) of 126 devices Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 33 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Storage Area Network Silberschatz, Galvin and Gagne ©2005 34 Disk Management Common in large storage environments (and becoming more common) Low-level formatting, or physical formatting — Dividing a disk into sectors that the disk controller can read and write. Multiple hosts attached to multiple storage arrays - flexible To use a disk to hold files, the operating system still needs to record its own data structures on the disk. Partition the disk into one or more groups of cylinders. Logical formatting or “making a file system”. Boot block initializes system. The bootstrap is stored in ROM. Bootstrap loader program. Methods such as sector sparing used to handle bad blocks. Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 35 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 36 Disk Scheduling The operating system is responsible for using hardware efficiently — for the disk drives, this means having a fast access time and disk bandwidth. Access time has two major components Disk Scheduling Seek time is the time for the disk are to move the heads to the cylinder containing the desired sector. Rotational latency is the additional time waiting for the disk to rotate the desired sector to the disk head. Minimize seek time Seek time ≈ seek distance Disk bandwidth is the total number of bytes transferred, divided by the total time between the first request for service and the completion of the last transfer. Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 37 Disk Scheduling (Cont.) Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 38 FCFS — First Come First Served Illustration shows total head movement of 640 cylinders. Several algorithms exist to schedule the servicing of disk I/O requests. We illustrate them with a request queue (0-199). 98, 183, 37, 122, 14, 124, 65, 67 Head pointer 53 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 39 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 40 SSTF — Shortest Seek Time First SCAN (aka. Elevator Algorithm) Selects the request with the minimum seek time from the current head The disk arm starts at one end of the disk, and moves toward the other position. end, servicing requests until it gets to the other end of the disk, where the head movement is reversed and servicing continues. SSTF scheduling is a form of SJF scheduling; may cause starvation of Illustration shows total head movement of 208 cylinders. some requests. Illustration shows total head movement of 236 cylinders. Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 41 C-SCAN Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 42 C-LOOK Provides a more uniform wait time than SCAN. Somewhat smarter version of C-SCAN The head moves from one end of the disk to the other, servicing requests Arm only goes as far as the last request in each direction, then reverses as it goes. When it reaches the other end, however, it immediately returns to the beginning of the disk, without servicing any requests on the return trip. direction immediately, without first going all the way to the end of the disk. Treats the cylinders as a circular list that wraps around from the last cylinder to the first one. (→“Circular Scan”) Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 43 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 44 Selecting a Disk-Scheduling Algorithm SSTF is common and has a natural appeal. SCAN and C-SCAN perform better for systems that place a heavy load on the disk. Performance depends on the number and types of requests. RAID-Systems (Disk Arrays) Requests for disk service can be influenced by the file-allocation method. The disk-scheduling algorithm should be written as a separate module of the operating system, allowing it to be replaced with a different algorithm, if necessary. Either SSTF or LOOK is a reasonable choice for the default algorithm. Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 45 46 RAID Levels RAID-Systems — Basic Ideas • Idea came up in the 1980s Large variety of exact definitions of the • „Winchester disk drives“: smaller, cheaper, less power & space/ volume consumption, no air conditioning, ... than state-of-the-art mainframe disk drives • But: Many disk drives largely reduced MTTF (by factor 1/n, assuming independence) levels, plus additional levels. Various choices as to how and where to implement RAID functionality (OS, drive, controller, SAN interconnect; HW/SW;...) Additional functionality can be integrated, such as replication and snapshots. • Need to use redundancy to improve/regain reliability • Many disks can also provide much better performance • Can also avoid restrictions of file systems: Volume mgmt. software offers a lot of additional functionality. Hot spare disks can be reserved as • files must fit onto single disk → „logical volumes“ spanning disks „immediate“ replacement. • Striping: use a group of disks as a single storage unit • Mirroring/Shadowing or Parity/ECCs improve reliability by storing redundant data Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 47 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 48 Hierarchical Storage Management (HSM) A hierarchical storage system extends the storage hierarchy beyond primary memory and secondary storage to incorporate tertiary storage — usually implemented as a jukebox of tapes or removable disks. Usually incorporate tertiary storage by extending the file system. More Aspects Automatically migrate data up and down the storage hierarchy Small and frequently used files remain on disk. Large, old, inactive files are archived to the jukebox. HSM is usually found in supercomputing centers and other large installations that have enormous volumes of data. Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 49 Cost Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 50 Price per Megabyte of DRAM (1981 to 2004) Main memory is much more expensive than disk storage The cost per megabyte of hard disk storage is competitive with magnetic tape if only one tape is used per drive. The cheapest tape drives and the cheapest disk drives have had about the same storage capacity over the years. Tertiary storage gives a cost savings only when the number of cartridges is considerably larger than the number of drives. Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 51 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 52 Price per Megabyte of Magnetic Hard Disk (1981 to 2004) Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 53 Price per Megabyte of a Tape Drive (1984-2000) Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 Silberschatz, Galvin and Gagne ©2005 54 Intended Schedule 0 1 2 3 4 5 6 7 8 9 10 11 12 ✏ Date 20.04. 27.04. 04.05. 11.05. 18.05. 25.05. 01.06. 08.06. 15.06. 22.06. 29.06. 06.07. 13.07. 20.07. 27.07. 12.10. Lecture Introduction to Operating Systems Systems Programming using C (File Subsystem) Systems Programming using C (Process Control) Process Scheduling Process Synchronization Inter Process Communication Pfingstmontag Deadlocks Memory Management Hand out Submission Course registration 1. Assignment 2. Assignment 1. Assignment 3. Assignment 2. Assignment 4. Assignment 3. Assignment 5. Assignment 4. Assignment 6. Assignment 5. Assignment 7. Assignment 6. Assignment 8. Assignment 7. Assignment 9. Assignment 8. Assignment 10. Assignment 9. Assignment 10. Assignment IX. File Systems Input / Output Filesystems Special subject: XQuery your Filesystem Wrap up session First examination date Second examination date Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 55 Operating Systems • Prof. Dr. Marc H. Scholl • DBIS • U KN • Summer Term 2009 56