* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Ch. 14 : UNIX Operating System with Linux
Survey
Document related concepts
Copland (operating system) wikipedia , lookup
MTS system architecture wikipedia , lookup
Distributed operating system wikipedia , lookup
Commodore DOS wikipedia , lookup
Library (computing) wikipedia , lookup
Security-focused operating system wikipedia , lookup
Process management (computing) wikipedia , lookup
Berkeley Software Distribution wikipedia , lookup
Plan 9 from Bell Labs wikipedia , lookup
Burroughs MCP wikipedia , lookup
Transcript
Ch. 14 : UNIX Operating System with Linux • History – UNIX System V – Linux • Design Goals • Memory Management • Processor Management • Device Management • File Management • Network Management • Security Management • User Interface Understanding Operating Systems 1 UNIX Runs On All Sizes of Computers Advantages: 1. Portable from large to small systems because written in C. 2. Powerful utilities are brief, single-operation commands that can be combined in single command line. 3. Device independent -- can be configured to run any device because it includes device drivers as part of OS. Understanding Operating Systems Disadvantages 1. No single standardized version of OS. – AT&T’s UNIX V standard (late 1980s). 2. Commands are so brief that novice users find it unfriendly 2 Year Internal release 1971 Version 1 1972 1973 1973 1975 1979 Version 2 Version 3 Version 4 Version 6 Version 7 1980 1981 1983 Release 3.0 Release 4.0 1984 1989 1991 1993? 199x? Understanding Operating Systems External release Features Based on MULTICS; introduced shell concept, written in assembly language Added pipes and filters Kernel and I/O, first version written in C UNIX V4 UNIX V6 UNIX V7 UNIX System III UNIX System V Release 1 UNIX System V Release 2 UNIX System V Release 4 Novell’s Unix Ware Solaris ? First version to become commercially available More powerful shell added: string variables, structured programming, trap handling First version used in 16-bit microcomputers First version available for a mainframe Added small general-purpose programs Added features from Berkeley version: shared memory, more commands, vi editor, termcap database, flex file names Open Software Foundation founded Combined features from BSD, SunOS and Xenix Novell’s version of System V release 4 designed to work with NetWare Sun’s version designed to run to Sun workstations, derived from AT&T’s UNIX System V, Release 4 3 UNIX System V • AT&T entered computer industry by offering line of PCs powered by UNIX System V. • 1990 -- +24 versions of UNIX (AT&T’s UNIX System V, A/UX, Ultrix, Microsoft’s XENIX & University of California at Berkeley’s UNIX Versions 4.1 bsd, 4.2 bsd, and 4.3 bsd. • 1991 -- AT&T’s Unix System Labs developed System V release 4 which incorporated almost all features found in System V release 3 plus BSD release 4.3, SunOS, and Xenix. – Other companies established Open Software Foundation. • 1993 -- Berkeley released 4.4BSD based on AT&T’s UNIX, requiring customers to obtain licenses from AT&T to use it. • Novell acquired UNIX from AT&T & released own version, UNIX Ware. Designed to interact with Novell’s NetWare system. Understanding Operating Systems 4 New Releases of UNIX • Although newer commands become more difficult to use, they can be adapted to new situations with relative ease. • Offer full support for LANs. • Comply with international OS standards. • System security has been greatly improved & meets many of the U.S. government security requirements. • Most UNIX systems feature Common Desktop Environment (CDE). – Uniform GUI designed to give UNIX systems a standard look. • Many “standards,” each supported by coalition of vendors. Understanding Operating Systems 5 Linux • Developed in Finland, by Linus Torvalds in 1991. • Linux was based on UNIX, multi-platform OS. • Brought speed, efficiency & flexibility of UNIX to a PC environment. • Open-source program -- source code is freely available to anyone for improvement. – Under constant development by uncounted people around world. • Linux is not UNIX, it’s clone. • Linux’s ncurses (new curses) -- screen handling library that improved on curses libraries featured in UNIX Release V. • First used by programmers at universities and research environments, but becoming widely adopted in commercial & industrial markets. Understanding Operating Systems 6 UNIX Design Goals “OS by programmers for programmers” 1. Develop OS to support software development. – Utilities for which programmers typically need to write code. – Each utility was designed for simplicity. – Designed to be used in combination with each other. 2. Keep algorithms as simple as possible (without becoming rudimentary). – UNIX can be mastered by experienced programmers in weeks. 3. Make OS & any application software developed for it, portable from machine to machine. – Reduces conversion costs & doesn’t cause application packages to become obsolete with every change in hardware. Understanding Operating Systems 7 Linux Design Goals Function Multiple processes & multiple processors Multiple platforms Multiple users Inter-process communications Terminal management Peripheral devices Buffer cache Demand paging memory management Dynamic and shared libraries Disk partitions Network protocol Understanding Operating Systems Purpose Linux can run +1 program at a time using one or several processors. Operate on Intel’s x86 processors & other platforms including Alpha, Sparc, certain 6800 platforms, certain MIPS machines, and Power PCs. Allows several users to work on the same machine at same time. Linux supports pipes, IPC, sockets, etc. Its terminal management conforms to POSIX standards and it also supports pseudoterminals as well as process control systems. Supports a wide range of devices including sound cards, graphics interfaces, networks, SCSI, etc. Supports a memory area reserved to buffer the input and output from different processes. Loads pages into memory only when they’re needed. Dynamic libraries are loaded only when they’re needed & their code is shared if several applications are using them. Linux allows file partitions used by file systems such as Ext2 and partitions having other formats (MS-DOS, ISO 9660, etc.) It supports TCP/IP and other network protocols. 8 UNIX Memory Management • Beginning with Version 4, available for multi-users. • Swapping or demand paging memory management used. • Best depends on kind of applications that will run. – Small jobs -- swapping. Many large jobs -- demand paging. • Swapping requires entire program in RAM before execution. – Imposes size restriction on programs. – Uses round robin policy which slows system when traffic is heavy. • Paging requires more complicated hardware configurations. – Increases system overhead & under heavy loads,thrashing can occur. – Advantage of implementing the concept of virtual memory. Understanding Operating Systems 9 Typical Memory Layout for Single User-Memory Part UNIX Image Stack Highest Memory Address Data Program Code Understanding Operating Systems Lowest Memory Address • Image -- computer execution environment composed of: usermemory part, general register values, status of open files, & current directory. • Must remain in memory during execution of a process 10 Program Code • Sharable portion of program. • Because this code will be physically shared by several processes it must be written in reentrant code. – Code protected -- instructions not modified during normal execution. – All data references made without use of absolute physical addresses. • Memory Manager gives program code special treatment. – Space allocated to program code can’t be released until all of processes using it have completed their execution. – UNIX uses “text table” to track which processes using which code. – Memory isn't released until the program code is no longer needed. Understanding Operating Systems 11 Data Segments & Stack Segments • Data segment starts after program code & grows toward higher memory locations as needed by program. • Stack segment starts at highest memory address & grows downward as subroutine calls & interrupts add info to it. • Data & stack are non-sharable sections of memory, so when original program terminates memory space is released. • While each process is in memory, Memory Manager protects them from each other so they don’t overlap . Understanding Operating Systems 12 UNIX Kernel • Part of OS that implements “system calls” to set up memory boundaries so several processes coexist in memory. – Permanently resides in memory – Processes use calls to interact with File Manager & request I/O. • Kernel -- set of programs that implements most primitive of system’s functions. – Only part of OS to permanently reside in memory. • Remaining sections of OS handled in same way as any large program. – Pages of OS brought into memory on demand. – Uses least-recently-used (LRU) page replacement algorithm. Understanding Operating Systems 13 Linux Memory Management • Allocates memory space to each process. – Address space divided : process code, process data, code & shared library data used by process, & stack used by process. • When process begins execution its segments have fixed size but sometimes process handles variables with unknown number & size. – System calls change size of process data segment (expanding it to accommodate extra data values or reducing it when certain values positioned at end of data segment no longer needed). Understanding Operating Systems 14 Linux Memory Management - 2 • Memory protection based on type of info stored in each region belonging to process address space. – If process modifies access authorization assigned to a memory region, kernel changes protection info assigned to corresponding memory pages. • When process requests pages, Linux loads them into memory. • When kernel needs the memory space, pages are released using a least recently used (LRU) algorithm. Understanding Operating Systems 15 Linux Memory Management - 3 • Linux maintains dynamically managed area in memory (page cache) where new pages inserted or old pages deleted. – If necessary, Linux deactivates swap devices without rebooting. • To track free/busy pages, uses system of page tables. • With certain chip architectures, memory access uses segments. • Uses VM mechanism provided by processor on which it is executing, converting virtual addresses into physical addresses. Understanding Operating Systems 16 UNIX Processor Management • • • • Allocates CPU, schedules process, & handles process requests. – Kernel maintains several tables to coordinate execution of processes & allocation of devices. Using predefined policy, Process Scheduler selects process from READY queue & begins its execution for given time slice. Process scheduling algorithm picks process with highest priority to be run first; any processes using a lot of CPU time get lower priority. – System updates compute-to-total-time ratio for each job every second. – If processes have same computed priority, handled round-robin. Effect of negative feedback -- system balances I/O-bound jobs with CPU-bound jobs. Understanding Operating Systems 17 UNIX Processor Management Policies • To load process from READY queue, Processor Manager chooses process with longest time spent on secondary storage. • To select process to temporarily move out, Processor Manager chooses process that’s either waiting for disk I/O or currently idle. • If process is waiting for completion of I/O request & isn’t ready to run, UNIX dynamically recalculates all process priorities to determine which inactive but ready process will begin when processor available. • Policies seem to work well & don’t impact on running processes. • If disk is used for secondary file storage & as “swapping area,” then heavy traffic significantly slow disk I/O because job swapping may take precedence over file storage. Understanding Operating Systems 18 UNIX Uses Process & User Table to Keep System Running Smoothly • Process table always resides in memory. – Set up when process created; deleted when process terminates. • Each entry contains: process identification #, user identification #, process memory/secondary storage address, process size, & scheduling info. • For processes with sharable code, process table maintains text table. – Contains: memory/ secondary storage address of text segment (sharable code) & count to track # of processes using this code. • When count =0, code not needed. – Table entry & any memory locations allocated to code segment released. Understanding Operating Systems 19 User Table • User table resides in memory only while process is active. – Allocated to each active process. – Kept in transient area of memory. • Contains info needed when process is running: user & group identification # to determine file access privileges; pointers to system’s File Table for every file being used by process; pointer to current directory; & list of responses for various interrupts. • User table, process data segment and code segment (sharable code, can be swapped into/out of main memory as needed. Understanding Operating Systems 20 Synchronization • UNIX is true multitasking OS. – Achieves process synchronization by requiring that processes wait for certain events. – Each event is represented by integers that are equal to address of table associated with event. • Race occurs if event happens during process’s transition between deciding to wait for event & entering WAIT state. – Isn’t a problem in single-processor environments. – May pose a problem in multiprocessor environments. Understanding Operating Systems 21 fork and wait Commands • fork -- gives user capability of executing 1 program from another. – Gives 2nd program all attributes of 1st program. – Splits program into 2 copies, both running from statement after fork. – Process id (pid) generated so each process has unique ID number. • wait -- allows programmer to synchronize process execution by suspending parent until child is finished. Understanding Operating Systems 22 exec Commands (execl, execv, execls, execlp, execvp) • Used to start execution of new program from another program. • Successful exec call overlays 2nd program over 1st. – Leaves only 2nd program in memory. – 2nd program considered new process but has pid of 1st program. • No return from successful exec call. – Concept of parent-child doesn’t hold. • Programmer can use fork, wait, & exec commands in this order to create parent-child relationship & then have child be overlaid by another program that, when finished, awakens parent so that it can continue its execution Understanding Operating Systems 23 Linux Processor Management • • • Linux scheduler scans list of processes in READY state & uses predefined criteria to choose which process to execute. 3 different scheduling policies: 1 for "normal" processes, and 2 for "real time" processes. Each process has associated process type, fixed priority & variable priority. – Used by scheduler to determine which scheduling policy used on processes in READY queue. Understanding Operating Systems 24 Process Types in Linux SCHED_FIFO for non-preemtible "real time" processes SCHED_RR for preemptible "real time" processes SCHED_OTHER for "normal" processes • SCHED_INFO -- executed immediately; scheduler selects process with highest priority & executes it. – Non-preemptible; normally runs to completion. • SCHED_RR -- executed using round robin with small time quantum. • SCHED_OTHER -- executed only when "real time" processes do not exist in READY queue. – dynamic priorities set by user-specified level using system calls "nice" and "setpriority," & by factor computed by system. Understanding Operating Systems 25 Linux Supports Pipes & Clones • Linux supports pipes to allow executing processes to exchange data. • Extension permits process clones to be created. – Clone process is created using primitive clone, by duplicating parent process. – Allows both processes to share same segment of code & data. – Any modification of 1 1 visible to other, unlike classical processes. • Ability to clone processes allows implementation of servers where several threads may be executing. – Operations may simply share data, without making use of inter-process communication mechanisms. Understanding Operating Systems 26 Organization of Table of Processes in Linux • Each process is referenced by descriptor. – Approximately 70 fields describing process attributes & info needed to manage process. – Kernel dynamically allocates descriptors when processes begin execution. – All process descriptors organized in doubly linked list. – Descriptors of ready/in execution processes put in another doubly linked list with fields indicating "next run" and “previously run." • Several macro instructions used by scheduler to manage & update process descriptor lists as needed. Understanding Operating Systems 27 Process Synchronization in Linux • Provides wait queues & semaphores to allow 2 processes to synchronize with each other. • Wait queue -- linked and circular list of process descriptors. • Semaphores used to solve problems of mutual exclusion & problems of producers & consumers. – Contains 3 fields: semaphore counter, number of waiting processes, & list of processes waiting for semaphore. – Semaphore counter may contain only binary values. – Semaphore counter takes value of number of units which are accessible concurrently. Understanding Operating Systems 28 UNIX Device Management : Device Drivers • Innovative feature is treatment of devices—device independent. – Treats each I/O device as special type of file. – Every device assigned name & descriptors that identify devices, contain info about them, & stored in device directory. • Device drivers -- subroutines working with OS to supervise transmission of data between main memory & peripheral unit. • Incorporation of driver into kernel done during system configuration. – Config.c automatically creates conf.c file for any hardware config. – Contains parameters that control resources & 2 tables (bdevsw, cdevsw). 29 Each Physical Device Identified By 1. Major device number -- used as index to array to access appropriate code for specific device driver. 2. Minor device number -- passed to device driver as argument & is used to access one of several identical physical devices. 3. Class • Each class has Configuration Table with array of entry points into device drivers. – Only connection between system code & device drivers. – Allows systems programmers to create new device drivers quickly to accommodate differently configured systems Understanding Operating Systems 30 UNIX Divides I/O System Into 2 Separate Systems 1. Block I/O system (structured I/O system). – – Used for devices addressed as sequence of 512-byte blocks. Allows Device Manager to use buffering to reduce I/O traffic. – UNIX has from 10 to 70 buffers for I/O, & buffer-related info. 2. Character I/O system (unstructured I/O system). – – – • Device drivers that implement character lists. Subroutine puts a character on list, or queue, and another subroutine retrieves character from list. E.g., terminal -- 2 input (raw, canonical) & 1 output queue. Some devices actually belong to both classes. Understanding Operating Systems 31 Linux Device Drivers • Building on UNIX architecture, each device has special section in kernel (device driver). – Includes all instructions necessary to communicate with device. • When new device is developed, it can be used with Linux by writing its own device driver. • Instructions for talking to devices are loaded as needed (seldom used devices) or can be kept in memory all the time when OS system boots. • Although device files may be kept anywhere on file system, kept in /dev directory. Understanding Operating Systems 32 Linux Device Classifications • Devices defined as either character mode or block mode. • Information is kept in device file. • Since there may be +1 device of same type in system, each device is identified by 2 device numbers. – Major number -- device driver to be used. – Minor number -- device number. – Assigned to ensure each device has unique code. Understanding Operating Systems 33 UNIX File Management : Types of Files 1. Directories -- files used by system to maintain hierarchical structure of file system. • Users can read info, but only system can modify. 2. Ordinary files -- where users store information. • Protection is based on user’s requests (read, write, execute, & delete functions). 3. Special files -- device drivers that provide interface to I/O hardware. • • • Appear as entries in directories. Part of file system & reside in /dev directory. Name of each special file indicates type of device with which it’s associated. Understanding Operating Systems 34 UNIX File Structure • Stores files as sequences of bytes. • Doesn’t impose any structure on them. – Text files are strings of characters with lines delimited by line feed, or new line, character. – Binary files are sequences of binary digits grouped into words as appear in memory during execution. • Structure of files is controlled by programs that use them, not by system. Understanding Operating Systems 35 UNIX File Management System Organizes Disk Into 512-Byte Blocks • 1. 2. 3. Disk divided into 4 basic regions: Address 0 reserved for booting. Size of disk & boundaries of other regions. i-list -- list of file definitions using combination of major & minor device numbers & i-numbers to uniquely identify file. 4. Free blocks available for file storage -- kept in linked list where each block points to next available empty block. – As files grow, noncontiguous blocks linked to chain. Understanding Operating Systems 36 UNIX File Storage • Whenever possible files are stored in contiguous empty blocks. • Allocation is very simple. • No need to compact files until become large & more dispersed—so that file retrieval becomes cumbersome. – Compaction to bring retrieval time back to normal. Understanding Operating Systems 37 i-list • Each entry in i-list called i-node with 13 disk addresses. – 1st 10 addresses point to 1st 10 blocks of a file. – If file > 10 blocks, 11th address points to block with addresses of next 128 blocks of file. • Each i-node contains info on specific file (e.g., , owner’s identification, protection bits, physical address, file size, time of creation, last use/last update, number of links, & of file is directory, ordinary file, or special file). Understanding Operating Systems 38 File Names • UNIX is case sensitive. • Most UNIX versions allow file names with unlimited length. • Some system programs (e.g., compilers) expect files to have specific “suffixes”. • UNIX supports a hierarchical tree file structure. – Root directory is identified by slash (/). – Names of other directories are preceded by (/) symbol, which is used as a delimiter. Understanding Operating Systems 39 A Few Rules Apply to All Path Names 1. If path name starts with a slash, path starts at root directory. 2. A path name can be either one name or list of names separated by slashes. 3. Using 2 periods (..) in path name will move you upward in hierarchy (closer to root). 4. Spaces are not allowed within path names. Understanding Operating Systems 40 File Directories • Information for each file: access control, number of links, name of group and owner, byte size of file, date & time of last modification, and file name. Access control drwxrwxr-x drwxrwxr-x -rwxr-xr-x -rwxr--r--rwx------ Understanding Operating Systems No. of links 2 2 1 1 1 Group Owner journal journal journal journal journal comp comp comp comp comp No. of bytes 128 128 11904 12556 10362 Date Time Jan 10 Jan 15 Jan 6 Jan 20 Jan 17 19:32 09:59 11:38 18:08 07:32 File name chap7 chap8 ms-dos unix vax 41 Data Structures for Accessing Files • Info presented in directory isn’t all kept in same location. • UNIX divides the file descriptors into parts. – Hierarchical directories contain only name of file & “i-number.” • All i-nodes are stored in reserved part of device where directory resides, usually in Block 1. • Each i-node has room for 13 pointers (0–12). – 1st 10 block # stored in i-node list relate to 1st 10 blocks of file. – For files larger than 138 blocks, 12th entry points to a block that contains a list of 128 indirect blocks. – For extremely large files of more than 16,522 blocks, 13th entry points to a “triple indirect block.” Understanding Operating Systems 42 Open/Create/Link/Delete Files • File opened -- its device, i-number, & read/write pointer are stored in System File Table & indexed by the i-node. • File created -- i-node is allocated to it, & directory entry with file name & its i-node number is created. • File linked -- directory entry is created with new name & original i-node number, & link-count field in the i-node is incremented by one. • Delete shared file -- link-count field in i-node is decremented by 1. – When count =0, directory entry is erased & all disk blocks allocated to file, along with its i-node block, are deallocated. Understanding Operating Systems 43 Linux File Management • All Linux files organized in directories connected to each other in treelike structure. • Linux specifies 5 types of files used by system to determine what file is to be used for. File type Directory Ordinary file Symbolic link Special file Named pipe Understanding Operating Systems File functions Contains lists of file names. Contains data or programs belonging to users. Contains path name of another file that it is linking to. Assigned to device controller located in kernel. When accessed, physical device associated with it is activated & put into service. Used as communication channel among several processes to exchange data. Creation is same as any sort of file. Carried out via a C function named mkfifo. 44 Linux File Names & File Directories • File names can be up to 256 characters long and contain alphabetic character, underscores, & numbers. • Linux actually uses only 1st 32 or 64 characters of name. • File directories contain same info, in same order as UNIX. • Linux supports—for each type of member, owner, group, or world—3 types of file permissions: read, write, & execute. Understanding Operating Systems 45 Linux Data Structures • Linux support several file types. – Some inherent to this OS, such as Ext2FS. – Others from other OS such as UNIX System V. • Kernel has layer of software to maintain interface between system calls related to files & file management code. – Virtual File System (VFS). – Any process-initiated system call to files is directed to VFS. – Performs file operations independent of format of file system involved. – VFS then redirects request to module managing file. Understanding Operating Systems 46 UNIX User Interface Command (filename) ls ls -l cd cp rm mv more lpr date date -u mkdir grep cat format diff pwd Stands for Run File List Directory Long List Change Directory Copy Remove Move Show More Print Date Universal Date/Time Make Directory “Global Regular Expression/Print” Concatenate Format Different Path Working Directory Action to be performed Run/Execute file with that name. Show a listing of file names in directory. Show a comprehensive directory list. Change working directory. Copy a file into another file or directory. Remove/delete a file or directory. Move or rename a file or directory. Type the file’s contents to the screen. Print out a file. Show date and time. Show date and time in universal format. Make a new directory. Find a specified string in a file. Create a file or append to existing file. Format a volume. Compare two files. Show the path name of this directory. 47 Command Files (Script Files) • Used to automate repetitious tasks. – Each line of file is valid UNIX instruction and can be executed by user simply by typing sh and the name of the script file. – E.g., bash (Bourne Again Shell), csh (C shell) & ksh (Korn shell). • Script files used to automate repetitive tasks & to simplify complex procedures. • If script file included in user’s configuration file, automatically executed every time user logs on. – Exact name of user configuration file varies from system to system. – Common names are .profile and .login. Understanding Operating Systems 48 Redirection • To send output to a file or to another device, use symbol > between command & output destination. – ls > myfiles • To concatenate files (copy their contents into file), use cat file1 file2 > newfile • Symbol >> appends new file to existing file. • Reverse redirection takes input for program from existing file. mail ann roger < memo Understanding Operating Systems 49 Pipes • Pipes & filters allow you to redirect output or input to selected files or devices based on commands given to command interpreter. – UNIX does that by manipulating I/O devices as special files. • Pipe -- open file connecting 2 programs: info written to it by 1 program may be read immediately by other, with synchronization, scheduling, & buffering handled automatically by system. – Programs are executing concurrently, not one after other. – who | sort • Pipeline -- several programs simultaneously processing same I/O stream. – who | sort | lpr Understanding Operating Systems 50 Filters • UNIX has many programs that read some input, manipulate it in some way, & generate output – filters. • wc (word count) -- counts lines, words, & characters in file. • sort -- contents of file sorted & displayed on screen or accepts input from keyboard & directs output to screen. – – – – sort -f sortednames sort -n sortednums sort -nr sortednums sort +2f sortedempl Understanding Operating Systems 51 Other Commands • man – displays on-line manual supplied with OS. • grep -- looks for specific patterns of characters. – grep Pittsburgh maillist – grep -v Pittsburgh maillist – grep -c Pittsburgh maillist • pg or more -- displays output one screen at a time. • nohup -- starts its execution & then log off system without having to wait for it to finish. – nohup cp oldlargefile newlargefile & • nice – lowers priority of a program if it uses large number of resources & you’re not in hurry for results. – nice cp oldlargefile newlargefile & Understanding Operating Systems 52 Linux User Interface • Linux can be used in command-based mode. • Also has built-in GUI support for X-Windows developed at MIT as part of Athena project. – Windows Application Binary Interface (Wabi) • Linux tape archive utility is significant improvement over UNIX tar utility. – Features built-in error recovery. – Accommodates both command line interface and menu-driven GUI. Understanding Operating Systems 53