Download What is an operating system? - KOVAN Research Lab

CENG334
Introduction to Operating Systems
Introduction
Topics
What’s an operating system?
•Course policy
•
Erol Sahin
Dept of Computer Eng.
Middle East Technical University
Ankara, TURKEY
URL: http://kovan.ceng.metu.edu.tr/~erol/Courses/CENG334
Some of the following slides are adapted from Matt Welsh, Harvard Univ.
Week 1
1
Welcome to CENG334!
What is this course about?



Operating Systems drive the inner workings of
virtually every computer in the world today
PCs, servers, iPods, cell phones, missile guidance systems,
etc. all have an OS that dictate how they operate.
The OS manages many aspects of how programs run, and how
they interact with hardware and the outside world.
2
3
Welcome to CENG334!
What is this course about?



Operating Systems drive the inner workings of
virtually every computer in the world today
PCs, servers, iPods, cell phones, missile guidance systems,
etc. all have an OS that dictate how they operate.
The OS manages many aspects of how programs run, and how
they interact with hardware and the outside world.
Understanding the OS is essential for understanding:





System performance and reliability
Resource management
Virtualization and abstraction
Concurrency and parallelism
Hardware interfaces and I/O
This course is about more than just “kernel internals”

It is really about learning complex systems design.
4
What is an operating system?
Software that provides an elaborate illusion to applications
User application
User application
User application
Protection boundary
Kernel
Memory management
Process management
Accounting
Device drivers
Filesystem
TCP/IP stack
Disk I/O
CPU support
Hardware/software
interface
Gnarly world of hardware
5
One OS Function: Concurrency
Give every application the illusion of having its own CPU!
I think I have my own CPU
So do I!
6
One OS Function: Concurrency

The OS timeslices each application on a single CPU

Switches between applications extremely rapidly, i.e., 100 times/sec
Kernel Scheduler
Timeslice on
single CPU system
time
7
Another OS Function: Virtual Memory

Give every application the illusion of having infinite memory


And, that it can access any memory address it likes!
In reality, RAM is split across multiple applications
Code
Data
Physical RAM
Stack
VM System
Code
Data
Stack
Swap out to disk
8
More OS Functions
Multiprocessor support



Modern systems have multiple CPUs
Can run multiple applications (or threads within applications) in parallel
OS must ensure that memory and cache contents are consistent across CPUs
Filesystems



Network protocols



Real disks have a hairy, sector-based access model
User applications see flat files arranged in a hierarchical namespace
Network interface hardware operates on the level of unreliable packets
User apps see a (potentially reliable) byte-stream socket
Security and protection

Prevent multiple apps from interfering with each other and with normal system
operation
9
Why bother with an OS?

Not just to give Slashdot readers something to argue about...

What do you think?
10
Why bother with an OS?

Not just to give Slashdot readers something to argue about...

Abstract away messy details of hardware




Safety!



Give apps a nice clean view of the system
Save programmers a lot of trouble when building applications
Allow apps to be ported across a wide range of hardware platforms
Don't let applications run amok – keep them in a “sandbox”
e.g., Access to unallocated memory address crashes only the program, not the
whole system
 Segmentation fault – core dumped
Efficiency


Share one machine across many different apps: concurrent execution
You would be surprised how much slack there is in a typical computer system
11
Why study operating systems?

Most people will never write one from scratch...
(Though if you do you stand to get incredibly rich)
Although more and more people are hacking them (e.g., Linux and BSD)
You need to understand the “big picture” in order to hack the details




This class is about much more than the kernel!



Data structures, concurrency, performance, resource management, synchronization,
networks, distributed systems, databases ...
The ideas and skills you pick up in this class have broad applications
 And it doesn't hurt for those Microsoft interviews either
This course is the basis for future work in other areas of systems

Distributed systems, P2P, sensor nets, etc.
12
Major OS Design Issues

Structure


Sharing


How to prevent malicious users from compromising the system?
Performance


How to programs and users name and access resources?
Security


How are limited resources multiplexed across users?
Naming


How is the OS itself organized? Lots of modules? One big blob of code?
How to keep it all fast?
Reliability

What happens when a program (or the OS itself) has a bug or failure?
13
More OS Design Issues

Extensibility


Communication


What happens when demands on resources increase?
Distribution


How are multiple concurrent activities created and controlled?
Scale


How do programs exchange information?
Concurrency


How do users and developers add new features?
How do many computers interact with each other, e.g., to pool resources?
Accounting

How do you track (and maybe charge for) resource usage?
14
Teaching staff
– Instructor:
• Section 1, 2: Dr. Erol Sahin
 Location: B-111, Tel: 210 5539,
 E-mail: erol@ceng
 Office hours: by appointment.
– Teaching assistant:
• Can Hosgor
• hosgor@ceng
• Erdal Sivri
 E-mail: erdal@ceng
15
Textbook

Operating System Concepts





Avi Silberschatz
Peter Baer Galvin
Greg Gagne
John Wiley & Sons, Inc.
ISBN 0-471-76907-X
Modern Operating Systems
 Andrew S. Tanenbaum
 Prentice Hall., 2001,
 ISBN-10: 0130313580
 ISBN-13: 97801303135
Both books should be available at the bookstore..
16
Grading

Midterm: 30 % [April 14]

Assignments: 30 %

Final: 40 %
17
Assignments

Assignment 1: Basic Shell


Assignment 2: Basic Threading


process management, Unix system calls
thread management and synchronization
Assignment 3: File System

ext2 file system implementation, inode/block concepts, file/directory
handling
18
Policies
• Late assignments:
– Late submission policy will be announced for
each assignment.
• Academic dishonesty:
– All assignments submitted should be fully
your own. We have a zero tolerance policy
on cheating and plagiarism. Your work will be
regularly checked for such misconduct and
any such attempts will be prosecuted:
• At all times you have the right to challenge our
decisions on cheating, upon which the case
will be processed through the disciplinary
channels of the university. However, we would
like to remind you that, if found guilty, the legal
code of the university proposes a minimum of
six month expulsion from the university.
19
Cheating
• What is cheating?
– Sharing code: either by copying, retyping, looking at, or supplying a copy of a
file.
• What is NOT cheating?
– Helping others use systems or tools.
– Helping others with high-level design issues.
– Helping others debug their code.
20
Communication
•
•
•
•
Online information about the course will be available on the CENG334 web
page: http://kovan.ceng.metu.edu.tr/~erol/Courses/CENG334/
Announcements about the course will be made at the CENG334
newsgroup at news://metu.ceng.course.334
Please put Section 1,2 on the subject line of your posting.
If you have a specific question you can send an e-mail to the us. However
make sure that the subject line starts with CENG334 [capital letters, and no
spaces] to get faster reply.
21
Operating System Overview
Topics
Brief History
CENG334
•OS Services
Introduction
to Operating Systems
•System calls
•Basic Operation
•OS structures
•
Erol Sahin
Dept of Computer Eng.
Middle East Technical University
Ankara, TURKEY
URL: http://kovan.ceng.metu.edu.tr/~erol/Courses/CENG334
Some of the following slides are adapted from Matt Welsh, Harvard Univ.
22
In the Beginning...

There was no OS – just libraries

Computer only ran one program at a time, so no need for an OS

Programming through wiring..
Harvard Mark I, 1944
ENIAC, 1945
IBM 360,
1960's
23
In the Beginning...

There was no OS – just libraries


Computer only ran one program at a time, so no need for an OS
And then there were batch systems



Programs printed on stacks of punchhole cards
OS was resident in a portion of machine memory
When previous program was finished, OS loaded next program to run
24
Punch Card
25
In the Beginning...

There was no OS – just libraries


And then there were batch systems




Computer only ran one program at a time, so no need for an OS
Programs printed on stacks of punchhole cards
OS was resident in a portion of machine memory
When previous program was finished, OS loaded next program to run
Disk spooling



Disks were much read stack onto disk while previous program is running
With multiple programs on disk, need to decide which to run next!
But, CPU still idle while program accesses a peripheral (e.g., tape or disk!)
26
Multiprogramming
To increase system utilization, multiprogramming OS’s were
invented

keeps multiple runnable jobs loaded in memory at once
Overlaps I/O of a job with computing of another

While one job waits for I/O to compile, CPU runs instructions from another job
To benefit, need asynchronous I/O devices

need some way to know when devices are done performing I/O
Goal: optimize system throughput

perhaps at the cost of response time…
Dennis Ritchie and Ken
Thompson at a PDP11, 1971
27
Timesharing
To support interactive use, timesharing OS's were created

multiple terminals connected to one machine

each user has illusion of entire machine to him/herself

optimize response time, perhaps at the cost of throughput
Timeslicing


divide CPU fairly among the users
if job is truly interactive (e.g. editor), then can switch between programs and users faster
than users can generate load
MIT Multics (mid-1960’s) was the first large timeshared system

nearly all modern OS concepts can be traced back to Multics
28
Personal Computing
Apple I, 1976
Totally changed the computing industry.
CP/M: First personal computer OS


IBM needed OS for their PCs, CP/M behind schedule
Bill Gates to the rescue: Bought 86-DOS and made MS-DOS
 DOS is basically a subroutine library!
Apple LISA, 1983
Many popular personal computers follow

Apple, Commodore, TRS-80, TI 99/4, Atari, etc...
IBM PC, 1981
Bill Gates and Paul Allen, c.1975
Commodore VIC-20
29
30
Parallel Computing and Clusters
High-end scientific apps want to use many CPUs at once

Parallel processing to crunch on enormous data sets

Need OS and language primitives for dividing program into parallel activities

Need OS primitives for fast communication between processors

degree of speedup dictated by communication/computation ratio
Many kinds of parallel machines:



SMPs: symmetric multiprocessors – several CPUs accessing the same memory
MPPs: massively parallel processors – each CPU may have its own memory
Clusters: connect a lot of commodity machines with a fast network
31
Distributed OS
Goal – Make use of geographically distributed resources

workstations on a LAN

servers across the Internet
Supports communication between applications

interprocess communication (on a single machine):


message passing and shared memory
networking procotols (across multiple machines):

TCP/IP, Java RMI, .NET SOAP
“The Grid”, .NET, and OGSA

Idea: Seamlessly connect vast computational resources across the Internet
32
Embedded OS
The rise of tiny computers everywhere – ubiquitous computing

Processor cost low enough to embed in many devices


PDAs, cell phones, pagers, ...
How many CPUs are in your car? On your body right now?
Gets more interesting with ubiquitous networking!

Wireless networks becoming pervasive

Sensor networks are an exciting new direction here

Little “motes” with less 4KB of RAM, some sensors, and a radio
Typically very constrained hardware resources

slow processors

very small amount of memory (e.g. 8 MB)

no disk – but maybe quasi-permanent storage such as EEPROM
33
Operating System Overview
User application
User application
User application
Protection boundary
Kernel
Memory management
Process management
Accounting
Device drivers
Filesystem
Disk I/O
TCP/IP stack
CPU support
Hardware/software
interface
34
Operating System Services
(What things does the OS do?)
Services that (more-or-less) map onto components

Program execution


I/O operations



Standardized interfaces to extremely diverse devices
File system manipulation

How do you read/write/preserve files?

Looming concern: How do you even find files???
Communications


How do you execute concurrent sequences of instructions?
Networking protocols/Interface with CyberSpace?
User interface- Almost all operating systems have a user interface (UI)

Varies between Command-Line (CLI), Graphics User Interface (GUI), Batch
Cross-cutting capabilities

Error detection & recovery

Resource allocation

Accounting

Protection
35
User Operating System Interface - CLI
CLI allows direct command entry



Sometimes implemented in kernel, sometimes by
systems programs
Sometimes multiple flavors implemented – shells
Primarily fetches a command from user and
executes it
 Sometimes commands built-in, sometimes just
names of programs

If the latter, adding new features
doesn’t require shell modification
36
User Operating System Interface - GUI
User-friendly desktop metaphor interface
•
•
•
•
•
Usually mouse, keyboard, and monitor
Icons represent files, programs, actions, etc
Various mouse buttons over objects in the interface
cause various actions
• provide information, options,
• execute function, open directory (known as a folder)
Invented at Xerox PARC
Many systems now include both CLI and GUI
interfaces
•
•
•
Microsoft Windows is GUI with CLI “command” shell
Apple Mac OS X as “Aqua” GUI interface with UNIX
kernel underneath and shells available
Solaris is CLI with optional GUI interfaces (Java
Desktop, KDE)
37
Xerox PARC Alto
38
System Calls
Programming interface to the services
provided by the OS
Typically written in a high-level language (C
or C++)
Mostly accessed by programs via a highlevel Application Program Interface (API)
rather than direct system call use
Three most common APIs are
•
•
•
•
Win32 API for Windows,
POSIX API for POSIX-based systems (including
virtually all versions of UNIX, Linux, and Mac OS X),
and
Java API for the Java virtual machine (JVM)
Why use APIs rather than system calls?
39
Example of Standard API
Consider the ReadFile() function in the
Win32 API—a function for reading from a file
A description of the parameters passed to ReadFile()

HANDLE file—the file to be read

LPVOID buffer—a buffer where the data will be read into and written from

DWORD bytesToRead—the number of bytes to be read into the buffer

LPDWORD bytesRead—the number of bytes read during the last read

LPOVERLAPPED ovl—indicates if overlapped I/O is being used
40
System Call Implementation
Typically, a number associated with each system
call
•
•
•
•
System-call interface maintains a table indexed according
to these numbers
The system call interface invokes intended system call in
OS kernel and returns status of the system call and any
return values
The caller need know nothing about how the system call is
implemented
Just needs to obey API and understand what OS will do as
a result call
Most details of OS interface hidden from
programmer by API Managed by run-time
support library (set of functions built into
libraries included with compiler)
41
API – System Call – OS Relationship
42
System Programs
System programs provide a convenient
environment for program development and
execution. They can be divided into:

File manipulation

Status information

File modification

Programming language support

Program loading and execution

Communications

Application programs
Most users’ view of the operation system is
defined by system programs, not the actual
system calls
43
System Programs
Provide a convenient environment for program development
and execution

Some of them are simply user interfaces to system calls;
others are considerably more complex
File management - Create, delete, copy, rename, print, dump,
list, and generally manipulate files and directories
Status information




Some ask the system for info - date, time, amount of available
memory, disk space, number of users
Others provide detailed performance, logging, and debugging
information
Typically, these programs format and print the output to the
terminal or other output devices
Some systems implement a registry - used to store and
retrieve configuration information
44
System Programs (cont’d)
File modification


Text editors to create and modify files
Special commands to search contents of files or
perform transformations of the text
Programming-language support - Compilers,
assemblers, debuggers and interpreters
sometimes provided
Program loading and execution- Absolute loaders,
relocatable loaders, linkage editors, and overlayloaders, debugging systems for higher-level and
machine language
Communications - Provide the mechanism for
creating virtual connections among processes,
users, and computer systems

Allow users to send messages to one another’s
screens, browse web pages, send electronic-mail
messages, log in remotely, transfer files from one
machine to another
45
Operating System operation
The OS kernel is just a bunch of code that sits around in memory,
waiting to be executed
Memory
Emacs
Firefox
OS Kernel
(device drivers,
file systems,
virtual memory, etc.)
xmms
sshd
46
Operating System operation
The OS kernel is just a bunch of code that sits around in memory,
waiting to be executed
Memory
Emacs
xmms
Firefox
System call
(open network socket)
OS Kernel
(device drivers,
file systems,
virtual memory, etc.)
Interrupt (disk block read)
sshd
OS is triggered in two ways: system calls and hardware interrupts
System call: Direct “call” from a user program

For example, open() to open a file, or exec() to run a new program
Hardware interrupt: Trigger from some hardware device

For example, when a disk block has been read or written
47
Interrupts – a primer
An interrupt is a signal that causes the CPU to jump to a predefined instruction – called the interrupt handler
•
•
Hardware interrupt examples
•
•
•
Interrupt can be caused by hardware or software
Timer interrupt (periodic “tick” from a programmable timer)
Device interrupts
• e.g., Disk will interrupt the CPU when an I/O operation has completed
Software interrupt examples (also called exceptions)
•
•
•
Division by zero error
Access to a bad memory address
Intentional software interrupt – e.g., x86 “INT” instruction
• Can be used to trap from user program into the OS kernel!
• Why might this be useful?
48
Interrupt handler example
1) OS fills in interrupt handler
table (usually at boot time)
Interrupt handler table
Interrupt handler
for interrupt 4
2) Interrupt occurs – e.g., hardware
signal
Interrupt handler
for interrupt 5
!!!
3) CPU state saved to stack
49
Interrupt handler example
1) OS fills in interrupt handler
table (usually at boot time)
Interrupt handler table
Interrupt handler
for interrupt 4
2) Interrupt occurs – e.g., hardware
signal
Interrupt handler
for interrupt 5
!!!
3) CPU state saved to stack
4) CPU consults interrupt table
and invokes appropriate handler
50
Protection
A major job of the OS is to enforce protection
Prevent malicious or buggy programs from:


Allocating too many resources (denial of service)
Corrupting or overwriting shared resources (files, shared memory, etc.)
Prevent different users, groups, etc. from:


Accessing or modifying private state (files, shared memory, etc.)
Killing each other's processes
How does the OS enforce protection boundaries?
51
Enforcing Resource Limits
The OS limits what resources user programs can access



For example, Emacs can't modify memory in use by Mozilla.
xmms can't hog the CPU and prevent other programs from running.
One user cannot read/write another user's files
 (Unless permissions are set appropriately)
How does the OS enforce these limits?


This implies that regular user programs cannot “break out” of these limits!
We'll see how on the next slide.
A lot of viruses, worms, etc. exploit security holes in the OS

Overrunning a memory buffer in the kernel can give a non-root process root privileges
 Kernel code needs to be rock solid in order to be secure!!!
52
User mode vs. kernel mode
What makes the kernel different from user programs?

Kernel can execute special privileged instructions
Examples of privileged instructions:




Access I/O devices
 Poll for IO, perform DMA, catch hardware interrupt
Manipulate memory management
 Set up page tables, load/flush the TLB and CPU caches, etc.
Configure various “mode bits”
 Interrupt priority level, software trap vectors, etc.
Call halt instruction
 Put CPU into low-power or idle state until next interrupt
These are enforced by the CPU hardware itself.



CPU has at least two protection levels: Kernel mode and user mode
CPU checks current protection level on each instruction
What happens if user program tries to execute a privileged instruction?
53
Boundary Crossing
Mozilla calls read() system call
User mode
Trap to kernel mode
Save application registers and state
Restore app registers
Return CPU to user mode
Kernel mode
Kernel trap handler
Lookup read() in system call table
Invoke internal read() function
Return to trap handler
read() system call
Perform internal read()
54
Web surfing homework for Wednesday!
Learn
•
More about XEROX PARC
•
•
More about Ken Thomson and Dennis
Ritchie
•
•
How did MS-DOS become so successful?
More about Apple
•
•
What are they known for
More about Microsoft
•
•
What else had they invented
What’s the relation between XEROX PARC GUI and
Apple GUI?
Use Wikipedia, and google the web..
55
OS design and implementation
There is no ultimate OS that would satisfy all requirements:
•
Trade-offs have to made at each level and for all aspects.
Important principle to separate
•
•
•
Policy: What will be done?
Mechanism: How to do it?
Mechanisms determine how to do something, policies decide what
will be done
•
The separation of policy from mechanism is a very important principle, it allows
maximum flexibility if policy decisions are to be changed later
56
Operating Systems Structure
(What is the organizational Principle?)
Simple

Only one or two levels of code
Layered

Lower levels independent of upper levels
Microkernel

OS built from many user-level processes
Modular

Core kernel with Dynamically loadable modules
57
Simple Structure
MS-DOS – written to
provide the most
functionality in the least
space
Not divided into modules
 Interfaces and levels of
functionality not well
separated

58
Monolithic Kernels
Most common OS kernel design (used in UNIX and Linux)



Kernel code is privileged and lives in its own address space
User applications are unprivileged and live in their own separate address spaces
All kernel functions loaded into memory as one large, messy program
User application
User application
User application
System call
Protection boundary
Kernel
Memory management
Process management
Accounting
Device drivers
Filesystem
Disk I/O
TCP/IP stack
CPU support
Pros and cons???
59
Monolithic Kernels
Most common OS kernel design



Kernel code is privileged and lives in its own address space
User applications are unprivileged and live in their own separate address spaces
All kernel functions loaded into memory as one large, messy program
User application
User application
User application
System call
Protection boundary
Kernel
Memory management
Process management
Accounting
Device drivers
Filesystem
Disk I/O
TCP/IP stack
CPU support
Pros and cons




Pro: Overhead of module interactions within the kernel is low (function call)
Pro: Kernel modules can directly share memory
Con: Very complicated and difficult to organize
Con: A bug in any part of the kernel can crash the whole system!
60
Layered kernels
Operating system is divided many layers (levels)
•
•
•
•
Each layer uses functions (operations) and services
of only lower-level layers
•
•
•
Each built on top of lower layers
Bottom layer (layer 0) is hardware
Highest layer (layer N) is the user interface
Advantage: modularity Easier debugging/Maintenance
Not always possible: Does process scheduler lie above or
below virtual memory layer?
•
Need to reschedule processor while waiting for paging
•
May need to page in information about tasks
Important: Machine-dependent vs independent
layers
•
•
Easier migration between platforms
Easier evolution of hardware platform
61
Microkernels
Use a very small, minimal kernel, and implement all other functionality
as user level “servers”



Kernel only knows how to perform lowest-level hardware operations
Device drivers, filesystems, virtual memory, etc. all implemented on top
Use inter-process procedure call (IPC) to communicate between
applications and servers
User application
User application
Memory management
Accounting
Device drivers
User application
Process management
Filesystem
Disk I/O
Inter-process
procedure call
TCP/IP stack
CPU support
Protection boundary
Microkernel
Pros and Cons???
62
Microkernels - 2
Pros and cons
•
•
•
Pro: Kernel is small and simple, servers are protected from each other
Con: Overhead of invoking servers may be very high
• e.g., A user process accessing a file may require inter-process communication
through 2 or 3 servers!
Microkernels today
•
•
•
Lots of research in late 80's and early 90's
Windows NT uses “modified microkernel”:
• Monolithic kernel for most things, OS APIs (DOS, Win3.1, Win32, POSIX)
implemented as user-level services
Mac OS X has reincarnated the microkernel architecture as well:
• Gnarly hybrid of Mach (microkernel) and FreeBSD (monolithic)
63
Modules-based Structure
Most modern operating systems implement modules

Uses object-oriented approach

Each core component is separate

Each talks to the others over known interfaces

Each is loadable as needed within the kernel
Overall, similar to layers but more flexible
64
Virtual Machines
A virtual machine takes the layered approach to its logical
conclusion. It treats hardware and the operating system kernel
as though they were all hardware
A virtual machine provides an interface identical to the underlying
bare hardware
The operating system creates the illusion of multiple processes,
each executing on its own processor with its own (virtual)
memory
Virtualbox.org
Vmware.com
parallels.com
65
VMware Architecture
66
Virtualization
67
The Java Virtual Machine
68