Download Lecture 15 - Department of Math and Computer Science

Chapter 21: The Linux
System
Operating System Concepts – 8th Edition,
Silberschatz, Galvin and Gagne ©2009
History
„ Linux is a modern, free operating system based on UNIX
standards
„ First developed as a small but self-contained kernel in 1991 by
Linus Torvalds, with the major design goal of UNIX compatibility
„ Its history has been one of collaboration by many users from all
around
d th
the world,
ld corresponding
di almost
l
t exclusively
l i l over th
the
Internet
„ Designed to run efficiently and reliably on common PC hardware,
but also runs on a variety of other platforms
„ OS kernel is entirely original, but it can run much existing free
UNIX software, resulting in an entire UNIX-compatible operating
system free from proprietary code
„ Many, varying Linux Distributions including the kernel,
applications, and management tools
Operating System Concepts – 8th Edition
21.2
Silberschatz, Galvin and Gagne ©2009
The Linux Kernel
„ Version 0.01 (May 1991) had no networking, ran only on 80386-
compatible Intel processors and on PC hardware, had extremely limited
device-drive
device
drive support, and supported only the Minix file system
„ Linux 1.0 (March 1994) included these new features:
z
z
z
z
z
z
Support for UNIX’s standard TCP/IP networking protocols
BSD-compatible
p
socket interface for networking
gp
programming
g
g
Device-driver support for running IP over an Ethernet
Enhanced file system
Support for a range of SCSI controllers for
high-performance
f
disk access
Extra hardware support
„ Version 1.2 (March 1995) was the final PC-only Linux kernel
Operating System Concepts – 8th Edition
21.3
Silberschatz, Galvin and Gagne ©2009
Linux 2.0
„
„
Released in June 1996, 2.0 added two major new capabilities:
z
Support for multiple architectures, including a fully 64-bit native Alpha port
z
Support for multiprocessor architectures
Other new features included:
z
Improved memory-management code
z
Improved TCP/IP performance
z
Support for internal kernel threads, for handling dependencies between
loadable modules, and for automatic loading of modules on demand
z
Standardized configuration interface
„
Available for Motorola 68000-series processors, Sun Sparc systems, and for
PC and PowerMac systems
„
2.4
2
4 and 2.6
2 6 increased SMP support
support, added journaling file system,
system preemptive
kernel, 64-bit memory support
Operating System Concepts – 8th Edition
21.4
Silberschatz, Galvin and Gagne ©2009
The Linux System
„ Linux uses many tools developed as part of Berkeley’s BSD
operating system, MIT’s X Window System, and the Free
Software Foundation's GNU project
„ The min system libraries were started by the GNU project,
with improvements
p
p
provided by
y the Linux community
y
„ Linux networking-administration tools were derived from
4.3BSD code; recent BSD derivatives such as Free BSD
have borrowed code from Linux in return
„ The Linux system is maintained by a loose network of
developers collaborating over the Internet, with a small
number of public ftp sites acting as de facto standard
repositories
Operating System Concepts – 8th Edition
21.5
Silberschatz, Galvin and Gagne ©2009
Linux Distributions
„ Standard, precompiled sets of packages, or distributions, include
the basic Linux system, system installation and management
utilities,
tiliti
and
d ready-to-install
d t i t ll packages
k
off common UNIX tools
t l
„ The first distributions managed these packages by simply providing
a means of unpacking all the files into the appropriate places;
modern distributions include advanced package management
„ Early distributions included SLS and Slackware
z
Red Hat and Debian are popular distributions from commercial
and noncommercial so
sources,
rces respecti
respectively
el
„ The RPM Package file format permits compatibility among the
various Linux distributions
Operating System Concepts – 8th Edition
21.6
Silberschatz, Galvin and Gagne ©2009
Linux Licensing
„ The Linux kernel is distributed under the GNU General Public
License (GPL), the terms of which are set out by the Free Software
F
Foundation
d ti
„ Anyone using Linux, or creating their own derivative of Linux, may
not make the derived product proprietary; software released under
the GPL may not be redistributed as a binary-only product
Operating System Concepts – 8th Edition
21.7
Silberschatz, Galvin and Gagne ©2009
Design Principles
„ Linux is a multiuser, multitasking system with a full set of UNIX-
compatible tools
„ Its file system adheres to traditional UNIX semantics, and it fully
implements the standard UNIX networking model
„ Main design goals are speed, efficiency, and standardization
„ Linux is designed to be compliant with the relevant POSIX
documents; at least two Linux distributions have achieved official
POSIX certification
„ The Linux programming interface adheres to the SVR4 UNIX
semantics, rather than to BSD behavior
Operating System Concepts – 8th Edition
21.8
Silberschatz, Galvin and Gagne ©2009
Components of a Linux System
Operating System Concepts – 8th Edition
21.9
Silberschatz, Galvin and Gagne ©2009
Components of a Linux System (Cont)
„ Like most UNIX implementations, Linux is composed of three main
bodies of code; the most important distinction between the kernel
and
d allll other
th components
t
„ The kernel is responsible for maintaining the important abstractions
of the operating system
z
Kernel code executes in kernel mode with full access to all the
physical resources of the computer
z
All kernel code and data structures are kept in the same single
address space
Operating System Concepts – 8th Edition
21.10
Silberschatz, Galvin and Gagne ©2009
Components of a Linux System (Cont)
„ The system libraries define a standard set of functions through
which applications interact with the kernel, and which implement
much
h off the
th operating-system
ti
t
functionality
f
ti
lit th
thatt d
does nott need
d th
the
full privileges of kernel code
„ The system utilities perform individual specialized management
tasks
Operating System Concepts – 8th Edition
21.11
Silberschatz, Galvin and Gagne ©2009
Kernel Modules
„ Sections of kernel code that can be compiled, loaded, and
unloaded independent of the rest of the kernel
„ A kernel module may typically implement a device driver, a file
system, or a networking protocol
„ The module interface allows third parties to write and distribute,
on their own terms, device drivers or file systems that could not
be distributed under the GPL
„ Kernel modules allow a Linux system to be set up with a
standard minimal kernel
standard,
kernel, without
itho t an
any e
extra
tra de
device
ice dri
drivers
ers b
built
ilt in
„ Three components to Linux module support:
z
module management
z
driver registration
z
conflict resolution
Operating System Concepts – 8th Edition
21.12
Silberschatz, Galvin and Gagne ©2009
Process Management
„ UNIX process management separates the creation of processes
and the running of a new program into two distinct operations.
z
The fork system call creates a new process
z
A new program is run after a call to execve
„ Under UNIX, a process encompasses all the information that the
operating system must maintain to track the context of a single
execution of a single program
„ Under Linux, process properties fall into three groups: the
process’s identity, environment, and context
Operating System Concepts – 8th Edition
21.13
Silberschatz, Galvin and Gagne ©2009
Process Identity
„ Process ID (PID). The unique identifier for the process; used to
specify processes to the operating system when an application makes
a system call to signal,
signal modify,
modify or wait for another process
„ Credentials. Each process must have an associated user ID and one
or more group IDs that determine the process’s rights to access
system resources and files
„ Personality. Not traditionally found on UNIX systems, but under Linux
each process has an associated personality identifier that can slightly
modify the semantics of certain system calls
z Used primaril
primarily b
by em
emulation
lation libraries to req
request
est that ssystem
stem calls
be compatible with certain specific flavors of UNIX
Operating System Concepts – 8th Edition
21.14
Silberschatz, Galvin and Gagne ©2009
Process Environment
„ The process’s environment is inherited from its parent, and is
composed of two null-terminated vectors:
z
The argument vector lists the command-line arguments used to
invoke the running program; conventionally starts with the name of
the program itself
z
The environment vector is a list of “NAME=VALUE” pairs that
associates named environment variables with arbitrary textual
values
„ Passing environment
en ironment variables
ariables among processes and inheriting
variables by a process’s children are flexible means of passing
information to components of the user-mode system software
„ The environment-variable mechanism provides a customization of the
operating system that can be set on a per-process basis, rather than
being configured for the system as a whole
Operating System Concepts – 8th Edition
21.15
Silberschatz, Galvin and Gagne ©2009
Process Context
„ The (constantly changing) state of a running program at any point in time
„ The scheduling context is the most important part of the process context; it
is the information that the scheduler needs to suspend and restart the
process
„ The kernel maintains accounting information about the resources currently
being
b
i consumed
db
by each
h process, and
d th
the ttotal
t l resources consumed
db
by th
the
process in its lifetime so far
„ The file table is an array of pointers to kernel file structures
z
When making file
f I/O
/O system calls, processes refer
f to files
f
by their index
into this table
Operating System Concepts – 8th Edition
21.16
Silberschatz, Galvin and Gagne ©2009
Process Context (Cont)
„ Whereas the file table lists the existing open files, the
file-system context applies to requests to open new files
z
The current root and default directories to be used for new file searches
are stored here
„ The signal-handler table defines the routine in the process’s address
space to
t be
b called
ll d when
h specific
ifi signals
i
l arrive
i
„ The virtual-memory context of a process describes the full contents of the
its private address space
Operating System Concepts – 8th Edition
21.17
Silberschatz, Galvin and Gagne ©2009
Processes and Threads
„ Linux uses the same internal representation for processes and threads; a
thread is simply a new process that happens to share the same address
space as its parent
„ A distinction is only made when a new thread is created by the clone
system call
z
f k creates a new process with its own entirely new process context
fork
z
clone creates a new process with its own identity, but that is allowed to
share the data structures of its parent
„ Using
U i clone gives
i
an application
li ti fifine-grained
i d control
t l over exactly
tl what
h t iis
shared between two threads
Operating System Concepts – 8th Edition
21.18
Silberschatz, Galvin and Gagne ©2009
Kernel Synchronization
„ A request for kernel-mode execution can occur in two ways:
z
A running program may request an operating system service, either
explicitly via a system call, or implicitly, for example, when a page fault
occurs
z
A device driver may deliver a hardware interrupt that causes the CPU to
start
t t executing
ti a kernel-defined
k
l d fi d h
handler
dl ffor th
thatt iinterrupt
t
t
„ Kernel synchronization requires a framework that will allow the kernel’s
critical sections to run without interruption by another critical section
Operating System Concepts – 8th Edition
21.19
Silberschatz, Galvin and Gagne ©2009
Kernel Synchronization (Cont)
„ Linux uses two techniques to protect critical sections:
1. Normal kernel code is nonpreemptible (until 2.6)
– when a time interrupt is received while a process is
executing a kernel system service routine, the kernel’s
need_resched flag is set so that the scheduler will run
once the system call has completed and control is
about to be returned to user mode – so kernel code gets another turn
2. The second technique applies to critical sections that occur in an
interrupt
p service routines
– By using the processor’s interrupt control hardware to disable
interrupts during a critical section, the kernel guarantees that it can
proceed without the risk of concurrent access of shared data structures
Operating System Concepts – 8th Edition
21.20
Silberschatz, Galvin and Gagne ©2009
Kernel Synchronization (Cont)
„ To avoid performance penalties, Linux’s kernel uses a synchronization
architecture that allows long critical sections to run without having interrupts
disabled for the critical section’s
section s entire duration
„ Interrupt service routines are separated into a top half and a bottom half.
z
The top half is a normal interrupt service routine, and runs with
recursive
i iinterrupts
t
t di
disabled
bl d
z
The bottom half is run, with all interrupts enabled, by a miniature
scheduler that ensures that bottom halves never interrupt themselves
z
This architecture is completed by a mechanism for
f disabling selected
bottom halves while executing normal, foreground kernel code
Operating System Concepts – 8th Edition
21.21
Silberschatz, Galvin and Gagne ©2009
Interrupt Protection Levels
„ Each level may be interrupted by code running at a higher
level, but will never be interrupted by code running at the
same or a lower level
„ User processes can always be preempted by another process
when a time-sharing scheduling interrupt occurs
Operating System Concepts – 8th Edition
21.22
Silberschatz, Galvin and Gagne ©2009
Process Implementation
„ Same fork()/exec() interface you are familiar with.
„ fork() is actually implemented with more flexible clone() method
z
can create a task with a virtual memory context (process) or without
(thread)
„ Each thread (task) has
z
unique Program Counter
z
stack
z
register set
z
address space pointer (optional)
Operating System Concepts – 8th Edition
21.23
Silberschatz, Galvin and Gagne ©2009
Process Implementation (2)
„ Each task has a process descriptor
„ struct task_struct (include/linux/sched.h, line 335 in 2.4.24 kernel source)
„ Contains all relevant info about a process
„ Stored at the "end" of the kernel stack:
z
Register-poor
Register
poor architectures don't
don t need
Stack
High addrs
extra register for task_struct
„ The task_struct instance for the current
taskk iis available
il bl via
i the
h current pointer
i
(see include/asm-arm/current.h)
„ current is actually calculated from stack
pointer value
Operating System Concepts – 8th Edition
21.24
task_struct
space
Low addrs
Silberschatz, Galvin and Gagne ©2009
task_struct fields of interest
„ volatile long state – process state (-1 unrunnable, 0 runnable, >0 stopped)
z
Actual values are:
#define TASK_RUNNING
0
#define TASK_INTERRUPTIBLE
1
#define TASK
TASK_UNINTERRUPTIBLE
UNINTERRUPTIBLE
2
#define TASK_ZOMBIE
4
#define TASK_STOPPED
8
#define TASK_EXCLUSIVE
32
„ int sigpending – set if a signal is waiting for this process
„ struct mm_struct *mm – pointer to virtual address space
„ pid_t pid – process ID (default max is 32768)
Operating System Concepts – 8th Edition
21.25
Silberschatz, Galvin and Gagne ©2009
task_struct fields of interest (cont)
„ struct task_struct *p_opptr, *p_pptr, *p_cptr, *p_ysptr, *p_osptr;
z
Process tree pointers: parent, youngest child, younger sibling, older
sibling
„ uid_t uid, euid, suid, fsuid;
gid_t gid, egid, sgid, fsgid;
z
user and group IDs
„ struct thread_struct thread
z
CPU specific
p
state of this task. see ((include/asm-arm/processor.h
p
l. 64))
„ struct fs_struct *fs
z
file system info (current working directory, etc.)
„ struct files_struct
files struct *files
z
open file info
Operating System Concepts – 8th Edition
21.26
Silberschatz, Galvin and Gagne ©2009
Process context
„ Normal program execution occurs in user space (like Nachos)
„ Exception or system call switches execution to kernel space (like Nachos)
z
still executing in the process user context
„ Interrupt causes context switch to interrupt context
z
This is a kernel thread with no associated virtual address space
Operating System Concepts – 8th Edition
21.27
Silberschatz, Galvin and Gagne ©2009
System Calls
„ User program makes call: e.g. read( ... )
„ read() is a C library wrapper (glibc, typically)
„ library generates "software interrupt" (0x80 on intel
architectures) with the appropriate system call number
„ This causes switch to kernel space and execution of
system call handler system_call() (which is actually an
assembly
asse
b y language
a guage routine)
out e)
„ This looks up the appropriate method to call for the
supplied system call number, e.g. sys_read(...)
„ Very similar to Nachos, except the stubs are hidden by a C
library
Operating System Concepts – 8th Edition
21.28
Silberschatz, Galvin and Gagne ©2009
Adding a new system call
„ Add an entry to the end of the system call table (see
linux/arch/armnommu/kernel/calls.S) for your system call:
#ifndef NR
NR_syscalls
syscalls
#define NR_syscalls 288
#else
__syscall
y
_start:
/* 0 */
/* 5 */
.long
SYMBOL_NAME(sys_ni_syscall)
.long
SYMBOL_NAME(sys_exit)
.long
SYMBOL_NAME(sys_fork_wrapper)
.long
SYMBOL_NAME(sys_read)
.long
SYMBOL_NAME(sys_write)
.long
SYMBOL_NAME(sys_open)
.long
SYMBOL_NAME(sys_madvise)
.long
long
SYMBOL NAME(sys fcntl64)
SYMBOL_NAME(sys_fcntl64)
...
/* 220 */
__syscall_end:
.rept
NR_syscalls - (__syscall_end - __syscall_start) / 4
.long
g
SYMBOL_NAME(sys
( y _ni_syscall)
y
)
.endr
Operating System Concepts – 8th Edition
21.29
Silberschatz, Galvin and Gagne ©2009
Adding a new system call
„ For a real system call that was generally useful, you'd do this in the
subdirectories for all of the supported architectures. We're only interested in
the iPod processor
„ Add the resulting system call number to include/asm-armnommu/unistd.h
#define __NR_mincore
(__NR_SYSCALL_BASE+219)
#define __NR_madvise
(__NR_SYSCALL_BASE+220)
#define __NR_fcntl64
(__NR_SYSCALL_BASE+221)
„ Now,, your
y
syscall
y
has to be compiled
p
into the kernel. To avoid having
g to
frob with the Makefile, you can add your code to
arch/armnommu/kernel/sys_arm.c, source code for miscellaneous "special"
system calls
Operating System Concepts – 8th Edition
21.30
Silberschatz, Galvin and Gagne ©2009
Using your new sys call
„ You could modify the source for glibc to add the appropriate library call
„ OR, use the provided _syscalln() macros, where n is in the range 0 to 6
and indicates how many parameters the sys call will pass
„ So, for an imaginary sys call # 42, reopen, with the following signature:
long reopen(const char *filename, int flags, int mode)
we would add
z
#define __NR_reopen 42
_syscall3(long,
syscall3(long reopen,
reopen const char *,
* filename,
filename
int flags, int mode)
to the program that wanted to call reopen()
„ See IBM DeveloperWorks article for details
Operating System Concepts – 8th Edition
21.31
Silberschatz, Galvin and Gagne ©2009
Kernel programming tips
„ Example from Love:
/*
* silly_copy - utterly worthless syscall that copies the len bytes from
* 'src' to 'dst' using the kernel as an intermediary in the copy for no
* good reason. But it
i makes for a good example!
! */
/
asmlinkage long sys_silly_copy(unsigned long *src, unsigned long *dst,
unsigned long len) {
unsigned long buf;
/* fail if the kernel wordsize and user wordsize do not match */
if (len != sizeof(buf))
return -EINVAL;
/* copy src, which is in the user's address space, into buf */
if (copy_from_user(&buf, src, len))
return -EFAULT;
/* copy buf into dst, which is in the user's address space */
if (copy_to_user(dst, &buf, len))
return
t
-EFAULT;
EFAULT
/* Return amount of data copied */
return len;
}
Operating System Concepts – 8th Edition
21.32
Silberschatz, Galvin and Gagne ©2009
Compiling/Installing the kernel
„ Unless you have added files, compiling should be simple – just type "make"
in the top level of the source tree.
„ There will be an executable called "linux" at the top level of the source tree.
This isn't the one you want.
„ Copy arch/armnommu/boot/Image to linux.bin in the root of the Linux
partition
titi on the
th iPod
iP d
„ Reboot the iPod – hold down "Menu" on the click wheel and the center
button together for several seconds
Operating System Concepts – 8th Edition
21.33
Silberschatz, Galvin and Gagne ©2009
Compiling user-level programs
„ You are compiling on an x86 platform and need to produce an executable
for the ARM processor on the iPod
„ Department Linux boxes have the appropriate cross-compiler installed: arm-
elf-gcc. This is the 2.95 toolchain.
„ See http://www.ipodlinux.org/wiki/Toolchain if you'd like to install on your
own system
t
„ To compile myuserprog.c:
arm-elf-gcc –o myuserprog myuserprog.c –elf2flt
„ The –elf2flt creates the executable format expected by the iPod
„ You will have to copy onto the iPod from a Ubuntu system
„ You can use /home on the iPod's
iPod s Linux partition for your user programs.
Remember to use sudo when you do your copy
„ Check permissions before copying – should be 755 (all users have
read/exec))
„ arm-elf-gcc produces two executables, use the one without .gdb ext.
Operating System Concepts – 8th Edition
21.34
Silberschatz, Galvin and Gagne ©2009
Executing user-level programs
„ One of the top-level choices in podzilla is "File Browser"
„ Use it to browse to your user program
„ With your executable selected, press AND HOLD the center button
„ Select Execute
„ Select "Read
Read Output"
Output , unless you're
you re trying something interactive
„ Press MENU on the clickwheel to exit
Operating System Concepts – 8th Edition
21.35
Silberschatz, Galvin and Gagne ©2009
Resources
„ http://www.netlibrary.com/library_home_page.asp - you can get to Robert
Love's Linux Kernel Development through here. See Ch. 5
„ http://www.ibm.com/developerworks/linux/library/l-system-calls/ - Nice
article from IBM DeveloperWorks about writing new sys calls
„ http://lxr.linux.no/linux-old+v2.4.24// - Linux Cross reference, searchable
version
i off the
th kkernell source code
d
„ http://www.faqs.org/docs/kernel_2_4/lki.html - Linux Kernel Internals by
Tigran Aivazian
„ http://www.gelato.unsw.edu.au/~dsw/public-files/kernel-docs/kernel//
/
/
f /
/
api/index.html - Linux Kernel API
Operating System Concepts – 8th Edition
21.36
Silberschatz, Galvin and Gagne ©2009