Download Dept. of CSE, BUAA

Unix Programming Environment
Part 2 – An Introduction to Unix Systems
Dept. of CSE
BUAA
Agenda
 1. Overview
 2. A Brief History of Unix Systems
 3. Organizations and Standards about Unix
 4. Unix Nowadays
 Internal structure of modern Unix, system interfaces, and tools
 5. Summary - Unix’s legend
 6. The mandate for change
 7. Looking back, looking forward
 8. Supplementary Readings
Unix Programming Environment
Dept. of CSE, BUAA
1. Overview (1)
 Since its inception in 1969, the Unix system has grown into a
versatile operating platform:
 small embedded processors
 workstations and desktop systems
 high-performance multi-processor systems
 “The UNIX system” consists of a collection of user programs,
libraries, and utilities, running on the UNIX operating system kernel,
which provides a run-time environment and system services for
these applications.
 Baseline releases:
 System V Release 4( SVR4)
 UC’s Berkeley Software Distribution (4.xBSD )
 Linux
 CMU’s Mach
Unix Programming Environment
Dept. of CSE, BUAA
1. Overview (2) - How UNIX is organized
The UNIX system is functionally
organized at three levels:
 The kernel, which schedules
tasks and manages storage;
 The shell, which connects and
interprets users' commands, calls
programs from memory, and
executes them;
 The tools and applications that
offer additional functionality to the
operating system
 Tools can be added or removed
from a UNIX system, depending
upon the applications required.
Unix Programming Environment
Dept. of CSE, BUAA
2. A Brief History of Unix (1)
 2.1 The Beginning
 In the late 1960s, BTL & GE & MIT:
Multics: Multiplexed Information and Computing System
 One of the principal developers, Ken Thompsons, Game program
called “Space Travel”:
PDP-7: lacking a program development environment
Honeywell 635 running the GECOS: cross-assembler
 Ken Thompsons & Dennis Ritchie: develop an operating
environment for PDP-7
A file system: the ancestor of s5fs
A process subsystem: like “fork”
A simple command interpreter called the “shell”
Brian Kernighan called this system as “Unics”( Uniplexed
Information and Computing system) => “UNIX”
Unix Programming Environment
Dept. of CSE, BUAA
2. A Brief History of Unix (2)
Unix Programming Environment
Dept. of CSE, BUAA
2. A Brief History of Unix (2)
 Develop a text-processing system for the patent department at BTL:
Port UNIX to the PDP-11
Several text-processing utilities including “ed”, “runoff”, etc
A new language called “B”:
 BCPL, CPL, Algol60
 B: an interpretive language
 To improve the performance, Ritchie evolved it into “C”: compilable and supporting
data types and data structures
 1973 – 1974: having a tremendous impact on Unix’s future success
Unix Programmer Manual
Unix was rewritten in C( resulting in v4 )
The first Unix paper on ACM SOSP – “The Unix Time Sharing
System”( Supplementary reading )
Distributing Unix to outside including its source code
Unix Programming Environment
Dept. of CSE, BUAA
2. A Brief History of Unix (3)
 2.2 Proliferation
 As a result of antitrust litigation by the Dept. of Justice, AT&T
distributed Unix for educational and research purpose royaltyfree: one of such licensees was U.C. at Berkeley at 1973.
 Mel Ferentz: <Unix NewsLetter> => USENIX Conference
 John Lions from Australia: lecture notes => <A commentary on
the UNIX operating System>
 Many people ported UNIX to various architectures: Interdata 8/32,
Interdata 7/31, VM/370, VAX-11( UNIX/32V ), Intel 8086( by SCO &
MS ), etc
The spirit of cooperation between its keepers and users was a
key factor in the rapid growth and rising popularity of UNIX.
Version 7 Unix, released in 1979, was the first truly portable
UNIX system.
Unix Programming Environment
Dept. of CSE, BUAA
2. A Brief History of Unix (4)
 2.3 Baseline Release 1: BSD
 Utilities: ex->vi, a Pascal compiler, C Shell( job control, command
history )
 1978: VAX-11/780( 32-bit architecture, 4G address space, but only
2M physical memory) => a paging-based virtual memory system
for UNIX
 The Defense Advanced Research Projects Agency( DARPA )
funded the development of Unix systems at Berkeley:
Integrating TCP/IP network protocol suite => Internet
 4.4BSD is the last release from CSRG at Berkeley.
FreeBSD and OpenBSD, basing on 4.4BSD, still evolve now.
 Contributions from BSD: VM, TCP/IP, Fast file system( FFS), a
reliable signals implementation, and the sockets facility, a logstructured file system, etc
Unix Programming Environment
Dept. of CSE, BUAA
2. A Brief History of Unix (5)
 2.4 Baseline Release 2: System V
 After “baby Bells” were born, AT&T was allowed to enter the
computer business in 1982.
 AT&T marketed Unix aggressively: System III at 1982, System V
in 1983, SVR2 in 1985, SVR3 in 1987:
A VM implementation quite different from that of BSD:
“regions”(!!!)
System V IPC: shared memory, semaphores and message
queues
The STREAMS framework for device drivers and network
protocols: XTI
Shared libraries, remote file sharing, etc
Unix Programming Environment
Dept. of CSE, BUAA
2. A Brief History of Unix (6)
 SVR4, developed jointly by AT&T and Sun, was first released in
1989:
Integrating features from SVR3, 4BSD, SunOS and XENIX
Virtual File System( VFS ) and Virtual Memory( VM ) from Sun
Real-time scheduling classes
Supporting Symmetric Multiple Processors( SMP )
 Kernel-level threads
 Fine-grained locks
Unix Programming Environment
Dept. of CSE, BUAA
2. A Brief History of Unix (7)
 2.4 Baseline Release 3: Mach
 The Unix kernel is small and simple, yet offered many useful
facilities. As more and more features were incorporated into the
kernel, the internal elegancy disappeared.
 The microkernel approach: export a few simple abstractions, do
provide most of the functionality thorough a collection of userlevel tasks.
 But the performance becomes the critical hurdle for the
“microkernel” architecture.
Unix Programming Environment
Dept. of CSE, BUAA
3. Organizations and Standards
 Organization:
 UI & OSF
Unix International( UI ): AT&T and Sun, System V-based
systems
Open Software Foundation( OSF ): Motif, OSF/1, DCE
 USENIX:
 But the growth, and even survival, of UNIX has been jeopardized
by Microsoft Windows.
 Standards:
 System V Interface Description( SVID ) from USL
 POSIX from IEEE
 XPG from X/Open
Unix Programming Environment
Dept. of CSE, BUAA
4. Unix Nowadays (1)
Internal Structure of Traditional Unix
Unix Programming Environment
Dept. of CSE, BUAA
4. Unix Nowadays (2)
Internal Structure of modern Unix
Unix Programming Environment
Dept. of CSE, BUAA
4. Unix Nowadays (3)
Unix Programming Environment
Dept. of CSE, BUAA
4. Unix Nowadays (4)
 Summary
 Architecture: Functions,
Internal Organization
 Dynamic loading
Unix Programming Environment
Dept. of CSE, BUAA
4. Unix Nowadays (5) - Standards (1)
Unix
Unix Programming Environment
Dept. of CSE, BUAA
4. Unix Nowadays (6) - Standards (2)
 X/Open Common Applications Environment (CAE)
Portability Guide Issue 3 (XPG3) and Issue 4 (XPG4)
 SUS( Single UNIX Specification ), SUSv2
 XNS4: Networking Services Issue4
 http://www.unix-systems.org
 Notes:
 1. The developers of SVID3( UNIX Systems Laboratories) are no
longer in business, and this specification defers to POSIX and
X/Open CAE.
 2. POSIX Standards: we can get them from the library.
Unix Programming Environment
Dept. of CSE, BUAA
4. Unix Nowadays (7) - Standards (3)
POSIX.1
XPG.3
BSD
Interfaces
XPG.4
XNS4
POSIX.2
POSIX
Standard
POSIX.1b
SUS(
XPG4v2)
ISO-C
Amendement 1
POSIX.1c
SVR4
XPG.3
LP64-clean
derivative of
XNS4
SUSV2
XNS5
Solaris8
SUSV2
Unix Programming Environment
Dept. of CSE, BUAA
4. Unix Nowadays (8)
 Software Development Toolset for Unix
 GNU Development Toolset – gcc, ld, gdb, building system
Unix Programming Environment
Dept. of CSE, BUAA
5. Unix’s Legend (1)
Unix Programming Environment
Dept. of CSE, BUAA
5. Unix’s Legend (2)
Unix Creators & Gurus
Brian Kernighan
Dennis Ritchie
Unix Programming Environment
Ken Thompson
Bill Joy
Linus Torvalds
Dept. of CSE, BUAA
What’s GNU?
Richard Stallman – the founder and author of many GNU programs
Free Software Foundation
GPL - Copyright is called…Copyleft.
Linux => GNU/Linux
Unix Programming Environment
Dept. of CSE, BUAA
6. The Mandate for Change
 1. Functionalities:
 Pipe -> System V IPC -> POSIX IPC
 Multithread
 2. Networking
 TCP/IP Protocol Stack
 Distributed File System: NFS from Sun, AFS/Coda from CMU
 RPC-based Services: NIS from Sun, DCE from OSF
 3. Performance
 FFS, Log FS
 SMP Architecture
 4. Computation Paradigm Shifts
 A Host + terminals – networked workstations – C/S model –
Distributed object/component-based computation
Unix Programming Environment
Dept. of CSE, BUAA
7. Looking back, Looking forward( 1 )
 1. “Small is beautiful”:
 2. Flexibility: the core is a framework
 Simplicity, clarity and flexibility
 Methodology: combination, interface-oriented
 3. what’s wrong with Unix?
 Although initially simple, it did not remain the way: Standard I/O
 No uniform user interfaces
Ritche said: “Unix is simple and coherent, but it takes a genius to
understand and appreciate its simplicity.”
The building-block approach requires creativity and imagination to
use effectively.
cat menu | grep shrimp | test -lt $10
 Because of too many value-added features, Unix became chaotic.
Standards like POSIX are partially successful now.
Unix Programming Environment
Dept. of CSE, BUAA
7. Looking back, Looking forward( 2 )
 Philosophy Matters and Traditions in Unix
 Lessons of Unix can be applied in our programming practice
 ‘KISS Principle’: Keep it simple, stupid! / Simplicity, clarity and generality
Write small pieces connected by clean interfaces.
Design programs to communicate easily with other programs.
Robustness is the child of transparency and simplicity.
Design for simplicity; add complexity only where you must.
Design for transparency; spend effort early to save effort later.
In interface design, obey the Rule of Least Surprise.
Programmer time is expensive; conserve it in preference to machine
time.
Avoid hand-hacking; write programs to write programs when you
can.
Use smart data so program logic can be stupid and robust.
Prototype, then polish. Get it working before you optimize it.
Unix Programming Environment
Dept. of CSE, BUAA
7. Looking back, Looking forward( 3 )
Everything that can be a device-independent filter should be.
Data streams should, if at all possible, be textual (so they can
be viewed and filtered with standard tools).
Database layouts should if at all possible be textual (humaneditable).
Application protocols should if at all possible be textual
(human-readable).
Complex front ends (user interfaces) should be cleanly
separated from complex back ends.
Whenever possible, prototype in an interpretive language
before coding C.
Mixing languages is better than writing everything in one, if
using only that one will over-complicate the program.
Unix Programming Environment
Dept. of CSE, BUAA
8. Supplementary Readings
 The UNIX Time-Sharing System
Dennis M. Ritchie and Ken Thompson
 Early Unix history and evolution
Dennis M. Ritchie
 Themes in Operating Systems
Dennis M. Ritchie
 25th Anniversary of Unix
 An introduction to Solaris
Unix Programming Environment
Dept. of CSE, BUAA