Download file (1.5 MB, ppt)

1
Course Staff
B.S.Mahanand .B.E.,(M.Tech).,
Dept of Computer Science and Engg
S.J.C.E.
Mysore.
Introduction to Unix
2
Course Texts
Text Book: Unix Concepts &Applications
By Sumitabha Das
Reference: Learning Unix - for those
who have not used Unix before.
Introduction to Unix
3
Operating Systems
 An Operating System controls (manages) hardware
and software.
 provides support for peripherals such as keyboard,
mouse, screen, disk drives, …
 software applications use the OS to communicate with
peripherals.
 The OS typically manages (starts, stops, pauses, etc)
applications.
Introduction to Unix
4
Single vs. Multitasking
 Some old operating systems could only do one thing at
a time (DOS).
 Most modern systems can support multiple
applications (tasks) and some can support multiple
users (at the same time).
 Supporting multiple tasks/users means the OS must
manage memory, CPU time, network interfaces, ...
Introduction to Unix
5
User Interfaces
 The User Interface is the software that supports
interactions with a human.
 Some operating systems directly provide a user
interface and some don't.
 Windows is an example of an Operating System that
includes a user interface.
 Unix (the OS) does not directly provide a user
interface.
Introduction to Unix
6
What is UNIX?
 UNIX is a popular operating system (OS) in both
engineering and business world
 OS is a “super program” that controls sharing of
resources and communication between machines
 E.g., Windows, OS/2, VMS, MacOS, UNIX
 UNIX is available for all platforms
Introduction to Unix
7
Unix and Users
 Most flavors of Unix (there are many) provide the
same set of applications to support humans
(commands and shells).
 Although these user interface programs are not part of
the OS directly, they are standardized enough that
learning your way around one flavor of Unix is enough.
Introduction to Unix
8
Flavors of Unix
 There are many versions of Unix that are used by lots
of people:
 SysV (from AT&T)
 BSD (from Berkeley)
 Solaris (Sun)
 IRIX (SGI)
 AIX (IBM)
 LINUX (free software)
Introduction to Unix
9
Unix History and Motivation
 The first version of Unix came from AT&T in the early
1970s (Unix is old!).
 Unix was developed by programmers and for
programmers.
 Unix is designed so that users can extend the
functionality - to build new tools easily and efficiently
(this is important for programmers).
Introduction to Unix
10
Unix History and Motivation
 K Thomson and D Ritchie at Bell Labs
 Wrote first version in assembly then the next version
in C; the first “understandable” OS!
 UC Berkeley graduate students
 Enhanced with memory management and
networking capabilities (BDS UNIX)
 UNIX Today: contain both features
 Commercial UNIX
 Linux
Introduction to Unix
11
Some Basic Concepts
 Unix provides a simple interface to peripherals (it's
pretty easy to add support for a new peripheral).
 Unix includes a basic set of commands that allow the
user to view/change the system resources (filesystem,
processes, peripherals, etc.).
Introduction to Unix
12
What we will look at
 In this course we will learn about:
 Unix user accounts
 the core set of Unix commands
 the Unix filesystem
 A couple of special programs called "shells".
 A number of commonly used applications:

Window system, text editors, programming tools.
Introduction to Unix
13
The power of Unix is that you can
extend the basic commands
 We will also look at how to extend the basic
functionality of Unix:
 customize the shell and user interface.
 string together a series of Unix commands to create new
functionality.
 create custom commands that do exactly what we want.
Introduction to Unix
14
UNIX Services
 UNIX facts about programs and files
 A file is a collection of data stored on disk
 A program is a collection of bytes representing code and
data that is stored in a file
 When a program is started, it is loaded from disk into
RAM. A “running” program is called a process
 Processes and files have an owner and a group, and they
protected from unauthorized access
 UNIX supports a hierarchical directory structure
 Files have a location within the directory hierarchy
Introduction to Unix
15
UNIX Services
 Sharing resources
 UNIX shares CPUs among processes by dividing CPU time
into slices (typically 0.1 second) and allocating time slices
based on their priorities
 UNIX shares memory among processes by dividing RAM
into equal-sized pages (e.g., 4K bytes) and allocating them;
only those pages of a process that are needed in RAM are
loaded from disk while pages of RAM that are not accessed
are saved back to disk
 UNIX shares disk space among users by dividing disks into
equal-sized blocks and allocating them
Introduction to Unix
16
UNIX Services
 UNIX provides communication mechanism between
processes and peripherals
 Pipe is one-way medium-speed data channel that allows
two processes on the same machine to talk
 Socket is two-way high-speed data chanel that allows two
processes on different machines to talk
 UNIX allows server-client model of communication

e.g., X Window systems
Introduction to Unix
17
UNIX for Users
 Basic commands
 man, ls, cat, more, page, head, tail, less
 mv,cp,rm,ln
 pwd,cd,mkdir,rmdir,file
 chmod, groups, chgrp
 Editing utilities and terminal type
 vi, emacs
Introduction to Unix
18
Getting Started in UNIX
 Obtain an account and logging-in
 passwd: utility for changing password
 Shells
 A middleman program between you and UNIX OS
that executes your commands

Shell commands vs. utilities vs. system calls
 Three popular shells: Bourne, Korn, C
 Shell has a programming language that are tailored
to manipulate processes and files; such program is
called shell script
Introduction to Unix
19
Utilities for Manipulating File
 mv, cp, rm, ln
 move,copy,remove,make links to files
mv [-fi] soutce targetFile
.. rename source filename
mv [-fi] soutceList targetDirectory
.. move source files into target
directory
cp [-fip] soutceFile targetFile
.. make copy of sourceFile
named targetFile
cp [-fip] soutceFileList targetDirectory .. copy source files into target
directory
cp -rR [-fip] soutceDirList targetDirectory
.. copy source files
into target directory, and copy the directory structure.
rm [-fi] fileList
rm -rR [-fi] directoryList
ln [-fns] source [target]
ln [-fns] sourceList targetDirectory
.. make links to sour files into
target directory
 file - determine file type
hello.c:
$ file hello.c
c program text
Introduction to Unix
20
Utilities for Hanling Directory
 pwd, cd, mkdir, rmdir
 print working directory
 change directory

cd, pushd, popd
 make directory
 remove directory
Introduction to Unix
21
Vi - Editor
 vi is a visual text editor.
 Most of you are too young to understand what a non-
visual editor might be!
 check out the ed editor!
 vi shows you part of your file and allows you to enter
commands that change something (add new stuff,
delete a char or line, etc).
Introduction to Unix
22
vi modes
vi has a couple of modes:
 command mode: move the cursor around, move to a
different part of the file, issue editing commands,
switch to insert mode.
 insert mode: whatever you type is put in the file (not
interpreted as commands).
when you first start vi you will be in command
mode.
Introduction to Unix
23
Cursor Movement Commands
(only in command mode!)
h
l
j
k
move left one position
move right one position
move up one line
move down one line
Your arrow keys might work (depends on the version of
vi and your terminal)
Introduction to Unix
24
More Cursor Movement
w
b
e
)
(
move forward one word
move backward one word
move to the end of the word
move to beginning of next sentence
move to beginning of current sentence
Introduction to Unix
25
Scrolling Commands
CTRL-F
CTRL-B
CTRL-D
CTRL-U
scroll forward one screen
scroll backward one screen
scroll forward 1/2 screen
scroll backward 1/2 screen
Introduction to Unix
26
Command that delete stuff
x
dw
dd
X
3x
5dd
delete character (the one at the cursor)
delete word
delete line
delete back one character (backspace)
delete 3 characters (any number works)
delete 5 lines (any number works)
Introduction to Unix
27
Changing Text
cw
cc
C
rx
change word (end with Esc)
change line (end with Esc)
change rest of the line
replace character with 'x' (could be anything, not
just 'x')
Introduction to Unix
28
Insert Mode
 In insert mode whatever you type goes in to the file.
There are many ways to get in to insert mode:
i
insert before current position
a
A
R
append (insert starting after cursor)
append at end of line
begin overwriting text
Introduction to Unix
29
Ending Insert Mode
 To get out of insert mode (back to command mode)
you press "Esc" (the escape key).
 There is a status line (bottom of screen) that tells you
what mode/command you are in.
Introduction to Unix
30
Saving and Exiting
ZZ save if changes were made, and quit.
:wq Write file and quit
:w Write file
:w file Write to file named file
:q Quit
:q! Really quit (discard edits)
Introduction to Unix
31
Searching Commands
/text
?text
n
N
search forward for text
search backward for text
repeat previous search
repeat search in opposite direction
Introduction to Unix
32
Other Stuff
 Copying and Yanking (Paste)
 Remembering positions
 Switching files
 Repeating commands
 Display line numbers
 Run Unix commands (for example: emacs)
Introduction to Unix
33
Emacs -Editor
 emacs is every bit as cryptic as vi!
 emacs allows you to customize it with new
commands and keyboard shortcuts.
 The emacs commands are written in elisp (a dialect
of Lisp), so you need to understand elisp to do
serious customization.
Introduction to Unix
34
Emacs meta key
 Many emacs commands are invoked with sequence
of keystrokes.
 Emacs doesn't have modes like vi – you can always
enter text (at the current cursor position) or
commands.
 Many commands start with a special keystroke
called the metakey. (others use the control key).
 The ESC key is (usually) the meta key.
Introduction to Unix
35
Command List Syntax
 The book shows a list of tons of emacs commands. The
syntax used to show this list looks like this:
C-a C-b (means Ctrl-a , Ctrl-b)
M-a M-b (means Esc, a, Esc, b)
Introduction to Unix
36
Important Commands
 Exit: C-x C-c
 Save file : C-x C-s
 Undo: C-x u
 Get out of a command: C-g
Introduction to Unix
37
Cursor movement
 Cursor keys usually work (it depends on how your
terminal is set up and how emacs is configured).
C-f: forward (right arrow)
C-b: backward (left arrow)
C-p: previous line (up arrow)
C-n: next line (down arrow)
Introduction to Unix
38
Other stuff in emacs
 Move by words, sentences, paragraphs
 File handling – save/load file, etc.
 Delete char, word, sentence, region
 Buffer manipulation (multiple buffers)
 Searching, replacing
 Automatic indentation (major mode)
 Lots more (try the tutorial, read the book!)
Introduction to Unix
39
The Environment
The Unix system environment can be set by user
System variables:
Unix system is controlled by number of shell variables
set by system called system variables which can be
seen by executing $set command
ExHOME,PATH,IFS,MAIL,PS1,PS2,SHELL,TERM,LOGN
AME
Introduction to Unix
40
.profile : It is the shell script executed during login
time.
stty : To set the terminal characteristics
history :It displays previously executed commands.
Introduction to Unix
41
histsize : Used to store more commands
$_ : Stores the last arguments of the last command
Changing Directory : ~(tilde)
~ acts as short hand representation of home
Directory.
Introduction to Unix
42
UNIX Utilities: man & ls
 man
 on-line help for UNIX commands
man [-s section] word
man -k keyword
 Man pages are divided into 8 sections


1. Commands & application programs
2. System calls, 3. Library functions ...
 ls
 list files
ls [options] {fileName}*
Introduction to Unix
43
UNIX Utilities
 ls example
$ ls -alFs
total 15
1 drwxr-xr-x 2 jaemok mass
3 drwxr-xr-x 46 jaemok mass
5 -rwxr-xr-x 1 jaemok mass
5 -rwxr-xr-x 1 jaemok mass
1 -rw-r--r-- 1 jaemok mass
5
drwxr-xr-x
1
jaemok
mass
5091
Mar 2 01:04
a.out
512 Mar 2 01:04 ./
3072 Mar 2 01:04 ../
5091 Mar 2 01:04 a.out*
5091 Mar 2 01:04 hello*
36 Mar 2 01:04 hello.c
size in block
type and file permission
hard-link count
owner
group
size in bytes
modified time
filename
 Miscellaneous: date, clear
Introduction to Unix
44
Utilities to List File Contents
 cat,more,page,head,tail,less
cat displays contents of files given on command line (or from
standard input) to standard output
$ cat > heart
blah blah blah
^D
$_
more and page displays contents of files(or input) fitted to
rows of terminal, and allows you to scroll.
head and tail only displays given number of lines in the first
part or the last part of files(or input).
less is similar to more, but has more functions
Introduction to Unix
45
File Owner and Group
Every UNIX process has a owner which is typically the
user name who started it

My login shell’s owner is my user name
When a process creates a file, its owner is set to the
process’ owner
UNIX represents the user name as user ID
Every UNIX user is a member of a group and an UNIX
process belongs to a group

My login shell’s group is my group name (ID)
Introduction to Unix
46
File Permissions
 File Permissions (ex: rw-r--r--)
 owner: rw-, group: r--, others: r- r: read, w: write, x: execute
 When a process executes, it has four values related
to file permission
 a real user ID, an effective user ID
 a real group ID, an effective group ID
 When you login, your login shell process’ values are
your user ID and group ID
Introduction to Unix
47
File Permission Application
When a process runs, file permission applies
If process’ effective user ID = file owner

Owner permission applies
If process’ effective group ID=file group ID

Group permission applies
Otherwise, others permission applies
Super user’s process automatically has all access rights
When creating a file, umask affect permission
Introduction to Unix
48
Effective User and Group ID
 A process’ effective user ID
 depends on who executes the process, not who owns the
executable
 E.g., if you run passwd (owned by root), the effective user ID
is your ID, not root; then how can it update /etc/passwd file
owned by root ?
 Two special file permissions



set user ID and set group ID
When an executable with set user ID permission is executed, the
process’ effective user ID becomes that of executable; the real user
ID is unaffected
File permission of /bin/passwd is r-sr-sr-x
Introduction to Unix
49
Changing File Permissions
 chmod
 change the permission mode of files
chmod [-fR] <absolute-mode> fileList
chmod [-fR] <symbolic-mode-list> fileList
Introduction to Unix
50
chmod
 absolute-mode is octal
number
if you want a file to have permission setting: rwxr- x--then, it is an octal number o750 (111 101 000)
$ chmod 750 myfile
 symbolic-mode-list is comma-separated list of symbolic
expressions of form
[who] operator [permissions]
who : characters u,g,o,a
operator : -,=.+
permissions : r,w,x,s
$ chmod g-rx myfile
Introduction to Unix
51
Text Handling Commands
 There are many Unix commands that handle textual
data:
 operate on text files
 operate on an input stream
 Functions:
 Searching
 Processing (manipulations)
Introduction to Unix
52
Searching Commands
 grep, egrep, fgrep : search files for text
patterns
 strings: search binary files for text strings
 find: search for files whose name matches a pattern
Introduction to Unix
53
grep - Get Regular Expression
grep [options] regexp [files]
regexp is a "regular expression" that describes
some pattern.
files can be one or more files (if none, grep reads
from standard input).
Introduction to Unix
54
grep Examples
 The following command will search the files a,b
and c for the string "foo". grep will print out any
lines of text it finds (that contain "foo")
grep foo a b c
 Without any files specified, grep will read from
standard input:
grep I
Introduction to Unix
55
Regular Expressions
 The string "foo" is a simple pattern.
 grep actually understands more complex patterns that
are described using regular expressions.
 We will look at regular expressions used by grep and
other programs later.
 In case you can't wait - here is a sample:
grep "[A-Z]0{2,3}" somefile
Introduction to Unix
56
grep options
-c
-h
-l
-n
-v
print only a count of matched lines.
don't print filenames
print filename but not matching line
print line numbers
print all lines that don't match!
Introduction to Unix
57
grep, egrep and fgrep
 All three search files (or stdin) for a text pattern.
 grep supports regular expressions
 egrep supports extended regular expressions
 fgrep supports only fixed strings (nothing fancy)
 All have similar forms and options.
Introduction to Unix
58
strings
 The strings command searches any kind of file
(including binary data files and executable programs)
for text strings, and prints out each string found.
 strings is typically used to search for some text in a
binary file.
 strings [options] files
Introduction to Unix
59
The find command
 Find searches the filesystem for files whose name
matches a pattern*.
 Here is a simple example:
find . -name unixtest -print
 *Actually find can do lots more!
Introduction to Unix
60
Text Manipulation
 There are lots of commands that can read in text (from
files or standard input) and print out a modified
version of the input.
 Some possible examples:
 force all characters to lower case
 show only the first word on each line
 show only the first 10 lines
Introduction to Unix
61
Common Concepts
 These commands are often used as filters, they read
from standard input and send output to standard
output.
 Different commands for different specific functions
 another way is to build one huge complex command
that can do anything. This is not the Unix way!
Introduction to Unix
62
Simple Filters
head tail - show just part of a file
cut paste join - deal with columns in a text file.
sort - reorders the lines in a file
tr - translate characters
uniq - find repeated or unique lines in a file.
Introduction to Unix
63
head or tails?
 head shows just the "head" (beginning) of a file.
 tail shows just the "tail" (end) of a file.
 Both commands assume the file is a text file.
Introduction to Unix
64
The head command
head [options] [files]
By default head shows the first 10 lines.
Options:
-n print the first n lines.
Example:
head -20 /etc/passwd
Introduction to Unix
65
The tail command
tail [options] [files]
By default tail shows the last 10 lines.
Options:
-n print the last n lines.
-nc print the last n characters
+n print starting at line number n
+nc print starting at character number n
Introduction to Unix
66
The tail command (cont.)
More Options:
-r show lines in reverse order
-f don't quit at end of file.
Examples:
tail -100 somefile
tail +100 somefile
tail -r -c somefile
Introduction to Unix
67
The cut command
 cut selects (and prints) columns or fields from lines of
text.
cut options [files]
 You must specify an option!
Introduction to Unix
68
cut options
-clist cut character positions defined in list.
list can be:
number (specifies a single character position)
range (specifies a sequence of positions)
comma separated list (specifies multiple positions or
ranges)
Introduction to Unix
69
cut -c examples
cut -c1
cut -c1-10
cut -c1,10
prints first char. (on each line).
prints first 10 char
prints first and 10th char.
cut -c5-10,15,20prints 5,6,7,8,9,10,15,20,21,… char on each line.
Introduction to Unix
70
more cut options
-flist cut fields identified in list.
a field is a sequence of text that ends at some separator
character (delimiter).
You can specify the separator with the -d option. -dc
where c is the delimiter.
The default delimiter is a tab.
Introduction to Unix
71
Specifying a delimiter
cut -d: -f1
prints everything before the
first ":" (on each line).
What if we want to use space as the delimiter?
cut -d" " -f1
Introduction to Unix
72
cut -f examples
cut -f1
prints everything before the first tab.
cut -d: -f2,3 prints 2nd and 3rd : delimited columns.
cut -d" " -f2 prints 2nd column using space as the
delimiter.
Introduction to Unix
73
The paste command
 paste puts lines from one or more files together in
columns and prints the result.
paste [options] files
 The combined output has columns separated by tabs.
Introduction to Unix
74
paste cands votes
cands
Gore
Bradley
Bush
McCain
Trump
Letterman
votes
10
10
10
10
10
100
Introduction to Unix
Gore
Bradley
Bush
McCain
Trump
Letterman
10
10
10
10
10
100
75
paste options
-dc separate columns of output with character c.
you can use different c between each column.
-s
merge subsequent lines from a single file.
Introduction to Unix
76
paste -s -c"\t\n" records
records
Gore
10
Bradley
10
Bush
10
Letterman
100
Gore
Bradley
Bush
Letterman
10
10
10
100
paste -s -c"\t\t\n" records
Gore
10
McCain
100
Introduction to Unix
10
Bradley
Bush 10
10
Letterman
77
The join command
 join combines the common lines of 2 sorted files.
 Useful for some text database applications, but not a
very general command.
 Look at examples in the book if you are interested.
Introduction to Unix
78
The sort command
 sort reorders the lines in a file (or files) and prints
out the result.
sort [options] [files]
Introduction to Unix
79
sort options
-b
-d
-n
-r
ignore leading spaces and tabs
sort in dictionary order (ignore punctuation)
sort in numerical order
reverse the order of the sort
tons more options!
Introduction to Unix
80
Numeric
vs.
Alphabetic
By default, sort uses an alphabetical ordering.
sort
1256875
18
27
38
66
875
38
18
27
1256875
66
875
Introduction to Unix
sort -n
18
27
38
66
875
1256875
81
Alphabetic Ordering (uses ASCII)
'0' < '9' <'A' < 'Z'< 'a' < 'z' <
bbbb
BBBB
aaaa
AAAA
0000
####
$$$$
Introduction to Unix
sort
####
$$$$
0000
AAAA
BBBB
aaaa
bbbb
82
ASCII codes
32:
33:! 34:" 35:#
40:( 41:) 42:* 43:+
48:0 49:1 50:2 51:3
56:8 57:9 58:: 59:;
64:@ 65:A 66:B 67:C
72:H 73:I 74:J 75:K
80:P 81:Q 82:R 83:S
88:X 89:Y 90:Z 91:[
96:` 97:a 98:b 99:c
104:h 105:i 106:j 107:k
112:p 113:q 114:r 115:s
120:x 121:y 122:z 123:{
Introduction to Unix
36:$
44:,
52:4
60:<
68:D
76:L
84:T
92:\
100:d
108:l
116:t
124:|
37:%
45:53:5
61:=
69:E
77:M
85:U
93:]
101:e
109:m
117:u
125:}
38:& 39:'
46:. 47:/
54:6 55:7
62:> 63:?
70:F 71:G
78:N 79:O
86:V 87:W
94:^ 95:_
102:f 103:g
110:n 111:o
118:v 119:w
126:~
83
The tr command
 tr is short for translate.
 tr translates between two sets of characters.
 replace all occurrences of the first character in set 1
with the first character in set 2, the second char in set 1
with the second char in set 2, …
tr [options] [string1 [string2]]
Introduction to Unix
84
tr Example
Replace 'A' with 'a', 'B' with 'b', … 'Z' with 'z'
Gore
Bradley
Bush
McCain
Trump
Letterman
tr A-Z a-z
Introduction to Unix
gore
bradley
bush
mccain
trump
letterman
85
tr can delete
-d option means "delete characters that are found in
string1".
Gore
Bradley
Bush
McCain
Trump
Letterman
tr -d aeiou
Introduction to Unix
Gr
Brdly
Bsh
McCn
Lttrmn
86
Another tr example - remove newlines
Gore
Bradley
Bush
McCain
Trump
Letterman
tr -d '\n'
GoreBradleyBushMcCainTrumpLetterman
Introduction to Unix
87
The uniq Command
 uniq removes duplicate adjacent lines from a file.
 uniq is typically used on a sorted file (which forces
duplicate lines to be adjacent).
 uniq can also reduce multiple blank lines to a single
blank line.
Introduction to Unix
88
uniq examples
Gore
Bradley
Bush
McCain
Trump
Letterman
10
10
10
10
10
100
uniq
uniq
Introduction to Unix
Gore
Bradley
Bush
McCain
Trump
Letterman
10
100
89
Shell - a user interface
 A shell is a command interpreter turns text that you
type (at the command line) in to actions:
 runs a program, perhaps the ls program.
 allows you to edit a command line.
 can establish alternative sources of input and
destinations for output for programs.
Introduction to Unix
90
Running a Program
 You type in the name of a program and some
command line options:
 The shell reads this line, finds the program and runs
it, feeding it the options you specified.
 The shell establishes 3 I/O channels:
 Standard Input
 Standard Output
 Standard Error
Introduction to Unix
91
Programs and Standard I/O
Standard Input
(STDIN)
Program
Standard Output
(STDOUT)
Standard Error
(STDERR)
Introduction to Unix
92
Unix Commands
 Most Unix commands (programs):
 read something from standard input.
 send something to standard output (typically depends
on what the input is!).
 send error messages to standard error.
Introduction to Unix
93
Defaults for I/O
 When a shell runs a program for you:
 standard input is your keyboard.
 standard output is your screen/window.
 standard error is your screen/window.
Introduction to Unix
94
Terminating Standard Input
 If standard input is your keyboard, you can type stuff
in that goes to a program.
 To end the input you press Ctrl-D (^D) on a line by
itself, this ends the input stream.
 The shell is a program that reads from standard input.
 What happens when you give the shell ^D?
Introduction to Unix
95
Popular Shells
sh
ksh
csh
bash
Bourne Shell
Korn Shell
C Shell
Bourne-Again Shell
Introduction to Unix
96
Customization
 Each shell supports some customization.
 User prompt
 Where to find mail
 Shortcuts
 The customization takes place in startup files – files
that are read by the shell when it starts up
Introduction to Unix
97
Startup files
sh,ksh:
/etc/profile (system defaults)
~/.profile
bash:
~/.bash_profile
~/.bashrc
~/.bash_logout
csh:
~/.cshrc
~/.login
~/.logout
Introduction to Unix
98
Wildcards (metacharacters) for
filename abbreviation
 When you type in a command line the shell treats
some characters as special.
 These special characters make it easy to specify
filenames.
 The shell processes what you give it, using the special
characters to replace your command line with one that
includes a bunch of file names.
Introduction to Unix
99
The special character *
 * matches anything.
 If you give the shell * by itself (as a command line
argument) the shell will remove the * and replace it
with all the filenames in the current directory.
 “a*b” matches all files in the current directory that
start with a and end with b.
Introduction to Unix
100
Understanding *
 The echo command prints out whatever you give it:
> echo hi
hi
 Try this:
> echo *
Introduction to Unix
101
* and ls
 Things to try:
ls *
ls –al *
ls a*
ls *b
Introduction to Unix
102
Other metacharacters
? Matches any single character
ls Test?.doc
[abc…] matches any of the enclosed characters
ls T[eE][sS][tT].doc
[a-z] matches any character in a range
ls [a-zA-Z]*
[!abc…] matches any character except those listed.
ls [!0-9]*
Introduction to Unix
103
Input Redirection
 The shell can attach things other than your keyboard
to standard input.
 A file (the contents of the file are fed to a program as if
you typed it).
 A pipe (the output of another program is fed as input as
if you typed it).
Introduction to Unix
104
Output Redirection
 The shell can attach things other than your screen to
standard output (or stderr).
 A file (the output of a program is stored in file).
 A pipe (the output of a program is fed as input to
another program).
Introduction to Unix
105
How to tell the shell to redirect
things
 To tell the shell to store the output of your program in
a file, follow the command line for the program with
the “>” character followed by the filename:
ls > lsout
the command above will create a file named lsout and
put the output of the ls command in the file.
Introduction to Unix
106
Input redirection


To tell the shell to get standard input from a file, use
the “<“ character:
sort < nums
The command above would sort the lines in the file
nums and send the result to stdout.
Introduction to Unix
107
You can do both!
sort < nums > sortednums
tr a-z A-Z < letter > rudeletter
Introduction to Unix
108
Output and Output Append
 The command ls > foo will create a new file
named foo (deleting any existing file named foo).
 If you use >> the output will be appended to foo:
ls /etc >> foo
ls /usr >> foo
Introduction to Unix
109
Pipes
 A pipe is a holder for a stream of data.
 A pipe can be used to hold the output of one
program and feed it to the input of another.
prog1
prog2
STDOUT
Introduction to Unix
STDIN
110
Asking for a pipe
 Separate 2 commands with the “|” character.
 The shell does all the work!
ls | sort
ls | sort > sortedls
Introduction to Unix
111
Building commands
 You can string together a series of unix commands to
do something new!
 Exercises:
 List all files in the current directory but only use upper
case letters.
 List only those files that have permissions set so that
anyone can write to the file.
Introduction to Unix
112
Stderr
 Many commands send error messages to standard
error.
 This is a different stream than stdout.
 The “>” output redirection only applies to Stdout
(not to Stderr).
 Try this:
ls foo blah gork > savedls
(I’m assuming there are no files named foo, blah
or gork!).
Introduction to Unix
113
Capturing stderr
 To redirect stderr to a file you need to know what
shell you are using.
 When using sh, ksh or bash it’s easy:
ls foo blah gork 2> erroroutput
It’s not so easy with csh...
Introduction to Unix
114
File Descriptors
 Unix progams write to file descriptors, small integers
that are somehow attached to a stream.
STDIN is 0
STDOUT is 1
STDERR is 2
“2>” means redirect stream #2 (sh, ksh and bash)
Introduction to Unix
115
Csh and Stderr
 >& merges STDOUT and STDERR and sends to a file:
ls foo blah >& saveboth
 >>& merges STDOUT and STDERR and appends to a
file:
ls foo blah >>& saveboth
Introduction to Unix
116
More Csh
 |& merges STDOUT and STDERR and sends to a pipe:
ls foo blah |& sort
 To send STDERR to file “err” and STDOUT to file “out”
you can do this:
(ls foo blah > out) >& err
Introduction to Unix
117
Shell Variables*
 The shell keeps track of a set of parameter names
and values.
 Some of these parameters determine the behavior
of the shell.
 We can access these variables:
 set new values for some to customize the shell.
 find out the value of some to help accomplish a task.
* From now on I'll focus on sh and ignore csh
Introduction to Unix
118
Example Shell Variables
sh / ksh / bash
PWD current working directory
PATH list of places to look for commands
HOME home directory of user
MAIL where your email is stored
TERM what kind of terminal you have
HISTFILE where your command history
is saved
Introduction to Unix
119
Displaying Shell Variables
 Prefix the name of a shell variable with "$".
 The echo command will do:
echo $HOME
echo $PATH
 You can use these variables on any command line:
ls -al $HOME
Introduction to Unix
120
Setting Shell Variables
 You can change the value of a shell variable with an
assignment command (this is a shell builtin
command):
HOME=/etc
PATH=/usr/bin:/usr/etc:/sbin
NEWVAR="blah blah blah"
Introduction to Unix
121
set
command (shell builtin)
 The set command with no parameters will print out a
list of all the shell varibles.
 You'll probably get a pretty long list…
 Depending on your shell, you might get other stuff as
well...
Introduction to Unix
122
$PS1 and $PS2
 The PS1 shell variable is your command line prompt.
 The PS2 shell variable is used as a prompt when the
shell needs more input (in the middle of processing a
command).
 By changing PS1 and/or PS2 you can change the
prompt.
Introduction to Unix
123
Fancy bash prompts
 Bash supports some fancy stuff in the prompt string:
\t is replace by the current time
\w is replaced by the current directory
\h is replaced by the hostname
\u is replaced by the username
\n is replaced by a newline
Introduction to Unix
124
Example bash prompt
======= [foo.cs.rpi.edu] - 22:43:17 =======
/cs/hollingd/introunix echo $PS1
======= [\h] - \t =======\n\w
 You can change your prompt by changing PS1:
PS1="Yes Master? "
Introduction to Unix
125
Making Changes Stick
 If you want to tell the shell (bash) to always use the
prompt "Yes Master ?", you need to store the
change in a shell startup file.
 For bash - change the file ~/.bashrc.
 Wait a few minutes and we will talk about text editors
- you need to use one to do this!
Introduction to Unix
126
The PATH
 Each time you give the shell a command line it does
the following:
 Checks to see if the command is a shell built-in.
 If not - tries to find a program whose name (the
filename) is the same as the command.
 The PATH variable tells the shell where to look for
programs (non built-in commands).
Introduction to Unix
127
echo $PATH
======= [foo.cs.sjce.edu] - 22:43:17 =======
/cs/bsm/introunix echo $PATH
/home/hollingd/bin:/usr/bin:/bin:/usr/local/b
in:/usr/sbin:/usr/bin/X11:/usr/games:/usr/lo
cal/packages/netscape
 The PATH is a list of ":" delimited directories.
 The PATH is a list and a search order.
 You can add stuff to your PATH by changing the
shell startup file.
Introduction to Unix
128
Job Control
 The shell allows you to manage jobs
 place jobs in the background
 move a job to the foreground
 suspend a job
 kill a job
Introduction to Unix
129
Background jobs
 If you follow a command line with "&", the shell will
run the job in the background.
 you don't need to wait for the job to complete, you can
type in a new command right away.
 you can have a bunch of jobs running at once.
 you can do all this with a single terminal (window).
ls -lR > saved_ls &
Introduction to Unix
130
Listing jobs
 The command jobs will list all background jobs:
> jobs
[1] Running
>
ls -lR > saved_ls &
 The shell assigns a number to each job (this one is job
number 1).
Introduction to Unix
131
Suspending and Killing the
Foreground Job
 You can suspend the foreground job by pressing ^Z
(Ctrl-Z).
 Suspend means the job is stopped, but not dead.
 The job will show up in the jobs output.
 You can kill the forground job by pressing ^C (Ctrl-
C).
 It's gone...
Introduction to Unix
132
Moving a job back to the foreground
 The fg command will move a job to the
foreground.
 You give fg a job number (as reported by the jobs
command) preceeded by a %.
> jobs
[1] Stopped
> fg %1
ls -lR > saved_ls
Introduction to Unix
ls -lR > saved_ls &
133
Quoting - the problem
 We've already seen that some characters mean
something special when typed on the command
line: * ? []
 What if we don't want the shell to treat these as
special - we really mean *, not all the files in the
current directory:
echo here is a star *
Introduction to Unix
134
Quoting - the solution
 To turn off special meaning - surround a string with
double quotes:
echo here is a star "*"
echo "here is a star"
Introduction to Unix
135
Careful!
 You have to be a little careful. Double quotes
around a string turn the string in to a single
command line parameter.
> ls
fee file? foo
> ls "foo fee file?"
ls: foo fee file?: No such file or
directory
Introduction to Unix
136
Quoting Exceptions
 Some special characters are not ignored even if
inside double quotes:
 $ (prefix for variable names)
 " the quote character itself
 \ slash is always something special (\n)
 you can use \$ to mean $ or \" to mean "
echo "This is a quote \" "
Introduction to Unix
137
Single quotes
 You can use single quotes just like double quotes.
 Nothing (except ') is treated special.
> echo 'This is a quote \" '
This is a quote \"
>
Introduction to Unix
138
Backquotes are different!
 If you surround a string with backquotes the string is
replaced with the result of running the command in
backquotes:
> echo `ls`
foo fee file?
> PS1=`date`
Tue Jan 25 00:32:04 EST 2000
Introduction to Unix
139
Communication & E-mail
News : It is invoked by user to read any message that is sent
by system adminstator.
Write : Used for two way communications.
Mesg :Makes the user to block the message.
Talk :It is popular communication program which is superior
to write.
Introduction to Unix
140
Communication & E-mail
Mail : Used when user is not logged in and message
can be viewed,replied,deleted.
Elm and pine : These are mail handlers which are
menu driven.
Finger :Gives the details of user with communication
features.
Introduction to Unix
141
Shell Programming
 Shell script – Group of commands has to be
executed & stored in file with extension .sh
 Exit – Shell Script termination.
 Expr – Used for computation of basic four
operation
 Set – Assigning values to positional parameters.
Introduction to Unix
142
Shell Programming
 Conditional Statements
- If
- If- elif
- Case
- While
- Until
- For
Introduction to Unix
143
Introduction to Unix
144
introduction and history
 Practical Extraction and Report Language
 Pathologically Eclectic Rubbish Lister?
 the Swiss Army chainsaw of scripting languages
 combines the best of C, sh, awk, and sed
 released in 1987 by Larry Wall
 initially ported to MPE by Mark Klein
 re-ported by Mark Bixby in 1997 with periodic
updates since then
Introduction to Unix
145
current status
 latest & greatest Perl release v5.6.0 available for MPE
from bixby.org
 Perl is not supported by HP, but if your use of Perl
uncovers any underlying MPE or POSIX bugs, then we
certainly want to hear from you!
 the best way to get assistance with Perl on MPE is to
post your questions to HP3000-L
Introduction to Unix
146
variable names
 scalar values
 $days
# the simple scalar value "days"
 $days[28]
# the 29th element of array
@days
 $days{'Feb'}
# the 'Feb' value from hash
%days
 $#days
# the last index of array @days
 entire arrays or array slices (aka lists)
 @days
# ($days[0], $days[1],... $days[n])
 @days[3,4,5]
Introduction to Unix
# same as @days[3..5]
147
value constructors
 scalar values
 $abc = 12345;
 $abc = 12345.67;
 $abc = 0xffff;
# hex
 $abc = "a string with a newline\n";
 list values
 @abc = ("cat", "dog", $def);
 ($dev, $ino, undef, undef, $uid, $gid) =
stat($file);
 hash values
 $abc{'December'} = 12;
 $month = $abc{'December'};
Introduction to Unix
148
scalar vs. list context
 the context of some operations will determine the
type of the data returned
 scalar
 list
 assignment to a scalar variable will evaluate the
righthand side in a scalar context
 $onerecord = <STDIN>
 assignment to a list variable will evaluate the
righthand side in a list context
 @entirefile = <STDIN>
Introduction to Unix
149
simple statements
 terminated with a semicolon
 may be followed by one optional modifier
 if EXPR
 unless EXPR
 while EXPR
 until EXPR
 foreach EXPR
 $os = 'mpe';
 $os = 'mpe' if $model == 3000;
Introduction to Unix
150
compound statements
 a block is a sequence of statements
delimited by curly brackets (braces) that
defines a scope
 compound statements that control flow:
 if (EXPR) BLOCK
 if (EXPR) BLOCK else BLOCK
 if (EXPR) BLOCK elsif (EXPR) BLOCK ...
else BLOCK
 LABEL while (EXPR) BLOCK
 LABEL while (EXPR) BLOCK continue BLOCK
 LABEL for (EXPR; EXPR; EXPR) BLOCK
Introduction to Unix
151
subroutines
sub max {
my $max = shift(@_);
foreach $foo (@_) {
$max = $foo if $max < $foo; }
return $max;
}
$bestday = max($mon,$tue,$wed,$thu,$fri);
 parameters passed via @_ array
 @_[0] = parm1, @_[1] = parm2, etc
 @_ is an alias (i.e. call by reference)
 private variables declared with my
 return or the value of
the last expression is the
functional return value
Introduction to Unix
152
arithmetic operators
 addition: +
 subtraction:  multiplication: *
 division: /
 modulus: %
 exponentiation: **
 auto-increment and -decrement: ++ - ++$a - increments $a, returns new value
 $a++ - returns current value, then increments $a
Introduction to Unix
153
assignment operators
 works like C
 $a += 2; is equivalent to $a = $a + 2;
 **=
+=
*=
&=
<<=
&&=
-=
>>=
||=
.=
%=
^=
x=
/=
 |=
Introduction to Unix
154
relational operators
 numeric comparisons:
 < > <= >= == != <=>
 <=> returns -1, 0, or 1 depending on whether the left
argument is numerically less than, equal to, or greater
than the right argument
 string comparsions:
 lt gt le ge eq ne cmp
 cmp returns -1, 0, or 1 depending on whether the left
argument is stringwise less than, equal to, or greater than
the right argument
Introduction to Unix
155
bitwise operators
 shift left: <<
 shift right: >>
 AND: &
 OR: |
 XOR: ^
 negation: ~
Introduction to Unix
156
i/o and file handles
 open files are identified via file handles
 uppercase handle names by convention
 predefined file handles: STDIN, STDOUT, STDERR
 <FILEHANDLE> in a scalar context reads the next
record from the file
 <FILEHANDLE> in a list context reads ALL of the
remaining records from the file
Introduction to Unix
157
opening files with open()
 open(HANDLE,
reading
 open(HANDLE,
reading
 open(HANDLE,
writing
 open(HANDLE,
appending
 open(HANDLE,
pipe for writing
"/file/path") - open for
"< /file/path") - open for
"> /file/path") - open for
">> /file/path") - open for
"| shell command") - open
Introduction to Unix
158
regular expressions
 a vast superset beyond standard Unix
regexps
 a ? modifier to make patterns non-greedy
 zero-width lookahead and lookbehind
assertions
 conditional expressions
Introduction to Unix
159
using regular expressions
$showme=`callci showme`;
if ($showme =~ /RELEASE: ([AZ]\.(\d)(\d)\.\d\d)/) {
$release = $1; # the matching V.UU.FF
$mpe = "$2.$3"; # the matching U and U
(i.e. 7.0)
}
$showme =~ s/LDev/Logical Device/gi; #
global substitution
Introduction to Unix
160
predefined variables
 $| or $OUTPUT_AUTOFLUSH
 By default, all Perl output is buffered (0). To enable
automatic flushing, set this variable to 1. Needed
when doing MPE I/O which is usually unbuffered.
 $$ or $PID
 POSIX PID of the current process
 $^O or $OSNAME
 operating system name (mpeix)
 @ARGV
Introduction to Unix
161
debugging
 invoke the debugger by starting Perl with the -d
parameter
 #!/PERL/PUB/perl -d
 examine or modify variables
 single-step execution
 set breakpoints
 list source code
 set actions to be done before a line is executed
Introduction to Unix
162
perl extensions
 binary code residing in an external NMXL
loaded at run time
 a thin layer of C that allows the Perl interpreter
to call compiled code written in other languages
 several extension libraries come bundled with
Perl (sockets, POSIX, etc)
Introduction to Unix
163
System Administration
 fdisk – Dividing a disk into partitions
 Useradd – Adding users
 Usermod and userdel – Modifying and removing
users
 Umask – To make default file permissions
 Fsck – File system checking
 Mkfs – Creating file systems
Introduction to Unix
164
System Administration
 Mount - To mount (i.e, attach) a file system to the
root file system.
 Init – This daemon maintains the system at one
run level.
 Lpadmin – Configuring a printer
 Lpsat – Obtaining printer and job status
Introduction to Unix
165
TCP\IP Network Administration
 Netconfig – Confiuring the network interface.
 Ifconfig – Interface Configuration
 Ping – Checking the network.
 Route – Used to build the kernel routing table
to indicate the gateways that have to be used for
routing packets destined.
Introduction to Unix
166