Download Operating Systems File System File System File System Functions

File System Functions
√
identification and location of files,
Operating Systems
File System
√
usage of directories,
√
user to files access control,
√
blocking of files during access to files,
dr. Tomasz Jordan Kruk
√
free blocks allocation management,
[email protected]
√
free blocks space management.
Institute of Control & Computation Engineering
Warsaw University of Technology
Faculty of E&IT, Warsaw University of Technology
Criteria of files organization:
√
access performance,
√
flexibility,
√
storage efficiency,
√
manageability,
√
fault tolerance.
Operating Systems / File System – p. 1/37
Faculty of E&IT, Warsaw University of Technology
File System
Types of File Structure
File management:
√
it must be possible to store a very large amount of information.
√
the information must survive the termination of the process using it.
√
multiple processes must be able to access the information concurrently.
1 Record
1 Byte
Cat
√
Operating Systems / File System – p. 3/37
field, basic data unit, contains single value characterized by a size and
a type,
Cow
Dog
Hen
√
record, collection of related to each other fields treated as a whole,
(a)
√
file, collection of similar records treated as a whole, identified by a
unique name, with an access restricted by given access rights.
Three kinds of files:
Ibis
(b)
Ant
Fox
Pig
Goat
Lion
Owl
Pony
Rat
Worm
Lamb
(c)
a. byte sequence,
b. record sequence,
c. tree.
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 2/37
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 4/37
int main(int argc, char *argv[])
{
int in_fd, out_fd, rd_count, wt_count;
char buffer[BUF _SIZE];
File Operations (II)
/* Open the input file and create the output file */
in_fd = open(argv[1], O_RDONLY); /* open the source file */
if (in_fd < 0) exit(2);
/* if it cannot be opened, exit */
out_fd = creat(argv[2], OUTPUT _MODE); /* create the destination file */
if (out_fd < 0) exit(3);
/* if it cannot be created, exit */
/* Copy loop */
while (TRUE) {
rd_count = read(in_fd, buffer, BUF _SIZE); /* read a block of data */
if (rd_count <= 0) break;
/* if end of file or error, exit loop */
wt_count = write(out _fd, buffer, rd_count); /* write data */
if (wt_count <= 0) exit(4);
/* wt_count <= 0 is an error */
}
/* Close the files */
close(in _fd);
close(out _fd);
if (rd_count == 0)
exit(0);
else
exit(5);
Meaning
Who can access the file and in what way
Password needed to access the file
ID of the person who created the file
Current owner
0 for read/write; 1 for read only
0 for normal; 1 for do not display in listings
0 for normal files; 1 for system file
0 for has been backed up; 1 for needs to be backed up
0 for ASCII file; 1 for binary file
0 for sequential access only; 1 for random access
0 for normal; 1 for delete file on process exit
0 for unlocked; nonzero for locked
Number of bytes in a record
Offset of the key within each record
Number of bytes in the key field
Date and time the file was created
Date and time the file was last accessed
Date and time the file has last changed
Number of bytes in the file
Number of bytes the file may grow to
Faculty of E&IT, Warsaw University of Technology
/* syntax error if argc is not 3 */
if (argc != 3) exit(1);
Attribute
Protection
Password
Creator
Owner
Read-only flag
Hidden flag
System flag
Archive flag
ASCII/binary flag
Random access flag
Temporary flag
Lock flags
Record length
Key position
Key length
Creation time
Time of last access
Time of last change
Current size
Maximum size
Possible File Attributes
/* no error on last read */
/* error on last read */
}
Operating Systems / File System – p. 5/37
Faculty of E&IT, Warsaw University of Technology
File Operations (I)
Operating Systems / File System – p. 7/37
Memory-Mapped Files
/* File copy program. Error checking and reporting is minimal. */
#include <sys/types.h>
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>
/* include necessary header files */
Program
text
Program
text
Data
(a)
int main(int argc, char *argv[]);
/* ANSI prototype */
#define BUF _SIZE 4096
#define OUTPUT _MODE 0700
/* use a buffer size of 4096 bytes */
/* protection bits for output file */
Data
xyz
(b)
Process segments:
a. A segmented process before mapping files into its address space.
b. The process after mapping an existing file abc into one segment and
creating a new segment for file xyz.
int main(int argc, char *argv[])
{
int in_fd, out_fd, rd_count, wt_count;
char buffer[BUF _SIZE];
if (argc != 3) exit(1);
abc
/* syntax error if argc is not 3 */
/* Open the input file and create the output file */
in_fd = open(argv[1], O_RDONLY); /* open the source file */
exit *Systems
/
if (in_fd <Faculty
0) exit(2);
of E&IT, Warsaw University of Technology/* if it cannot be opened,Operating
/ File System – p. 6/37
out_fd = creat(argv[2], OUTPUT _MODE); /* create the destination file */
if (out_fd < 0) exit(3);
/* if it cannot be created, exit */
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 8/37
Two-level Directory Systems
Directories
√
directories contain information about files: attributes, address, owner
information,
√
directory may be a file itself (like under Unix),
√
the main purpose of directories is to keep and enable translation
between files and their names.
Structures of directories organization:
√
single-level structure,
√
two-levels directory systems (one directory per each user),
√
hierarchical directory systems.
Root directory
A
A
B
A
B
C
C
? files are identified by paths,
? it is possible to have more than one name for the same file,
? current directory (working directory) idea, absolute and relative path
names.
Faculty of E&IT, Warsaw University of Technology
B
C
Root directory
User
directory
A
Faculty of E&IT, Warsaw University of Technology
B
B
B
C
B
B
A single-level directory system containing four files, owned by three different
people, A, B and C.
Operating Systems / File System – p. 10/37
Operating Systems / File System – p. 11/37
Hierarchical Directory Systems
A
A
C
Faculty of E&IT, Warsaw University of Technology
Single-level Directory Systems
A
C
Files
Operating Systems / File System – p. 9/37
Root directory
User
directory
C
C
C
C
User subdirectories
C
Faculty of E&IT, Warsaw University of Technology
C
C
C
User file
Operating Systems / File System – p. 12/37
A Unix Directory Tree
Contiguos Allocation
/
bin
File A
(4 blocks)
Root directory
File E
(12 blocks)
File C
(6 blocks)
File G
(3 blocks)
etc
lib
…
usr
tmp
bin
etc
lib
usr
File D
(5 blocks)
File B
(3 blocks)
tmp
ast
File F
(6 blocks)
(a)
jim
lib
(File A)
ast
lib
dict.
(File E)
(File C)
(File G)
…
jim
/usr/jim
5 Free blocks
File B
6 Free blocks
(b)
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 13/37
Faculty of E&IT, Warsaw University of Technology
Disk Space Management
Linked List Allocation (I)
Methods of file allocation:
√
contiguous allocation,
? FAT entry = name, start block, size,
√
linked list allocation,
? FAT entry = name, start block, size,
? in each block field with the reference to the next data block.
√
i-nodes (index nodes),
? FAT entry = name, reference to the block with indexes,
? i-node block contains references to data blocks.
? possible extensions with introduction of areas with local continuity,
i-node block entry would have reference to data block and the count
of blocks located there,
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 15/37
Operating Systems / File System – p. 14/37
File A
0
Physical
block
File
block
0
File
block
1
File
block
2
File
block
3
File
block
4
4
7
2
10
12
File B
0
Physical
block
File
block
0
File
block
1
File
block
2
File
block
3
6
3
11
14
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 16/37
Implementing Directories
Linked List Allocation (II)
Physical
block
0
1
2
10
3
11
4
7
File A starts here
6
3
File B starts here
7
2
games
attributes
games
mail
attributes
mail
news
attributes
news
work
attributes
work
5
(a)
(b)
Data structure
containing the
attributes
8
9
10
12
11
14
12
-1
a. A simple directory containing fixed-size entries with the disk addresses
and attributes in the directory entry.
13
14
-1
15
b. A directory in which each entry just refers to an i-node.
Unused block
Linked list allocation using a file allocation table in main memory.
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 17/37
Faculty of E&IT, Warsaw University of Technology
An Example i-node
Operating Systems / File System – p. 19/37
Long File Names Handling
File 1 entry length
Pointer to file 1's name
File 1 attributes
File 1 attributes
File Attributes
Entry
for one
file
Address of disk block 0
Address of disk block 1
Address of disk block 2
p
e
b
e
Address of disk block 3
r
o
j
c
t
u
d
g
t
File 2 entry length
Entry
for one
file
Pointer to file 2's name
File 2 attributes
Pointer to file 3's name
File 3 attributes
File 2 attributes
Address of disk block 4
p
o
l
Address of disk block 5
Address of disk block 6
e
n
r
n
s
e
File 3 entry length
Address of disk block 7
File 3 attributes
Address of block of pointers
f
Disk block
containing
additional
disk addresses
o
o
(a)
p
e
b
e
e
n
r
c
u
t
r
n
f
o
t
d
s
e
o
j
g
p
o
l
o
Heap
(b)
Two ways of handling long file names in a directory:
a. in-line,
b. in a heap.
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 18/37
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 20/37
File System Containing Shared Files
Free Disk Space Management
Free disk blocks: 16, 17, 18
Root directory
A
B
A
B
C
B
B
C
C
C
B
?
42
230
86
136
162
234
1001101101101100
0110110111110111
210
612
897
1010110110110110
97
342
422
0110110110111011
41
214
140
1110111011101111
63
160
223
1101101010001111
21
664
223
0000111011010111
48
216
160
1011101101101111
262
320
126
1100100011101111
310
180
142
0111011101110111
516
482
141
1101111101110111
C
C
C
C
A 1-KB disk block can hold 256
32-bit disk block numbers
Shared file
A bitmap
(a)
(b)
a. storing the free list on a linked list,
b. a bitmap.
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 21/37
Faculty of E&IT, Warsaw University of Technology
Directed Acyclic Graphs
C's directory
B's directory C's directory
Disk Quota
B's directory
Open file table
Quota table
Attributes
disk addresses
User = 8
Hard block limit
Quota pointer
Owner = C
Count = 1
Owner = C
Count = 2
Operating Systems / File System – p. 23/37
Soft block limit
Current # of blocks
# Block warnings left
Soft file limit
Owner = C
Count = 1
Quota
record
for user 8
Hard file limit
Current # of files
# File warnings left
(a)
(b)
(c)
Symbolic linking
a. situation prior to linking,
b. after the link is created,
Quotas controlled on a per-user basis in a quota table.
c. after the original owner removes the file.
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 22/37
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 24/37
Disk Backups
3
4
Directory
that has not
changed
5
16
18
6
17
19
7
8
10
9
11
12
13
Bytes
Root directory
1
2
The Unix V7 File System (I)
20
14
21
15
File that
has changed
14
27
30
31
23
25
File name
29
28
22
24
2
26
I-node
number
32
File that has
not changed
A Unix V7 directory entry.
A file system to be dumped.
√
each file and directory labeled by its i-node number,
√
the shaded items has been modified since the last dump.
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 25/37
Faculty of E&IT, Warsaw University of Technology
Usage of Bitmaps for Backup
Operating Systems / File System – p. 27/37
The Unix V7 File System (II)
I-node
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
(b)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
(c)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
(d)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Attributes
Disk addresses
(a)
Single
indirect
block
Addresses of
data blocks
Double
indirect
block
Triple
indirect
block
A Unix i-node.
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 26/37
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 28/37
The Unix V7 File System (III)
Root directory
1
.
..
4
bin
7
dev
1
14
9
I-node 6
is for /usr
Mode
size
times
132
lib
etc
6
usr
8
tmp
Looking up
usr yields
i-node 6
Block 132
is /usr
directory
6
1
19
dick
30
erik
51
jim
26
45
I-node 6
says that
/usr is in
block 132
I-node 26
is for
/usr/ast
Mode
size
times
406
ast
/usr/ast
is i-node
26
Block 406
is /usr/ast
directory
size in blocks of i-nodes list,
26
√
sieze in blocks of the file system,
6
√
number of free blocks in a file system,
√
index of the next free block on the free blocks list,
√
number of free i-nodes in the file system,
minix
√
index of the next free i-node on the free i-node lists,
src
√
modification marker of the superblock,
√
time of modification and the name of the file system.
64
grants
92
books
60
mbox
17
I-node 26
says that
/usr/ast is in
block 406
/usr/ast/mbox
is i-node
60
The steps in looking up usr/ast/mbox.
Faculty of E&IT, Warsaw University of Technology
Superblock contains:
√
81
bal
The Unix V7 File System (V)
Modern operating systems may have additional features:
√
many copies of the superblock,
√
journaling,
√
snapshot awareness.
Operating Systems / File System – p. 29/37
Faculty of E&IT, Warsaw University of Technology
The Unix V7 File System (IV)
Under Unix the file system is usually written to the disk in four separate
sections:
√
block 0 (boot block), used for booting the operating system,
√
block 1, (superblock), contains information about the structure of the
file system,
√
blocks 2 – m, (i-nodes), i-nodes lists,
√
blocks m+1 – n, data blocks.
Operating Systems / File System – p. 31/37
Example File System Structure
Entire disk
Disk partition
Partition table
MBR
Boot block
Super block
Free space mgmt
I-nodes
Root dir
Files and directories
I-nodes and special files:
√
in case of special files in i-nodes, instead of the first data pointer,
number of device driver handling procedure is written,
√
that number consists of two parts: major number and minor number,
√
special file may be created with the mknod command.
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 30/37
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 32/37
1 TB
2 TB
2 TB
2 TB
128 MB
256 MB
512 MB
1024 MB
2048 MB
FAT-32
FAT-16
FAT-12
2 MB
4 MB
8 MB
16 MB
The CP/M directory entry format.
Disk block numbers
Extent Block count
File type
(extension)
User code
Block size
0.5 KB
1 KB
2 KB
4 KB
8 KB
16 KB
32 KB
File name
16
2
1
3
8
Bytes 1
The MS-DOS File System (II)
The CP/M File System
Maximum partition size for different block sizes. The empty boxes represent
forbidden combinations.
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 33/37
Faculty of E&IT, Warsaw University of Technology
The MS-DOS File System (I)
Bytes
8
3
1
10
2
2
2
The File System under Windows 98 (I)
4
8
3
Base name
Ext
Bytes
Size
File name
1 1 1
N
T
Attributes
Extension Attributes
Reserved
Time Date
Operating Systems / File System – p. 35/37
First
block
number
The MS-DOS directory entry.
4
Creation
date/time
Sec
2
Last
access
2
4
Last write
date/time
Upper 16 bits
of starting
block
2
4
File size
Lower 16 bits
of starting
block
The extended MS-DOS directory entry used in Windows 98.
Bytes
1
Sequence
10
1 1 1
12
2
4
5 characters
0
6 characters
0
2 characters
Attributes
Checksum
An entry for (part of) a long file name in Windows 98.
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 34/37
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 36/37
The File System under Windows 98 (II)
68
d
o
g
e
3
o
v
2
w
n
1
T
h
f
e
T H E Q U I ~ 1
Bytes
o
q
C
A 0 K
C
A 0 K
C
A 0 K
C
A 0 K
N
A T S
0
t
h
x
u
i
Creation
time
e
j
u
c
k
Last
acc Upp
l
a
0
z
m
p
0
s
0
r
b
Last
write
Low
y
o
Size
An example of how a long name is stored in Windows 98.
Faculty of E&IT, Warsaw University of Technology
Operating Systems / File System – p. 37/37