Download File-System Interface and Implementation

Chapter 11 & 12:
File System Interface and
Implementation
Operating System Concepts – 9th Edition
Modified by Dr. Neerja Mhaskar for CS 3SH3
Silberschatz, Galvin and Gagne ©2013
File-System Structure
 File system resides on secondary storage (disks)

Provides user interface

File system algorithms and data structures map the
logical file system onto physical secondary storage.

Provides efficient and convenient access to disk by
allowing data to be stored, located and retrieved easily
 Logical storage unit in a file system is called a file
 I/O transfers performed in blocks (stored on one or more
sectors) (usually 512 bytes)
 File system organized into layers - Layering are useful for
reducing complexity and redundancy, but adds overhead
and can decrease performance
Operating System Concepts – 9th Edition
12.2
Silberschatz, Galvin and Gagne ©2013
File-System Structure Cont…
 Many different types of file systems exist,
sometimes many within an operating system.
Each with its own format.

UFS – Unix File system

Extended file system (ext3 and ext 4) – LINUX

FAT, FAT32, NTFS - Windows
Operating System Concepts – 9th Edition
12.3
Silberschatz, Galvin and Gagne ©2013
File
A file is associated with the following:
 File attributes – Name, Identifier (e.g. inode number), text etc.
 File operations – Create, write, read, delete, truncate, repositioning
within a file.
 File type – Executable (.bin, .exe), text file (.txt, .doc)
 File structure/Internal File structure – Files have internal structure -
used by Oses and programs to work with them.

Supporting different file structure can increase the size of the OS
making it hard to manage and work with.

Therefore, Unix treats all files as sequence of bytes

All operating systems must support the executable file structure to
load and run programs.

Disk files are accessed in units of physical blocks, 512 bytes.
Internally files are organized in logical units (>= 1 byte).

The number of logical units which fit into one physical block
determines its packing – suffers from internal fragmentation.
Operating System Concepts – 9th Edition
12.4
Silberschatz, Galvin and Gagne ©2013
File System Layers
 I/O control level – Transfers information
from main memory to the disk system
 Consists of device drivers (manage I/O
devices) and interrupt handlers.
 Device drivers act as translators, which
take high level commands like “retrieve
block 123” and outputs low-level hardware
specific commands to hardware controller.
 Basic file system - issues generic
commands to the appropriate device driver
to read and write physical blocks on the
disk

Manages memory buffers and caches.
Buffers hold data in transit, and
Caches hold frequently used data
Operating System Concepts – 9th Edition
12.5
Silberschatz, Galvin and Gagne ©2013
File System Layers (Cont.)
 File organization module

Keeps track of files, their logical blocks and
physical blocks.

Translates logical block addresses to
physical block addresses

Manages free space, disk allocation
 Logical file system manages metadata (all
information related to the file-system structure
except the actual data)

Manages the directory structure.

Maintains file structure via file-control blocks

A file control block (FCB) (inode in Unix
file systems) contains all the information
about the file (e.g.: ownership, permissions,
and location of the file contents.)

Also, responsible for Protection.
Operating System Concepts – 9th Edition
12.6
Silberschatz, Galvin and Gagne ©2013
Disk Structure
 A disk can be used in its entirety for a file system.
 Alternatively a physical disk can be broken up into
multiple partitions, slices, or mini-disks, each of which
becomes a virtual disk and can have its own filesystem. ( or
be used for raw storage, swap space, etc. )
 Or, multiple physical disks can be combined into one volume,
i.e. a larger virtual disk, with its own filesystem spanning the
physical disks.
 Each volume containing file system also tracks that file
system’s info in device directory or volume table of contents
Operating System Concepts – 9th Edition
12.7
Silberschatz, Galvin and Gagne ©2013
A Typical File-system Organization
Operating System Concepts – 9th Edition
12.8
Silberschatz, Galvin and Gagne ©2013
Directories
 Directories – enable segregating files into groups and
managing them as groups.
 Operations that are to be performed on a directory:

Operations regarding a file: Create, delete rename, search.

Traverse the file system and list directory.
 Unix treats directories and files as same entities, but Windows
treats them as different entities.
 Various schemes for defining logical structure of a directory
exist.

Single-level, two-level, tree-structure etc.
Operating System Concepts – 9th Edition
12.9
Silberschatz, Galvin and Gagne ©2013
Directory Structure
 A collection of nodes containing information about all files
Directory
Files
F1
F2
F3
F4
Fn
Both the directory structure and the files reside on disk
Operating System Concepts – 9th Edition
12.10
Silberschatz, Galvin and Gagne ©2013
Directory Organization - Single-Level Directory
 A single directory for all users
 Naming problem
 Grouping problem
Operating System Concepts – 9th Edition
12.11
Silberschatz, Galvin and Gagne ©2013
Directory Organization - Two-Level Directory
 Separate directory for each user
 Path name to access other user’s files
 Can have the same file name for different user
 Efficient searching
 No grouping capability
Operating System Concepts – 9th Edition
12.12
Silberschatz, Galvin and Gagne ©2013
Directory Organization - Tree-Structured Directories
 Allows users to create their own subdirectories and to organize
their files.
 Efficient searching and Grouping Capability
 Most common
Operating System Concepts – 9th Edition
12.13
Silberschatz, Galvin and Gagne ©2013
Directory Organization - Acyclic-Graph Directories
 Have shared subdirectories and files
Operating System Concepts – 9th Edition
12.14
Silberschatz, Galvin and Gagne ©2013
Directory Organization - General Graph Directory
 Issue with acyclic graph directories is cycle detection.
 General graph directories have cycles.
Operating System Concepts – 9th Edition
12.15
Silberschatz, Galvin and Gagne ©2013
Directory Implementation
 Linear list of file names with pointer to the data blocks

Simple to program

Time-consuming to execute as it has linear search time.
 Hash Table – linear list with hash data structure.

Decreases directory search time

Issue: Collisions – situations where two file names hash
to the same location.
 Resolve
collisions by maintaining each hash entry as
linked list, and adding a new entry to the linked list.

Only good if entries are fixed size.
Operating System Concepts – 9th Edition
12.16
Silberschatz, Galvin and Gagne ©2013
File System Mounting
 Just as a file must be opened before it is used, a file system must
be mounted before it can be available on the system.

More specifically, the directory structure may be built out of
multiple volumes, which must be mounted to make them
available within the file-system name space.
 An unmounted file system (i.e., Fig. (b)) is mounted at a mount
point - the location within the file structure where the file system is
to be attached.

In UNIX its an empty directory (typically empty).
Operating System Concepts – 9th Edition
12.17
Silberschatz, Galvin and Gagne ©2013
Partitions and Mounting
 Partition can be a volume containing a file system (“cooked”) or
raw – just a sequence of blocks with no file system
 Boot block can point to boot volume or boot loader set of blocks that
contain enough code to know how to load the kernel from the file
system

Or a boot management program for multi-os booting
 Root partition contains the OS, other partitions can hold other
Oses, other file systems, or be raw

Mounted at boot time

Other partitions can mount automatically or manually
 At mount time, file system consistency checked

Is all metadata correct?

If not, fix it, try again

If yes, add to mount table, allow access
Operating System Concepts – 9th Edition
12.18
Silberschatz, Galvin and Gagne ©2013
Question
What problems could occur if a system allowed a file system to be
mounted simultaneously at more than one location?
Operating System Concepts – 9th Edition
12.19
Silberschatz, Galvin and Gagne ©2013
File-System Implementation
 The file-system implementation consists of three
major layers,

First layer - is the file-system interface, based on
the open(), read(), write(), and close() calls and
on file descriptors.

Second layer is called the virtual file system
(VFS) layer.

Third layer – is the layer implementing the filesystem type or the remote-file-system protocol.
Operating System Concepts – 9th Edition
12.20
Silberschatz, Galvin and Gagne ©2013
File-System Implementation – First layer
 Several on-disk and in-memory structures are used to implement
a file system (in particular, implementation of file system
operations.)
On Disk Structures
 Boot control block (boot block in Unix) is per volume

contains information needed by the system to boot OS from
that volume. Needed only if volume contains OS, and is
usually first block of volume.
 Volume control block (superblock in UNIX) is per volume

Contains volume (or partition) details (e.g.: Total # of blocks,
# of free blocks, block size, free block pointers or array etc.)
 Directory structure is per file system

Used to organizes the files. In Unix file system, it contains file
names and associated inode numbers.
Operating System Concepts – 9th Edition
12.21
Silberschatz, Galvin and Gagne ©2013
File-System Implementation (Cont.)
 File Control Block (FCB) is per file

Contains details about the file (e.g. inode number,
permissions, size, dates)
A typical file-control block
Operating System Concepts – 9th Edition
12.22
Silberschatz, Galvin and Gagne ©2013
In-Memory File System Structures
In-memory structures are used for both file-system management and
performance improvement via caching.
 Mount table - contains information about each mounted volume
 In-memory directory-structure cache - holds the directory
information of recently accessed directories.
 System-wide open-file table contains a copy of the FCB of each
open file, as well as tracks the number of processes that have the
file open.
 Per-process open-file table contains a pointer to the appropriate
entry (for the file) in the system-wide open-file table and some other
fields.
 Buffers hold file-system blocks when they are being read from disk
or written to disk.
Operating System Concepts – 9th Edition
12.23
Silberschatz, Galvin and Gagne ©2013
In-Memory File System Structures Cont…
 open(filename) – system call returns a pointer to
the appropriate entry (matching the filename) in
the per-process file-system table.

All file operations use this pointer

UNIX refer to it as a file descriptor (an integer
value).
Operating System Concepts – 9th Edition
12.24
Silberschatz, Galvin and Gagne ©2013
In-Memory File System Structures
In-memory file-system structures. (a) File open. (b) File read
Operating System Concepts – 9th Edition
12.25
Silberschatz, Galvin and Gagne ©2013
Virtual File Systems (VFS) – Second Layer
 VFS provide a common interface to multiple
different filesystem types

Thus, it separates file-system-generic
operations from their implementation detail.
 It provides a unique identifier (vnode) for files
across the entire space, including across all
filesystems of different types. (Note that: UNIX
inodes are unique only across a single
filesystem)

Therefore, provides a mechanism for uniquely
representing a file throughout a network.
Operating System Concepts – 9th Edition
12.26
Silberschatz, Galvin and Gagne ©2013
Virtual File Systems (Cont.)
Operating System Concepts – 9th Edition
12.27
Silberschatz, Galvin and Gagne ©2013
Access Methods
The information in the file can be accessed in several
ways.
 Sequential access: Information in the file is
processed in order, one record after the other.
 Direct access: information in the file in no
particular order.
Operating System Concepts – 9th Edition
12.28
Silberschatz, Galvin and Gagne ©2013
Allocation Methods - Contiguous
An allocation method refers to how disk blocks are allocated for files:
Three allocation methods are in practice.
 Contiguous allocation – each file occupies set of contiguous blocks

Simple – only starting location (block #) and length (number of
blocks) are required

Problems include finding space for file, knowing file size, external
fragmentation, need for compaction off-line (downtime) or online
 Many newer file systems (i.e., Veritas File System) use a modified
contiguous allocation scheme
 Extent-based file systems allocate disk blocks in extents
(contiguous block of disks). Instead of blocks, extents are allocated
for file allocation

A file consists of one or more extents
Operating System Concepts – 9th Edition
12.29
Silberschatz, Galvin and Gagne ©2013
Contiguous Allocation
 Mapping from logical address to
physical address
 Block size = 512 = 29 bytes
 LA = Logical address
 LA/512 = Q (integer division)
 R = LA % 512 (remainder)
Q
LA/512
R
 Block to be accessed = Q +
starting address
 Displacement into block = R
Operating System Concepts – 9th Edition
12.30
Silberschatz, Galvin and Gagne ©2013
Allocation Methods - Linked
 Linked allocation – each file is a linked list of blocks

File ends at nil pointer

No external fragmentation, therefore no need for compaction

Each block contains pointer to next block

Free space management system called when new block
needed

Improve efficiency by clustering blocks into groups but
increases internal fragmentation

Reliability can be a problem (pointer is lost or damaged)

Locating a block can take many I/Os and disk seeks
Operating System Concepts – 9th Edition
12.31
Silberschatz, Galvin and Gagne ©2013
Linked Allocation
 Each file is a linked list of disk blocks: blocks may be scattered
anywhere on the disk
block
=
pointer
 Pointer = 4 bytes
 Mapping
Q
LA/508
R
 Block to be accessed is the Qth block in the linked chain of blocks
representing the file.
 Displacement into block = R + 4
Operating System Concepts – 9th Edition
12.32
Silberschatz, Galvin and Gagne ©2013
Linked Allocation
Operating System Concepts – 9th Edition
12.33
Silberschatz, Galvin and Gagne ©2013
Allocation Methods – FAT
 FAT (File Allocation Table) variation of Linked Allocation

Beginning of volume has table, indexed by block number

Directory entry contains the block# of the first block of the file

The table entry indexed by that block# contains the block# of
the next block in the file

Chain continues until it reaches the last block


Its table entry has a special end-of-file value
An unused block is indicated by a table value of 0
 Much like a linked list, but faster on disk and cacheable
 New block allocation simple
 Can result in a significant number of disk head seeks, unless the
FAT table is cached.
 Random-access time is better than Linked allocation.
Operating System Concepts – 9th Edition
12.34
Silberschatz, Galvin and Gagne ©2013
File-Allocation Table
Operating System Concepts – 9th Edition
12.35
Silberschatz, Galvin and Gagne ©2013
Allocation Methods - Indexed
 Indexed allocation

Each file has its own index block(s) of pointers to its data blocks

ith entry in the index block points to the ith block of the file

Directory contains the address of the index block
 Logical view
index table
Operating System Concepts – 9th Edition
12.36
Silberschatz, Galvin and Gagne ©2013
Indexed Allocation (Cont.)
 Need index table
 Supports Random Access
 Dynamic access without external fragmentation, but have overhead
of index block
 Number of entries in the index block depends on the size of the
block and size of the pointer holding block addresses. If block size
= 512 bytes and pointer size = 4 bytes:

Total number of entries in the index block = 512/4 = 128
 Mapping from logical to physical in a file of maximum size of
64K bytes we need only 1 block for index table
Q = displacement into index table
R = displacement into block
Operating System Concepts – 9th Edition
12.37
Q
LA/512
R
Silberschatz, Galvin and Gagne ©2013
Example of Indexed Allocation
Operating System Concepts – 9th Edition
12.38
Silberschatz, Galvin and Gagne ©2013
Indexed Allocation Cont…
What happens if file is large such that the index block runs out of space
to hold data block addresses? Various scheme to deal with this:
 Linked Scheme - Link together several index blocks. The last entry
of an index block either holds the address of the next index block
(for large file) or is NULL (for small file)
 Multilevel Scheme - Multilevel index – A variant of linked
representation uses a first-level index block to point to a set of
second-level index blocks, which in turn point to the file blocks.

This approach could be continued to a third, fourth or n-th level,
depending on the desired maximum file size.

Two-level index (4K blocks could store 1,024 four-byte pointers
in outer index -> 1,048,567 data blocks and file size of up to
4GB)
 Combined Scheme - Uses direct blocks (addresses of the data
blocks) and indirect blocks (addresses of the index blocks).
Operating System Concepts – 9th Edition
12.39
Silberschatz, Galvin and Gagne ©2013
Indexed Allocation – Multi-Level Scheme
Operating System Concepts – 9th Edition
12.40
Silberschatz, Galvin and Gagne ©2013
Combined Scheme: UNIX UFS
Operating System Concepts – 9th Edition
12.41
Silberschatz, Galvin and Gagne ©2013
Performance
 Best method depends on file access type

Contiguous great for sequential and random for
relatively small files. However, suffers from external
fragmentation.
 Linked good for sequential access for large files, but
inefficient for random access
 Indexed more complex

Good for both sequential and random access, for large
files.
 Declare access type at creation -> select either contiguous
or linked
Operating System Concepts – 9th Edition
12.42
Silberschatz, Galvin and Gagne ©2013
Performance (Cont.)
 Adding instructions to the execution path to save one disk I/O is
reasonable

Intel Core i7 Extreme Edition 990x (2011) at 3.46Ghz = 159,000
MIPS


Typical disk drive at 250 I/Os per second


http://en.wikipedia.org/wiki/Instructions_per_second
159,000 MIPS / 250 = 630 million instructions during one
disk I/O
Fast SSD drives provide 60,000 IOPS

159,000 MIPS / 60,000 = 2.65 millions instructions during
one disk I/O
Operating System Concepts – 9th Edition
12.43
Silberschatz, Galvin and Gagne ©2013
Free-Space Management
 File system maintains free-space list to track available
blocks/clusters
 Bit vector or bit map (n blocks)
0 1
2
n-1

…
bit[i] =
1  block[i] free
0  block[i] occupied
 Simple and easy to find first free block or n consecutive blocks
 Need to be kept in memory for efficiency.

Also, to be written to disk occasionally – Why?
Operating System Concepts – 9th Edition
12.44
Silberschatz, Galvin and Gagne ©2013
Free-Space Management (Cont.)
 Inefficient as it requires lot of space.

Example:
block size = 4KB = 212 bytes
disk size = 240 bytes (1 terabyte)
n = 240/212 = 228 bits (or 32MB)
if clusters of 4 blocks -> 8MB of memory
Operating System Concepts – 9th Edition
12.45
Silberschatz, Galvin and Gagne ©2013
Linked Free Space List on Disk

Linked list -keep track of all free blocks.

Traversing the list and/or finding a
contiguous block of a given size are not
easy (we must read each block, which
requires substantial I/O time), but
fortunately are not frequently needed
operations.

Generally the system just adds and
removes single blocks from the beginning
of the list.
Operating System Concepts – 9th Edition
12.46
Silberschatz, Galvin and Gagne ©2013
Free-Space Management (Cont.)
 Grouping

Modify linked list to store address of next n-1 free blocks in first
free block, plus a pointer to next block that contains free-blockpointers.
 Counting

Because space is frequently contiguously used and freed, with
contiguous-allocation allocation, extents, or clustering

Keep address of first free block and count of following free
blocks

Free space list then has entries containing addresses and
counts
Operating System Concepts – 9th Edition
12.47
Silberschatz, Galvin and Gagne ©2013
Free-Space Management (Cont.)
 Space Maps

Sun's ZFS file system was designed for HUGE numbers and sizes of
files, directories, and even file systems.

The resulting data structures could be VERY inefficient if not
implemented carefully.
 Divides device space into metaslab units and manages metaslabs
Given volume can contain hundreds of metaslabs
 Each metaslab has associated space map
 Uses counting technique, records to log file rather than file
system


An in-memory space map is constructed using a balanced tree data
structure, constructed from the log data.

The combination of the in-memory tree and the on-disk log provide
for very fast and efficient management of these very large files and
free blocks
Operating System Concepts – 9th Edition
12.48
Silberschatz, Galvin and Gagne ©2013
Sharing and Protection
 Sharing of files on multi-user systems is desirable

Enabling this results in storing more information about the file.
 Sharing may be done through a protection scheme
 Files must be kept safe for reliability ( against accidental damage ),
and protection ( against deliberate malicious access. )
 Types of access

Read, Write, Execute, Append, Delete and List

UNIX uses a set of 9 access control bits, in three groups of
three. These correspond to R (read), W (write), and X (execute)
permissions for each of the Owner, Group, and

In addition to the above Unix has other special bits to control
access.
Operating System Concepts – 9th Edition
12.49
Silberschatz, Galvin and Gagne ©2013
Access Lists and Groups in Linux
 Mode of access: R (read), W (write), and X (execute)
 Three classes of users on Unix / Linux: Owner, Group, and Others.
 Each class of user is given RWX permission using bits (0 = deny
access and 1 to grant access).
 File/Directory permissions set using chmod command.
 Example: Suppose we want to set the below permissions on a file
named test. txt:
a) owner access
7

b) group access
6

c) public access
1

RWX
111
RWX
110
RWX
001
 The command in UNIX is => chmod 761 test.txt
Operating System Concepts – 9th Edition
12.50
Silberschatz, Galvin and Gagne ©2013
Question
What access rights does the following command:
chmod 751 test.txt
specify on the file test.txt?
Operating System Concepts – 9th Edition
12.51
Silberschatz, Galvin and Gagne ©2013
End of Chapter 11 & 12
Operating System Concepts – 9th Edition
Silberschatz, Galvin and Gagne ©2013