Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Implementing Processes, Threads, and Resources
Document related concepts
Object storage wikipedia , lookup
Library (computing) wikipedia , lookup
MTS system architecture wikipedia , lookup
Plan 9 from Bell Labs wikipedia , lookup
Commodore DOS wikipedia , lookup
Spring (operating system) wikipedia , lookup
Burroughs MCP wikipedia , lookup
File locking wikipedia , lookup
Transcript
Slide 13-1
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-2
File
Management
Copyright © 2004 Pearson Education, Inc.
13
Operating Systems: A Modern Perspective, Chapter 13
Fig 13-2: The External View of the File Manager
Application
Program
Memory Mgr
Process Mgr
File Mgr
UNIX
Device Mgr
WriteFile()
CreateFile()
CloseHandle() ReadFile()
SetFilePointer()
Memory Mgr
Process Mgr
Device Mgr
File Mgr
mount()
write()
close() open()
read()
lseek()
Windows
Hardware
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-3
Why Programmers Need Files
<head>
…
</head>
<body>
…
</body>
HTML
Editor
Slide 13-4
Web
Browser
foo.html
<head>
…
</head>
<body>
…
</body>
File
Manager
• Persistent storage
• Shared device
Copyright © 2004 Pearson Education, Inc.
File
Manager
• Structured information
• Can be read by any applic
• Accessibility
• Protocol
Operating Systems: A Modern Perspective, Chapter 13
File Management
• File is a named, ordered collection of
information
• The file manager administers the collection
by:
–
–
–
–
Storing the information on a device
Mapping the block storage to a logical view
Allocating/deallocating storage
Providing file directories
• What abstraction should be presented to
programmer?
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-5
Information Structure
Applications
Records
Structured Record Files
Record-Stream Translation
Byte Stream Files
Stream-Block Translation
Storage device
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-6
Byte Stream File Interface
fileID = open(fileName)
close(fileID)
read(fileID, buffer, length)
write(fileID, buffer, length)
seek(fileID, filePosition)
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-7
Low Level Files
fid = open(“fileName”,…);
…
read(fid, buf, buflen);
…
close(fid);
int
int
int
int
int
open(…) {…}
close(…) {…}
read(…) {…}
write(…) {…}
seek(…) {…}
b0 b1 b2
...
Slide 13-8
bi ...
Stream-Block Translation
Storage device response to commands
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Structured Files
Records
Record-Block Translation
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-9
Record-Oriented Sequential Files
Logical Record
fileID = open(fileName)
close(fileID)
getRecord(fileID, record)
putRecord(fileID, record)
seek(fileID, position)
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-10
Slide 13-11
Record-Oriented Sequential Files
Logical Record
H byte header
k byte logical record
...
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-12
Record-Oriented Sequential Files
Logical Record
H byte header
k byte logical record
...
...
Physical Storage Blocks
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Fragment
Electronic Mail Example
Slide 13-13
struct message {
putRecord(struct message *msg) {
/* The mail message */
putAddress(msg->to);
address to;
putAddress(msg->from);
address from;
putAddress(msg->cc);
line subject;
putLine(msg->subject);
address cc;
putString(msg->body);
string body;
}
};
struct message *getRecord(void) {
struct message *msg;
msg = allocate(sizeof(message));
msg->to = getAddress(...);
msg->from = getAddress(...);
msg->cc = getAddress(...);
msg->subject = getLine();
msg->body = getString();
return(msg);
}
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Indexed Sequential File
• Suppose we want to directly access records
• Add an index to the file
fileID = open(fileName)
close(fileID)
getRecord(fileID, index)
index = putRecord(fileID, record)
deleteRecord(fileID, index)
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-14
Indexed Sequential File (cont)
Slide 13-15
Application structure
Account #
012345
123456
294376
...
529366
...
965987
Copyright © 2004 Pearson Education, Inc.
Index
index = i
i
k
index = k
j
index = j
Operating Systems: A Modern Perspective, Chapter 13
More Abstract Files
• Inverted files
– System index for each datum in the file
• Databases
– More elaborate indexing mechanism
– DDL & DML
• Multimedia storage
– Records contain radically different types
– Access methods must be general
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-16
Implementing Low Level Files
• Secondary storage device contains:
– Volume directory (sometimes a root directory
for a file system)
– External file descriptor for each file
– The file contents
• Manages blocks
– Assigns blocks to files (descriptor keeps track)
– Keeps track of available blocks
• Maps to/from byte stream
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-17
Disk Organization
Boot Sector
Volume Directory
…
Blk0
Blk1
Blkk
Blkk+1
Blkk-1
Track 0, Cylinder 0
Blk2k-1
Track 0, Cylinder 1
…
Blk
Track 1, Cylinder 0
…
Blk
Track N-1, Cylinder 0
…
Blk
Track N-1, Cylinder M-1
…
…
Blk
Blk
…
Blk
Blk
…
Blk
Copyright © 2004 Pearson Education, Inc.
Blk
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-18
Low-level File System Architecture
Block 0
b0 b1 b2 b3 …
…
bn-1
...
Sequential Device
Copyright © 2004 Pearson Education, Inc.
Randomly Accessed Device
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-19
File Descriptors
•External name
•Current state
•Sharable
•Owner
•User
•Locks
•Protection settings
•Length
•Time of creation
•Time of last modification
•Time of last access
•Reference count
•Storage device details
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-20
An open() Operation
Slide 13-21
• Locate the on-device (external) file
descriptor
• Extract info needed to read/write file
• Authenticate that process can access the file
• Create an internal file descriptor in primary
memory
• Create an entry in a “per process” open file
status table
• Allocate resources, e.g., buffers, to support
file usage
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-22
File Manager Data Structures
2 Keep the state
of the processfile session
3 Return a
reference to
the data
structure
Process-File
Session
Open File
Descriptor
External File Descriptor
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
1 Copy info from
external to the
open file
descriptor
Opening a UNIX File
fid = open(“fileA”, flags);
…
read(fid, buffer, len);
0
1
2
3
stdin
stdout
stderr
...
Slide 13-23
On-Device File Descriptor
File structure
inode
Open File Table
Internal File Descriptor
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Block Management
• The job of selecting & assigning storage
blocks to the file
• For a fixed sized file of k blocks
– File of length m requires N = m/k blocks
– Byte bi is stored in block i/k
• Three basic strategies:
– Contiguous allocation
– Linked lists
– Indexed allocation
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-24
Contiguous Allocation
Slide 13-25
• Maps the N blocks into N contiguous blocks
on the secondary storage device
• Difficult to support dynamic file sizes
File descriptor
Head position
…
First block
Number of blocks
Copyright © 2004 Pearson Education, Inc.
237
785
25
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-26
Linked Lists
• Each block contains a header with
– Number of bytes in the block
– Pointer to next block
• Blocks need not be contiguous
• Files can expand and contract
• Seeks can be slow
First block
…
Head: 417
...
Copyright © 2004 Pearson Education, Inc.
Length
Length
Length
Byte 0
...
Byte 4095
Byte 0
...
Byte 4095
Byte 0
...
Byte 4095
Block 0
Block 1
Block N-1
Operating Systems: A Modern Perspective, Chapter 13
Indexed Allocation
Slide 13-27
• Extract headers and put them in an index
• Simplify seeks
• May link indices together (for large files)
Index block
…
Head: 417
...
Byte 0
...
Byte 4095
Length
Block 0
Byte 0
...
Byte 4095
Length
Length
Byte 0
...
Byte 4095
Block N-1
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Block 1
Slide 13-28
DOS FAT Files
File Descriptor
43
Disk
Block
254
Disk
Block
…
107
Disk
Block
File Descriptor
43
107
254
43
Disk
Block
Disk
Block
…
254
File Access Table (FAT)Operating Systems: A Modern Perspective, Chapter 13
Copyright © 2004 Pearson Education, Inc.
107
Disk
Block
inode
UNIX Files
mode
owner
…
Direct block 0
Direct block 1
…
Direct block 11
Single indirect
Double indirect
Triple indirect
Slide 13-29
Data
Data
Data
Index
Data
Index
Data
Index
Index
Data
Index
Index
Data
Index
Index
Index
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Data
Data
Unallocated Blocks
Slide 13-30
• How should unallocated blocks be managed?
• Need a data structure to keep track of them
– Linked list
• Very large
• Hard to manage spatial locality
– Block status map (“disk map”)
• Bit per block
• Easy to identify nearby free blocks
• Useful for disk recovery
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Marshalling the Byte Stream
Slide 13-31
• Must read at least one buffer ahead on input
• Must write at least one buffer behind on
output
• Seek flushing the current buffer and
finding the correct one to load into memory
• Inserting/deleting bytes in the interior of the
stream
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Full Block Buffering
Slide 13-32
• Storage devices use block I/O
• Files place an explicit order on the bytes
• Therefore, it is possible to predict what is likely to
be read after bytei
• When file is opened, manager reads as many
blocks ahead as feasible
• After a block is logically written, it is queued for
writing behind, whenever the disk is available
• Buffer pool – usually variably sized, depending on
virtual memory needs
– Interaction with the device manager and memory
manager
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Directories
• A set of logically associated files and sub
directories
• File manager provides set of controls:
– enumerate
– copy
– rename
– delete
– traverse
– etc.
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-33
Directory Structures
• How should files be organized within
directory?
– Flat name space
• All files appear in a single directory
– Hierarchical name space
• Directory contains files and subdirectories
• Each file/directory appears as an entry in exactly
one other directory -- a tree
• Popular variant: All directories form a tree, but a
file can have multiple parents.
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-34
Directory Implementation
Slide 13-35
• Device Directory
– A device can contain a collection of files
– Easier to manage if there is a root for every file
on the device -- the device root directory
• File Directory
– Typical implementations have directories
implemented as a file with a special format
– Entries in a file directory are handles for other
files (which can be files or subdirectories)
Copyright © 2004 Pearson Education, Inc.
Operating Systems: A Modern Perspective, Chapter 13
Slide 13-36
UNIX mount Command
/
/
bin usr etc
bill
bin usr etc
foo
bill
nutt
foo
/
nutt
FS
abc cde
xyz
/
blah
FS
abc cde
xyz
blah
Copyright © 2004 Pearson Education, Inc.
mount FS at foo
Operating Systems: A Modern Perspective, Chapter 13
VFS-based File Manager
Exports OS-specific API
File System Independent
Part of File Manager
Virtual File System Switch
MS-DOS Part of
File Manager
Copyright © 2004 Pearson Education, Inc.
ISO 9660 Part of
File Manager
…
Operating Systems: A Modern Perspective, Chapter 13
ext2 Part of
File Manager
Slide 13-37
Rensselaer Polytechnic Institute
CSCI-4210 – Operating Systems
David Goldschmidt, Ph.D.
very fast
very small
volatile
managed as
part of the
filesystem
very slow
non-volatile
very large
A File Management System is a set of OS
services that supports files and directories
for user applications, OS programs, etc.
Data should be organized and
available in a convenient and
efficient manner
Files are the basic
building blocks
A file is an abstraction that represents user
data, OS data, an executable, a device, etc.
A file is simply a sequence of bytes
Actual storage location (via network?) and
format are transparent to users
Storage scheme on disk is also transparent
▪ Typically involves cylinders, tracks, sectors, etc.
File attributes include:
Human-readable symbolic name
Type (e.g. executable, directory, text, PDF, etc.)
Logical location (i.e. containing directory or path)
Physical location on disk
Size (in bytes)
Protection or security (i.e. permissions)
Timestamps (created, last modified, last accessed)
The pathname (or just path) of a file
specifies the sequence of directories
one must traverse to locate
the file
An absolute path starts
at the root node
A relative path starts
anywhere
A link provides a shortcut to a file and may
circumvent the given directory hierarchy
A hard link in Unix is
indistinguishable from
the original file
A symbolic link in Unix
is merely a shortcut
A Windows shortcut
is just a symbolic link
File creation requires space allocation
Opening a file returns a handle or file descriptor
Read and write operations
use the handle and an
offset (or file pointer)
The close operation simply
deletes the handle and
deallocates any memory
Deleting a file deallocates all disk space
marked as in use by the file
But likely does not erase file contents
Deleted files are recoverable until the disk space
is used for (and overwritten by) other file(s)
The delete operation also removes the
corresponding entry in the containing directory
An access method describes the manner
and mechanisms by which a process
accesses the data in a file
Two common access methods:
Sequential access (open, read, write, close)
Random access (open, read, write, seek, close)
In a contiguous disk space allocation scheme,
files are allocated to contiguous blocks of
disk space
A
A
A
A
Four files allocated contiguously to disk:
A
A
B
B
C
C
C
D
File B outgrows its space and is reallocated:
A
A
C
C
C
D
B
B
B
B
File D outgrows its space and is reallocated:
A
A
C
C
C
B
B
B
B
D
D
Defragmentation combines free disk space:
A
A
C
C
C
B
B
B
B
D
D
In a clustered disk space allocation scheme,
files are allocated to clusters of disk space
on an as needed basis
A
A
A
Four files allocated contiguously to disk:
A
A
B
B
C
C
C
D
File B outgrows its space, so one or more
new clusters are allocated:
A
A
B
B
C
C
C
D
B
B
File D outgrows its space, so one or more
new clusters are allocated:
A
A
B
B
C
C
C
D
B
B
D
File accesses may cross cluster boundaries,
causing slower response times
Using a linked approach,
each cluster has a
pointer to the next cluster
Using an indexed approach, a single
table maintains pointers to each
individual cluster
Managing free blocks (or free clusters) may
follow the same scheme as for a single file
Linked scheme
Indexed approach
Bitmapped approach:
A bitmap is maintained in memory
Each bit corresponds to a block or cluster
A 0 indicates a free block
A 1 indicates the block is in use
A file allocation table (FAT) maps logical files
to their physical addresses on a disk
A pure FAT would have an entry
for each sector (e.g. 512 bytes)
To improve performance, clusters
of contiguous sectors are used
Clusters range from 4 to 64 sectors
(e.g. 2,048 to 32,768 bytes)
Microsoft FAT-32 filesystem maps to clusters
May require linked clusters to house entire FAT
Linux uses indexed clusters
In Unix, file information is stored in an index
node (I-node), which contains:
Mode and type of file
Number of links to the file
File owner’s userid and groupid
Size of file (in bytes)
Last access and last modified times
Number of blocks allocated to the file
Pointers to the first twelve blocks
Pointers to three additional blocks of pointers
inode
12
…
1024
single indirect
10242
10243
cluster
cluster
cluster
inode
1024
cluster
…
cluster
double indirect
inode
1024
triple indirect
inode
10242
cluster
…
…
inode
1024
inode
1024
…
…
A virtual filesystem provides transparent
access to different filesystem types on
multiple device types and disk partitions
A virtual machine is a layered approach that
logically combines the kernel operating
system and hardware
Creates the illusion
of multiple processes,
each executing on its
own virtual processor
with its own virtual
memory
http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9002552
Non-virtual machine
Virtual machine
Java programs execute on a native Java
Virtual Machine (JVM)
Virtual machines provide complete
protection of system resources
Each virtual machine is isolated from
all other virtual machines
▪ which prohibits direct sharing
of system resources
Virtual machines can be difficult to implement
due to the effort required to provide
an exact duplicate of each underlying machine
