Download Data::Dumper - Monash University

Topic 8: Databases
CSE2395/CSE3395
Perl Programming
Llama3 chapter 16, pages 221-224
Camel3 pages 363-398
Programming the Perl DBI chapters 2-5, pages 7-135
perlfunc, perlmod, perlmodlib, Data::Dumper, perltie,
DB_File, DBI manpages
In this topic
 Modules
►
►
►
►
use Module;
Perl standard library
CPAN
writing modules
 Databases
►
►
►
persistent data
Data::Dumper
DBM
– databases in files
►
pack and unpack
– flat-file databases
►
DBI
– interfacing with SQL databases
2
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Modules
 Much of Perl’s functionality is contained in
optional modules
 A module is a file which contains re-usable Perl
code for a particular task
Your Perl
code
Module
imported
module
Your code is importing;
module is exporting
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
3
Modules
 Modules can be
►
found in the Perl Standard Library
– supplied with Perl and already installed
►
downloaded from CPAN
– tar xzf Module.tar.gz
– cd Module
– perl Makefile.PL
– make
– make install
►
written by you
– using package keyword and saving file as Module.pm
Camel3 pages 299-307, 831-915
perlmod, perlmodlib, perlfunc manpages
4
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Using a module
 Import a module into your code with use
keyword at top of your program
 use Data::Dumper;
 use CGI ":all";
 use Dog qw(bark bite);
 Read module documentation to see how to
access module’s functionality
►
►
perldoc Data::Dumper
perldoc CGI
Llama3 pages 282-283
Camel3 pages 299-300, 822-823
perlfunc manpage
5
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Writing a module
 To create a module called Dog:
►
start file with package Dog;
– package keyword ensures that Dog’s variables and functions don’t
interfere with anyone else’s
►
►
►
use Exporter;
@ISA = qw(Exporter);
set @EXPORT_OK to a list of function names (bark, bite)
– these two lines allow Perl to put Dog’s functions into code that says
use Dog ("bark", "bite");
►
►
write the functions for Dog (bark, bite)
end file with a nonzero expression
– needed or else use Dog will fail
►
save file as Dog.pm in current directory
– or any directory named in special Perl @INC array
Camel3 pages 301-305
perlmod, perlfunc manpages
6
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Timeout
package DayOfWeek;
use Exporter; @ISA = qw(Exporter);
@EXPORT_OK = "weekday"; # Allow import of &weekday.
# weekday(y,m,d): Return weekday of date (0 = Sun).
sub weekday
{
my ($y, $m, $d) = @_;
# Uses array references (topic 10).
( [[6,2,2,5,0,3,5,1,4,6,2,4],
[6,2,3,6,1,4,6,2,5,0,3,5]] -> [
((! ($y % 400) || ($y % 100)) && (! ($y % 4)))
|| 0 ][ $m-1 ] + $d + $y + int(($y-1)/4)
- int(($y-1)/100) + int(($y-1)/400)
) % 7;
}
1; # Indicate successful definition of module.
7
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Timeout
# Import the weekday function from DayOfWeek.pm
use DayOfWeek qw(weekday);
# Names of days of week.
@days = qw(Sunday Monday Tuesday
Wednesday Thursday Friday Saturday);
# Check usage.
die "Usage: $0 year month day\n"
if @ARGV != 3;
$daynum = weekday(@ARGV[0..2]);
print "That day is $days[$daynum]\n";
8
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Databases
 A database
contains organized data
► contains relations between data
► is persistent
►
 A hash
contains organized data
► has a key/value relationship
► ceases to exist at end of scope
►
– never beyond end of program
 A persistent hash would be great
►
but how?
9
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Persistent data
 To make a variable persistent, need a way to make it last
between invocations of program
►
by saving its state in a file
 Write-back (“freeze/thaw”, serialization) strategy
►
►
►
►
dump the contents of variable to file at program exit (“freeze”)
initialize variable with dumped contents at program entry (“thaw”)
file is up-to-date only when program is not running
for example: Data::Dumper, Storable
 Write-through (“tie”) strategy
►
►
►
►
update file every time the variable is changed
requires help from Perl to tie the variable to the file
file is always up-to-date
for example: DBM files
10
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Data::Dumper
 Perl Standard Library module
►
use Data::Dumper;
 Example of write-back (freeze/thaw) strategy
 Writes Perl code to file
►
when later evaluated by Perl, data is reconstructed
 To dump (freeze) a variable %hash:
►
►
open the persistent file for writing
Data::Dumper->Dump([\%hash], [qw(*hash)]);
– [...] is array reference, see topic 10.
 To restore (thaw) from a file file:
►
►
do "file";
Perl evaluates the statements in file, reconstructing the
variable %hash
Camel3 pages 702, 882
Data::Dumper, perlfunc manpages
11
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Timeout
# Freezing and thawing some variables.
use Data::Dumper;
# Thaw saved values, if they exist.
if (-f "storage") {
do "storage"; # Restore previous values.
} else {
$important = 0; %valuable = (); # First run.
}
# ... do stuff with $important and %valuable ...
# Freeze values for next time before program exit.
open STORAGE, ">storage";
print STORAGE Data::Dumper->Dump(
[$important, \%valuable],
[qw(important *valuable)]);
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
12
DBM files
 DBM files are a simple database format common
to Unix
►
also available on other platforms
 Data stored in a binary file
►
usually with a sophisticated data structure like a Btree
 Roughly equivalent to a hash
can look up value given key
► keys are strings, values are strings
► may be arbitrary limits on data size
►
Llama3 pages 221-222
Camel3 pages 696-697, 883-884, 985
perlmod manpage
13
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
DBM files
 Perl provides a simple interface to DBM files
using dbmopen or tie functions
► connects a hash to the DBM file
► accessing the hash transparently accesses the file
instead
►
– all hash operations supported
►
example of write-through (tie) strategy
 To tie a hash %hash to a file file:
►
either dbmopen %hash, "file", 0600
– mode (0600) only used if file needs to be created
►
or tie %hash, "DB_file", "file"
Llama3 pages 222-224
Camel3 pages 363-398, 697-698
perltie, perlfunc manpages
14
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Timeout
# Use DBM to track mail aliases.
# Use Berkeley-DB format for DBM file.
# This is the most flexible one.
use DB_File;
dbmopen %alias, "aliases", 0600
or die "Can't tie to DBM file: $!\n";
# Define functions that modify file
# through %alias hash.
sub get_alias { return $alias{$_[0]}; }
sub set_alias { $alias{$_[0]} = $_[1]; }
sub delete_alias { delete $alias{$_[0]}; }
sub exists_alias { return exists $alias{$_[0]}; }
15
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
pack and unpack
 Files store only strings
 If data other than strings are to be stored, need to
encode into strings before storing
►
and decode from strings when fetching
 To encode data into a string, use pack
►
$string = pack "a10 l", $word, $int;
 To decode data from a string, use unpack
►
($word, $int) = unpack "a10 l", $string;
 Format string ("a10 l") specifies how to pack or
unpack
►
dozens of different specifiers
Llama3 page 224
Camel3 pages 757-762, 819-821
perlfunc manpage
16
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Random-access files
 Can store fixed-length records in a file
►
e.g., 30 bytes per record
– 20 for name, 8 for ID number, 1 for age, 1 for mark
bytes 0 to 29 for first record
► bytes 30 to 59 for second record
► etc.
►
record 1
record 2
record 3
17
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Random-access files
 Open file for both reading and writing
 To modify or create record
create record with pack
► seek to position in file with seek
► write packed record to file with print
►
 To read record
seek to position in file with seek
► read appropriate number of bytes with read
► extract fields from string with unpack
►
Llama3 pages 225-227
Camel3 pages 769, 780
perlfunc manpage
18
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Timeout
open FILE, "+< database"; # Open for read/write.
# write_record(recno, name, id, age, mark)
sub write_record {
my $recno = shift;
my $string = pack "a20 a8 c c", @_;
seek FILE, ($recno-1)*30, 0;
print FILE $string;
}
# (name, id, age, mark) = read_record(recno)
sub read_record {
my $recno = shift;
seek FILE, ($recno-1)*30, 0;
read FILE, $string, 30;
return unpack "a20 a8 c c", $string;
}
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
19
Client/server databases
 Sometimes necessary to interact with an existing
database
►
database is kept on a server
– e.g., Oracle, MySQL, Microsoft Access (ODBC), etc.
– server handles mundane aspects like storage, access control,
file/record locking, etc.
►
your program is client
– makes requests to server on database’s behalf
 Need to interact with server using its interface
►
usually requires making a network connection
 Commonly done with SQL
►
►
Structured Query Language, a standard way of phrasing queries
and database transformations
SQL is not examinable in this subject!
 But
SQL dialects vary slightly
► network protocols and interfaces vary
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
►
20
DBI
 Perl provides a unified interface for all database
servers
►
►
called Database Interface, or DBI
use DBI;
 Client uses DBI to access database
DBI speaks to database through a system-specific
database driver (DBD)
► DBD translates client’s SQL request for specific
server dialects
► DBD massages server responses into standard form
for client
►
21
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
DBI
DBD::Oracle
(driver)
Your Perl
program
Oracle
database
Oracle
database
DBI
(interface)
DBD::CSV
(driver)
CSV
database
DBI system
Always one
instance of DBI
One driver per type
of database
22
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
DBI handles
 DBI system uses handles to manage
connections to database system
►
database handle
– one per database
– used to perform instantaneous actions and to initiate longer
statements
►
statement handle
– one per SQL statement
– usually used to perform lengthy SELECT queries
23
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Connecting to a database
 Need to know database’s data source name
Always
DBI
Database driver
DBI:mSQL:frotz.org:base:4910
Database identifier; format differs
for each driver (here: hostname,
database name, port number
24
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Timeout
use DBI;
# Connect to the database.
# OO: class->method() notation is how to call
# a class method (in this case, a constructor).
my $db_handle = DBI->connect(
"DBI:Ingres:data", $username, $passwd,
{ PrintError => 1 } # Hash ref: topic 10
) or die "Can't connect: $DBI::errstr\n";
# ... do stuff ...
# Now disconnect.
# OO: $object->method() notation is how to call
# an object method (here, a destructor).
$db_handle->disconnect()
or warn "Disconnection error: $DBI::errstr\n";
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
25
Executing DBI statements
 Actions on database are performed through database
handle
 Simple (atomic) statements can be performed using do
method
►
►
$db_handle->do("DELETE FROM ...");
return value indicates success or failure
 Complex statements, which may produce several results
in sequence, require a statement handle
►
►
$st_handle = $db_handle->prepare("SELECT...");
statement handle used to iterate over results
– @fields = $st_handle->fetchrow_array()
26
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Timeout
# Insert a name into a phone database
# usage: thisprogram name number
use DBI;
# ... connect to the database ...
# quote() method correctly escapes strings
# with metacharacters in them.
$db_handle->do("
INSERT INTO phonedir (name, phone_number)
VALUES ( " . $db_handle->quote($ARGV[0]) .
", " . $db_handle->quote($ARGV[1]) .
")"
) or warn "Insert error: $DBI::errstr\n";
27
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Timeout
# Extract entries from the phone database
# usage: thisprogram name
# ... connect to the database ...
$st_handle = $db_handle->prepare("
SELECT name, phone_number
FROM phonedir
WHERE name LIKE " . $db_handle->quote($ARGV[0])
);
$st_handle->execute();
# Iterate over the data returned by the query.
while (($name, $num) = $st_handle->fetchrow_array())
{
print "$name: $num\n";
}
28
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Covered in this topic
 Modules
►
►
►
►
use Module;
Perl standard library
CPAN
writing modules
 Databases
►
►
►
persistent data
Data::Dumper
DBM
– databases in files
►
pack and unpack
– flat-file databases
►
DBI
– interfacing with SQL databases
29
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Going further
 Tying
adding new behaviour to built-in-types
► perldoc -f tie
►
 Transactions
protecting databases from corruption
► supported by some DBD modules
►
 Multi-level DBM files
storing nested data structures in a DBM file
► MLDBM module
►
 ODBC
Microsoft’s database standard
► DBI::ODBC and Win32::ODBC modules
►
30
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University
Next topic
 The World Wide Web
 Writing a Perl web client
►
LWP module
 Dynamic web pages
►
Common Gateway Interface (CGI)
LWP, CGI manpages
31
Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University