Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Topic 8: Databases CSE2395/CSE3395 Perl Programming Llama3 chapter 16, pages 221-224 Camel3 pages 363-398 Programming the Perl DBI chapters 2-5, pages 7-135 perlfunc, perlmod, perlmodlib, Data::Dumper, perltie, DB_File, DBI manpages In this topic Modules ► ► ► ► use Module; Perl standard library CPAN writing modules Databases ► ► ► persistent data Data::Dumper DBM – databases in files ► pack and unpack – flat-file databases ► DBI – interfacing with SQL databases 2 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Modules Much of Perl’s functionality is contained in optional modules A module is a file which contains re-usable Perl code for a particular task Your Perl code Module imported module Your code is importing; module is exporting Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University 3 Modules Modules can be ► found in the Perl Standard Library – supplied with Perl and already installed ► downloaded from CPAN – tar xzf Module.tar.gz – cd Module – perl Makefile.PL – make – make install ► written by you – using package keyword and saving file as Module.pm Camel3 pages 299-307, 831-915 perlmod, perlmodlib, perlfunc manpages 4 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Using a module Import a module into your code with use keyword at top of your program use Data::Dumper; use CGI ":all"; use Dog qw(bark bite); Read module documentation to see how to access module’s functionality ► ► perldoc Data::Dumper perldoc CGI Llama3 pages 282-283 Camel3 pages 299-300, 822-823 perlfunc manpage 5 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Writing a module To create a module called Dog: ► start file with package Dog; – package keyword ensures that Dog’s variables and functions don’t interfere with anyone else’s ► ► ► use Exporter; @ISA = qw(Exporter); set @EXPORT_OK to a list of function names (bark, bite) – these two lines allow Perl to put Dog’s functions into code that says use Dog ("bark", "bite"); ► ► write the functions for Dog (bark, bite) end file with a nonzero expression – needed or else use Dog will fail ► save file as Dog.pm in current directory – or any directory named in special Perl @INC array Camel3 pages 301-305 perlmod, perlfunc manpages 6 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Timeout package DayOfWeek; use Exporter; @ISA = qw(Exporter); @EXPORT_OK = "weekday"; # Allow import of &weekday. # weekday(y,m,d): Return weekday of date (0 = Sun). sub weekday { my ($y, $m, $d) = @_; # Uses array references (topic 10). ( [[6,2,2,5,0,3,5,1,4,6,2,4], [6,2,3,6,1,4,6,2,5,0,3,5]] -> [ ((! ($y % 400) || ($y % 100)) && (! ($y % 4))) || 0 ][ $m-1 ] + $d + $y + int(($y-1)/4) - int(($y-1)/100) + int(($y-1)/400) ) % 7; } 1; # Indicate successful definition of module. 7 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Timeout # Import the weekday function from DayOfWeek.pm use DayOfWeek qw(weekday); # Names of days of week. @days = qw(Sunday Monday Tuesday Wednesday Thursday Friday Saturday); # Check usage. die "Usage: $0 year month day\n" if @ARGV != 3; $daynum = weekday(@ARGV[0..2]); print "That day is $days[$daynum]\n"; 8 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Databases A database contains organized data ► contains relations between data ► is persistent ► A hash contains organized data ► has a key/value relationship ► ceases to exist at end of scope ► – never beyond end of program A persistent hash would be great ► but how? 9 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Persistent data To make a variable persistent, need a way to make it last between invocations of program ► by saving its state in a file Write-back (“freeze/thaw”, serialization) strategy ► ► ► ► dump the contents of variable to file at program exit (“freeze”) initialize variable with dumped contents at program entry (“thaw”) file is up-to-date only when program is not running for example: Data::Dumper, Storable Write-through (“tie”) strategy ► ► ► ► update file every time the variable is changed requires help from Perl to tie the variable to the file file is always up-to-date for example: DBM files 10 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Data::Dumper Perl Standard Library module ► use Data::Dumper; Example of write-back (freeze/thaw) strategy Writes Perl code to file ► when later evaluated by Perl, data is reconstructed To dump (freeze) a variable %hash: ► ► open the persistent file for writing Data::Dumper->Dump([\%hash], [qw(*hash)]); – [...] is array reference, see topic 10. To restore (thaw) from a file file: ► ► do "file"; Perl evaluates the statements in file, reconstructing the variable %hash Camel3 pages 702, 882 Data::Dumper, perlfunc manpages 11 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Timeout # Freezing and thawing some variables. use Data::Dumper; # Thaw saved values, if they exist. if (-f "storage") { do "storage"; # Restore previous values. } else { $important = 0; %valuable = (); # First run. } # ... do stuff with $important and %valuable ... # Freeze values for next time before program exit. open STORAGE, ">storage"; print STORAGE Data::Dumper->Dump( [$important, \%valuable], [qw(important *valuable)]); Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University 12 DBM files DBM files are a simple database format common to Unix ► also available on other platforms Data stored in a binary file ► usually with a sophisticated data structure like a Btree Roughly equivalent to a hash can look up value given key ► keys are strings, values are strings ► may be arbitrary limits on data size ► Llama3 pages 221-222 Camel3 pages 696-697, 883-884, 985 perlmod manpage 13 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University DBM files Perl provides a simple interface to DBM files using dbmopen or tie functions ► connects a hash to the DBM file ► accessing the hash transparently accesses the file instead ► – all hash operations supported ► example of write-through (tie) strategy To tie a hash %hash to a file file: ► either dbmopen %hash, "file", 0600 – mode (0600) only used if file needs to be created ► or tie %hash, "DB_file", "file" Llama3 pages 222-224 Camel3 pages 363-398, 697-698 perltie, perlfunc manpages 14 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Timeout # Use DBM to track mail aliases. # Use Berkeley-DB format for DBM file. # This is the most flexible one. use DB_File; dbmopen %alias, "aliases", 0600 or die "Can't tie to DBM file: $!\n"; # Define functions that modify file # through %alias hash. sub get_alias { return $alias{$_[0]}; } sub set_alias { $alias{$_[0]} = $_[1]; } sub delete_alias { delete $alias{$_[0]}; } sub exists_alias { return exists $alias{$_[0]}; } 15 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University pack and unpack Files store only strings If data other than strings are to be stored, need to encode into strings before storing ► and decode from strings when fetching To encode data into a string, use pack ► $string = pack "a10 l", $word, $int; To decode data from a string, use unpack ► ($word, $int) = unpack "a10 l", $string; Format string ("a10 l") specifies how to pack or unpack ► dozens of different specifiers Llama3 page 224 Camel3 pages 757-762, 819-821 perlfunc manpage 16 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Random-access files Can store fixed-length records in a file ► e.g., 30 bytes per record – 20 for name, 8 for ID number, 1 for age, 1 for mark bytes 0 to 29 for first record ► bytes 30 to 59 for second record ► etc. ► record 1 record 2 record 3 17 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Random-access files Open file for both reading and writing To modify or create record create record with pack ► seek to position in file with seek ► write packed record to file with print ► To read record seek to position in file with seek ► read appropriate number of bytes with read ► extract fields from string with unpack ► Llama3 pages 225-227 Camel3 pages 769, 780 perlfunc manpage 18 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Timeout open FILE, "+< database"; # Open for read/write. # write_record(recno, name, id, age, mark) sub write_record { my $recno = shift; my $string = pack "a20 a8 c c", @_; seek FILE, ($recno-1)*30, 0; print FILE $string; } # (name, id, age, mark) = read_record(recno) sub read_record { my $recno = shift; seek FILE, ($recno-1)*30, 0; read FILE, $string, 30; return unpack "a20 a8 c c", $string; } Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University 19 Client/server databases Sometimes necessary to interact with an existing database ► database is kept on a server – e.g., Oracle, MySQL, Microsoft Access (ODBC), etc. – server handles mundane aspects like storage, access control, file/record locking, etc. ► your program is client – makes requests to server on database’s behalf Need to interact with server using its interface ► usually requires making a network connection Commonly done with SQL ► ► Structured Query Language, a standard way of phrasing queries and database transformations SQL is not examinable in this subject! But SQL dialects vary slightly ► network protocols and interfaces vary Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University ► 20 DBI Perl provides a unified interface for all database servers ► ► called Database Interface, or DBI use DBI; Client uses DBI to access database DBI speaks to database through a system-specific database driver (DBD) ► DBD translates client’s SQL request for specific server dialects ► DBD massages server responses into standard form for client ► 21 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University DBI DBD::Oracle (driver) Your Perl program Oracle database Oracle database DBI (interface) DBD::CSV (driver) CSV database DBI system Always one instance of DBI One driver per type of database 22 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University DBI handles DBI system uses handles to manage connections to database system ► database handle – one per database – used to perform instantaneous actions and to initiate longer statements ► statement handle – one per SQL statement – usually used to perform lengthy SELECT queries 23 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Connecting to a database Need to know database’s data source name Always DBI Database driver DBI:mSQL:frotz.org:base:4910 Database identifier; format differs for each driver (here: hostname, database name, port number 24 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Timeout use DBI; # Connect to the database. # OO: class->method() notation is how to call # a class method (in this case, a constructor). my $db_handle = DBI->connect( "DBI:Ingres:data", $username, $passwd, { PrintError => 1 } # Hash ref: topic 10 ) or die "Can't connect: $DBI::errstr\n"; # ... do stuff ... # Now disconnect. # OO: $object->method() notation is how to call # an object method (here, a destructor). $db_handle->disconnect() or warn "Disconnection error: $DBI::errstr\n"; Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University 25 Executing DBI statements Actions on database are performed through database handle Simple (atomic) statements can be performed using do method ► ► $db_handle->do("DELETE FROM ..."); return value indicates success or failure Complex statements, which may produce several results in sequence, require a statement handle ► ► $st_handle = $db_handle->prepare("SELECT..."); statement handle used to iterate over results – @fields = $st_handle->fetchrow_array() 26 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Timeout # Insert a name into a phone database # usage: thisprogram name number use DBI; # ... connect to the database ... # quote() method correctly escapes strings # with metacharacters in them. $db_handle->do(" INSERT INTO phonedir (name, phone_number) VALUES ( " . $db_handle->quote($ARGV[0]) . ", " . $db_handle->quote($ARGV[1]) . ")" ) or warn "Insert error: $DBI::errstr\n"; 27 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Timeout # Extract entries from the phone database # usage: thisprogram name # ... connect to the database ... $st_handle = $db_handle->prepare(" SELECT name, phone_number FROM phonedir WHERE name LIKE " . $db_handle->quote($ARGV[0]) ); $st_handle->execute(); # Iterate over the data returned by the query. while (($name, $num) = $st_handle->fetchrow_array()) { print "$name: $num\n"; } 28 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Covered in this topic Modules ► ► ► ► use Module; Perl standard library CPAN writing modules Databases ► ► ► persistent data Data::Dumper DBM – databases in files ► pack and unpack – flat-file databases ► DBI – interfacing with SQL databases 29 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Going further Tying adding new behaviour to built-in-types ► perldoc -f tie ► Transactions protecting databases from corruption ► supported by some DBD modules ► Multi-level DBM files storing nested data structures in a DBM file ► MLDBM module ► ODBC Microsoft’s database standard ► DBI::ODBC and Win32::ODBC modules ► 30 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University Next topic The World Wide Web Writing a Perl web client ► LWP module Dynamic web pages ► Common Gateway Interface (CGI) LWP, CGI manpages 31 Original Slides by Debbie Pickett, Modified by David Abramson, 2006, Copyright Monash University