Download Week 12

Practical Session 12
SQL
Basic Python
SQLite
Relational Databases
• A relational database is a collection of data items organized as a set of
formally described tables from which data can be accessed easily
• A relational database is created using the relational model:
• Data Definition Language
• Used to build and destroy databases, create, and drop
• Data Manipulation Language
• Used to manipulate data in databases, insert, delete, and retrieve
• We will use SQL data definition and query language.
• Each database consists of a number of tables and each table has its own
primary key.
• Relational:
• Because you may have relations between the different tables.
Very good tutorial at: http://www.w3schools.com/sql/default.asp
Table
ID
Name
OfficeHours
1
2
3
4
5
6
Majeed
Boaz
Matan
Dan
Hagit
Tom
Wed 10:00-12:00
Tue 16:00-18:00
Tue 14:00-16:00
Tue 12:00-14:00
Tue 16:00-18:00
Tue 19:00-21:00
- Table name: TEACHING_ASSISTANTS
- Column name: Id, Name, Office Hours
- Column type: INT, VARCHAR(20), VARCHAR(20)
- Each table has primary key, must be unique and non-null: example: id
- Each line in table is called a record
- You may have foreign key columns, which is a primary key in other table, to denote relationship
between two different tables
ANSI SQL Data Types
• Character strings
•
•
•
•
CHARACTER(n) or CHAR(n) — fixed-width n-character string, padded with spaces as needed
CHARACTER VARYING(n) or VARCHAR(n) — variable-width string with a maximum size of n characters
NATIONAL CHARACTER(n) or NCHAR(n) — fixed width string supporting an international character set
NATIONAL CHARACTER VARYING(n) or NVARCHAR(n) — variable-width NCHAR string
• Bit strings
•
•
BIT(n) — an array of n bits
BIT VARYING(n) — an array of up to n bits
• Numbers
•
•
•
INTEGER and SMALLINT
FLOAT, REAL and DOUBLE PRECISION
NUMERIC(precision, scale) or DECIMAL(precision, scale)
• Date and time
•
•
•
•
•
DATE — for date values (e.g., 2011-05-03)
TIME — for time values (e.g., 15:51:36). The granularity of the time value is usually a tick (100 nanoseconds).
TIME WITH TIME ZONE or TIMETZ — the same as TIME, but including details about the time zone in question.
TIMESTAMP — This is a DATE and a TIME put together in one variable (e.g., 2011-05-03 15:51:36).
TIMESTAMP WITH TIME ZONE or TIMESTAMPTZ — the same as TIMESTAMP, but including details about the time zone in question.
Creating/Deleting a table
Primary Key
• A primary key is used to uniquely identify each row in a table.
• A primary key can consist of one or more fields on a table.
• When multiple fields are used as a primary key, they are called a
composite key.
• Primary key is inheritably unique.
Foreign Key
• A foreign key is a field(s) that point to the primary key of another
table.
• The purpose of the foreign key is to ensure referential integrity of the
data.
• Only values that are supposed to appear in the database are permitted.
Code
Example
TEACHING_ASSISTANTS
Column
type
characteristic
ID
int
Primary Key
Name
VARCHAR(50)
Office hours
VARCHAR(9)
PRACTICAL_SESSIONS
Column
type
characteristic
TA
int
Foreign Key
Group
int
Primary Key
Location
VARCHAR(50)
Time
VARCHAR(9)
Effect: PRACTICAL_SESSIONS table cannot contain information on a TA that is not in the
Teaching Assistant table.
SQL Commands
PRACTICAL_SESSIONS
TEACHING_ASSISTANTS
TA Group
Location Time
3
11
90-234
Sun 14-16
ID
Name
OfficeHours
3
12
34-205
Sun 18-20
4
13
90-125
Thu 14-16
6
21
28-145
Thu 14-16
1
22
28-107
Thu 14-16
2
23
72-213
Thu 14-16
5
31
90-145
Thu 14-16
1
2
3
4
5
6
Majeed
Boaz
Matan
Dan
Hagit
Tom
Wed 10:00-12:00
Tue 16:00-18:00
Tue 14:00-16:00
Tue 12:00-14:00
Tue 16:00-18:00
Tue 19:00-21:00
6
32
90-127
Thu 14-16
2
33
90-134
Thu 14-16
1
41
90-235
Thu 14-16
5
42
90-235
Thu 14-16
Insert/Delete/Update [record]
Select
• The most common operation in SQL is the query, which is performed
with the declarative SELECT statement.
• SELECT retrieves data from one or more tables, or expressions.
• Standard SELECT statements have no persistent effects on the
database.
• Some non-standard implementations of SELECT can have persistent
effects, such as the SELECT INTO syntax that exists in some databases.
Select
• A query includes a list of columns to be included in the final result immediately
following the SELECT keyword.
• An asterisk ("*") can be used to specify that the query should return all columns of
the queried tables.
• SELECT is the most complex statement in SQL, with optional keywords and clauses
that include:
• FROM:
•
The FROM clause which indicates the table(s) from which data is to be retrieved.
• WHERE:
•
•
The WHERE clause includes a comparison predicate, which restricts the rows returned by the query.
The WHERE clause eliminates all rows from the result set for which the comparison predicate does not
evaluate to True.
• GROUP BY:
•
•
The GROUP BY clause is used to project rows having common values into a smaller set of rows.
GROUP BY is often used in conjunction with SQL aggregation functions or to eliminate duplicate rows
from a result set. The WHERE clause is applied before the GROUP BY clause.
• HAVING:
•
•
The HAVING clause includes a predicate used to filter rows resulting from the GROUP BY clause.
Because it acts on the results of the GROUP BY clause, aggregation functions can be used in the
HAVING clause predicate.
• ORDER BY:
•
•
The ORDER BY clause identifies which columns are used to sort the resulting data, and in which
direction they should be sorted (options are: ASC ascending or DESC descending).
Without an ORDER BY clause, the order of rows returned by an SQL query is undefined.
SQL Aggregation Functions
• SQL aggregate functions return a single value, calculated from values
in a column.
•
•
•
•
•
•
•
AVG() - Returns the average value
COUNT() - Returns the number of rows
FIRST() - Returns the first value
LAST() - Returns the last value
MAX() - Returns the largest value
MIN() - Returns the smallest value
SUM() - Returns the sum
FROM
• * returns all columns.
• You may specifically choose columns you want, and their order
WHERE
•
•
•
•
•
•
•
AND
OR
IS
IN
BETWEEN
LIKE
http://en.wikipedia.org/wiki/Where_%28SQL%29
HAVING/GROUP BY
HAVING/GROUP BY
ORDER BY
• If Unspecific, default order is undefined.
JOIN
• The JOIN keyword is used in an SQL statement to query data from two or
more tables, based on a relationship between certain columns in these
tables.
• Tables in a database are often related to each other with keys.
• Different SQL JOINs:
• INNER JOIN/JOIN: Return rows when there is at least one match in both tables.
• LEFT JOIN: Return all rows from the left table, even if there are no matches in the
right table.
• RIGHT JOIN: Return all rows from the right table, even if there are no matches in the
left table.
• OUTER JOIN/FULL JOIN: Return rows when there is a match in one of the tables.
Examples: http://www.w3schools.com/sql/sql_join.asp
JOIN/INNER JOIN Example
TA Group
Location Time
ID
Name
OfficeHours
3
11
90-234
Sun 14-16
3
12
34-205
Sun 18-20
4
13
90-125
Thu 14-16
6
21
28-145
Thu 14-16
1
22
28-107
Thu 14-16
1
2
3
4
5
6
Majeed
Boaz
Matan
Dan
Hagit
Tom
Wed 10:00-12:00
Tue 16:00-18:00
Tue 14:00-16:00
Tue 12:00-14:00
Tue 16:00-18:00
Tue 19:00-21:00
2
23
72-213
Thu 14-16
5
31
90-145
Thu 14-16
6
32
90-127
Thu 14-16
2
33
90-134
Thu 14-16
1
41
90-235
Thu 14-16
5
42
90-235
Thu 14-16
Query:
1 SELECT ta.name, ps.group, ps.Location , ps.Time
FROM TEACHING_ASSISTANTS AS ta
2 JOIN PRACTICAL_SESSIONS AS ps
ON ta.id = ps.ta
3
Python
• Python is an open-source, general purpose programming language that is
dynamic, strongly-typed, object-oriented, functional, and memory-managed.
• Python is an interpreted language, meaning that it uses an interpreter to
translate and run its' code.
• The interpreter reads and executes each line of code one at a time, just like a script, and
hence the term "scripting language"
• Python is dynamic which means that the types are checked only at runtime.
• Python is also strongly typed, just like Java. You can only execute operations that
are supported by the target type.
• Python file are called “modules”, and suffixed by “.py”
Simple Python Example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
# This is a comment.
# Import sys module
import sys
# Gather our code in a main() function
def main():
print 'Hello, ', sys.argv[1]
# Command line args are in sys.argv[1], sys.argv[2] ...
# sys.argv[0] is the script name itself and can be ignored
# Standard boilerplate to call the main() function to begin
# the program.
if __name__ == '__main__':
main()
Python Built in functions
• The Python interpreter has a number of functions built into it that are
always available.
• Complete list of these functions can be found here:
• https://docs.python.org/2/library/functions.html
• Examples:
• len(s) - Return the length (the number of items) of an object.
• str(o) - Return a string containing a printable representation of an object.
• int(x) - Return an integer object constructed from a number or string x.
Python String
• String are enclosed by either double or single quotes.
• String are immutable.
• Method examples:
•
•
•
•
•
s.lower() – returns the lowercase version of a string
s.upper() – returns the upper version of a string
s.strip() – returns a string with the whitespaces removed from the start to the end
s.replace('old', 'new') – returns a new string where all occurrences of 'old' have been replaced by 'new'
s.split('delim') – returns a list of substrings separated by the given delimter.
For example: 'aaa,bbb,ccc'.split(',') gives ['aaa', 'bbb', 'ccc'].
1 s = 'hello'
print s[1] # e
2 print len(s) # 5
print s + ' there' # hello there
3 pi = 3.14
text = 'The value of pi is ' + str(pi)
4 print text # The value of pi is 3.14
https://docs.python.org/2/library/string.html
5
Lists
•
•
Core data structure in python.
• Lists can be used as: Stacks, Queues, Arrays, Sets (Unordered List)
• Lists can contain Lists and used as: Matrices
Method examples:
• list.append(elem) - adds a single element to the end of the list. Does not return the new list, just modifies
the original
• list.insert(index, elem) - inserts the element at the given index, shifting elements to the right
• list.remove(elem) - searches for the first instance of the given element and removes it
• list.extend(list2) - adds the elements in list2 to the end of the list. You can also use + or += on a list for the
same effect
• list.sort() - sorts the list in place (does not return it)
• list.pop(index) - removes the returns the element at the given index
1 cars = ['Ford', 'Honda', 551]
print cars[0] # Ford
2 print cars[1] # Honda
print cars[2] # 551
3 print len(cars) # 3
https://docs.python.org/2/tutorial/datastructures.html
4
Tuples
• A tuple is a sequence of immutable Python objects.
• Tuples are sequences, just like lists.
• Tuples cannot be changed unlike lists.
• Tuples use parentheses, whereas lists use square brackets.
• Tuples are immutable:
• You can't add elements.
• Tuples have no append or extend method.
• You can't remove elements.
• Tuples have no remove or pop method.
• You can find elements in a tuple, since this doesn’t change the tuple.
• You can also use the in operator to check if an element exists in the tuple.
• Why tuples? Tuples are faster than lists.
How much faster? x2-x3 times faster.
$ python3.1 -mtimeit -s'x,y,z=1,2,3' '[x,y,z]'
1000000 loops, best of 3: 0.379 usec per loop
$ python3.1 -mtimeit '[1,2,3]'
1000000 loops, best of 3: 0.413 usec per loop
$ python3.1 -mtimeit
10000000 loops, best
$ python3.1 -mtimeit
10000000 loops, best
-s'x,y,z=1,2,3' '(x,y,z)'
of 3: 0.174 usec per loop
'(1,2,3)'
of 3: 0.0602 usec per loop
$ python2.6 -mtimeit -s'x,y,z=1,2,3' '[x,y,z]'
1000000 loops, best of 3: 0.352 usec per loop
$ python2.6 -mtimeit '[1,2,3]'
1000000 loops, best of 3: 0.358 usec per loop
$ python2.6 -mtimeit
10000000 loops, best
$ python2.6 -mtimeit
10000000 loops, best
-s'x,y,z=1,2,3' '(x,y,z)'
of 3: 0.157 usec per loop
'(1,2,3)'
of 3: 0.0527 usec per loop
Dictionaries
• Dictionaries are the same as HashMaps.
• Each Dictionary entry has Key:Value:
dict = {'Name': 'Zara', 'Age': 7, 'Class': 'First'}
• Keys are unique.
• Complete information here:
https://www.tutorialspoint.com/python/python_dictionary.htm
Slicing
• Slicing of an object, ‘obj’,is done by using a range [a:b]
• ‘a’ denotes the beginning of the range.
• If ‘a’ is omitted, then a=0
• ‘b’ denotes the end of the range(excluding)
• If ‘b’ is omitted, then b=len(obj)
• Slicing can be done to objects containing ‘getitem’ attribute, i.e. strings and lists.
1 s = “Hello"
str1 = s[1:4] # str1 = “ell"
2 str2 = s[1:] # str2 = “ello"
str3 = s[:] # str3 = “Hello"
3 str4 = s[1:100] # str4 = “Hello" - an index that is too big is truncated down to the string length
4
5
If Statement / in
1 if x >= 0.5 and y >= 0.5: # if syntax
print 'All above half'
2 elif x >= 0.5 or y >= 0.5: #else if syntax
print 'One of them is above half.'
3 else: #else syntax
print 'None of them is above half.'
4
5
1 cars = ['Ford', 'Honda', 551]
6 if 'Ford' in cars:
2
print 'Yay!'
3
Loops
• Python does not support “for i=0;i<....;....” loops.
• ‘in’ loops:
1 for car in cars:
2
print car
• ‘while’ loops:
1 i=0
while i < len(cars):
2
print cars[i]
i += 1
3
4
Handling user input
1 import sys
2
3 def main():
4
while True:
5
print("Input: ")
6
inputline = sys.stdin.readline() # read a line from the user
7
inputline = inputline.strip('\n') # strip the '\n' from the line of the user
8
if inputline == 'exit':
9
return
10
else:
11
outline = inputline + ".." + inputline[len(inputline)-2:] + ".." + inputline[len(inputline)-2:]
12
print("Got echo: %s" % (outline))
13
14 if __name__ == '__main__':
15
main()
File read/write
1 import sys
2
3 def main(args):
4
inputfilename = args[1]
5
if (not os.path.isfile(filename)): #check if file exists
6
return
7
with open(inputfilename) as inputfile: #try-with-resources
8
for line in inputfile:
9
print(line)
10
11 if __name__ == '__main__':
12
main(sys.argv)
https://docs.python.org/2/tutorial/inputoutput.html
SQLite
• The module used to connect to an SQL database to apply commands and queries.
• To execute commands and run queries:
• We connect to the database.
• We must ask for a Cursor object.
1 import sqlite3
dbcon = sqlite3.connect('example.db')
2 with dbcon:
cursor = dbcon.cursor()
• To get the results3we use the following commands:
• cursor.fetchone(): fetches one of the results of the query in the form of a tuple, or None if none is
4
available
• cursor.fetchall(): fetches all of the results of the query, returning a list of tuples
Executing Commands and Queries
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
import sqlite3
import os
databaseexisted = os.path.isfile('example.db')
dbcon = sqlite3.connect('example.db')
with dbcon:
cursor = dbcon.cursor()
if not databaseexisted: # First time creating the database. Create the tables
cursor.execute("CREATE TABLE Students(ID INTEGER PRIMARY KEY, NAME TEXT NOT NULL)")
# create table students
cursor.execute("INSERT INTO Students VALUES(?,?)", (1, 'Morad',))
# add entry 'id = 1, name = Morad' into the table.
cursor.execute("INSERT INTO Students VALUES(?,?)", (2, 'Harry Potter',))
# let's get all students and print their entries
cursor.execute("SELECT * FROM Students")
studentslist = cursor.fetchall()
print("All students as list:")
print(studentslist)
print("All students one by one:")
for student in studentslist:
print("Student name: " + str(student)) # let's get the name of the student of id 1
cursor.execute("SELECT NAME FROM Students WHERE ID=(?)", (1,))
studentwithid1 = cursor.fetchone()
print("Student with id 1: " + str(studentwithid1))