Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Practical Session 12 SQL Basic Python SQLite Relational Databases • A relational database is a collection of data items organized as a set of formally described tables from which data can be accessed easily • A relational database is created using the relational model: • Data Definition Language • Used to build and destroy databases, create, and drop • Data Manipulation Language • Used to manipulate data in databases, insert, delete, and retrieve • We will use SQL data definition and query language. • Each database consists of a number of tables and each table has its own primary key. • Relational: • Because you may have relations between the different tables. Very good tutorial at: http://www.w3schools.com/sql/default.asp Table ID Name OfficeHours 1 2 3 4 5 6 Majeed Boaz Matan Dan Hagit Tom Wed 10:00-12:00 Tue 16:00-18:00 Tue 14:00-16:00 Tue 12:00-14:00 Tue 16:00-18:00 Tue 19:00-21:00 - Table name: TEACHING_ASSISTANTS - Column name: Id, Name, Office Hours - Column type: INT, VARCHAR(20), VARCHAR(20) - Each table has primary key, must be unique and non-null: example: id - Each line in table is called a record - You may have foreign key columns, which is a primary key in other table, to denote relationship between two different tables ANSI SQL Data Types • Character strings • • • • CHARACTER(n) or CHAR(n) — fixed-width n-character string, padded with spaces as needed CHARACTER VARYING(n) or VARCHAR(n) — variable-width string with a maximum size of n characters NATIONAL CHARACTER(n) or NCHAR(n) — fixed width string supporting an international character set NATIONAL CHARACTER VARYING(n) or NVARCHAR(n) — variable-width NCHAR string • Bit strings • • BIT(n) — an array of n bits BIT VARYING(n) — an array of up to n bits • Numbers • • • INTEGER and SMALLINT FLOAT, REAL and DOUBLE PRECISION NUMERIC(precision, scale) or DECIMAL(precision, scale) • Date and time • • • • • DATE — for date values (e.g., 2011-05-03) TIME — for time values (e.g., 15:51:36). The granularity of the time value is usually a tick (100 nanoseconds). TIME WITH TIME ZONE or TIMETZ — the same as TIME, but including details about the time zone in question. TIMESTAMP — This is a DATE and a TIME put together in one variable (e.g., 2011-05-03 15:51:36). TIMESTAMP WITH TIME ZONE or TIMESTAMPTZ — the same as TIMESTAMP, but including details about the time zone in question. Creating/Deleting a table Primary Key • A primary key is used to uniquely identify each row in a table. • A primary key can consist of one or more fields on a table. • When multiple fields are used as a primary key, they are called a composite key. • Primary key is inheritably unique. Foreign Key • A foreign key is a field(s) that point to the primary key of another table. • The purpose of the foreign key is to ensure referential integrity of the data. • Only values that are supposed to appear in the database are permitted. Code Example TEACHING_ASSISTANTS Column type characteristic ID int Primary Key Name VARCHAR(50) Office hours VARCHAR(9) PRACTICAL_SESSIONS Column type characteristic TA int Foreign Key Group int Primary Key Location VARCHAR(50) Time VARCHAR(9) Effect: PRACTICAL_SESSIONS table cannot contain information on a TA that is not in the Teaching Assistant table. SQL Commands PRACTICAL_SESSIONS TEACHING_ASSISTANTS TA Group Location Time 3 11 90-234 Sun 14-16 ID Name OfficeHours 3 12 34-205 Sun 18-20 4 13 90-125 Thu 14-16 6 21 28-145 Thu 14-16 1 22 28-107 Thu 14-16 2 23 72-213 Thu 14-16 5 31 90-145 Thu 14-16 1 2 3 4 5 6 Majeed Boaz Matan Dan Hagit Tom Wed 10:00-12:00 Tue 16:00-18:00 Tue 14:00-16:00 Tue 12:00-14:00 Tue 16:00-18:00 Tue 19:00-21:00 6 32 90-127 Thu 14-16 2 33 90-134 Thu 14-16 1 41 90-235 Thu 14-16 5 42 90-235 Thu 14-16 Insert/Delete/Update [record] Select • The most common operation in SQL is the query, which is performed with the declarative SELECT statement. • SELECT retrieves data from one or more tables, or expressions. • Standard SELECT statements have no persistent effects on the database. • Some non-standard implementations of SELECT can have persistent effects, such as the SELECT INTO syntax that exists in some databases. Select • A query includes a list of columns to be included in the final result immediately following the SELECT keyword. • An asterisk ("*") can be used to specify that the query should return all columns of the queried tables. • SELECT is the most complex statement in SQL, with optional keywords and clauses that include: • FROM: • The FROM clause which indicates the table(s) from which data is to be retrieved. • WHERE: • • The WHERE clause includes a comparison predicate, which restricts the rows returned by the query. The WHERE clause eliminates all rows from the result set for which the comparison predicate does not evaluate to True. • GROUP BY: • • The GROUP BY clause is used to project rows having common values into a smaller set of rows. GROUP BY is often used in conjunction with SQL aggregation functions or to eliminate duplicate rows from a result set. The WHERE clause is applied before the GROUP BY clause. • HAVING: • • The HAVING clause includes a predicate used to filter rows resulting from the GROUP BY clause. Because it acts on the results of the GROUP BY clause, aggregation functions can be used in the HAVING clause predicate. • ORDER BY: • • The ORDER BY clause identifies which columns are used to sort the resulting data, and in which direction they should be sorted (options are: ASC ascending or DESC descending). Without an ORDER BY clause, the order of rows returned by an SQL query is undefined. SQL Aggregation Functions • SQL aggregate functions return a single value, calculated from values in a column. • • • • • • • AVG() - Returns the average value COUNT() - Returns the number of rows FIRST() - Returns the first value LAST() - Returns the last value MAX() - Returns the largest value MIN() - Returns the smallest value SUM() - Returns the sum FROM • * returns all columns. • You may specifically choose columns you want, and their order WHERE • • • • • • • AND OR IS IN BETWEEN LIKE http://en.wikipedia.org/wiki/Where_%28SQL%29 HAVING/GROUP BY HAVING/GROUP BY ORDER BY • If Unspecific, default order is undefined. JOIN • The JOIN keyword is used in an SQL statement to query data from two or more tables, based on a relationship between certain columns in these tables. • Tables in a database are often related to each other with keys. • Different SQL JOINs: • INNER JOIN/JOIN: Return rows when there is at least one match in both tables. • LEFT JOIN: Return all rows from the left table, even if there are no matches in the right table. • RIGHT JOIN: Return all rows from the right table, even if there are no matches in the left table. • OUTER JOIN/FULL JOIN: Return rows when there is a match in one of the tables. Examples: http://www.w3schools.com/sql/sql_join.asp JOIN/INNER JOIN Example TA Group Location Time ID Name OfficeHours 3 11 90-234 Sun 14-16 3 12 34-205 Sun 18-20 4 13 90-125 Thu 14-16 6 21 28-145 Thu 14-16 1 22 28-107 Thu 14-16 1 2 3 4 5 6 Majeed Boaz Matan Dan Hagit Tom Wed 10:00-12:00 Tue 16:00-18:00 Tue 14:00-16:00 Tue 12:00-14:00 Tue 16:00-18:00 Tue 19:00-21:00 2 23 72-213 Thu 14-16 5 31 90-145 Thu 14-16 6 32 90-127 Thu 14-16 2 33 90-134 Thu 14-16 1 41 90-235 Thu 14-16 5 42 90-235 Thu 14-16 Query: 1 SELECT ta.name, ps.group, ps.Location , ps.Time FROM TEACHING_ASSISTANTS AS ta 2 JOIN PRACTICAL_SESSIONS AS ps ON ta.id = ps.ta 3 Python • Python is an open-source, general purpose programming language that is dynamic, strongly-typed, object-oriented, functional, and memory-managed. • Python is an interpreted language, meaning that it uses an interpreter to translate and run its' code. • The interpreter reads and executes each line of code one at a time, just like a script, and hence the term "scripting language" • Python is dynamic which means that the types are checked only at runtime. • Python is also strongly typed, just like Java. You can only execute operations that are supported by the target type. • Python file are called “modules”, and suffixed by “.py” Simple Python Example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 # This is a comment. # Import sys module import sys # Gather our code in a main() function def main(): print 'Hello, ', sys.argv[1] # Command line args are in sys.argv[1], sys.argv[2] ... # sys.argv[0] is the script name itself and can be ignored # Standard boilerplate to call the main() function to begin # the program. if __name__ == '__main__': main() Python Built in functions • The Python interpreter has a number of functions built into it that are always available. • Complete list of these functions can be found here: • https://docs.python.org/2/library/functions.html • Examples: • len(s) - Return the length (the number of items) of an object. • str(o) - Return a string containing a printable representation of an object. • int(x) - Return an integer object constructed from a number or string x. Python String • String are enclosed by either double or single quotes. • String are immutable. • Method examples: • • • • • s.lower() – returns the lowercase version of a string s.upper() – returns the upper version of a string s.strip() – returns a string with the whitespaces removed from the start to the end s.replace('old', 'new') – returns a new string where all occurrences of 'old' have been replaced by 'new' s.split('delim') – returns a list of substrings separated by the given delimter. For example: 'aaa,bbb,ccc'.split(',') gives ['aaa', 'bbb', 'ccc']. 1 s = 'hello' print s[1] # e 2 print len(s) # 5 print s + ' there' # hello there 3 pi = 3.14 text = 'The value of pi is ' + str(pi) 4 print text # The value of pi is 3.14 https://docs.python.org/2/library/string.html 5 Lists • • Core data structure in python. • Lists can be used as: Stacks, Queues, Arrays, Sets (Unordered List) • Lists can contain Lists and used as: Matrices Method examples: • list.append(elem) - adds a single element to the end of the list. Does not return the new list, just modifies the original • list.insert(index, elem) - inserts the element at the given index, shifting elements to the right • list.remove(elem) - searches for the first instance of the given element and removes it • list.extend(list2) - adds the elements in list2 to the end of the list. You can also use + or += on a list for the same effect • list.sort() - sorts the list in place (does not return it) • list.pop(index) - removes the returns the element at the given index 1 cars = ['Ford', 'Honda', 551] print cars[0] # Ford 2 print cars[1] # Honda print cars[2] # 551 3 print len(cars) # 3 https://docs.python.org/2/tutorial/datastructures.html 4 Tuples • A tuple is a sequence of immutable Python objects. • Tuples are sequences, just like lists. • Tuples cannot be changed unlike lists. • Tuples use parentheses, whereas lists use square brackets. • Tuples are immutable: • You can't add elements. • Tuples have no append or extend method. • You can't remove elements. • Tuples have no remove or pop method. • You can find elements in a tuple, since this doesn’t change the tuple. • You can also use the in operator to check if an element exists in the tuple. • Why tuples? Tuples are faster than lists. How much faster? x2-x3 times faster. $ python3.1 -mtimeit -s'x,y,z=1,2,3' '[x,y,z]' 1000000 loops, best of 3: 0.379 usec per loop $ python3.1 -mtimeit '[1,2,3]' 1000000 loops, best of 3: 0.413 usec per loop $ python3.1 -mtimeit 10000000 loops, best $ python3.1 -mtimeit 10000000 loops, best -s'x,y,z=1,2,3' '(x,y,z)' of 3: 0.174 usec per loop '(1,2,3)' of 3: 0.0602 usec per loop $ python2.6 -mtimeit -s'x,y,z=1,2,3' '[x,y,z]' 1000000 loops, best of 3: 0.352 usec per loop $ python2.6 -mtimeit '[1,2,3]' 1000000 loops, best of 3: 0.358 usec per loop $ python2.6 -mtimeit 10000000 loops, best $ python2.6 -mtimeit 10000000 loops, best -s'x,y,z=1,2,3' '(x,y,z)' of 3: 0.157 usec per loop '(1,2,3)' of 3: 0.0527 usec per loop Dictionaries • Dictionaries are the same as HashMaps. • Each Dictionary entry has Key:Value: dict = {'Name': 'Zara', 'Age': 7, 'Class': 'First'} • Keys are unique. • Complete information here: https://www.tutorialspoint.com/python/python_dictionary.htm Slicing • Slicing of an object, ‘obj’,is done by using a range [a:b] • ‘a’ denotes the beginning of the range. • If ‘a’ is omitted, then a=0 • ‘b’ denotes the end of the range(excluding) • If ‘b’ is omitted, then b=len(obj) • Slicing can be done to objects containing ‘getitem’ attribute, i.e. strings and lists. 1 s = “Hello" str1 = s[1:4] # str1 = “ell" 2 str2 = s[1:] # str2 = “ello" str3 = s[:] # str3 = “Hello" 3 str4 = s[1:100] # str4 = “Hello" - an index that is too big is truncated down to the string length 4 5 If Statement / in 1 if x >= 0.5 and y >= 0.5: # if syntax print 'All above half' 2 elif x >= 0.5 or y >= 0.5: #else if syntax print 'One of them is above half.' 3 else: #else syntax print 'None of them is above half.' 4 5 1 cars = ['Ford', 'Honda', 551] 6 if 'Ford' in cars: 2 print 'Yay!' 3 Loops • Python does not support “for i=0;i<....;....” loops. • ‘in’ loops: 1 for car in cars: 2 print car • ‘while’ loops: 1 i=0 while i < len(cars): 2 print cars[i] i += 1 3 4 Handling user input 1 import sys 2 3 def main(): 4 while True: 5 print("Input: ") 6 inputline = sys.stdin.readline() # read a line from the user 7 inputline = inputline.strip('\n') # strip the '\n' from the line of the user 8 if inputline == 'exit': 9 return 10 else: 11 outline = inputline + ".." + inputline[len(inputline)-2:] + ".." + inputline[len(inputline)-2:] 12 print("Got echo: %s" % (outline)) 13 14 if __name__ == '__main__': 15 main() File read/write 1 import sys 2 3 def main(args): 4 inputfilename = args[1] 5 if (not os.path.isfile(filename)): #check if file exists 6 return 7 with open(inputfilename) as inputfile: #try-with-resources 8 for line in inputfile: 9 print(line) 10 11 if __name__ == '__main__': 12 main(sys.argv) https://docs.python.org/2/tutorial/inputoutput.html SQLite • The module used to connect to an SQL database to apply commands and queries. • To execute commands and run queries: • We connect to the database. • We must ask for a Cursor object. 1 import sqlite3 dbcon = sqlite3.connect('example.db') 2 with dbcon: cursor = dbcon.cursor() • To get the results3we use the following commands: • cursor.fetchone(): fetches one of the results of the query in the form of a tuple, or None if none is 4 available • cursor.fetchall(): fetches all of the results of the query, returning a list of tuples Executing Commands and Queries 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 import sqlite3 import os databaseexisted = os.path.isfile('example.db') dbcon = sqlite3.connect('example.db') with dbcon: cursor = dbcon.cursor() if not databaseexisted: # First time creating the database. Create the tables cursor.execute("CREATE TABLE Students(ID INTEGER PRIMARY KEY, NAME TEXT NOT NULL)") # create table students cursor.execute("INSERT INTO Students VALUES(?,?)", (1, 'Morad',)) # add entry 'id = 1, name = Morad' into the table. cursor.execute("INSERT INTO Students VALUES(?,?)", (2, 'Harry Potter',)) # let's get all students and print their entries cursor.execute("SELECT * FROM Students") studentslist = cursor.fetchall() print("All students as list:") print(studentslist) print("All students one by one:") for student in studentslist: print("Student name: " + str(student)) # let's get the name of the student of id 1 cursor.execute("SELECT NAME FROM Students WHERE ID=(?)", (1,)) studentwithid1 = cursor.fetchone() print("Student with id 1: " + str(studentwithid1))