Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Let’s talk about us… in a couple of sentences • Your programming experience: • How much • Which programming languages • Why python? Day 1 Introduction • Python is a renowned programming language for a few reasons: – Easy to pick up – The same executable can run anywhere – Widely used in Linux environment – Widely used in Web and Internet Development Let’s get started… Installation and settings For this course you need to have: • Python 3.5 installed • PyCharm IDE NOTE: For Windows users, if you use Cygwin or another “special” shell you may experience issues Interpreted vs compiled • Nowadays many languages have different implementation as interpreted or compiled • Also, these implementation only loosely fall into categories any more Basics Interpreted vs compiled (and bytecode) • Bytecode, also known as portable code, represents a sort step in the middle and it’s for example used by Java virtual machines and by the main Python interpreter • This bytecode approach may also allow for a “just-in-time” language implementation like PyPy Basics Python 2 vs Python 3 • Currently there are two supported versions of Python APIs: Python 2.7.12 – its lifespan has been extended because of the large existing repository of scripts and libraries, but it will only be supported until 2020 Python 3.5.2 – current release, not backward compatible with 2.x. Version 3.6 is planned for the end of the year We will be using Python 3.5.2 for this training Basics Python Features pip • pip is a Python package installer. It integrates with PyPI, the Python Package Index (https://pypi.python.org/pypi) • pip is already installed if you have downloaded from python.org: Python 2 release 2.7.9 or higher or Python 3 release 3.4 or higher Basics Python Features pip & Homebrew • pip syntax is very simple: pip install modulename • However, if you are working on a Mac, you may have some issue with pip. In that case, a workaround is to use another package installer, Homebrew • Simple installation guide: http://docs.python-guide.org/en/latest/starting/install/osx/ Basics Instructions virtualenv • virtualenv is an utility that creates a virtual Python interpreter and isolates any packages installed for that interpreter from others on the system • This is useful when you have a number of projects, and don't want them to all share the same Python installation For example, you could have two project with conflicting requirements (like two versions of the same library) Basics Python Features virtualenv • virtualenv has one basic command: $ virtualenv ENV where ENV is the new directory where you want to place the new virtual environment The python in your new virtual environment is effectively isolated from the python that was used to create it • In a newly created ENV there will also be the activate and deactivate shell scripts These are convenience scripts that will modify the $PATH environment variable to ENV Basics Instructions Python Features The basics of Python scripting Course logbook • Before you close your shell use the following code to get the history of everything you have typed in your Python35 folder in the text file logbook.txt import readline my_file = open('logbook.txt','a') my_logbook = "" for i in range(readline.get_current_history_length()): my_logbook += readline.get_history_item(i + 1)+"\n" my_file.write(my_logbook) my_file.close() Basics Python’s command line • At first we will work with the command line • Type python in the shell (python -i in case of Cygwin) to start working • The prompt will change to >>> • The character \ allows you to continue a command in the next line (useful for readability) Basics Python Features Commenting in Python • There two type of commenting in Python – Use # to make an inline comment – Use ''' or (""")to start and end a multiline comment • We will be using this in class as you will be recovering your Python session later to have it as logbook Basics Instructions Variables declaration and use • A variable’s type doesn’t have to be declared in advance • The variable is defined when you assign it a value • ALL variables are objects in Python • The equivalent of NULL in Python is None. However it has some differences • Trying to use a variable that has not been defined previously will generate a blocking error Basics Whitespaces and indentation • Python doesn’t use punctuation to define blocks of code but indentation • Blocks of code are called ‘suites’ in Python environment • In order to create a standard between different script, the use of multiple of 4 whitespaces is suggested for each level of indentation Basics Python Features Tabs and implicit line joining • Tabs should be avoided unless working on a existing script which uses them • Python 3 disallows mixing tabs and spaces • Code using parenthesis, bracket and braces allows implicit line joining Check PEP 8 (Style Guide for Python code) for important suggestions about proper indentation and style: Basics https://www.python.org/dev/peps/pep-0008/ Python Features Python namespaces • In many cases a Python script may be split into multiple.py files (modules), or requires functions that are not built in, like the more complex mathematical formulas Basics Python Features Python namespaces • Is important to know that Python defines an enclosed environment of variables for each module file This environment is called ‘namespace’ In short it means that by default, variables and functions of one module are not visible to another module Basics Python Features Modules as libraries • However, when executing the statements of one module we may want to use the commands and variables defined in another module • In order to do that, we use the command import Basics Instructions Importing modules: math • An important module commonly used is math • While the basic arithmetical operations are immediately available, math allows using advanced mathematical operations like: math.ceil(x) and math.floor(x) math.trunc(x) and math.modf(x) math.pow(x,y) and math.sqrt(x) and many more Basics Instructions print function in Python 3 • An important difference between Python 2 and 3 is that the instruction print is not available any more and it’s replaced by the function print() • To avoid the automatic newline after the output, use print(str, end="") • This functions has a few interesting options like its interaction with the str.format() Basics function Instructions Python Features Prompt for keyboard input: input() • The command input(str) shows the string str on the screen and pause the virtual machine until the user submit keyboard input • The user input is redirected as stdin input and can be assigned to a variable • The command input(str) is equivalent to the raw_input(str) command of Python 2.x Basics Instructions Python Features Types Common Basic Types • The main built-in types are: numerics (int, float, complex) sequences (list, tuple, range, string) mappings (dict) classes and instances For more information about all built-in type and their supported operations: Basics https://docs.python.org/3/library/stdtypes.html Data Structures Python Features Truth values • Whenever a condition is tested any of the following is considered false: Object None Object False Zero of any numeric type. For example 0, 0.0, 0j Any empty sequence, or mapping (more on this later) Instances of user-defined classes appositely designed Python Features Conditional statements (with if example) • The if statement works like in many other languages, with a couple of syntax differences The if statement is immediately followed by the colon symbol : No parenthesis The optional elif and else are available The suite that will run if the if statement resolves as True is defined by indentation and must be, of course, immediately after the statement Instructions Booleans in Python • Python uses the following priority sequence for boolean operators, from the lowest to the highest: not and or • When in doubts, uses parenthesis For the full operator precedence list, check: https://docs.python.org/3/reference/expressions.html Python Features switch • There is no switch command in Python • Seriously! No joke Instructions Python Features Casting • In Python different type can’t be used interchangeably • Casting is for example required when you want to use "5" as 5 • Built-in casting functions are int(x), float(x) and str(x) Data Structures Python Features Mutable and immutable objects • One important distinction of objects in Python is whether they are mutable or immutable • Immutable object are just that: their content cannot be changed after they have been created In case of containers, that means their references cannot be changed, i.e. they cannot point to different object • Trying to assign a new value to a immutable object will result in the creation of a new one Python Features Immutable objects • Some immutable Objects: int, float, long, complex str bytes tuple frozen set Python Features Mutable objects • Some mutable Objects: byte array list dict objects created from user-defined classes set Python Features Python's data structures Strings • Strings in Python allow classic operations and can be used with built-in functions: Both single and double quotes can be used (in pairs) for strings Concatenation can be done using the + or the * operators str.lower(), str.upper(), len(str) str1.index(str2) The character \ allows escaping as in other programming languages The format() method allows strings to be partially dynamic Data Structures Sequences • There are three basic sequence types in Python: Tuples Lists Range • They are collections of elements • They are subclasses of the iterable class • They have many methods for to access or use their element Tuples • Tuple are immutable sequences of values • That means that once created they can’t be modified in any way • They can, however, be used to create new lists • Their elements are retrieved using the same notation as array Data Structures Python Features Tuples • Tuples may be constructed in a number of ways: – Using a pair of parentheses for the empty tuple: () – Using a trailing comma for a singleton tuple: a, or (a,) – Separating items with commas: a,b,c or (a,b,c) – Using the built-in function: tuple() or tuple(iterable) Data Structures Python Features Lists • Lists are mutable sequences: typically used to store collections of homogeneous items, but this is not forced in Python Their elements are retrieved using the same notation as array They are, in fact, equivalent to arrays of other programming languages like PHP Lists also accept a negative value as index. In this case, the index starts from the end, with -1 as the last element Data Structures Python Features Lists • Lists may be constructed in several ways: Using a pair of square brackets to denote the empty list: [] Using square brackets, separating items with commas: [a], [a, b, c] Using a list comprehension: [x for x in iterable] Using the type constructor: list() or list(iterable) Data Structures Python Features List slicing • Slicing is a way to create shallow copies of a list in Python x[start:end] - items start through end-1 x[start:] # items start through the rest of the array x[:end] # items from the beginning through end-1 x[:] # a copy of the whole array Instructions Data Structures Python Features List slicing • There is also the step value, which can be used with any of the above by using a second colon symbol: a[start:end:step] #same as above, by step • Be careful that Python ignores errors with indexes while slicing, returning only the elements in the slicing range This may lead to hard to find bugs Instructions Data Structures Python Features List operations • Being mutable, lists allow modification on their elements. Some useful methods and functions are: append(x) insert(index,x) remove(x) pop(index) sort() del list[index] – built-in function Instructions Data Structures Ranges • Ranges are simply an immutable sequence of numbers • They are constructed with the built-in functions range(stop) or range(start,stop[,step) • They are commonly used for looping a specific number of times in Python’s for loops Instructions Data Structures Python Features Common sequence operations • x in s - True if an item of s is equal to x, else • • • • False Concatenation with + and * len(s) - length of s s.index(x[, i[, j]]) - index of the first occurrence of x in s (at or after index i and before index j) s.count(x) - total number of occurrences of x in s Instructions Data Structures Mapping objects: Dictionaries • A mapping object maps hashable values to arbitrary objects. Mappings are mutable objects • There is currently only one standard mapping type in Python, the dictionary • In Python, all immutable object (numbers, strings and tuples) and objects from userdefined classes are hashable, while mutable objects (lists and dictionaries) are not Data Structures Python Features dict • The dict class allows creating dictionaries in Python • A dict object is a collection of pairs key/value Anything that is hashable can be used as key Anything can be the value • dict objects can be created by: Placing a comma-separated list of key: value pairs within braces Instructions Using the dict constructor Data Structures Python Features dict • dict basic constructor: dict(**kwarg) Using **kwargs in a constructor or function allows for a dictionary of any length to be passed as parameter • Dictionaries in Python works similarly to associative arrays Elements can be retrieved using the same annotations of associative array in PHP Instructions Data Structures dict • Common operations in dictionaries include: key in d - returns True if d has a key key, else False copy() - method that returns a shallow copy of the dictionary clear() - method that removes all items from the dictionary. get(key[, default]) - method that returns the value for key if key is in the dictionary, else default. If default is not given, it defaults to None Instructions Data Structures dict • Common operations in dictionaries include: keys(), values(), items() - these methods create dictionary view objects on the keys, values or both. These view objects are useful for iteration purposes and optimization update(other) - this method is useful to append a new dictionary to a another dictionary. If a key overlaps, the final result is taken from other Instructions Data Structures set • A set is an unordered collection with no duplicate elements • Basic uses include membership testing and eliminating duplicate entries • Set objects also support mathematical operations like union, intersection, difference, and symmetric difference NOTE: a set use the curly braces for representation, like dictionaries. But they have simple elements instead of key/value pairs Data Structures Python Features Lists in dictionaries in lists in dictionaries in lists... • Remember that tuples, list and dictionaries work with references • Also, that trying to change an immutable object will create a new one. That means that if this happens after it has been included in a list (or dictionary), the one in the list will stay unchanged • There is no limit in nesting list, tuples and dictionaries – Try to avoid too much nesting as it may make things unnecessary complex to manage Data Structures Python Features JSON and Python • The JSON library is available for Python and offers the operations to creates JSON dumps from Python sequences and dict objects and vice versa This encoding/decoding is very straightforward due to the perfect match between the lists and dictionaries of Python and the array and objects of the JSON notation Data Structures Python Features Day 2 Functions • In Python the keyword def introduces a function definition The parameters are always object references Remember that the behaviour of immutable objects may make it look otherwise The function works with a local set of variables Global variables can be referenced, but no value can be assigned unless they are named in a global statement Instructions Python Features Basic syntax • As usual, it is followed by the function name and a list of the accepted parameters • Also in this case indentation defines the suite runs when the function is called • It allows to return a user defined value In fact, if no return statement is written, an implicit return None is run at the end of the function’s suite Instructions Python Features Basic syntax • Functions in Python also allow for default values for one or more parameters Those parameters can be omitted when calling the function. Default value will be used instead Order is important, as they must be omitted from right to left Alternatively, the keyword argument form can be used to call the function instead of the positional form Instructions Python Features Functions in Python • While it may looks obvious, remember when writing a simple script that a function must be defined before the non-function code that use it That is true also if you write functions in the same file of the main .py file (the one that you run with the shell command python) Python Features Built-in functions • Useful built-in Python functions: globals() - returns a dictionary representing the current global symbol table locals() - updates and returns a dictionary representing the current local symbol table len(s) - returns the length (the number of items) of an object s. The argument may be a sequence or a collection print(*objects, sep=' ', end='\n', file=sys.stdout, flush=False) - prints objects to the text stream file, separated by sep and Instructions followed by end Python Features Built-in functions • Useful built-in Python functions: bool([x]) - returns one of the two boolean objects (True or False) base on x enumerate(iterable, start=0) - returns an enumerate object. iterable must be a sequence, an iterator, or some other object which supports iteration filter(function, iterable) - constructs an iterator from those elements of iterable for which function returns true Instructions Python Features Built-in functions • Useful built-in Python functions: min(iterable, *[, key, default]) or min(arg1, arg2, *args[, key, default]) returns the smallest item in an iterable object or the smallest of two or more arguments max(iterable, *[, key, default]) or max(arg1, arg2, *args[, key, default]) - returns the largest item sorted(iterable[, key][, reverse]) -returns a new sorted list from the items in iterable Instructions Python Features Built-in functions for iterable • Useful built-in Python functions: sum(iterable[, start]) - sums start and the items of an iterable from left to right and returns the total. start defaults to 0 next(iterator[, default]) - retrieves the next item from the iterator object by calling its __next__() method. If default is given, it is returned if the iterator is exhausted, otherwise an exception StopIteration is raised Instructions Python Features Loops for loop in Python • for loop in Python used the same logic of for in DOS and Unix shell scripting • It iterates through a sequence of items, running the suite for each one • If that sequence is a ordered sequence of numbers, the loop will behave much like for loop in other languages • To obtain this easily, range sequence objects are used Instructions Data Structures Python Features for with range() • The for statement is used to iterate over the elements of a sequence (such as a string, tuple or list) or other iterable object • As mentioned earlier, using a sequence object of the type range allows the for loop working similarly to other programming languages Instructions Python Features Deeper into for loops • A few extra option available for your for loops: continue - skip the rest of the suite and continues with the next iteration of the loop. Usually used in an if statement else - it is executed when the loop terminates through exhaustion of the list break - like in other programming languages, breaks out of the smallest enclosing for loop and skips the else clause. Usually used with an if statement Instructions Python Features while • while(condition) in Python works like in other programming languages as it repeatedly test the condition and runs the following suite if the condition is resolved to True • continue, break and else works in the same way as in the for loop Instructions for and while tips • The length of the list of elements for the for loop is evaluated at the beginning and an internal counter move through it That means that modifying the list may bring unexpected behaviour as length and counter position are not recalculated • There is no do...while in Python but it can be easily recreated by using a variable that starts as True object and checking the condition at the end of the suite Instructions Python Features Creating a python script __main__ • '__main__' is the name given to the scope in which top-level code executes • A module’s property __name__ is set equal to '__main__' when read from standard input, a script, or from an interactive prompt • This allows to run the code when a module us run as script or with python -m but not when is imported Basics Instructions Python Features Package system for Python • Packages are a way of structuring Python’s module namespace by using dotted module names • Python bases the package system on instances of the file __init__.py When importing a module, Python will look for the __init__.py file. If found, it will look also in the subdirectories and start building a hierarchy with all the directories containing the file The file __init__.py is often empty but it can also execute initialization code for the package or set the __all__ variable Instructions Python Features Importing modules: import all (*), import variables, import functions • There are a few ways to import modules to use their variable and functions in your scripts import x - this does not import the names of the functions or variables defined in x directly. Dot notation is needed in order to use them from x import y - this import the name of the function or variable y. However it doesn’t import the module name x from x import * - variation of the above, it imports all names except those beginning with an Instructions underscore Python Features Importing modules: circular references • One issue you must be aware of is circular references • When designing your script’s structure, you must avoid that two modules directly import from each other That creates a loop that will stop your script The error message that you get shows that but it may not be very clear Basics Python Features Solving circular references • Solving a circular reference may require to reorganise the structure of your modules Moving functions, classes and variables (names) between modules Creating new modules to contain names that are shared between modules • Don’t be afraid of changing the structure of your scripts, even if it may take some time Basics Python Features Running your script • The convention for Python scripts is to have the following code at the end of your module if __name__ == "__main__": # execute the following code only # if run as a script main() Basics Python Features File Input and Output Working with files • Working with text files is easy with Python through a few simple commands: open(filename[,mode]) – filename must be a string containing the file name, while mode is another string that defines how to use the file: 'r' when the file will only be read (default if mode is omitted) 'w' for only writing (an existing file with the same name will be erased), 'a' opens the file for appending 'r+' opens the file for both reading and writing Instructions Working with files • It is good practice to use the with...as: keywords when dealing with file objects The read or write operation are done in the suite following with...as: This has the advantage that the file is automatically properly closed after its suite finishes, even in case of error Instructions Reading files • To read the already opened files: read([size]) - reads the file’s contents. size is optional numeric argument. Be careful about the size of the file as if size is omitted the entire content will be read readline() - reads a single line from the file; a newline character (\n) is left at the end of the string, and is only omitted on the last line of the file if the file doesn’t end in a newline Instructions Writing on files • To write on the file: write(str) - put the contents of string into the stream to the file, returning the number of characters written flush() - commit the changes to the file close() - to close the file and free up any system resources taken up by the open file Instructions File access and buffering • When opening a file, you set the named parameter buffering to 1 to enable line buffering. That means that content is kept in the buffer only until a write with the characters '\n' is executed NOTE: While the file is open, trying to work on it with a word processor may lead to unexpected results Instructions Python Features File state boolean: closed • An stream object has the useful property closed which keeps track of the state of the connection If it’s correctly closed it will resolve to True, otherwise to False Instructions with ... as • When used, with...as allows to define some repeatedly use code as an object with initialisation code in the __enter__ method and closing code and safety measures in the __exit__ method This allows for example, to design an object to handle database connections as an alternative to try...finally (not part of this course) A return True in the __exit__ code will allow the program to continue even if an error occurs Instructions Using with ... as with custom object • You can use the with...as statement with objects created from user-defined classes if they implement the following methods: __enter__(self) will contain your initialization code that is run when the new object is created using the with...as statement __exit__(self, exc_type, exc_value, traceback) will contain the code that is run when all the code in the following suite is executed or the execution of the suite stops because of an error Instructions Day 3 Object Oriented Programming in Python Object Oriented Programming • Object-oriented programming (OOP) is a programming language model organized around objects rather than actions • The programmers define not only the data type of a data structure, the object, but also the types of operations (methods) that can be applied to the data structure • In addition, programmers can create relationships between one object and another. • For example, objects can inherit characteristics from other objects Basics Classes in Python • An important characteristic of Python is that nearly everything is an object. That includes classes! • Class objects support two kinds of operations: attribute references and instantiation Instructions Python Features Classes in Python • Class Attribute references use the standard dot notation syntax used for all attribute references in Python: obj.name • Attribute references are created for all data attributes and methods in the class definition • data attributes correspond to the “data members” of C++ but don’t need to be declared in the class definition • methods are functions that “belong to” an object Instructions Python Features Classes in Python • Class instantiation uses function notation i.e. nameofclass(parameter1, parameter2,...) • Attribute references are created in the namespace for all variables and methods of the class definition • A class object can be seen as a function that returns a new instance of the class (In Python functions are objects) Instructions Python Features __init__() • The instantiation operation (“calling” a class object) creates an empty object • To create objects with instances customized to a specific initial state, a class may define a special method named __init__() • Note that __init__() is not a constructor, but a method that is called immediately after the object has been created Instructions Data Structures Python Features __init__() • The __init__() method may have arguments for greater flexibility in the initialisation • For example: >>>class Cat ... def __init__(self, colour, breed): ... self.c = colour ... self.b = breed >>>myCat = Cat("White","Persian") Instructions Data Structures Python Features The super() Function • super() can be used to refer to parent classes without naming them explicitly, thus making the code more maintainable For example, a common use is to use it to easily call the __init__() method of the base class in the __init__() method of the derived class Instructions Data Structures Python Features The self Variable • The parameter self keeps a reference to the object itself that be used in the method’s code similarly to the this of C and Java • When calling a method of an object, the reference of the object itself is added as first parameter • Using the word self is just a name convention However, for readability, it is suggested to follow the convention Data Structures Python Features Inheritance • Inheritance is supported in Python with the following syntax: class DerivedClassName(BaseClassName): • In fact, the basic syntax for class calling is a shortcut for: class ClassName(Object): as by default they are derived class of the Object class Instructions Data Structures Python Features Inheritance • Execution of a derived class definition proceeds the same as for a base class When the class object is constructed, the base class is remembered This is used for resolving attribute references: if a requested attribute is not found in the class, the search proceeds to look in the base class. This rule is applied recursively if the base class itself is Instructions derived from some other class Data Structures Python Features Overriding • Derived classes may override methods of their base classes • Method references are resolved as follows: the corresponding class attribute is searched, descending down the chain of base classes if necessary, and the method reference is valid if this yields a function object Instructions Data Structures Python Features Overriding • When overriding methods, you should check it’s behaviour in a object of the derived class: In the derived class, if a method y from the base class was called by another method x of the same base class, the derived version of y will be called instead If you don’t consider this when writing a derived class of you may incur in unexpected behaviour • To call the base class method directly: BaseClassName.methodname(self, arguments) Instructions Data Structures Python Features OO Built-in Functions • A couple of useful built-in Python functions: isinstance(object, classinfo) - returns true if the object argument is an instance of the classinfo argument, or of a subclass thereof issubclass(class, classinfo) Return true if class is a subclass of classinfo. A class is considered a subclass of itself Instructions Python Features Database use in Python Python and Databases • For Python-database interaction we will use MySQL as example • Some knowledge about SQL queries is useful for this part although not necessary • In any case, in your scripting you may need to implement queries created by others and to be able to handle the results Basics Making Queries • The steps to interact with a MySQL database are always the same, and very easy with Python: Open the connection Make all the needed queries If no more queries are going to be done immediately, close the connection Basics Instructions Making Queries - Tips • Enclose the code connecting to the database in with...as statement The connection closing code goes into the __exit__() method Create a class for the connection as seen in this course for opening a file (the logic is the same) Alternatively, you can learn about the try...finally syntax for equivalent result Instructions Handling the Results • MySQL SELECT queries made with the pymysql module will return the result record in the following format: All the records will be returned as elements of a list Each record will be returned as a dictionary with the columns’ names as keys Basics Handling the Results • The values can be then accessed as normal for nested list and dictionaries NOTE: other connection modules for MySQL or other database engines may return the values in a different format Basics More on Python Replacements for the Switch Statement • As mentioned before, there is no “switch” statement in Python • The simplest ways to implemented it is with the use of if-elif-else or, in some cases, with dictionary mapping • Dictionary mapping makes use of the fact that dictionaries values may be references to functions Data Structures Python Features Ternary if • While looking into scripts code you may found the Python version of ternary if which has a very different from other programming languages: a if condition else b • This statement returns a if condition resolves to True and returns b if instead it resolves to False Instructions Python Features List Comprehension • List comprehensions provide a concise way to create lists • A list comprehension consists of brackets containing an expression followed by a for clause, then zero or more for or if clauses • The result will be a new list resulting from evaluating the expression in the context of the for and if clauses which follow it Instructions Data Structures Python Features *args, **kwargs • Two interesting tools for Python programmers are the parameters *args and **kwargs “args” and “kwargs” are just naming conventions, the asterisks symbols are what matters • They allow you to pass a variable number of arguments to a function With *args, they will be collected in the list args With **kwargs, they will be collected in the dictionary kwargs • This is transparent to the function user Instructions Data Structures Python Features Use of Function Objects • As mentioned, in Python function are stored as object too • One use of that can for example to use a different function on some data based on some previous conditions • Also they can passed around as easy as any other variable They can, for example, be passed as the return value of another function Python Features lambda • Another feature of Python is the availability of small anonymous functions, called lambda • Lambda functions can be used wherever function objects are required They are, however, syntactically restricted to a single expression • Once learned, lambda functions are quite useful from a readability point of view Instructions Python Features