* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Python Crash Course – Programming tools
Abstraction (computer science) wikipedia , lookup
Logic programming wikipedia , lookup
Comment (computer programming) wikipedia , lookup
Programming language wikipedia , lookup
Falcon (programming language) wikipedia , lookup
C Sharp (programming language) wikipedia , lookup
Functional programming wikipedia , lookup
Software bug wikipedia , lookup
Control flow wikipedia , lookup
Corecursion wikipedia , lookup
Reactive programming wikipedia , lookup
Team Foundation Server wikipedia , lookup
Object-oriented programming wikipedia , lookup
Python syntax and semantics wikipedia , lookup
Python Crash Course – Programming tools Python Crash Course – Programming tools Give me six hours to chop down a tree and I will spend the first four sharpening the axe. Abraham Lincoln Steffen Brinkmann Max-Planck-Institut für Astronomie, Heidelberg IMPRESS, 2016 1 / 60 Python Crash Course – Programming tools The ingredients I A working Python installation I Internet connection I Passion for Python If anything of the above is missing, please say so now! 2 / 60 Python Crash Course – Programming tools Outline The actual problem Who you are Who you can be Defensive programming Some thoughts Version control Introduction Hello git Documentation Documentation levels docstrings in Python doxygen Testing Testing strategies doctests in Python Unit tests in Python 3 / 60 Python Crash Course – Programming tools The actual problem Who you are Outline The actual problem Who you are Who you can be Defensive programming Some thoughts Version control Introduction Hello git Documentation Documentation levels docstrings in Python doxygen Testing Testing strategies doctests in Python Unit tests in Python 4 / 60 Python Crash Course – Programming tools The actual problem Who you are 5 / 60 Python Crash Course – Programming tools The actual problem Who you are 6 / 60 Python Crash Course – Programming tools The actual problem Who you can be Outline The actual problem Who you are Who you can be Defensive programming Some thoughts Version control Introduction Hello git Documentation Documentation levels docstrings in Python doxygen Testing Testing strategies doctests in Python Unit tests in Python 7 / 60 Python Crash Course – Programming tools The actual problem Who you can be The problem I A computer scientist is the analyst, designer, programmer, maintainer, tester, support and customer. I You cannot do all of this and do science and publish. I But you can do much more than you do now with the help of tools. All of the problems in the comics are communication problems. Either with your (future) self or with your collaborators. 8 / 60 Python Crash Course – Programming tools The actual problem Who you can be What you do now The (modified) Joel test Do you manage your source code, version branches and release cycles using a version control system? Examples for version control systems are cvs, svn, git and mercurium. Can you make a build in one step? A build means getting from source code to a distributable package or executable. One step means one command on the command line or one click of a button. http://www.joelonsoftware.com/articles/fog0000000043.html 9 / 60 Python Crash Course – Programming tools The actual problem Who you can be What you do now The (modified) Joel test Do you use a continuous integration and automated testing platform? Examples for CI and automated testing software are Jenkins and Hudson Do you make daily builds? Imagine a colleague asks for your code as you arrive at the office. Could you pass him your code within a minute and be sure she gets it to work (and it works as expected) without further interaction from your side. http://www.joelonsoftware.com/articles/fog0000000043.html 10 / 60 Python Crash Course – Programming tools The actual problem Who you can be What you do now The (modified) Joel test Do you have a bug database? Do you document bugs and feature requests at some well accessible place? Do you fix bugs before writing new code? Is fixing bugs the first priority? http://www.joelonsoftware.com/articles/fog0000000043.html 11 / 60 Python Crash Course – Programming tools The actual problem Who you can be What you do now The (modified) Joel test Are input and output formats standardized? Do you use common file formats, such as csv, fits, png or do you have a written document that specifies the input and output comprehensively? Do you provide comprehensive source and usage documentation? Is every function, method, class module and file of your code documented in-code, e.g. with robodoc or doxygen? Do you provide README files and/or pdf documents as a manual on how to install and use your program? http://www.joelonsoftware.com/articles/fog0000000043.html 12 / 60 Python Crash Course – Programming tools The actual problem Who you can be What you do now The (modified) Joel test How many questions did you answer with Yes? 0-3 Look no further. This course is for you. over 3 You’re on the right track. There is always room for enhancements. http://www.joelonsoftware.com/articles/fog0000000043.html 13 / 60 Python Crash Course – Programming tools The actual problem Who you can be What you want to do from now on I Defensive programming I Use a version control system I Document your code I Write tests I Automate the build process I Automate the tests 14 / 60 Python Crash Course – Programming tools The actual problem Who you can be how you want to do it from now on I Defensive programming → programming skills I Use a version control system → git I Document your code → doxygen I Write tests → doctests, unittest I Automate the build process → bash, configure, make I Automate the tests → Jenkins 15 / 60 Python Crash Course – Programming tools Defensive programming Some thoughts Outline The actual problem Who you are Who you can be Defensive programming Some thoughts Version control Introduction Hello git Documentation Documentation levels docstrings in Python doxygen Testing Testing strategies doctests in Python Unit tests in Python 16 / 60 Python Crash Course – Programming tools Defensive programming Some thoughts Defensive programming I Write comprehensible code I Check input and make sure, the program reacts in a predictable manner I Throw and catch exceptions I Fix bugs before you implement the next feature 17 / 60 Python Crash Course – Programming tools Defensive programming Some thoughts Comprehensive code I Beautiful is better than ugly. I Explicit is better than implicit. I Simple is better than complex. I Complex is better than complicated. I Flat is better than nested. I https://www.python.org/dev/peps/pep-0020/ 18 / 60 Python Crash Course – Programming tools Defensive programming Some thoughts Check input and intermediate results I Check for type of function arguments I Expect user input to be bogus I Check for empty strings, negative values etc. I Use assert <expression>[, <description>] statements. It will throw an AssertionError if <expression> is False I In case of an error, print error message to stderr and return or exit with a well defined and documented return value I Include these cases in your testing routines 19 / 60 Python Crash Course – Programming tools Defensive programming Some thoughts Throw and catch expressions def checkedDivide (a , b ): if b == 0: raise Exception ( " Do not divide by zero !!! " ) return a / b def t e s t C h e c k e d D i v i d e (a , b ): try : result = checkedDivide ( a , b ) except Exception as e : sys . stderr . write ( " Division Error : % s \ n " % str ( e )) return math . nan else : sys . stderr . write ( " successful division \ n " ) finally : sys . stderr . write ( " checked division performed .\ n " ) return result https://docs.python.org/3/tutorial/errors.html 20 / 60 Python Crash Course – Programming tools Defensive programming Some thoughts Throw and catch expressions def checkedDivide (a , b ): if b == 0: raise Exception ( " Do not divide by zero !!! " ) return a / b def t e s t C h e c k e d D i v i d e (a , b ): try : result = checkedDivide ( a , b ) except Exception as e : logging . error ( " Division Error : % s \ n " % str ( e )) return math . nan else : logging . info ( " successful division \ n " ) finally : logging . debug ( " checked division performed .\ n " ) return result https://docs.python.org/3/howto/logging.html 21 / 60 Python Crash Course – Programming tools Defensive programming Some thoughts Fix bugs Find bugs : in-code checks, tests, debugging Document bugs : Mention known bugs in a comment line. A documented bug is better than an undocumented one Evaluate bugs : Does it effect one feature? Many features? The whole program? Will the program crash? Will the results be obviously wrong? Will the results be slightly wrong? Will the user notice? Fix bugs : Do not let the bugs propagate into future versions. Fix the bug in every version (branch). 22 / 60 Python Crash Course – Programming tools Version control Introduction Outline The actual problem Who you are Who you can be Defensive programming Some thoughts Version control Introduction Hello git Documentation Documentation levels docstrings in Python doxygen Testing Testing strategies doctests in Python Unit tests in Python 23 / 60 Python Crash Course – Programming tools Version control Introduction Introduction I Version (revision, source) control systems are tools to help you manage the ever changing source you are working on. I A version is identified by a unique number or letter code. I A commit is a change to the source (i.e. in one or more files) added to the current branch, making it the head revision of that branch. I Branches are parallel development lines emerging from the same commit as the result of forking. I Branches can (and mostly should) be merged at some point. 24 / 60 Python Crash Course – Programming tools Version control Introduction Why and why not Advantages I Easy recovery of former (working) versions. I Documentation of development progress in the commit messages. I Branching enables you to work on different features at the same time. I Branching enables different programmers to work in an ordered fashion to work on the same program I Separate bug-fixes from adding new features and development of new versions. 25 / 60 Python Crash Course – Programming tools Version control Introduction Why and why not Cost I Learn how to use a VCS: this course, once in a while half an hour. I Set up the VCS (git): this course, once a few minutes I Set up a repository: once every time you start a new project 1 minute I Commit your changes: several times a day 30 seconds. 26 / 60 Python Crash Course – Programming tools Version control Hello git Outline The actual problem Who you are Who you can be Defensive programming Some thoughts Version control Introduction Hello git Documentation Documentation levels docstrings in Python doxygen Testing Testing strategies doctests in Python Unit tests in Python 27 / 60 Python Crash Course – Programming tools Version control Hello git Why git I Used in many large projects: Linux kernel, Qt, Gnome, eclipse, KDE, X11, Perl, Rails,. . . I Decentralised: Every local copy is a complete repository. No need for a central server. I Very fast I Powerful branching and merging 28 / 60 Python Crash Course – Programming tools Version control Hello git Hello git One time set up of git I Install git I Documentation: http://www.git-scm.com/documentation I Set up you user name and email address: $ git config -- global user . name " YOUR NAME " $ git config -- global user . email " NAME@mpia . de " 29 / 60 Python Crash Course – Programming tools Version control Hello git Hello git Let’s start a new project We started an new project: # !/ bin / env python from __future__ import print_function print ( " hello git " ) Let’s set up a repository for it. 30 / 60 Python Crash Course – Programming tools Version control Hello git Hello git In the new project’s directory Let’s set up a repository for the new project. $ ls -A hello_git . py $ git init Initialized empty Git repository in /.../. git / $ ls -A . git hello_git . py 31 / 60 Python Crash Course – Programming tools Version control Hello git Hello git In the new project’s directory Let’s add the source file to the new repository. $ git status On branch master Initial commit Untracked files : ( use " git add < file >... " to include in what will be committed ) hello_git . py nothing added to commit but untracked files present ( use " git add " to track ) $ git add hello_git . py $ git status On branch master Initial commit Changes to be committed : ( use " git rm -- cached < file >... " to unstage ) new file : hello_git . py 32 / 60 Python Crash Course – Programming tools Version control Hello git Hello git In the new project’s directory Let’s commit the source file to the new repository. $ git commit -m " initial commit " [ master ( root - commit ) bb32989 ] initial commit 1 file changed , 5 insertions (+) create mode 100644 hello_git . py $ git status On branch master nothing to commit , working directory clean $ git log commit b b 3 2 9 8 9 2 f 1 e 7 5 e 8 e 2 3 2 2 e 6 6 2 1 4 5 a d e f 7 8 4 3 6 4 6 d 7 Author : Steffen Brinkmann < brinkmann@mpia . de > Date : Tue Feb 23 20:49:31 2016 +0100 initial commit 33 / 60 Python Crash Course – Programming tools Version control Hello git Hello git git background 34 / 60 Python Crash Course – Programming tools Version control Hello git Hello git In the new project’s directory Let’s change a source file and add a new one. $ emacs hello_git . py $ cat > README . md hello_git ========= A very simple program $ git add hello_git . py README $ git commit -m ’ added README and in - code documentation ’ [ master 9 ed1ab4 ] added README and in - code documentation 2 files changed , 5 insertions (+) create mode 100644 README 35 / 60 Python Crash Course – Programming tools Version control Hello git Hello git going remote I I I I Setting up a remote repository serves for backing up and sharing. Many free services available: gitlab, github, bitbucket, sourceforge,. . . Create the repository online local set up (once): $ git remote add origin git@gitlab . com : szs / test . git $ git push -u origin master Counting objects : 7 , done . Delta compression using up to 2 threads . Compressing objects : 100% (5/5) , done . Writing objects : 100% (7/7) , 689 bytes | 0 bytes /s , done . Total 7 ( delta 0) , reused 0 ( delta 0) To git@gitlab . com : szs / test . git * [ new branch ] master -> master Branch master set up to track remote branch master from origin . https://en.wikipedia.org/wiki/Comparison of source code hosting facilities#Available version control systems 36 / 60 Python Crash Course – Programming tools Version control Hello git Hello git Further workflow $ git pull Already up - to - date . $ emacs hello_git . py $ git add hello_git . py $ git commit -m ’ some more output ’ [ master edcaeef ] some more output 1 file changed , 1 insertion (+) $ git push Counting objects : 3 , done . Delta compression using up to 2 threads . Compressing objects : 100% (3/3) , done . Writing objects : 100% (3/3) , 327 bytes | 0 bytes /s , done . Total 3 ( delta 1) , reused 0 ( delta 0) To git@gitlab . com : szs / test . git 9 ed1ab4 .. edcaeef master -> master 37 / 60 Python Crash Course – Programming tools Version control Hello git git with branches Next steps: branches I Branches enable you to work on several fixes/features at the same time I Branches enable you to work with other programmers I Branches should be merged often I Commits should only be made to feature or fix branches I Merge these branches to a central development branch I Test the development branch I Merge the development branch to the master branch I The master branch should only contain versions which are distributable 38 / 60 Python Crash Course – Programming tools Version control Hello git git with branches Next steps: branches http://nvie.com/posts/a-successful-git-branching-model/ 39 / 60 Python Crash Course – Programming tools Version control Hello git git with branches Exercises Using $ git help branch $ git help checkout $ git help merge do the following: I Create a development branch called ”develop” I Create a feature branch called ”features” I Change the source and/or add files in two or more commits I Merge this branch back to ”development” I Merge the ”development” branch back to ”master” 40 / 60 Python Crash Course – Programming tools Documentation Documentation levels Outline The actual problem Who you are Who you can be Defensive programming Some thoughts Version control Introduction Hello git Documentation Documentation levels docstrings in Python doxygen Testing Testing strategies doctests in Python Unit tests in Python 41 / 60 Python Crash Course – Programming tools Documentation Documentation levels Documentation levels I README, CHANGES, INSTALL, LICENSE files I in-code (doxygen, robodoc, docstrings) I technical documentation I usage manual I online manuals, cookbooks, examples 42 / 60 Python Crash Course – Programming tools Documentation Documentation levels How much documentation is just enough I Code should explain itself I ”Don’t comment your code, code your comments” I Explain what you do and why you do it, not how you do it. Rules of thumb: I I I I I Every file gets a block comment at the beginning, describing what its purpose is, and what the interface is Every class, method and function gets a block comment describing input, output, side effects (there should be none) and additional information (e.g. literature) Variables get a comment line, if they are of major importance for the code Code blocks get a comment if they implement a non-trivial algorithm. 43 / 60 Python Crash Course – Programming tools Documentation docstrings in Python Outline The actual problem Who you are Who you can be Defensive programming Some thoughts Version control Introduction Hello git Documentation Documentation levels docstrings in Python doxygen Testing Testing strategies doctests in Python Unit tests in Python 44 / 60 Python Crash Course – Programming tools Documentation docstrings in Python docstrings I In-code documentation for module, class, function or method I Enclosed by ’’’ or """ I Is read by built-in documentation system 45 / 60 Python Crash Course – Programming tools Documentation docstrings in Python docstrings def inc (a , b =1): ’’’ inc (a , b =1) -> sum of a and b inc ( a ) -> a +1 ’’’ return a + b >>> import mymodule >>> help ( mymodule . inc ) 46 / 60 Python Crash Course – Programming tools Documentation doxygen Outline The actual problem Who you are Who you can be Defensive programming Some thoughts Version control Introduction Hello git Documentation Documentation levels docstrings in Python doxygen Testing Testing strategies doctests in Python Unit tests in Python 47 / 60 Python Crash Course – Programming tools Documentation doxygen Getting started I doxygen generates technical documentation from in code documentation I It can generate minimal documentation from undocumented code... I Or generate in-depth documentation from special markers in your in-code documentation $ doxygen -g doxygen . cfg $ emacs doxygen . cfg $ doxygen doxygen . cfg 48 / 60 Python Crash Course – Programming tools Testing Testing strategies Outline The actual problem Who you are Who you can be Defensive programming Some thoughts Version control Introduction Hello git Documentation Documentation levels docstrings in Python doxygen Testing Testing strategies doctests in Python Unit tests in Python 49 / 60 Python Crash Course – Programming tools Testing Testing strategies Why test I Tests define the proper behaviour of the code. I Tests help you detect bugs due to side effects of new features I A reasonable strategy: Define tests before you start programming: Test Driven Development (TDD) Caution: I A passed test is no guarantee for correct code. I Correct code is not the same as good code (readability, modularity, locality, portability, efficiency,. . . ) I Seduces to program only what is necessary for the next test. I Not every requirement can be translated into a test. 50 / 60 Python Crash Course – Programming tools Testing Testing strategies Testing levels Unit testing : Verify the functionality of a specific section of code, usually at function or class level. Integration testing : Test interfaces between modules or similar components (e.g. of a pipeline) System testing : Check the whole set of modules, files, programs, pipelines etc. to verify that it meets its requirements. 51 / 60 Python Crash Course – Programming tools Testing doctests in Python Outline The actual problem Who you are Who you can be Defensive programming Some thoughts Version control Introduction Hello git Documentation Documentation levels docstrings in Python doxygen Testing Testing strategies doctests in Python Unit tests in Python 52 / 60 Python Crash Course – Programming tools Testing doctests in Python doctests I In Python, unit tests can be defined within the docstrings I doctests are executed calling doctest.testmod() 53 / 60 Python Crash Course – Programming tools Testing doctests in Python docstrings """ This is the factorial module . The factorial module supplies one function , factorial (). For example , >>> factorial (5) 120 """ def factorial ( n ): """ Return the factorial of n , an exact integer >= 0. >>> [ factorial ( n ) for n in range (6)] [1 , 1 , 2 , 6 , 24 , 120] >>> factorial (30) 265252859812191058636308480000000 L >>> factorial ( -1) Traceback ( most recent call last ): ... ValueError : n must be >= 0 """ if n < 0: raise ValueError ( ’n must be >= 0 ’) if n == 0: return 1 return n * factorial (n -1) if __name__ == " __main__ " : import doctest doctest . testmod () 54 / 60 Python Crash Course – Programming tools Testing doctests in Python docstrings """ This is the factorial module . The factorial module supplies one function , factorial (). For example , >>> factorial (5) 120 """ def factorial ( n ): """ Return the factorial of n , an exact integer >= 0. >>> [ factorial ( n ) for n in range (6)] [1 , 1 , 2 , 6 , 24 , 120] >>> factorial (30) 265252859812191058636308480000000 L >>> factorial ( -1) Traceback ( most recent call last ): ... ValueError : n must be >= 0 """ if n < 0: raise ValueError ( ’n must be >= 0 ’) if n == 0: return 1 return n * factorial (n -1) >>> python -m doctest test _doctest s1 . py 55 / 60 Python Crash Course – Programming tools Testing Unit tests in Python Outline The actual problem Who you are Who you can be Defensive programming Some thoughts Version control Introduction Hello git Documentation Documentation levels docstrings in Python doxygen Testing Testing strategies doctests in Python Unit tests in Python 56 / 60 Python Crash Course – Programming tools Testing Unit tests in Python import unittest I More complex unit tests can be done using the module unittest I Derive a class from unittest.TestCase. . . I and implement methods starting with test. These methods can call I I I I I assertEqual() to check for an expected result assertTrue() or assertFalse() to verify a condition; assertRaises() to check for raises exceptions For more complex settings overwrite setUp() and tearDown() 57 / 60 Python Crash Course – Programming tools Testing Unit tests in Python docstrings from test_doctests import factorial import unittest class TestFactorial ( unittest . TestCase ): def test_valid ( self ): self . assertEqual ( factorial (5) , 120) self . assertEqual ([ factorial ( n ) for n in range (6)] , [1 , 1 , 2 , 6 , 24 , 120]) self . assertEqual ( factorial (30) , 2 6 5 2 5 2 8 5 9 8 1 2 1 9 1 0 5 8 6 3 6 3 0 8 4 8 0 0 0 0 0 0 0 ) def test_invalid ( self ): self . assertRaises ( ValueError , factorial , -1) if __name__ == ’ __main__ ’: unittest . main () 58 / 60 Python Crash Course – Programming tools Testing Unit tests in Python docstrings from test_doctests import factorial import unittest class TestFactorial ( unittest . TestCase ): def test_valid ( self ): self . assertEqual ( factorial (5) , 120) self . assertEqual ([ factorial ( n ) for n in range (6)] , [1 , 1 , 2 , 6 , 24 , 120]) self . assertEqual ( factorial (30) , 2 6 5 2 5 2 8 5 9 8 1 2 1 9 1 0 5 8 6 3 6 3 0 8 4 8 0 0 0 0 0 0 0 ) def test_invalid ( self ): self . assertRaises ( ValueError , factorial , -1) >>> python -m unittest te st_unitt est1 . py 59 / 60 Python Crash Course – Programming tools Now what? I I Some action is better than no action! Start slow: I I I I I git init in every new code directory Only master branch Get used to write in-code documentation: docstrings use built scripts (makefiles, bash scripts) Then, accelerate as you become comfortable. 60 / 60