Download Problem Solving with Python Challenges 6 – Reading and writing

 Problem Solving with Python Challenges 6 – Reading and writing structured file data (and validating input) Contents 1 Introduction ........................................................................................................................................... 1 2 Reading and writing structured (CSV) data ............................................................................. 2 3 Useful string methods ........................................................................................................................ 4 4 Combining boolean expressions with boolean operators (and, or, not) ................. 5 4.1 Combining conditions to validate user input .................................................................. 6 5 Sorting using the sorted function ............................................................................................. 7 6 Using the Python random module .............................................................................................. 8 7 The user data challenge .................................................................................................................... 9 1 Introduction Challenge 6 is to process and verify user data in simplified version of how Web sites manage user ids, passwords and favourite words. By the end we will see: •
•
•
how to store and retrieve information from a CSV (comma separated values) file, how to check that user input meets certain requirements (e.g. that a password is a certain length and has upper case letters), how to verify randomly selected characters from a favourite word Section 2 is an exercise to read and write structured (CSV) data. Section 3 introduces some useful string functions/methods, including how to verify properties of a string such as whether it is all lower case. Section 4 looks at combining boolean conditions and how to validate user input (using string methods). Section 5 covers sorting sequences, which is useful for sorting a list of values for veification. Section 6 covers generating random numbers and sequences, which is useful for picking random values to verify. Section 7 introduces the challenge to put everying together to manage user data. 1 2 Reading and writing structured (CSV) data In this section we will follow an exercise to: •
•
•
create a CSV (comma separated value) file in Excel read and modify the file using Python import the modified file back into Excel. We will use a list of lists to store the information in Python. It is just like a spreadsheet in Excel that is made up of a list of rows and each row is a list of cells. For example the following Excel spreadsheet: can be represented by the following list of lists in Python: List indexes 0 1 2 0 [ 'bob',
'bobspass',
'bobsword' ]
1 [ 'tom',
'tomspass',
'tomsword' ]
2 [ 'alice',
'alicespass',
'alicesword' ]
Row 1 of the spreadsheet comprises cell A1, B1 and C1 or (bob, bobspass, bobword). List 0 of the list of lists is ['bob', 'bobspass', 'bobsword']. Cell A1 in the spreadsheet is element [0][0] of the list of lists, cell B1 is element [0][1], cell C1 iselement [0][2] etc. Complete the following exercise to demonstrate this in practice. 1. In Excel create the following spreadsheet and save the spreadsheet as comma separated values (CSV) file called: test.csv Close the spreadsheet. You can open test.csv in a text editor such as Notepad to see the CSV format. 2 2. In the same folder as test.csv, create the following new Python program called csvtest.py: import csv
data = []
# the Python csv module
with open('test.csv', newline='') as csvfile:
for row in csv.reader(csvfile):
data.append(row)
print(data)
for user in data:
print(user)
3. Run the program and you should see the following output: [['bob', 'bobspass', 'bobsword'], ['tom', 'tomspass',
'tomsword'], ['alice', 'alicepass', 'alicesword']]
['bob', 'bobspass', 'bobsword']
['tom', 'tomspass', 'tomsword']
['alice', 'alicepass', 'alicesword']
The first line of output is the list of lists. Each subsequent line is a list in order from the list (equivalent to each row of the spreadsheet). 4. Add the following lines to the csvtest.py program to change Bob's password and save the file: data[0][1] = 'bobsnewpass'
with open('test.csv', 'w', newline='') as csvfile:
writer = csv.writer(csvfile)
for uer in data:
writer.writerow(user)
5. Use Excel to open test.csv and you should see the modified file: You can also open test.csv in a text editor to see the changes. 3 The complete program is: import csv
data = []
# open test.csv, read each row and append it to data
with open('test.csv', newline='') as csvfile:
for row in csv.reader(csvfile):
data.append(row)
# output the list of lists data
print(data)
# output each list/row of data
for user in data:
print(user)
# modify bob's password
data[0][1] = 'bobsnewpass'
# write each list/row of data to test.csv
with open('test.csv', 'w', newline='') as csvfile:
writer = csv.writer(csvfile)
for user in data:
writer.writerow(user)
See: https://docs.python.org/3/library/csv.html?highlight=csv#module-­‐csv for further information about the Python csv module. 3 Useful string methods Python provides a number of useful string methods/functions. For example, you can capitalize a string as follows: s = 'abc'
t = s.capitalize()
print(t)
# prints Abc
There are other methods that convert the case of a string, test whether a string ends with a particular substring etc. The following table lists some useful methods to verify the properties of a string. These are methods/functions of a string that return True if the string satisfies a property and False otherwise. 4 Method Description Examples str.isalnum()
True if all characters in str are alphanumeric and str has at least one character. s = 'xyz'
s.isalnum()
t = '123abc'
t.isalnum()
u = '-abc'
u.isalnum()
True if all characters in str are alphabetic and str has at least one character. s = 'xyz'
s.isalpha()
t = '123abc'
t.isalpha()
u = '-abc'
u.isalpha()
str.isalpha()
See also: isdecimal(), isdigit() and isnumeric()
str.islower()
True if all characters in str are lower case and str has at least one character. str.isupper()
True if all characters in str are upper case and str has at least one character. s = 'xyz'
s.islower()
t = 'xyz123'
t.islower()
u = 'Xyz'
u.islower ()
s = 'XYZ'
s.isupper()
t = 'XYZ123'
t.isupper()
u = 'xyz'
u.isupper()
# True
# True
# False
# True
# False
# False
# True
# True
# False
# True
# True
# False
See: https://docs.python.org/3/library/stdtypes.html#string-­‐methods for further examples. 4 Combining boolean expressions with boolean operators (and, or, not) A boolean expression is a statement that is either true or false. For example: •
•
•
It is raining The dog is black 1 + 1 is less than 3 Often we need to know: 1. Whether two or more boolean expressions are all true. For example, is a dog black and does it have a long tail? 2. Whether at least one of two or more boolean expressions is true. For example, is it raining or snowing? 5 3. Whether the opposite of the boolean expression is true. For example, give me a car that is any colour but black (not black) In programming we use boolean AND, OR and NOT to combine boolean expressions in this way. The equivalent keywords in Python are and, or, not. For example: Boolean Description operator Python x AND y True if both x and if x and y:
print('both x and y are true')
y are true. else:
False if either x or print('one of x or y is false')
y is false. x OR y True if either x or if x or y:
print('at least one of x or y is true')
y is true. else:
False if both are x print('both x and y are false')
and y are false. NOT x True if x if false. False if x is true. if not x:
print('x is not true (false)')
else:
print('x is true')
4.1 Combining conditions to validate user input We can combine boolean conditions to validate user input. For example, the following program asks for user input and then validates that the input has 7 characters and is alphabetic. user_input = input('Enter a 7 letter alphabetic word: ')
if len(user_input) == 7 and user_input.isalpha():
print('Well done!', user_input, 'is valid')
else:
print(user_input,'is invalid')
Tasks 1. Change the above program so that the user is given 3 attempts to enter a valid word (you will need to use a for loop or a while loop) 2. Try to work out the combination of string methods to use to verify that a string input by a user satisfies the following constrains: • It is alphanumeric • It contains at least one numeric value • It contains at least one alphabetic value • It is mixed case • It is at least 8 characters long 6 3. Write a program that asks a user to input a new password and checks that the password satisfies the preceding constraints. Hints •
•
•
You can use a series of if statements and/or boolean operators to combine tests. Try writing an input_isvalid()function that includes the string validation tests. You can use the keyword break to break out of a for loop early. Alternatively you can use a while loop that only continues if a condition is true: while somecondition:
print('this is repeated as long as some_condition is true')
print('somecondition is no longer true')
5 Sorting using the sorted function Python provides the built-­‐in sorted function to sort items that are part of a sequence such as characters in a string, items in a list or numbers in a range. The sorted(items) function returns a new list of the items in ascending order. To sort in descending (reverse) order, set the reverse parameter to True: sorted(items, reverse=True)
Try the following three examples. 1. Sorting a list of numbers in ascending order unsorted_nums = [1, 3, 6, 2, 5, 4]
sorted_nums = sorted(unsorted_nums)
print(sorted_nums)
# [1, 2, 3, 4, 5, 6]
2. Sort a string in ascending order unsorted_str = 'adcb'
sorted_chars = sorted(unsorted_str) # ['a','b','c','d']
sorted_str = ''.join(sorted_chars)
print(sorted_str)
# 'abcd'
3. Sort a list in descending order unsorted_list = ['tom', 'dick', 'harry']
sorted_list = sorted(unsorted_list, reverse = True)
print(sorted_list)
# ['tom', 'harry', 'dick']
7 6 Using the Python random module The Python random module provides methods to generate random numbers and random sequences and to shuffle sequence. Try the following five examples. 1. Use random.randint to generate a random number in a specified range, e.g.: 0 <= n <= 10 import random
n = random.randint(0, 10)
print(n)
2. Use random.choice to select a random element from a sequence: import random
mylist = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
random_item = random.choice(mylist)
print(random_item)
mystring = 'alongstringofcharacters'
random_char = random.choice(mystring)
print(random_char)
3. Use random.shuffle to shuffle a list in place: import random
mylist = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
random.shuffle(mylist)
print(mylist)
mystring = 'alongstringofcharacters'
mystring_aslist = list(mystring)
random.shuffle(mystring_aslist)
myshuffled_string = ''.join(mystring_aslist)
print(myshuffled_string)
4. Use random.sample to generate a list of elements (or sample) from a sequence (or population): import random
# select 4 elements from mylist
mylist = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
random_items = random.sample(mylist, 4)
print(random_items)
# select 3 random characters from mystring
mystring = 'alongstringofcharacters'
random_chars = random.sample(mystring, 3)
print(random_chars)
8 5. Use random.sample, range and sorted to generate the indexes in order of 3 random characters from a word: import random
favouriteword = 'python'
length_of_word = len(favouriteword)
random_indexes = random.sample(range(length_of_word), 3)
sorted_indexes = sorted(random_indexes)
print(sorted_indexes)
See: https://docs.python.org/3/library/random.html for full details. 7 The user data challenge You have been given two Python files: •
•
userdata.py – a module that stores user data (user id, password and favourite word) in a CSV file called userdata.csv. The userdata module provides the following functions: o load_users – to load user data from userdata.csv (if the file exists) o save_users(users) – to save the users list of lists of user to userdata.csv o add_user(user, users) – to add the specified user [userid, password, favouriteword] to the users file o update_user(user, users) – to update the data for an exising users (if the user exists) o get_user(userid, users) – to get the specified user's data from the list of users (if the userid exists) o get_user_index(userid, users) – to get the index of the specified user manageusers.py – the start of a Python program that uses the userdata
module and is intended to mimic the following typical user options: o registration (if not already known) o login (if not already logged in) o favourite word verification by verifying 3 random characters of the word (if logged in) o password reset (if logged in) o favourite word reset (if logged in) o logout (if logged in) The challenge is to complete the manageusers.py program and to test that it works as expected. The expected behaviour is described in comments in the program. The basis of some of functionality is provided. For example, the login functionality is complete (apart from allowing 3 attempts to enter a correct password). Registration is mostly complete, except there is no verification of the data entered by a user. To start with run the manageusers.py program and see what happens when you select the various options. 9