Download Introduction to APIs with Python

USING APIS WITH PYTHON
Introduction to gathering and analyzing data via APIs
Gus Cavanaugh
Agenda
■ Brief overview of Data Analytics Office Hours Meetup
■ Introduce the Application Programming Interface (API)
■ Overview of different types of APIs
■ General process for finding and using an API
■ Practical example: gather tweets from Twitter
■ Practical example: perform sentiment analysis with IBM’s Watson
■ Resources for future learning
Data Analytics Office Hours
■ About me: Consultant trying to use data to solve business problems
■ Meetups have been super helpful for learning technical subjects. But most are in-depth
presentations on a topic. As such, I often quickly get lost.
■ When I gave meetup talks, the best questions were often asked by individuals after the
talk was over. Worse, they were usually foundational questions that meant the attendee
missed out on a lot of value
■ I want this meetup to be the place where those types of questions are asked.
■ Presentations will be short or non-existent – just enough to cover some basic ideas. After
that, it’s up to us.
■ To put it crudely, I want this to be the forum for “stupid” questions. If one person asks
one question they would be afraid to raise their hand and ask at a large meetup, this will
have been a success
My ask of you
■ Introduce yourself
■ Ask Questions…
– Through the webinar
– Through the meetup: http://www.meetup.com/DataAnalytics-Office-Hours/
– Via Email: [email protected]
– On Twitter: @GusCavanaugh
On to the material!
Application Programming Interfaces (APIs)
allow you to access data programmatically
(with code). Generally speaking this means
you get way more better data from a given
website than if you used only your web
browser.
What this looks like with Python
■ Hit Twitter API
■ Hit Watson API
What is an API?
■ Application Programming Interface (API): code talking to code
■ When we use a computer, we generally visit web sites using our browser. Here we see the
web sites Graphical User Interface (GUI). While GUIs are great for interacting with the web
site on a small scale, we may want to gather or provide more information than is
convenient with via manually doing that with a web browser
■ To solve this problem, companies will create APIs such that users of their site can gather
or provide data by writing code rather than using their browser. The data and functionality
they make available is entirely up to them.
■ Most companies offer some combination of free and paid access. Companies will often
enforce this by limiting the data one can access or limiting the frequency of access.
– Yelp will only allow free users of their API to access only one random review of a
restaurant
– Twitter will provide the same data to free and paid users, but will restrict the number
of calls free users can make in a given time period
REST APIs – a “good enough” intro
■ REST stands for Representational State Transfer
■ What you need to know: when an application is RESTful, you can access data at
specific URLs via HTTP
■ Example:
– When I visit my twitter timeline, https://twitter.com/GusCavanaugh, I’m making
an HTTP GET request with my browser. Twitter sends HTML, CSS, and JS to my
browser, which my browsers renders into the Twitter website
– With Twitter’s REST API, I write Python code to make an HTTP GET request to
the essentially the same URL. Twitter returns JSON formatted data to me (just
text). I can now parse that JSON with Python to get the data I want
Finding and Accessing APIs
■ If you find yourself heavily using a website, it may be a good candidate for using the
website’s API (almost always the case if one is gathering data for analysis)
■ To find out if a website has an API, look for a “developers” link on the companies home
page. Or just google “<website> + API”
– Most companies with APIs will provide documentation at a url similar to
dev.company.com, .e.g., https://dev.twitter.com/overview/documentation
■ You will need to register with each with each company
– You may be asked to provide a url: http:127.0.0.1 (your local host) will work – any
valid url, regardless of whether it is a real website, will work.
– You will be given API credentials, usually a key & secret
– DO NOT SHARE THESE WITH ANYONE EVER!!!
■
The keys and secrets you see in this presentation are fake
Using the API
■ Read the documentation!
– This is where you’ll find the data you can access and the url you will need to
use
– You will need to make your API call – generally a GET or PUT request – with the
information specified in the company’s documentation. Think of this like a
secret handshake. You need to begin the sequence
– They will respond with data in a format that they have specified in the
documentation
■ We’ll use the Python library Requests to make our HTTP Requests
– If it’s not installed, you can install it with pip install requests
Twitter Example
■ Libraries required:
– python-twitter
■ Make API call with pythontwitter library
– Pass in arguments for
username and number of
tweets
■ Parse response with pandas
– Not required, just nice
Watson Example
■ Libraries required:
– requests, json
■ Unlike with twitter, we don’t have
a Python wrapper for Watson
■ We have to use requests to
make a post request
■ Then we need to use the json
library to parse the results
■ Finally we use pandas (optional)
to clean the response