top band

Saturday 1:55 p.m.–2:25 p.m.

Cutting Off the Internet: Testing Applications that Use Requests

Ian Cordasco

Audience level:
Intermediate
Category:
Best Practices & Patterns

Description

A brief and opinionated view of testing applications and libraries that use requests by a core-developer of requests. You will receive an overview of testing with responses, vcr, httpretty, mock, and betamax.

Abstract

Requests has over 23 million downloads from PyPI alone and is relied on by open source projects and companies alike but there's no established best practices to testing an application that uses requests. I am a core developer of requests and the author of github3.py. I use requests on an almost daily basis and have developed some simple and sensible ways of testing libraries and applications that use requests. I am also the author of betamax, one of the libraries we'll be discussing in this talk. **Why test at all?** Testing is generally accepted as a best practice in our industry. The difficulty is in testing anything that talks to the internet. We all know how to test Django apps that talk to a database. We know how to test most other applications we write. Clearly our applications that use requests can benefit from tests as well. **What tools exist?** There are many tools that already exist to help you test your code that uses requests. Some of the most popular include responses, httpretty, and vcr.py. Betamax's popularity is growing especially given its simplicity. There is a fundamental flaw with both responses and httpretty though: they encourage the user to write fixture data which means that any time the service changes, the user has to fix their fixtures to match the real world data. Tools like vcr.py and betamax relax that constraint by allowing a real interaction to take place, saving it to disk, and then re-using it. At any time, this interaction can be re-recorded to keep the fixture data up-to-date. What responses and httpretty especially excel at is making the user specify, for a given test, what request the user expects the code to make. In the simplest case, this can be done with the wildly popular mock library which has been included in the Python 3 standard library. **What patterns emerge?** In my experience, the best approach is a combination of integration and unit tests. Using mock to test the calls made to requests help narrow down test failures or real code failures in dependency upgrades or contribution acceptance. Similarly, integration tests can alert you to a potentially harmful change made in requests during an upgrade. Writing unit tests without providing fixture data also forces the developer to write code that is not coupled tightly. This is a pattern we as a community recognize as beneficial and maintainable.
bottom band background