PyCon 2016 in Portland, Or
hills next to breadcrumb illustration

Tuesday 11:30 a.m.–noon

Better Testing With Less Code: Property Based Testing With Python

Matt Bachmann

Audience level:
Intermediate
Category:
Testing

Description

Standard unit tests have developers test specific inputs and outputs. This works, but often what breaks code are the cases we did not think about. Property based testing has developers define properties of output and has the computer explore the possible inputs to verify these properties. This talk will introduce property based testing and provide real world examples and patterns.

Abstract

Strong testing is the best safety net a developer can have when changing code. A good test suite can prevent regressions and provide confidence that what you put out to the world actually works. However, in the real world writing these tests is extremely difficult. Testing individual cases tends to lead to large amounts of code that must be maintained and often we fail to catch all the corner cases and potential points of failure. Property Based Testing is a method that tries to overcome these failures by offloading the creation of test cases to the computer. As a developer you describe the input to your algorithms and then define what must be true of any valid output. The computer then takes this and throws random data into your algorithm trying to find a failure. A good framework will then take a failure and reduce it down to a simple case that the developer can debug and solve. The catch is coming up with these properties can be very difficult. You need to define useful properties! This talk goes over a few patterns that should inspire developers to get started. First we go over reversible operations. Using serialization as an example we will write a test to ensure serializing and deserializing an object can be done without losing data. Next we will show how property based testing can assist with rewriting or optimizing legacy code. Finally this talk show how this style of testing can be used to write tests that work with stateful systems. Using a priority queue as an example we will show how a property based testing framework can verify the properties of the queue (items come out in priority order) remain true no matter what sequence of operations are applied to the queue.