07-09, 13:30–17:30 (US/Pacific), Room 318
Writing correct software is difficult, and even scientists don’t always get it right
[citation needed].
Hypothesis is a testing package that will search for counterexamples to your
assertions – so you can write tests that provide a high-level description of your
code or system, and let the computer attempt a Popperian falsification. If it
fails, your code is (probably) OK… and if it succeeds you have a minimal input
to debug.
Come along and learn the principles of property-based testing, how to use
Hypothesis, and how to use it to check scientific code – whether highly-polished
or quick-and-dirty!
This tutorial is designed to promote simple but powerful methods for bolstering your software-based work/research with property-based testing, and thereby help to close the aperture on “the things that could go wrong” because of your code. It is structured as four blocks, each consisting of a short talk, live-coded demo, and extensive exercises for attendees:
- Property-Based Testing 101: core concepts and the core of the Hypothesis library
- Describe your Data: from numbers, to arrays, to recursive and more complicated things
- Common Tests: from "does not crash" to "write+read == noop" to 'metamorphic relations'
- Putting it into Practice: use what you've learned to find real bugs in a real project!
Each of these blocks runs for 40-60 minutes, consisting of a 5-15 minute presentation, occasional live-coded demos, and guided exercises. Along with a short break in the middle, we find this pattern balances content, practice, and focus well for most classes.
Because this tutorial covers testing scientific code with Hypothesis (contrast
Python code in general), a substantial fraction of the time is spent working with
data in Numpy arrays and Pandas dataframes.
The only assumed knowledge is the existence of and simple use of those data
structures, as I don't have to do an introduction to Numpy/Pandas as well as
Hypothesis!
Zac grew up in Australia eating dark chocolate, reading books, and occasionally indulging in both at the same time. By day he works at Anthropic, an AI safety and research company in San Francisco; by night he (co-) maintains Hypothesis and Pytest, and contributes to a range of other Python projects. You can read more about him at https://zhd.dev/