07-11, 13:55–14:25 (US/Pacific), Ballroom
Python notebooks are a workhorse of scientific computing. But traditional notebooks have problems — they suffer from a reproducibility crisis; they are difficult to use with interactive widgets; their file format does not play well with Git; and they aren't reusable like regular Python scripts or modules.
This talk presents a marimo, an open-source reactive Python notebook that addresses these concerns by modeling notebooks as dataflow graphs and storing them as Python files. We discuss design decisions and their tradeoffs, and show how these decisions make marimo notebooks reproducible in execution and packaging, Git-friendly, executable as scripts, and shareable as apps.
This talk explores the following question: How might one design a Python notebook that blended the best parts of interactive computing with the reproducibility, maintainability, and reusability of traditional software?
Python notebooks are a workhorse for scientific computation and research. And yet, while great for exploring data, traditional notebooks — in which scientists run cells one at a time, manually managing the state of their kernel in a tedious and error-prone fashion, while also manually managing their notebook's dependency environment — suffer from a well-documented reproducibility crisis. One study, by Pimentel et al., found that just a quarter of notebooks on GitHub could be run, and just four percent reproduced their outputs when re-executed. Additionally, because they are typically stored as JSON, notebooks fail to enjoy the usual benefits of code — maintainability, reusability, and interoperability with the Python ecosystem.
This talk presents a marimo, an open-source reactive Python notebook that addresses these concerns by modeling notebooks as dataflow graphs and storing them as Python files. marimo notebooks are reproducible, with a reactive runtime that eliminates hidden state and opt-in package management; interactive, with UI elements that are automatically synchronized with Python (no callbacks); expressive, supporting markdown and SQL that can be parametrized by arbitrary Python values; stored as pure Python files, so they are Git-friendly; executable as scripts; and shareable as web apps or WASM-powered static HTML.
To keep code and outputs consistent, marimo models notebooks as directed acyclic graphs on cells, based on the variables declared and referenced by each cell, and pairs this graph with a reactive runtime. Run or delete a cell and the runtime automatically marks affected cells as stale, optionally running them to eliminate hidden state and hidden bugs. We discuss our implementation, which is powered by static analysis and takes inspiration from reactive notebooks for other languages, particularly Pluto.jl. As a bonus, reactive execution makes it easy to work with interactive widgets, which are made available to the user upon importing marimo as a library into their notebook, and reuse notebooks as scripts or apps.
The marimo file format has the following properties: it is human readable, Git-friendly, usable as a regular Python module, executable as a script, and editable with a text editor. We show how the file format paves the way for virtual environment management as well as integration with other tools designed for code, like pytest.
marimo's design decisions come with tradeoffs, which we discuss. Reactive execution hardens reproducibility and enables rapid experimentation with data, but it imposes constraints on the ways that variables can appear in the notebook. Storing notebooks as code simplifies package management (and versioning with git), but means a separate solution must be developed for serializing and viewing notebook outputs. These tradeoffs may not be acceptable to all notebook users, especially those who use notebooks as extended REPLs or scratchpads — however, through a series of examples, we hope to demonstrate why these tradeoffs are worth it for researchers who use notebooks as a core part of their scientific process.
This talk is directed at anyone with an interest in about notebooks and interactive computation. By using marimo as a case study, we hope to answer a broader question. If notebooks were re-imagined as reactive and reusable Python programs, instead of REPL-like scratchpads — would that change the way you worked with code and data?
Akshay Agrawal is currently building marimo, a new kind of reactive notebook for Python that is reproducible, git-friendly (stored as Python files), executable as a script, and deployable as an app.
He is both a researcher, focusing on machine learning and optimization, and an engineer, having contributed to several open source projects, including TensorFlow during his time at Google, and CVXPY, of which he is a maintainer. He holds a PhD from Stanford University, where he was advised by Stephen Boyd, as well as a BS and MS in computer science from Stanford.