SciPy 2024

ultrack: large-scale versatile cell tracking in Python under segmentation uncertainty
07-11, 10:45–11:15 (US/Pacific), Room 316

Accurate cell tracking is essential to various biological studies. In this talk, we present Ultrack, a novel Python package for cell tracking that considers a set of multiple segmentation hypotheses and picks the segments that are most consistent over time, making it less susceptible to mistakes when traditional segmentation fails.
The package supports various imaging modalities, from small 2D videos to terabyte-scale 3D time-lapses or multicolored datasets in any napari-compatible image format (e.g. tif, zarr, czi, etc.).
It is available at https://github.com/royerlab/ultrack


Accurate cell tracking is essential to various biological studies, enabling the analysis of cell behavior, migration, and interactions. We introduce ultrack, a novel Python package for cell tracking that:
- Achieves state-of-the-art accuracy on the Cell Tracking Challenge.
- Is the fastest ILP-based tracker to date.
- Is easy to use on 2D, 3D, and multichannel datasets.
- Can process arbitrarily large datasets thanks to efficient out-of-memory processing.
- Scales with available computing resources, from modest laptops and Google Colab notebooks to large high-performance computer clusters.
- Is compatible with most image timelapse file formats.

Ultrack considers multiple candidate segmentations for tracking and picks the most temporal consistent segments. This enables combining the result of different segmentation approaches, increasing the chance that correct segments are present among all candidates.

The package's Python interface was designed to support a broad audience, from biologists with minimal coding experience who need to quickly interact with a tracking algorithm to bioimage analysts/machine learning researchers who want to customize it to meet specific needs and requirements.

Furthermore, it supports processing any napari-compatible file format (e.g. tif, zarr, czi, etc.), maximizing user-friendliness while avoiding data duplication. This ensures smooth integration into existing workflows and data management pipelines.

The main goals of the talk are to dive deep into Ultrack's inner workings and how most of it is accomplished with an extension of the classical image processing operator "watershed" plus an ILP formulation and to showcase a comprehensive collection of time-lapse datasets acquired on different types of microscopes (confocal, light-sheet, …), on different model organisms, and for a variety of dimensionalities: from 2D+t to 3D+t and multicolor. Tracking hundreds to tens of millions of segments per dataset in a few hours.

The package is fully available at https://github.com/royerlab/ultrack