07-07, 08:00–12:00 (US/Pacific), Ballroom C
In this tutorial, you will learn how to integrate Large Language Models (LLMs) directly into Python programs as thoughtfully-designed core components of the program rather than bolt-on additions. This hands-on session teaches design principles and practical techniques for incorporating LLM outputs into program control flow. We will use LlamaBot, an open-source Python interface to LLMs, focusing on local execution with local and efficient models.
This hands-on tutorial teaches practical integration of Large Language Models (LLMs) into Python programs using LlamaBot and Ollama. Working with locally-run models that fit within 16GB RAM, participants will build a git commit message generator while learning core concepts of LLM application development.
Through Jupyter notebooks, we'll progress from basic LLM interactions with SimpleBot to structured outputs using Pydantic, culminating in systematic evaluation and practical deployment. The tutorial emphasizes learning-by-doing: participants will experiment with different models, prompting strategies, and temperature settings to understand their effects on output quality.
Key learning outcomes include mastering prompt design, implementing structured generation with schema validation, developing systematic evaluation approaches, and integrating LLM-powered features into existing workflows. The session concludes with a class-chosen discussion on broader implications of LLM applications in practice.
Installation instructions will be provided on the official GitHub repository.
Prerequisites –- Python programming experience
- Hardware Requirements:
- Mac: 16GB RAM minimum
- Linux/Windows: 16-32GB RAM
- Ability to install and run Ollama and the
smollm2,gemma2:2b,mistral, andphi4models
As Senior Principal Data Scientist at Moderna Eric leads the Data Science and Artificial Intelligence (Research) team to accelerate science to the speed of thought. Prior to Moderna, he was at the Novartis Institutes for Biomedical Research conducting biomedical data science research with a focus on using Bayesian statistical methods in the service of discovering medicines for patients. Prior to Novartis, he was an Insight Health Data Fellow in the summer of 2017 and defended his doctoral thesis in the Department of Biological Engineering at MIT in the spring of 2017.
Eric is also an open-source software developer and has led the development of pyjanitor, a clean API for cleaning data in Python, and nxviz, a visualization package for NetworkX. He is also on the core developer team of NetworkX and PyMC. In addition, he gives back to the community through code contributions, blogging, teaching, and writing.
His personal life motto is found in the Gospel of Luke 12:48.