07-09, 13:55–14:25 (US/Pacific), Room 315
Large language models (LLMs) enable powerful data-driven applications, but many projects get stuck in “proof-of-concept purgatory”—where flashy demos fail to translate into reliable, production-ready software. This talk introduces the LLM software development lifecycle (SDLC)—a structured approach to moving beyond early-stage prototypes. Using first principles from software engineering, observability, and iterative evaluation, we’ll cover common pitfalls, techniques for structured output extraction, and methods for improving reliability in real-world data applications. Attendees will leave with concrete strategies for integrating AI into scientific Python workflows—ensuring LLMs generate value beyond the prototype stage.
LLMs have transformed the landscape of data-driven software, enabling applications in information retrieval, automated summarization, and intelligent assistants. However, many teams struggle to move beyond early-stage demos—where models appear to work well in controlled environments but fail in production due to hallucinations, non-determinism, and poor evaluation practices.
This talk addresses that gap. It presents a structured framework for incorporating LLMs into real-world applications, grounded in software engineering best practices and scientific computing principles. Rather than focusing solely on model performance, we’ll emphasize how to design, evaluate, and iterate on AI-powered systems effectively.
Attendees will gain insights into:
- The LLM software development lifecycle (SDLC)—how it differs from traditional ML and software workflows.
- Evaluating business and scientific value—ensuring LLM outputs align with real-world needs.
- Handling non-determinism and hallucinations—logging, monitoring, and structured output techniques.
- Beyond conversations: Automating structured workflows—using LLMs for knowledge extraction, document processing, and decision support.
Intended Audience
This talk is for data scientists, software engineers, and AI/ML practitioners looking to:
- Move beyond toy LLM demos and build production-ready systems.
- Understand how software engineering principles apply to AI-driven applications.
- Learn how to evaluate and iterate on LLM outputs to ensure robustness and reliability.
Prior experience with Python and scientific computing is expected, but attendees don’t need prior LLM expertise—this talk focuses on software and systems principles applicable across AI applications.
Key Takeaways
By the end of this talk, attendees will understand:
- Why many LLM projects stall in proof-of-concept purgatory.
- The key differences between LLM development and traditional software engineering.
- How to design an iterative LLM SDLC, incorporating monitoring, evaluation, and structured outputs.
- Strategies for handling non-determinism and ensuring AI models work reliably in production.
Hugo Bowne-Anderson is an independent data and AI consultant with extensive experience in the tech industry. He is the host of the industry Vanishing Gradients, where he explores cutting-edge developments in data science and artificial intelligence.
As a data scientist, educator, evangelist, content marketer, and strategist, Hugo has worked with leading companies in the field. His past roles include Head of Developer Relations at Outerbounds, a company committed to building infrastructure for machine learning applications, and positions at Coiled and DataCamp, where he focused on scaling data science and online education respectively.
Hugo's teaching experience spans from institutions like Yale University and Cold Spring Harbor Laboratory to conferences such as SciPy, PyCon, and ODSC. He has also worked with organizations like Data Carpentry to promote data literacy.
His impact on data science education is significant, having developed over 30 courses on the DataCamp platform that have reached more than 3 million learners worldwide. Hugo also created and hosted the popular weekly data industry podcast DataFramed for two years.
Committed to democratizing data skills and access to data science tools, Hugo advocates for open source software both for individuals and enterprises.