SciPy 2024

Andrew Huang

I'm an atmospheric scientist, python developer, and open source contributor working on the HoloViz ecosystem.

I am the lead developer of the Panel chat components to easily build an interface for interacting with Large Language Models (LLMs). I have shared applicable examples of integrating Panel chat components with LangChain, OpenAI, Mistral, LlamaCpp on

Connect with me on

The speaker's profile picture


From RAGs to riches: Build an AI document inquiry web-app
Pavithra Eswaramoorthy, Dharhas Pothina, Andrew Huang

As we descend from the peak of the hype cycle around Large Language Models (LLMs), chat-based document inquiry systems have emerged as a high-value practical use case. Retrieval-Augmented Generation (RAG) is a technique to share relevant context and external information (retrieved from vector storage) to LLMs, thus making them more powerful and accurate.

In this hands-on tutorial, we’ll dive into RAG by creating a personal chat app that accurately answers questions about your selected documents. We’ll use a new OSS project called Ragna that provides a friendly Python and REST API, designed for this particular case. We’ll test the effectiveness of different LLMs and vector databases, including an offline LLM (i.e., local LLM) running on GPUs on the cloud-machines provided to you. We'll then develop a web application that leverages the REST API, built with Panel–a powerful OSS Python application development framework.

Ballroom B/C