AI Inference Infrastructure

The emergence of foundation models has revolutionized AI applications, but the most powerful AI systems today are increasingly compound systems - compositions of multiple models, retrievers, tools, and traditional software components. This research seminar explores the intersection of cloud computing infrastructure and compound AI systems, covering both the infrastructure required to build and deploy these systems at scale and the architectural patterns that make them effective. Students will participate in guest lectures from leading experts across academia and industry, including leaders of companies like Databricks, OpenAI, Foundry, NVIDIA, Microsoft, and more, read and discuss papers, and complete a quarter-long project in groups of 2-3.