Self Improving AI Agents

This graduate seminar course covers the latest techniques and applications of AI agents that can continuously improve themselves through interaction with themselves and the environment. The course will start with self-improvement techniques for LLMs, such as constitutional AI, using learned/domain-specific verifiers, scaling test-time compute, and combining search with LLMs. We will then discuss the latest research in augmenting LLMs with tool use and retrieval techniques, and orchestrating AI capabilities with multimodal web interaction. We will next discuss multi-step reasoning and planning problems for agentic workflows, and the challenges in building robust evaluation and orchestration frameworks. Industry applications that will be discussed include coding agents, research assistants in STEM, robotics and more. Students will work on an original research project in this area, discuss the suggested readings in each class, and learn from invited academic and industry speakers. Prerequisites: CS224N or CS229S; Fluency in Python programming and using large language model APIs.