Week 2
How an AI agent works
Wednesday, October 7, 11:00–11:50 a.m., Slichter 3853
Joining remotely? The Zoom link is on the home page.
An agent is a loop of API calls. This week we open one call and look inside: what a request holds, how the model chooses between writing text and calling a tool, why tokens cost what they cost, and how an agent keeps its context from overflowing.
In class
Four parts. Each card opens the slides at the start of that part.
- Part 1One callWhat an API, HTTP and JSON are, then one real request to the OpenAI Responses API and its reply. The two kinds of output: text or a tool call.
- Part 2Inside the callTool calls as tokens, strict JSON, reasoning effort, the context window, the KV cache, and why cached tokens now cost 1/10 to 1/50 of fresh ones.
- Part 3The loopThe agent loop in 15 lines, what fills its context, what one run costs, and five ways to keep the context small.
- Part 4Using itWhat a $200 subscription is worth at API prices, the basics before your first call, and research hygiene.
Further reading
- Week 2 readingThe details and sources behind every part of the slides, on one page.
- OpenAI: function callingThe request and response shapes we read in class.
- OpenAI: prompt cachingHow the prefix cache works, and how to check that it hits.
- OpenAI: the harmony response formatThe token stream the model actually sees, with roles, reasoning and tool calls.
- Thinking Machines: defeating nondeterminism in LLM inferenceWhy the same prompt gives different answers, even at temperature 0.