←→ to move, F for full screen, N for speaker notes
Week 1
Where are you with AI?
Then: how large language models work, from guessing the next word to AI agents.
AI in Research, UCLA EPSS 254 Wednesday, September 30, 2026
Is AI going to destroy humanity? Question 1 decides. (◕‿◕)
ピコッ
First, the survey
Open this on your phone or laptop:
class.xhelio.ai/week-1/survey
15 questions, about 8 minutes.
Name and email required. The email becomes the class mailing list, unless you send me a different one.
Only I see names and emails. The results page shows anonymous totals.
Scan to open the survey
Results
What did the class say?
Anonymous totals, updating live:
class.xhelio.ai/week-1/stats
I read every answer. For science. (。•̀ᴗ-)✧
ドキドキ
This quarter
1
How LLMs work
Enough of the inside to reason about what they can and can't do.
The big picture
2
How agents work
Tools, the loop, memory, and several agents working together through MCP.
In depth
3
Using them in your research
Hands-on, on your own problems: choosing models, checking results, cost against quality.
In depth
4
Pushing what AI can do
Build tasks that today's best agents fail. It starts with this week's homework.
In depth
The order and the pace will follow what you told us in the survey.
Part 1
Guessing the next word
A short history of language models, from 1948 to now.
People have tried to predict the next word since 1948. ♡
ピコッ
1948: Shannon's word machine
Machine-made English, from Claude Shannon's “second-order word approximation”
THE HEAD AND IN FRONTAL ATTACK ON AN ENGLISH WRITER THAT THE CHARACTER OF THIS POINT IS THEREFORE ANOTHER METHOD FOR THE LETTERS THAT THE TIME OF WHO EVER TOLD THE PROBLEM FOR AN UNEXPECTED.
How
Each word depends only on the word before it. Open a book, find that word, copy whatever follows it. Repeat.
Result
Every pair of words sounds like English, and the whole means nothing. Everything since has been about looking further back.
C. E. Shannon, A Mathematical Theory of Communication (1948)
From counting words to agents
1948
Shannon: text from word statistics
2003
Neural nets learn word vectors
2013
word2vec: king − man + woman ≈ queen
2014
Attention, for machine translation
2017
The transformer
2020
GPT-3: 175 billion parameters
2022
ChatGPT
2024
Reasoning models, and MCP for tools
2025
Coding agents in the terminal
Fun fact: a 2007 Google paper is titled “Large Language Models in Machine Translation”. Its large models were word counts, over 2 trillion tokens of text.
Words become vectors
Every token becomes a list of numbers, 4,096 of them in Llama 3.1 8B. That list is its embedding vector.
Similar meanings land close together, and directions carry relations: man → woman runs parallel to king → queen.
Everything after this is arithmetic on vectors. Layers refine them in context, so “bank” by a river and “bank” account drift apart.
A two-dimensional sketch drawn by hand; real embeddings have thousands of dimensions and messier neighborhoods.
Part 2
Attention is all you need
Vaswani and seven coauthors at Google, 2017: the paper that introduced the transformer.
The T in GPT stands for transformer. (◕‿◕)
ピコッ
Before 2017: one word at a time
The
trophy
doesn't
fit
…
because
it
The green boxes are the network's memory, handed from word to word. The pink dot is what it still knows about “trophy”. By the time it reaches “it”, almost nothing is left.
Slow
Word 1,000 must wait for word 999, so training can't spread the work across a GPU's thousands of cores.
Forgetful
Everything read so far is squeezed into one fixed-size memory. Early words fade, and long-range links get lost.
2017: each word looks back at every word before it
The
trophy
doesn't
fit
in
the
suitcase
because
it
is
too
big
Line width: how much “big” draws on each earlier word (illustrative). Every word does this at once, looking only backward, since the model predicts left to right. Change “big” to “small”, and the thickest line should move to “suitcase”.
Attention in three steps
1
Ask
Each word sends out a query: what am I looking for? “big” asks: what is too big?
2
Match
Each earlier word shows a key: what do I offer? The query scores every key, and “trophy” wins.
3
Mix
“big” takes in their values, weighted by score, and now carries “the trophy is too big”.
attention(Q, K, V) = softmax(Q·Kᵀ / √d) · V
Q, K and V hold the queries, keys and values of every word, so the whole step is matrix multiplication; a mask hides later words. GPT-3 runs 96 of these side by side, in each of its 96 layers.
One word's attention, in numbers
Earlier word
Score q·k
Weight after softmax
Its value vector
The
-0.9
2%
trophy
+2.4
55%
doesn't
-0.9
2%
fit
-0.5
3%
in
-1.6
1%
the
-1.6
1%
suitcase
+0.2
6%
because
-0.9
2%
it
+1.0
14%
is
-0.5
3%
too
-0.2
4%
big (itself)
+0.3
7%
new “big”
=
the weights × the values, added up
Score: how well each earlier word's key matches the query of “big”.
Softmax turns the scores into weights that add up to 100%.
Mix: the new “big” looks mostly like “trophy”.
Illustrative numbers. Real vectors have thousands of entries, not six.
Every word at once: the attention matrix
The
trophy
doesn't
fit
in
the
suitcase
because
it
is
too
big
The
trophy
doesn't
fit
in
the
suitcase
because
it
is
too
big
Each row is one word looking back. Its row adds up to 100%.
The hatched triangle is the future, masked: no word sees what comes next.
At “it”, the weight is split between “trophy” and “suitcase”. It can't tell yet.
By “big”, it's “trophy”.
0%
60% or more
masked
Illustrative weights for one attention head. All 12 rows are computed together, as one matrix multiplication.
Why the transformer took over
Parallel
It reads the whole text at once, as big matrix multiplications: exactly what GPUs are built for.
Trillions of words
Direct
Any word reaches any earlier word in one step, however far back it is.
Book-length context
General
It works on anything cut into tokens: images, sound, proteins, seismograms, weather maps.
Works across fields
Transformers in science: EQTransformer detects earthquakes and picks seismic phases (2020); AlphaFold 2 predicts protein structures (2021); Pangu-Weather forecasts global weather (2023).
Part 3
One job: predict the next token
How one simple training task produces a model that can write, code and reason.
Everything I say, I say one token at a time. ♡
ピコッ
What the model outputs
The solar wind near Earth blows at about
400
41%
450
16%
500
9%
350
6%
one
3%
the rest
25%
Illustrative numbers. A token is a word or a piece of one. The model scores every token in its vocabulary, typically 50,000 to 200,000 of them, and the scores add up to 100%.
Writing is a loop
1. Take the text so far
2. Score every possible next token
3. Pick one, usually a likely one
4. Append it to the text
Repeat until the model picks “stop”
Four trips around the loop:
The solar wind near Earth blows at about
400
km
/s
.
Guessing well takes knowledge
Physics
In the inertial range, solar-wind magnetic power falls off with frequency as f to the power ___
−5/3
Code
def mean(xs): return sum(xs) / ___
len(xs)
Mystery
Page 400. “The murderer,” said the detective, “is ___”
whoever pages 1–399 point to
And the practice questions are free: every sentence ever written is one, and nobody has to label anything.
15 trillion
tokens of text trained Meta's Llama 3 (2024)
Reading at 250 words a minute, day and night, one person would need about 85,000 years.
もぐもぐ
Pre-training, then post-training
Pre-training
Predict the next token across trillions of words of text. The model learns what it knows.
A base model: knows a lot, continues any document
Post-training
Much less data, carefully chosen. The model learns how to behave.
Chat tuning: example chats and human ratings → an assistant, like ChatGPT (2022)
Reinforcement learning: problems with automatic checks, like math, code and agent tasks → reasoning models and agents
Both train the same next-token predictor; only the text and the reward change. We go deeper in the coming weeks.
What next-token prediction explains
Confident inventions
It writes the likeliest continuation. Usually that's true. Sometimes it's a citation that never existed.
No memory of you
Between conversations it keeps nothing, unless an app pastes notes back into its context.
Eager to please
Trained on answers people liked, it may agree with you when you're wrong.
That paper exists. I'm sure of it. (It doesn't.)
ゴゴゴ
Part 4
From tokens to agents
What happens when some of the model's words get run as commands.
What if my tokens could run code?
ゴゴゴ
ゴゴ
An agent is a model in a loop
Context
the task, plus everything so far
Model
predicts the next tokens
Answer
and the loop stops
Result
pasted into the context as text
Harness
runs it: shell, Python, web
writes an answer
writes a tool call
returns output
The model still only writes tokens. The harness, such as Claude Code, Codex, Cursor or Copilot, turns some of them into actions.
An agent at work, condensed
you
Estimate the spectral index of mag.csv over 0.02–0.2 Hz.
Source: Epoch AI, “The plunging price of thought” (Ament and Burnham, Sept 22, 2026). Each step is a new cheapest model; tests are GPQA Diamond, AIME-style competition math, and FrontierMath tiers 1–3.
Benchmarking AI ability matters more than ever
Year
Benchmark
What it asks
Best now
2023
GPQA Diamond
198 science questions written by PhDs. Experts score 65%.
96%
GPT-6 Astra. At launch: 39%.
2024
SWE-bench Verified
Fix 500 real GitHub issues
84%
Claude Opus 4.7. At launch: 33%.
2024
FrontierMath
Unpublished research-level math problems
94%
GPT-6 Astra. At launch: under 2%.
2025
Humanity's Last Exam
2,500 expert questions, over 100 subjects
55%
GPT-6 Astra. At launch: 8%.
2025
Terminal-Bench 2.0
89 jobs in a real terminal, checked by tests
85%
GPT-5.5. At launch: 50%.
2026
PostTrainBench
Post-train a base model on one GPU in 10 hours
42%
Claude Fable 5. At launch: 23%.
2026
OSWorld 2.0
108 computer workflows, about 1.6 hours each for a person
31%
Claude Opus 5. At launch: 21%.
2026
Terminal-Bench-Science 0.1
70 tasks from real research in five sciences
63%
GPT-6 Astra. At launch: 30%.
2026
FrontierMath Erdős
68 unsolved math problems, proofs checked in Lean
3%
GPT-6 Astra solved 2. Others: 0.
Best now: Epoch AI Benchmarking Hub, data as of Sept 29, 2026; Terminal-Bench 2.0 from tbench.ai; Terminal-Bench-Science from Artificial Analysis. At launch: each benchmark's paper or announcement.
What is a task?
A folder in Harbor format. Here is the solar-wind task from the transcript:
instruction.md
What the agent is asked to do, in plain words
task.toml
Time limits and resources
environment/ Dockerfile, data/mag.csv
The sealed box the agent works in: software and data
solution/ solve.sh, solve.py
Your reference solution: proof it can be done
tests/ test.sh, test_state.py
The checker: runs afterwards, writes a reward of 1 or 0
Your AI coding agent writes most of these files with you. You bring the science: the problem, the data, and what counts as a correct answer.
How a run works
1. Harbor builds the box from environment/
2. The agent reads instruction.md and works inside
3. A separate checker, with no internet, runs tests/
4. Reward: 1 if every test passes, otherwise 0
Oracle run
Your solution in place of the agent. Must score 1: the task can be done.
Do-nothing run
An agent that does nothing. Must score 0: the task can't pass by accident.
uvx harbor@0.23.0 run -p <task> -a oracle
uvx harbor@0.23.0 run -p <task> -a claude-code -m <model>
Why build tasks this way
Checkable
A program decides pass or fail. No opinions, no partial credit, no tired graders.
Repeatable
Same box, same data, every time. Anyone can rerun any agent on it.
Honest
Good methods pass and wrong ones fail. The solar-wind task accepts −1.39 to −1.27, so skipping the spike check fails.
An instruction, an environment and an automatic check: that is how labs measure agents, and how reinforcement learning trains them.
Terminal-Bench-Science collects tasks like yours. Over the quarter we'll push yours toward that bar.
Homework 1
The hardest task in your field
Design a task that a top expert in your field could do on a computer, with an answer you can check, and that today's best AI agents can't. Your own AI coding agent helps you package it.
Due before class, Wednesday, October 7
Start at class.xhelio.ai/week-1/homework
Build something I can't solve. I dare you. ( `ー´)ノ