Language models in depth

Week 4 covers what a language model does and why it fabricates. This page carries the rest: the field these models came out of, what changed as they got bigger, what has been added around them since 2023, and the argument over whether any of it resembles human intelligence. None of it is required for the Week 4 lab. It is here for the questions the seminar tends to raise, and for anyone who wants the mechanism in more detail than one session allows.

Deep learning

Language models are one branch of a field that predates them by decades. The “deep” in deep learning refers to stacked layers of simple units, each layer learning a representation built out of the one below it, with the connection weights set by gradient descent against a training objective. LeCun, Bengio, and Hinton’s 2015 review in Nature lays out the method and its history.

A network called AlexNet won the 2012 ImageNet image-recognition contest with a 15.3 percent top-five error rate against 26.2 percent for the runner-up, and the field turned toward deep networks after it. Image recognition, speech recognition, and protein structure prediction are built with deep networks too, which is part of why Bender restricts the stochastic-parrots critique to text generation. Bengio, Hinton, and LeCun received the 2018 ACM Turing Award, and Hopfield and Hinton received the 2024 Nobel Prize in Physics for “foundational discoveries and inventions that enable machine learning with artificial neural networks”.

Scale and emergent abilities

Next-token prediction sounds too thin to produce what these systems do, and the earliest evidence that it is not came in 2017, when an OpenAI model trained only to predict the next character of product reviews turned out to have become a strong sentiment classifier on its own. Nothing in its training asked for sentiment; tracking it simply helped predict the next character.

Scale then complicated the picture. As models grew from GPT-3’s 175 billion parameters toward PaLM’s 540 billion, some capabilities appeared abruptly instead of improving smoothly: identifying a movie from a string of emoji, for example, and about 5 percent of tasks in the BIG-bench evaluation suite show that kind of sharp jump. The training compute behind the jumps is hard to picture; running the training of the largest of these models on a machine doing a billion operations per second would take well over 100 million years. Researchers disagree over what the jumps are; some read them as genuinely new capability, others as statistical pattern-matching crossing a threshold that makes it visible.

Growth in these systems has been regular enough to be written down as a formula. Kaplan and colleagues showed in 2020 that prediction error falls as a power law in model size, training data, and compute across more than seven orders of magnitude, which turned “make it bigger” into a budgeted engineering decision. DeepMind’s 2022 Chinchilla paper corrected the recipe by showing that model size and training data should grow together, and that a 70-billion-parameter model trained on 1.4 trillion tokens beat the 280-billion-parameter Gopher trained on the same compute budget. A smaller model trained on more text can beat a bigger one on the same budget.

The budgets that buy this scale sit with very few organizations. Stanford’s AI Index priced the whole arc: the 2017 transformer that introduced the architecture cost around 900 dollars to train, GPT-4 cost an estimated 78 million, and Google’s Gemini Ultra an estimated 191 million, so Gemini Ultra’s training cost more than two hundred thousand times what the architecture’s first model cost seven years earlier. Epoch AI, working from a different accounting method, puts GPT-4’s final training run nearer 40 million dollars and finds the cost of the largest runs growing about 2.4 times a year since 2016, with a billion-dollar run arriving before 2027. The two estimates differ because one prices rented compute and the other amortizes hardware, and the gap between them is a good reminder to ask what any headline AI number is counting.

Retrieval and reasoning

The bare mechanism is not what anyone meets in a product. Two additions sit between the model and the user, and both are sold as fixes for the problems the mechanism creates.

Retrieval-augmented generation, introduced by Lewis and colleagues in 2020, searches a document collection first and hands the model the retrieved passages to answer from, so the answer is grounded in real sources instead of the blur. Legal vendors market this as the end of fabricated citations. A preregistered Stanford evaluation of the leading products found that Lexis+ AI and Thomson Reuters’s Westlaw AI-Assisted Research and Ask Practical Law AI “each hallucinate between 17% and 33% of the time.” Retrieval lowers the rate and does not remove it, because the model still writes the sentence.

Reasoning models are the second addition. Chain-of-thought prompting showed in 2022 that a model asked to work through steps solves more problems than one asked for the answer.

The technique arrived as a way of writing prompts and then became training. OpenAI’s o1, in September 2024, trained the behavior in, spending more computation before replying. The visible steps are not a record of the computation. Anthropic’s 2025 study gave models a hint, confirmed they used it, and found Claude 3.7 Sonnet mentioned the hint in its reasoning only 25 percent of the time and DeepSeek R1 only 39 percent. The explanation a model offers for its own decision is more generated text, produced by the same mechanism, and it can leave out what actually drove the answer.

Agents

The systems arriving in agencies now do not stop at text. An agentic system calls other software and takes many steps without checking back, so its output is a completed action such as a booking, a filing, or a sent message. Anthropic released the Model Context Protocol in November 2024 as a standard way to plug models into other software, and OpenAI’s Operator began clicking through websites on a user’s behalf in January 2025.

Capability here is measured in task length. METR’s 2025 study timed how long a human expert takes on tasks agents can finish with 50 percent reliability, and found that horizon had doubled about every seven months across the six years to 2025. METR keeps the chart current, and fitted to data from 2023 onward it now reports a doubling time of 124 days.

Deployment is going worse than the capability curve suggests. Gartner predicts more than 40 percent of agentic AI projects will be cancelled by the end of 2027 on cost, unclear value, or missing risk controls, and estimates that only about 130 of the thousands of vendors selling agentic AI are selling anything agentic, a practice it calls agent washing.

The question of who answers for an agent’s actions has already been tested. In Moffatt v. Air Canada, a British Columbia tribunal ruled in February 2024 that Air Canada owed a customer the bereavement fare its website chatbot had described, and rejected the airline’s argument that the chatbot was a separate entity responsible for its own information. The damages came to about 650 Canadian dollars, and the ruling holds an organization answerable for what its chatbot tells a client, which covers a benefits eligibility question as much as a fare question.

Machine intelligence

The premise that these systems imitate human intelligence goes back to the field’s founding document. The 1955 proposal for the Dartmouth summer research project, which gave the field its name, proceeds “on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.” Five years earlier, Turing had set aside the question of whether machines can think as too meaningless to discuss and substituted a test of whether one could pass for a person in conversation. Both take human intelligence as the reference point, and both were written before anyone knew how to build any of it.

Weizenbaum produced the first evidence that the resemblance is easy to fake. His 1966 program ELIZA matched patterns in a user’s sentences and reflected them back as questions, and the paper reports that subjects were hard to convince it was not human. He spent the rest of his career on what that meant. Computer Power and Human Reason (1976) recounts people confiding in the program and asking to be left alone with it, and argues that the readiness to attribute understanding tells you about the human, and that some judgments should not be handed to machines whatever they can do.

The resemblance is not only in the reader. Schrimpf and colleagues report in PNAS that transformer models predict close to 100 percent of the explainable variance in human brain responses to sentences, and that a model’s fit to the brain tracks how well it predicts the next word. Goldstein and colleagues, recording directly from the cortex of nine people listening to a podcast find computational principles shared between deep language models and the brain, with next-word prediction the point of contact. Neither result shows that the model understands anything. Both report that next-word prediction, the mechanism Bender calls mere form, is what tracks these brain measurements most closely.

Shanahan argues that a conversational agent creates an illusion of being in the presence of a thinking creature while being fundamentally not like us, and that words like “knows” and “believes” should be used with care. Placani argues that anthropomorphism works as both hype and fallacy, inflating what the systems can do and distorting judgments about responsibility and trust. The vocabulary carries the assumption in either direction, and “hallucination” is the clearest case, naming a statistical error with a word for human perception.

Artificial general intelligence

AGI names the endpoint, and the labs building toward it do not define it the same way. OpenAI’s charter sets an economic threshold, “highly autonomous systems that outperform humans at most economically valuable work.” Google DeepMind’s Levels of AGI replaces the single line with a grid of six performance tiers crossed with breadth, so a system can be superhuman at one thing and absent at everything else. Chollet defines intelligence as skill-acquisition efficiency on unknown tasks and built the ARC-AGI benchmark to measure that instead of accumulated knowledge.

That benchmark has since recorded a fast climb and a wall. OpenAI’s o3 scored 87.5 percent on ARC-AGI-1 in December 2024, at about 4,560 dollars per task, passing a bar that people clear routinely and cheaply. On the harder ARC-AGI-2, the 2025 results put the best verified commercial system at 37.6 percent and the competition winner at 24.03 percent, with the 85 percent grand prize unclaimed.

The people building these systems disagree about the timeline by a wide margin. A survey of 2,778 researchers who had published in top AI venues put the median 50 percent chance of machines outperforming humans at every task at 2047, thirteen years earlier than the same survey’s answer one year before, while the median date for the full automation of all human occupations came out at 2116, 69 years later, from the same respondents.