Language models in depth

Week 4 covers what a language model does and why it fabricates. The mechanism has a longer history than a single session can cover. These models descend from a research field with its own decades of failure, they acquired several of their abilities only at scale, they have been surrounded since 2023 by retrieval and tool use that is not the model itself, and the question of whether any of it resembles human intelligence remains unsettled among the people who build them. None of it is required for the Week 4 lab.

Deep learning

Language models are one branch of a field that predates them by decades. The “deep” in deep learning refers to stacked layers of simple units, each layer learning a representation built out of the one below it, with the connection weights set by gradient descent against a training objective. LeCun, Bengio, and Hinton’s 2015 review in Nature describes the method and its history.

A network called AlexNet won the 2012 ImageNet image-recognition contest with a 15.3 percent top-five error rate against 26.2 percent for the runner-up, and the field turned toward deep networks after it. Image recognition, speech recognition, and protein structure prediction are built with deep networks too, which is one reason Bender restricts the stochastic-parrots critique to text generation. Bengio, Hinton, and LeCun received the 2018 ACM Turing Award, and Hopfield and Hinton received the 2024 Nobel Prize in Physics for “foundational discoveries and inventions that enable machine learning with artificial neural networks”.

Scale and emergent abilities

Next-token prediction sounds too thin to produce what these systems do, and the earliest evidence that it is not came in 2017, when an OpenAI model trained only to predict the next character of product reviews proved to have become a strong sentiment classifier on its own. Nothing in its training asked for sentiment; tracking it merely helped predict the next character.

Scale then complicated the picture. As models grew from GPT-3’s 175 billion parameters toward PaLM’s 540 billion, some capabilities appeared abruptly instead of improving smoothly: identifying a movie from a string of emoji, for example, and about 5 percent of tasks in the BIG-bench evaluation suite show that kind of sharp jump. The training compute behind the jumps is hard to picture; running the training of the largest of these models on a machine doing a billion operations per second would take well over 100 million years. Researchers disagree over what the jumps are; some read them as genuinely new capability, others as statistical pattern-matching crossing a threshold that makes it visible.

Growth in these systems has been regular enough to be written down as a formula. Kaplan and colleagues showed in 2020 that prediction error falls as a power law in model size, training data, and compute across more than seven orders of magnitude, which turned “make it bigger” into a budgeted engineering decision. DeepMind’s 2022 Chinchilla paper corrected the recipe by showing that model size and training data should grow together, and that a 70-billion-parameter model trained on 1.4 trillion tokens beat the 280-billion-parameter Gopher trained on the same compute budget. A smaller model trained on more text can beat a bigger one on the same budget.

Very few organizations have the budgets that buy this scale. Stanford’s AI Index priced the whole arc: the 2017 transformer that introduced the architecture cost around 900 dollars to train, GPT-4 cost an estimated 78 million, and Google’s Gemini Ultra an estimated 191 million, so Gemini Ultra’s training cost more than two hundred thousand times what the architecture’s first model cost seven years earlier. Epoch AI, working from a different accounting method, puts GPT-4’s final training run nearer 40 million dollars and finds the cost of the largest runs growing about 2.4 times a year since 2016, with a billion-dollar run arriving before 2027. The two estimates differ because one prices rented compute and the other amortizes hardware, and the gap between them is a good reminder to ask what any headline AI number is counting.

Retrieval and reasoning

The bare mechanism is not what anyone meets in a product. Two additions come between the model and the user, and both are sold as fixes for the problems the mechanism creates.

Retrieval-augmented generation, introduced by Lewis and colleagues in 2020, searches a document collection first and hands the model the retrieved passages to answer from, so the answer is grounded in real sources instead of the blur. Legal vendors market this as the end of fabricated citations. A preregistered Stanford evaluation of the leading products found that Lexis+ AI and Thomson Reuters’s Westlaw AI-Assisted Research and Ask Practical Law AI “each hallucinate between 17% and 33% of the time.” Retrieval lowers the rate and does not remove it, because the model still writes the sentence.

Reasoning models are the second addition. Chain-of-thought prompting showed in 2022 that a model asked to reason in steps solves more problems than one asked for the answer.

The technique arrived as a way of writing prompts and then became training. OpenAI’s o1, in September 2024, trained the behavior in, spending more computation before replying. The visible steps are not a record of the computation. Anthropic’s 2025 study gave models a hint, confirmed they used it, and found Claude 3.7 Sonnet mentioned the hint in its reasoning only 25 percent of the time and DeepSeek R1 only 39 percent. The explanation a model offers for its own decision is more generated text, produced by the same mechanism, and it can omit the factors that produced the answer.

Agents

The systems arriving in agencies now produce more than text. An agentic system calls other software and takes many steps without checking back, so its output is a completed action such as a booking, a filing, or a sent message. Anthropic released the Model Context Protocol in November 2024 as a standard way to plug models into other software, and OpenAI’s Operator began clicking through websites on a user’s behalf in January 2025.

Capability here is measured in task length. METR’s 2025 study timed how long a human expert takes on tasks agents can finish with 50 percent reliability, and found that horizon had doubled about every seven months across the six years to 2025. METR maintains the chart, and fitted to data from 2023 onward it now reports a doubling time of 124 days.

Deployment is going worse than the capability curve suggests. Gartner predicts more than 40 percent of agentic AI projects will be canceled by the end of 2027 on cost, unclear value, or missing risk controls, and estimates that only about 130 of the thousands of vendors selling agentic AI are selling anything agentic, a practice it calls agent washing.

The question of who answers for an agent’s actions has already been tested. In Moffatt v. Air Canada, a British Columbia tribunal ruled in February 2024 that Air Canada owed a customer the bereavement fare its website chatbot had described, and rejected the airline’s argument that the chatbot was a separate entity responsible for its own information. The damages came to about 650 Canadian dollars, and the ruling holds an organization answerable for what its chatbot tells a client, which covers a benefits eligibility question as much as a fare question.

Machine intelligence

The premise that these systems imitate human intelligence goes back to the field’s founding document. The 1955 proposal for the Dartmouth summer research project, which gave the field its name, proceeds “on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.” Five years earlier, Turing had dismissed the question of whether machines can think as too meaningless to discuss and substituted a test of whether one could pass for a person in conversation. Both take human intelligence as the reference point, and both were written before anyone knew how to build any of it.

Weizenbaum produced the first evidence that the resemblance is easy to fake. His 1966 program ELIZA matched patterns in a user’s sentences and reflected them back as questions, and the paper reports that subjects were hard to convince it was not human. He spent the rest of his career on what that meant. Computer Power and Human Reason (1976) recounts people confiding in the program and asking to be left alone with it, and argues that the readiness to attribute understanding is evidence about the human, and that some judgments should not be handed to machines whatever they can do.

The resemblance is not only in the reader. Schrimpf and colleagues report in PNAS that transformer models predict close to 100 percent of the explainable variance in human brain responses to sentences, and that a model’s fit to the brain tracks how well it predicts the next word. Goldstein and colleagues, recording directly from the cortex of nine people listening to a podcast find computational principles shared between deep language models and the brain, with next-word prediction the point of contact. Neither result shows that the model understands anything. Both report that next-word prediction, the mechanism Bender calls mere form, is what tracks these brain measurements most closely.

Shanahan argues that a conversational agent creates an illusion of being in the presence of a thinking creature while being fundamentally not like us, and that words like “knows” and “believes” should be used with care. Placani argues that anthropomorphism works as both hype and fallacy, inflating what the systems can do and distorting judgments about responsibility and trust. The vocabulary embeds the assumption in either direction, and “hallucination” is one such case, naming a statistical error with a word for human perception.

Consciousness

Whether a model understands and whether one could be conscious are different questions, and the second is the one a student is asked at a family dinner. It became a public argument through a Google engineer’s 2022 claim that the LaMDA chatbot was sentient, which the company called wholly unfounded and which cost him his job a month later.

Chalmers works the question philosophically and finds current models short on several conditions that most theories of consciousness require: recurrent processing, a global workspace, and unified agency. His conclusion is narrow in both directions. Present models are probably not conscious, and the obstacles he names are engineering properties rather than laws, so successors could satisfy them within a decade.

Butlin, Long and eighteen co-authors take the empirical route, deriving indicator properties from the leading neuroscientific theories, stating them in computational terms, and assessing existing systems against them. Their own caution is the part worth keeping: satisfying the indicators would not establish that a system is conscious, and no current system satisfies many of them.

For this field the question is practical before it is metaphysical. A person in front of a worker may already treat a chatbot as something that knows them, which is what Week 4 reports about teenagers and companion apps, and what Weizenbaum found in 1966. Nothing in the consciousness literature resolves what a worker should say to that person, and a practitioner who waits for it to be resolved has no answer in the meantime.

Artificial general intelligence

AGI names the endpoint, and the labs building toward it do not define it the same way. OpenAI’s charter sets an economic threshold, “highly autonomous systems that outperform humans at most economically valuable work.” Google DeepMind’s Levels of AGI replaces the single line with a grid of six performance tiers crossed with breadth, so a system can be superhuman at one task and absent at everything else. Chollet defines intelligence as skill-acquisition efficiency on unknown tasks and built the ARC-AGI benchmark to measure that instead of accumulated knowledge.

That benchmark has since recorded a fast climb and a wall. OpenAI’s o3 scored 87.5 percent on ARC-AGI-1 in December 2024, at about 4,560 dollars per task, passing a bar that people clear routinely and cheaply. On the harder ARC-AGI-2, the 2025 results put the best verified commercial system at 37.6 percent and the competition winner at 24.03 percent, with the 85 percent grand prize unclaimed.

The people building these systems disagree about the timeline by a wide margin. A survey of 2,778 researchers who had published in top AI venues put the median 50 percent chance of machines outperforming humans at every task at 2047, thirteen years earlier than the same survey’s answer one year before, while the median date for the full automation of all human occupations was 2116, 69 years later, from the same respondents.