Kinds of AI systems

Week 1 asks the class to define “AI,” Week 4 dismantles a generative language model, and Week 3 runs the fairness math on a risk score. Those are three different machines, met weeks apart. None of what follows is required for a lab. It exists so that a claim about “AI” can be met with a question about which kind, since the kind decides how the system works and how it fails.

Umbrella term

The word “AI” covers systems that share almost nothing in how they work. Arvind Narayanan and Sayash Kapoor, whose AI Snake Oil the course reads in Week 4, put the problem in one line: using “AI” as a single label is as problematic “as using ‘vehicle’ without differentiating between bikes, cars and spaceships.” A chatbot and the software a county uses to score a child-welfare referral are both called AI, and in how they are built, what they are used for, and how they go wrong they have little in common.

Origin of the term

The name was a deliberate choice. John McCarthy, with Marvin Minsky, Nathaniel Rochester, and Claude Shannon, put “artificial intelligence” in the title of a 1955 proposal for a summer workshop at Dartmouth the following year. The proposal asked for “a 2 month, 10 man study” and set an aim that still sounds ambitious today: to find how to make machines “use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves.” Its working definition of the problem was circular, “making a machine behave in ways that would be called intelligent if a human were so behaving,” which puts human behavior at the center of the standard and leaves the word “intelligent” undefined.

McCarthy later said the name resolved a matter of territory as much as of science. “One reason for inventing the term ‘artificial intelligence’ was to escape association with cybernetics,” he wrote, wanting to avoid “having either to accept Norbert Wiener as a guru or having to argue with him” (Nilsson, 2010). Not everyone in the room liked the result. The Stanford Encyclopedia of Philosophy records that the term was “disliked by some of the attendees” and that a consensus definition of it may be “eternally unanswerable.”

It was one branding choice among several that were live at the time. Arthur Samuel called his checkers-playing program a study in “machine learning” in 1959, Donald Michie ran “machine intelligence” workshops in Britain, and cybernetics was the term McCarthy was steering around. Of these, “artificial intelligence” was the most anthropomorphic, and it invites a comparison to the human mind that the systems do not earn.

The word has remained a moving target since. Larry Tesler’s own formulation was “Intelligence is whatever machines haven’t done yet”, usually misquoted as “AI is whatever hasn’t been done yet”; either way, the effect it names is that a capability stops being called AI once it works and becomes ordinary software, which confines the word to whatever is still unproven. The label has also been turned back on the human labor it can hide: Amazon named its crowd-work marketplace Mechanical Turk “artificial artificial intelligence,” work that looks automated but runs on people paid by the task. Emily Bender and Alex Hanna argue in The AI Con that the term now functions largely as marketing, and the label has a market value that regulators have begun to police. A 2019 review of 2,830 European companies presented as “AI startups” found evidence that AI was material to the value proposition in about 60 percent of them, with the tag often applied by investors and the press rather than the firms. The familiar “40 percent of AI startups use no AI” headline is the complement of that figure, restated by the press rather than written in the report. By 2024 the pattern had a name, “AI washing,” and the Securities and Exchange Commission was fining investment advisers for it, charging two firms 400,000 dollars for false claims about using AI; the Federal Trade Commission announced five enforcement actions over deceptive AI claims in September 2024.

After ChatGPT

The confusion is recent. Before late 2022, among the systems called “AI,” the one a social-services worker was most likely to meet was predictive: a risk score on a child-welfare referral, a fraud flag on a benefits claim, an eligibility rule. The Allegheny tool had been scoring referrals since 2016, COMPAS longer than that, and Michigan had already run MiDAS through tens of thousands of families. Then ChatGPT launched in November 2022 and reached about 100 million monthly users within two months, the fastest ramp analysts could recall for a consumer app. Generative AI adoption went on to spread faster than the personal computer or the internet at the same point in their histories, and ChatGPT passed 800 million weekly users by October 2025. The predictive systems named above predate that surge by a decade or more.

Structured and unstructured data

Two questions separate the families, and both come from the OECD’s framework for classifying AI systems, a government-grade typology written for non-specialists: what kind of data the system runs on, and what kind of task it performs.

The first question is whether the input is structured or unstructured. Structured data is rows and columns: a person’s age, income, prior contacts with an agency, months since a last claim. Unstructured data is free text, images, and audio, which have no columns until a model imposes some. The OECD framework lists the options plainly: unstructured, semi-structured, structured. A risk score reads a table and a language model reads text, which is already enough to put them in different families.

Prediction and generation

The second question is what the model does with the data. A discriminative model predicts a label, a score, or a rank for the input it is given: fraud or not fraud, high risk or low, this face or that one. A generative model produces new output that was not in the input: a paragraph, an image, a line of code. This is a technical distinction, stated formally by Ng and Jordan in 2001: a discriminative model estimates the probability of a label given the data, while a generative model models the joint distribution of the data itself. Because it represents that distribution, a generative model can also be sampled to produce new examples, which is what makes generation possible. Next-token prediction is a generative task; scoring a referral is a discriminative one, even when both are built from neural networks. The OECD framework makes the same cut, asking of any system whether the model is “discriminative” or “generative.”

Much of what is called AI on the discriminative side is continuous with statistics a practitioner may already have used. A predictive risk model is often logistic regression or a near relative, the same method that estimates the odds of an outcome from a set of variables, scaled up with more data and the more flexible machine-learning and deep-learning algorithms built on top of it. The scale and flexibility changed; the core move, fitting a model to past data to predict a label, has been in the statistics toolkit for decades. Ng and Jordan’s example of a discriminative model is logistic regression itself.

The distinction also governs which evidence transfers. Two studies on models of roughly the same size point in opposite directions because the task differs. Running Llama 3, Gemma 2, and Mistral Nemo on a sixteen-gigabyte laptop against thirty real outpatient cases, blinded consultants judged the models’ diagnostic and treatment recommendations potentially unsafe in 32 to 54 percent of responses, against 1 percent for the physicians. Asked instead to identify constructs already present in a written record, models reached agreement with expert coders above 0.93. A result about generation says little about classification. Two cautions on the clinical study: the cases were in German and some models answered in English, so it is partly a result about performance outside English, and the rated clinicians and the rating consultants worked at the same institution, which the authors note may favor the humans.

Three families

The course meets three of these families. Predictive risk modeling scores rows and columns, NLP and computer vision classify text and images, and generative AI writes new text and makes new images. Each one fails in ways the other two do not, so the word AI on its own says almost nothing about what an agency has bought.

Predictive risk modeling disappears from view once “AI” means a chatbot, and it is the family social services deployed first, years before ChatGPT. Generative AI has a session on how language models work and a longer resource behind it, and it comes back in the AI-literacy, prompting, and adoption weeks. The Allegheny tool has scored child-neglect referrals since 2016 from birth, Medicaid, and justice records; it has no generative component and never touched a language model, and its problems, bias and questionable calibration, are not the problems a chatbot has, so a fix for one is not a fix for the other.

Rules without a model

A fourth kind has no learned model in it at all, and it causes more of the public confusion than the other three together. Many systems that news coverage calls an “algorithm” or “AI” are hand-written if-then rules, what the OECD framework files under symbolic or human-written-rules systems, with no statistical learning in them. The distinction matters for accountability, because rules can be audited by reading the code, while a learned model usually cannot.

Michigan’s MiDAS is the local, cautionary case, and an unresolved one. Reporting describes it operationally as flagging any data discrepancy as fraud, the pattern of a rules engine, and the state agency never publicly disclosed the exact architecture. Whether MiDAS was a trained model or a rule set has not been established in public documentation. Neither journalists nor courts could obtain a clear answer about which family this deployed government system belonged to, and that opacity is the accountability gap the course’s protocol is built to expose. Week 1 already reads MiDAS as rules-based automation of a kind decades old.

Types of AI models

Inside the generative family, two further distinctions decide what an organization can run and on whose terms: how large the model is, and how much of it anyone outside the vendor can see. Both of those are procurement questions that arrive before any technical one does.

Size

There is no agreed threshold for a small model; published definitions run from under one billion parameters to twenty. Size sets the hardware, the running cost, and whether the model can run inside the building at all, which is the same question 42 CFR 2.12 asks of any vendor that would receive substance use disorder records. Larger is not automatically better for a given job. Evaluating seven model sizes from 0.6 to 32 billion parameters against four benchmarks built from child welfare investigation summaries, Qi, Perron, and colleagues found a four-billion-parameter model with extended reasoning outperforming models eight times larger. In a companion study a twenty-billion-parameter model running entirely on local hardware classified substance categories in the same kind of record at 0.94 to 1.00 agreement, both studies from the University of Michigan School of Social Work.

Openness

The Open Source Initiative’s Open Source AI Definition, published in October 2024, requires the weights, the training code, and a description of the training data, but not the training data itself, and critics including the Software Freedom Conservancy argue that this fails to require reproducibility and devalues the term. Licenses that call themselves open often are not: Meta’s Llama license requires a separate grant from Meta above 700 million monthly active users, and the OSI has stated that the Llama license is not open source. Stanford’s Foundation Model Transparency Index found the average score across major developers fell from 58 out of 100 in 2024 to 40 in 2025, with training data the worst-disclosed category. A product page rarely says which of the two it means, so the question has to be asked directly.

Failure modes and regulation

Sorting a system into a family changes what to look for. The families have separate failure modes, and regulators are increasingly writing separate rules for each.

Predictive risk modeling fails mostly through the validity of its predictions and the bias in its historical labels. The label it predicts is usually a proxy: the Allegheny tool does not measure future maltreatment, it predicts whether a referral matches patterns in past agency actions, and, as the ACLU’s review of family-screening algorithms puts it, “being under arrest doesn’t necessarily mean you did something illegal.” Generative AI fails through hallucination, synthetic-content misuse, and unclear training-data provenance, the failures Week 4 covers. Those failures arise from a different mechanism than the ones described above.

Regulators have drawn the same line. The EU AI Act puts tools like AFST and COMPAS under its high-risk regime for systems used in essential services and law enforcement, and regulates general-purpose generative models through a separate chapter with its own rules. NIST issued a type-agnostic AI Risk Management Framework and then, because generative systems raised risks the base framework did not cover, a separate Generative AI Profile on top of it. That generative systems needed a separate profile is a sign the families do not share one risk model.

Inside a predictive risk model

Predictive risk modeling is worth taking apart, because the harm and the help both enter at specific stages of building it. Suresh and Guttag map where bias enters across the machine-learning life cycle; running their map over a tool like the Allegheny screening model, stage by stage:

• Defining the outcome. The model predicts a proxy, not harm. The Allegheny tool’s designers describe its output as the estimated probability that county agents will remove a child, not that a child will be harmed (Logic Magazine, 2023); COMPAS predicts re-arrest, and only about a fifth of those it flagged for a violent crime went on to commit one (ProPublica, 2016). This is the selective-labels problem: outcomes are recorded only for cases someone already acted on. Writing the judgment down makes it consistent, and it fixes a stand-in in place of the outcome of interest.
• Collecting the data. The model reads administrative records: birth, Medicaid, behavioral health, juvenile probation, prior benefits. That history contains more than a rushed intake worker can, and it is a record of contact with public systems, so it over-represents families who are poor, Black, or already known to them. An ACLU audit found the tool would label 33 percent of Black households high risk against 20 percent of others.
• Learning from past decisions. The model is fit to earlier human decisions, so it reproduces them at scale. The predictive-policing literature proves the runaway feedback loop formally, and Eubanks documents it across welfare and child welfare in Automating Inequality. A model can be audited where a caseworker’s private bias cannot, and it can also launder past enforcement into an objective-looking score.
• Setting the threshold. A continuous score becomes a decision at a cutoff, and the cutoff is a value choice. Kleinberg, Mullainathan, and Raghavan proved that calibration and equal error rates cannot all hold at once across groups with different base rates, and ProPublica found COMPAS wrongly flagged Black defendants at nearly twice the rate of white ones. The class computes this tradeoff by hand in the fairness explorer.
• Deploying it with a person. A worker is meant to catch the model’s mistakes, but Green’s survey of 41 oversight policies found people cannot perform that function, and Elish names the place blame is assigned the moral crumple zone. The same machinery can also be pointed at help: LA County’s Homelessness Prevention Unit routes high-risk people to case management and about 6,500 dollars in assistance, and early enrollees were 71 percent less likely to enter a shelter within 18 months.
• Reading the frame. The tabular risk model that flags a family for investigation is structurally the one that flags a person for assistance, and which deployment a person receives is decided upstream. Michigan’s MiDAS shows the punitive end, wrongly flagging tens of thousands on the assumption that every claimant was committing fraud, with 93 percent of reviewed determinations involving no fraud. Eubanks’ point is that the automation alone does not make a system a weapon against the poor; that comes from the institution deploying it.

Black box

A risk model built as logistic regression can at least be read: its weights say which variables push a score up and by how much. Most of the rest of what is called AI cannot be read that way, and the gap has widened as the models have grown.

Cynthia Rudin argues that for high-stakes decisions the field should use models that are interpretable from the start rather than fitting a black box and explaining it afterward, since explaining a black box “is likely to perpetuate bad practices and can potentially cause catastrophic harm to society.” Her working example is criminal-risk scoring, where a fully transparent rule list matched COMPAS’s accuracy on the same data, so the opacity bought nothing. Predictive risk models are at the readable end of this line, and the opacity problem is sharpest for deep neural networks.

Transformers and large language models made it worse. Their behavior is spread across billions of parameters, and a single unit can respond to many unrelated concepts at once, so there is no one neuron to point to. The attention weights the architecture exposes are not accounts of what the model did: Jain and Wallace found that they “largely do not” correspond to gradient-based measures of feature importance, though Wiegreffe and Pinter answered that this depends on what is meant by explanation, and the dispute is unsettled. Even a model that shows its work can misreport it: Anthropic found that when a model used a planted hint, Claude 3.7 Sonnet mentioned the hint only 25 percent of the time and DeepSeek R1 39 percent, so a stated chain of reasoning is not a reliable record of the real computation.

Two research programs address this, and they differ in where they intervene. Explainable AI, often shortened to XAI and the aim of efforts like DARPA’s XAI program, attaches an explanation to a model that is already a black box; post-hoc tools such as LIME and SHAP are its workhorses, and their explanations can be unfaithful to what the model actually computes, which is Rudin’s objection and the finding of sanity-check studies of saliency maps.

Interpretable machine learning takes the other route, building models that are transparent by design, and Rudin’s case is that high-stakes decisions should use those. A newer line, mechanistic interpretability, tries to reverse-engineer a trained network after the fact: Anthropic’s sparse-autoencoder work pulled millions of human-readable features out of a production model, while the authors caution that the method is incomplete and has no rigorous way yet to confirm that the features correspond to the model’s computation.

Where a model cannot explain itself, accountability shifts to process. Documentation standards such as model cards and datasheets for datasets ask builders to state what a system was trained on and tested for; external audits of commercial AI products have moved vendors to fix measured disparities; and law is starting to require transparency, though less than its reputation suggests, since the GDPR “right to explanation” is more contested than often claimed. The most common safeguard is a person placed over the system. As in the risk-model case, Green found that a human placed over an algorithm usually cannot oversee it, and the requirement mostly legitimates the tool.

A tabular risk model is often a logistic regression a person can inspect, and Rudin’s argument is that high-stakes decisions should remain that inspectable. A language model is a far deeper black box, so moving one into a consequential decision raises the accountability bar rather than resolving it.

These questions are not resolved inside social impact settings alone, and practitioners are thinly present where they are settled. Algorithmic fairness, interpretability, accountability, AI safety, and trustworthy AI are studied intensively in computer science and human-computer interaction, at venues like the ACM Conference on Fairness, Accountability, and Transparency (FAccT), the conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO), and the AAAI/ACM Conference on AI, Ethics, and Society (AIES). Practitioners encounter these systems where they are deployed, in child welfare, benefits, and the courts.

Self-check

Each item gives a definition and an example, and many of these scenarios combine more than one family. The feedback names the data type and the task that place each one. Nothing is graded, and answers remain in this browser until you press reset.

Each item takes at least one family, and several take two. Select every family that applies, then press Check for that item. An item cannot be changed once it is checked.

Common misconceptions

• “AI means ChatGPT.” ChatGPT is one family, generative models on unstructured data. Risk scorers, fraud flaggers, and facial recognition are different families that predate it by years and work on different principles.
• “If generative AI is unreliable, everything called AI is.” A risk score and a chatbot have different failure modes; the bias in a predictive model and the hallucination in a generative one come from different mechanisms, and one does not imply the other. IBM’s own comparison notes predictive estimates are comparatively more explainable than generative outputs.
• “The algorithm understands the case.” A discriminative model like AFST is fitting correlations in historical agency data; whatever it is doing, it has never met the child and contains nothing about them that is not in the record. A generative model is producing likely text, and whether that amounts to understanding is contested, which Week 4 argues rather than resolves. Neither one has a case, and a claim that it does should be refused.
• “A risk score predicts what will happen.” It predicts a proxy label drawn from past agency decisions, not the outcome of concern. The tool measures whether data patterns correlate with prior actions, not future harm.
• “More data makes a system more objective.” Training on more historical data can increase the bias already in it. The Allegheny tool draws on justice and disability-service records that over-represent Black families and people with mental-health histories, so more of that data deepens the pattern rather than correcting it.
• “If it is called an algorithm, it is a trained AI model.” Many deployed government “algorithms” are hand-written rules with no learning in them. Reading the code audits a rule set; it does not audit a learned model, and the two involve different accountability.

Note on the nesting diagram

The most common picture of AI is a set of nested circles, AI containing machine learning containing deep learning containing generative AI, as in IBM’s version. As a picture of technical lineage it is defensible, but it is a poor guide here, because it suggests generative AI is simply further along the same line as predictive modeling, when the two behave, fail, and are regulated differently. It shows where these methods came from, and the table above shows how they differ in use.