Sociotechnical protocol

Seventeen questions in three sets. Side A applies to any technology, Side B to any claimed success, and Side M to a technology an organization has no choice about. The protocol is used in every week of the course and is part of the rubric for every major assignment.

Behind any automated decision there are human decisions about what to build, whose data to use, and who absorbs the cost of an error. “The algorithm did it” names none of them, and the questions below are written to find them.

Use

Run Side A on everything, including the successes. Run Side B on everything, including the failures, because a failure’s Side B shows what was missing. Add Side M when a funder or a law requires the deployment.

Each question has a label, so a memo, a slide, or a rubric comment can point at A5 or B3 without quoting it in full. Questions that cannot be answered are findings as well, and the record should say what would have to be asked, and of whom.

You can fill the protocol in online and copy the assembled result into your notes; nothing you type there is saved.

Side A, questions for any technology

Origin

• A1. Who designed this, and who was in the room when the problem was defined?
• A2. Whose problem does it solve, and whose problem does it claim to solve? These are often different, and the difference should be stated plainly.

Inputs

• A3. Whose data and whose labor make it run?
• A4. What does it assume about its users, and who breaks those assumptions? A benefits portal that assumes a smartphone, a fixed address, and fluent English has already chosen its users.

Consequences

• A5. Who bears the risk when it fails, and who does the work of appealing? In MiDAS, the state made the error and the claimant did the paperwork.
• A6. What social arrangement does it create or reinforce, and how hard would that be to undo?
• A7. What did it replace, and what was lost in the replacement?

Side B, Conditions of success

Success stories receive the same rigor as failures. Admiring a win without understanding its conditions is how bad replications happen.

Cause

• B1. What made this work: the tool, the institution, or the people it amplified?
• B2. Whose participation shaped the design, and at what stage? Advising at the end is not the same as deciding at the start.

Evidence and safeguards

• B3. Which guardrails were built in before deployment, and which were added after failure?
• B4. Who verified the claims of success, and what would falsify them?

Durability

• B5. Would this replicate without its champions, its funding model, or its political moment?
• B6. Who maintains this in year five?

Side M, mandated deployments

Some technologies arrive as a condition of funding or law: the Homeless Management Information System (HMIS) for HUD homeless-assistance grantees, electronic visit verification for Medicaid personal care services. Refusal is not among the options, and the protocol still earns its time, because its answers locate the terms that remain negotiable.

• M1. Which terms of this deployment are negotiable, and who is present when they are set? The mandate usually sets a floor; the software, the configuration, and any data collected beyond the required minimum are often decided locally.
• M2. What is monitored after launch, and who sees the numbers?
• M3. Who can override the system’s output, and does the override stand?
• M4. What would trigger rollback or exit, and is that written down where the agency can point to it?

Self-check

Each item is a fact from a case this course covers, and the answer is the protocol question it belongs to. The drill is the one that makes the protocol usable under time: hearing which question a piece of evidence answers, rather than reading seventeen of them in order. Nothing is graded, and answers remain in this browser until you press reset.

Each item takes at least one question, and one takes two. Select every question that applies, then press Check for that item. An item cannot be changed once it is checked.