Y12W38VC Machines making decisions about people

In 2016, journalists examined a computer system used in American courts to predict which defendants were likely to commit future crimes. Judges were using its predictions to inform sentencing, bail, and parole. What the journalists found was that the system's errors fell unevenly across racial groups. This week's article examines what happens when machines make decisions about people — and what fairness in such decisions turns out to require.

Core Vocabulary

algorithmic

/ˌæl.ɡəˈrɪð.mɪk/|al.go.rith.mic

adjective

Relating to or produced by an algorithm; following a prescribed step-by-step computational procedure or set of rules. Algorithmic systems make decisions by applying predetermined logical operations.

Word Breakdown: algorithm + -ic (relating to)

Word family: algorithm (n.), algorithmically (adv.)

Synonyms: computational, procedural, systematic

Collocations: algorithmic bias, algorithmic fairness, algorithmic decision

Example: Algorithmic systems in criminal justice make predictions about defendants based on historical data and patterns.

In the articleThe algorithmic system used historical data to predict recidivism rates.

disproportionately

/dɪsˈprɔːr.ʃə.nət.li/|dis.pro.por.tion.ate.ly

adverb

Out of proper proportion; in a way that is not fair, balanced, or equal. Disproportionate effects occur when harm or benefit falls unequally on groups.

Word Breakdown: dis- (not) + proportionately (fairly)

Word family: disproportionate (n.)

Synonyms: unequally, unfairly, unbalancedly

Collocations: disproportionately affect, disproportionately impact, disproportionate error rates

Example: The algorithm's errors fell disproportionately on certain racial groups, meaning false predictions were more common for some groups than others.

In the articleThe system's errors fell unevenly across racial groups.

fairness

/ˈfɛər.nəs/|fair.ness

noun

Impartial and just treatment; equity in decision-making and distribution of outcomes. In algorithmic contexts, fairness means equal treatment or equal error rates across groups.

Word Breakdown: fair + -ness (quality of)

Word family: fair (n.), fairly (adv.)

Synonyms: equity, justice, impartiality

Collocations: algorithmic fairness, fairness criterion, fairness across groups

Example: Algorithmic fairness requires that error rates be equivalent across racial and demographic groups, not just that the algorithm seems impartial.

In the articleWhat fairness in such decisions turns out to require.

bias

/ˈbaɪ.əs/|bi.as

noun

A systematic distortion or prejudice in favor of or against a particular group, outcome, or interpretation. Bias can be intentional or unintentional, and it skews decisions or data analysis.

Word family: biased (n.), biasing (n.)

Synonyms: prejudice, distortion, skew

Collocations: algorithmic bias, racial bias, gender bias

Example: The algorithm exhibited bias because it made systematically different error rates for different racial groups.

In the articleThe system's errors revealed underlying bias in historical data and criminal justice records.

calibrated

/ˈkæl.ɪ.breɪ.tɪd/|cal.i.brat.ed

adjective

Adjusted or tuned to precise standards; in algorithmic contexts, calibrated means that predictions are accurate across different groups. Calibration ensures that a score of '60' means the same probability regardless of group membership.

Word Breakdown: calibr- (adjust) + -ated (made to be)

Word family: calibrate (n.), calibration (n.)

Synonyms: adjusted, tuned, precise

Collocations: calibrated to accuracy, calibrated algorithm, properly calibrated

Example: A calibrated algorithm would predict the same actual recidivism probability for all groups receiving the same score.

In the articleCalibration is one fairness criterion but achieves different error rates across groups.

automated

/ˈɔː.tə.meɪ.tɪd/|au.to.mat.ed

adjective

Operated or controlled by machine without human intervention; performed automatically according to a program or system. Automated systems replace human decision-making with algorithmic procedures.

Word Breakdown: auto- (self) + mated (made)

Word family: automate (n.), automation (n.)

Synonyms: mechanical, programmed, automatic

Collocations: automated decision, automated system, automated bias

Example: Automated systems in criminal justice raise concerns because once implemented, they apply decisions consistently without human discretion.

In the articleThe automated system made decisions without human oversight.

transparent

/trænsˈpɛər.ənt/|trans.par.ent

adjective

Open to inspection and understanding; allowing visibility of how something works. Transparent algorithms reveal their logic and reasoning, not operating as black boxes.

Word Breakdown: trans- (through) + parent (visible)

Word family: transparency (n.), transparently (adv.)

Synonyms: clear, open, understandable

Collocations: algorithmic transparency, transparent system, lack of transparency

Example: A transparent algorithm would explain which factors influenced its prediction and why.

In the articleTransparency is essential for accountability in algorithmic decision-making.

explanation

/ˌɛk.spləˈneɪ.ʃən/|ex.pla.na.tion

noun

A statement or account of how or why something happened or works. In algorithmic contexts, explanations provide understanding of decision logic and the factors influencing outcomes.

Word Breakdown: ex- (out) + -planation (flattening/making clear)

Word family: explain (n.), explanatory (n.)

Synonyms: account, reasoning, justification

Collocations: explanation for bias, lack of explanation, provide explanation

Example: Without explanation, defendants cannot challenge algorithmic predictions or understand why they received a particular score.

In the articleCourts should require explanation of algorithmic decisions.

Technical Terms

algorithmic fairness

/ˌæl.ɡəˈrɪð.mɪk ˈfɛər.nəs/|al.go.rith.mic.fair.ness

noun

The field of study focused on how to design and evaluate automated decision systems that treat groups equitably. Algorithmic fairness research grapples with competing definitions of what makes an algorithm fair.

Synonyms: fair algorithms, equitable automation

Collocations: algorithmic fairness research, algorithmic fairness criterion, achieve algorithmic fairness

Example: Algorithmic fairness researchers debate whether fairness means equal error rates or equal accuracy predictions across groups.

In the articleThe article examines what algorithmic fairness requires in practice.

COMPAS

/ˈkɒm.pæs/|COM.PAS

noun

A controversial risk-assessment algorithm used in American courts to predict defendants' likelihood of future criminal behaviour. Journalists revealed that COMPAS made different types of errors for different racial groups.

Synonyms: risk assessment algorithm, recidivism prediction tool

Collocations: COMPAS algorithm, COMPAS bias, COMPAS scores

Example: COMPAS became the subject of intense scrutiny when research revealed its differential error rates across racial groups.

In the articleThe article discusses findings about COMPAS and similar algorithmic decision systems.

equalised odds

/ˈiː.kwə.laɪzd ɒdz/|e.qual.ised.odds

noun

A fairness criterion requiring that error rates be equal across groups; if an algorithm has a 10% false positive rate for one group, it should have a 10% false positive rate for all groups. Equalised odds ensure that algorithmic mistakes affect all groups equally.

Synonyms: equal error rates, equitable error distribution

Collocations: equalised odds criterion, achieve equalised odds

Example: Equalised odds is one definition of fairness: an algorithm is fair if it makes mistakes at the same rate for all racial groups.

In the articleDifferent fairness criteria cannot all be satisfied simultaneously.

calibration

/ˌkæl.ɪˈbreɪ.ʃən/|cal.i.bra.tion

noun

A fairness criterion requiring that a prediction score means the same probability of the outcome across all groups; a score of '60' indicates a 60% probability of recidivism regardless of the defendant's race. Calibration ensures predictive accuracy but may produce different error rates.

Word family: calibrated, calibrate

Synonyms: prediction accuracy, probability alignment

Collocations: calibration criterion, achieve calibration

Example: Calibration requires that a risk score accurately predict the actual probability of reoffending, but may result in unequal error rates across groups.

In the articleCalibration is a fairness criterion but conflicts with equalised odds.

impossibility theorem

/ɪmˌpɒs.ə.ˈbɪl.ɪ.ti ˈθɪə.rəm/|im.pos.si.bil.i.ty.the.o.rem

noun

A mathematical result showing that different fairness criteria cannot all be satisfied simultaneously. If an algorithm achieves equalised odds (equal error rates), it cannot achieve calibration (equal prediction accuracy), and vice versa.

Synonyms: fairness trade-off, incompatible fairness criteria

Collocations: impossibility theorem shows, fairness impossibility theorem

Example: The impossibility theorem reveals a fundamental tension: no single algorithm can satisfy all competing definitions of fairness at once.

In the articleThe article discusses how different fairness definitions conflict with each other.

Figurative Phrases

black box

An opaque system whose internal workings are invisible or incomprehensible to users. The phrase comes from aviation but is now used for any system that cannot be inspected or understood.

Etymology/Type: idiomatic

Synonyms: opaque system, hidden mechanism

Example: Algorithmic systems that offer no explanation for their decisions function as a black box, preventing defendants from challenging predictions.

In the articleAlgorithmic systems should not operate as a black box; their logic must be transparent.

the hand of the machine

Automated agency or control; the implication that a machine, not a person, made a decision. The phrase uses 'hand' figuratively to represent agency.

Etymology/Type: idiomatic

Synonyms: machine decision, algorithmic agency

Example: Defendants blame the hand of the machine rather than human judges, but the algorithm was designed by humans using biased data.

In the articleThe article examines responsibility for algorithmic decisions.

tip the scales

To bias an outcome; to influence a decision unfairly in one direction. The phrase uses literal scales metaphorically.

Etymology/Type: idiomatic

Synonyms: bias an outcome, skew a decision

Example: Biased training data tips the scales against certain groups by ensuring the algorithm makes systematically unfair predictions.

In the articleBias in data can tip the scales of algorithmic decision-making.

rubber-stamp

To approve something without scrutiny or genuine examination; to give automatic approval. The phrase comes from the bureaucratic practice of stamping documents.

Etymology/Type: idiomatic

Synonyms: automatic approval, passive acceptance

Example: Judges should not rubber-stamp algorithmic recommendations without examining their fairness and accuracy.

In the articleCourts cannot simply rubber-stamp algorithmic decisions.

play the numbers

To reason with or rely on statistics; to use numerical data to support an argument. The phrase suggests calculation and strategy.

Etymology/Type: idiomatic

Synonyms: use statistics, rely on data

Example: When courts play the numbers with algorithmic predictions, they may miss important individual circumstances that the algorithm overlooked.

In the articleAlgorithmic systems play the numbers but may miss human context.

the system is the problem

A diagnosis that structural or systemic issues, not individual failings, are responsible for a problem. The phrase treats 'system' figuratively.

Etymology/Type: idiomatic

Synonyms: structural bias, systemic failure

Example: If algorithmic bias reflects historical inequity in criminal justice, then the system is the problem: feeding biased data to an algorithm produces a biased algorithm.

In the articleAlgorithmic bias reveals that the underlying system may be the problem.

Confusing Words

disproportionately vs. inordinately

These near-synonyms both mean excessively, but disproportionately emphasises imbalance relative to other groups, while inordinately emphasises excess beyond normal limits.

  • disproportionately [out of proper proportion; affecting some groups unequally compared to others] — The algorithm's errors fell disproportionately on certain racial groups — false predictions were far more common for some groups than others.
  • inordinately [excessively; far beyond what is normal or reasonable] — The impact on defendants was inordinately harsh because the algorithm provided no explanation for its predictions.

Disproportionately = unequally relative to another group (comparison-based); Inordinately = excessively beyond normal (absolute excess). Ask: Is this about unequal distribution (disproportionately) or about being too much (inordinately)?

bias vs. prejudice

These near-synonyms both involve skewed thinking, but bias is systematic distortion while prejudice is pre-formed judgment often with a social element.

  • bias [a systematic distortion or skew that produces consistently unequal outcomes] — Algorithmic bias occurs when a system makes systematically different errors for different groups — bias can be unintentional.
  • prejudice [a pre-formed negative judgment about a group, often based on social stereotypes] — Historical prejudice in criminal justice created biased training data that led to algorithmic bias.

Bias = systematic distortion (can be unintentional, technical); Prejudice = pre-judgment (typically intentional, social, negative). Ask: Is this about systematic skew (bias) or pre-formed judgment (prejudice)?

transparent vs. translucent

These near-synonyms both involve visibility, but transparent means fully see-through while translucent means partly see-through.

  • transparent [completely open to inspection and understanding; allowing full visibility] — A transparent algorithm would fully explain its reasoning and allow inspection of how it reaches decisions.
  • translucent [allowing light or some visibility through, but not fully clear] — A translucent algorithm might reveal some information but still operate partially as a black box.

Transparent = fully see-through, complete clarity (no hidden parts); Translucent = partly see-through, partial clarity (some opacity remains). Ask: Is this about complete openness (transparent) or partial visibility (translucent)?