Y12W38VC Machines making decisions about people
In 2016, journalists examined a computer system used in American courts to predict which defendants were likely to commit future crimes. Judges were using its predictions to inform sentencing, bail, and parole. What the journalists found was that the system's errors fell unevenly across racial groups. This week's article examines what happens when machines make decisions about people — and what fairness in such decisions turns out to require.
Core Vocabulary
algorithmic
/ˌæl.ɡəˈrɪð.mɪk/|al.go.rith.mic
adjective
Relating to or produced by an algorithm; following a prescribed step-by-step computational procedure or set of rules. Algorithmic systems make decisions by applying predetermined logical operations.
Word Breakdown: algorithm + -ic (relating to)
Word family: algorithm (n.), algorithmically (adv.)
Synonyms: computational, procedural, systematic
Collocations: algorithmic bias, algorithmic fairness, algorithmic decision
Example: Algorithmic systems in criminal justice make predictions about defendants based on historical data and patterns.
disproportionately
/dɪsˈprɔːr.ʃə.nət.li/|dis.pro.por.tion.ate.ly
adverb
Out of proper proportion; in a way that is not fair, balanced, or equal. Disproportionate effects occur when harm or benefit falls unequally on groups.
Word Breakdown: dis- (not) + proportionately (fairly)
Word family: disproportionate (n.)
Synonyms: unequally, unfairly, unbalancedly
Collocations: disproportionately affect, disproportionately impact, disproportionate error rates
Example: The algorithm's errors fell disproportionately on certain racial groups, meaning false predictions were more common for some groups than others.
fairness
/ˈfɛər.nəs/|fair.ness
noun
Impartial and just treatment; equity in decision-making and distribution of outcomes. In algorithmic contexts, fairness means equal treatment or equal error rates across groups.
Word Breakdown: fair + -ness (quality of)
Word family: fair (n.), fairly (adv.)
Synonyms: equity, justice, impartiality
Collocations: algorithmic fairness, fairness criterion, fairness across groups
Example: Algorithmic fairness requires that error rates be equivalent across racial and demographic groups, not just that the algorithm seems impartial.
bias
/ˈbaɪ.əs/|bi.as
noun
A systematic distortion or prejudice in favor of or against a particular group, outcome, or interpretation. Bias can be intentional or unintentional, and it skews decisions or data analysis.
Word family: biased (n.), biasing (n.)
Synonyms: prejudice, distortion, skew
Collocations: algorithmic bias, racial bias, gender bias
Example: The algorithm exhibited bias because it made systematically different error rates for different racial groups.
calibrated
/ˈkæl.ɪ.breɪ.tɪd/|cal.i.brat.ed
adjective
Adjusted or tuned to precise standards; in algorithmic contexts, calibrated means that predictions are accurate across different groups. Calibration ensures that a score of '60' means the same probability regardless of group membership.
Word Breakdown: calibr- (adjust) + -ated (made to be)
Word family: calibrate (n.), calibration (n.)
Synonyms: adjusted, tuned, precise
Collocations: calibrated to accuracy, calibrated algorithm, properly calibrated
Example: A calibrated algorithm would predict the same actual recidivism probability for all groups receiving the same score.
automated
/ˈɔː.tə.meɪ.tɪd/|au.to.mat.ed
adjective
Operated or controlled by machine without human intervention; performed automatically according to a program or system. Automated systems replace human decision-making with algorithmic procedures.
Word Breakdown: auto- (self) + mated (made)
Word family: automate (n.), automation (n.)
Synonyms: mechanical, programmed, automatic
Collocations: automated decision, automated system, automated bias
Example: Automated systems in criminal justice raise concerns because once implemented, they apply decisions consistently without human discretion.
transparent
/trænsˈpɛər.ənt/|trans.par.ent
adjective
Open to inspection and understanding; allowing visibility of how something works. Transparent algorithms reveal their logic and reasoning, not operating as black boxes.
Word Breakdown: trans- (through) + parent (visible)
Word family: transparency (n.), transparently (adv.)
Synonyms: clear, open, understandable
Collocations: algorithmic transparency, transparent system, lack of transparency
Example: A transparent algorithm would explain which factors influenced its prediction and why.
explanation
/ˌɛk.spləˈneɪ.ʃən/|ex.pla.na.tion
noun
A statement or account of how or why something happened or works. In algorithmic contexts, explanations provide understanding of decision logic and the factors influencing outcomes.
Word Breakdown: ex- (out) + -planation (flattening/making clear)
Word family: explain (n.), explanatory (n.)
Synonyms: account, reasoning, justification
Collocations: explanation for bias, lack of explanation, provide explanation
Example: Without explanation, defendants cannot challenge algorithmic predictions or understand why they received a particular score.
Technical Terms
algorithmic fairness
/ˌæl.ɡəˈrɪð.mɪk ˈfɛər.nəs/|al.go.rith.mic.fair.ness
noun
The field of study focused on how to design and evaluate automated decision systems that treat groups equitably. Algorithmic fairness research grapples with competing definitions of what makes an algorithm fair.
Synonyms: fair algorithms, equitable automation
Collocations: algorithmic fairness research, algorithmic fairness criterion, achieve algorithmic fairness
Example: Algorithmic fairness researchers debate whether fairness means equal error rates or equal accuracy predictions across groups.
COMPAS
/ˈkɒm.pæs/|COM.PAS
noun
A controversial risk-assessment algorithm used in American courts to predict defendants' likelihood of future criminal behaviour. Journalists revealed that COMPAS made different types of errors for different racial groups.
Synonyms: risk assessment algorithm, recidivism prediction tool
Collocations: COMPAS algorithm, COMPAS bias, COMPAS scores
Example: COMPAS became the subject of intense scrutiny when research revealed its differential error rates across racial groups.
equalised odds
/ˈiː.kwə.laɪzd ɒdz/|e.qual.ised.odds
noun
A fairness criterion requiring that error rates be equal across groups; if an algorithm has a 10% false positive rate for one group, it should have a 10% false positive rate for all groups. Equalised odds ensure that algorithmic mistakes affect all groups equally.
Synonyms: equal error rates, equitable error distribution
Collocations: equalised odds criterion, achieve equalised odds
Example: Equalised odds is one definition of fairness: an algorithm is fair if it makes mistakes at the same rate for all racial groups.
calibration
/ˌkæl.ɪˈbreɪ.ʃən/|cal.i.bra.tion
noun
A fairness criterion requiring that a prediction score means the same probability of the outcome across all groups; a score of '60' indicates a 60% probability of recidivism regardless of the defendant's race. Calibration ensures predictive accuracy but may produce different error rates.
Word family: calibrated, calibrate
Synonyms: prediction accuracy, probability alignment
Collocations: calibration criterion, achieve calibration
Example: Calibration requires that a risk score accurately predict the actual probability of reoffending, but may result in unequal error rates across groups.
impossibility theorem
/ɪmˌpɒs.ə.ˈbɪl.ɪ.ti ˈθɪə.rəm/|im.pos.si.bil.i.ty.the.o.rem
noun
A mathematical result showing that different fairness criteria cannot all be satisfied simultaneously. If an algorithm achieves equalised odds (equal error rates), it cannot achieve calibration (equal prediction accuracy), and vice versa.
Synonyms: fairness trade-off, incompatible fairness criteria
Collocations: impossibility theorem shows, fairness impossibility theorem
Example: The impossibility theorem reveals a fundamental tension: no single algorithm can satisfy all competing definitions of fairness at once.
Figurative Phrases
black box
An opaque system whose internal workings are invisible or incomprehensible to users. The phrase comes from aviation but is now used for any system that cannot be inspected or understood.
Etymology/Type: idiomatic
Synonyms: opaque system, hidden mechanism
Example: Algorithmic systems that offer no explanation for their decisions function as a black box, preventing defendants from challenging predictions.
the hand of the machine
Automated agency or control; the implication that a machine, not a person, made a decision. The phrase uses 'hand' figuratively to represent agency.
Etymology/Type: idiomatic
Synonyms: machine decision, algorithmic agency
Example: Defendants blame the hand of the machine rather than human judges, but the algorithm was designed by humans using biased data.
tip the scales
To bias an outcome; to influence a decision unfairly in one direction. The phrase uses literal scales metaphorically.
Etymology/Type: idiomatic
Synonyms: bias an outcome, skew a decision
Example: Biased training data tips the scales against certain groups by ensuring the algorithm makes systematically unfair predictions.
rubber-stamp
To approve something without scrutiny or genuine examination; to give automatic approval. The phrase comes from the bureaucratic practice of stamping documents.
Etymology/Type: idiomatic
Synonyms: automatic approval, passive acceptance
Example: Judges should not rubber-stamp algorithmic recommendations without examining their fairness and accuracy.
play the numbers
To reason with or rely on statistics; to use numerical data to support an argument. The phrase suggests calculation and strategy.
Etymology/Type: idiomatic
Synonyms: use statistics, rely on data
Example: When courts play the numbers with algorithmic predictions, they may miss important individual circumstances that the algorithm overlooked.
the system is the problem
A diagnosis that structural or systemic issues, not individual failings, are responsible for a problem. The phrase treats 'system' figuratively.
Etymology/Type: idiomatic
Synonyms: structural bias, systemic failure
Example: If algorithmic bias reflects historical inequity in criminal justice, then the system is the problem: feeding biased data to an algorithm produces a biased algorithm.
Confusing Words
disproportionately vs. inordinately
These near-synonyms both mean excessively, but disproportionately emphasises imbalance relative to other groups, while inordinately emphasises excess beyond normal limits.
- disproportionately [out of proper proportion; affecting some groups unequally compared to others] — The algorithm's errors fell disproportionately on certain racial groups — false predictions were far more common for some groups than others.
- inordinately [excessively; far beyond what is normal or reasonable] — The impact on defendants was inordinately harsh because the algorithm provided no explanation for its predictions.
Disproportionately = unequally relative to another group (comparison-based); Inordinately = excessively beyond normal (absolute excess). Ask: Is this about unequal distribution (disproportionately) or about being too much (inordinately)?
bias vs. prejudice
These near-synonyms both involve skewed thinking, but bias is systematic distortion while prejudice is pre-formed judgment often with a social element.
- bias [a systematic distortion or skew that produces consistently unequal outcomes] — Algorithmic bias occurs when a system makes systematically different errors for different groups — bias can be unintentional.
- prejudice [a pre-formed negative judgment about a group, often based on social stereotypes] — Historical prejudice in criminal justice created biased training data that led to algorithmic bias.
Bias = systematic distortion (can be unintentional, technical); Prejudice = pre-judgment (typically intentional, social, negative). Ask: Is this about systematic skew (bias) or pre-formed judgment (prejudice)?
transparent vs. translucent
These near-synonyms both involve visibility, but transparent means fully see-through while translucent means partly see-through.
- transparent [completely open to inspection and understanding; allowing full visibility] — A transparent algorithm would fully explain its reasoning and allow inspection of how it reaches decisions.
- translucent [allowing light or some visibility through, but not fully clear] — A translucent algorithm might reveal some information but still operate partially as a black box.
Transparent = fully see-through, complete clarity (no hidden parts); Translucent = partly see-through, partial clarity (some opacity remains). Ask: Is this about complete openness (transparent) or partial visibility (translucent)?
- Choosing a selection results in a full page refresh.
- Opens in a new window.