Y12W38WR Machines making decisions about people

Design
The writing prompt

Design the specific regulatory framework Australia should adopt for high-stakes algorithmic decision-making, given both the bias research and the capacity of well-designed systems to reduce human bias.

1Retrieval check

Q1.What did Obermeyer, Mullainathan et al. (2019) find about a widely-used healthcare algorithm?

  • AIt was fair across demographic groups
  • BIt systematically under-served Black patients because it used past spending as a proxy for need
  • CIt matched human clinicians exactly
  • DIt was never deployed

Q2.What does the blind-audition research add to the algorithmic-bias conversation?

  • AIt shows algorithms always help
  • BIt shows removing human judgement can dramatically reduce bias — the comparison must be to real human alternatives, not to perfect decision-makers
  • CIt shows humans are always unbiased
  • DIt disproves the bias literature
Show answer key

Q1 → B. It systematically under-served Black patients because it used past spending as a proxy for need.The proxy problem — healthcare spending as need — embedded historical access inequality into the algorithm.

Q2 → B. It shows removing human judgement can dramatically reduce bias — the comparison must be to real human alternatives, not to perfect decision-makers.Blind auditions dramatically increased women’s hiring in orchestras — a clean case of ‘algorithm beats biased human’.

2Prompt deconstruction

Stimulus
COMPAS / Gender Shades / Obermeyer; the blind-audition research.
Scope
Australian regulatory framework; reference the specific studies.
Thinking
Which high-stakes domains; what kinds of audit/transparency/accountability; who enforces.
Position
Between minimal regulation and heavy restriction.
Output
Named framework covering specific domains, audit requirements, and enforcement — plus honest concession of what it cannot achieve.

3Position nudge

Where on the range does your proposal sit?

Pole A
Pole B

Pole AMinimal regulation

Pole BHeavy restriction on algorithmic decisions

Commit to a specific point; defend it in your planner.

4Planner — design the thing, then the trade-offs

My proposal
One sentence — the framework’s core.
Covered domains
Which decisions are high-stakes enough for coverage (hiring, lending, healthcare, criminal justice, immigration, benefits).
Audit requirements
What algorithms must disclose; what independent testing is required.
Accountability
Who is liable when decisions produce harm; what remedy applies.
Comparison standard
Against what baseline are algorithms judged — human status quo, or an ideal?
Enforcement body
Who enforces, and how (inspection, response to complaints, certification).
What it doesn’t achieve
A specific harm the framework accepts it cannot eliminate.

5Sentence stems

  • My proposal is ___.
  • I am grounding this in [researcher]’s finding that ___.
  • The main trade-off is ___: this design gains ___ but loses ___.
  • The most predictable objection is ___, and my response is ___.
  • I would know it was working after [time] if ___.
  • What I am most likely to abandon is ___, so I will build in ___ to prevent that.

6Exemplar paragraph (not about this article)

(1) My proposal is a tiered algorithmic-accountability framework: (1-a) Tier-1 high-stakes domains (criminal justice, healthcare rationing, welfare eligibility, child-protection triage) require independent pre-deployment audit, mandatory disclosure of training data composition and outcome metrics across demographic groups, and ongoing monitoring; (1-b) Tier-2 domains (hiring, lending, insurance) require disclosure and audit-on-request; (1-c) a statutory individual right to a meaningful human review of any algorithmic decision materially affecting you. (2) I am grounding this in the Obermeyer finding that proxy selection can embed historical inequity invisibly, in Buolamwini’s demographic accuracy gaps, and in the blind-audition research that the comparison must be to real human alternatives. The main trade-off is innovation cost: this design gains accountability but loses some deployment speed. (3) The most predictable objection is that audit requirements will entrench incumbents who can afford compliance, and my response is that the tiering is calibrated — small-scale / low-stakes systems sit outside Tier-1 entirely, and the audit regime is scaled to risk, not to system size. (4) I would know it was working after two years if the published demographic-outcome metrics for Tier-1 systems show measurable narrowing of accuracy and treatment gaps. (5) What I am most likely to abandon is the individual-right-to-human-review under volume pressure, so I will specify response-time statutory minimums. (6) What the framework doesn’t achieve: it cannot eliminate bias where the underlying social conditions are themselves unequal; it can only prevent algorithms from compounding those conditions.

What this paragraph does, move by move

  1. Names a three-element tiered framework precisely.
  2. Grounds in three specific research findings with their mechanisms.
  3. Handles the incumbent-entrenchment objection by risk-based tiering.
  4. Specifies a measurable two-year success test.
  5. Statutorily locks the individual-review mechanism.
  6. Honestly names what the framework cannot do (baseline inequality).