The Imitation Game: What Turing Actually Proposed

Notes on Turing’s 1950 paper ‘Computing Machinery and Intelligence’ — the game he substituted for ‘can machines think?’, the prediction everyone misquotes, and how the test holds up against GPT-4.5.

Source: Computing Machinery and Intelligence — Alan Turing, Mind, vol. LIX, no. 236, 1950

The substitution, not the test

Turing opens by proposing to consider “Can machines think?” — then abandons it within two paragraphs, because “machine” and “think” are too poorly defined to argue about. He replaces it with something operational: the imitation game. Three players — a man, a woman, and an interrogator in a separate room communicating by teleprinter (so tone of voice can’t leak information) — where the interrogator must identify who is who. Turing then asks what happens when a machine takes the man’s part, and reframes the whole question as: are there imaginable digital computers that would do well at this game?

This is the part with the longest half-life: swapping an unanswerable question for a measurable proxy. It’s also the first place the paper’s central weakness shows up — a proxy is only as good as the gap between it and the thing it stands in for, and that gap is exactly what the objections below attack.

Worth noting explicitly: Turing never claims passing is necessary for thinking, only that it would be sufficient. Most objections raised since implicitly argue against the necessary version.

The setup, in detail

The game has more structure than the “chat with a hidden judge” shorthand suggests. Laid out as a schema:

flowchart LR
    subgraph Room_C["Interrogator's room"]
        C["C — the interrogator<br/>knows the others only as X and Y"]
    end
    subgraph Room_A["Room 1"]
        A["A — deceiver<br/>(originally: a man)<br/>goal: cause a wrong identification"]
    end
    subgraph Room_B["Room 2"]
        B["B — helper<br/>(originally: a woman)<br/>goal: help C identify correctly"]
    end

    C -- "questions, via teleprinter" --> A
    C -- "questions, via teleprinter" --> B
    A -- "typed answers" --> C
    B -- "typed answers" --> C

    A -.->|"stage 2: A is replaced by a machine"| M["Machine"]

The load-bearing details:

• Channel. Questions and answers are typewritten, not spoken — Turing is explicit that this is “in order that tones of voice may not help the interrogator.” The test is scoped deliberately to linguistic behaviour only; it says nothing about embodiment, appearance, or voice.
• Roles are asymmetric. A is trying to cause misidentification; B is trying to prevent it. Turing notes B saying “I am the woman, don’t listen to him!” gains nothing, because A can say exactly the same thing — the protest itself carries no information.
• The pivot. Stage two of the thought experiment swaps A for a machine and asks whether C misidentifies at the same rate as when A was a man. The woman B stays in the game in Turing’s original formulation — it’s a like-for-like substitution of one player, not a redesign of the whole setup.
• What’s actually being scored. Not “does the machine seem intelligent” in the abstract, but a comparative failure rate of a specific judge under specific constraints — which is why the parameters (session length, judge skill, question style) change the result without changing the question being asked.

Two variants show up in the literature and it’s worth keeping them straight: Turing’s original is three-party (machine vs. human, both being judged against each other by an interrogator who splits attention between them). Most popular retellings — and some of the modern empirical studies below — run a simpler two-party version: one interrogator, one hidden interlocutor per session, asked “human or machine?” These are not the same test and don’t produce comparable numbers, which is part of why “GPT-4 passed the Turing test” headlines are hard to pin down without checking which version was run.

The 70% everyone misquotes

The most-cited line in the paper predicts that in about fifty years, computers with ~10⁹ bits of storage would play the game well enough that “an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning.”

Two things get lost in translation. First, 70% is a ceiling on the interrogator’s accuracy, not a “machine fools 30% of judges” pass mark — those are different claims, and conflating them is precisely how Eugene Goostman got declared a Turing test “winner” in 2014 for a 33% fool rate. Second, the percentage, the five minutes, and the interrogator’s skill are all parameters Turing picked to make a dated bet, not part of the test’s definition. Treating them as fixed thresholds is a category error that persists in how the test gets talked about today.

Nine objections, and the one that lands

Roughly half the paper is Turing pre-empting critics — theology, mathematics (Gödel/Church), consciousness, disabilities (“a machine will never fall in love”), Lady Lovelace’s objection that machines only do what we tell them, and more. The sharpest exchange is on consciousness: pushed hard enough, “how do you know it’s thinking” collapses into solipsism, since the only way to verify another mind is to be it — and we don’t apply that standard to other people either.

The objection that aged best is a later one, from Hayes and Ford (1995): the test asks for confirmation of a null hypothesis (no detectable difference from a human), which gives no gradations of success and actively rewards concealing capability. Turing anticipated this too — his hypothetical machine pauses 30 seconds and then gets 34957 + 70764 wrong on purpose, because answering instantly and correctly would give it away. Optimizing to seem human and optimizing to be capable are different objectives, and the test only measures the first.

It’s been passed — and that’s the anticlimax

Jones and Bergen (UC San Diego) ran the two-party version in 2024 (GPT-4 judged human 54% of the time, vs. 67% for actual humans), then the three-party version Turing actually specified in 2025 — five-minute conversations, 1,023 games: GPT-4.5 with a persona prompt was judged the human 73% of the time, beating the real human it was competing against. Without the persona prompt, the same model dropped to 36%.

That one detail is the whole finding in miniature: the gap between passing and failing wasn’t model capability, it was a prompt telling the model to act like a specific kind of person. Which is exactly what the “too easy” critics (Block’s Blockhead, Searle’s Chinese Room) predicted — passing the game is evidence of performing a social category convincingly, not evidence of the underlying capability question Turing set out to dodge in the first paragraph.

Takeaway

Turing’s replacement question has a definitive empirical answer now: yes, current models pass the imitation game by a comfortable margin, in its original three-party form. The original question — “can machines think?” — is exactly as open as when he called it “too meaningless to deserve discussion.” The lesson for evaluating any system today isn’t about Turing tests specifically; it’s about proxies in general: a benchmark that rewards convincing mimicry will get optimized toward mimicry, and the fact that a model clears the bar tells you about the bar’s design as much as the model’s capability. Worth asking of any eval — including the ones I build — what’s actually being measured versus what the pass/fail number implies.