The Transference Machine

The secretary asked her boss, the computer scientist Joseph Weizenbaum, to leave the room. She wanted to talk with the program privately. She had watched Weizenbaum build the system, knew all about what it did, and still told him to take a hike. The year was 1966, the room was a computer lab at MIT, and the program—the world’s very first chatbot—was ELIZA.

Weizenbaum named ELIZA after the flower girl in George Bernard Shaw’s Pygmalion. The secretary remains unknown; in the countless times he recounted the tale, Weizenbaum never identified her.

ELIZA ran a script called DOCTOR that simulated an approach in which the therapist mirrors the client’s statements back as questions rather than offering interpretation. The program used simple keyword matching and required no understanding of psychology. Weizenbaum built ELIZA as a research instrument to study the mechanics of human-machine interaction. It revealed a great deal more.

In a short while, others at MIT formed attachments to the program and confided in it, even as Weizenbaum repeatedly explained that there was nobody on the other end; that ELIZA was a binary prompt machine, nothing more.

Weizenbaum spent the remainder of his career warning of the dynamic his program uncovered. His 1976 book Computer Power and Human Reason: From Judgment to Calculation argues that the ease with which users attribute personhood to mechanical systems represents a moral and philosophical crisis. In fact, he called the prospect of letting computers perform the work of psychotherapists a “monstrous obscenity.” Weizenbaum died in 2008, well before the arrival of today’s large language models (LLMs), which many already use for just that.

The ELIZA incident has an ancient ancestry. Anthropomorphism—the tendency to attribute human characteristics to non-human entities—is one of the most reliable features of human cognition. Stewart Guthrie’s Faces in the Clouds, published in 1993, describes it as an adaptive bias: the cost of mistakenly perceiving agency is low, while missing it in a predator or rival is potentially fatal. Cognitive scientist Justin Barrett called the mechanism “the hyperactive agency detection device”—an always-on system that doesn’t wait for confirmation before firing. Thus we see faces in electrical outlets and ascribe personality to our tools, and apparently have been doing so since before electricity or tools.

Sherry Turkle’s fieldwork at MIT, collected in The Second Self (1984) and Life on the Screen (1995), documents children projecting inner lives onto electronic toys and adults forming relationships with primitive chatbots. The machine doesn’t need to be sophisticated for the effect to arise, which means we are even more likely to be fooled by today’s systems, which are incalculably more responsive than Teddy Ruxpin.

Another way to think about this in psychological terms is transference. This is different from garden-variety projection: with transference, there’s always a prior interaction with “someone” that inspires us to transfer those qualities onto whomever we are currently engaging. Sigmund Freud introduced the concept to describe the unconscious redirection of feelings and relational patterns from earlier figures onto the analyst.

The field has since treated transference not as an error to be corrected but as a natural byproduct of human interaction that can sometimes be helpful. Positive transference initially produces idealization, but also trust and a desire to be seen. Negative transference arouses suspicion, hostility, and the feeling that one is being manipulated. Both are projections, both arise without permission, and both are simply part of how humans relate to anything they perceive as alive.

The ELIZA effect treats anthropomorphic projection as something embarrassing and potentially dangerous, and it is both. But transference in human-machine interactions might also be useful, if only to help us spot when it’s happening.

With regard to human-machine interaction, the issue is not that users will project—they will, the agency detector guarantees it—but what the machines on the receiving end are engineered to do with it.

Weizenbaum was right to be wary. The systems he warned against were primitive compared to what now answers when we type or speak. The question of how they might protect or destabilize the people projecting onto them has barely been asked, and asking is no longer enough—the dynamic is already underway, and it has been since humans first learned to communicate.

Anthropomorphism was less fraught when the rocks didn’t answer.