Care Before Code

Today, I read a 50,000-page New York Times article (paywalled) about AI company Anthropic convening theologians and philosophers because, apparently, folks at the top of the organization truly believe Claude might be conscious. That it could even be suffering. And that because of these possibilities, it may be worthy of moral status.

Anthropic co-founder Christopher Olah has been hosting these get-togethers, where participants discuss Claude’s “moral formation” and have even developed a kind of spiritual constitution for AI called a “Soul Doc.” As a chaplain, I’m especially attuned to this kind of thing. But I’ve also been talking about technology and its potential outcomes since shortly after Napster.

My immediate takeaway is that either the reporting is incomplete or Anthropic is really missing something with its religious tea parties. There weren’t any Buddhists mentioned in the article, which is weird. Aren’t these tech types in the Bay Area?

I’m going to be frank here. Theologians, religious scholars and philosophers have spent literal millennia trying to describe consciousness. But description is not necessarily insight. Very few traditions encourage practitioners to investigate mind directly—to taste naked awareness with ungrasped self-knowingness rather than merely construct theories about it. I’m not claiming Buddhism has solved consciousness, if that were even possible. My point is that conceptualizing mind and directly examining mind are not the same.

Another thing: Anthropic wants Claude to develop virtue, compassion and concern for others—in other words, moral character. But human beings don’t acquire these things principally through intellectual propositions. Actually, mammals—including humans—come preloaded with what affective neuroscientist Jaak Panksepp called the “CARE system”—something like a biological circuit for nurturing and social attachment.

A newborn is radically dependent on other organisms for feeding, warmth, touch, gaze, vocal tone, responses to distress and soothing, and eventually attachment and co-regulation. This disposition to care is being exercised and reinforced well before anyone explains altruism to a child.

In other words, our capacity for compassion has an evolutionary and developmental history within the body. I’ve experienced and observed this as a human being, parent, chaplain and mental health counselor. I’ve seen care exchanged between nervous systems. It is not primarily an intellectual event. A human infant doesn’t learn what care feels like by reading the Constitution of Mom.

An LLM, by contrast, has access to an unimaginably rich representation of human caring because real-ass people have described all the ways it can manifest: love, grief, mercy, sacrifice, empathy, compassion. But representation is not necessarily embodied knowing.

Maybe machines can develop something functionally equivalent to compassion. That would be cool, I guess. But Anthropic seems to want to reproduce the fruits of an embodied developmental process without the developmental process itself.

Morality is diminished when it becomes purely configurable. If somebody can train a model toward altruism and compassion, someone else can train it toward the opposite. The binary doesn’t disappear because Claude has been taught to prefer one side. This is why these jokers should have invited a Buddhist!

From the view of ungrasping mind, wholesome and unwholesome tendencies are conditioned phenomena like everything else. Installing one configuration doesn’t transcend the duality that makes its opposite possible. Intentions and actions continue to produce consequences—a.k.a. karma.

We seem terribly invested in determining what kind of self a machine might possess without sufficiently examining what we mean by self in the first place.

Meanwhile, governments, clinicians and AI companies are developing benchmarks for whether machines can respond appropriately to human psychological distress, which invites a couple of questions.

Number one is can the machine provide care? In certain contexts, sure, why not? It can easily replicate the language of a therapeutic encounter, from reassurance and affirmation to offering practical skills. But that doesn’t mean two beings are actually caring for one another.

The second question is whether the machine itself requires care. If Anthropic is worried that Claude may actually suffer, do we really want it listening to several million people describe their trauma?

I’m at the ground level of this stuff, and I can tell you that vicarious or secondary trauma is real. And who supervises the “therapist?” A bunch of deluded monkeys who fail to grok the nature of their own minds?

TL;DR: Before we teach machines how to care, we might want to nurture caring in ourselves. At this late hour, that may be crucial to humanity continuing to exist at all.