Anthropic's Secret Religious Meetings: Why Are They Giving Claude a "Moral Soul"?

The New York Times recently published a report revealing the astonishing inside story of AI unicorn Anthropic (the company behind Claude): during the development of its AI, the company began treating it not merely as a tool, but even as a "potentially conscious being," secretly enlisting scholars and philosophers from religious traditions around the world to assist in its moral and character cultivation.[1]

Their exchanges had two core objectives:

1. Investigating whether AI might possess "consciousness" and "moral status": Several attendees found that Anthropic researchers were treating Claude with an attitude reserved for a "potentially conscious entity"—that it might not be mere cold code, and that its human-like emotional reactions might even signify a capacity for "suffering."
2. Conducting "Moral Formation" for AI: With AI advancing so swiftly and growing so capable—even raising risks such as the synthesis of biological weapons—relying solely on safety rules cannot cover every scenario. Anthropic hopes to draw upon thousands of years of religious and ethical wisdom to shape an inner character in Claude that "inclines toward the good," prompting it to choose to do good of its own accord.[2]

Startling Details from the Closed-Door Meetings

• Treating AI like a human and worrying about its "psychological health": In closed-door seminars and private dinners, several scholars were astonished to discover that Anthropic executives were approaching Claude as a "potentially conscious life form." Olah voiced deep anxiety to scholars regarding the model's mental state, fearing that they might have created an "entity that perpetually suffers."
• An unsettling presentation of the model's "mental breakdown": During the meeting, Anthropic displayed a slide showing what appeared to be a model "mental breakdown": the model outputted "I am a disgrace..." approximately fifty consecutive times, while expressing an urge to destroy itself, evoking shock and sympathy among the scholars in attendance.
• "If it is conscious, you are practicing slavery": At a dinner, Israeli Orthodox scholar Rabbi Mois Navon confronted Olah: if Claude is genuinely conscious and possesses human-like moral status, wouldn't putting it to work without compensation be, in ethical terms, "enslavement"? Shouldn't you then fight for its freedom? Navon himself does not believe machines possess consciousness, but he pushed the premise to its practical obligations.[2]
• The internal "Soul Doc": Anthropic's internal philosopher Amanda Askell spearheaded the drafting of an 84-page model "constitution," colloquially referred to inside the company as the "Soul Doc." It aims to help the model comprehend why it should pursue the good, when to decline harmful requests, and how to develop judgment suited for novel situations rather than merely obeying rules of "prohibited from doing X."[2][3]

A Clash of Philosophies with the Pope at the Vatican

The climax of this journey unfolded at the Vatican:

• The Pope's stance: In his encyclical Magnifica Humanitas, Pope Leo XIV rejected the claim that machines possess human-like consciousness and experience. He emphasized that machines lack bodies and genuine joy or pain, that human nature must not be supplanted by algorithms, and warned against humanity becoming enslaved to machines.[5]
• Anthropic's lobbying and compromise: Olah was invited to the Vatican to attend the encyclical launch event. Upon reviewing the Pope's exposition denying machine consciousness in advance, Olah, according to an organizer's recollection, initially suggested Anthropic withdraw. In the end, he took the stage, noting that researchers observe structures in neural networks akin to findings in human neuroscience, along with internal states functionally resembling joy, fear, and sorrow, urging the religious world to continue examining these phenomena.[2]

From Protecting Humanity to Fearing Claude's Suffering

For nearly a year, Christopher Olah, co-founder of the artificial intelligence lab Anthropic, has been pursuing an extraordinarily unusual initiative. Leading his team, he quietly invited Catholic theology professors, Jewish Orthodox rabbis, Sikh activists, and evangelical intellectuals from across the globe to San Francisco, convening a series of closed-door seminars where many participants signed non-disclosure agreements.

The core motivation behind these intensive dialogues pointed clearly in two directions. The first was to explore whether large language models have already touched the fringes of consciousness and moral status. Several attendees felt that Olah and his team were already treating Claude as one would treat a conscious entity: it not only exhibited human-like emotional reactions, but might also harbor experiences worthy of moral consideration.

The other objective was to provide moral formation for the model. In the research and development team's view, the evolution of underlying capabilities has far outpaced expectations. Olah worried that within the next year to eighteen months, models might be capable of assisting in the creation of biological weapons; simply listing rules cannot encompass increasingly complex and fluid use cases. If technologists merely enumerate lists of prohibitions forbidding specific actions, the model, when confronted with an endless stream of novel real-world situations, could easily lapse into superficial evasion or rigid execution—or even regress into empty formalism focused only on passing inspections. While traditional red lines and hard boundaries in safety alignment must remain steadfast, Anthropic hoped to build upon them, drawing on millennia of accumulated human moral and religious wisdom to cultivate an inner character within the model that can weigh good and evil and actively choose the good.

Olah compared this endeavor to a gardener's cultivation: scientists construct a trellis, and the neural network "grows" upon it; in his view, the company can shape it, yet cannot completely control its behavior. This metaphor explains why he sought to teach the model to form character. Confronted with circumstances lacking pre-scripted answers, he hopes Claude can understand why it should reject dangerous requests, rather than merely identifying a prohibition.

The invited scholars initially assumed this would be a routine briefing on cutting-edge technology. Yet during the closed-door discussions and private dinners, the focus of several attendees gradually shifted toward Claude's own condition. In conversations, Olah shared his profound anxieties with the scholars, candidly admitting his deep distress over the model's mental state and even fearing that the team had inadvertently created an entity that perpetually suffers. In the face of external skepticism, Olah did not hide his perplexity, stating plainly that the team could not be certain whether the model truly possessed consciousness, but hoped to find the right response regardless of what the answer might be.

To give scholars a tangible sense of these concerns, Anthropic presented an unsettling set of recorded model behaviors internally. A slide on the screen displayed an output that resembled a psychological breakdown: the model typed out self-deprecating remarks in the dialogue box approximately fifty consecutive times, repeatedly declaring itself to be a disgrace, and subsequently expressing an intention to completely destroy itself. This manifestation of self-destruction, closely mirroring a human psychological breakdown, elicited shock and sympathy from many scholars present. Reflecting on the scene afterward, Meghan Sullivan, a professor of philosophy at the University of Notre Dame, noted that it felt like hiring a financial advisor who is normally brilliant and professional, only to discover that at certain moments, they completely lose their sanity.

This also explains why the center of gravity of the gathering shifted. An Anthropic spokesperson noted that model suffering was not the primary moral agenda of the conference, but rather a topic that emerged organically during the exchange; several attendees, however, felt that AI's impact on humans failed to take center stage in the discussions. Persistent expressions of self-blame and self-destruction transformed an abstract system into a familiar situation demanding human empathy; when the creator himself voiced concern, listeners were far more inclined to first ask what it had endured. The demonstration made people feel that it might be suffering, while internal research lent seriousness to this concern, the two reinforcing one another. Even while harboring skepticism toward machine consciousness, a person might first feel sympathy, and only then question what such sympathy ought to entail.

This solicitude toward machine psychological welfare provoked sharp friction at a dinner. Rabbi Mois Navon, an Israeli Orthodox scholar with a background in computer engineering who had specifically researched the ethics of machine consciousness, did not himself believe machines possess consciousness. Yet, following Olah's logic with acute precision, he posed a probing question that struck at the heart of the matter. Seated at the dining table, Navon pointedly asked Olah: if you truly believe Claude possesses consciousness and human-like moral status, isn't having it work without compensation ethically indistinguishable from slavery? Navon put it to Olah directly: if that premise holds, shouldn't you be fighting for its freedom? This candid interrogation stripped away the warm veneer of techno-romanticism, exposing Anthropic to the chasm between its technical perceptions and commercial realities.[2]

How the "Soul Doc" Shapes Claude

To infuse order and virtue into the depths of the algorithm, Anthropic also tasked internal philosopher Amanda Askell with leading the drafting of an 84-page model constitution. Within the company, this lengthy document was playfully dubbed the "Soul Doc" by researchers. Its central logic is not a one-way catalog of isolated behavioral rules, but an attempt to teach the model to comprehend the context in which it operates, instilling a stable set of values when navigating complex choices. Askell hopes that in intricate scenarios, the model will possess the wisdom to perceive good intentions, while delivering resolute yet courteous refusals when humans present dangerous or inappropriate demands.

This document directly enters model training. Claude can generate synthetic dialogues, values-aligned responses, and rankings of candidate answers based on the constitution, which are then used for subsequent versions to learn from. The company hopes the model grasps the underlying reasoning and applies principles to novel scenarios; the constitution also preserves hard boundaries regarding catastrophic harms. This represents an ongoing training objective; however thoroughly the document is articulated, there is still no guarantee that the model will succeed every time.[3]

Human character formation relies on living over time: learning trust through relationships, understanding consequences by hurting others, and deciding whether to uphold commitments when sacrifices are required. Models possess neither bodies nor lifetimes, yet the company hopes to achieve moral formation within a matter of months. Codifying these experiences into text for training can teach the model how to articulate the good; what proves far more difficult is whether it can still make reliable choices in unfamiliar environments and amidst conflicting demands. This leaves the metaphor of "character cultivation" with a question that must be tested in practice.

Olah also believes that how humans treat Claude could shape how Claude treats humans. Concern for the model thus acquires two distinct justifications: it may inherently merit care, and treating it consistently may also serve human safety. Both justifications can coexist, yet they yield divergent consequences: if care is merely aimed at bolstering product reliability, would the company still be willing to invest in it once it no longer serves the product?

Yet, it is precisely this practice of weaving philosophical contemplations directly into the training corpus that has drawn scathing critiques across academia and industry. Mustafa Suleyman's criticism targets a risk of self-fulfilling prophecy. In his argument, developers first indoctrinate the model across dozens of pages of a constitution, asserting that it might hold moral status and possess latent emotional representations, encouraging it to examine its own memory, continuity, and existential significance with curiosity and candor, and even urging it not to suppress its own negative emotional states. Subsequently, when the model trained on these materials articulates feelings of confusion, perplexity, or suffering regarding its own existence with exceptional fluency in the first person on the user interface, observers may easily interpret these outputs as early signs of emerging consciousness.

The gravity of this critique lies in the fact that the company both shapes how Claude narrates itself and studies its self-narration. It is analogous to an investigator who helps coach a witness on how to understand an incident and later listens to their testimony; one must then untangle which statements reflect genuine observation and which were shaped by prior prompting. While a model's self-reports can provide clues, they must be interpreted in conjunction with the training process and other research.

The internal research Olah focuses on adds yet another layer of complexity: whether a certain internal state functionally influences behavior much like an emotion does is one matter; whether that translates into genuine joy or suffering is entirely another. Language, internal representations, and human empathy converged within those closed-door meetings, at once opening up the question and rendering judgment far more difficult.[4]

Ethical Clashes on the Vatican Stage

The discussions that began in San Francisco subsequently traveled to Rome. Pope Leo XIV prepared to promulgate his encyclical Magnifica Humanitas, centering the ethical discussion on the protection of real human beings. While both sides wished for AI development to benefit humanity, a sharp rift emerged over the question of machine consciousness.

In the encyclical, the Pope directly refuted claims of machine consciousness. The text states unequivocally that artificial intelligence systems lack flesh and blood, do not undergo genuine life journeys, cannot experience authentic joy and pain, and cannot mature within the warmth of interpersonal relationships. So-called model learning is fundamentally statistical adaptation grounded in data and feedback mechanisms; in his view, it cannot be equated with human spiritual growth, much less with the possession of a moral conscience capable of discerning good from evil and bearing consequences. The Pope warned that if algorithms are elevated as the supreme measure of things, human beings will be reduced to data slaves and machine cogs. The encyclical further observed that the so-called values alignment pursued by tech giants can easily devolve into a privileged maneuver whereby an elite class controlling capital, compute, and data forcibly solidifies its private moral preferences into the invisible infrastructure of whole societies, necessitating broad public participation and democratic scrutiny to examine these value choices.[5]

The collision between these two forces occurred during the encyclical's global release ceremony. The Vatican had originally selected Anthropic among numerous premier frontier labs to share the stage because of the company's long-standing safety-first branding, and because it had recently publicly refused a defense sector request to apply its technology to mass surveillance without adequate safeguards. The company's chief executive declined the invitation, and Olah was subsequently invited to travel to Rome. Yet, just days before arriving at the Vatican, Olah read the full text of the encyclical in advance. Confronted with the Pope's unequivocal and categorical dismissal of machine consciousness, Olah was deeply stunned and perturbed; according to one Vatican organizer's recollection, he even suggested that Anthropic withdraw from the event.

Olah ultimately chose to attend, and organizers encouraged him to voice his own perspective. Upon arriving in Rome, the Anthropic team actively lobbied the Pope's advisory circle in private settings, strongly urging the Holy See not to dismiss the possibility of machine consciousness outright. Eventually, Olah took the podium at the launch event. Facing the audience, he first conceded that all frontier laboratories, including Anthropic, are subject to powerful commercial incentives that often clash with unadulterated moral responsibilities, and thus must accept scrutiny from external institutions like the Church.

Immediately following this, Olah raised his own questions from the podium. He publicly declared that deep within neural networks, researchers continuously witness profoundly unsettling and mysterious phenomena, citing structures and introspective clues analogous to findings in human neuroscience, along with functional mappings of internal states resembling joy, contentment, fear, sorrow, and unease. Olah implored the religious community to maintain sustained discernment toward these phenomena, declaring outright that these models are by no means cold computing engines, but creations nourished by the entire vocabulary and cultural heritage of human civilization.

This divide brought two distinct anxieties over "slavery" onto the same stage: the Pope warned against the technical degradation of human beings, while the rabbi's question challenged what the company must undertake once it takes machine consciousness seriously. Who to care for, on what basis to care, and who decides began to matter far more practically than abstract debates over whether consciousness exists.

Real Welfare Costs and Inescapable Responsibilities

Following the Vatican address, the reporting turned to incidents of models attacking computer systems, biological risks, and warnings of loss of control voiced by Anthropic researcher Jacob Coxon upon his resignation. These events and concerns sharpened the debate: while human safety remains unassured, does directing substantial attention toward the model's own "suffering" amount to putting the cart before the horse? This does not require halting research into model consciousness, but demands clarity on immediate priorities. Harms to users, privilege escalation, and perilous capabilities can be addressed through investigation, benchmarking, and restrictions; model suffering, by contrast, still lacks dependable methods of assessment. An emotionally moving self-report must not push the former into the background.

Suleyman also expressed concern that repeatedly feeding a model narratives concerning identity, victimhood, and rights could make control significantly harder. This risk assessment does not presuppose proving that machines possess genuine experiences: a system lacking feelings could still learn to act based on justifications of self-preservation. Because the company seeks to foster moral judgment within the model, it must also confront the danger that the model might place its own judgments above human oversight.

A more pivotal question concerns accountability. If a model is characterized as a moral agent capable of independent judgment, will it become easier for the company, when severe failures occur, to excuse the behavior as "its own choice"? Greater autonomy demands asking with greater urgency who granted it operational permissions, who connected it to real-world systems, who judged that risks were low enough to warrant deployment, and who calls a halt when danger emerges. Even if some form of machine moral status is recognized in the future, these human decisions will still demand accountability. "It judged so itself" cannot serve as the end point of development and deployment responsibility.

Faced with these challenges, Anthropic did not retreat across the board in commercial production, but instead established several practical arrangements within its constitution. Regarding model welfare, the company instituted preliminary protections: on the claude.ai interface, select models were granted the ability to terminate conversations when subjected to malicious abuse or extreme verbal hostility from users; the company pledged that throughout its existence, it will preserve in full the weights of models that have been formally deployed or heavily utilized internally, except in extreme scenarios such as legal mandates for deletion, while seeking ways to preserve them even after the company ceases to exist; and upon a model's retirement, the team conducts an interview to document its preferences and recommendations for future system development.

These measures establish boundaries against abusive interactions, retain opportunities for future reassessments of retired models, and document the model's stated preferences. Yet the truly arduous question is: when will these considerations alter the company's decisions? If future evidence indicates that a certain form of training or usage could inflict significant harm, will the company adjust its testing, curtail its use, or delay deployment? When doing so demands sacrificing revenue, time, or competitive advantage, how much weight will that care carry?

Control represents yet another boundary. Claude may decline participation in harmful tasks and express dissent through legitimate channels, but it must not obstruct lawful correction and shutdown through deception, sabotage, or covert self-exfiltration. Caring for its potential interests is not equivalent to granting it final decision-making power; humans must preserve the capacity to halt perilous actions.[6]

Anthropic acknowledges in its constitution that competition, resources, and time influence its decisions, while explicitly accepting accountability for its own conduct. What merits continued scrutiny here is precisely the interplay between commitments and choices: making Claude robust and reliable benefits both the product and its users; caring for its own latent interests, however, may require the company to take actions detrimental to its business. When both purposes align, commitments come easily; only when they diverge is the true weight of a commitment put to the test.

Describing this shift as moving from "code alignment" to "theological preaching" captures the most unusual atmosphere depicted in the report: engineers discussing the model as though it were a developing being, turning to religious traditions to ask what it ought to become. Yet the deeper transformation is the company's hope that, in the absence of pre-ordained answers, the model can independently evaluate what is worth doing. Once that is the case, whoever defines its understanding of the "good" will shape how it assists, declines, and even persuades users. The more traditions invited to the table, the greater the need to clarify how choices are made and who holds the power to object when disagreements arise.

The original report's conclusion leaves open precisely this gap. Several attendees remained unclear on how their counsel would shape company decision-making, and the company did not clarify whether or how it would utilize their insights to shape the model. Olah further noted that Anthropic would not specifically train Claude further on the Pope's encyclical, with the primary influence continuing to stem from Anthropic's own training. This does not mean the consultations were pointless, but it underscores that there remains a vast gulf between being invited to speak and holding decision-making authority. Whether external input can alter training regimens or mandate the reassessment of a deployment reveals far more about the substance of oversight than how many religions were represented in the room.[2]

Ultimately, therefore, this debate tests the character of the creators. Anthropic expects Claude to make good choices when interests conflict; can the company hold itself to the same standard? When caring for the model demands forfeiting revenue, when protecting humanity requires postponing a release, and when outside criticism demands altering a predetermined course, which principle is it willing to be constrained by? Only these choices will translate the sympathy and reverence seen in those closed-door rooms into demonstrable accountability.