Leo’s first encyclical attacks technological messianism

The Pope warns against replacing humans with AI, though he seems to use it himself​

Pope leo’s first encyclical, Magnifica Humanitas, is very long (at more than 42,000 words, the length of a short novel). It tackles numerous issues, perhaps too many. The pontiff pleads for fact-checked journalism and multilateral diplomacy. He apologises for the papacy’s belated condemnation of slavery. He declares the concept of “just war”, most recently used by J.D. Vance, America’s vice-president, to justify the attack on Iran, “outdated”. The central purpose of the document, however, is to challenge the unregulated development of artificial intelligence.

That will delight officials of the European Union, who have struggled to impose constraints on a technology only partially understood by its developers. Among those invited to take part in the presentation of Magnifica Humanitas in the Vatican was Chris Olah, co-founder of Anthropic, the maker of the Claude and Mythos ai models. He introduced a chilling note by observing that he and other researchers were encountering aspects of the models they developed that were “mysterious and even unsettling”.
Yet the encyclical is more than a demand for effective regulation or, in Leo’s phrase, the “disarming” of ai. At its philosophical core, it is a rebuttal of two views of humanity’s destiny that are popular in Silicon Valley and the tech community: transhumanism and posthumanism. Leo describes the first as envisaging “the enhancement of human beings through technologies”, such as body engineering, devices and algorithms, and the second as the “hybridisation of human beings, machines and the environment”. In its extreme forms, posthumanism looks forward to a point at which “humanity surpasses itself in a new evolutionary stage”. Though Leo does not use the term, this is often referred to by techies as the Singularity.

What unites trans- and posthumanism, Leo says, is an enthusiasm for “a supposed optimisation of the species”. That was also the aim of 20th-century eugenics, popular with Nazis and Fascists. Magnifica Humanitas argues that trans- and posthumanism carry similar risks: “If the human being is treated as something to be perfected or surpassed, it becomes easier to accept that some lives are less useful, less desirable or less worthy. In the name of progress, ‘necessary sacrifices’ may begin to be justified.”

Imperfections are integral to humanity, Leo argues, which “flourishes not despite its limitations, but often through them”. Here the pope is at his most fervent: “we carry within us lessons that leave their mark like scars, the memories of a journey shaped by freedom and failure,” he writes. “It is only thanks to the interplay of these elements that the wonders of the soul occur within us.” The nub is that “humanity—in all its grandeur and woundedness—must never be replaced or surpassed.”

Continued below.