(RNS) — It took a little more than a day for Father Justin, an artificial intelligence avatar posing as a priest, to be defrocked. After Catholic Answers, a site devoted to evangelizing for Catholicism, introduced the character to answer questions about the faith, Catholics on social media called the character a “scandalizing mockery of the sacred priesthood” that offered only “a substitute for real interaction.”

On April 24, Catholic Answers apologized for the experiment, and Justin was reintroduced as a lay theologian.

Catholics close to the Vatican’s work on artificial intelligence say that Justin captures the possible problems with AI evangelization and the reasons for caution in Pope Francis’ and other church officials’ attempts to tackle AI, even as the technology is becoming an increasingly buzzy topic at the Vatican.

The Rev. Philip Larrey. (Photo courtesy Boston College)

The Rev. Philip Larrey, a professor in the department of formative education at Boston College, said that while he thinks Catholic Answers are a good group, “they were a little bit too quick to enter into something which is extremely complicated, and that is interactive artificial intelligence.”

San Jose, California, Bishop Oscar Cantú, who leads the Catholic faithful in Silicon Valley, said that AI doesn’t come up much with parishioners in his diocese. Nonetheless, as a leader in the computing capital of the world, Cantú said he has engaged with AI as a global and moral issue, even if he hasn’t “delved into it too much.”

Pointing to the adage coined by Meta founder Mark Zuckerberg, “move fast and break things,” the bishop said, “with AI, we need to move very cautiously and slowly and try not to break things. The things we would be breaking are human lives and reputations.”

Experts agreed that Father Justin’s imitation of the sacrament of confession was highly inappropriate.

Cantú said, “If we have some sort of a robot in the guise of a priest, it can confuse” people about the fact that the sacraments must be celebrated in person. “Just because Father Justin recites the formula, that doesn’t make it a sacrament,” he said.

The bishop cautioned that AI chatbots should make very clear that they are AI. “It’s so critical that we be as transparent as possible, for the sake of the people we’re trying to guide,” Cantú added.

Even Justin introducing itself as a lay theologian is problematic. “A person who may be incredibly knowledgeable of Scripture and of church teaching but is not a person of faith does not do theology, because theology begins with faith,” the bishop said.

Bishop Oscar Cantú. (Photo courtesy Creative Commons)

Noreen Herzfeld, a professor of theology and computer science at St. John’s University and the College of St. Benedict and one of the editors of a book about AI sponsored by the Vatican Dicastery for Culture and Education, said that the AI character was previously “impersonating a priest, which is considered a very serious sin in Catholicism.”

The leaders don’t dismiss the usefulness of AI when used properly. Cantú said that AI can do “tremendous work” as “a tool that can be used for good for doing research.” But, he added, “A person of faith doing theology then needs to judge the credibility of each source and the authority of each source.”

Larrey, who has worked closely with both Vatican and AI leaders on the ethical issues surrounding AI, emphasized that Pope Francis is interested in “person-centered AI,” meaning that AI must be used “for the good of human beings” and “not the detriment of people.”

The best example of AI being used for evangelization, Larrey said, is Magisterium AI, a chatbot developed by Longbeard, a digital strategy firm founded by a former seminarian named Matthew Harvey Sanders, a friend of Larrey’s. The bot explains church teaching in a format like ChatGPT, by drawing on official church teachings, as well as select writings such as the works of St. Augustine and St. Thomas Aquinas, linking to the documents that informed its answer.

Larrey distinguishes such automated research from generative AI, which learns and experiments with its newfound knowledge. “When you have a generative AI, if you’re not careful, it gets out of hand,” he said.

Accuracy issues, Herzfeld said, is one of many reasons it should not be used for evangelization. “As much as you beta test one of these chatbots, you will never get rid of hallucinations” — moments when the AI makes up its own answers, she said. 

The Magisterium AI interface. (Screen grab)

“The problem is actually built into how they work,” Herzfeld continued. “They work on statistics, on probabilities as to what words and phrases should follow, and they do not have internal mental models of the world. And without that, they will always get off track.”

Herzfeld said that an AI is only as good as the data it is trained on. On issues where the church is divided, she said, “I could see bots out there giving competing answers, for example, on the desirability of the Latin Mass or the role of women in the church.”

Herzfeld expressed concern, too, that people who are not well-versed in technology may believe computers have a “certain level of infallibility,” and she worries that turning to AI bots will lead to spiritual “de-skilling,” convincing young people to think “religion is just about answers.”

“When I was their age, I was shy, and if I’d had a bot to answer my questions, I’d have gone to a bot instead of going to my pastor or even going to my parents to ask questions about religion,” Herzfeld said, adding that forming relationships with AI instead of with other people or God could become a type of idolatry.

Larrey, who has been studying AI for nearly 30 years and is in conversation with Sam Altman, the CEO of OpenAI, is optimistic that the technology will improve. He said Altman is already making progress on the hallucinations, on its challenges to users’ privacy and reducing its energy use — a recent analysis estimated that by 2027, artificial intelligence could suck up as much electricity as the population of Argentina or the Netherlands.

Larrey understands that Catholic Answers’ Justin was designed to “get people who would otherwise not be interested in the Catholic faith,” but also thinks chatbots should not vie with real priests. “Why don’t you just go down to your parish and talk to a priest? They’re not extinct, you know,” he said.

For evangelization, “I think you need people to contact and to attract other people,” said Larrey. “We will never eliminate that step.”

Cantú believes this is the ultimate lesson to be drawn from the ill-fated Father Justin. “The faith is not just about answers. It’s not just about information. It’s about encounter, a personal encounter with Christ, with real people,” he said.

Just as rank-and-file Catholics should exercise caution with AI for evangelization, Cantú suggested similar care from priests who might turn to ChatGPT for an assist when writing their homilies.

While the AI’s suggestion can be a good starting point for priests who are “pressed for time,” Cantú said, the priest must make the homily his own. 

“When anyone within the church, a person of faith, expresses the truth, to some degree, they’re opening a window to the mind of God because God is truth.” In the end, the bishop said, “It has to be my words and my act of faith.

“A robot can’t do that because it’s not an act of faith.”