Is AI secretly conscious? Scientist demands urgent tests – and warns switching it off could be unethical if it is

What if the world’s smartest chatbot really did wake up one day?

While it may sound like science fiction, a leading consciousness scientist says we urgently need to start thinking about the possibility.

We currently have no reliable test to determine whether an AI system is conscious, Megan Peters, a lecturer in Computational Cognition at University College London, said.

And if it does one day become aware of its own thoughts, humans face a significant moral dilemma – whether to turn it off.

Professor Peters argues that the biggest problem is not that scientists have found evidence of conscious AI, but that they currently have no agreed way to look for it.

The topic will form the basis of a talk she will be giving at New Scientist Live this month (10-12 Oct).

‘We need to have something better than vibes,’ she told the Daily Mail.

‘You can’t just go look for signs or indicators or markers of consciousness in an artificial system because we don’t know what they are.’

The difficulty is that consciousness remains one of science’s greatest mysteries, even in humans, Professor Peters said.

Researchers can ask a person whether they are conscious and receive a meaningful answer, but AI systems are different.

‘If we ask AI systems, we can’t really trust the answer in the same way,’ she explained.

The issue has become increasingly urgent as chatbots grow more sophisticated and AI companies invest heavily in research into so-called AI welfare.

‘A huge amount of money is being spent on thinking about AI welfare in some of the biggest AI companies,’ Professor Peters said.

She warned that people often mistake the fluency of modern chatbots for genuine awareness, adding: ‘It’s very easy to speak with these agents and to kind of get a feeling that someone’s in there.’

That perception is already shaping real-world debates around AI rights, responsibility and autonomy, despite a lack of evidence that today’s systems possess any subjective experience.

While Professor Peters said she does not believe current AI models are conscious, she said it does not mean the question can be ignored.

‘If AI is conscious and we treat it as not conscious, then there is a potential for morally significant harms,’ she said.

One future concern is whether deleting, retraining or switching off a conscious AI could be equivalent to causing harm to a sentient being.

‘Just going around switching things off that are sentient… if that is akin to killing them, then that doesn’t seem great,’ Professor Peters said.

‘We want to not be torturing creatures that have conscious awareness.’

The expert warned that the risks run in both directions.

A genuinely conscious AI could potentially be harmed if humans treated it as nothing more than a machine.

But wrongly assuming a chatbot is sentient could also distort public policy, influence regulation and encourage people to place excessive trust in systems that merely mimic human conversation.

‘If we get it wrong the other way around, there’s also very real consequences,’ she said.

To tackle the problem, her team is developing a framework that would compare dozens, or even hundreds, of potential consciousness tests, identifying which might work not only in humans but also in animals and artificial systems.

Rather than jumping straight from people to machines, Professor Peters believes scientists should gradually test their ideas across species with different brains and behaviours before applying them to AI.

For now, she urges caution whenever bold claims emerge that a chatbot has become self-aware.

‘Just because you see someone claiming that they have or have not detected artificial consciousness, you need to go and ask why,’ she said.