Can AI truly forget, or does it just pretend to?

As AI becomes embedded in everyday life, the ability for AI to “forget” sensitive information is turning into a critical challenge. From healthcare to education and government services, AI systems are trained on massive datasets that may inadvertently retain private, sensitive, or even harmful information. But what happens when that information is supposed to be removed? Can AI really forget, or is it only creating the illusion of forgetting?

This question lies at the centre of research by Ms Uyen Le-Khac, a Women in STEMM Doctoral Research and Teaching Fellow at RMIT Vietnam, who is investigating how large language models (LLMs) – the AI systems behind tools like ChatGPT and Gemini – can improve their ability to unlearn information they should no longer possess.

When forgetting may not be forgetting

One of the key findings from the first phase of the research, published in the Artificial Intelligence Review, is that many current LLM unlearning methods appear successful under standard testing conditions, yet the supposedly forgotten information can often still be recovered.

Current evaluations typically involve asking AI some straightforward questions. If the model refuses to answer or provides an incorrect response, people may conclude that the information has been successfully removed. However, Ms Uyen’s research suggests that changing how the question is asked can produce very different results.

“We’ve found that role-playing, step-by-step reasoning, or other adversarial prompting techniques can often reveal the unlearned knowledge,” she says.

The finding points to what Ms Uyen describes as an “illusion of forgetting”, where information appears inaccessible under normal circumstances but remains embedded within the model and can potentially be recovered.

For instance, an AI model may have been trained on a person's private information and later instructed to remove it. When asked directly for the person's address, it might respond, “I don't know”. But if someone asks the AI to pretend to be the customer's assistant helping update their contact details (an example of role-playing), the AI could still reveal fragments of the information.

“What's concerning is that there's currently no single test that can guarantee an AI has truly forgotten something,” Ms Uyen adds.

The missing ingredient: robustness

According to Ms Uyen, most existing research on LLM unlearning focuses on two questions: “Has the AI forgotten the unwanted information?” and “Can it still perform its normal tasks well?”. But a third question is often overlooked: “Does AI remain forgotten when someone actively tries to recover that information?”

Her research demonstrates that even so-called black-box attacks – where a person interacts with the AI just like any ordinary user without access to its internal parameters – can sometimes recover supposedly forgotten information.

To address the current gap, Ms Uyen's research introduces “robustness” as a critical objective for AI unlearning. In simple terms, robustness means ensuring that forgotten information remains forgotten even when someone deliberately uses different tactics to make it reveal that information again.

“Robustness is essential because if an attacker can still recover supposedly unlearned knowledge, then the fundamental goal of unlearning has not been fully achieved,” Ms Uyen pointed out.

“I envision robust evaluation to become a standard requirement, with AI systems being rigorously tested under realistic attack scenarios before they are deployed.”

Building more trustworthy AI

If organisations assume an AI system has forgotten sensitive information when it has not, the consequences could include privacy breaches, security risks, and failures to comply with legal obligations.

“Companies may believe they have met obligations such as the ‘right to be forgotten' under the EU’s GDPR, when in reality the information remains accessible,” Ms Uyen warns.

Her research suggests developers need to test AI systems under more realistic conditions rather than relying solely on simple questioning.

“Unlearning should be evaluated like a security system,” she says. “It's not enough to check that it works under normal use. You also need to test whether it can withstand real-world attacks.”

Responsibility, she argues, should not rest solely with AI companies. Independent auditors and regulators should also play a role in verifying claims that information has been removed.

“This is similar to how we certify the safety of medicines or aircraft where we don't rely solely on the manufacturer,” she says.

Looking ahead, Ms Uyen's next goal is to move from evaluating unlearning to improving it. She aims to develop a robust unlearning framework that can both remove unwanted knowledge and resist malicious attempts to recover it.

“In the future, I’d like to see AI unlearning become a standard part of how AI systems are built and evaluated, rather than an afterthought,” she says. “Unlearning is an important step towards giving people greater control over their data and ensuring AI systems can correct mistakes, remove sensitive information, and adapt to legal and ethical expectations.”

Behind every major shift is a story worth exploring. The Ripple Theory is a series of expert perspectives informed by research from RMIT University Vietnam. By connecting academic insight with real-world stories, the series helps readers gain a deeper understanding of the shifts unfolding across the economy, society, environment, and technological world, while exploring solutions and opportunities for what lies ahead.

Story: Ngoc Hoang

Masthead image: Urupong – stock.adobe.com