AI and data protection · 5 minute read

The Black-Box Blind Spot: Why the Right to Erasure Challenges Neural Networks

Deleting a record is not the same as removing its influence from a trained model. Businesses need to plan for valid erasure requests before personal data enters an AI system.

An opaque black glass cube suspended inside an interconnected silver lattice
Illustrative image

The rush to deploy AI has brought a difficult question into the boardroom: what happens when someone asks you to delete personal data that has already been used to train a model?

The familiar analogy is that you cannot unbake a cake. It captures part of the problem, but not the whole picture. Machine learning can make erasure technically difficult without making it impossible in every case. Nor does that difficulty remove the organisation's data protection responsibilities.

The right is real, but it is not absolute

Under Article 17 of the UK GDPR, the right to erasure applies in specified circumstances. These include when personal data is no longer needed, when consent is withdrawn and there is no other legal ground for processing or when the data has been processed unlawfully. An objection can also trigger erasure where the relevant conditions are met, including where there are no overriding legitimate grounds for continuing the processing.

The right is not absolute. Exceptions can apply, for example where processing is necessary to comply with a legal obligation or establish, exercise or defend legal claims. Each request needs a reasoned assessment. A business cannot assume that every request requires deletion, but it cannot dismiss a valid request simply because its chosen technology makes deletion inconvenient.

The neural network paradox

A traditional database usually lets you locate a person's record and remove it. Even there, deletion may also need to cover linked systems, copies and back-ups. An AI system adds further layers: source datasets, cleaned training files, prompts, logs, retrieval stores and the trained model itself.

Training changes the model's mathematical parameters. A person's contribution may be distributed across those parameters rather than held as a neat, separate record. Removing an entry from the original training file does not necessarily remove its influence from the model that has already been trained.

It is also misleading to say that personal data always disappears once it becomes part of a model. Some models can memorise and reproduce training information, including identifiable details. Whether a model contains personal data or can genuinely be treated as anonymous requires a careful assessment of identification and extraction risks. It is not settled by calling the data mathematical weights.

Technical complexity is not an exemption

Organisations building or fine-tuning AI need to know which parts of their system contain personal data and how valid rights requests will be handled. This means tracing information through the full pipeline rather than treating the training dataset as the only place it exists.

Possible responses may include deleting relevant records and copies, addressing information held in retrieval stores or considering retraining or machine unlearning where model-level action is required. Machine unlearning aims to remove the influence of particular training data, but its effectiveness depends on the model and the method. It is not a universal deletion button. Output filters alone do not establish that personal data has been erased.

The appropriate response depends on the processing, the request and the evidence available. Controllers remain accountable for their own decisions and processors have obligations of their own, including assisting with rights requests under the agreed processing arrangements. Buying a model from a third party does not remove the need to understand those roles and responsibilities.

Build the response before you build the model

Start with data provenance. Record where training inputs come from, establish the lawful basis for personal data and identify any additional condition needed for special category information. Public availability does not make personal data unrestricted. Permission to process it does not automatically settle copyright or licensing questions either.

Minimise personal data before ingestion and assess whether it needs to be in the model at all. Complete a Data Protection Impact Assessment where processing is likely to result in a high risk to individuals. Consider whether a less intrusive system would achieve the purpose and whether information can be genuinely anonymised before training.

Check supplier terms for retention, training reuse, deletion and assistance with rights requests. Test your proposed response, record its limitations and keep evidence of the action taken. Requests must generally be answered without undue delay and within one calendar month, subject to applicable extensions and exceptions.

How Loopol can help

At Loopol, we help businesses and developers cut through the technical hype and put practical AI governance in place. We can review your AI uses, map the information involved and help build acceptable use policies, supplier checks and clear request-handling processes.

Our AI Governance and Acceptable Use Set-up provides a governance framework, a risk review of your AI tools and practical staff controls. Buy the service below or contact Loopol to discuss model development, data flows or erasure concerns. Specialist technical testing, model retraining, DPIAs and ongoing request management are scoped separately. We do not promise that a policy alone can solve a model's technical limitations.

Tailored to your business

Put practical AI governance in place

AI Governance and Acceptable Use Set-up: £1,500

An AI governance framework built with you: acceptable use policy, a risk review of the AI tools your business uses and practical controls for staff. Wider assessments and ongoing support are agreed separately. No VAT is added while Loopol is not VAT registered.

Contact Loopol

helloloopol@gmail.com

All articles