🔗 Share this article

Technology companies and child protection agencies will receive permission to assess whether AI systems can generate child abuse material under new UK legislation.

Substantial Rise in AI-Generated Harmful Content

The announcement coincided with revelations from a protection watchdog showing that reports of AI-generated child sexual abuse material have more than doubled in the last twelve months, rising from 199 in 2024 to 426 in 2025.

New Legal Structure

Under the changes, the authorities will permit designated AI companies and child protection groups to examine AI models – the underlying technology for chatbots and image generators – and ensure they have adequate safeguards to prevent them from creating images of child exploitation.

"Ultimately about preventing exploitation before it occurs," stated Kanishka Narayan, adding: "Experts, under strict protocols, can now identify the risk in AI models early."

Tackling Regulatory Obstacles

The changes have been introduced because it is illegal to create and own CSAM, meaning that AI developers and other parties cannot create such content as part of a testing process. Until now, authorities had to delay action until AI-generated CSAM was uploaded online before dealing with it.

This law is designed to preventing that problem by helping to halt the production of those materials at source.

Legal Framework

The changes are being added by the government as revisions to the crime and policing bill, which is also establishing a ban on possessing, producing or distributing AI systems designed to generate child sexual abuse material.

Practical Impact

This recently, the official visited the London headquarters of a children's helpline and heard a mock-up call to counsellors featuring a account of AI-based exploitation. The call depicted a teenager requesting help after being blackmailed using a explicit AI-generated image of themselves, constructed using AI.

"When I hear about young people experiencing blackmail online, it is a cause of intense frustration in me and justified concern amongst families," he stated.

Alarming Data

A prominent online safety organization stated that instances of AI-generated abuse material – such as online pages that may contain multiple images – had more than doubled so far this year.

Instances of the most severe content – the gravest form of abuse – increased from 2,621 images or videos to 3,086.

• Girls were overwhelmingly victimized, making up 94% of illegal AI images in 2025
• Portrayals of newborns to toddlers increased from five in 2024 to 92 in 2025

Industry Response

The legislative amendment could "represent a vital step to ensure AI tools are secure before they are launched," stated the head of the internet monitoring foundation.

"AI tools have enabled so survivors can be targeted repeatedly with just a simple actions, providing offenders the capability to create possibly limitless amounts of advanced, lifelike child sexual abuse material," she added. "Content which additionally commodifies survivors' trauma, and renders young people, especially female children, less safe both online and offline."

Support Interaction Information

Childline also published details of support interactions where AI has been mentioned. AI-related harms discussed in the conversations include:

• Using AI to rate weight, body and appearance
• Chatbots discouraging children from consulting safe guardians about harm
• Facing harassment online with AI-generated content
• Online extortion using AI-manipulated pictures

Between April and September this year, the helpline delivered 367 support interactions where AI, conversational AI and related topics were discussed, four times as many as in the same period last year.

Half of the references of AI in the 2025 sessions were connected with psychological wellbeing and wellness, including utilizing chatbots for assistance and AI therapy applications.