Korean Police Arrest 419 in Deepfake Crackdown Over 7 Months

South Korean police arrested 400 suspects in deepfake crimes over the first seven months of this year, deploying a three-layered response system that combines detection software, expert video analysis and AI forensic techniques.

Police arrested 419 suspects in fake-video crimes between January and July and detained 17 of them, the National Investigation Headquarters of the Korean National Police Agency said on the 13th. To counter increasingly sophisticated deepfake crimes, police have built and operate a three-stage response system: screening with deepfake detection software, review by forensic video analysts, and analysis of the AI tools used to commit the crimes.

In March, the police cyber crime reporting system received a series of complaints that sexually explicit fake videos, made by superimposing victims' photographs, had been posted on overseas adult websites. After securing evidence in cases filed with four regional police agencies, police asked the distribution sites to delete the offending accounts and posts.

An analysis of the circulated videos using deepfake detection software then found the same characteristics across the clips. Police, working on the premise that the same person had produced the videos, conducted an intensive investigation, arrested a male suspect in his 20s and referred him for detention.

AI forensic technology is also being used in investigations. After referring a man in his 20s for detention on charges of producing and selling sexually exploitative fake videos through social media, police discovered fake videos with a different pattern from earlier footage while investigating those who had commissioned the videos. Following an initial screening with detection software, they applied AI digital forensic techniques to additionally identify the generative AI program used to create the videos and even the original photographs before editing. Police are investigating the commissioning party's production charges based on this evidence.

The deepfake detection software used by police determines whether an image has been synthesized by comprehensively analyzing signs of face generation or swapping, biological features such as eye blinking and blood-flow changes, and the structure of file metadata. It also carries an AI model specialized in detecting fake videos in South Korea. Since its development, it has been used in a total of 1,636 investigations, and this year it has been used in 72 cases in other investigative areas such as election crimes.

Videos in which the software cannot clearly determine manipulation are further examined by forensic video analysts. Specialist analysts verify tampering or alteration using a separate video-analysis program.

At the digital forensics stage, police use an in-house "AI forensic technique." Specialist analysts at regional police agencies technically reconstruct the production and distribution process, from which program a suspect used to manipulate a video to which part of the video was altered. Police use this to confirm additional offenses and as evidence to prove intent.

Going forward, police plan to expand deepfake detection technology beyond digital sex crimes to crimes that misuse AI more broadly, including the spread of disinformation and telecommunications financial fraud.

"We are advancing our response technology faster than the pace at which the technology itself is evolving," the National Investigation Headquarters said. "Through upgrades to related systems and intensive crackdowns, we will respond strongly to fake-video crimes under a zero-tolerance principle."

Original reporting by Lee Yu-jin for Seoul Economic Daily.

AI-translated from Korean. Quotes from foreign sources are based on Korean-language reports and may not reflect exact original wording.

Watch · Seoul Economic Daily