In the age of AI, it's easy to make deepfake porn. But victims find it hard to undo the damage

“The scandalise was something I didn’t await.” 

That’s how L., a 21-year-old jurisprudence educatee at the University of Hong Kong, remembers intuitive feeling the moment she discovered another student had created pornographic images of her without consent using artificial intelligence.

Initially, she blamed herself, wondering if she had done something he had misunderstood. 

The laptop contained around 20 folders of women, which included screenshots from their social media taken without their consent and explicit photos depicting them naked or doing sexual acts, created using AI tools, L., B. And H. Said.

The three HKU law students decided to go public about the incident in July. They set up an Instagram account to publish a letter detailing what had happened.

They said that after being questioned by his ex-partner, the male student admitted using photos of the other students, screenshotted from social media, to create AI-generated pornographic images with their faces.

H. said the University of Hong Kong initially asked him to write an apology letter, which he did, and sent him a warning letter to be kept in his personal file for internal reference. H. didn't think this went far enough in holding him accountable.

John Lee, who leads the city of Hong Kong, even weighed in on the case, pledging in July to research international “best practices” on regulating AI.

The notorious AI porn kingpin with a double life

The case points to a worldwide problem when it comes to policing and regulating AI porn made without consent.

“Once [someone has] possession of these images, you won't know how they would be used. They could be used personally, or they could be spread underground or publicly," said H. "You cannot control [it]."

B. Said there was also a chance the perpetrator’s computer could be hacked, so whether or not the male student wanted to publish the photos, there was still a risk the images could become widely available.

“Criminalization is a signal to the public that this action is publicly wrong,” B. Added.

In 2021, Hong Kong passed legislation that covered publishing or threatening to publish intimate images or videos without consent, including “deepfakes," which are images or videos that have been digitally altered using AI to replace the face of one person with another. The offences are punishable by up to five years in jail. 

However, that doesn't include their creation or possession. Legal experts say that loophole needs to be closed.

Sharing non-consensual deepfake porn isn't a crime in Canada. Prime Minister Mark Carney pledged to pass a law that would criminalize “producing and distributing non-consensual sexual deepfakes” during his 2025 federal election campaign.

South Korea passed a law last year that criminalizes not only the possession but the consumption of such content. The United Kingdom has also made the creation or distribution of sexually explicit deepfakes a criminal offence, following a surge in incidents in recent years.

"Nudify" apps typically use AI to create fake images of people without their consent. Pop star Taylor Swift and U.S. Democratic politician Alexandria Ocasio-Cortez are among the victims of non-consensual explicit deepfakes. 

She said the issue is increasingly widespread and pointed to the work of San Francisco city attorney David Chiu, who took legal action in August 2024 against 16 of the most-visited websites creating AI-generated non-consensual explicit images.

His office said the websites targeted in the lawsuit had been visited more than 200 million times in the first half of 2024. In June of this year, his office said 10 of the sites were now offline or no longer accessible in California. The attorney’s office named some of the companies that operated the websites, including U.S.-based Sol Ecom and Briver and U.K.-based Itai Tech Ltd.

McGlynn said even more mainstream AI tools, such as Elon Musk's Grok chatbot, have “very few guardrails." According to a report by The Verge, Grok Imagine's new "spicy" mode "didn't hesitate to spit out fully uncensored topless videos" of Taylor Swift without even being asked to make explicit content.

Meanwhile, OpenAI’s popular ChatGPT is going to allow erotic content soon, according to OpenAI boss Sam Altman.

Canada has been debating regulating platforms, such as forcing companies to take down content deemed to be child sexual abuse or intimate photos and videos shared without consent, including deepfakes.

McGlynn believes if governments look into the large platforms, they wouldn't need to worry as much about the smaller ones.

“Those are the gateways, the pipeline. It's on TikTok. The boys see the nudify apps advertised.... If we stopped that, then we could really deal with this,” McGlynn said. 

Vince Chan from the Hong Kong-based Association Concerning Sexual Violence Against Women said it was becoming increasingly difficult to identify content that was AI-generated, as the technology has advanced quickly.

For example, he said that just in the last two years, it has become harder to identify face-swapping onto existing pornographic content.

“Glitches and distortions were more obvious [a couple of years ago], but now, the results can be very lifelike unless the content is scrutinized very closely,” he said.

In the past year, his organization received 11 requests for help involving deepfake intimate images, a 38 per cent increase from the year before.

Hong Kong's privacy watchdog has launched a criminal investigation into the case of the University of Hong Kong law students. But barrister Michelle Wong, who works with victims of sexual assault and harassment, said the privacy commissioner's power is limited.

She says their primary focus is the disclosure or misuse of personal data, which makes it difficult to fit this case into their remit.

Wong said Hong Kong criminalizes creating fake photos of children under the age of 16, and therefore there's a system that caters to these kinds of offences.

“I believe the original legislative intent is that children need to be protected. But obviously now we may see a trend saying that actually, adults above 16 should also be protected by the same legal framework,” she said.

W., 30, who was a victim of AI porn about five years ago and asked to remain anonymous because she doesn't know the identity of the perpetrator, remembers when her friends forwarded photos they received through a direct message on social media, asking if it was her. 

W. Said at the time, she cried every day and had nightmares. She skipped work, too afraid to leave the house because she was worried the person who sent the photos knew where she lived. 

She also didn't report it to the police, because she was wary they would handle her case sensitively. 

W. believed the perpetrator took photos from her social media account. Four years later, she still wonders about their identity and wishes they would be arrested. She remains nervous of men she doesn’t know approaching her.

While she's glad there's growing awareness of AI-generated pornography, she feels helpless about what can be done about it.

“The internet feels quite lawless," she said. "Even if a law is implemented, how it’s enforced remains a larger question.”

---

---