Europol has supported authorities from 19 countries in a large-scale hit against child sexual exploitation that has led to 25 arrests worldwide. The suspects were part of a criminal group whose members were engaged in the distribution of images of minors fully generated by artificial intelligence (AI).
Open-source models exist and can be forked
…and then we’re back at “someone can take that model and tag real images to appear AI-generated.”
You would need a closed-source model run server-side in order to prevent that.