A sex offender convicted of making more than 1,000 indecent images of children has been banned from using any “AI creating tools” for the next five years in the first known case of its kind.

Anthony Dover, 48, was ordered by a UK court “not to use, visit or access” artificial intelligence generation tools without the prior permission of police as a condition of a sexual harm prevention order imposed in February.

The ban prohibits him from using tools such as text-to-image generators, which can make lifelike pictures based on a written command, and “nudifying” websites used to make explicit “deepfakes”.

Dover, who was given a community order and £200 fine, has also been explicitly ordered not to use Stable Diffusion software, which has reportedly been exploited by paedophiles to create hyper-realistic child sexual abuse material, according to records from a sentencing hearing at Poole magistrates court.

  • xmunk@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    2
    arrow-down
    20
    ·
    8 months ago

    It isn’t misinformation, though, generative AI needs a basis for it’s generation.

    • rebelsimile@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      25
      arrow-down
      2
      ·
      edit-2
      8 months ago

      The misinformation you’re spreading is related to how it works. A generative AI system will (without prompting away from it) create people with 3 heads, 8 fingers on each hand and multiple legs connecting to each other. Do you think it was trained on that? This argument of “it can generate it, therefore it was trained on it” is ridiculous. You clearly don’t understand how it works.

      • xmunk@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        12
        ·
        8 months ago

        You’re extremely correct when it comes to combining different aspects of existing works to generate something new - but AI can’t generate something it doesn’t know about. If a generative model knows what a prepubescent naked body looks like it has been exposed to them before. The most generous way to excuse this is that medical diagrams exist and supplied the majority of inputs for any prompts about cp to work off of. A must more realistic view is that some cp made it into the training set.

        I don’t disagree with any of your assessments but if you wanted a Van Gogh painting of a Glorp from Omnicron Persei 8, you’ll get out… something, but because the model has no reference for Glorps it’ll be hallucinations or guesses based on other terms it can find.

        To be clear, I’m coming at this from the angle as someone who has trained and evaluated models in a company that’s used them for the better part of a decade.

        I understand I’m going up against your earnestly held belief, but I’ve seen behind the curtain on a lot of this stuff and hopefully in time the way it works becomes demystified for more people.

        • Leate_Wonceslace@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          6
          arrow-down
          1
          ·
          8 months ago

          For reference, the comment I made was improperly displayed, and I thought I replied to the wrong person. It said:

          Hi, I’m a mathematician that’s been following the development of generative neural networks for about a decade or more.

          You’re wrong. Your knowledge of the inner workings of these AI is accurate, but somehow you’ve reached an incorrect conclusion. I sometimes run a local instance of Stable Diffusion on my home PC, and it can make things that have never existed look totally unlike anything it’s ever seen, and yet match certain specifications in principle.

          I don’t use it to generate porn, so I can’t speak to the difficulties in avoiding csam while doing so. Mostly I generate is paintings in the style of Van Gogh, and it does a remarkable job of doing so, even when I can’t get it to do what I want. For example: it generated a painting of him in profile wearing armor when I asked for a weapon. I don’t think Van Gogh ever painted himself in profile, and he certainly never did so in armor. And yet the model was capable of imagining what this human-like figure so closely associated with the artist style “Van Gogh” would look like in profile because it knew what humans tend to look like in profile, and it could conceptualize how the features would present themselves. I’m certain that an AI can imagine a convincing image of simulated csam without ever having seen it, because these models really are just that good at imagining new things.

          • PotatoKat@lemmy.world
            link
            fedilink
            English
            arrow-up
            2
            ·
            8 months ago

            Has your model seen humans in a profile view? Has it seen armor? Has it seen Van Gogh style paintings? If yes then it can create a combo of those things.

            For CSAM it needs to know what porn looks like, what a child looks like and what a naked pubescent body looks like to create it. It didn’t make your van Gogh painting from nothing it had an idea of what those things were.

            • Leate_Wonceslace@lemmy.dbzer0.com
              link
              fedilink
              English
              arrow-up
              2
              ·
              edit-2
              8 months ago

              it can create a combo of those things

              Yes, that’s my point. It didn’t need to be trained on a portrait of Van Gogh in profile; it had several portraits of Van Gogh, a bunch of faces in profile, and used them to create something new. In the exact same way, a network trained on photos of people that include nude adult bodies and children in innocent situations can feasibly create facsimiles of csam without ever having been trained on it.

            • xmunk@sh.itjust.works
              link
              fedilink
              English
              arrow-up
              1
              arrow-down
              1
              ·
              8 months ago

              Yea, specifically, the model shouldn’t have had access to a significant training set on naked prepubescent bodies - that’s been my main objection in this thread.

                • xmunk@sh.itjust.works
                  link
                  fedilink
                  English
                  arrow-up
                  3
                  arrow-down
                  1
                  ·
                  8 months ago

                  Awesome link, I’ll share it up thread where someone was asking for it. Yea, it’s something that’s hard to prove since models aren’t upfront with how they’re sourcing their data.

              • Leate_Wonceslace@lemmy.dbzer0.com
                link
                fedilink
                English
                arrow-up
                3
                ·
                8 months ago

                Are you paying attention? It didn’t need to be trained on a portrait of Van Gogh in profile; it had several portraits of Van Gogh, a bunch of faces in profile, and used them to create something new. In the exact same way, a network trained on photos of people that include nude adult bodies and children in innocent situations can feasibly create facsimiles of csam without ever having been trained on it.

                • xmunk@sh.itjust.works
                  link
                  fedilink
                  English
                  arrow-up
                  3
                  arrow-down
                  2
                  ·
                  8 months ago

                  The model should not have had access to naked prepubescent imagery. If it did, that’s a problem. My argument in this thread is that it did have access to csam and thus is able to regurgitate them.

                  I honestly think you and I are in agreement. I’m not arguing that the model is regurgitating known csam but the model ingested csam[1] and the output is derived from that csam. The fact that it can now make csam in the style of Van Gogh is a property of how these models can combine motifs… the fact that it understands how to generate csam at all is the problem.

                  1. https://cyber.fsi.stanford.edu/news/investigation-finds-ai-image-generation-models-trained-child-abuse
                  • Leate_Wonceslace@lemmy.dbzer0.com
                    link
                    fedilink
                    English
                    arrow-up
                    1
                    arrow-down
                    2
                    ·
                    8 months ago

                    Ah, I see. I’m sorry; I misunderstood your argument. Yes, given the fact that csam is part of the training data, it would likely be able to reproduce it. I thought your argument was the reverse hypothetical: “If the model is able to produce csam then it must have been trained on csam.” which is incorrect. Again, my apologies for misunderstanding.

                • PotatoKat@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  2
                  arrow-down
                  3
                  ·
                  edit-2
                  8 months ago

                  The bodies of children are not just small versions of adult bodies.There are meaningful differences that an ai wouldn’t be able to just guess. Also do you not see any problem in using photos of real children to generate csam? Imagine someone used a picture of your child/niece/nephew to generate porn. Does that not feel wrong to you? It’s still using real photos of real children either way, even if it’s abstracted through training data.

                  • Leate_Wonceslace@lemmy.dbzer0.com
                    link
                    fedilink
                    English
                    arrow-up
                    2
                    arrow-down
                    1
                    ·
                    8 months ago

                    do you not see any problem

                    I’m discussing hypotheticals of cause-and-effect, not ethics. The question is if it possible not if it’s moral to do so. Please don’t try to shift the topic or try to portray me as possessing an opinion I don’t have again.

                    meaningful differences that an ai wouldn’t be able to just guess

                    While I am aware that there are such differences, I don’t think it’d be impossible for AI to guess them accurately. Lack of training data would make such less probable, since it’d be less likely to know which nude forms better approximate a realistic depiction of the imagined subject. Essentially, certain distributions of outputs have different probabilities depending on if the training data has csam, but due to the diversity of adult bodies it becomes possible for the model to stumble upon a convincing facsimile. How the images of nude adults are labeled can also impact these distributions.

          • xmunk@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            4
            arrow-down
            2
            ·
            8 months ago

            Because it’s a transformation that can be accurately predicted, at least as far as we can conceive. This is sort of the problem with this thread - there are plenty of examples of derivative combinations that are being presented as counter examples but naked children don’t just look like adults scaled down. This is a rather unique situation because most people have been parents or siblings and know what naked children look like but photographs of that nudity are restricted and shouldn’t be included in model training.

            The other example we might have to work with would be copywrited material but we know that models did consume material they weren’t licensed to - as a result AI has been able to generate Disney characters and the like in a recognizable way.