• muelltonne@feddit.org
    link
    fedilink
    English
    arrow-up
    4
    ·
    2 months ago

    If it is counting website visits, I’m wondering how they are filtering out bots using selenium on a linux system to crawl their sites. That should be a huge amount of traffic

    • drphungky@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      2 months ago

      Anyone using bots that are openly bots would be limiting their shit to robots.txt. Anyone who is out trawling the whole website is probably obfuscating what they’re doing by spoofing new user agents.