lemmyreader to Security@lemmy.mlEnglish • 3 months agoAmazon storing classified US government documents improperlylemmy.mlimagemessage-square78fedilinkarrow-up1556arrow-down124file-text
arrow-up1532arrow-down1imageAmazon storing classified US government documents improperlylemmy.mllemmyreader to Security@lemmy.mlEnglish • 3 months agomessage-square78fedilinkfile-text
minus-squareAmbiguousPropslinkEnglish7•3 months agorobots.txt doesn’t have to be followed. It doesn’t block crawling.
robots.txt doesn’t have to be followed. It doesn’t block crawling.