lemmyreader@lemmy.ml to Security@lemmy.mlEnglish · 11 months agoAmazon storing classified US government documents improperlylemmy.mlexternal-linkmessage-square76fedilinkarrow-up1557arrow-down124file-text
arrow-up1533arrow-down1external-linkAmazon storing classified US government documents improperlylemmy.mllemmyreader@lemmy.ml to Security@lemmy.mlEnglish · 11 months agomessage-square76fedilinkfile-text
minus-squareAmbiguousPropslinkfedilinkEnglisharrow-up7·11 months agorobots.txt doesn’t have to be followed. It doesn’t block crawling.
robots.txt doesn’t have to be followed. It doesn’t block crawling.