Drunk & Root@sh.itjust.works to Selfhosted@lemmy.worldEnglish · 2 days agoHow to combat large amounts of Ai scrapersmessage-squaremessage-square52fedilinkarrow-up1115arrow-down14file-text
arrow-up1111arrow-down1message-squareHow to combat large amounts of Ai scrapersDrunk & Root@sh.itjust.works to Selfhosted@lemmy.worldEnglish · 2 days agomessage-square52fedilinkfile-text
everytime i check nginx logs its more scrapers then i can count and i could not find any good open source solutions
minus-squareSayCyberOnceMore@feddit.uklinkfedilinkEnglisharrow-up6·1 day agoIf you’re able to, use GeoIP ranges to only allow access from the countries you want. That immediately limits a lot of everything Then - again if you’re able to - use a block list that covers known scrapers in case they’re in your country. I use pfBlockerNG on my pfSense firewall for exactly this.
If you’re able to, use GeoIP ranges to only allow access from the countries you want.
That immediately limits a lot of everything
Then - again if you’re able to - use a block list that covers known scrapers in case they’re in your country.
I use pfBlockerNG on my pfSense firewall for exactly this.