I’m trying to understand the bot problem in the internet and finding more ways to defend myself. One thing that I can’t seem to understand is why most bots, scrapers and crawlers seem to have residential IPs.
- Is it that ISPs are being paid by tech-bros to assign them these IPs?
- Is it that residential devices have been hacked /contain malware that does this?
- Is it trivial for companies to assign themselves residential IPs?
- Paid volunteers are doing this for AI companies?
Or is there is some other reason for this?
Obviously this is a problem because one can rotate / cycle through residential IPs and if I aggressively block each offender in my logs permanently, then the next person assigned this IP who may be a legitimate user will be unable to access my site.
It would be like running TOR, but not a relay, it would be like an exit node.
That should be enough to warn anyone away from using them.
Running an exit node is perfectly legal though. There would be no evidence you have done anything wrong very quickly.
You can just look at the testimonies from others who have run exit nodes. The cost of your “free” VPN is that law enforcement will constantly be in contact and investigating you because your network/machine is being used to download CSAM.
There is no “oh don’t worry, A.B.C.D is just a tor node, we can give it a pass”. Every time that happens, it has to be treated with a full investigation.