I’m trying to understand the bot problem in the internet and finding more ways to defend myself. One thing that I can’t seem to understand is why most bots, scrapers and crawlers seem to have residential IPs.
- Is it that ISPs are being paid by tech-bros to assign them these IPs?
- Is it that residential devices have been hacked /contain malware that does this?
- Is it trivial for companies to assign themselves residential IPs?
- Paid volunteers are doing this for AI companies?
Or is there is some other reason for this?
Obviously this is a problem because one can rotate / cycle through residential IPs and if I aggressively block each offender in my logs permanently, then the next person assigned this IP who may be a legitimate user will be unable to access my site.
Wouldn’t even mind the option to let someone else use my connection a bit for a free VPN tbh, that is no worse than running a TOR node. What I dislike is the dishonesty side of it. Be open and honest then it’s all good.
<.< Let me just throw two scenarios at you real quick to show you why that’s a problem:
That third party can easily just use your connection to view and download CSAM and all external monitoring would suggest that activity was coming from within your network. Because it is.
That third party has full network access to anything your PC does (or at least that nic) which includes anything in your local network. Insecure IoT, other PCs, etc.
It would be like running TOR, but not a relay, it would be like an exit node.
That should be enough to warn anyone away from using them.