Warning: Some posts on this platform may contain adult material intended for mature audiences only. Viewer discretion is advised. By clicking ‘Continue’, you confirm that you are 18 years or older and consent to viewing explicit content.
Plus how do you spot the difference between a good bot and a bad bot? Web crawlers from search engines are for example inherently good, so they should still be able to operate, but if it is easy to register a good bot in WEI, it is also easy to register a bad bot. If it is hard to register a good bot, then you’re effectively gatekeeping the automated part of the internet (something that actually might be Google’s intention).
Plus how do you spot the difference between a good bot and a bad bot? Web crawlers from search engines are for example inherently good, so they should still be able to operate, but if it is easy to register a good bot in WEI, it is also easy to register a bad bot. If it is hard to register a good bot, then you’re effectively gatekeeping the automated part of the internet (something that actually might be Google’s intention).
I was thinking the same thing about Google wanting their bots to be the only ones allowed to crawl and index the internet.
A bot that only reads your website is good, one that posts things or otherwise changes your database less so.