The Arch Wiki (and possibly other Arch websites, but when I checked I wasn't able to catch it) uses Anubis (https://anubis.techaro.lol/) to mitigate AI crawlers causing issues on their servers. Anubis works by making your computer do some computations before being let in. For users like you and me, it'd be a minor inconvenience, but for these AI crawlers, it'd cost them a lot of computation as they hit these sites a huge number of times.
Using Anubis is said to be a nuclear option, but as these AI crawlers don't care about that, sites, especially FOSS ones, started using it.
9 Comments
MyNameIsRichard@lemmy.ml · 11 pts · 1y
I've had a brief look around and it seems fine for me.
the_citizen@lemmy.world · 3 pts · 1y
So probably it's about my network. Thanks for reply.
N0x0n@lemmy.ml · 8 pts · 1y
The wiki, it's because they changed the waifu to a stupid gear !
https://anubis.techaro.lol/
megane_kun@lemmy.dbzer0.com · 7 pts · 1y
The Arch Wiki (and possibly other Arch websites, but when I checked I wasn't able to catch it) uses Anubis (https://anubis.techaro.lol/) to mitigate AI crawlers causing issues on their servers. Anubis works by making your computer do some computations before being let in. For users like you and me, it'd be a minor inconvenience, but for these AI crawlers, it'd cost them a lot of computation as they hit these sites a huge number of times.
Using Anubis is said to be a nuclear option, but as these AI crawlers don't care about that, sites, especially FOSS ones, started using it.
b_tr3e@feddit.org · 2 pts · 1y
Because it has to be recompiled for every update.
Cyber@feddit.uk · 6 pts · 1y
Wouldn't that be Gentoo's website?
the_citizen@lemmy.world · 1 pts · 1y
Oh I understand it now. Thank you for information.
just_another_person@lemmy.world · 1 pts · 1y
Website, or package repos? Two different things.
the_citizen@lemmy.world · 1 pts · 1y
Website.