It’s supposed to be a dot (.) character. The project’s name is n.eko.

You have a point here.

But when you consider the current worlds web traffic, this isn’t actually the case today. For example Gnome project who was forced to start using this on their gitlab, 97% of their traffic could not complete this PoW calculation.

IE - they require only a fraction of computational cost to serve their gitlab, which saves a lot of resources, coal, and most importantly, time of hundreds of real humans.

(Source for numbers)

Hopefully in the future we can move back to proper netiquette and just plain old robots.txt file!

If you remember the project I would be interested to see it!

But I’ve seen some AI poisoning sink holes before too, a novel concept as well. I have not heard of real world experiences of them yet.

They’re working on no-js support too, but this just had to be put out without it due to the amount of AI crawler bots causing denial of service to normal users.

1. Doesn’t run against Firefox only, it runs against whatever you configure it to. And also, from personal experience, I can tell you that majority of the AI crawlers have keyword “Mozilla” in the user agent.

2. Yes, this isn’t cloudflare, but I’m pretty sure that’s on the Todo list. If not, make an issue to the project please.

3. The computational requirements on the server side are a less than a fraction of the cost what the bots have to spend, literally. A non-issue. This tool is to combat the denial of service that these bots cause by accessing high cost services, such as git blame on gitlab. My phone can do 100k sha256 sums per second (with single thread), you can safely assume any server to outperform this arm chip, so you’d need so much resources to cause denial of service that you might as well overload the server with traffic instead of one sha256 calculation.

And this isn’t really comparable to Tor. This is a self hostable service to sit between your web server/cdn and service that is being attacked by mass crawling.

Edit: If you don’t like the projects stickers, fork it and remove them. This is open source project.

And Xe who made this project is quite talented programmer. More than likely that you have used some of Xe’s services/sites/projects before as well.

Yes, Anubis uses proof of work, like some cryptocurrencies do as well, to slow down/mitigate mass scale crawling by making them do expensive computation.

https://lemmy.world/post/27101209 has a great article attached to it about this.

–

Edit: Just to be clear, this doesn’t mine any cryptos, just uses same idea for slowing down the requests.

In general, to everyone who finds Yacy as an interesting project, just give it a try!

It’s relatively light weight, and having millions of pages indexed does not take that much disk space, in my case: 3.5 million indexed pages is around 200 gigabytes only.

Yacy is far from perfect, and it’s an ancient project. But it’s still alive and kicking strong!

Hi!

I’ve been selfhosting Yacy for some years, even tho I rarely use it (I’m mostly using Kagi these days).

But some tips:

• Set up something like this to your browser, this sends Yacy to crawl pages that you visit https://github.com/JeremyRand/YaCyIndexerGreasemonkey .
• Get familiar with blacklists and try to find some public ones to filter out bad sites and adult content.
• Tinker with Ranking & heuristics -> Solr boosts to get results that fit your use case more.
• And in general, tinker with all the settings you can find!

And not directly Yacy related, but you can use your own Yacy through Searxng as well, even in ‘private’(non P2P) mode.

Yacy.