GitHub - searxng/searxng: SearXNG is a free internet metasearch engine which aggregates results from various search services and databases. Users are neither tracked nor profiled.
github.com
SearXNG is a free internet metasearch engine which aggregates results from various search services and databases. Users are neither tracked nor profiled. - searxng/searxng

All the posts about Reddit blocking everyone except Google and Brave got me thinking: What if SearNGX was federated? I.E. when data is retrieved via a providers API, that data is then federated to all other instances.

It would spread the API load out amongst instances, removing the API bottlenecks that come from search providers.

It would allow for more anonymous search, since users could cycle between instances and get the same results.

Geographic bias would be a thing of the past.

Other than ActivityPub overhead and storage, which could be reduced by federating text-only content, I fail to see any downside.

Thoughts?

  • BuelldozerAEnglish
    8·
    5 months ago

    There’s only been about 700 yacy peers online in the last 30 days which is pretty low for a “crowd sourced” search engine, especially when many of those are, I think, temporary peers that come and go. It looks like it has only maybe 200 “master” servers which wouldn’t be nearly enough to keep up with the Internet these days.

    The good news is that if there’s websites / urls that you care about you can point your own yacy instance at them and schedule the crawls to keep up with content changes.

    I remember reading about yacy some years ago and now that I’ve bumped it into again it’s sparked my interest. I may stand up a docker instance and play with it for awhile. If nothing else it could make a very useful “arrrrr” search engine.