- cross-posted to:
- selfhosted@lemmy.world
- cross-posted to:
- selfhosted@lemmy.world
Crossposted from https://lemmy.ml/post/50758732
Hi everyone,
I am the original author of Searx. I started Hister with a similar motivation: reducing our dependence on external search engines while keeping searches and personal data under our control.
Searx is a metasearch engine that forwards queries to other search providers. Hister takes a different approach. It builds a private full text index from content you choose, then searches that index entirely on your own infrastructure.
Hister can automatically index pages through its Firefox and Chrome extensions. It can also watch local directories, import browser history and bookmarks, index individual URLs, and crawl complete documentation sites.
The feature I find most useful is offline previews. Hister stores the readable content and HTML of indexed pages locally. You can open a result in a clean and sanitized preview beside the search results without visiting the original website again.
Some other features:
- Full text search across web pages, PDFs, docx files, Markdown, OrgMode and text files
- Phrase searches, field filters, date filters, wildcards, negation, aliases, labels, facets, and result priorities
- Optional semantic search using an embeddings endpoint you configure
- Persistent website crawls
- Imports from browser history, Linkwarden, Karakeep, Shaarli, Wallabag, and Linkding
- Web, terminal, command line, HTTP API, and MCP interfaces
- SQLite and PostgreSQL support, plus optional multiple user hosting
Hister cannot replace a global search engine (yet) for subjects you have never encountered because it only searches what you have indexed. My workflow is to search Hister first, then use its shortcut to fall back to traditional search when I need broader web results.
The project is free software under the AGPLv3+ license. It can be installed as a standalone binary or with Docker.
Project: https://github.com/asciimoo/hister
Website and documentation: https://hister.org/
Small read-only demo: https://demo.hister.org/
I’d appreciate feedback, questions, and suggestions as well as joining our growing community.
AI disclosure: AI assisted contributions are not strictly prohibited, but all contributions should be made by humans. More details: https://github.com/asciimoo/hister/blob/master/CONTRIBUTING.md#ai-policy
You must log in or register to comment.
Being able to navigate only through the content of my favorites websites, oh my, excellent news. I’m truly eager to try it.
I have a newbie question tho : I would like to use Hister while still supporting the people who create website content. Say I’m online and don’t wish to use the local preview, can I access to the website instead of loading my local content ? If I access the website via Hister instead of Searx for instance, would my request be added to their traffic numbers, or not ? (those lowering their referencing and their visibility).
(Not like my individual usage would change anything anyway but still asking to get a graspe of what it implies).
Thank you for the hard work !
The results are standard search results, if you click on the title, it opens the original URL, just like in any other regular search engine. You can even close the preview pane if you are not interested in it.
I realize my question was more about direct links as a whole and the way they can impact website visibility (because no click registered by search engines), than about Hister specifically. Did some searching in the meantime.
In any case thank you for your work and your answer ! Time to set up now. 😈
I set hister up for a while, completely forget about it and checked it again after reading this Lemmy post. Well there are now 6.600 pages and 480 MB RAM usage.
Looks like I need to integrate hister more into my web usage. But maybe I need some training. It isn’t possible to setup hister as my default search engine in browser with automated fallback to normal Meta-search with whatever engine? And I use login credentials maybe a faster way like passkey possible in the future?
It isn’t possible to setup hister as my default search engine in browser with automated fallback to normal Meta-search with whatever engine?
It can be added as a default search engine and it provides a link and a hotkey to open the search query with an external search engine, however I have a different workflow: https://hister.org/posts/how-i-use-hister
And I use login credentials maybe a faster way like passkey possible in the future?
That’s a good idea, thanks for the suggestion.
Yay, bindsym, good to see i3 again, Bad that I switched to KDE, but there is an equivalent. Anyway it’s a habit changer. And Lemmy is the perfect place for your app, because here are the last people that know how to change habits nowadays. I’ll give it a chance again. Default search engine is a good starting point for me.
it’s a habit changer.
It took some time to get used to it, but I find it more efficient, to spawn a Hister with a single hotkey and use the URL bar with a traditional search engine when I know that I’m looking for something new. But, of course this is subjective, everbody has their preferred workflows, I shared mine as an inspiration.
Btw, you can start your query with
!!, in this case Hister automatically redirects you to your external search engine with the rest of the search query.
Very cool! Is there a place to share index DB? Having a pre-fetched index on awesome lists and fmhy would be amazing!
I’ve started creating smaller, pre-fetched, thematic datasets: https://hister.org/datasets . It is still very rudimentary, but the concept is the same. Both datasets and improvement ideas are appreciated.
Ver cool!
Big fan of searx and use it on all of my devices. This will be a welcome addition.
What are the Hardware requirements? I imagine the index will become quite large, no?
The storage requirement is around 100KB/page on average.
Memory usage can exceed 1GB momentarily for searches when using language detection and multi-language indexes (it is the default config). Without language detection Hister has a much smaller memory footprint (~30MB default with ~100-150MB peaks).
I’ve been slowly migrating from buku to hister, but it’s still early days. I do like þat hister indexes sites, which beats having to manually tag everyþing. So far it’s looking pretty good, þough. Þanks for writing it.
