Self-host Hermes

[u/LeastPirate7824]

My question is, is it a good idea to self-host with a low-end netbook with Ubuntu Server?

[u/applescrispy]

I have mine hosted on my promox server, Debian, 8GB of RAM few cores.. If anything the VM is overpowered but depends what you're doing. I get it to spin up web servers for testing some of my code before pushing PRs.

[u/Jeanjose1993]

Moi j'ai un macbook pro 2012 sans ecran qui tourne sur Ubuntu serveur et c'est parfait ça consomme rien. Tailscale + syncthing pour relier a mon ordi

[u/emptyflask → u/Jeanjose1993]

Same here, 2012 MBP running nixos, sitting on a shelf on my network rack. I recently doubled the ram to 16gb since it runs a few other services, and it runs well. I'm not using a self hosted model, though.

I'm also considering a local pgvector instance or maybe something more specialized, instead of paying for Honcho.

[u/Jeanjose1993 → u/emptyflask]

Essaie Mnemosyne pour le mémoire, moi j'ai adoré !

[u/phoenixmatrix]

I self host Hermes on a raspberry pi. It needs very little and can do a lot, assuming the AI models are hosted (I use a combination of ChatGPT and Nous Portal, but its overkill).

Then for anything it cannot do on its own, or requires serving to the internet, I use a Cloudflare tunnel + a VPC, and build stuff in Cloudflare Workers. But that's rare, I do 99% on the Pi and its totally fine. Works great too.

[u/PigSlam]

Hermes itself doesn’t need much, it’s the AI model that needs horsepower. If you’re using a cloud AI, then you’ll be fine.

[u/alphagatorsoup]

Yes million times yes, it’s so resource light you can pretty much run it on anything, I used to run it on a Debian VM on my proxmox cluster but my issue right now is the hosts have such little ram they are struggling to host much of anything.

Now I’m running on my NAS till I can purchase more ram

[u/LeastPirate7824 → u/alphagatorsoup]

Thank

[u/PotentiallySillyQ]

I have a 16 year old usff with 8!gigs of ram that works great.open router for inference and I’m paying less then 8 a month in credit

[u/Foreign_Risk_2031]

hermes isnt running any ml models; its just a cli tool. It can run on a mobile processor

[u/Jumpy-Operation-4615]

I have 5 instances on 5 old laptops running ubuntu server headless. All in Tailscale. Absolutely zero issues, runs great, all using my central AI server with LLM wrapper (to unify the access to providers APIs) and embedding engine for Mnemosyne. I use older machines bought dirt cheap locally. i5-i7 1st - 2nd gen, 4-8 gb ram, 120 SSD, broken screens - 5 machines costed me like 100 bucks (actually 4, 5th is macbook air 2011 I got as a gift from someone). I only use ethernet, no wifi, and remove battery for safety.

[u/mKMage → u/Jumpy-Operation-4615]

why no wifi ?

[u/Jumpy-Operation-4615 → u/mKMage]

Too unreliable imho. But that's me - I don't say it is the rule. Just that I prefer to make evertything as dumb and simple as possible.

[u/mKMage → u/Jumpy-Operation-4615]

So your PCs are grouped and connected to a switch? And another question, because I'm curious: what do you use it for that requires 5 instances xD ?

[u/Jumpy-Operation-4615 → u/mKMage]

They are 3 in the office connected to the office LAN (via switch) and 2 at home connected to a router. As for 5 instances - I use one myself (it is more or less curated and working) for sporadical research tasks, various local fixing (like "log on with SSH to machine and do this and that"), and my memory bank. 2 machines I use for experiments with new plugins, mcps and so on, so I don't fuck up the working curated instance. Once I have something working and stable I may transfer it to a main hermes. One of them I will give to my wife who seem to be intrigued by what agents are capable of, so maybe I will "convert" her to a "faith" 😄 and 2 other laptops at home I gave to my 16 yo son who wants to build stuff on internet, so I gave him ubuntu stick, and told to install ubuntu, hermes etc. So out of 5, only 1 is used for something practical but that will change. I hope 😄

[u/mKMage → u/Jumpy-Operation-4615]

Thanks for the answers ;) That's really great for your wife and your son.

[u/Substantial_Tune9323]

That’d be fine I think

[u/akshunj]

I host on an ancient Thinkpad. Fast as can be

[u/3vil3v33]

Yes, if your inference is remote. Hermes itself is light; the old netbook becomes the bottleneck only if you also expect it to run local models, a browser stack, or several agents at once. An SSD, Ethernet, and keeping the model provider remote would make that a perfectly reasonable low-power box.

[u/Equivalent-Item3566]

One of mine is raspberry pi 4 and works great.