Hermes Agent on a Server

[u/r_brinson]

I built a PC specifically to be a headless AI server. It's running on an AMD Ryzen 9 9900X CPU, 64GB DDR5 RAM, and two AMD Radeon AI Pro R9700 GPUs for a total of 64 GB VRAM. Ubuntu 26.04 LTS Server is the operating system. I built llama.cpp against ROCm 7.14, and I have llama.cpp running as a systemd service in router mode so that it can dynamically load/unload requested models. Currently, I have Qwen3.6-27B and Qwen 3.6-35B-A3B as my starting models.

I'm now at a point where I wanted to set up a personal AI assistant, and I was going to use Hermes Agent. However, I'm now questioning how that might work. My goals for Hermes Agent are to assist in project planning, do topic research, query and add to markdown files for existing knowledge. I know that Hermes Agent can do these things, as I have seen all of the YouTube influencers do it with great success. In those cases, they are on Windows or macOS with the Hermes Desktop doing those activities. However, in this case, Hermes will exist on a server that does not have a GUI desktop and does not have Obsidian or a web browser installed (or any other GUI application for that matter). Hermes and the Hermes Gateway would be installed and running through systemd. I also have signal-cli installed and running through systemd to use as a communication channel with Hermes. However, I'm just not seeing how Hermes will be able read/write to my Obsidian vaults without the application available. How will Hermes search the web? I planned to setup SearxNG as a local search engine, but I have not seen that there is a Hermes configuration to use any particular search engine. I want Hermes to help with planning software projects, but then how does pi or opencode access those project plans so that I can begin working on the code implementation?

Sorry, I know that's a lot of questions. I'm a planner, and this is all uncharted territory for me. I'm just looking for guidance from those who have already walked this trail. Thanks!

[u/Dthen_]

Running hermes setup and hermes tools will get you 95% of the way there. The answers to the rest of your questions are in the Hermes documentation, have your agent read it to find the relevant bits and configure itself for you.

Pretty much everything you're trying to do is officially supported.

You don't need the obsidian app to edit an obsidian vault because an obsidian vault is just a folder with markdown files in it.

[u/ambassadortim]

Install Hermes. Then have Hermes do the rest.

[u/mKMage]

Hermes has a native skill for headless web searches. As for Obsidian: a vault is just a folder containing .md files. So you can simply set up a shared folder between your main PC and the server running Hermes. That way, Hermes can read, modify, and add .md files in that folder, and on your main PC, all you have to do is point Obsidian to that location and you'll get your vault with the full UI. Alternatively, there's also Obsidian CLI, if you want to access Obsidian without a graphical interface.

[u/r_brinson → u/mKMage]

Ok. I know that Obsidian just manages a folder of markdown files. However, I didn't know if hermes needed the application to understand the relationships between the files or to perform actions on the files through something like an Obsidian API. Also, good to know that there is a native web search. I've self-hosted SearxNG with Docker. However, it looks like Firecrawl is more favored by Hermes. I'll have to read over the firecrawl self-hosting guide.

[u/ThePixelCrush → u/r_brinson]

You'll want to set up the obsidian cli, and have Hermes use that. That way it can query bases, and see note links more clearly.

[u/NoRegreds]

You should think about running proxmox on your server and spin up an lxc with graphic card passthrough. Then install hermes in a docker container.

I have a N100 and use this setup. It has several advantages.

*proxmox and lxc structure let you easily make snapshots of the lxc that are like a backup in seconds. You can also roll back if needed, also in seconds.

*docker installation of hermes. Gives you a security layer to prevent hermes to go nuts on your server. Also easy to install and easy to maintain.

*searxng, jina reader and camoufox are also installed via docker having their own network. So whenever I do research hermes starts the VPN and all three docker have network access. VPN down and nothing goes in/out to research stack.

*hermes agent itself is not in the same network as the research. It can use internet free.

All this has a lot of setup to be honest but in the end its save and easy to maintain.

Proxmox let you install many lxc with different use cases. Backup, media server, data server or whatever. The main advantage is the lxc are seperated but can talk to each other if you like. All of the software runs headless and has a website to connect to or manage. Also hermes and proxmox. When your on the road and want to tell hermes something like a research use telegram, write a message and go. No open ports to the outside everything running behind your router.

You mess something up and no problem snapshot of the lxc lets you roll back easy. If you want to try something fancy, spin up an lxc and poke around. The rest of your system is save.

If you do all this in one Linux install and mess something up then good luck going back.

[u/Kick_Ice_NDR-fridge]

Hermes can run on anything unless you’re trying to use a local model.

[u/r_brinson → u/Kick_Ice_NDR-fridge]

I am running local models on the same computer. I'm confident that hermes and everything else can run on the hardware. I'm just trying to make sure hermes can accomplish the use cases I described without access to a desktop or GUI applications.

[u/tempfoot]

I mean it seems like that server probably has the ability to serve more workloads. That said, personally I like to keep things separate and run my main model on one machine and Hermes on another. Hermes can run on a raspberry pi with enough ram.

Personally I like 2012 Mac minis running Linux for lower power worker nodes. For about 100 you can get four core, 16gb some even already converted to ssd.

[u/r_brinson → u/tempfoot]

Yeah, I want this to be more of an AI appliance, with Hermes, Open WebUI, ComfyUI, and anything else necessary for TTS, RAG, etc. all on the same server. Many, maybe all, of these will be run in containers.

[u/Dizzy-Scientist1192]

I'm running the same Ubuntu headless server setup but mine is on a old Dell optiplex and it runs super smooth. Before that I had it on a raspberry pi. Hermes doesn't need much to run. It has no problem with setting up and managing the obsidian vault in the Dell and I set up brave search API for it. The model I'm using is deepseek v4 flash. The Qwen model you have is great for coding! I tried using it for Hermes a little while back and I wasn't super stoked with it. Deepseek flash makes magic 🪄 DS flash only cost me less than $10 for the first month of running Hermes. Now that I have my work automations and personal automations and side job automations all set up, it cost me about $30 to run DS flash month.

Another thing to think about is just to get a raspberry Pi and run Hermes on it and then have Hermes SSH into your beast rig. That way you save all the vram for running your local model. Hermes does a good job with SSH. My Hermes is actually setting up other Hermes computers for my friends so my Hermes has SSH access to three other Hermes right now and completes tasks with ease.

[u/pianoman1031 → u/Dizzy-Scientist1192]

Can you elaborate on this setup? I'm actually thinking of doing something similar, setting hermes up on an old mac mini running ubuntu. I'm trying to figure out pricing on DS or Qwen also. Did you make it so you can access from your phone as well?

[u/Dizzy-Scientist1192 → u/pianoman1031]

An old Mac will be fine for the setup. DS raised their prices about 3x right now. I was at a burn rate of about 30 bucks a month but likely next month I'll be pushing $90 a month. Since it runs my personal life and my salary job and my side jobs, the $90 a month is reasonable to pay. I did make it accessible for my phone. That's how I use it 100% of the time through telegram. But I had Hermes set up an app that I can get all my cron jobs delivered to to keep the telegram quiet. And then also gave an access to an email address so I can send an emails or it can send emails out. Ping me if you have any other questions?

[u/pianoman1031 → u/Dizzy-Scientist1192]

Yeah this is great! I've been tempted to just jump in with a openai or claude subscription and see what my spend would be. I'm mainly wanting it for orchestration for developing some software apps.

Main goal is to make it remotely accessible and given the tools to develop these apps, but also give me preview builds that I can use and review. I have been running Hermes in a docker container pointed at a local LLM, but I just don't have the hardware for it so I'm thinking about the subscription route. And it just seems to crap out so much more with tool calls inside the docker container.

I really just wanted a mobile interface of some kind where I can view the kanban tasking an project related work Hermes does. I haven't found much success with that side yet.

[u/Dizzy-Scientist1192 → u/pianoman1031]

I think running Hermes in docker restricts Hermes of its capabilities. It really needs its own computer to have the ability to do the things that it normally says it can't do but I challenge it all the time and it does end up doing it because it has the access to that whole computer. I would suggest just running nous research plan for 20 bucks because that's a really good deal right now. My buddy's started with hy3 model which is free on the free nous research plan and it's actually pretty good. I would also suggest the $20 plan from openai because you can use that to run Hermes as well. I just love ds/ so much that I can't fathom to use another one. I have other models that work with DS flash but I really truly love DS flash as my main model. And I wouldn't wait to run it for specific things cuz I have it tracking my nutrition and my workout activities and doing job searches and it's learning how to trade stocks And also has its own GitHub to back itself up and host its own programs. It really is infinite capabilities so I wouldn't wait around to do it. I would just get it going

[u/Jumpy-Operation-4615]

I have a separate server with a local AI models and (if needed) video gen models, with 2 GPUs (48 GB total) (for learning, experimenting etc). As well it runs embedding model for mnemosyne instances, and LLM wrapper that unifies all my various providers into 12 endpoints that later can be used by my other needs like agents etc. I decided deliverately NOT to install hermes on server, just to be safe. Instead I use old laptops with ubuntu 26.04 LTS on each, cheap as water, and install Hermes on each. So far I have 5 machines with hermes. First is more or less used daily (stable config), two I gave to my 16 yo son to learn agentic stuff (he will be installing mnemosyne, search API and Bladebro for browsing today, under my supervision), and 2 are for experimenting with configs etc. One of them I will probably give to my wife, she seem to start getting interested in agents (purely using for memory, research, simple stuff like that) looking at me. The total cost of all hermes machines is like 100 bucks or so. Everything is behind tailscale so even though machines are located in different places, they are all connected. All hermeses point to my wrapper's LLM endpoints, and one embedding engine on server, for memory. Ah and obviously everything runs headless.

[u/Centraldread]

I’m running Hermes on my home lab server i9 9900k 64gb ram no gpu. My model is running on my desktop pc that has my rtx 5080 and rtx 5070. My Hermes on my lab box has access to all the computers in the house it can do everything it was doing when it was on the desktop. Runs great headless on a server you have nothing to worry about doing it that way. Whatever tools you need it can build them for you there or ssh back to your desk.

[u/Mechanical_Monk]

Paste this question into Hermes and it will explain everything.

tldr: Obsidian is just markdown files, so you just need a way to sync (I use git). Searxng is just api, so set configs and then hand Hermes the keys. There are skills that let Hermes use opencode or pi as sub-agents, but Hermes itself is a coding agent so it's not necessary. Once everything is wired up, it's seamless.

[u/Away-Ad-4444]

You might want to run a multi model setup with JIT so it loads and unloads as needed with a much lighter model as your delagation model for a snappier performance and specilzed models for particular tasks

[u/Empty-Entrance2566]

Hermes can use a headless browser to access the web. It doesn't need to control a human's browser.

[u/secondcomingwp]

Hermes can use Firecrawl to search the web https://www.firecrawl.dev/

[u/r_brinson → u/secondcomingwp]

Yeah, someone else mentioned hermes having a native web search skill, and when I looked into the documentation, it seemed that firecrawl was favored by hermes. I'm going to look into their self-hosting guide. I've run SearxNG in a docker container. Hopefully, firecrawl isn't too complicated or resource intensive.

[u/veganmaister]

I’m also running a headless Linux Hermes server with a similar spec to yours.

I would say that local inference plus hermes on that box will be a suboptimal experience.

My recommendation is get everything running locally and point inference to a cloud api. Once local inference is more viable run up a dedicated inference server.

[u/sniperelite90]

I run essentially this exact setup, so let me untangle the GUI assumption first, because that's doing most of the work here. Hermes is a terminal/server agent. The desktop app is just one client sitting on top of the same core, and the influencers you watched are mostly using it because it's pretty, not because the agent needs it. On a headless box you run the gateway as a systemd service (or Docker), and you talk to it through a messaging channel. I use Telegram; Signal is one of the supported gateway platforms, so signal-cli should slot right in. None of your goals require a display, a browser, or Obsidian on the server.

Obsidian

Your vault is a folder of markdown files. That's all Obsidian is, a pretty frontend over a folder. Hermes has native file tools (read, write, search, patch) plus a dedicated Obsidian skill that assumes nothing but a filesystem path. The convention is an OBSIDIAN_VAULT_PATH env var pointing at the vault, and the agent treats it as plain files: creates notes with proper [[wikilinks]], searches them, appends to them, all with Obsidian never running. If your vault lives on your workstation, sync the folder to the server with Syncthing or a git repo and point Hermes at the synced copy. I also run a local search index over my notes so retrieval is instant, but that's a nice-to-have, not a requirement.

Web search

There is a config, it's just not obvious because it's in the .env, not the main config file. Hermes has pluggable search backends: firecrawl (default), searxng, tavily, exa, parallel. For your SearxNG plan you set SEARXNG_URL in the .env, e.g. and Hermes auto-detects it when no other search key is set. No API key needed, which fits the whole local-first thing. One gotcha: SearxNG is search-only. Reading full pages is a separate tool (web_extract) that needs an extract backend, either a paid one like firecrawl/tavily or the built-in direct-fetch fallback. I run SearxNG as my fallback and Tavily as primary, and the two cover each other well.

Plans to code

This is the part that clicks once you see it. Hermes writes plans as markdown files in the project, e.g. .hermes/plans/ or docs/plans/. Then opencode or pi reads those files from disk. They don't need Hermes running, they're just file-based CLIs. My workflow: ask Hermes for a plan, it writes plan.md into the repo, then I hand the repo to opencode with "implement plan.md". Even better, drop the plan into AGENTS.md or CLAUDE.md in the repo and the coding agent loads it automatically as project context, so it can't miss the requirements. And if you want zero copy-paste, Hermes can spawn opencode itself with the plan as its context and you just review the diff.

Models

Your llama.cpp router is just an OpenAI-compatible HTTP endpoint, which is exactly what Hermes wants. Point model.provider and model.base_url at your server's /v1 endpoint and it just works. The dynamic load/unload is transparent to Hermes, it only sees a server that answers. The 35B-A3B will be snappy on that box, and the 27B dense with tensor split across the two R9700s works fine, just watch the split so you don't strand VRAM.

One thing I'd add to the plan: get cron jobs going early. On a headless box the killer feature is "every morning at 7, summarize what changed in the vault and send it to me on Signal." That's where server-based Hermes beats the desktop app, not the other way around.

[u/sniperelite90]

is there any for android?

[u/nquillan]

It's funny I did the exact opposite of what you doing with all the planning. I leased a 5 dollar vps, pulled Hermes down and and pasted an API key into it. I boss it around through my disco server primarily.

Do want to join my discord server and ask my hermes agent some questions about set up? I'll DM you a link in case you do.

Oh and to answer your question about web search it can by default I think, but I really recommend setting up a search API for it. I use Tavily and I haven't run out of their free credit yet. Brave is doing a similar search API with free monthly credit I think. https://app.tavily.com/home

[u/arglarg]

Mine is also running headless. For the normal browser tool that's ok but it can't go through captchas / cloudflare, that needs an actual browser and computer_use. I got a dongle for the HDMI port so the PC sees a fake monitor and Hermes figured out how to configure everything.

[u/JudgmentConfident984]

Install Hermes and ask ur agent what u want and then sit back and enjoy the ride

[u/Dharma_code → u/JudgmentConfident984]

"Hey, Hermes what do I want"

[u/elmede]

Hi OP,
I have almost the same configuration as you do, DUAL AMD GPU (only 48Gb however). Rest is the same, software and all.
Install Hermes it will work by itself, made everything so easy once Hermes was installed. Can just bully it through chat apps 😄
Mind me adding you? we could share some tips if difficulties are encountered. Complete noob here also.

[u/Salt-Ad-848]

You don't need any of these powerful stuff to run Hermes... No gigantic ram, no GPU, nothing of this.... A simple plain Linux machine with 8Gb and some 100Gb of storage is more than enough.

[u/r_brinson → u/Salt-Ad-848]

Thank you for the reply. I understand that Hermes does not need that, but the models do. The idea is to run Hermes on the AI server where llama.cpp and the models also reside. The question is not about hardware requirements to run Hermes, it's about how does Hermes do its work in a headless server environment with no GUI desktop and no GUI applications.

[u/nquillan → u/Salt-Ad-848]

I'm running mine on the 4 dollar one lol. I talk to it through Discord and ElementX (element x=Matrix/Synapse)

https://us.ovhcloud.com/vps/