Self-hosted AI agents: the honest guide

What self-hosting gets you

Three things, and they are real. Sovereignty: on hardware you own, nothing leaves the building except what you send to the model provider you chose, and with an open model on your own GPU not even that. Cost shape: the software is often free, a small server is a few dollars a month, and model tokens are the only line that grows with use. Control: you decide when to update and which network the agent can see. Whether those outweigh being the operator is argued on a separate page; this one assumes you have decided and helps you do the job well.

Installation is the short part

Most self-hosting guides are installation guides, which is one reason self-hosted agents tend to fail in month two rather than on day one. Installing is an afternoon. What you have afterwards is a service: a process that has to be awake at 3am, holding connections to platforms that revise their rules, carrying tokens that expire on a timetable you did not set, on a disk that fills without saying so. Each job below is ordinary, each arrives unscheduled, and on your own box it arrives for you.

A machine that stays up

The failure modes of a sleeping laptop are set out on the diagnostic page. The practical conclusion is a machine that is never suspended: a desktop in a cupboard or a small rented server. If it is at home, put it on a wired connection, turn off every power-saving setting that touches sleep, and accept that the router and the electricity are now part of the deployment.

Two things fail on a machine that never sleeps, and both are quiet. The disk fills - transcripts, container images, package caches, logs - and a full disk stops everything at once; Olano watches it, warns as it fills, and can clear caches, superseded images and old logs on its own. And memory runs out on a small box, at which point the kernel kills the largest process, usually the agent runtime; the supervisor brings it back, and the run in flight is lost rather than paused.

Reachability, the part everyone underestimates

There are two directions, and only one is free. Outbound: the runtime calling a model provider, or holding a connection open to a messaging platform, works from behind any home router on any changing IP and needs nothing from you. Inbound is the problem. A sign-in callback from a service you are connecting, a webhook from a platform that pushes events, a link you send to a colleague's phone so they can finish a sign-in - all of these need the outside world to reach a fixed address on your box, over https, and to find the same address again next week.

Home connections defeat this in several ways at once. The public IP changes without notice. Many providers put customers behind carrier-grade NAT, where there is no inbound path at all and port forwarding does nothing. A laptop on hotel wifi sits behind a captive portal that swallows every connection until someone clicks Accept in a browser, which an unattended agent cannot do. And certificates: a self-signed one is refused by every third-party callback, so https means a real domain name and a certificate that renews itself.

The three workable answers are a rented server with a static IP and a domain pointed at it; a domain plus a reverse proxy at home with a dynamic-DNS updater, which works until the NAT case bites; or a tunnel that gives you a stable public hostname and forwards it to your box, trading sovereignty over the front door for not having to open one - the Olano installer includes a tunnel client by default for this, and you can decline it. Whichever you pick, tell the deployment its public address. In Olano, a magic link sent to a phone comes back without a working address until the deployment knows its own, exposing an agent over MCP or A2A is refused on a plain-http public address, and the dashboard installs to a phone's home screen only over https. A box on plain http still works in the browser; it just cannot do those three things.

A process supervisor

An agent runtime is one long-running service. In Olano it boots the agents, their channels, the scheduler, Olano Cortex and the self-learning loop together, and if the process dies they all die with it. Something has to bring it back: after a crash, after an out-of-memory kill, and after a reboot nobody was there for. The standard answers are a system service on the host or a container runtime with a restart policy; the all-in-one deployment shape Olano ships tells the container runtime to bring every container back after a crash or a reboot unless you stopped it yourself, so the stack returns with nobody logged in. A terminal session left open in a multiplexer is what most people start with, and it is what is not running on the morning after the first reboot.

Two consequences to know before the first outage. A run in flight when the process died is gone, not paused, and has to be started again. A schedule that fell inside the outage is not run afterwards: the scheduler keeps no record of what it missed while the process was down, so there is no backlog to work through.

Secrets handling

The rule that survives every setup: a credential never belongs in a configuration file, a committed environment file or a shell history. Olano keeps every API key, bot token and service password in an encrypted vault and resolves them per agent, so two agents using one service can be different identities. An agent asking for a stored value gets a masked preview; turning one back into plaintext is an explicit, owner-only request that needs a person's approval. The sandbox an agent runs commands in holds no credential at all, and the revision history redacts secrets before anything is stored.

The self-hosting decision inside this is the restart. After the runtime restarts, the vault is locked. Either a person unlocks it from the dashboard before anything that needs a credential can run, or the box holds unlock material of its own - a key in the environment or the system keyring - and opens it unattended. The first is safer and means a 2am reboot leaves anything that needs a stored credential waiting until someone wakes up; the second is what unattended operation requires, and it puts the secret that protects all the other secrets at rest on the same machine. Pick one deliberately, and if it is the second, that machine's disk encryption and login are now the boundary.

Backups you have actually restored

The version history built into the platform is not a backup. Olano's revision history tracks a deliberate allow-list - deployment settings, the org chart, each agent's configuration and the markdown in its workspace (identity files, memory, daily logs, skills) - so you can answer "who changed this, and what did it look like before". Databases, tokens and the vault are never tracked, by design. A backup has to cover what it does not: the home directory holding every workspace, the conversation transcripts and the encrypted vault, plus the conversation-state database if you chose the database backend. When that database runs as its own container the two live in different places, and a backup that copies one and forgets the other restores agents with their memory and no thread history, or the reverse.

Two things about the vault. It is encrypted, so the copy is safe to keep elsewhere and useless to anyone without the passphrase - including you, if the passphrase was only ever in one person's head and that person has changed jobs. And a backup nobody has restored is a hypothesis. Restore it once onto a spare machine, before the day it matters.

Connector renewals

This is the one nobody predicts, and the one that gets reported as "the integration randomly broke". Connect an account over OAuth and the service hands over a short-lived access token and a way to renew it. Renewal has to happen ahead of expiry and needs a running process at that moment - exactly what a laptop asleep over a weekend does not have. Some providers also expire the renewal itself if it goes unused long enough, at which point no software can recover the connection: consent cannot be renewed by a machine.

The runtime's job is to make this rare, and Olano's health loop does three things on its own: it renews grants ahead of expiry on a regular check even while the agent is idle, it re-discovers a provider's token endpoint if the one it had stopped answering, and it reconnects servers that timed out or errored with a wait that lengthens on each failure. What it never does is retry a server that needs a sign-in you have not given, because reconnecting cannot invent consent. At that point you get a notification naming the server and the agent, with a link to where a person re-authorises it, and a connection-health view shows every server's state without waiting for one. On your own box that person is you, and the lesson for the machine is the same as above: keep the process running through the renewal window.

Updates

On a managed deployment a new release is made available and you are told when it is ready, so you choose when to take it. On your own box you are the one who notices a release exists. The habit that works is the boring one: read what changed, take a backup or a snapshot, update, restart, then check the connection-health view and the notifications, because an update is also a restart and everything above applies to it. Two specifics for Olano: the files under each workspace's system folder are owned by the platform and replaced on upgrade, while everything else in the workspace is yours and left alone; and the installer is safe to re-run, so refreshing dependencies is the same step as installing them.

The sandbox question

Where an agent's shell commands run is the decision with the most security in it. On a managed Olano deployment the answer is fixed: never on the host. Each agent gets an isolated container that holds neither the platform's source nor its secrets, and if that container cannot start, the agent is reduced to file tools with no shell rather than being handed a shell on the machine. On your own box the default is different, and it is the honest trade-off of self-hosting: an agent's commands run on the host, rooted at its workspace, with a scrubbed environment. That is fine for one technical person whose agent does their own work, and the wrong posture for an agent that reads email from strangers, because a shell on the host is a shell on the box that holds your vault.

The alternative needs a container runtime, which is why the installer sets one up by default and why declining it is a decision rather than a shortcut. With one present you can switch any agent, one at a time, to a container of its own - the arrangement a managed deployment enforces on every agent - with a standard toolkit inside it (git, Python, Node, ripgrep, a terminal multiplexer, ffmpeg, pandoc, PDF tools, LibreOffice and OCR) and per-agent ceilings on memory and CPU that are limits, not reservations, so an idle agent costs nothing. Packages an agent installs land in a shared area that survives restarts. On your own box the container settings - network mode, image, whether it persists - are honoured as written; a managed deployment pins them.

The two choices that matter at install time

Where conversation state lives. Olano offers two stores: a database, the installer's default and the one built for many agents writing at once, and a single-file store that needs no configuration and suits one machine. Selecting the database without telling it where the database is fails loudly at startup rather than silently dropping to the file store - deliberately, because a deployment that quietly changed where it keeps every conversation is the kind of thing you discover during a restore.

Whether you have a container runtime. Everything in the sandbox section follows from this one. Without it, agents can only run commands on the host and the database falls back to the file store; with it, the database gets its container and any agent can be given an isolated place to run. Skipping it to keep the install small removes the only way out of the open posture; it is not a lighter version of the safe one.

No GPU, and no third-party embedding service

A common assumption is that running AI on your own server means a GPU. For the agent runtime it does not. The models that do the reasoning are called over the network from whichever provider you hold keys for, and the only model that runs locally by default is the small one that turns memory and documents into searchable vectors. Olano's default install runs that on the CPU with no heavyweight machine-learning framework, no GPU driver and no third-party token, so search over memory and the knowledge base works on the same small server as everything else. The heavier legacy backend exists as an explicit opt-in; a missing GPU is not a problem to fix.

The third option: open models on your own GPU

Where the requirement is that prompts and documents never leave the building at all, the answer is an open model on a GPU you control, served locally and addressed by the runtime like any other provider. Olano supports this through Ollama - the same route its Private AI plans use on the managed side, where the GPU box is rented for you. The trade is real: a model that fits on one card is not the frontier model you were using through an API, so the work has to be something a smaller model does well - a customer-service agent answering from a defined knowledge base often is, open-ended research often is not. The GPU tiers and their prices are on the pricing page rather than here, because they change.

Self-hosting Olano itself

Everything above applies to Olano as much as to anything else, because Olano can be self-hosted: install it on your own machine or server, bring your own model provider keys, and you get the same product the managed plans run - the dashboard, the agents, the channels, Cortex, approvals and the audit trail. A few defaults differ on a manual install. Team accounts are off until you switch them on. The vault opens unattended after a restart only where the box has a system keyring or you have given it key material of its own. There is no credit balance, so a provider key is required, and cost tracking is in dollars against your own provider bills. And the all-in-one deployment shape puts the runtime, the dashboard and the database on one machine behind one reverse proxy with automatic certificates, on one origin - a requirement rather than a preference, because the session cookie, the dashboard terminal and long chat streams all depend on it.

The managed plans exist for people who read the eight jobs above and would rather not own them; the trade is on the pricing page, and because the same software runs in both places the choice is not permanent in either direction.

What we would choose, and why

For one technical person: a small rented server with a static IP and a domain, the all-in-one shape with the database store, the container runtime installed so the sandbox is available, unlock material on the box so a reboot is unattended, and a nightly copy of the home directory and the database volume to somewhere that is not that server, restored once to prove it. That answers every job on the list except updates, and updates are a calendar entry.

For an agent a customer can message: the same machine, and one more question - who reads the re-authorisation notification when it arrives at night. If a named person has agreed to, self-host and enjoy it. If nobody has, the problem is not the machine and nothing on this page fixes it. That is the case for a managed deployment, ours or anyone's, and the one place on this page where we would say so.