Build a Self-Hosted AI Server From Scratch: The Layers, the Build Order and a Free Setup Kit

Every few weeks someone who has read the sovereign AI posts on this site asks me a version of the same question: what would it take to build the whole thing from an empty server? Not the ideas, the actual build. Which machine, which services, in which order, and what breaks along the way.

Until now the honest answer was "read a dozen articles and stitch them together." This post replaces that answer with two complete guides and a setup kit of 35 templates, generalized from the server I work on every day. All of it is free to download, with no signup and no email.

The guides are long, about 25,000 words between them, because a server like this has many moving parts and nearly every part has one trap that costs an evening. This post is the map: what each piece does, why it is shaped the way it is, and where in the guides the exact steps live. If you are new to Linux, start with the beginner path. If you have built servers before, skip to the lessons.

The downloads

• The complete kit (zip): both guides plus all 35 templates, with a README at the root that says where to start. This is the one to grab.
• Guide 1: how the server works, layer by layer: about 9,400 words on what each layer does, how it connects to the next, the config shape and what to tune.
• Guide 2: the build, step by step, plus typed decisions with Jev: about 15,300 words on choosing the server, a beginner path, every layer in order, CPU and Ubuntu tuning, updates, 30 lessons, a from-scratch checklist and a troubleshooting table.

The most useful single files, if you want to read one before taking the whole zip:

Every file is a template. Replace example.com, <user> and the other bracketed values with your own, and keep secrets in the owner-only files each template names. Never put a secret in the template itself, and never paste one into a chat with an assistant.

Who this is for, and what it costs

This is for a small business owner, a consultant or a one-person shop that already pays for Claude or ChatGPT and wants those assistants working on a machine it controls: files that stay put, jobs that keep running when the laptop closes, local models for the private and repetitive work, and a phone alert when something finishes or fails.

The money side is smaller than most people expect. A CPU server of the class the guides use, with two 12-core Xeons, about 96 GB of RAM and mirrored NVMe drives, was listed from $125 a month plus a $125 setup fee in OVHcloud's US Eco range when I checked on October 5, 2026 (the full cost comparison). The software in the guides is free to self-host. The other paid parts are the assistant subscriptions you probably have already.

What a CPU server will not give you is a local model that writes long answers as fast as a hosted one. That is a deliberate choice, and it shapes everything else.

The one design rule

The whole stack follows one rule: the expensive, smart models do the writing and the judgment; cheap local models do triage and extraction; plain code does everything deterministic; and a person approves anything that leaves the server. Every web page sits behind one identity check, and no service listens on a public port.

Once that rule is in your head, most of the choices below stop being choices. Should the local model write the client report? No, it sorts and extracts. Should n8n send the email? No, it prepares it and a person sends it. Should the chat page talk to the model runner directly? No, every call goes through one gateway, so every call has a key, a budget and a line in the log.

The stack on one page

+-----------------------------------------------------------------------------------------+
| L8 CONTROL PLANE  prompt hub: prompt -> plan -> Run -> job log -> phone alert           |
|                   ops portal: fixed commands with approval, plus the link directory     |
+-----------------------------------------------------------------------------------------+
| L7 JOBS           systemd user timers | detached agent jobs | parallel runners | n8n    |
+-----------------------------------------------------------------------------------------+
| L6 PREPROCESSING  raw text -> small local model -> JSON schema -> validate -> hand-off  |
+-----------------------------------------------------------------------------------------+
| L5 AGENTS         Claude Code | Codex CLI | Cowork | Desktop Commander | Hermes 3 8B     |
+-----------------------------------------------------------------------------------------+
| L4 MEMORY         Qdrant vectors | SQLite FTS5 keywords | notes as files in git         |
+-----------------------------------------------------------------------------------------+
| L3 GATEWAY        LiteLLM: one OpenAI-compatible endpoint, keys, budgets, spend logs    |
+-----------------------------------------------------------------------------------------+
| L2 INFERENCE      Ollama on CPU: qwen2.5 1.5B, hermes3 8B, nomic-embed-text             |
+-----------------------------------------------------------------------------------------+
| L1 ACCESS         Cloudflare Tunnel + Access | SSH keys | browser desktop | Guacamole   |
+-----------------------------------------------------------------------------------------+
| L0 HARDWARE/OS    CPU server, 64 to 96 GB RAM, NVMe RAID 1, Ubuntu 26.04 LTS, Docker    |
+-----------------------------------------------------------------------------------------+
  Cross-cutting: ntfy alerts | restic backups | update tool + GitHub Actions | monitors

Reading from the bottom:

• L0, hardware and operating system: a rented dedicated server running Ubuntu 26.04 LTS, with Docker installed from Docker's own repository.
• L1, access: Cloudflare Tunnel and Access in front of every web page, SSH with keys only, and two kinds of desktop in a browser tab.
• L2, local inference: Ollama running three small models on the CPU.
• L3, the gateway: LiteLLM, one OpenAI-compatible endpoint with named aliases, keys, budgets and a spend log.
• L4, memory: Qdrant for search by meaning, SQLite FTS5 for exact keywords, and the notes themselves as files in git.
• L5, agents: Claude Code and Codex on their subscriptions, plus Claude Cowork, Desktop Commander and the local Hermes 3 model.
• L6, preprocessing: raw pages and documents become small, validated JSON records before any expensive model reads them.
• L7, jobs: systemd timers, detached agent jobs, parallel runners and n8n.
• L8, the control plane: a prompt hub where every prompt becomes a plan, and nothing runs until you press Run.

Across all of them sit phone alerts through ntfy, restic backups to Google Drive, an update tool with rollback, and GitHub Actions checking the server from outside.

Choosing the server

Start from the work, not the model. If the main writing and coding is done by Claude Code and Codex on subscriptions, the server's job is to run services, desktops, schedules, small models and backups. That is a CPU and RAM job. If you need a local model that writes long answers at interactive speed, that is a GPU job, and video memory decides which models you can run.

The CPU rows are planning examples, not measured minimums. For a sense of how much room a CPU server has: in one measured window, five unattended coding agent jobs running together used about 2 percent of a 48-core server, because model response time, not the CPU, set the pace (the operating guide).

When you compare providers, look for AVX2 at minimum and AVX-512 with VNNI if you can get it. Favor memory channels over clock speed, because generating tokens on a CPU is limited mostly by memory bandwidth. Get two NVMe drives you can mirror, an out-of-band console (your way back in after a firewall mistake), and /dev/kvm if you want a Windows VM. Run lscpu on the provider's rescue system before you sign a long contract.

Renting a GPU by the hour is the right tool for bursts, such as trying a 70B model for a day. A dedicated server is the right tool for services that should cost the same whether you use them or not. Plenty of builders end up with both. Guide 2, section 2 has the details and current examples, and running AI on-prem versus in the cloud has the longer cost argument.

Getting in safely: Cloudflare Tunnel and Access

The first rule of the access layer is that no web page on the server can be reached from the internet directly. Only SSH is open, and it accepts keys only.

 you: laptop or phone browser                         the server
 ---------------------------                          ----------
       |  https://llmhub.example.com                   cloudflared (user service)
       v                                                 |  opened FROM the server:
 +------------------------------+                        |  4 outbound connections
 | Cloudflare edge              |<-----------------------+
 |  Access: sign in, check the  |
 |  email policy, add a signed  |   allowed   +---------------------------------------------+
 |  JWT to the request          |------------>| ingress rules, first match wins             |
 +------------------------------+             |  llmhub.example.com -> 127.0.0.1:8790 hub   |
       | not allowed                          |  chat.example.com   -> 127.0.0.1:8080 chat  |
       v                                      |  desk.example.com   -> 127.0.0.1:3000 desk  |
   refused at the edge,                       |  win.example.com    -> 127.0.0.1:8007 guac  |
   never reaches the server                   |  ...one rule per service you add            |
                                              |  anything else      -> 404                  |
                                              +---------------------------------------------+
                                               every rule: access.required = true, so the
                                               tunnel also rejects requests without a valid
                                               Access token; the hub and portal check it again

The program cloudflared runs on the server and opens connections outward to Cloudflare. You map each hostname, such as llmhub.example.com, to one port on 127.0.0.1. Cloudflare Access sits in front: you sign in, it checks your email against a policy, and it adds a signed token to the request. The tunnel turns away anything without a valid token, and the apps that make decisions about identity check the token again themselves.

Four details matter more than the rest:

1. Create the Access application before the tunnel route. Cloudflare's own guide recommends that order and warns that without an Access application, the published app is available to anyone on the internet. Then test it: in a private window, every hostname must send you to the sign-in page, and an email address that is not on the policy must be refused.
2. Publish every container port on 127.0.0.1. Docker's documentation says plainly that Docker and ufw use firewall rules in ways that make them incompatible: Docker routes container traffic before ufw's rules ever see it. A port published as 8080:8080 is open to the internet even while ufw reports it closed. A port published as 127.0.0.1:8080:8080 is not.
3. Verify the token in your own apps, with the audience set up front. The token arrives in the Cf-Access-Jwt-Assertion header. Check its signature against your team's public keys, its issuer, and its audience, which is the AUD tag of the Access application, written into the app's config from the first day. An app that learns its audience from the first request it sees trusts whoever arrives first.
4. Turn off password SSH before anything else goes on the server. Put PasswordAuthentication no, KbdInteractiveAuthentication no and PermitRootLogin no in a drop-in file whose name sorts first, such as 00-hardening.conf, because the first value sshd reads wins. Keep a second session open until a key login works. Then give every automation its own forced-command key that can run exactly one program.

Desktops in a browser tab

Two desktops make the server usable from any laptop or phone:

• A Linux desktop (the webtop image) gives you Chrome, extensions and Claude Cowork running on the server, so a long browser task survives closing your laptop. Its documentation is blunt: there is no authentication by default, the terminal has passwordless sudo, and you should not expose it to the internet unless you have secured it properly. Keep it on loopback behind Access, set a password and turn sudo off.
• Apache Guacamole opens a Windows 11 virtual machine, or any machine that speaks RDP or VNC, in a tab. It can sign you in by trusting the email header that Cloudflare Access adds. Guacamole's manual warns that with this extension it is trivial for anyone to add that header themselves unless every outside request is cleaned first, so the web app must listen on loopback and the tunnel must require Access. One version trap: Guacamole 1.6 calls the switch HTTP_AUTH_ENABLED, the 1.5 name is ignored, and 1.6 can switch the feature on when a related variable is present even without it. Set HTTP_AUTH_ENABLED=true explicitly so the config says what it does.

A Windows desktop and Drive backups on the same server covers the Windows side in more depth.

Local models, one gateway and memory

Ollama: small models for small jobs

Three models do almost all of the local work:

Two settings deserve your attention. Ollama's default context window is 4,096 tokens, so set OLLAMA_CONTEXT_LENGTH to what your prompts actually need. And parallel slots multiply memory: Ollama's FAQ says the RAM required scales with OLLAMA_NUM_PARALLEL times the context length. Parallel slots raise total throughput, not the speed of one answer. On our 48-thread server, four parallel requests to the small model delivered about 2.7 times the throughput of one at a time, while each request took a little longer (the benchmark). Measure on your own prompts before you tune anything.

The local models are not the main workers. A 1.5B model on a CPU is fast and costs nothing per call, which makes it good at "what kind of request is this?" and poor at writing your client proposal.

LiteLLM: one endpoint for every model call

Every caller, whether it is the prompt hub, n8n, the chat page or a script, talks to one OpenAI-compatible endpoint on 127.0.0.1:4000 and asks for an alias such as triage or extract instead of a host and a port. LiteLLM routes the alias, retries, falls back, caches, and writes a spend row for every call. Swapping a model means editing one alias.

Three rules make the gateway worth having:

• Every caller gets its own virtual key, limited to the models it needs and, for anything metered, given a budget. The master key is for creating keys. No app should ever call models with it.
• Subscription logins never go into the gateway. Your Claude and ChatGPT plans are a different billing lane with their own terms. Only metered API keys belong in LiteLLM.
• Nothing falls back to a paid API silently. A fallback to a cloud model on a private document sends that document out. The kit's config falls back from the small local model to the larger local one, and reaches a cloud alias only when an input is too long for the local context, and only on keys allowed to use it.

Open WebUI sits on top as a chat page for people, with its direct Ollama connection turned off so every chat shows up in the gateway's log.

Memory: Qdrant beside plain keyword search

Agents do better work when they can find the right note before they act. The guides run two kinds of search side by side. Qdrant stores embeddings of your notes, reports and agent briefs for search by meaning, with an admin key for the indexer and a read-only key for everything that only searches. A SQLite FTS5 index finds exact phrases, rule names and identifiers, which vector search regularly misses (what a mobile workspace taught us).

Keep the notes themselves as files in git; the vector index must be rebuildable, never the only copy. Pin the embedding model, because a different model needs a new collection and a full re-index. And take a Qdrant snapshot before every upgrade, because a snapshot restores only to the same minor version or the next one.

Hermes: the model and the agent

"Hermes" means two different things, and the guides keep them apart. Hermes 3 is Nous Research's open-weights model; the 8B version runs in Ollama as the local writer and extractor. Hermes Agent is Nous Research's MIT-licensed agent runtime with persistent memory, skills and chat gateways. The agent is optional, and its documentation says it needs at least 64,000 tokens of context for agent use with tools, which is slow and memory-hungry for an 8B model on a CPU. If you add it, point it at a hosted model through the gateway so its calls land in the same spend log, and keep it away from unattended jobs that change files.

Preprocessing: small JSON before anything expensive reads it

This layer saves the most money and gets the least attention. Instead of pasting a raw web page, feed item or PDF into a hosted model, the server reduces it first:

1. Clean it in code: strip markup and scripts, remove duplicates, pull out dates and prices with patterns, and record a hash of the source.
2. Split it if it is too long for the context budget.
3. Ask a small local model to fill a strict JSON schema, passed in Ollama's format field, at temperature 0.
4. Validate the schema first, then the meaning: values come from the allowed lists, dates parse, and every quoted excerpt appears word for word in the source.
5. On failure, retry once with the validator's errors in the prompt, then escalate to a larger model or a person.
6. Store the record with its source hash, and hand only the compact record to the expensive model.

The size difference is large. One measured page was 71,005 bytes as HTML and 5,743 bytes as clean Markdown (agent readiness beyond llms.txt). At roughly four characters per token, that is about 17,700 tokens against 1,400, and a JSON record of the facts you need is often a few hundred. As arithmetic only: at an example rate of $3 per million input tokens, 1,000 raw pages a day is about $53 of input, and 1,000 records of 400 tokens is about $1.20. The local model costs CPU time instead, and output, retries and escalations still count, so a 90 percent cut in tokens is not a 90 percent cut in the bill.

The rule that keeps this honest: prefer null to an invented value, and never let an automatic action depend on a model saying it is confident. Depend on evidence you can inspect.

The prompt hub: plan first, then Run

Once you have two assistants and two machines, you need one place to send prompts. The guides call it the prompt hub, and I wrote up how we built ours earlier this week. From a phone or a laptop you open one address, type a prompt, and the hub works out which engine and which machine should run it, shows you the plan, and waits.

 phone or laptop
   | 1 open https://llmhub.example.com/?q=<your prompt>     (a GET only shows the page)
   v
 Cloudflare Access sign-in -> tunnel -> hub on 127.0.0.1:8790
   |   hub verifies the Access JWT (signature, issuer, audience) and the email allowlist
   | 2 the page POSTs /api/plan (JSON only)
   v
 PLAN, nothing runs yet
   - triage: qwen2.5:1.5b returns {"kind", "changes_files"}; keyword heuristic if slow
   - memory: Qdrant search, top 5 hits scoring 0.66 or more (titles and paths)
   - engine: first available of claude -> codex -> local; skip any engine cooling down
   - target: host | browser desktop | Windows VM      mode: ask (read only) | work
   - returns the plan and a one-time run token
   | 3 you press Run (POST with the run token; the token is then deleted)
   v
 QUEUE in SQLite -> worker, at most 2 jobs at once -> engine runs in the job's own folder
   |  events stream to log.txt (the page polls it); result.md at the end
   |  usage limit hit -> mark that engine cooling down (30 or 20 min) -> next engine
   v
 4 ntfy push "job <id> done via codex <link>"; the link opens the job page behind Access

The choices that make it safe:

• Opening a link never runs anything. A link with a prompt in it only fills the box and creates a plan. Link previews, chat apps and crawlers follow links, so they must never be able to start work.
• Run takes a deliberate press that sends a one-time token, and every request that changes anything must be JSON, which stops a plain form on another site from triggering it.
• Engines go in order of strength and cost: the subscription coding agents first, the local model last. When one hits a usage limit, it is marked as cooling down and the job moves to the next.
• Each job gets its own folder holding the prompt, the log, the event stream and the result. After a restart, running jobs are marked interrupted instead of pretending to be alive.
• Phone alerts carry the job id, the status and a link, never the prompt, because the alert passes through a public relay.

The kit's hub/server.py is a reduced version to build from: the Access token check, a typed plan, the run token, SQLite and a small worker.

Typed decisions with Jev

The newest piece is optional and comes last. TypeSafe's Jev is what the company calls a System One model: you send it some material and typed questions, and it returns typed answers with probabilities instead of text. A Choice picks one of your options, a Score places the material on your rubric, and a Noul gives the probability that a statement is true. It does not write anything.

That fits every spot where a stack like this asks a small model for JSON and then parses, retries and checks it. A Choice can only return one of your labels, with a probability for each, so there is nothing to parse. TypeSafe defines a Choice's confidence as (p_top minus 1/n) divided by (1 minus 1/n), which is zero for an even split and one for certainty. Its own example sends anything under 0.5 to a person and asks for confirmation below 0.9 before an action with consequences, and it says plainly that the right thresholds depend on your data.

Where it changes the build:

• The hub's plan becomes one request with typed questions: work or read-only, which engine fits, whether the prompt asks to send, pay, sign or publish, and whether it contains a password or key. Code owns the outcome. A credential flag stops the plan, an outward flag means a person sends, and low confidence shows the alternatives instead of guessing.
• Preprocessing gets one question before extraction (is there anything here worth extracting?) and per-field checks after it, so a repair touches one field instead of the whole record.
• Guardrails become a set of yes-or-no questions asked of every inbound message and outbound reply.

Hosted Jev costs $0.042 per million input tokens, and output is free, so a 2,000-token decision costs about $0.000084 and 10,000 of them a day cost about 84 cents. The material you send does leave the server, so the guides split the work: public and synthetic material can go to hosted Jev, and anything private goes to a local decision model through Ollama 0.35 or later, which serves a System One endpoint at /v1/systemone with open decision models such as Nimble and Tev1. Run both in shadow mode for a week over your own labelled history before anything acts on them. Jev and System One models in a sovereign AI stack has the full trial plan.

Where n8n fits, and where it does not

n8n is the integration layer beside the hub and the job runner, not a replacement for either. It earns its place for webhook glue between software-as-a-service tools, for approvals (a workflow can pause until a person submits a form), for scheduled integrations that someone who does not code must be able to read and edit, and for simple AI steps routed through the gateway so they keep their budgets and logs.

It is the wrong tool for long coding-agent jobs, heavy data pipelines, anything that must be reviewed in version control first, and anything that sends, books, pays or signs for a person without an approval step. The kit's example workflow is the safe shape in miniature: a webhook, the local model turning the input into strict JSON through LiteLLM, validation in code, a phone alert whose link opens an approval form, and a wait for the person.

Two practical notes. Back up n8n's database and its encryption key together, because a restored database without the key cannot decrypt its stored credentials. And read the licence: n8n's Sustainable Use License allows use for your own internal business purposes and personal use, and its FAQ says you should not make n8n available to your customers so they can connect their accounts and build workflows. The n8n guide goes further.

Jobs, alerts and checks from outside

Timers and detached jobs

Scheduled work runs as systemd user timers, with linger enabled so they keep running after you log out, inside one resource slice so a batch job cannot starve your desktop. Each run takes a lock, writes a "started" receipt, works in its own folder, and always writes a final state, even when there was nothing to do. Long agent jobs start detached from your SSH session with setsid nohup ./run.sh > run.log 2>&1 < /dev/null & and record their process id, so you stop them by that id rather than by a name pattern that also matches your own SSH command.

ntfy: the alert on your phone

Every alert goes through one small script to ntfy, which pushes to the ntfy phone app. The public ntfy.sh relay has no sign-up, so, in ntfy's own words, "the topic is essentially a password". Make the topic long and random, keep it in an owner-only file, never send it through a chat, and put only ids, statuses and links in messages. If you want more control, run your own ntfy server with access tokens.

ntfy also makes a cheap dead-man's switch. At the end of every good backup, the script schedules a delayed "no backup in 26 hours" message with a fixed sequence id. Each new run replaces the pending message before it is delivered, so it only reaches your phone if the runs stop. ntfy accepts delays from 10 seconds to 3 days, so 26 hours fits. Use an ntfy server other than the machine it is watching.

GitHub Actions: maintenance from outside

You do not need a self-hosted runner. GitHub's own runners reach the server two ways: through an Access service token for health checks, and through a forced-command SSH key for updates, so a workflow can ask the server's update tool for one action and can never open a shell. Design around a few facts from GitHub's documentation: scheduled workflows run in UTC unless you set a timezone, only on the default branch, at most every five minutes, can be delayed at busy times, and are disabled after 60 days without activity in a public repository. Keep the repository private, give the workflow token read access only, and pin each action to a full commit SHA. The kit includes a maintenance workflow, a backup heartbeat that alerts when last night's backup is missing or stale, and a Dependabot file.

Backups: restic locally and on Google Drive

Build the backups before you trust the server with anything, and test a restore before you call them done.

 BACKUP AND RESTORE FLOW

 nightly    backup.sh (restic-backup.timer, OnFailure -> phone alert)
   |
   +--> consistent dumps into a staging folder
   |      Qdrant snapshot API per collection      (never the live storage folder)
   |      pg_dump -Fc per database, read back      (gateway, n8n, Guacamole)
   |      sqlite3 .backup, then integrity_check    (hub, anything on SQLite)
   |      inventory: model list, image digests, timers, packages
   |
   +--> restic backup: home, configs, staging --> LOCAL repository (a second disk if you can)
   |      excludes: caches, venvs, node_modules, model blobs, VM disks,
   |                the secrets that open the backups
   +--> restic check (local)
   +--> restic copy --> OFF-SITE repository on Google Drive
   |      (rclone backend, own OAuth client, drive.file scope)
   +--> restic check --read-data-subset=5% (off-site)
   +--> status file ok; re-arm the dead-man's switch (delayed ntfy alert 26 h out)
   |
 Sunday     retention: local 7 daily + 4 weekly; off-site 7 daily + 5 weekly + 12 monthly;
            then full checks
 monthly    restore-test.sh: restore into a scratch folder, list dumps, integrity-check copies
 daily      outside the server: GitHub Actions reads the status through Access
            (a second dead-man's switch)

 RESTORE
   one file:   restic restore latest --target /tmp/r --include /home/<user>/path/file
   one service: stop it -> restore its dump -> pg_restore, SQLite copy or Qdrant
                snapshot recover -> start
   the server: new box -> section 4 in order with the kit -> restic restore -> dumps
               -> start services

The shape, and why:

• Dump first, then back up. A copy of a live database file can capture a half-written state. So the script takes a Qdrant snapshot of each collection through its API, a pg_dump of each PostgreSQL database, and an online sqlite3 .backup of each SQLite file, and stages them. Docker named volumes live outside your home folder, so list them with docker volume ls and make sure each one has a dump step.
• restic for the snapshots. Snapshots are encrypted before they leave the server and deduplicated, so a nightly run stores only what changed, and each one restores on its own. The password lives in a password manager and in an owner-only file, never inside the repository it opens.
• Two repositories joined by restic copy, not a mirrored folder. A mirror made with rclone sync copies your local pruning to the cloud and deletes the off-site history with it. restic copy only adds.
• Google Drive through rclone, with your own OAuth client. rclone's documentation says its shared client "is being retired and will stop working during 2026", and that each client has a global rate limit shared by everyone using it. Use the drive.file scope so rclone can see only the files it created, and publish the OAuth app instead of leaving it in Testing, where rclone notes that grants expire after a week. An expired grant is how a nightly copy fails quietly.
• Verify before deleting. Local copies are pruned only after the off-site copy has been checked.
• Alarms for silence. An OnFailure= hook on the backup units alerts your phone when the script dies, the ntfy dead-man's switch covers runs that never start, and the daily GitHub workflow checks the status from outside.
• A restore test every month into a scratch folder, listing every dump and checking every copy.

RAID is not a backup. Mirrored drives protect you from one disk failing, not from deleting the wrong folder or losing the server. Guide 2 has the full restore runbook, from one file to a rebuild on new hardware.

CPU and Ubuntu tuning

Most of the speed on a CPU server comes free. On our Cascade Lake Xeons, Ollama loaded its matching CPU backend with AVX-512 and VNNI turned on, with no flags set. You can confirm yours: after a model loads, the line load_backend: loaded CPU backend from ... in Ollama's log names the variant it chose. The system_info line describes how that variant was built, so read /proc/cpuinfo for what your hardware actually supports.

After that, tuning is mostly measuring:

• Threads and NUMA. On a two-socket server, each processor has its own memory, and reading model weights across the link between sockets costs bandwidth. Yet short tests on our server ran faster at 24 threads than at 12. Pin Ollama to one socket and match num_thread to it only if your own measurements say it wins.
• Quantization. On a CPU, use Q4 builds of 1.5B to 8B models. Fewer bytes per weight means fewer bytes read per token, which is what limits speed on this hardware.
• Swap. Many provider images ship with no swap at all. A small compressed zram swap lets a memory spike slow a job down instead of waking the out-of-memory killer. Keep memory limits on background jobs so a runaway job is the one that gets stopped.
• Leave these alone. Transparent hugepages on madvise and the none scheduler on NVMe are already the right defaults. Note that IOSchedulingClass=idle in a unit file does nothing with the none scheduler.
• A reboot window. Ubuntu installs security updates automatically right after installation, but its automatic reboot setting defaults to off. Pick a window so kernel fixes actually take effect.
• No snaps for core services. Use Docker's own packages and each vendor's own install.
• The CPU governor. Test performance against the default for steadier response times, knowing it uses more power.

Guide 2, section 5 has the read-then-change commands for each of these, plus a section for when you add a GPU.

The beginner path: Desktop Commander plus ChatGPT

You do not need to know Linux to build this, as long as you keep the approvals and the snapshots. The guides set up two assistants with separate jobs:

• Claude with Desktop Commander does the hands-on work. Desktop Commander is an MCP server, and its remote mode links your server to your claude.ai account so Claude can run commands and edit files there. It needs Node.js 22.12 or later. You start it on the server with npx @wonderwhy-er/desktop-commander@0.3.0 remote, confirm the code it shows, and add the connector in claude.ai. It accepts commands only while that process is running.
• ChatGPT plans, explains errors and reviews configs, with no access to the server at all.
• Codex CLI on the server reviews in read-only mode in its own folder. codex exec runs in a read-only sandbox by default, and the old --full-auto flag is no longer supported; --sandbox workspace-write replaces it when Codex must edit files.

The loop for every step: the assistant proposes one step and explains what could go wrong; it shows a command that reads the current state, then the command that changes it; nothing runs until you type "approve"; it runs the step, reads the whole output and confirms the result with a second read-only check; and it adds a line to ~/setup-log.md with the time, the command, the result and how to undo it. Before any firewall, SSH, disk, kernel or Docker network change, you take a snapshot at your provider first.

The two starter prompts in the downloads build that loop and the build order into the conversation. Paste the Desktop Commander one into a new Claude chat, fill in the bracketed facts about your server, and it starts by reading the system and proposing step one.

Three safety facts about Desktop Commander, from its own documentation. The blocked-commands list is meant to reduce accidents and is not a security boundary, since shell substitution, absolute paths or another interpreter get around it. The allowed-directories setting covers file operations, not terminal commands, and an empty list grants the whole file system. And it keeps a local log of every tool call with its arguments, unredacted, so a password typed into a command lands in that log. Real isolation comes from the operating system: a separate account, a container or a virtual machine. After the build, narrow its folders, decide on telemetry, and revoke devices you no longer use. Keep the two assistants out of the same files at the same time: each works in its own folder, and one of them, or you, merges.

Desktop Commander, Codex and Claude for a small business is the longer walkthrough.

Keeping it current

An update is done when the new version is installed, running, checked, pinned and reversible. Security patches arrive automatically. Everything else goes weekly, one component at a time, through a tool that records the old version, takes a backup, applies the change, runs a real smoke test and rolls back on failure, with the person's desktop last. Our own tunnel once kept running its old binary for 70 hours after the new one was installed, because nothing compared the running process with the file on disk (one safe update command). Check what is running, not what is installed.

The lessons that cost the most to learn

Guide 2 lists 30 lessons, each tied to something that went wrong or nearly did. These are the ones I would tape to the monitor:

1. A green status proves nothing; check the content. Sign-in pages, empty pages and error pages can all report success.
2. Docker-published ports bypass ufw. Bind every one to 127.0.0.1.
3. Create the Access application before the route, then run a refusal test in a private window.
4. The agent fills, the person sends. An agent that can submit can sign, book or pay on someone's behalf.
5. A message from another agent is data, not consent. Otherwise a refused action can be laundered through a peer.
6. Never put backticks or $( inside a command string you hand to a tool. Write a script file and run that. One stray backtick in a text-editing command once ran a production deploy.
7. Detach long jobs properly, and stop them by their recorded process id. A job tied to your SSH session dies with it.
8. External quotas are account-wide. Keep one ledger per quota and one scheduler per ledger.
9. Pin the embedding model. Mixed embeddings return plausible nonsense.
10. After a major version bump, look for renamed settings. A renamed variable is ignored without an error.
11. RAID is not a backup, and a backup is not proven until you restore it. Check the real exclude file after adding anything large: a 150 GB VM disk nearly went into a nightly backup.
12. Enable and start the timers, and enable linger. A unit file on its own starts nothing.

Where to start

Download the zip, read its README, then read section 1 of guide 2, which lists the decisions you will make whether you plan them or not. Build one layer at a time in the order the guide gives, and do not move on until the layer you just built passes its check. Most people can stop after the gateway and the hub and already have something useful; the rest can come a weekend at a time.

If this saves you an agency invoice, pass the link to one other small business owner who has been quoted thousands of dollars for the same thing.

Related reading

• Sovereign AI made simple: the short version of why a machine of your own is worth it, and where each detailed guide fits.
• The work around the model: the operating practices that keep a server like this healthy day to day.
• Sovereign AI workstation best practices: access, persistence and parallel jobs.
• The sovereign advantage of a dedicated AI workstation: hardware and running costs.
• Build your own AI prompt hub: the plan-then-Run hub in detail.
• One safe update command: update order, checks and rollback.
• Jev and System One models in a sovereign AI stack: typed decisions and what leaves the machine.

Fact-check notes and sources

• The guides and the kit: generalized from a working server. Every hostname, address and account in them is a placeholder, and the templates are not copies of any live server's files. Vendor facts in the guides were checked against vendor documentation on October 15, 2026; versions change often, so check the linked pages before you run a command.
• Server price: from the cost comparison, which records OVHcloud's US Eco catalog at $125 a month with a $125 installation fee on October 5, 2026. Prices and availability change; check the provider's page.
• Our own measurements: the 2 percent load figure comes from the operating guide; the 2.7 times throughput figure (2.66 in the original table, 24 threads, four parallel requests) from the benchmark; the 70-hour tunnel from one safe update command; the page sizes from agent readiness beyond llms.txt. Four characters per token is a rule of thumb for English text, and $3 per million tokens is an example rate for the arithmetic, not a quoted price.
• Docker and ufw: Docker, packet filtering and firewalls, read October 15, 2026: "Docker and ufw use firewall rules in ways that make them incompatible with each other."
• Cloudflare Access: self-hosted applications recommends creating the Access application before the tunnel route, warns that without one the app is available to anyone on the internet, and says Access applications deny by default. Validating JWTs names the Cf-Access-Jwt-Assertion header and the team certs endpoint, and its examples check the issuer and the AUD tag. Both read October 15, 2026.
• Desktops: the webtop documentation states there is no authentication by default and the terminal has passwordless sudo. The Guacamole 1.6.0 manual documents HTTP_AUTH_ENABLED and HTTP_AUTH_HEADER and warns that unsanitized requests make the header trivial to forge. Both read October 15, 2026.
• Ollama: the FAQ gives the 4,096-token default context, the parallel default of 1, RAM scaling with parallel slots times context length, the five-minute keep-alive, and K/V cache quantization with flash attention. Structured outputs covers the schema in format and temperature 0. Decision models requires v0.35.0 or later and lists Nimble, Tev1, Clef and Clef Flash at /v1/systemone. Read October 15, 2026.
• LiteLLM and Qdrant: LiteLLM virtual keys shows keys limited to a model list through /key/generate. Qdrant snapshots restore only to the same minor version or the next one. Read October 15, 2026.
• Hermes Agent: the repository is MIT licensed; the providers page says it "requires at least 64,000 tokens of context for agent use with tools." Read October 15, 2026.
• TypeSafe Jev: the models page lists jev-latest as jev-1.13.0, $0.042 per million input tokens, free output and a 64k-token default request limit. The confidence page gives the formula and the 0.5 and 0.9 example thresholds. Speed and accuracy claims for these models are the vendors' own. Read October 15, 2026.
• n8n licence: the Sustainable Use License page, read October 15, 2026, allows internal business and personal use; its FAQ says not to make n8n available to customers to connect their accounts and build workflows.
• ntfy: the publishing guide, read October 15, 2026: the topic "is essentially a password"; delays run from 10 seconds to 3 days; a new message with the same sequence id replaces a scheduled one before delivery.
• GitHub Actions: events that trigger workflows (five-minute minimum, default branch only, delays at busy times, 60 days in public repositories, UTC unless a timezone is set) and secure use (pin actions to a commit SHA, read-only token by default). Read October 15, 2026.
• rclone and Google Drive: the rclone Drive documentation, read October 15, 2026: the shared client "is being retired and will stop working during 2026", each client has a global rate limit, drive.file sees only files rclone created, and grants for an app left in Testing expire after a week.
• Desktop Commander and Codex: Desktop Commander's security notes and README (advisory command and folder limits, unredacted local tool-call log, remote mode); the npm package lists version 0.3.0 and Node 22.12.0 or later; Codex non-interactive mode says codex exec defaults to read-only and --full-auto is no longer supported. Read October 15, 2026.
• Ubuntu: the 26.04 release notes give 23 April 2026 and the name Resolute Raccoon; automatic updates says security updates are enabled right after installation and Unattended-Upgrade::Automatic-Reboot defaults to false. Read October 15, 2026.

This post is informational, not security or IT consulting advice. Test every template on a server you can rebuild before you trust it with real data. Mentions of Cloudflare, Docker, Ollama, LiteLLM, Qdrant, Apache Guacamole, LinuxServer.io, n8n, ntfy, restic, rclone, Google, GitHub, TypeSafe, Nous Research, OpenAI, Anthropic, Desktop Commander and OVHcloud are nominative fair use. No affiliation is implied.