# Hardening a One-Admin Linux Server: Survey First, Measure Before You Restrict

A hardening checklist never told us about a free security subscription we were missing. Here is a method that surveys the machine first, with a tested kit to download.

Author: J.A. Watte
Published: November 18, 2026
Source: https://jwatte.com/blog/harden-single-admin-linux-server-survey-first/

---

Most hardening advice is a list of settings. Turn this off, set that to no, install this tool. We followed a fair amount of it on our own server and still missed a free subscription that covered a third of our packages. The list was not wrong. It was answering a different question from the one we needed answered.

This guide is the method we used the day after, when we asked what else was missing. It is written for the common small-business case: one Linux server, one person who administers it, a handful of services in containers, and no security team. The worked example, with our own numbers and three recommendations that turned out wrong, is in [the companion article](/blog/server-hardening-eight-items-three-changes/).

Everything here is also in a single file you can follow by hand or give to an AI coding agent: the [Server Hardening Survey Kit](/downloads/server-hardening-survey-kit.md). It includes the commands, a decision table, a container probe script, audit rules, an integrity-check script with its timer, and a paste-ready agent prompt. We ran the kit's install and self-test sequence in a clean Ubuntu 24.04 container before publishing it.

## The method in seven steps

1. Survey the machine with read-only commands.
2. Measure your own traffic before adding any limit.
3. Check entitlements as well as settings.
4. Decide each item on purpose, and let "skip" be a real answer.
5. Probe a throwaway copy before hardening a container.
6. Add detection that stays quiet enough to be read.
7. Check that your update routine covers everything pending.

## 1. Survey first

A generic checklist cannot know what your machine runs. A survey can. Every command in this step only reads.

```bash
pro security-status                 # which packages have security coverage, and from where
sudo ss -tlnp                       # what is listening
sudo ufw status verbose             # what the firewall allows
sudo sshd -T | grep -E '^(passwordauthentication|permitrootlogin|persourcepenalties) '
grep . /sys/devices/system/cpu/vulnerabilities/*
lsblk -o NAME,TYPE,FSTYPE,MOUNTPOINT
systemctl is-active auditd
systemctl list-timers --all --no-pager
```

The kit has the full set, including one line per container that shows its user, privilege flags and dropped capabilities.

Two habits make the survey worth more than its output. Write down what was already right, so the result does not read as though the machine were open. And treat the survey output as sensitive. It is a description of how to attack the machine, so it does not belong in a chat with an outside service or a public ticket.

## 2. Measure before you restrict

A limit that would have blocked your own work last week will block it next week. Before adding a rate limit, an allowlist or a dropped permission, count what it would have stopped.

For logins, three numbers are enough:

```bash
# successful logins in a week
sudo journalctl -u ssh --since "7 days ago" | grep -c 'Accepted '

# the busiest single minute
sudo journalctl -u ssh --since "7 days ago" -o short-iso | grep 'Accepted ' \
  | cut -c1-16 | sort | uniq -c | sort -rn | head -3

# failed attempts in a day
sudo journalctl -u ssh --since "24 hours ago" | grep -cE 'Failed |Invalid user '
```

Then compare with what the limit does. The firewall's `limit` rule, per the [ufw manual](https://manpages.ubuntu.com/manpages/noble/man8/ufw.8.html), will "deny connections if an IP address attempts to initiate 6 or more connections within 30 seconds." If your own automation logs in more often than that from one address, the rule locks it out. Ours peaked at 19 logins in a minute.

Three more things to know before restricting logins:

- **OpenSSH may already be doing it.** Since [version 9.8](https://www.openssh.com/txt/release-9.8) the server can penalize addresses that repeatedly fail authentication and refuse them for a time. Check `sshd -T` for `persourcepenalties` before installing a second tool for the same job.
- **Hosted CI runners change address.** If a build service logs in to deploy, an address allowlist cannot work. Closing the port means building a tunnel or private network path for those jobs first.
- **With key-only login, failed password attempts cannot succeed.** They are noise in the log. They are not progress for the attacker.

The same rule holds for every other restriction. Before dropping a capability from a container, find out what the process reads. Before making a filesystem read-only, find out what it writes.

## 3. The subscription blind spot

This is the step no settings checklist has. A scanner audits configuration. An entitlement, a thing you are allowed to switch on, is outside its view. And the package manager compares your system only against the repositories that are enabled, so a source you never turned on reports no gap at all.

Check these by name:

| Entitlement | Check | What it covers |
|---|---|---|
| Ubuntu Pro ESM | `pro security-status` | Security fixes for community-maintained packages. Free for personal use on up to five machines |
| Livepatch | `pro status` | High and critical kernel fixes applied without an immediate reboot |
| Firmware updates | `fwupdmgr get-updates` | BIOS, drive and device firmware |
| Repositories you added by hand | `ls /etc/apt/sources.list.d/` | Whatever that vendor ships. Confirm each is still wanted |
| The release's support window | `pro status` | Everything, once it ends |

We wrote up the first one in detail in [the Ubuntu Pro ESM article](/blog/ubuntu-pro-esm-free-security-updates/). On our server it surfaced 21 waiting fixes in ten minutes.

## 4. Decide each item, and let "skip" be an answer

The survey gives you a list of candidates. The mistake is to treat the list as a to-do list. Some items are cheap and obviously right. Some cost more than they return on your machine. Some cannot be done without rebuilding. The goal is that every row has a decision and a reason, not that every row is fixed.

| Item | The question that decides it |
|---|---|
| Security subscriptions | Is any source off that costs nothing to turn on? |
| Automatic security updates | Who installs fixes when you are away for two weeks? |
| Firewall default deny | Does anything listen that you did not put there? |
| Admin login reachable from the internet | What logs in, from where, how often? What breaks if it closes? |
| Password and root login | Does any account still need a password over the network? |
| Kernel live patching | How costly is a reboot, and is there already a tested reboot routine? |
| CPU side-channel mitigations | Does the fix need a boot flag and a reboot? Who shares the hardware? |
| Container privilege flags | Does a probe on a throwaway copy start cleanly? |
| Secrets at rest | What role does each readable file have, and is its directory closed? |
| Detection | Would anyone read the alerts? If not, make them quieter first |
| Disk encryption | Who has physical or hypervisor access? Can you enter a passphrase at boot? |
| Update coverage | Is every pending item in your update routine's step list? |
| Backups | When did a restore last work, not when did a backup last run? |
| Admin key protection | Is the key passphrase-protected or hardware-backed, and where do copies live? |
| Sudo scope | Does automation hold full sudo, or only the commands it needs? |
| Outbound traffic | Does anything need to reach the whole internet, or a short list of hosts? |
| Logs off the machine | If the server were wiped tonight, would any record of what happened survive? |
| Third-party access | Who else has a key or an account, and does it still need to exist? |
| Unused services and accounts | What is installed or enabled that nothing uses? |

Many of these have honest "not now" answers. Moving the admin login behind a private network breaks any outside job that logs in until that job has a path. A CPU mitigation that needs a boot flag can wait for a planned reboot. Disk encryption on a running server usually means a reinstall, so it is a decision for the next rebuild. Outbound filtering is a real project on a server that pulls packages and images from many places. Live patching is right for a server nobody reboots on a schedule and optional for one with a tested reboot routine. What matters is that the reason is written down with a date and a condition for looking again.

Two ways a first answer goes wrong, both of which happened to us:

- **A count that was not checked.** A quick look said six containers needed a fix. The careful count was eight, and they were not all the same kind.
- **A mode without a role.** A quick search said four of seven env files were readable by other accounts. The full survey found 43 files. All 28 real ones were already private. The readable ones were templates and test fixtures.

For secrets, three questions settle it. What is the file's role: a real secret, an example, a fixture, or public by nature? Is a directory above it closed to other accounts, which makes the file's own mode irrelevant? And does any other account on the machine exist that should not read it? `namei -l /path/to/file` shows every directory gate on the way down.

## 5. Probe before you harden containers

Two container settings are worth having on almost every service.

`no-new-privileges` stops a process from gaining privilege when it runs another program. The kernel's [documentation](https://docs.kernel.org/userspace-api/no_new_privs.html) names what it blocks: "the most obvious examples are setuid/setgid programs and file capabilities." Dropping capabilities removes specific root powers. Docker's [security page](https://docs.docker.com/engine/security/) says the engine already drops many by default, "an allowlist instead of a denylist approach", and `cap_drop: ALL` takes away the rest.

Either one can stop a service from starting. So try it on a copy that nothing depends on: same image, no network, capped memory, removed afterwards. The kit has a 15-line script for this. We ran it on the public Postgres 16 image two ways.

With every capability dropped, the container exited at once:

```text
state: exited exit=1
chmod: changing permissions of '/var/lib/postgresql/data': Operation not permitted
chmod: changing permissions of '/var/run/postgresql': Operation not permitted
error: failed switching to 'postgres': operation not permitted
```

With five capabilities added back (CHOWN, DAC_OVERRIDE, FOWNER, SETGID, SETUID), it started and reported "database system is ready to accept connections". Those five are what its startup script uses to fix ownership of its data folder and switch to its service user. The [capabilities manual](https://man7.org/linux/man-pages/man7/capabilities.7.html) describes each one.

In a Compose file the result looks like this:

```yaml
services:
  db:
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    cap_add: [CHOWN, DAC_OVERRIDE, FOWNER, SETGID, SETUID]
```

A probe is necessary and not sufficient. Three things it does not show:

- **Existing data.** A database's first start and its second start on existing data take different paths. Probe the second one too.
- **Live mounts.** A clean copy mounts none of your host folders. If root in the real container reads a folder that only another user may read, it is relying on a capability, and dropping everything breaks it with no clear error. List the real mounts and who owns them. This is how we found that one of our services needed DAC_READ_SEARCH added back.
- **Paths the probe never ran.** Migrations, first-run setup, plugins. Say which ones you did not exercise.

Then change the definition only, and prove the change only added lines: render the full configuration before and after with `docker compose config` and compare. The flags take effect the next time the container is recreated. Keep the old file so you can go back.

Be honest about how much this buys. A container that already runs as root gains little from `no-new-privileges` alone. The capability drop is the larger change for those. And two stronger measures are left out here on purpose. User-namespace remapping changes who owns existing data, and Docker's [own guide](https://docs.docker.com/engine/security/userns-remap/) says it is best enabled on a new installation. Read-only root filesystems need their own probe.

## 6. Detection that stays quiet

Prevention fails silently. If someone adds a login key or a scheduled job, nothing in a firewall or a patch level will tell you. Detection is the answer, and the usual way it fails is noise. A check that fires every day gets ignored by the second week.

**Audit rules: watch configuration, not commands.** Put file watches on the places that change who can log in, what runs with privilege and what starts by itself: the account files, sudo rules, the SSH server configuration, authorized keys, scheduled jobs, service definitions, the firewall, the container engine's configuration, the package sources, and the audit configuration itself. Add kernel module loading. Do not add a rule that records every command. Sudo already writes an audit record for each command once the audit service runs, and on a server with automation the flood hides everything else. The kit has a 20-rule file to start from.

One trap for scripts and agents: the audit search tools read standard input when they are not attached to a terminal. In a script they wait forever or swallow the rest of the script. The [manual](https://man7.org/linux/man-pages/man8/ausearch.8.html) gives the fix, `--input-logs`, which it says "is needed if you are using ausearch from a cron job."

**An integrity baseline that sorts out package updates.** [AIDE](https://aide.github.io/) records what a set of files looks like and reports differences. Out of the box, every package update is a difference. Two changes make it readable.

First, narrow the scope to places where a change means someone altered how the machine boots, logs in, schedules work or runs privileged code. Leave out data, caches and anything that routine updates regenerate, and write each exclusion down as a known gap. Record ownership, permissions, link targets and a content hash. Leave out timestamps, which change on every update without the file changing.

Second, ask the package manager about each difference. If the file belongs to an installed package and `dpkg --verify` says it matches what the package shipped, it is an ordinary update, and the baseline can move forward by itself. Anything else waits for a person. The kit's wrapper script does this and requires a written reason to accept a change.

We tested the published script in a clean container. The results show the behavior better than a description:

| What we did | Result |
|---|---|
| Built the baseline | 644 entries |
| Planted a file in a system folder | Flagged: "no package owns it" |
| Installed a package of plain files | Accepted automatically, baseline moved forward |
| Edited a packaged configuration file | Flagged: "differs from package" |
| Added a line to `authorized_keys` | Flagged |
| Tried to accept with no reason | Refused |
| Removed a package | Accepted automatically |
| Deleted a watched file with no package activity | Flagged |

The test also found a limit and a bug. The limit: installing a package that ships a symbolic link left one entry for a person, because the package database stores no link targets to verify against. The [dpkg manual](https://man7.org/linux/man-pages/man1/dpkg.1.html) says "the only functional check performed is an md5sum verification of the file contents." Expect an occasional accept after installing new software. The bug: the accept command failed on a system without systemd. That is fixed in the published version, and it is the reason we test a kit before handing it to anyone.

Wire the failed check into something you already look at, such as the list of failed units or an existing health script. A new alert channel is one more thing to ignore.

**One baseline scan.** [Lynis](https://cisofy.com/lynis/) is a good one-time second opinion. Treat its hardening index as a number to compare against on this machine later, not a grade. Most of its suggestions are generic, and on a key-only server with one administrator, password ageing and legal banners matter far less than a service you did not know was listening. Check your timers afterwards: on our server, installing the package scheduled a daily scan by itself.

Know the limit of all of this. A baseline stored on the same machine can be rewritten by someone who already has root. This catches the tier below that: a changed key file, a new unit, an edited configuration, a replaced binary.

## 7. Does your update routine cover everything?

"We have an update command" and "everything that needs updating is in it" are different claims. List every pending item and write beside it which step of your routine handles it: operating system packages, the kernel, the container engine, each service image, tools installed outside the package manager, snaps, firmware, and bundled certificate stores inside images and language runtimes.

On our server this took ten minutes and found three gaps. One of them is easy to miss anywhere: when an operating system stops trusting a certificate authority, copies of the old list can live on inside a language package manager's vendored bundle, in caches and in runtimes. None of those is updated by updating the operating system.

If you add live patching, count its delivery channel too. On Ubuntu it arrives as a snap, and snaps by default [refresh four times a day](https://snapcraft.io/docs/how-to-guides/manage-snaps/manage-updates/) by themselves. If your rule is that every change goes through one reviewed routine, that is an exception to record, and `snap refresh --hold` is the control.

## What this does not protect against

- Someone who already has root, or who has your admin key and its passphrase.
- A vulnerable application. Hardening the host does not fix the code running on it.
- A compromised dependency or image. That needs version pinning, a hold on brand-new releases and a supply-chain check.
- Physical or hypervisor access when disks are not encrypted.
- Anything you chose to skip. That is what the written reasons are for.

## Using the kit

[Download the Server Hardening Survey Kit](/downloads/server-hardening-survey-kit.md). It is one Markdown file. Sections 1 and 2 are read-only and safe to run first. Section 0 holds rules for an AI agent: read-only before anything else, never print the contents of a secret, one change at a time with a way back, no restart without a yes, and correct a wrong first answer plainly. The last section is a prompt you can paste with the file attached.

If you only do three things from this guide: run `pro security-status`, count your own logins before adding any limit, and write one line of reason beside everything you decide to leave alone.

## Fact-check notes and sources

- **Our records:** the kit's files were extracted from the published Markdown and run in a clean Ubuntu 24.04 container on November 18, 2026, with AIDE 0.18.6 and Python 3.12.3: baseline of 644 entries and the eight results in the table above. The container probe script was run the same day against the public Postgres 16 image with no network, twice. The login count (peak of 19 in one minute), the 43 env files and the container counts come from the survey of our own server described in [the companion article](/blog/server-hardening-eight-items-three-changes/). The 21 waiting fixes are from [the Ubuntu Pro ESM article](/blog/ubuntu-pro-esm-free-security-updates/).
- **Firewall limit rule:** [ufw manual page](https://manpages.ubuntu.com/manpages/noble/man8/ufw.8.html).
- **OpenSSH penalties:** [OpenSSH 9.8 release notes](https://www.openssh.com/txt/release-9.8) and the [sshd_config manual](https://man.openbsd.org/sshd_config), `PerSourcePenalties`.
- **Livepatch:** Canonical, [Canonical Livepatch Client](https://snapcraft.io/canonical-livepatch), which "patches high and critical linux kernel vulnerabilities" and is delivered as a snap. **Snap refresh schedule and holds:** Canonical, [Manage updates](https://snapcraft.io/docs/how-to-guides/manage-snaps/manage-updates/).
- **The no-new-privileges flag:** kernel documentation, [No New Privileges Flag](https://docs.kernel.org/userspace-api/no_new_privs.html). **Capabilities:** [capabilities(7)](https://man7.org/linux/man-pages/man7/capabilities.7.html).
- **Docker:** [Docker Engine security](https://docs.docker.com/engine/security/), [docker container run](https://docs.docker.com/reference/cli/docker/container/run/), [Compose service options](https://docs.docker.com/reference/compose-file/services/) and [user namespace remapping](https://docs.docker.com/engine/security/userns-remap/).
- **Audit tools:** [ausearch(8)](https://man7.org/linux/man-pages/man8/ausearch.8.html) and [auditctl(8)](https://man7.org/linux/man-pages/man8/auditctl.8.html). The auditctl page marks the older `-w` watch form as deprecated in favor of the syscall form; the kit notes both.
- **Integrity checking:** [AIDE](https://aide.github.io/) and its [manual](https://aide.github.io/doc/). **Package verification:** [dpkg(1)](https://man7.org/linux/man-pages/man1/dpkg.1.html), `--verify`.
- **Baseline scan:** [Lynis](https://cisofy.com/lynis/), CISOfy.

## Related reading

- [We Surveyed Our Server for 8 Hardening Gaps](/blog/server-hardening-eight-items-three-changes/): this method applied to our own machine, including the three first answers that were wrong.
- [A Third of Our Server's Packages Had No Security Promise](/blog/ubuntu-pro-esm-free-security-updates/): the subscription blind spot in full, with its own setup kit.
- [Updated on Disk, Old in Memory](/blog/safe-update-command-self-hosted-ai-server/): why installing a fix is not the same as running it.
- [Build a Self-Hosted AI Server From Scratch](/blog/self-hosted-ai-server-from-scratch/): the kind of server this guide is written for.
- [The check was green because it measured a state no visitor is ever in](/blog/blog-green-checks-that-cannot-fail/): on checks that pass without proving anything.

*This post is informational, not security-consulting or legal advice. Read every command before you run it and test changes on a copy of your own system first. Mentions of Ubuntu, Canonical, Livepatch, OpenSSH, Docker, PostgreSQL, AIDE, Lynis and CISOfy are nominative fair use. No affiliation is implied.*


---

Canonical HTML: https://jwatte.com/blog/harden-single-admin-linux-server-survey-first/
RSS: https://jwatte.com/feed.xml
JSON Feed: https://jwatte.com/feed.json
Hero image: https://jwatte.com/images/harden-single-admin-linux-server-survey-first.webp
