# Sovereign Git Server and Isolated Runner Kit Version: November 1, 2026. Companion article: https://jwatte.com/blog/sovereign-git-server-self-hosted-runners/ **Keep GitHub when it meets the need. Start with one isolated runner if compute is the problem. Move the forge only when ownership, contracts, disconnected operation or measured costs justify operating it.** This is a phased setup guide for a new server, not an unattended installer. The commands are adapted from the official documentation linked beside them. Replace every angle bracket placeholder before running a command. The deployment paths below describe directories you create on your own target machines. They are not links to private source material. This kit has not been deployed or load tested for the article. Validate syntax, login, job isolation and recovery on disposable infrastructure before moving real repositories. Do not run these instructions on a machine that already hosts unrelated work. No service should become public during setup. ## 1. Make the decision - [ ] Identify the problem: CI charges, artifact retention, GPU work, residency, disconnected operation or seat costs. - [ ] Record the current bill, actual runner minutes, peak concurrency, package storage, LFS traffic and administrator time. - [ ] Compare GitHub with hosted runners, GitHub with your runners, and a self hosted forge with your runners. - [ ] Name the administrator and backup administrator. Give both an independent provider recovery route. - [ ] Decide which client data may reach GitHub, Cloudflare, the GPU provider and model APIs. A private server does not change a provider's contract. - [ ] Set recovery point and recovery time objectives. These mean the amount of recent work you can lose and how long recovery can take. - [ ] Price migration and feature gaps separately: identity, code scanning, project management, compliance records and integrations. - [ ] Reject a single server where its outage would exceed your business tolerance. Our foundation is the [server build guide](https://jwatte.com/downloads/self-hosted-ai-server-build-guide.md), [hardening checklist](https://jwatte.com/downloads/self-hosted-ai-hardening-checklist.md), [safe update kit](https://jwatte.com/downloads/safe-update-command-kit.md) and [workstation operations kit](https://jwatte.com/downloads/sovereign-ai-workstation-ops-kit.md). Keep deterministic work in ordinary code, use local models for suitable private extraction, and give expensive models the work that needs their judgment. A person approves releases, production changes and external messages. ## 2. Hardware and hosting worksheet These are fields to measure, not performance promises. Separate the forge from machines that execute repository code. | Item | Your value | Acceptance check | | --- | --- | --- | | Forge host, region and monthly term | `` | Location and recovery access documented | | CPU, RAM, disk and growth | `` | Clone, search and restore fit with spare capacity | | CPU runner VM pool | `` | Each VM handles one job and is then destroyed | | GPU type and video memory | `` | Your model and context fit; timed workload completes | | Maximum simultaneous jobs | `` | No memory exhaustion, excessive queue or disk pressure | | Primary data, artifacts and LFS | `` | Retention limits and growth alerts set | | Separate local backup disk | `` | Loss of the primary disk leaves a readable copy | | Off site encrypted backup | `` | Credentials and encryption password recoverable separately | | Recovery test interval | `` | Calendar owner and independent failure alert | | Setup and monthly labor | `` | Low and high cases both budgeted | A small VPS can host a lightweight forge. A dedicated CPU server suits sustained builds and several isolated guests. Rent GPU capacity for bursts unless measured utilization supports a fixed commitment. Vercel is an option for the public static site and previews; it is not the persistent forge database or runner host. ## 3. Install Forgejo with PostgreSQL Prerequisites: a new Linux machine, administrator access through the provider console, a supported Docker Engine and Compose installation, an approved DNS name, and a Cloudflare account if using the tunnel below. Use your distribution and Docker's official installation instructions. Do not paste an unknown installer into a privileged shell. Use the [Forgejo Docker installation guide](https://forgejo.org/docs/latest/admin/installation/docker/) for the image, volume and database configuration. Select supported releases and record their immutable image digests. This template deliberately leaves version selection to the operator instead of silently following a floating tag. As the administrator, create the deployment folder: ```sh sudo mkdir -p /srv/forge/data /srv/forge/postgres /srv/forge/cloudflared sudo chown -R "$(id -u):$(id -g)" /srv/forge cd /srv/forge umask 077 ``` Create `.env` with an editor. Generate a long, fresh random hexadecimal database password in your password manager. Restrict this value to hexadecimal characters so Compose interpolation and comment syntax cannot alter it. The same value supplies Forgejo and PostgreSQL. Never commit this file or paste the resulting values into chat. Docker administrators can inspect container environments, so treat engine access as privileged. ```dotenv FORGEJO_IMAGE=codeberg.org/forgejo/forgejo@sha256: POSTGRES_IMAGE=postgres@sha256: CLOUDFLARED_IMAGE=cloudflare/cloudflared@sha256: FORGE_DOMAIN=git. DB_PASSWORD= ``` Save `compose.yaml`. This adapts the official Forgejo PostgreSQL example and keeps services on private container networks. There is no published host port. The database major version and its data directory must match the image you chose; the example mount below fits PostgreSQL 17 and earlier. Review the [official PostgreSQL image instructions](https://hub.docker.com/_/postgres) before choosing a newer major. ```yaml services: db: image: ${POSTGRES_IMAGE} restart: unless-stopped environment: POSTGRES_DB: forgejo POSTGRES_USER: forgejo POSTGRES_PASSWORD: ${DB_PASSWORD} volumes: - ./postgres:/var/lib/postgresql/data networks: [database] healthcheck: test: [CMD-SHELL, 'pg_isready -U forgejo -d forgejo'] interval: 10s timeout: 5s retries: 12 server: image: ${FORGEJO_IMAGE} restart: unless-stopped environment: USER_UID: '1000' USER_GID: '1000' FORGEJO__database__DB_TYPE: postgres FORGEJO__database__HOST: db:5432 FORGEJO__database__NAME: forgejo FORGEJO__database__USER: forgejo FORGEJO__database__PASSWD: ${DB_PASSWORD} FORGEJO__server__DOMAIN: ${FORGE_DOMAIN} FORGEJO__server__ROOT_URL: https://${FORGE_DOMAIN}/ FORGEJO__server__DISABLE_SSH: 'true' FORGEJO__service__DISABLE_REGISTRATION: 'true' FORGEJO__actions__ENABLED: 'true' volumes: - ./data:/data networks: [database, edge] depends_on: db: condition: service_healthy tunnel: image: ${CLOUDFLARED_IMAGE} restart: unless-stopped command: [tunnel, run] volumes: - ./cloudflared:/etc/cloudflared:ro networks: [edge] networks: database: internal: true edge: ``` Check the UID and GID against your intended Forgejo account. Follow the image's ownership guidance for the data folder. Do not grant the runner access to these volumes or to the forge host's Docker socket. Finish section 4 before starting the tunnel. Then run the following in this deployment folder. `config -q` checks without printing the interpolated password. See the [Compose command reference](https://docs.docker.com/reference/cli/docker/compose/). ```sh chmod 600 .env docker compose config -q docker compose pull docker compose up -d docker compose ps ``` Open the protected forge address. Complete the initial administrator setup, confirm registration remains closed, enable two factor authentication and create a separate daily account. Create one disposable repository. Test a clone, a commit, an issue and a release before importing anything valuable. ## 4. Put Access before the tunnel Follow [Cloudflare's locally managed tunnel guide](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/get-started/create-local-tunnel/) and [origin Access validation settings](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/configure-tunnels/origin-configuration/). The existing [Cloudflare development server kit](https://jwatte.com/downloads/kimsufi-vps-cloudflare-tunnel-access-kit.md) explains the surrounding account and recovery steps. 1. Create a self hosted Access application covering `git.` before adding its DNS route. 2. Add an Allow policy for named email addresses or your identity provider group. Require the appropriate identity provider MFA. Do not select everyone. 3. Record the Access application audience identifier and team name. 4. Install the verified `cloudflared` client on your administration machine, then run the commands below. The browser login belongs to the human administrator. ```sh cloudflared tunnel login cloudflared tunnel create sovereign-forge cloudflared tunnel route dns sovereign-forge git. ``` Copy only this tunnel's generated credential JSON to the target server's `cloudflared` directory as `tunnel.json`. Keep the account management certificate off the server. Give the connector read access to its configuration and tunnel credential only. Save the following as `cloudflared/config.yml`: ```yaml tunnel: credentials-file: /etc/cloudflared/tunnel.json ingress: - hostname: git. service: http://server:3000 originRequest: access: required: true teamName: audTag: - - service: http_status:404 ``` The HTTP hop is confined to the private container network. Cloudflare terminates public TLS and is a trusted intermediary. Choose private end to end access instead if that trust conflicts with your requirements. A tunnel does not make the app air gapped. From an external browser, test an approved account, an unapproved account and a signed out session. Also test the API, Git clone, LFS and release download paths. Confirm the provider firewall exposes no forge port. Keep provider recovery access independent of the tunnel. ### Automation and Git authentication Create a dedicated [Access service token](https://developers.cloudflare.com/cloudflare-one/access-controls/service-credentials/service-tokens/) with a Service Auth policy and an expiry. An Allow policy alone will still ask an unattended client to sign in. Access authorization and Forgejo authorization are separate: passing Access does not grant repository access. For an HTTP smoke test, put these two lines in a protected curl configuration file, `access-test.conf`, replacing placeholders with the token values in an editor. The flags are curl configuration keys; the secrets stay out of the command line. ```text header = "CF-Access-Client-Id: " header = "CF-Access-Client-Secret: " ``` ```sh chmod 600 access-test.conf curl -f -sS -K access-test.conf https://git./api/v1/version ``` For Git over HTTPS, put the headers under a URL-scoped `http` section in a protected Git configuration file on the authorized client. Use a credential helper for Forgejo credentials, never a password inside the remote URL. See [Git configuration](https://git-scm.com/docs/git-config) and Cloudflare's service token guide. ```gitconfig [http "https://git./"] extraHeader = CF-Access-Client-Id: extraHeader = CF-Access-Client-Secret: ``` Do not place those headers in job repository files. A runner also calls the forge API through its own HTTP client. Git headers alone do not configure that client. The following gateway supplies them to both kinds of request without exposing the token to the job container. Never add an Access Bypass rule to make a runner work. ### Complete runner route through an Access gateway This is an operator template assembled from [Caddy reverse proxy headers](https://caddyserver.com/docs/caddyfile/directives/reverse_proxy), [Caddy interface binding](https://caddyserver.com/docs/caddyfile/directives/bind), [environment placeholders](https://caddyserver.com/docs/caddyfile/concepts), [Docker bridge options](https://docs.docker.com/engine/network/drivers/bridge/) and [Ubuntu firewall instructions](https://documentation.ubuntu.com/server/how-to/security/firewalls/). Its combined behavior still needs the runtime acceptance tests below. Run it on each disposable runner VM, never the forge host. Create one private job network. The bridge name is fixed so the firewall can recognize it. Record the gateway IPv4 printed by the second command as ``: ```sh docker network create -o com.docker.network.bridge.name=forge-jobs forge-job docker network inspect -f '{{(index .IPAM.Config 0).Gateway}}' forge-job ``` Choose an approved Caddy release and its architecture-specific archive from the [official releases](https://github.com/caddyserver/caddy/releases), following [installation guidance](https://caddyserver.com/docs/install). Verify the downloaded archive against the release's published checksum before extracting and installing it. This example is for Linux AMD64: ```sh CADDY_VERSION='' curl -fLo caddy.tar.gz "https://github.com/caddyserver/caddy/releases/download/v${CADDY_VERSION}/caddy_${CADDY_VERSION}_linux_amd64.tar.gz" printf '%s %s\n' '' caddy.tar.gz | sha256sum -c - tar -xzf caddy.tar.gz caddy sudo install -m 0755 caddy /usr/local/bin/caddy sudo useradd -r -s /usr/sbin/nologin forge-gateway sudo mkdir -p /etc/forge-gateway /var/lib/forge-gateway sudo chown forge-gateway:forge-gateway /var/lib/forge-gateway ``` Create `/etc/forge-gateway/gateway.env` with an editor. Give it root ownership, group `forge-gateway` and mode 640. The administrator injects these values during provisioning; they never enter a repository or job image: ```dotenv RUNNER_PRIVATE_IPV4= FORGE_DOMAIN=git. CF_ACCESS_CLIENT_ID= CF_ACCESS_CLIENT_SECRET= XDG_CONFIG_HOME=/var/lib/forge-gateway/config XDG_DATA_HOME=/var/lib/forge-gateway/data ``` After saving the environment file, set its access explicitly: ```sh sudo chown root:forge-gateway /etc/forge-gateway/gateway.env sudo chmod 640 /etc/forge-gateway/gateway.env ``` Save `/etc/forge-gateway/Caddyfile`. This is a fixed upstream reverse proxy, not a general internet proxy. It overwrites Access headers and preserves the independent Forgejo Authorization header. Its HTTP listener exists only on the VM's private job bridge; the upstream uses verified HTTPS: ```caddyfile { admin off auto_https off } http://:8080 { bind {$RUNNER_PRIVATE_IPV4} reverse_proxy https://{$FORGE_DOMAIN} { header_up Host {$FORGE_DOMAIN} header_up CF-Access-Client-Id {env.CF_ACCESS_CLIENT_ID} header_up CF-Access-Client-Secret {env.CF_ACCESS_CLIENT_SECRET} } } ``` On this new runner VM, establish a default-deny incoming host firewall before starting Caddy. Preserve your already verified management access rule and provider console route. The exact management rule depends on your network. These UFW rules admit only the local host and this VM's job bridge to the gateway: ```sh sudo ufw default deny incoming sudo ufw allow in on lo to port 8080 proto tcp sudo ufw allow in on forge-jobs to port 8080 proto tcp sudo ufw enable sudo ufw status verbose ``` Do not enable a firewall over an unprotected remote session before preserving its management rule. No Docker host port is published for Caddy; it runs directly as the dedicated unprivileged host account. This distinction matters because Docker-published ports can bypass ordinary UFW rules. In a separate supervised session, run `sudo -u forge-gateway sh -c 'set -a; . /etc/forge-gateway/gateway.env; exec /usr/local/bin/caddy run --config /etc/forge-gateway/Caddyfile'`. The runner VM lifecycle owns this process. If it exits, jobs should fail rather than falling back to an unprotected forge address. Test the route as the runner account: ```sh curl -f -sS http://:8080/api/v1/version git config -f "$HOME/.gitconfig" 'url.http://:8080/.insteadOf' 'https://git./' ``` The URL rewrite is narrowly scoped to this one forge. It contains no credential. Add the same `git config` invocation as the first job step before checkout, because each job container has its own Git configuration. The gateway supplies Access identity; Forgejo still requires its own job token or repository credential. For a real clone test, create a temporary account restricted to the disposable test repository, then create a token with that repository selected and `read:repository` scope, following [Forgejo token scopes](https://forgejo.org/docs/latest/user/token-scope/). Inject the token as `/run/job-secrets/forge-token`, mode 600, owned by the job user. Revoke it after the test. The job is deliberately allowed to read this narrow credential; it receives no administrator credential. Use Git's documented [askpass mechanism](https://git-scm.com/docs/gitcredentials) instead of putting a token in a URL or command argument. Save this helper as `/opt/job-tools/forge-askpass`, make it executable, and include it in the approved job image: ```sh #!/bin/sh case "$1" in *Username*) printf '%s\n' "$FORGE_GIT_USERNAME" ;; *Password*) cat "$FORGE_GIT_TOKEN_FILE" ;; *) exit 1 ;; esac ``` Then run this inside the one-job container. It uses the gateway URL explicitly, so the credential is requested for that exact clone connection rather than relying on a checkout action's header matching. Git's existing credential helpers are disabled for this command only: ```sh LC_ALL=C \ GIT_TERMINAL_PROMPT=0 \ GIT_ASKPASS=/opt/job-tools/forge-askpass \ FORGE_GIT_USERNAME='' \ FORGE_GIT_TOKEN_FILE=/run/job-secrets/forge-token \ git -c credential.helper= clone http://:8080//.git source git -C source rev-parse HEAD ``` Compare that commit with the approved test commit in the forge. An unexpected hostname or redirect is a failed acceptance test, not a reason to broaden credential scope. For a checkout action, explicitly configure its server URL and token inputs for this route and test the pinned action version; the generic Git rewrite alone does not prove its authentication headers match. Acceptance tests: from a job container on `forge-job`, the version request and authorized Git clone must work; an unauthorized Forgejo repository must remain inaccessible; another VM and the public internet must not reach port 8080. Also test artifact upload and download, LFS, clone redirects and release links. Forgejo retains its public root URL, so a client following an absolute URL may need an explicit gateway route; do not assume the Git rewrite configures non-Git clients. Resolve those paths in the tested job image before calling the setup ready. Then revoke the Access service token and confirm requests fail. Restore a newly issued token before real work. Verify the gateway process cannot read forge or backup data. The gateway's plaintext HTTP hop is confined to this single-use VM and its one job network. Do not use this design for mixed-trust concurrent jobs. ## 5. Register an isolated CPU runner Use [Forgejo's binary installation](https://forgejo.org/docs/latest/admin/actions/installation/binary/), [registration](https://forgejo.org/docs/latest/admin/actions/registration/), [configuration](https://forgejo.org/docs/latest/admin/actions/configuration/) and [security guide](https://forgejo.org/docs/latest/admin/actions/security/). This design requires a separate disposable runner VM. Containers inside the forge host are not the security boundary. 1. Build a clean runner VM image with Git and a supported container engine. Download the selected runner release and detached signature using the official installation guide's URL pattern: ```sh RUNNER_VERSION='' RUNNER_ARCH='' RUNNER_URL="https://code.forgejo.org/forgejo/runner/releases/download/v${RUNNER_VERSION}/forgejo-runner-${RUNNER_VERSION}-linux-${RUNNER_ARCH}" curl -fLo forgejo-runner "$RUNNER_URL" curl -fLo forgejo-runner.asc "${RUNNER_URL}.asc" ``` Check the signing key fingerprint against Forgejo's installation guide and import that verified key. Run `gpg --verify forgejo-runner.asc forgejo-runner`, and stop on any signature failure. Only then install it: ```sh sudo install -m 0755 forgejo-runner /usr/local/bin/forgejo-runner forgejo-runner -v ``` 2. Create a dedicated unprivileged runner account. Give its engine access only to this disposable VM. Keep provider administration, backup deletion and production credentials out of the image. 3. Complete the Access gateway above and its negative access tests. Deny access from this VM to database, backup and infrastructure administration networks. Allow only needed dependency sources and artifact destinations. 4. Generate the configuration as the runner account: ```sh umask 077 forgejo-runner generate-config > runner-config.yml chmod 600 runner-config.yml ``` 5. Register at repository scope for the initial trial. Current Forgejo has a Create new runner dialog that returns a UUID and token for `server.connections`. For this ephemeral path, use the [server CLI](https://forgejo.org/docs/latest/admin/command-line/) from the forge deployment directory. Confirm the application account and locate its configuration without printing that file's secrets: ```sh docker compose exec -T server sh -c 'id git; ls /data/*/conf/app.ini' ``` Use the reported configuration path as `` below. This standard image uses the `git` application account; stop and inspect the selected image if that identity check fails. Generate the runner secret directly into a protected file: ```sh umask 077 docker compose exec -T -u git server forgejo -c '' forgejo-cli actions generate-secret > runner-secret chmod 600 runner-secret ``` Then run `docker compose exec -T -u git server forgejo -c '' forgejo-cli actions register --name -s / --secret-file /dev/stdin --ephemeral < runner-secret`. The documented secret-file option reads the protected input without placing its value in the command line. Record the returned UUID. Transfer the secret into the runner's protected configuration through your provisioning system, then remove the temporary secret file. Do not print it, retain it in terminal history or give the runner the forge administrator's credential. The registration runs outside job execution. A short lived credential is still a credential. 6. Put the returned UUID and token into the generated configuration. Merge these fields without discarding the other generated defaults: ```yaml runner: capacity: 1 labels: - 'cpu:docker://@sha256:' container: privileged: false network: forge-job valid_volumes: [] docker_host: '-' server: connections: private-forge: url: http://:8080/ uuid: token: ``` The shared `forge-job` network is limited to the single disposable VM and one job. Never reuse this network across mixed-trust jobs or put the forge or database on it. The `docker_host` value shown lets the runner use its engine without mounting that socket into job containers. See [Forgejo's Docker access documentation](https://forgejo.org/docs/latest/admin/actions/docker-access/). Do not change it to automount merely to make a build work. 7. Start the single-job runner using the documented ephemeral mode: ```sh forgejo-runner one-job -c runner-config.yml ``` 8. After the job ends, export the necessary job log, revoke residual credentials and destroy the entire VM and its writable disk. Provision the next VM from the clean image. Do not restart this VM as a fresh runner. Forgejo's ephemeral registration limits job assignment; it does not erase a disk. Put a maximum lifetime on idle VMs too. For an ordinary persistent runner, the documented command is `forgejo-runner daemon -c runner-config.yml`. Use that only in a separately justified trusted pool. Never execute untrusted pull request code on a persistent runner that can reach secrets. Test with a harmless `.forgejo/workflows/smoke.yml` before allowing a real build. The label must match the registered runner: ```yaml name: runner-smoke on: workflow_dispatch jobs: check: runs-on: cpu steps: - run: git config -f "$HOME/.gitconfig" 'url.http://:8080/.insteadOf' 'https://git./' - run: id - run: git version ``` The test must show the expected restricted identity. Also verify that the job cannot read forge data, query the database, open the engine socket or reach backup administration. Do not turn a failed negative test into an accepted exception without a documented isolation redesign. ## 6. Add a GPU runner without giving it the forge Rent or provision a separate GPU VM or a dedicated machine with GPU passthrough. This runner procedure requires control of the host driver and container engine. RunPod Pods, used in the cost worksheet, are container-based inference rentals; do not assume they provide a disposable VM, nested Docker or equivalent isolation. That cost scenario uses the controller to launch approved inference in a Pod and collect outputs. It is not a claim that this VM runner procedure runs inside a Pod. See [RunPod storage and container behavior](https://docs.runpod.io/pods/storage/types). Install the provider-approved driver and NVIDIA Container Toolkit, then confirm the device is visible. Use the [Docker Compose GPU guide](https://docs.docker.com/compose/how-tos/gpu-support/) for the reservation syntax. GPU labels select a machine; they do not install a driver or grant device access. For an initial trusted trial, register another ephemeral Forgejo runner with the label `gpu:host://`. Host execution is deliberate here: the dispatch step starts the approved GPU container on a single-use VM. Host jobs can read the runner account and control the VM's Docker daemon. That also lets them read the Access gateway credential through a host mount, regardless of the gateway's separate account. Treat that token as exposed to this trusted job: allow only reviewed workflows, use an application-scoped short lived token, and revoke it when the VM is destroyed. Give untrusted code a different pool with no secrets. The server remains elsewhere. Save `gpu-job.compose.yaml` on that runner image, pinning an approved CUDA image compatible with its driver: ```yaml services: gpu-check: image: @sha256: command: [nvidia-smi] deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu] ``` First run `docker compose -f gpu-job.compose.yaml run --rm gpu-check`. Then replace the smoke command with your reviewed inference task, mount only the input and output directories it requires, and give any model gateway credential a job-specific budget and expiry. Do not copy a personal agent login into the image. An example dispatch workflow, adapted from the Forgejo label and Docker GPU guides: ```yaml name: gpu-smoke on: workflow_dispatch jobs: gpu-check: runs-on: gpu steps: - run: nvidia-smi - run: docker compose -f /opt/approved-jobs/gpu-job.compose.yaml run -T gpu-check ``` The `run` command propagates the container exit status. It leaves a stopped job container for this example, which disappears when the whole VM is destroyed. If using `up` instead, the invocation `docker compose -f /opt/approved-jobs/gpu-job.compose.yaml up --abort-on-container-exit --exit-code-from gpu-check` supplies exit status propagation. Keep the reviewed file outside repository-controlled paths. Upload only approved artifacts, record GPU hours and destroy the VM after the job or timeout. Model weights, caches, attached disks and stopped instances can continue to cost money depending on the provider. ## 7. Hybrid: keep GitHub and attach your runner Use [GitHub's adding a runner guide](https://docs.github.com/en/actions/how-tos/manage-runners/self-hosted-runners/add-runners) and [ephemeral runner reference](https://docs.github.com/en/actions/reference/runners/self-hosted-runners). 1. Keep the repository on GitHub. Create a separate disposable CPU or GPU VM using the isolation rules above. 2. In repository Settings, Actions, Runners, select New self-hosted runner. Choose the VM's architecture and operating system. 3. Copy GitHub's displayed download command and checksum verification for that runner release. Do not substitute an old binary URL from an article. 4. Run its displayed configuration command with the additional `--ephemeral` option. Scope it to this repository or an explicitly limited runner group. Use `--labels cpu-private` or `--labels gpu-private` for routing. Let the interactive prompt collect the temporary registration token rather than placing it in shell history. 5. Start the listener with the documented command: ```sh ./run.sh ``` 6. Run a manual smoke workflow, then destroy the VM after its single job. Forward required logs before destroying it. A permanently running supervisor must never resurrect the used disk. ```yaml name: private-runner-smoke on: workflow_dispatch permissions: contents: read jobs: check: runs-on: [self-hosted, linux, x64, cpu-private] steps: - run: id - run: git version ``` A GPU job selects `gpu-private` instead and runs the GPU container wrapper from section 6. Do not add public pull request triggers to this example. Read GitHub's current billing terms before enabling a fleet; included hosted minutes and any platform charges are different from your own machine bill. ## 8. Read only public mirror options **Static content is the simplest public boundary.** Export approved documentation and release files from a reviewed commit, scan the output for secrets, and publish that artifact to a second server or static host. The public server receives no forge database, runner token, private repository or production credential. Use an atomic release directory switch so readers do not see half a publication. For public Git repositories, use [Forgejo push mirrors](https://forgejo.org/docs/latest/user/repo-mirror/) to a separate destination. Create an empty destination, give the primary a credential scoped to writing that repository, configure the push mirror, and verify that an anonymous reader cannot push. Keep the secondary's administrative access separate from its anonymous read interface. For an explicit command-line mirror, [Git's push documentation](https://git-scm.com/docs/git-push) describes mirror semantics. Run `git clone --mirror https://git.//.git public-repo.git`, then `git -C public-repo.git push --mirror `. Use only an empty dedicated destination: mirror pushes can overwrite and delete destination refs. Audit every branch, tag, hidden ref and reachable commit history for private data before publishing; an approved main-branch checkout does not establish that the rest is public. Prefer a static export when that history is not intended for publication. Validate the destination before scheduling. A Git mirror does not include issues, users, pull request discussions, database settings or all external LFS objects. Verify LFS separately. A public mirror is a distribution copy; deletions or a malicious push may propagate. It does not replace a versioned backup. ## 9. Backup, restore and prove recovery Follow [Forgejo's backup guide](https://forgejo.org/docs/latest/admin/setup/backup/), [PostgreSQL dumps](https://www.postgresql.org/docs/current/backup-dump.html), [restic repositories](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html), [backup](https://restic.readthedocs.io/en/stable/040_backup.html) and [restore](https://restic.readthedocs.io/en/stable/050_restore.html). Our policy is three copies: live data, an encrypted backup on a separate local device, and an encrypted off site backup. RAID is availability for a disk failure, not a historical copy. Keep the encryption password and provider recovery codes outside the primary server. Inventory repositories, database, `app.ini`, encryption keys, LFS, packages, attachments, Actions artifacts, release files, queues, external object storage, runner templates and infrastructure configuration. Store a protected copy of runner configuration and pin inventories in `/srv/forge/runner-inventory`. Re-register disposable runner credentials after recovery. Do not back up unnecessary job secrets. Create two restic repositories, one on your separate local disk and one in B2. Current restic documentation recommends B2's S3-compatible endpoint. Create a bucket-scoped key and enter it in a protected environment file through an editor: ```sh export AWS_ACCESS_KEY_ID='' export AWS_SECRET_ACCESS_KEY='' export RESTIC_REPOSITORY='s3:https:////forge' export RESTIC_PASSWORD_FILE='/etc/forge-backup/restic-password' ``` The password file contains the repository encryption password and is readable only by the backup account. Source this environment file in that account's restricted session. Initialize only a new repository: ```sh restic init restic -r /mnt/backup-disk/forge-restic init ``` Quiesce writers for a consistent snapshot. Pause new jobs, wait for current writes and take this outage in an agreed window. The following example assumes all Forgejo file storage is under `data`, uses the default configuration created above, and stages outside the deployment directory. Extend it if you changed any storage paths. Run it as a backup administrator with read access to all the selected files: ```sh set -eu umask 077 cd /srv/forge mkdir -p /srv/forge-snapshot /srv/forge/runner-inventory trap 'docker compose start server' EXIT docker compose stop server docker compose exec -T db pg_dump -U forgejo -d forgejo -Fc > /srv/forge-snapshot/database.dump tar -cf /srv/forge-snapshot/files.tar data compose.yaml .env cloudflared runner-inventory docker compose start server trap - EXIT restic -r /mnt/backup-disk/forge-restic backup /srv/forge-snapshot restic backup /srv/forge-snapshot restic snapshots ``` The PostgreSQL container stays up only to create its logical dump; Forgejo is stopped while files and database are captured. Prevent other applications from writing this database. Do not replace the dump with a live copy of the PostgreSQL data directory. Restrict the unencrypted staging folder and remove its contents according to your documented retention procedure after both backups succeed. A failed off site copy must alert independently even if the local copy passed. Configure retention deliberately. For example, the operator policy might be daily restore points for a month and monthly points for a year. Check the storage bill and legal retention requirements before choosing those numbers. Separate routine backup credentials from broader account administration and deletion permissions where the backend and retention design allow it. Test object lock or append-only behavior with restic before adopting it; an incompatible retention policy can break cleanup. ### Monthly restore exercise This monthly interval is our planning policy. Use a new isolated recovery VM with no outbound email, webhooks, runner dispatch or public DNS. Never restore onto the live data directory. 1. Recover the backup password from its independent location. Run `restic snapshots` and select the exact snapshot ID for this forge. 2. Run `restic check --read-data` against the off site repository. This reads the stored data and can incur transfer charges. A normal `restic check` alone does not read every stored data blob. 3. Restore with `restic restore --target /srv/restore-test`. Confirm the restored dump and file archive are present under the restored snapshot path. 4. Make an empty deployment directory on the recovery VM. Extract the restored `files.tar` into it. Keep the tunnel stopped, replace the public hostname with the isolated recovery hostname, and disable integrations before starting Forgejo. 5. From the recovery deployment directory, run `docker compose up -d --wait db` and stop if it fails. The healthcheck must pass before importing. Then import the dump using the command below. It assumes the restored dump has been placed at the named recovery path. Do not run it against the production database. ```sh docker compose exec -T db pg_restore -U forgejo -d forgejo -e < /srv/restore-test/database.dump docker compose up -d server ``` 6. Access the recovery instance through a private test route. Confirm administrator login and a normal account, a known repository commit, issue discussion, release, LFS file, package and permission boundary. Regenerate indexes and SSH authorization files if your configuration requires it, following Forgejo's restore instructions. 7. Run a harmless build on a newly registered isolated test runner. Compare outputs with the expected release and record elapsed recovery time, chosen snapshot time and any missing data. 8. Destroy the recovery instance after documenting the result. Alert on a missed test as well as a failed test. A successful `pg_restore` proves less than a successful clone and application login. ## 10. Updates, monitoring and security acceptance Use the [safe update kit](https://jwatte.com/downloads/safe-update-command-kit.md) for detect, back up, apply, verify, roll back and record. Its real host reboot recovery was not yet proved in the foundation article; rehearse your own recovery before relying on it. Our proposed cadence: review security notices daily, schedule ordinary changes weekly, test a full restore monthly, and respond to critical advisories according to exposure. These are operating assumptions, not vendor service promises. - [ ] Record installed and running image digests. Read the supported upgrade path before changing a database or forge major release. - [ ] Test upgrades in the isolated restored instance. A database migration may require restoring its previous data, not merely restarting an old image. - [ ] Drain runners before maintenance; block new jobs; retain the previous known working image and matching database backup. - [ ] Check forge HTTPS, login, clone, issue creation, a smoke build and artifact download after the change. - [ ] Alert on disk and inode pressure, database failures, queue age, failed backups, missed timers, rejected logins, new admin tokens and GPU spending. - [ ] Send an outside heartbeat. A monitor running only on the failed server cannot report its own silence. - [ ] Review Access events and forge audit records separately. Retain records according to the edition's actual capabilities; do not assume an open source forge has a commercial audit product's coverage. - [ ] Revoke departing staff in Access, the forge, provider accounts, tokens and active sessions. Remove their runner and mirror credentials too. Use this acceptance mapping with [OpenSSF's concise guide](https://best.openssf.org/Concise-Guide-for-Developing-More-Secure-Software), [Scorecard checks](https://github.com/ossf/scorecard#checks) and [SLSA build levels](https://slsa.dev/spec/v1.2/build-track-basics). The checks are controls to verify; a Scorecard badge or a signature alone is not proof of a secure pipeline. | Control | What to configure | Acceptance evidence | | --- | --- | --- | | Protected branches and review | Required reviews and checks, no routine direct pushes | Ordinary account cannot bypass a failing check | | Signed commits and releases | Verify identities, protect signing keys outside untrusted jobs | Known signed change verifies; wrong signer is rejected | | Token permissions | Repository scope, least privilege, expiry; read-only defaults | Job cannot write another repository or administer the forge | | Workflow supply chain | Pin actions and images to reviewed immutable references | Update changes go through review | | Dependencies | Renovate or equivalent, lockfiles, vulnerability review | Update proposal creates a reviewed change instead of direct production mutation | | Secrets | Secret store, no command echo, short lived job credentials | Test logs and artifacts contain no canary secret | | Untrusted code | Disposable isolated VM, no production secrets or privileged mounts | Negative access tests pass and used VM is destroyed | | Build provenance | Record source commit, builder and dependencies | Released artifact can be traced to its reviewed build | | SBOM | Generate a software bill of materials with the release | Component inventory accompanies the exact artifact digest | | Identity | Individual accounts; MFA through forge or SSO; recovery codes | Ordinary user and offboarded user tests pass | | Recovery | Coherent files and DB, encrypted off site history | Full isolated restore completes within your objective | Document which SLSA requirements are actually met. Generating provenance is only a starting point; stronger levels require protections for the build service and signing process. Do not claim a build level from this kit alone. ## 11. Cost worksheet These illustrations hold workload and storage constant between the low and high cases; only labor changes. Prices are US dollars before tax, observed November 1, 2026, 21:16 to 21:23 UTC. The $68.09 hourly labor proxy is the supplied October 29, 2026 estimate based on [BLS administrator wages](https://data.bls.gov/oesprofile/?areas=INDUSTRY,STATE,MSA&major_group=150000&measure=01&occupation=151244&year=2025) divided by the [wage share of employee compensation](https://data.bls.gov/timeseries/CMU2020000000000P). It is an employee cost proxy, not a managed service quote. Published inputs: [GitHub Team](https://github.com/pricing) $4 per person monthly; [standard Linux hosted runner](https://docs.github.com/en/billing/reference/actions-runner-pricing) $0.006 per minute and [Team included compute value](https://docs.github.com/en/billing/concepts/product-billing/github-actions) $18 monthly; [DigitalOcean](https://www.digitalocean.com/pricing/droplets) Basic 4 GiB $24 monthly and CPU-Optimized 4 GiB $42 monthly; [OVHcloud SYS-5](https://eco.us.ovhcloud.com/?range=kimsufi) advertised from $125 monthly plus $125 installation, final configuration and commitment unverified; [RunPod](https://www.runpod.io/pricing) RTX 4090 Secure Cloud on demand $0.89 per GPU hour and standard network storage $0.07 per GB monthly below 1 TB; [B2](https://www.backblaze.com/cloud-storage/pricing) $6.95 per decimal TB monthly after the first 10 GB. The current GitHub self hosted platform charge is zero. GitHub postponed its announced $0.002 per minute charge; see the [updated announcement](https://github.blog/changelog/2025-12-16-coming-soon-simpler-pricing-and-a-better-experience-for-github-actions/). Recheck before committing capacity. Keep artifact and package storage within the included allowance for these examples; LFS overages, extra transfers, taxes and model API charges are excluded. Independent backup exports and restore labor appear on both sides. [Cloudflare Zero Trust Free](https://www.cloudflare.com/plans/zero-trust-services/) contributes $0 for these examples within its 50-user limit, verified November 1, 2026. Paid identity features or a larger team change that input. Assume an existing administration computer and independent local backup disk common to both alternatives; new purchases belong in your own worksheet. Our workload assumptions: * Three people: $12 monthly Team seats, hosted CI within its allowance, 100 decimal GB retained in B2. Self hosting has a $24 forge VPS and a separate $42 CPU runner VPS. * Fifteen people: 200,000 standard Linux runner minutes monthly. Hosted seats and compute cost $60 + $1,200 minus $18 = $1,242. Hybrid costs $60 seats + $125 dedicated runner host. Full self hosting costs $24 forge VPS + $125 runner host. Keep 1,000 decimal GB in B2. Benchmark the workload on the dedicated hardware before claiming equivalent throughput. * Five people with nightly GPU work: 240 GPU hours monthly, the same 4090 Secure Cloud pool and runtime on both paths. Each has a $24 controller VPS, $213.60 GPU compute, $7 for 100 GB network storage, a $2 monthly container disk allowance and 500 decimal GB B2 history. The disk allowance assumes 20 GB at RunPod's published $0.10 per GB monthly running rate, reserving a full month even though actual running storage is prorated. See [Pod pricing](https://docs.runpod.io/pods/pricing). Hybrid retains $20 in GitHub seats; the full forge replaces those seats with a $24 forge VPS. An idle machine is not a discount: terminate compute and inspect the storage invoice. B2 monthly storage is $0.6255 for 100 GB, $3.4055 for 500 GB and $6.8805 for 1,000 GB. These are decimal GB, separate from GitHub's binary storage units. Transfers above B2's included allowance and any chargeable requests are extra. Formulas: ```text Setup = installation fee + setup hours × $68.09 Monthly = hosting + compute + seats + backups + admin hours × $68.09 36 month total = setup + 36 × monthly Savings = baseline total minus alternative total Break even = ceiling(setup difference ÷ monthly saving) ``` If the alternative has higher monthly cost and no setup advantage, break even is never. Do not round intermediate values. The following labor hours are our planning assumptions, not measurements: | Scenario | Setup hours, low / high | Admin hours per month, low / high | | --- | ---: | ---: | | Three people, GitHub | 2 / 4 | 0.5 / 1.5 | | Three people, own forge | 8 / 24 | 2 / 6 | | Heavy CI, GitHub hosted | 4 / 12 | 1 / 3 | | Heavy CI, hybrid | 16 / 40 | 4 / 10 | | Heavy CI, own forge | 24 / 60 | 6 / 16 | | GPU, hybrid | 8 / 24 | 2 / 6 | | GPU, own forge | 16 / 40 | 4 / 10 | | Scenario and labor case | Setup | Monthly including labor | 36 month total | | --- | ---: | ---: | ---: | | Three people, GitHub, low | $136.18 | $46.67 | $1,816.32 | | Three people, own forge, low | $544.72 | $202.81 | $7,845.72 | | Heavy CI, GitHub hosted, low | $272.36 | $1,316.97 | $47,683.30 | | Heavy CI, hybrid, low | $1,214.44 | $464.24 | $17,927.10 | | Heavy CI, own forge, low | $1,759.16 | $564.42 | $22,078.30 | | GPU, hybrid, low | $544.72 | $406.19 | $15,167.40 | | GPU, own forge, low | $1,089.44 | $546.37 | $20,758.60 | | Three people, GitHub, high | $272.36 | $114.76 | $4,403.74 | | Three people, own forge, high | $1,634.16 | $475.17 | $18,740.12 | | Heavy CI, GitHub hosted, high | $817.08 | $1,453.15 | $53,130.50 | | Heavy CI, hybrid, high | $2,848.60 | $872.78 | $34,268.70 | | Heavy CI, own forge, high | $4,210.40 | $1,245.32 | $49,041.94 | | GPU, hybrid, high | $1,634.16 | $678.55 | $26,061.80 | | GPU, own forge, high | $2,723.60 | $954.91 | $37,100.20 | At three people the own forge costs $6,029.40 more in the low labor case and $14,336.38 more in the high case. It never breaks even. For heavy CI, hybrid saves $29,756.20 low or $18,861.80 high, reaching break even in month 2 or 4. Full self hosting saves $25,605.00 low or $4,088.56 high against hosted GitHub, reaching month 2 or 17. Hybrid remains cheaper than moving the forge in both labor cases. For the GPU scenario, moving the forge costs $5,591.20 more low or $11,038.40 more high and never breaks even. Both pay the same GPU bill. An existing idle GPU, a different machine or a disconnected requirement changes the decision, but needs its own verified purchase or rental inputs. Migration, custom development, training, compliance assessments, on call coverage and major incidents are excluded from all rows. ## 12. Plain text architecture ```text ┌─────────────────┐ ┌────────────────────────────┐ │ Developer │ → │ Cloudflare Access + Tunnel │ └─────────────────┘ │ Email or SSO identity │ └──────────────┬─────────────┘ ▼ ┌────────────────────────────┐ │ Private forge + database │ │ Reviews, policy, job queue │ └──────────────┬─────────────┘ ▼ ┌────────────────────────┴─────────────────────┐ ▼ ▼ ┌────────────────────────┐ ┌────────────────────────┐ │ Disposable CPU runner │ │ Disposable GPU runner │ │ Private authenticated │ │ Scoped model budget │ │ connection to forge │ │ One job, then destroy │ └────────────┬───────────┘ └────────────┬───────────┘ └────────────────────────┬────────────────────┘ ▼ ┌────────────────────────────┐ │ Approved artifacts │ │ SBOM + provenance │ └──────────────┬─────────────┘ ▼ ┌────────────────────────────┐ │ Encrypted backup history │ │ Files + DB + configuration │ │ Local copy + off site copy │ └──────────────┬─────────────┘ ▼ ┌────────────────────────────┐ │ Isolated restore rehearsal │ │ Complete private recovery │ └────────────────────────────┘ Approved artifacts → reviewed publication → read only public mirror Forge + database + files + configuration → encrypted backup history Backup history → isolated recovery test; never publish backups ``` The lower flow separates two destinations: public artifacts go to the mirror, while complete encrypted backups stay private. The primary forge and its database also feed the backup process; runner output alone cannot recreate the forge. ## Sources and reading dates Official documentation read November 1, 2026. The observation windows below are UTC. Product releases and documentation can change; recheck the selected versions before installation. - Forgejo: [repository token scopes](https://forgejo.org/docs/latest/user/token-scope/), [server CLI and secret-file registration](https://forgejo.org/docs/latest/admin/command-line/), [configuration reference](https://forgejo.org/docs/latest/admin/config-cheat-sheet/), [Docker installation](https://forgejo.org/docs/latest/admin/installation/docker/), [runner installation](https://forgejo.org/docs/latest/admin/actions/installation/binary/), [registration](https://forgejo.org/docs/latest/admin/actions/registration/), [configuration](https://forgejo.org/docs/latest/admin/actions/configuration/), [runner security](https://forgejo.org/docs/latest/admin/actions/security/), [Docker access](https://forgejo.org/docs/latest/admin/actions/docker-access/), [backup and restore](https://forgejo.org/docs/latest/admin/setup/backup/), [mirrors](https://forgejo.org/docs/latest/user/repo-mirror/). Read 21:16 to 21:30 UTC. - GitHub: [add self hosted runners](https://docs.github.com/en/actions/how-tos/manage-runners/self-hosted-runners/add-runners), [ephemeral runners and lifecycle](https://docs.github.com/en/actions/reference/runners/self-hosted-runners). Read 21:16 UTC. - Gateway: [Caddy reverse proxy](https://caddyserver.com/docs/caddyfile/directives/reverse_proxy), [interface binding](https://caddyserver.com/docs/caddyfile/directives/bind), [configuration concepts](https://caddyserver.com/docs/caddyfile/concepts), [installation](https://caddyserver.com/docs/install), [official releases](https://github.com/caddyserver/caddy/releases); [Ubuntu UFW](https://documentation.ubuntu.com/server/how-to/security/firewalls/). Read 21:24 to 21:25 UTC. - Docker: [bridge networking](https://docs.docker.com/engine/network/drivers/bridge/), [port publishing and firewall behavior](https://docs.docker.com/engine/network/port-publishing/), [Compose reference](https://docs.docker.com/reference/cli/docker/compose/), [GPU reservations](https://docs.docker.com/compose/how-tos/gpu-support/). Read 21:16 to 21:25 UTC. - Cloudflare: [local tunnel setup](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/get-started/create-local-tunnel/), [configuration file](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/configure-tunnels/local-management/configuration-file/), [origin Access validation](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/configure-tunnels/origin-configuration/), [service tokens](https://developers.cloudflare.com/cloudflare-one/access-controls/service-credentials/service-tokens/). Read 21:16 to 21:17 UTC. - Git: [credential prompts](https://git-scm.com/docs/gitcredentials), [configuration](https://git-scm.com/docs/git-config), [mirror pushes](https://git-scm.com/docs/git-push). Read 21:16 to 21:29 UTC. - PostgreSQL: [logical backup and restore](https://www.postgresql.org/docs/current/backup-dump.html), [official container image and data layout](https://hub.docker.com/_/postgres). Read 21:17 to 21:21 UTC. - restic: [repository setup and B2](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html), [backup](https://restic.readthedocs.io/en/stable/040_backup.html), [checking repositories](https://restic.readthedocs.io/en/stable/045_working_with_repos.html), [restore](https://restic.readthedocs.io/en/stable/050_restore.html). Read 21:16 to 21:17 UTC. - Security framework: [OpenSSF concise guide](https://best.openssf.org/Concise-Guide-for-Developing-More-Secure-Software), [Scorecard](https://github.com/ossf/scorecard#checks), [SLSA build levels](https://slsa.dev/spec/v1.2/build-track-basics). Read 21:21 to 21:24 UTC. - Foundation: [server build](https://jwatte.com/blog/self-hosted-ai-server-from-scratch/), [safe updates](https://jwatte.com/blog/safe-update-command-self-hosted-ai-server/), [Cloudflare access](https://jwatte.com/blog/kimsufi-vps-cloudflare-tunnel-access/), [desktop access boundaries](https://jwatte.com/blog/kasm-workspaces-subdomain-cloudflare-access/), [backup and recovery](https://jwatte.com/blog/sovereign-windows-desktop-drive-backups/). Read November 1, 2026, 21:16 to 21:18 UTC. This is operational education, not a certification or a guarantee of recovery. Product names identify the software discussed; no affiliation is implied.