New to this? Start with Sovereign AI Made Simple.
The desktop connected. Then it disconnected. Then it did it again.
We had built a phone friendly interface, tested it in browsers, and promoted the container. The next user report described a repeating connection loop. That report was the reason to investigate, not a physical device certification. That was the point where a convincing deployment report met a less cooperative reality.
The server did not need a more ambitious name or another model. We needed to reproduce the failure, preserve the working data, and stop calling an open connection a usable desktop.
This is a field report from our October 4 through October 7, 2026 UTC work on the Sovereign AI Playbook. It covers what the release records actually show, what remained staged at the review checkpoint, and the choices I would carry into the next deployment.
The earlier architecture article explains the hardware and economic tradeoffs. Here, the subject is what happens after the diagram.
Use the lessons on your own server: Download the deployment workbook, the redacted evidence summary, or this article as Markdown.
A connection is not the same thing as a working screen
Our failing mobile test opened its stream successfully. The unrepaired mobile WebKit fixture opened four stream connections in about three minutes and was marked unstable. The repair notes traced the failure to the initial video negotiation rather than treating a fresh login as the cure.
The revealing detail was in the first negotiation. A mobile WebKit test with device scale three requested a 3840 by 2400 desktop at 288 DPI. Our outer interface subsequently imposed 1280 by 800 at 96 DPI. Setting sensible dimensions after an oversized stream had already begun was too late for this failing path.
The unrepaired test opened four stream connections in three minutes. The final recorded repair test held one connection for three minutes; a separate Chromium engine fixture held one for six minutes. These are bounded regression results, not an uptime guarantee. No physical iPhone or iPad acceptance had been completed by the checkpoint.
The automated stream tests used disposable desktops, not the live owner profile. The evidence also records a synthetic decoder failure test that preserved the unsent draft while recovering. Connection evidence.
The repair prepared the remote dimensions before the renderer started, offered JPEG compatibility for Safari, retained H.264 as a choice, and made the interface wait for rendered output before reporting Connected. JPEG was a compatibility decision, not a claim that it wins every bandwidth or motion comparison.
The general lesson is to measure the experience you promise. Record connection attempts, actual rendered output, close reasons and retry state. Then test long enough to catch the loop. A screenshot and a successful socket handshake cannot do that job.
Selkies already separates display resolution, scaling and stream controls. Those distinctions should remain visible in a custom interface rather than being folded into one mysterious Auto button. Selkies web client documentation.
Mobile work needs its own controls
The most useful phone improvements were ordinary interface decisions: reachable buttons, a local compose field, visible clipboard actions, and a way to select the remote window that matters.
A phone should not need to navigate a remote browser just to ask a project question. The useful pattern is to link directly to the knowledge page, terminal, account approvals and maintenance. The desktop remains available when the task needs a desktop.
We also separated local viewing from remote layout. Zooming a phone's view should not keep rearranging application windows on the server. Opening the keyboard should not become a series of remote resolution changes.
For terminal input, local composition is especially helpful. Write the command or prompt in a normal text field, review it, and explicitly send it. Our multiline paste path requires confirmation and does not append another Enter. Esc, Tab, arrows and temporary modifier keys make more sense than expecting somebody to reproduce every desktop shortcut on glass.
Recorded interface and input checks.
Clipboard access still belongs to the browser's security model. Safari requires appropriate user interaction and can show its own paste confirmation. A clear Copy or Paste button is more dependable than promising invisible clipboard synchronization. WebKit clipboard guidance.
A particularly frustrating case was an account approval link displayed inside the remote terminal. What looked like text on the phone was part of the desktop image. We built that page and tested its owner session, selectable fields, expiry and provider checks with synthetic accounts and codes. Those fixtures did not perform a fresh provider login. The lesson is not to weaken login.
It is to make the legitimate login usable.
Keep the workshop separate from its doorway
The phone is a doorway. Webtop is another doorway. Neither should be the only place a job or project exists.
The design gives host and desktop tools access to a shared working directory rather than requiring a new copy for every interface. The browser profile has its own persistent volume. Coding clients, model services and databases have separate roles and state.
Scroll to compare columns and rows when the table extends beyond the box.
| Component | The job it should do |
|---|---|
| Webtop and Selkies | Present the desktop and carry deliberate user input |
| Native coding CLIs | Inspect, edit and test the selected project through an authorized account |
| Ollama and Hermes | Serve the explicitly chosen local model workload |
| LiteLLM | Provide the application's configured model entry point |
| Open WebUI | Supply the chat interface, with its own account and storage boundaries |
| Qdrant | Hold an explicitly constructed semantic index, not automatically discover files |
| n8n | Coordinate defined workflows and record their execution |
An eight second synthetic job recorded state Completed and exit code zero. That record does not establish that a client detached during execution. It was a checkpoint demonstration, not proof of resilience after a disconnect or reboot. Job receipt summary.
A durable operation needs an identifier, a target project, an exit state and a saved result. On reconnection, read that record before starting again. Otherwise a helpful Retry button can submit the same work twice.
Our next deployment would establish that contract before adding a large collection of tools. It is cheaper to decide what Completed means while there are three steps than after there are thirty.
Installing Hermes did not give it our memory
A model file beside a project folder has no automatic understanding of the folder.
We transferred 409 exported Markdown memory notes, preserved their source paths, checked them against a manifest, and created screened retrieval copies. Eight notes needed credential related redactions. The source originals were preserved. That count describes the complete approved Markdown export, not every file or conversation on the old machine.
The verified project route used SQLite FTS5 to retrieve relevant text before asking Hermes to answer. It attached source references and checked exact quotations. The plain general model remained a separate route.
One failure was more instructive than a speed measurement: an early current release answer failed its expected version check. The implemented retrieval rule now replaces an older indexed copy with the live operational record for that specific current version question. A later test passed. Quoting a source and selecting current evidence are different responsibilities.
I would keep that lesson even with a much larger model. Current state, historical notes and proposed plans are different evidence classes. A system that mixes them indiscriminately can return an elegant answer about yesterday.
The local route used an 8192 token context and bounded output. That was an operating choice, not a claim that the entire library fits into one prompt. Ollama documents the memory consequences of longer contexts and more parallel requests. Ollama FAQ.
Qdrant and n8n are integrations, not decorations
It is easy to add two containers and say a product now has memory and automation. Our own evidence argues for more careful language.
At the editorial checkpoint, Qdrant had persistent storage and authenticated access, and an initial semantic indexing job was still building. The established Hermes answer route was the tested lexical route. A database health response was not evidence that semantic retrieval had passed relevance, freshness and failure tests.
The semantic design used local embeddings of already screened text, source coordinates, content hashes and product metadata. Results should be resolved against the current screened source, with stale mismatches rejected. A vector payload should not become a second, unverified source of truth. Its coverage needs a denominator, especially when only selected excerpts from large files are embedded.
n8n needed an equally explicit handoff: a fixed internal endpoint, an appropriate credential, a stable request identifier and a workflow owned by the intended account. Our setup work also encountered a local CLI task broker port collision. A command being installed did not mean it could run beside the active service without configuration.
Preserve the workflow store and the encryption key together as an operational responsibility, while protecting the key separately from public exports. n8n uses that key to encrypt stored credentials; replacing it casually can make a surviving database unusable. n8n encryption key documentation.
For the next project, I would begin with a harmless health workflow, then a controlled index refresh. I would not begin by granting an automation unrestricted shell access, arbitrary destination URLs and an unbounded model budget.
A new image should not require a new identity
The reusable image should contain software and reviewed configuration, not a photograph of a live account session.
Keep browser profiles, project files, model files, database storage and job records outside the replaceable image layer. Keep secrets out of Git and build arguments. Give each service only the runtime credential it needs, using supported secret files or protected configuration rather than assuming every application understands the same convention.
Our desktop repair promoted a specific image while preserving the profile volume and existing application environment. It did not recreate the model and database services just to change the desktop. That narrower change made the result easier to verify. Deployment evidence.
There is a subtle trap here. The host CLI, a CLI inside Webtop, an account in Chrome and a provider API can each have a different authorization state. Updating Netlify or Wrangler does not authorize the correct account. A working desktop password does not authorize GitHub.
Test from the exact execution environment you plan to use. Ask the CLI which account it sees, confirm access to the intended project, and perform a harmless read before a preview or production change. Keep private account details out of the published test report.
A gateway does not turn subscriptions into unlimited APIs
We retained the original separation between interactive subscription work and application model requests.
A native coding client can use a supported subscription login. The dashboard uses the model routes actually configured in LiteLLM. These are not interchangeable credentials, and a gateway does not manufacture missing provider access.
Our source aware local Hermes path was configured without a paid provider fallback. That is not an exhaustive test of every possible provider failure. LiteLLM supports configured fallback routes, but changing providers can change the cost, destination of the data and behavior of tools. That deserves explicit policy and tests.
LiteLLM fallback documentation.
The same care applies to remote prompting. Claude Code Remote Control lets a supported browser or mobile client continue work whose tools and files remain on the selected machine. A ChatGPT app needs an authorized connector that actually supports the requested action. Neither connection moves the provider's model onto your server or removes plan limits.
Claude Code Remote Control and OpenAI connected app guidance.
A useful remote prompt names the machine, project, permitted changes and completion evidence:
Use the enrolled server, not this device. Confirm its hostname and the project directory first. Read the current task record. Run the approved checks, save the result and show the changed files. Do not restart unrelated services or repeat a completed deployment. Ask for approval before any new production change.
The wording is not a security boundary by itself. The connector, account permissions and server controls must enforce the allowed actions. It does, however, make a wrong machine or missing result easier to notice.
Latest is a candidate, not a verdict
The update request was reasonable: bring the CLIs and desktop components current without losing the environment. The unsafe interpretation would be to replace everything with whatever a floating tag happened to resolve to that morning.
Our staged update work resolved an explicit package list to exact versions and a dependency lock. Its earlier dependency scan reported one critical finding and 48 high findings; a later scan reported zero critical and 37 high findings. Those are the two recorded scan results, not a claim that every dependency problem was eliminated.
That audit concerns the candidate CLI dependency tree, not a count of independent exploitable flaws in the whole server. It also does not justify marking the system universally secure. Scoped audit summary.
I would use a weekly schedule to discover and test changes. Promote only the immutable candidate that passed the checks, not a fresh pull after testing. Save the previous image reference and configuration, then replace the affected service and check it again.
For GitHub Actions, keep the trusted deployment mechanism separate from arbitrary repository scripts and untrusted pull requests. Give the workflow only the permissions it needs. A powerful persistent runner deserves particular care because changes to a workflow can become changes to the machine. GitHub secure use guidance.
The weekly updater was still under implementation at our reporting checkpoint. The workbook presents its acceptance contract, not a claim that an unattended pipeline had completed a successful update and rollback cycle.
Recovery deserves its own test
A candidate can pass syntax checks and still fail in use. Preserve the failing fixture, change the smallest relevant component, and run both the repaired scenario and the previously working paths.
For this desktop, that meant keeping the original Selkies entry and earlier controls available, testing decoder recovery, checking actual input and transfer bytes, and verifying that a second tab could not silently take control. It also meant leaving real phone acceptance separate from browser engine tests.
The backup policy had to be equally explicit. Our requested limit was 40 days for registered local backup files. That is a local operating choice, not a universal recommendation. A cleanup job must distinguish those backups from active profiles, original migration sources and the working repositories. It must not follow a convenient filename into somebody's live data.
A local archive can help undo a bad update on a working server. It does not prove recovery after losing that server. Database restoration, credential decryption and an independent recovery copy still need their own acceptance. The old environment was not cleared for retirement simply because the replacement desktop loaded.
What I would do first next time
Start with one protected desktop and one project. Confirm that the native account can read that project, and that a detached test job leaves an unambiguous result. Then replace the desktop container once and prove the project and profile survive.
Add the mobile controls next. Test the keyboard, clipboard, window focus and file delivery before adding another model. Include Retina scaling, a decoder failure, a second controller, an expired login and a simulated connection loss. Follow the browser tests with an actual phone.
Introduce project retrieval only after source identity and freshness are clear. Add semantic search and workflows as separate, bounded integrations. The two should improve the established path rather than turn one working service into a dependency on everything else.
Finally, automate the proven release procedure. The deployment workbook turns that order into a reusable checklist, a state inventory, test cases and prompts for a coding assistant. It deliberately is not a blind installer for an already occupied server.
The point of a capable server is not to make failure impossible. It is to give work a durable home, keep evidence when a client fails, and make the next action clear. A mobile AI workspace starts earning its keep when reconnecting feels routine rather than like starting over.
Update without interrupting the person using the screen
Treat the working desktop as occupied equipment. Pin its current release, back up and test the next image in a TEST copy. Promote one component at a time only when the live desktop is free, check the actual application and retain compatible rollback state. Bound retries so repeated failing builds cannot fill storage. Clean up after completion and preserve everything the rollback pins reference.
OS package upgrades belong in an execution context that will not be killed mid-install. After an interrupted upgrade, verify package-database consistency and the boot image before rebooting. A launcher handing work to a background service must wait for that service's terminal result and inspect the output; losing the phone connection or seeing the launcher exit says nothing about completion.
Pair semantic memory with keyword search for exact instructions and identifiers. Keep secrets in the process environment and scan commits. A monitor should alert only on changed findings, failed checks or lost coverage; first distinguish its own timeout or path failure from an application outage. Preserve completed checkpoints when the monitored inventory grows.
The operator guide covers the broader job, evidence and maintenance contract.
A phone message can request work without opening a shell
A small business owner asking for site status should reach a fixed operation through an authenticated private channel. Email also needs trusted DMARC and DKIM results. Self-hosted n8n can carry message data to fixed validated dispatcher commands over forced-command SSH; the message cannot rewrite the agent prompt.
Keep sandbox or fixed-tool limits, rate limits, a kill switch and append-only history. One-time confirmation binds outward actions to exact content and destination; replies pass the shareable gate. Channels remain inactive until owner credentials are configured.
The desktop needs its own update proof. Copy the live profile with ownership and UIDs preserved, validate the generated health check and promote only the proved digest after graceful browser closure and a fresh stopped-profile snapshot. Check after start and after a second restart; cap attempts and automatically restore image, configuration and profile on failure.
A clean test profile cannot establish that a populated mobile desktop will survive the same update.
The full chat, email and desktop update pattern defines the sequence and measured limits. The operator kit supplies its checklist.
Make the phone a real remote control, not a tiny monitor
Later sessions on the phone desktop, October 12 and 13, found problems that no desktop browser test had shown. Each one has a general fix you can apply to your own remote desktop.
Give the desktop container a fixed hostname. Chromium based browsers write the machine's hostname into their profile lock. A recreated container gets a new hostname unless you set one, so the next start finds the old lock, concludes that another computer is using the profile and stops at a "profile in use" prompt until someone clears it.
Declare the hostname in the service definition so every recreation looks like the same machine. Docker Compose supports this with the service's hostname setting.
Let the client choose the resolution. A fixed width and height set on the server overrides what the client asks for. A laptop hardly notices. A phone receives a desktop drawn for a monitor and shrunk until nothing is readable or tappable.
Selkies follows the browser window by default; keep that default, and on phones offer a phone shaped desktop with larger text instead of a fixed monitor size.
Mount overrides over the source, not the destination. Some desktop images copy their web pages into the web root when the container starts. A read only mount over the destination made that copy fail, and the pages disappeared. Mount your replacement over the file the startup script copies from, and the normal startup puts your version in place.
Let the stream claim touch gestures. Zoomed in on a phone, a one finger drag scrolled the outer page instead of moving the remote cursor. Setting touch-action: none on the stream element and on the elements inside it tells the browser not to handle panning and zooming there, so the page can turn the drag into pointer movement.
Keep the cursor in view by panning the local view toward the pointer position the server reports. Pinch to zoom, plus a control that zooms to the active window, makes small dialogs usable without changing the remote resolution.
Open the keyboard from a real input and send key events. Phone keyboards opened reliably only when focus landed in a real input field in the top level page. Forward each keystroke from that field as a key event over the remote desktop's own input channel, and let the server add Shift where a character needs it. Do not type through the clipboard.
A pasted password stays in a clipboard that other software can read, while forwarded key events leave nothing behind. The Copy and Paste buttons remain useful for ordinary text.
Prove input end to end before guessing. Two tools found the real cause quickly. A scripted phone emulation harness exercised the page itself. On the server, telemetry counted message types per connection, never their values, so it could show whether taps and keys arrived without recording what was typed.
The counts showed two browsers built on the same phone engine behaving differently: one sent hundreds of input events, the other sent none. That points at layers specific to one browser, such as extensions, content blockers or privacy settings, rather than at the engine or the server.
Test builds of a browser engine may not synthesize touch drags at all, so a drag that fails in automation needs a check on a real phone before it counts as a defect.
One tab controls the desktop. Only one tab can hold the control lease at a time. Two open tabs fought over it and silently dropped each other's input. Close the extra tab, and make any takeover explicit, which is why the second tab test in the recovery section matters.
Confirm focus before typing a secret. Remote typing goes wherever focus is. Before typing a password, look at the screen and confirm which field has focus. A password typed into a browser address bar becomes a web search and a history entry. If that happens, rotate the secret first, then remove the history entries. Clearing history does not make the exposed secret safe again.
These practices come from one deployment's sessions, not a compatibility certification for any phone or browser. The deployment workbook turns them into acceptance tests.
The best practices guide covers how the parallel agent jobs behind this work were run, and the operator guide covers the image cleanup that followed.
Keep the phone access route private
Reach the browser desktop through a tested SSH tunnel or a restricted VPN, with the server UI bound to loopback or a private interface. Test from the phone before removing an old access route, especially if mobile addresses change.
The host-hardening guide explains the binding and Docker firewall checks; the mobile workbook adds the acceptance test.
Send a prompt from a phone link, get the answer as a push
Two later additions changed how we use the phone. A private prompt hub accepts a request in its address: a link such as https://hub.example.com/?q=...&mode=ask opens a plan that names the engine, the target machine and the notes it will attach. Nothing runs until you tap Run. A GET request never runs anything, so a link preview in a messaging app, a browser prefetch or a stale bookmark cannot start a job. Bookmark the requests you repeat, such as a morning check of the booking page, and each one becomes two taps.
The answer comes back as a push. ntfy delivers a notification from anything that can call curl, so the hub sends a short "job finished" message with a link to the result page behind your own sign-in. The push never carries the prompt text or a customer's details: the public server caches messages for 12 hours by default, and its privacy policy describes no end-to-end encryption. On Android, ntfy can deliver instantly through a foreground service. On an iPhone with a self-hosted ntfy server, set upstream-base-url, or delivery can take hours.
If a job needs you halfway through, the hub can open it as a Claude Code Remote Control session, and you continue in the Claude app. Answering there, with the phone's own keyboard, avoids the remote desktop typing problems described above.
Build your own AI prompt hub covers the hub, and the ntfy alerts cookbook has the alert recipes.
Fact-check notes and sources
Phone prompt and alert update, October 15, 2026: the hub's plan-before-run behavior and its Remote Control button describe our own build. Sources read October 15, 2026 (UTC): ntfy phone apps, which describes instant delivery "achieved with a foreground service"; ntfy privacy for the 12-hour cache; ntfy configuration for upstream-base-url; and Claude Code Remote Control.
Dedicated-server security update, October 13, 2026: the new access and maintenance guidance links to the dated Ubuntu hardening sources, checked October 13, 2026, and is a recommendation, not a claim about an existing server.
The new phone-access test covers private desktop bindings and SSH or VPN reachability.
Phone input update, October 13, 2026: the new section reflects sessions between October 12 at 07:00 and October 13 at 06:00 UTC. Sources checked October 13, 2026 from 05:57:28 to 05:57:52 UTC, each read directly and through the alternate route with readable content, none unmeasured.
MDN touch-action defines none as "Disable browser handling of all panning and zooming gestures."
The Selkies web client guide says "by default the remote resolution follows your browser window exactly." Docker Compose services says hostname "declares a custom host name to use for the service container." The profile lock, keyboard and input count observations are our own and were not repeated on every phone or browser.
Workflow sources checked October 12, 2026 from 06:40:54 to 06:41:03 UTC: Email Trigger (IMAP), Telegram Trigger, Code, SSH, Send Email, Telegram and OpenSSH authorized_keys options.
Seven pages planned, seven measured with readable direct content and alternate-route checks, zero unmeasured. OpenSSH says the supplied command "is ignored" when the key has a forced command. The n8n nodes supply transport and workflow steps; sender authentication, argument validation, approval, sandboxing and output gates are application requirements, not guarantees supplied by those nodes.
Operational guidance updated October 12, 2026. The new practices are recommendations, not new measurements or claims that every integration is deployed. Existing dated tests retain their original conditions. The workstation operator guide supplies the detailed retrieval, scheduling, update and evidence policy.
The redacted evidence summary identifies the review window, failure reproduction, bounded connection tests, exported note coverage and staged update audit. It excludes account identifiers, private host addresses, session links, tokens and raw screen captures. Historical snapshots retain their dates.
No universal reliability rate, whole system security certification or completed independent disaster recovery test is claimed.
Vendor documentation supplies the general interfaces and constraints, rather than evidence that our deployment passed them: Webtop setup and exposure warnings, Selkies controls, WebKit clipboard behavior, Ollama memory and concurrency, Docker secrets, n8n encryption, and GitHub workflow security.
Continue with the Sovereign AI Playbook, small quantized models on a 96 GB server, and choosing desktop and browser agents.
Reporting reviewed October 7, 2026 UTC, using the preceding deployment records. The cover is a conceptual illustration, not a screenshot or proof of product capabilities. Product names identify the tools discussed and do not imply vendor endorsement.