---
permalink: false
eleventyExcludeFromCollections: true
layout: false
---

# A practical model-routing and concurrency playbook

Written October 18, 2026. Copy and adapt this guide for your own server.

Source article: https://jwatte.com/blog/sovereign-ai-model-routing-local-codex-claude/

These are operating patterns. Four local requests or eight hosted jobs are not a capacity promise for another machine. Start smaller and measure your own work.

## 1. Assign three tiers

| Tier | Good starting assignments | Required boundary |
| --- | --- | --- |
| Local code and models | Triage, extraction, embeddings, memory lookup | Validate output; preserve original evidence |
| Production agent | Research, drafts, edits, builds, renders | One owned checkout and branch per job |
| Review model and owner | Verify facts and diffs, fix, merge, prepare release | Person approves outward actions and deployment |

Use ordinary code before a model for exact arithmetic, hashes, identifiers and schema validation. Keep uncertain local results in a review queue. A valid JSON object can still contain an invented fact.

Codex with supported ChatGPT sign-in can serve the production tier; Claude or another capable model can review. Keep separately billed API credentials and subscription sign-ins distinct. Do not make a local failure silently become a paid API call.

Make the routing decision explicit in each brief. An interactive hub can use a different order from a batch launcher. Document the configured order rather than assuming it matches the architecture you want.

## 2. Use a remote Linux desktop as the workbench

Keep the desktop behind authenticated private access. Use it to inspect files, make edits, read rendered output and open terminal sessions beside the repository and data. A desktop terminal may connect to the host; record where commands actually execute.

Run repeatable CSV, JSON, catalog, ledger and index transformations as scripts on the server. Work on a copy or branch, keep source hashes and compare counts, totals, identifiers and a representative sample afterward. For financial data, reconcile totals and preserve the original ledger. Let the desktop display the result instead of moving the whole dataset to a laptop.

Persistent terminal sessions can outlast a dropped viewing connection. Reboots require separate job checkpoints and tested recovery. Give terminal sessions independent ownership so stopping one agent does not terminate unrelated work.

Use signed-in browsers only for approved tasks and within platform terms. The owner handles sign-in. Select the intended operating system rather than relying on a browser number that may change. Permit one controller at a time per shared browser.

Do not run a heavy refactor, inference benchmark and interactive desktop session without checking their combined CPU and memory use. Do not put credentials or existing clipboard contents into result records. Back up source files, configuration and persistent desktop data, with a tested off-site copy. Centralizing files does not itself create a backup.

## 3. Copy this job brief

```text
Objective:
Audience and requested output:
Owned repository, branch and files:
Read these sources before writing:
Evidence rules and source read dates:
Allowed actions:
Actions requiring owner approval:
Dependencies and exact successful completion condition:
Resource and request-rate budgets:
Acceptance checks:
Required result record:
Stop conditions and recovery checkpoint:
```

Give each run a unique identity. A retry gets a new identity or an explicit attempt number. Queued means waiting, start means work began, and exit means the process ended. Record the exit status separately from acceptance success.

Wait for dependencies in an ordinary scheduler before starting the agent. Avoid a maintenance worker waiting for no agents while an agent waits for that maintenance worker. Check a background service's completed state when the agent merely launched it.

Do not accept any historical exit line as proof of a current dependency. Match the run identity, require successful status and verify its expected result. Serialize merges, pushes and production deployments per target. Independent jobs must own different files or branches.

## 4. Copy this RESULT template

```text
Status: complete, partial or blocked, with reason
Objective achieved and behavior changed:
Branch and commit:
Files changed and scope:
Sources: public URL or dated record type; read date
Measurements: method, start/end UTC, sample size, units, limits
Checks: exact command, exit status, relevant output
Known gaps and untested environments:
Review questions and proposed cross-links:
Owner approval needed for the prepared release:
Recovery or rollback:
```

Keep private evidence separately. A shareable result cites public sources and dated observations, with needed images embedded. Remove private machine addresses, credentials, private locations, preview links and references to private captures from every shared artifact. Do not bundle internal logs with a reader's download.

## 5. Give the reviewer a narrow starting point

Begin with the brief, RESULT and diff. Then open the original evidence wherever it affects a claim. Reading only the summary is insufficient when a factual correction, privacy issue or security boundary depends on omitted context.

- Verify the base revision and owned scope. Identify unrelated changes and conflicting branches.
- Check every number against a dated record or source. Separate estimates, observations and controlled comparisons.
- Recheck current prices, limits and changing product claims using official documentation.
- Check sources behind important claims, including qualifiers and failed measurements.
- Remove em dashes, en dashes, filler and hype from the author's prose. Preserve needed compound-word hyphens.
- Scan source text, links, image references and metadata for secrets and private machine details.
- Read failed gates. Rerun checks appropriate to the actual change. Check generated examples against their source.
- Separate tested behavior from proposed integrations and untested operating systems.
- Confirm downloads match the reviewed content and the hero exists before publication.
- Prepare one integrated release and obtain the person's deployment approval.

Narrow review can reduce generation work. Do not claim a percentage saving without measuring comparable workflows, model usage, retries and correction time.

## 6. Size Ollama conservatively

| Control | Starting choice | Trade-off |
| --- | --- | --- |
| OLLAMA_NUM_PARALLEL | Start with 1 | More slots can improve aggregate throughput but consume context memory and increase latency |
| OLLAMA_MAX_LOADED_MODELS | Start with 1 | More resident models reduce switching waits if memory permits |
| Request context | Smallest sufficient tested budget | Truncation can remove corrections or exceptions |
| Request keep_alive | Explicit bounded duration | Warm models answer sooner but occupy memory between requests |

Our recorded installation uses four parallel slots and three loaded models. Those are ceilings, not reservations or a guarantee that any three models fit. Share the local request budget across callers. A four-worker indexer must not compete with another unrestricted four-worker queue by accident.

Pin runtime images and embedding identity. Change one setting at a time through the persistent service definition, with a health check and rollback. Test a short generation, embeddings and the intended combined workload before adopting higher limits. Do not copy administrative commands blindly.

Quantized weights are smaller than higher-precision copies, but weights alone do not predict runner RAM. Include context, concurrent slots, runtime overhead and all other services. Keep large optional models on demand. Do not keep every installed model warm.

## 7. Build memory that can be checked

Separate operating memory, job handoffs and audit material when their ownership and update rules differ. A small installation can also use one collection with clear payload categories. Choose from the data and access requirements.

Use a pinned local embedding model and record its identity, vector dimensions and retrieval prefixes. A model change may require a new collection and reindexing. Never mix incompatible embeddings because their names look similar.

Our implementation uses 1,600-character chunks with 200-character overlap, four shared embedding workers and references back to source files. Treat these as an example to test. Preserve headings, dates and exceptions, and verify retrieval with known questions.

Hash source content. Save a manifest after each processed file. Replace a changed file's old points and remove deleted files' points. Prevent overlapping refreshes. Run an incremental nightly refresh, retry failed work and catch up after missed runs. An unchanged pass should do little embedding work.

Keep private credentials outside arguments and logs. Do not print vectors. Search results are candidate evidence, so open the original records. Keep exact keyword search available for identifiers and rules that semantic search may miss.

## 8. Calculate actual overlap from your log

Save the following as `concurrency.py`, then run `python3 concurrency.py progress.txt` on a frozen copy of your log. Python's standard library is sufficient.

Accepted lines look like these generic examples:

```text
example-a queued 2026-10-18T09:00:00Z
example-a start 2026-10-18T09:01:00Z codex=version
example-a exit 0 2026-10-18T09:03:00Z
```

The calculator counts paired intervals, including failed exits. It ignores queue time, unpaired exits and starts without exits. A repeated start supersedes an unfinished attempt unless explicitly marked as resumed after reboot. Resume markers preserve the original start and launcher. A stop notice is diagnostic; only an exit closes an interval. Exit events sort before starts at equal timestamps, so adjacent runs do not overlap. Durations include waiting and are not labor time.

```python
import collections, datetime as dt, json, re, statistics, sys
from pathlib import Path
pending, runs = {}, []
counts = collections.Counter()
for line in Path(sys.argv[1]).read_text().splitlines():
    m = re.match(r'(\S+) (start|exit|stopped)\b(.*)', line)
    stamp = re.search(r'\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ', line)
    if not m or not stamp:
        continue
    job, event, rest = m.groups()
    time = dt.datetime.fromisoformat(stamp[0].replace('Z', '+00:00'))
    if event == 'stopped':
        counts['stop_markers'] += 1
    elif event == 'start':
        if job in pending:
            if 'resumed after reboot' in rest:
                counts['resume_markers'] += 1
                continue
            counts['superseded_starts'] += 1
        lane = 'Codex' if 'codex=' in rest else 'Claude' if 'model=' in rest else 'unlabelled'
        pending[job] = (time, lane)
    elif job in pending:
        start, lane = pending.pop(job)
        if time >= start:
            runs.append((start, time, lane, job))
    else:
        counts['unpaired_exits'] += 1
if not runs:
    raise SystemExit('No completed intervals')
events = sorted([(a, 1) for a,b,l,j in runs] + [(b, -1) for a,b,l,j in runs])
active = peak = 0
peak_times = []
for time, delta in events:
    active += delta
    if active > peak:
        peak, peak_times = active, [time.isoformat()]
    elif active == peak and delta == 1:
        peak_times.append(time.isoformat())
durations = [(b-a).total_seconds() for a,b,l,j in runs]
print(json.dumps({
    'completed_intervals': len(runs),
    'by_launcher': dict(collections.Counter(l for a,b,l,j in runs)),
    'by_exit_day_UTC': dict(sorted(collections.Counter(b.date().isoformat() for a,b,l,j in runs).items())),
    'median_seconds': statistics.median(durations), 'longest_seconds': max(durations),
    'peak_simultaneous': peak, 'peak_reached_UTC': peak_times,
    'first_start_UTC': min(a for a,b,l,j in runs).isoformat(),
    'last_exit_UTC': max(b for a,b,l,j in runs).isoformat(),
    'unmatched_starts_excluded': len(pending), 'diagnostics': dict(counts)
}, indent=2))
```

For production scheduling, prefer structured events with unique run and attempt identifiers. This small calculator is for the text format shown, not every scheduler's logs. Missing or inaccurate events can undercount or overstate actual overlap. Do not treat the measured peak as a safe future limit.

Tested October 18, 2026, on our 549-line progress snapshot: 169 paired intervals, median 774 seconds, longest 8,752 seconds and peak eight. Launcher labels identified 83 Codex and 47 Claude intervals; 39 were unlabelled. These observations describe our log, not your server's capacity.

## 9. Admit work below saturation

Before starting independent work, inspect logical CPU count, load averages, available memory and existing builds. On our 48-logical-CPU shared server, routine admission uses load below about 24 and more than 20 GiB available. Scale to your machine and leave headroom for the desktop, database and backups.

For optional local probes on our server, require load below 8, at least 25 GiB available and no build or deploy running. Wait at most 15 minutes for a build, then skip the probe. Recheck throughout and abort the whole probe if a condition fails.

Reuse equivalent recorded measurements before running another probe. If needed, test only a small already loaded model, concurrency 1, 2 and 4 within the configured limit, six short requests per level and at most 64 predicted tokens. Never load a large optional model just to fill a table. Keep all probe work under four minutes. Record start/end UTC and load before/after. Distinguish aggregate batch throughput from API-reported per-request generation rate.

For a retrieval-only gap, a bounded alternative is 20 sequential memory searches followed by 20 at concurrency four, reporting median and 95th percentile. Do not reindex as part of a latency probe. Apply the same admission and time limits.

Pace website reads across all jobs per origin. Try direct and check the alternate route for every planned page, then use passive browser rendering if those attempts fail. Do not interact with challenges, login or paywall controls. Preserve the original access finding and report planned, measured, route used and unmeasured pages. Share any intermediary service's rate budget.

Hosted inference has separate plan limits. More server processes cannot expand an account's allowance. Check the current usage dashboard and keep enough capacity for review and recovery.

## Sources and scope

Official references read October 18, 2026:

- Ollama configuration, context and residency: https://docs.ollama.com/faq
- Qdrant vector configuration and collections: https://qdrant.tech/documentation/concepts/collections/
- Codex scripting and permissions: https://learn.chatgpt.com/docs/non-interactive-mode
- Codex usage and separate API billing: https://developers.openai.com/codex/pricing
- Claude usage and length limits: https://support.claude.com/en/articles/11647753-how-do-usage-and-length-limits-work

Chunking, scheduling, guardrails and log results are our dated implementation choices and observations. They are not vendor guarantees. Use the source article for the full measurement window and qualifications.

This guide is informational, not technical consulting advice. Mentions of Ubuntu, Google Chrome, Ollama, Qdrant, nomic-embed-text, LiteLLM, OpenAI, ChatGPT, Codex, Anthropic, Claude and Python are nominative fair use. No affiliation is implied.