Our $125 CPU AI Server Versus GPU Servers: What the Extra Money Buys

Our OVH server costs $125 a month before tax. It carries our agents, databases, browser desktops and local models. It has no GPU.

Most of the heavy thinking does not happen on it. Our agents run here with their tools and files, but when they call Claude or ChatGPT, the model itself runs on the provider's hardware.

We would keep this CPU server. We would add a GPU when a measured local workload needs one. Here is what that decision costs, and what it can actually change.

What our server carries today

Our server is OVHcloud's So you Start SYS-5 from the US Eco catalog. The current listing names dual Xeon Silver 4214R processors; our running hardware reports two Intel Xeon Silver 4214 processors. We use the hardware's own report below. This is not a Rise server.

At about 01:40 UTC on October 18, the operating system reported 94 GB total memory, 58 GB used and 35 GB available. The 96 GB installed shows up as 94 GB in that report.

The same snapshot counted 993 processes and 14 running containers. The workload included databases, Ollama, LiteLLM, Open WebUI, Qdrant, n8n, integrations, Guacamole, a Windows 11 VM and two browser desktops. Coding agents and scheduled jobs share that machine.

Load averages were 2.72, 2.93 and 2.52 on 48 logical CPUs, after 13 days of uptime. Load average counts tasks that are running or waiting, so it is not a utilization percentage. Still, a load under 3 on 48 logical CPUs leaves most of the machine idle.

For a separate CPU observation, our workstation operator article records five unattended coding agent jobs running together using about 2 percent of the server. The earlier article calls it a 48-core server; strictly, it has 24 cores and 48 logical CPUs. Remote model response time and polite website request pacing limited that window.

Our job log adds one more number. Between October 11 and 18 it recorded 167 Claude Code and Codex jobs, and at the busiest moment, 02:43 UTC on October 12, eight of them were running at once.

So how many processes can it handle? We have seen 993 processes, 14 containers and eight agent jobs at once on this box. Those are observations, not a tested maximum. We have not established how many simultaneous heavy builds or local generations it can sustain.

Where the CPU stops feeling quick

Our small quantized model tests measured Hermes 3 8B Q4_0 through Ollama 0.35.1, with 24 requested CPU threads and a 4,096-token context. Nine short synthetic extraction calls produced a median total request time of 3.011 seconds and a median generation rate of 13.738 tokens per second. The first call took 12.462 seconds, including 7.489 seconds loading the model.

That is useful for short records. It does not establish fast long-form writing. At that measured generation rate, 1,000 output tokens alone would take about 73 seconds: 1,000 divided by 13.738. That is illustrative arithmetic, not a measured long-answer test. Prompt processing and loading would add time, and the rate could change.

A separate October 13 test of Qwen 2.5 1.5B Q4_K_M reached 103.98 aggregate output tokens per second with four simultaneous requests and 24 requested threads. Each output was capped at 64 tokens. Four-request batches took 2.37 to 2.56 seconds. Aggregate throughput is the combined work; it is not each user's answer speed.

Our rule: expensive smart models do writing and judgment; cheap local models do triage and extraction, with checks.

Dedicated GPU servers beside our $125 bill

The official US catalog confirms $125.00 a month excluding tax, plus $125.00 installation. The following are server rentals, not prices for buying a card. Setup is paid separately from the recurring difference.

InterServer's offer could not be read through our tested access routes today. Its figures come from the September 30, 2026 archive, captured at 19:29:55 UTC. The inventory may have changed. The page shows dollar signs without explicitly naming the currency. Its differences above assume the same dollar basis as our US bill. An unstated setup fee is not a zero fee.

NVIDIA specifies the 96 GB card and Spark's unified memory. Unified memory is shared with the system, not a separate 128 GB VRAM allocation.

Hetzner's GEX matrix and its price service give the EUR and USD figures separately, net of VAT, so we did not convert anything. Hetzner lists an hourly rate and a monthly maximum for each server; the table shows the maximum. Its page labels disagreed about tax, so confirm it at checkout. GEX131 comes in several configurations; the row shows the starting one.

These rows compare the GPU and the price. The servers also differ in CPU, RAM and disks, so read each listing before comparing like for like.

An hourly GPU can make a smaller experiment

DigitalOcean's GPU pricing offers a different commitment. Below, an always-on month means 730 hours, chosen for comparison. These totals are multiplication, not a published monthly cap.

Each total is the hourly rate multiplied by 730 or 40. No separate setup fee is listed for these on-demand Droplets. Taxes and additional billable services are outside this compute comparison.

The billing documentation specifies per-second billing, with a minimum of 60 seconds or $0.01, whichever is higher. Powering off does not stop billing. Destroy the Droplet to release it. We did not find a GPU monthly cap; the CPU Droplet cap should not be applied to GPUs.

The calculator did not expose GPU estimates in our initial read. We used explicit rates and billing rules.

If we keep our CPU server and add 40 hours of L40S time, the combined compute bill is $125 + $62.80 = $187.80. The added cost is $62.80. That is different from replacing the CPU server with an always-on GPU server.

Memory decides what fits; a benchmark decides what feels fast

Hugging Face's Llama 3.1 memory table estimates weight memory as follows:

These are approximate weights-only requirements. Context cache, runtime allocations and simultaneous requests need room too. A 35 GB weight estimate does not guarantee that our intended 70B service will work comfortably on a 48 GB card.

The gpt-oss model card gives model-specific examples: gpt-oss-20b fits within 16 GB, while gpt-oss-120b runs on one 80 GB GPU using MXFP4 quantization of its mixture-of-experts weights. This is not a general rule that every 120B model fits in 80 GB. The exact model, format and supported runtime matter.

GPU speed also needs a stated workload. Published aggregate serving throughput cannot be compared directly with our CPU's individual generation timer.

NVIDIA NIM's published tests used Llama-3.1-8B-Instruct, NIM 1.8.0, FP8, tensor parallelism 1, and 1,000 input plus 1,000 output tokens:

The H100 host was a DGX H100 with dual Xeon Platinum 8480CL processors. The L40S host was a Supermicro system with dual Xeon Platinum 8570 processors. Both used driver 570.124.06. NVIDIA defines system throughput across simultaneous requests. These are vendor measurements, with different models and conditions from our CPU tests, not a controlled CPU-versus-GPU speedup.

We found no comparable verified throughput figure for the exact Hetzner Blackwell configurations or InterServer inventory in this comparison. We would not estimate one from advertised TOPS. A measurement on the same GPU model is still not a performance guarantee for a particular rented server.

An API has more than one ceiling

Anthropic's rate-limit documentation separates requests per minute, input tokens per minute and output tokens per minute. For Claude Sonnet 5.5, its standard tiers show:

New organizations may start in Evaluation below those limits. Custom arrangements differ. Most Claude models count uncached input toward the input limit. A sudden traffic increase can also encounter acceleration limits.

Temporary limits return 429 responses. Respect retry-after when supplied. Reaching the tier's monthly spend cap also returns 429, but without that header; immediate retries will not restore the allowance.

A Claude subscription is a separate arrangement. Paid plans have a session limit resetting every five hours and weekly limits. Usage across Claude, Claude Code and Claude Desktop shares the allowance. There is no universal published message count we can turn into a guaranteed agent capacity.

OpenAI's API limits also depend on organization, project and model. Its Build, Launch and Grow tiers require $5, $100 and $500 in total credit purchases, with monthly usage limits of $500, $5,000 and $200,000. Those usage limits are separate from configurable spending controls.

For the documentation's grouped labels Astra, Sol and Terra, Build lists 5,000 requests and 1,000,000 tokens per minute; Launch lists 10,000 and 4,000,000; Grow lists 15,000 and 40,000,000. We preserve those labels rather than applying the numbers to every model. Temporary 429 responses can include Retry-After. A configured hard spend limit can also stop affected requests with 429.

Gemini's documentation measures requests per minute, input tokens per minute and requests per day. Limits belong to the project, not each API key. Actual per-model allowances are in AI Studio, so we cannot publish a universal RPM figure. Where applicable, its rolling ten-minute spend limits are $10 for Tier 1, $50 for Tier 2 and $200 for Tier 3. Exceeding these returns 429 RESOURCE_EXHAUSTED.

For a business, adding agent processes or API keys does not necessarily add allowance. Many jobs share the same quota. Long documents can exhaust tokens before requests. Queue work, bound retries and separate urgent requests from overnight batches. A local fallback helps only if it can perform the task to an acceptable standard.

When we would pay for a GPU

A rented dedicated machine still belongs in a hosting provider's environment. Local inference can avoid sending prompts to an LLM API, but backups, logs, gateways and fallback routes must follow the same data policy. Buying a GPU does not establish privacy by itself.

Predictable rental cost can help steady work. It also buys idle hours, maintenance and the responsibility for availability. We would compare cost per accepted result, including corrections, rather than cost per generated token alone.

Our practical next step

We would keep the CPU server and choose one repetitive task with known answers. Record document lengths, daily volume, queue delay, errors and review time. Try an hourly GPU for a day before committing to a monthly rental. For example, 24 hours at the listed L40S rate is 24 x $1.57 = $37.68 in compute.

Use our prompt hub approach and server setup to put API and local inference routes behind one controlled LiteLLM gateway. A tested endpoint change can then be configuration work. Model differences still need evaluation. Subscription clients keep their supported sign-ins; a gateway does not turn plan allowances into API credit.

We would move triage and extraction first. We would keep difficult judgment and final writing on the model that passes review. Add the GPU when the workload records justify its cost.

Fact-check notes and sources

All sources were read on October 18, 2026 (UTC). InterServer's live page showed our tools a "verify you are human" check, so its prices come from the Internet Archive copy listed below. DigitalOcean states, "Powering off a GPU Droplet does not stop billing."

• Server: our hardware and workload snapshot, about 01:40; SYS-5 at 01:44:45; US catalog at 01:45:08 confirms rental and installation. The initial product HTML omitted prices.
• Dedicated GPU prices: InterServer archive, read 01:45:07, capture September 30 at 19:29:55. Hetzner matrix and net price services for GEX45, GEX63, GEX131, 01:44:46 to 01:44:47.
• Hourly prices: DigitalOcean pricing, billing and calculator linked above, 01:44:46 to 01:44:47. Every monthly and part-time total is our stated arithmetic.
• Memory: NVIDIA specifications linked above, 01:46; Hugging Face model card and weight table linked above, 01:44:48 to 01:44:49.
• GPU measurements: NVIDIA performance tables, including complete Markdown, and metrics linked above, 01:45:08 to 01:45:09. No new benchmark was run.
• Provider limits: Anthropic, OpenAI and Gemini documentation, 01:44:47 to 01:44:48; Claude plan help, 01:44:48. These are dated examples, not account entitlements.
• CPU measurements: the two site articles linked above hold the October 4 extraction, October 13 concurrency and five-agent observations.
• Job peak: our own job log (progress.txt), October 11 to 18, 2026: 167 jobs with both a start and an exit line; the peak of eight running at once comes from pairing each start with its exit. Jobs with no exit line were left out.

Related reading

• The sovereign advantage
• Build a self-hosted AI server
• Small quantized models on our server
• Local AI, on-premises and cloud
• AWS Terraform costs versus sovereign AI
• A safe update command for a self-hosted server

This post is informational, not legal, licensing or financial advice. All vendor and product references, including OVHcloud, Intel, Samsung, Supermicro, Ollama, LiteLLM, Open WebUI, Qdrant, n8n, Guacamole, Microsoft Windows, NVIDIA, InterServer, Hetzner, DigitalOcean, AMD, Hugging Face, Meta Llama, Hermes, Qwen, Anthropic, Claude, OpenAI and Google Gemini, are nominative fair use. No affiliation is implied.