# How to Diversify Your AI API Spend Across Providers Before Your Vendor Cuts You Off

Most advice about escaping AI vendor lock-in makes your bill worse. The Anthropic batch discount does not exist on Bedrock or Vertex, and your likeliest cutoff is your own tier spend cap. Here is what actually works, with verified top-up URLs.

Author: J.A. Watte
Published: August 5, 2026
Source: https://jwatte.com/blog/diversify-ai-api-spend-across-providers/

---


A friend asked me how to move his workloads off a single AI vendor. He had the usual plan: sign up for a few cheaper providers, put a gateway in front of everything, and route by price. It is the plan almost everyone arrives with, and I have built enough of it to know that the first version of it costs you money instead of saving it.

So I went and checked. I pulled the current pricing and billing mechanics for nine providers against their own documentation, then had every price and every URL attacked by a second pass whose only job was to refute the first. All nine came back needing corrections. A promotional rate had been recorded as the standard price. Two URLs printed inside a vendor's own documentation were dead. A batch eligibility list turned out to apply to a region my friend cannot use. A rate limit rule was a retired policy still being repeated by third-party write-ups.

That is the state of the ground you are planning on. Underneath it, three things are true that reshape the whole exercise, and none of them are what he expected.

Your most likely cutoff is not a ban. The two most common diversification moves both roughly double the unit cost of the work you keep. And adding vendors is not the same as adding insurance.

## The cutoff you should actually plan for is your own spend cap

Everybody plans for the dramatic version. The account gets flagged, a false positive somewhere in a trust and safety pipeline, and you wake up to 401s. It happens, and I have written about the continuity plan for it before.

But if you are a small shop, there is a far more likely cutoff sitting in your account settings right now, and it is on a schedule.

Anthropic's rate limit tiers carry hard monthly spend ceilings. Start is $500 a month. Build is $1,000. Scale is $200,000. The documentation says plainly what happens when you get there: "Once you reach your tier's spend cap, API usage pauses until the next month unless you request a higher limit."

You cannot buy past it with prepaid credits. Loading more money into the account does not raise the ceiling, because the ceiling is not about your balance. So the failure mode is that a good month, a viral post, a runaway agent loop, or one badly scheduled backfill puts you against a wall in the third week and your pipeline stops until the calendar rolls over.

Mistral has the same shape with an org-level monthly spending limit that suspends API access. Kimi has a per-project daily budget cap that rejects all requests once hit. These are not obscure settings. They are the default behavior of the systems you already depend on.

Go look at your tier today. That is a five minute job and it is more likely to save your pipeline than anything else in this article.

## Both obvious diversification moves cost you money

Here is the part that surprises people, because it inverts the usual advice.

The Anthropic Message Batches API gives you 50% off. It exists on the first-party API and on Claude Platform on AWS. It does **not** exist on Amazon Bedrock, and it does **not** exist on Google Vertex AI. I want to be precise about this because the entire cost case for a Bedrock failover depends on it: if you move Claude work to Bedrock, every batchable job you had runs synchronously at full price.

It also cannot survive an OpenAI-shaped gateway. Batching is not a parameter, it is a separate endpoint with its own verb, its own request envelope, a polling lifecycle, and results that come back out of order keyed by an ID you assigned. A `/chat/completions` surface has nowhere to put any of that. So "proxy everything through one gateway and route by price" quietly converts your discounted asynchronous work into full-price synchronous work.

Stack that with the caching problem and it gets worse. Prompt caching reads cost about a tenth of input price, and caching is a prefix byte match: any change anywhere in the prefix invalidates everything after it. Render order is tools, then system, then messages, which means tools sit at position zero and are the most destructive thing to touch. Switching models invalidates the cache completely, because caches are model-scoped.

Read those two sentences together and you get the uncomfortable conclusion. The two things a portability layer exists to do, swap models and vary tool sets, are precisely the two things that destroy your caching discount.

None of this means do not use a gateway. It means know what you are handing over. A gateway is the right answer for observability, key management, and cheap failover on the workloads that were never batched or cached in the first place. It is the wrong answer for your batch pipeline. Keep a native path to the first-party batch endpoint no matter what else you build.

If you want a gateway anyway, self-hosted LiteLLM preserves the most of any option I checked. It normalizes cache control across seven providers, translating to Bedrock's `cachePoint` and Google's context caching API, and it exposes an Anthropic passthrough that reaches the batches endpoint natively, so the discount survives the hop. Its caveat is the same thing that makes it portable: it strips parameters a provider does not support, silently. Assert on the usage fields that come back. Do not trust that a parameter landed.

## Substituting a cheaper model wholesale usually does not work

I measured this on a news pipeline I run, which uses Claude for synthesis and a Chinese model for independent cross-reads. The question was the obvious one: what if we just moved everything to the cheap model?

The answer was that it would save about $0.97 a day, roughly $29 a month, because the cheap model still has to do the work. The tokens do not disappear when you change the model string.

Then it got worse. The tier that the fallback logic actually resolved to under load was priced at $2 per million input tokens, which is **more than Claude Sonnet costs at batch rates**. The naive version of the migration had negative savings, in the wrong direction, while breaking three product features that specifically depended on having two model lineages disagree with each other.

That is the general shape. Wholesale substitution moves work from a model with a discount structure you have already tuned to a model without one, and the arithmetic frequently comes out backwards. What works is matching each job to the cheapest model that can actually hold it, and being honest that some jobs are not movable at all.

Bulk classification, tagging, extraction, and dedupe are where the money is: high volume, low judgment, trivially verifiable. Condensing a large corpus before an expensive model reads it is the second best lever, because you pay once to shrink the input and then the expensive model reads less, and that compounds with volume. Final synthesis and anything a customer reads should stay on your best model, because quality is the product. Grading and scoring can move down a tier, but never to the same model being graded, since a model grading its own output produces a number with no information in it.

## The infrastructure change that matters more than the routing

Before you move a single workload, do this one thing, because it is what makes everything else reversible.

If every model string in your codebase is an inline literal, you cannot try a cheaper model without editing code, and you cannot revert during an incident without a deploy. Both of those facts make you route conservatively forever, which means you never capture the savings you are reading this article to find.

Replace the literals with per-job environment overrides and a kill switch per job:

```
MODEL_SUMMARIZE=deepseek-v4-flash
MODEL_CLASSIFY=gpt-oss-120b
MODEL_SYNTHESIZE=claude-opus-5

PILOT_SUMMARIZE=on
PROVIDER_SUMMARIZE=fireworks
```

Resolve those at call time with a null-safe fallback to the known-good model. The test for whether you did it right is simple: can you move one job to a different provider, and back again, by changing an environment variable and nothing else? Until that is true, every optimization you attempt is a deploy and a risk, and you will not attempt many.

## What actually protects you is portable weights

This is the finding that should drive your architecture, and it is not "use more vendors."

Eight accounts serving eight proprietary models is eight separate migrations waiting to happen. Every one of those models can be deprecated, repriced, or geofenced independently, and when it is, you rewrite. That is not insurance, it is diversified exposure.

What actually protects you is a small set of open-weight models that several independent providers serve at prices close enough that moving between them is a base URL and a model string. Three of them qualify right now.

**OpenAI's gpt-oss**, Apache 2.0 licensed, is the most portable model on the internet at the moment. Groq, Together, and Fireworks all serve the 120B at an identical $0.15 in and $0.60 out per million tokens, with the same 131,072 token context. Cerebras adds a fourth first-party leg at $0.35 and $0.75, which buys you roughly 3,000 tokens per second if you need it. OpenRouter fronts 19 more upstreams with a floor around $0.03 and $0.17. Groq and Fireworks both offer it at a 50% batch tier. If you standardize one model for portability, standardize this one.

**DeepSeek V4 Flash**, MIT weights, runs at exactly $0.14 and $0.28 at DeepSeek first-party, Together, and Fireworks, with a million tokens of context. Two of those three are US entities, so it is simultaneously a vendor hedge and a jurisdiction hedge. OpenRouter has 21 endpoints for it and the cheapest is about 40% below DeepSeek's own price, which is a strange and useful fact.

**Z.ai's GLM**, MIT weights, gives you GLM-5.2 at exactly $1.40 and $4.40 on Z.ai, Fireworks, and Together, plus roughly 30 more endpoints through OpenRouter. GLM-4.5-Air at $0.20 and $1.10 is the strategic sleeper: it is the one GLM on Together's batch-eligible list, so it is the GLM you can run at half price on a US provider. GLM-5.2 also exposes an Anthropic-shaped endpoint, which makes it the cheapest Claude-Code-compatible fallback available.

The trap on all three is context. The same model string carries wildly different context windows depending on the host. GLM-5.2 runs at 1,048,576 tokens on Z.ai and Fireworks, 512,000 on Together, 262,144 on several OpenRouter upstreams, and under 100,000 on one of them. Pin your effective context to whatever your smallest leg can serve, or you get silent truncation the first time you fail over, which is the worst possible moment to discover it.

The framing that follows from this: you are not diversifying vendors. You are diversifying vendors across a fixed set of portable weights. If your cutoff risk is regulatory rather than commercial, that distinction is exactly what saves you, because gpt-oss, DeepSeek V4, and GLM weights can all be served from a US or EU provider even if the lab that made them gets delisted.

## Where to actually load money, and the trap in the middle of it

Open and fund the accounts before you need them. A cold account is not a fallback, it is a form to fill out during an outage. Every provider has a first-payment delay of some kind, whether that is card verification, a tier threshold, or a payment rail you do not have.

Here are the pages, verified on the day of writing:

| Provider | Where you load credit |
|---|---|
| Anthropic | `platform.claude.com/settings/billing` |
| Moonshot / Kimi | `platform.kimi.ai/console/pay` |
| Alibaba Qwen | `billing-cost.console.alibabacloud.com/fortune/billing-account` |
| DeepSeek | `platform.deepseek.com/top_up` |
| Z.ai | `z.ai/manage-apikey/billing` |
| Groq | `console.groq.com/settings/billing/manage`, limits at `/settings/billing/limits` |
| OpenRouter | Credits page, reached from `openrouter.ai/workspaces/default/keys` |

On Alibaba the path after you land is Asset Information, then Cash Balance, then Top-up and Remittance. The older `console.anthropic.com` host still works but now 301-redirects to `platform.claude.com`.

Verify each of these yourself on the day you use it. Several vendor consoles are single-page apps that return HTTP 200 for **any** path, including ones that do not exist, so a link that loads is not evidence a page is real. That is not a hypothetical: two of the URLs I found printed inside Moonshot's own documentation were dead when checked.

### Payment rails, where the documentation and reality disagree

This is the part most likely to stop you on day one, and it is also the part where I would have misled you if I had trusted the docs alone.

**Kimi takes US credit cards in practice, even though the documentation does not say so.** Moonshot's international billing page lists the rails verbatim as "Online top-ups support WeChat Pay and Alipay QR code payments," with no card mentioned anywhere, and it names Beijing Moonshot AI Technology Co., Ltd. as the invoicing entity at a "tax-inclusive rate is 6%", which is a mainland VAT arrangement appearing in what is nominally the international document. Read that page cold and you would conclude a US developer cannot fund the account at all. The platform does accept US cards. The document is simply incomplete, which fits the other evidence that the international billing page is a partial translation of the mainland one.

The lesson generalizes past Kimi: **on these platforms, treat the billing documentation as a lower bound on what is accepted, and confirm at the checkout screen.** I have now been wrong in both directions on this, once by trusting a doc that understated reality.

**Alibaba is where a card is more likely to be declined, despite listing more of them.** Model Studio's documented card support is broad on paper: Visa, Mastercard, American Express, UnionPay, JCB, Discover, Diners Club, plus Alipay, PayPal, Apple Pay and Google Pay. The restrictions are what bite. Cards must support international transactions **and** 3D Secure. The platform explicitly rejects mainland-China-issued cards, prepaid cards, gift cards and virtual cards, and rejects PayPal accounts registered in mainland China. Each card binds to exactly one Alibaba Cloud account. In practice that funnels most people onto a plain Visa or Mastercard, and a virtual or privacy card of the kind many developers reach for first will be refused.

Alibaba is also not a prepaid API wallet at all. It is a cloud account with a payment method, settling pay-as-you-go against a threshold: around USD 1,000 for bank cards, and USD 8 to 1,000 for digital wallets depending on the account, with a monthly sweep for anything below. So "how much credit do I have" is not the right mental model, and there is no published minimum top-up.

Two adjacent Kimi facts worth knowing regardless. The international service is `api.moonshot.ai` with an account on `platform.kimi.ai`, and the mainland service is `api.moonshot.cn` with an entirely separate account, key, and balance. A great many tutorials and GitHub examples hardcode the `.cn` host, and copying one gives you an auth failure that looks exactly like a bad key. And the international contracting entity is Moonshot AI PTE. LTD., a Singapore company, with Singapore governing law and disputes going to arbitration in Singapore. There is no US entity and no small-claims path, so treat a first payment as unrecoverable and keep the balance you hold there small.

### Two funding traps that will page you at 3am

**Kimi returns HTTP 429 for an empty wallet, not 402.** When the available balance hits zero, requests fail with an `exceeded_current_quota_error` and a 429 status. Because that shares a status code with real rate limiting, a standard retry-with-backoff policy will spin against an empty wallet indefinitely, burning your latency budget while hiding the actual cause. Branch on the error type, not the status code. A quota error pages a human. A rate limit error backs off.

**Tier thresholds are cumulative, not monthly, and the bottom tier is unusable.** Kimi's minimum recharge is $1, and at $1 you get concurrency of 1 and three requests per minute. At $10 cumulative you get concurrency 50 and 200 requests per minute. Fund $10 immediately. Vouchers do not count toward the threshold, only cash does. OpenRouter has the same shape keyed differently: its limits are a function of **lifetime credit purchases**, not current balance, so topping up does nothing for your rate limit while cumulative spend does.

While you are collecting accounts, Alibaba gives you a million tokens per eligible model for 90 days, but only in the Singapore region with the deployment scope set to International. Pick Singapore, not US Virginia. Z.ai's flash models are genuinely free. Cerebras gives you $5 that expires 30 days after it is granted, which for a fallback leg sitting idle until a primary fails is close to worthless, so do not count it as insurance.

## Model IDs are a supply chain and nobody sends you a deprecation PR

Model strings are dependencies with sunset dates.

As I write this, Moonshot has already retired its entire k2 preview line, and the whole `moonshot-v1` family plus `kimi-k2.5` sunset around August 31, 2026. That matters more than it sounds, because the `moonshot-v1` family was the reliable fast tier for a long time and a lot of fallback ladders resolve to it. If yours does, your fallback disappears this month. Both of Moonshot's documentation domains also moved, so anything you read about Kimi from before roughly June 2026, including third-party pricing aggregators, is likely quoting dead model IDs.

Cerebras is deprecating its GLM-4.7 serving path on August 17, 2026. Anthropic's Sonnet 5 introductory pricing ends August 31, 2026, stepping from $2 and $10 to $3 and $15 per million tokens. If you built a budget on the introductory rate, it grows by half without any warning email.

Three habits keep this from biting. Never hardcode a model ID, so a swap is a config change. Resolve the available model list at runtime where the provider offers one, filter to the family you trust rather than taking the first match, and never silently downsize below the context your job needs. And put a recurring fifteen minute reminder to re-read the pricing and models pages of everything you depend on.

That last one sounds like busywork. It is the cheapest insurance on this page.

## The failures that actually waste your money are silent

Here is the turn in the argument. Everything above is about price. But when I audit a pipeline, most of the recoverable money is not in the rate card. It is in work that ran twice, work that ran on the expensive path by accident, and work that failed while reporting success.

I want to use a housing data site I run as the example, because it makes the point in the strongest possible way: **it has no LLM calls at all.** Every generative feature on it runs in the visitor's browser. And it made every mistake an LLM pipeline makes, which tells you these failures are not caused by models. They are caused by any pipeline with soft-failing external dependencies and no verification of its own output.

**A missing key silently published fake data.** The comment in the code says it better than I can: previously, a missing API key "automatically downgraded the entire pipeline to synthetic numbers and skipped Redfin entirely, which silently published fake prices to production." The fix was to separate "credential is missing" from "use the fake path" and make the fake path opt in by name, via a flag that CI never sets. Swap "synthetic numbers" for "a stub completion" and that is an LLM postmortem verbatim. If a missing key in your setup can route you to a cheaper model whose output still looks plausible, you have this bug today.

**A soft-fail destroyed the thing it was protecting.** One script fetches two large geographic datasets, catches any error, returns an empty object, and then writes the result unconditionally. It exits 0, deliberately, so that one upstream outage cannot block a weekly deploy. What it actually does is replace a good 1.7 MB artifact with `{}` and ship a map with zero markers. A fallback that writes over the last known good value converts a transient outage into permanent data loss. In an LLM pipeline this is the cached response overwritten with an empty completion, and the fix is one line: only write when you have something.

**Two whole page families produced nothing for three months.** The pipeline is deliberately fail-open per source, so one dead upstream does not take down twenty thousand pages. Reasonable. But the only record of a dead source is a warning line in a green build log, and two data families have been empty since early May across roughly twelve consecutive green weekly runs. The site even has a comment reading "never print a bare 0" so a dead source renders as "not yet published in this build", which sounds transient and has been true for a quarter. A degraded state described as temporary needs a check that escalates when it persists, or the euphemism becomes permanent and invisible.

**The freshness stamp lies.** The same repository contains both the correct rule, written in its own methodology page, that "a fresh build should not make stale data look new", and 51 pages that stamp the wall clock build date as their freshness signal regardless of whether the underlying data moved. A rerun that changes nothing relabels the output as new. The LLM version of this is stamping a generated-at timestamp on a cache hit.

That last pairing is my favorite thing in the whole audit, because it explains how these bugs survive. The correct rule and its violation ship from the same codebase, written by the same person, who is me.

## Ask the data, not the CI run

The other pipeline I audited, a news system, taught me the monitoring lesson the expensive way.

Per-step error isolation and honest job status are in direct conflict. In GitHub Actions, `continue-on-error` suppresses a step's **conclusion** but preserves its **outcome**. So a run in which every single step failed reports success, and looks byte-identical to a clean run.

One morning all eight ingest submits failed, because the target was unreachable from the runner for about eighteen minutes. Each one burned its five retries and exited 1. The run reported success. Seven sources self-healed at the next window. The eighth only runs in the window that failed, so it sat 47 hours stale and put Thursday's prices into a Saturday market brief.

You need two checks, because they answer different questions.

First, reconstruct honest job status from the outcomes that were preserved. Give each isolated step an `id`, then add a final gate that reads them:

{% raw %}
```yaml
      - name: Report submit failures
        if: always()
        env:
          OUTCOMES: |
            bea=${{ steps.bea.outcome }}
            fred=${{ steps.fred.outcome }}
        run: |
          FAILED=""
          while IFS='=' read -r src out; do
            [ -z "$src" ] && continue
            [ "$out" = "failure" ] && FAILED="$FAILED $src"
          done <<< "$(echo "$OUTCOMES" | sed 's/^[[:space:]]*//')"
          if [ -n "$FAILED" ]; then
            echo "::error::submit failed for:$FAILED"
            exit 1
          fi
```
{% endraw %}

Treat `skipped` as fine rather than as failure, because a source that deliberately runs in one window will skip in the other. And order this gate **after** your artifact upload, since step order is authoritative and an `exit 1` above the upload turns a recoverable incident into permanent data loss. One of my sources allows 25 calls a day and returns only the current snapshot, so a re-fetch can never recover an older value. The uploaded artifact is the only replay path that exists.

Second, probe the published artifact rather than the pipeline. A green run tells you the pipeline executed, not that it produced anything. Build a health endpoint that computes staleness from the data, with a per-source freshness budget set to roughly 1.5 times the producing schedule's cadence, so one delayed run does not flap but two missed runs alarm. Mark a source stale when its record is absent, its timestamp is unparseable, or its age exceeds budget. Then have an hourly job read that endpoint and open or close a labeled issue.

Three details separate a watchdog you trust from one you learn to ignore.

Retry the probe, and stay silent when the endpoint itself is unreachable. An unreachable site is indistinguishable from stale data, and a watchdog that files an issue during every deploy window trains you to ignore its own label.

Assert your denominator before you trust your numerator. This one embarrassed me. When my health endpoint's storage read throws, the staleness fields are absent from the response entirely. My watchdog read them with `jq -r '.ingestStaleCount // 0'`, got 0, concluded everything was fine, and **closed the open alarm**. A storage outage resolved the alarm instead of raising one. That `// 0` default is the standard defensive idiom, and here "field is missing" and "field is zero" mean opposite things. Check that the tracked count is above zero before you believe the stale count.

Self-heal before you escalate. Under four hours since the last green run, healthy, and close any open issue. At four hours, dispatch a recovery run, but only if nothing is already queued or running. At eight hours, open an issue too. That fixes the common transient failure with no human involved and only pages you after two failed recovery windows. Put a concurrency group on the target that queues rather than cancels, or your recovery dispatch will kill the run that was already recovering.

## The self-healing layers, in the order they should fire

Retries and watchdogs get lumped together, but they belong at different distances from the failure. Four layers, each idempotent, each a no-op when the layer below it already worked.

**Layer one is an in-process retry.** Exponential backoff with full jitter on 429, 408, and 5xx, and never on other 4xx. Use full jitter rather than plain doubling, or several simultaneous keepalives will thunder-herd the same blip. This layer exists because of a specific morning: a provider's batch endpoint returned intermittent 500s with empty bodies for about four hours, my submit code had no inline retry, and so every scheduled fire and every keepalive that touched a 500 failed outright. No work was queued, the drainer had nothing to drain, and the day's output and its email never happened. Five tries per fire beats an intermittent 500.

**Layer two is a freshness-gated keepalive.** A second scheduler fires the same idempotent endpoint, but a preceding step checks whether today's work is already filed and skips if it is. Mine now runs every thirty minutes across a thirteen-hour window. That sounds expensive and is nearly free, because once the work exists every subsequent fire is a no-op that costs one read. The gate is the whole trick: an ungated all-day keepalive is a cost leak, and a gated one is all-day insurance. Two independent schedulers matter more than they sound, because platform cron genuinely drops fires. I have watched a five-hour gap open in one scheduler's own timetable during a congested window.

**Layer three is a staleness watchdog that reads the output.** Covered above. It self-heals by dispatching a recovery run at four hours and only pages a human at eight.

**Layer four is a manual dispatch button** that takes the same parameters, so recovering by hand is not an improvisation under pressure.

Six mechanisms make those layers actually work rather than merely exist.

**Clear a wedged queue instead of just detecting it.** My drainer computed that a pending job was past its upstream expiry and then did nothing with that fact, so the queue stayed pinned forever and every later run found a job that would never complete. Computing a health signal and not acting on it is indistinguishable from not having the signal. It now clears the queue when the upstream job is past its documented 24 hour expiry.

**Give each job kind its own queue key and its own in-flight window.** When I added a second, smaller job to an existing pipeline, the tempting move was to reuse the pending key. That would have let either job orphan the other's in-flight work. Separate keys plus a 409 guard means a duplicate fire is free and the two jobs cannot corrupt each other.

**Freeze the date at submission, not collection.** Asynchronous work that recomputes "today" when results come back gets a different answer than when it was submitted, and the difference shows up as an off-by-one in day arithmetic on exactly the runs that straddle UTC midnight. My job map freezes the date at submit time so the collector cannot drift.

**Make later re-fires conditional on the actual output.** The first run of the day fires unconditionally. Later runs read the artifact they would produce and re-fire only if it is missing or a stub. Healthy days cost two reads. This is what turns "run it four times and hope" into "run it once and verify three times," and it fails open on a read error so a storage blip cannot suppress the real work.

**Bound the sum, not just each call.** A per-call timeout does nothing about N sequential calls that each have one. This is the bug I introduced in my own pipeline earlier today: moving to a slower model raised each call's ceiling to 150 seconds, and ten desks running sequentially made the pathological case 25 minutes against a 15 minute platform cap. The per-call timeout was correct and the aggregate was fatal. The fix is a wall-clock budget across the whole process, after which remaining work skips the enrichment and takes the documented degraded path. It almost never fires, which is the point.

**Cache before you call, and cap what you call.** Resolve the cache first, then cap items per run, then enforce a wall-clock budget in both the cache loop and the work loop. Cache-first ordering is what converts a recurring per-item cost into a one-time per-item cost, and it is the single highest-leverage line in my translation path.

## Three watchdog mistakes I made that looked like working watchdogs

These are the ones I would not have predicted, and each one produced a monitor that reported health while the thing it monitored was broken.

**A storage outage made my watchdog close the alarm.** Covered above, and worth repeating as a pattern rather than an anecdote: `jq '.field // 0'` cannot distinguish "the check returned zero" from "the check never ran." Assert the denominator before you trust the numerator.

**Shape-only validation refreshed the freshness clock.** My ingest endpoints validated that a field was a string and then wrote the record with a fresh timestamp. So a publish with every metric null **refreshed** the clock and read as perfectly fresh. Freshness monitoring detects an absent producer and is structurally blind to a broken one, because freshness and correctness are different questions and I had only built a monitor for one of them.

**A soft-fail overwrote the last known good value.** The fallback returned an empty object, the write was unconditional, and one upstream outage replaced a good 1.7 MB artifact with `{}` at exit code zero. In an LLM pipeline that is the cached response overwritten by an empty completion. Only write when you have something.

There is also a whole class of bug that only appears once you add concurrency. When I parallelized a scraper, a lazily-initialized browser singleton that had been safe for years became a race: two workers each passed the `if (!browser)` check, each launched one, and the cleanup could only ever reap one. The orphan kept the process alive for 137 minutes after a 28 minute job, so the symptom was "CI step hangs," not "scrape is slow." Caching the **promise** rather than the resolved value fixes it, and the general rule is that when you parallelize a loop which touched a lazy singleton, you have to guard the singleton. I keep an explicit `process.exit(0)` after the work completes as a backstop, because "exit when the event loop drains" is a liability when a handle you do not own is still registered.

## Make the artifact carry its own audit

The best monitoring idea in any of my pipelines is also the cheapest, and it has no cron at all.

One of my sites recomputes its own quality metrics over the built output at build time and publishes them on a public page: how many records exist, how their dates are distributed, how many are lagging the newest date, and which series contain duplicate dates. It is regenerated on every build, so it cannot go stale, and it inventories known defects in public rather than hiding them.

That ports directly to an LLM pipeline, and it is what I would build before another watchdog. For each run, publish counts computed **from the output artifact** rather than from logs: records processed, records skipped and why, records that took a fallback model and which one, records whose response failed to parse, and the model that actually served each class. Then the health question stops being "did the job exit zero" and becomes "does the artifact describe a run I would accept," which a person can answer in five seconds and a script can gate on.

The same discipline at row level: when my leaderboards drop rows for being the wrong vintage or too thin to be meaningful, they render the exclusion counts into the page. "Forty counties were dropped for reporting an older month, twelve for volume" tells a reader how much of the corpus was discarded. An LLM pipeline that silently skips records it could not process is hiding the same number.

## What I still owe my own pipelines

Auditing my own code for this article turned up gaps I had not fixed, which is the honest reason to publish them.

**My freshness map covers 19 of about 30 published endpoints.** Ten producers can die completely without the watchdog noticing, because they are simply not in the map. Widening it is a purely additive edit to one object literal, which is the most annoying kind of gap: cheap to close and easy to never get to.

**The outcome gate exists in one workflow and not its sibling.** The gate that reconstructs honest job status lives in the workflow where the incident happened. The other one still has eighteen `continue-on-error` steps, no step IDs, and no gate, so the exact failure I already diagnosed is still fully open there. Fixing one instance of a class and calling the class fixed is its own bug.

**Nothing gates on the freshness data I already publish.** The counts are computed and rendered, and no step asserts a minimum, compares this run to the last, or fails when a whole category comes back empty. That is how two data families produced nothing for roughly twelve consecutive green runs.

**My declared build and my CI step list have drifted.** The package script names seven steps and the pipeline runs five, so two of them never execute in production. One of them feeds a page that permanently renders "refreshes on next build," describing a run that never happens. The LLM version of this drift is the evaluation harness calling a different model than production does, which makes every eval result quietly meaningless.

**A balance alarm is missing.** Given that one provider returns a 429 for an empty wallet, poll the balance endpoint and alarm well above zero. Otherwise the first symptom of an empty account is a retry loop that looks like rate limiting.

## Attribute spend per call site, not per key

You cannot manage what you cannot attribute. I spent weeks believing one product was responsible for a Moonshot bill that turned out to be roughly 70% a different project sharing the same key.

Write a usage record on every model call, and record four things people usually skip. Record the model that **actually ran** rather than the one you requested, because with a fallback ladder those differ exactly when you most need to know. Record failed attempts too, at the attempted model's rate, because a ladder that fails twice before succeeding cost you three calls. Use race-free keys so concurrent writes cannot collide. And expose it behind an admin endpoint that aggregates per day, per call site, and per model.

Read the usage object carefully while you are at it, because the field names lie across providers. On Anthropic, `input_tokens` is the **uncached remainder only**. The real prompt size is that plus the cache creation tokens plus the cache read tokens. A dashboard that maps `input_tokens` to "prompt tokens" will report a well-cached agent that ran for hours as having sent four thousand tokens, and you will conclude your caching is broken when it is working perfectly.

Then record provenance in the output record itself, not just the usage log. One line is enough:

```js
record.source = record.source || 'fallback: glm-4.7 after claude timeout';
```

That is the entire difference between degraded output you can identify after the fact and degraded output that is indistinguishable from healthy output.

## Verify the cheaper model on your own workload

Never trust a vendor benchmark for your workload. Build a golden set instead, and make it small enough that you actually run it.

Pull 50 to 100 real inputs from production, weighted toward the awkward ones: the longest, the shortest, the ones in another language, the ones that have failed before. Record your current model's output as a reference. Run the candidate over the same inputs with the same prompt. Grade with a **third** model, never with either of the two being compared, and ask for a per-item verdict with a reason rather than a score, because reasons are auditable and scores are not. Then read the disagreements yourself, all of them if there are fewer than twenty. That is where you learn whether the cheaper model fails in a way that matters or in a way you can ignore.

Two things a benchmark will not tell you.

JSON mode is three different guarantees sold under one parameter name. Asking for a JSON object guarantees syntactically valid JSON and nothing else, so fields you marked required can be missing. A schema with strict mode guarantees conformance but restricts which schema features you may use. And a third class of provider accepts the parameter and treats your schema as a strong hint, which gives you plausible JSON that fails validation intermittently under load. Test with a deliberately awkward schema before you trust it.

And tool descriptions are not portable prompts. The same tool schema gets different trigger rates on different providers and different model generations. Recent Claude models reach for tools less readily than older ones and respond well to prescriptive "call this when X" descriptions, while descriptions written with "CRITICAL: you MUST" over-trigger on models that follow instructions literally. Swapping providers without re-tuning your tool descriptions is the most common silent quality regression there is.

## Parameter traps that will 400 a working codebase

A short list of the ones most likely to bite on the first swap.

Kimi's k2.x and k3 models hard-fix sampling parameters and **error** on your values rather than ignoring them. Strip `temperature` and `top_p` from the payload entirely. Do not send 1.0 to be safe, do not send null. The inverse trap matters too: `moonshot-v1` models do accept temperature, so a shared code path that worked there breaks the moment you switch to a k2.x string.

Thinking cannot be disabled on the newest Kimi models, and `kimi-k3` defaults its reasoning effort to **max**. A naive port silently buys the most expensive reasoning setting on a model that charges $15 per million output tokens. Set it low explicitly unless you need the depth.

The answer lives in the content field, while reasoning arrives separately and first. Code that renders the first non-empty delta shows your user raw chain of thought. Code that reads only the content field looks hung for the entire reasoning phase and then bursts, which is why people set aggressive timeouts and then wonder why calls fail. DeepSeek has the sharper version: with tools present, the reasoning field must be echoed back on every subsequent turn or you get a 400, and generic wrappers drop unknown assistant fields, so tool-calling conversations die on turn two.

DeepSeek's thinking mode accepts sampling parameters and ignores them. Your determinism knob becomes a no-op with no signal at all.

Current Anthropic models removed sampling parameters entirely, so `temperature` is a 400 on Opus 5, Sonnet 5, and the 4.7 and 4.8 family. So is the classic trailing assistant-turn prefill used to force JSON shape. And on Opus 5 thinking is on by default, where on 4.8 omitting the field meant off, with `max_tokens` capping thinking and response text together. A route that never set thinking and sized `max_tokens` tightly around its answer will truncate mid-response.

Qwen prices several models in context brackets, and the entire request bills at the bracket it lands in. A 40,000 token prompt does not pay the cheap rate for the first 32,000. It reprices the whole call. Qwen's region keys are also not interchangeable, so a key minted for Singapore silently fails against Virginia.

## Where to start on Monday

Six steps, in this order, because the order matters.

1. **Look up your tier's monthly spend cap** and set an alert well below it. Five minutes, and it addresses your likeliest outage.
2. **Find your duplicate pipeline.** Grep your schedules and look for anything firing more than once a day that writes to the same key each time. Every fire after the first is a paid no-op. This is the most common leak I find and it survives migrations, because the old synchronous path stays enabled after the batch migration ships.
3. **Add per-job model environment variables** with a kill switch. Until a swap is a config change, you will not try anything.
4. **Fund a second and third account** with real money before you need them, and run one request through each so you know the key works. Start with an open-weight model that three providers serve.
5. **Add the outcome gate and the freshness probe.** If your monitoring cannot see a full provider outage, adding another provider does not help you.
6. **Run the cutoff drill.** Revoke a key in staging, on a normal Tuesday, and time the fallback. Then read the output, not the status codes, and decide whether you would have shipped it.

I wrote the whole thing up as a checklist with the verified price tables, the top-up URLs, the code patterns, and the drill: **[/downloads/llm-provider-diversification-playbook.md](/downloads/llm-provider-diversification-playbook.md)**. It is plain Markdown, free to fork and hand to your team, and it carries the verified price and top-up tables plus copy-paste versions of every guard described above. That is the version my friend got.

The thing I would most want you to take from it is the reframing. You are not trying to leave your vendor, and you are probably not going to get banned. You are trying to make sure that no single company's pricing page, deprecation schedule, regional availability, or payment rail can stop your work. That is a smaller, more achievable, and considerably cheaper goal than the one most people set, and it starts with knowing where your money actually goes.

That is the wider argument my book *The $20 Dollar Agency* makes at length, if you want it: most of what small businesses get quoted four figures for is now a config change and an afternoon of attention.

## Fact-check notes and sources

All figures verified against official vendor documentation on **August 5, 2026**. Every price in this article survived a second adversarial pass whose job was to refute it; several first-pass figures did not survive and were corrected before publication.

- **Anthropic tier spend caps of $500, $1,000, and $200,000 per month, and the quoted pause behavior**: [Anthropic rate limits documentation](https://platform.claude.com/docs/en/api/rate-limits).
- **Anthropic model pricing, batch discount, cache read and write multipliers, and the Sonnet 5 introductory rate ending August 31, 2026**: [Anthropic pricing](https://platform.claude.com/docs/en/about-claude/pricing).
- **Message Batches API not available on Amazon Bedrock or Google Vertex AI**: [Anthropic batch processing documentation](https://platform.claude.com/docs/en/build-with-claude/batch-processing). This is the single load-bearing claim behind the cost argument against a Bedrock failover, and it was verified specifically.
- **Prompt caching prefix-match behavior, model-specific minimum cacheable prefix, and model-scoped cache invalidation**: [Anthropic prompt caching documentation](https://platform.claude.com/docs/en/build-with-claude/prompt-caching).
- **Kimi's documented payment rails, the Beijing invoicing entity, and the 6% tax-inclusive rate**: [Moonshot account and payments guide](https://platform.kimi.ai/docs/guide/account-and-payments.md). That page lists only WeChat Pay and Alipay. **Correction, 2026-08-05:** the platform does in fact accept US credit cards, so this page understates what is accepted and should not be read as an exhaustive list. An earlier version of this article drew the stronger conclusion that a US developer had no way to fund the account, which was wrong. Confirm the rails at the checkout screen rather than from the doc.
- **Alibaba Model Studio accepted cards and wallets, the 3D Secure and international-transaction requirements, the rejection of mainland-China-issued, prepaid, gift and virtual cards, one card per account, and the USD 1,000 bank-card / USD 8 to 1,000 digital-wallet settlement thresholds**: Alibaba Cloud billing and payment-method documentation, verified 2026-08-05.
- **Kimi pricing, the 40% batch discount at 60% of standard price, and the three batch-eligible models**: [Moonshot batch pricing](https://platform.kimi.ai/docs/pricing/batch.md) and the per-model pricing pages under `platform.kimi.ai/docs/pricing/`.
- **The moonshot-v1 family and kimi-k2.5 sunsetting August 31, 2026**: [Moonshot models list](https://platform.kimi.ai/docs/models.md).
- **Kimi tier thresholds keyed to cumulative recharge, and the zero-balance 429 with an `exceeded_current_quota_error` type**: [Moonshot limits](https://platform.kimi.ai/docs/pricing/limits.md) and [balance API](https://platform.kimi.ai/docs/api/balance.md).
- **Kimi fixed sampling parameters erroring rather than being ignored**: [Moonshot models overview](https://platform.kimi.ai/docs/api/models-overview.md).
- **Moonshot's built-in web search billing at $0.005 per successful call on top of tokens**: [Moonshot tool pricing](https://platform.kimi.ai/docs/pricing/tools.md).
- **Qwen pricing, context-bracket billing where all tokens bill at the bracket rate, and international batch eligibility limited to four legacy models**: [Alibaba Model Studio pricing](https://www.alibabacloud.com/help/en/model-studio/model-pricing) and [batch inference](https://www.alibabacloud.com/help/en/model-studio/batch-inference).
- **Qwen free quota of one million tokens per model for 90 days, restricted to the Singapore region with International deployment scope**: quoted from the Model Studio free quota documentation.
- **DeepSeek pricing including the cache-hit rate**: [DeepSeek pricing](https://api-docs.deepseek.com/quick_start/pricing).
- **Z.ai GLM pricing including the free flash models**: [Z.ai pricing](https://docs.z.ai/guides/overview/pricing).
- **Multi-provider price parity for gpt-oss-120b, DeepSeek V4 Flash, and GLM-5.2, and the per-endpoint price and context variation**: provider pricing pages plus the OpenRouter model endpoints API, which lists every upstream serving a given model with its own price and context window.
- **OpenRouter rate limits keyed to lifetime credit purchases rather than current balance, and the 5.5% card top-up fee**: [OpenRouter limits](https://openrouter.ai/docs/api-reference/limits).
- **Cerebras first-purchase tier promotion, per-model rather than per-tier limits, the 30-day expiry on free credits, and the GLM-4.7 deprecation on August 17, 2026**: [Cerebras rate limits](https://inference-docs.cerebras.ai/support/rate-limits) and [model overview](https://inference-docs.cerebras.ai/models/overview).
- **Together's batch discount described as "up to 50%" and limited to six named models, with DeepSeek and Kimi explicitly excluded**: [Together batch inference](https://docs.together.ai/docs/batch-inference).
- **Mistral current tier naming and the org-level monthly spending limit that suspends API access**: [Mistral tier documentation](https://docs.mistral.ai/admin/user-management-finops/tier).
- **LiteLLM Anthropic passthrough preserving the native batches endpoint**: LiteLLM proxy documentation. No public price is currently published for its Enterprise tier, so I have not quoted one.
- The pipeline incidents, code comments, and monitoring failures described in this article are from two systems I operate and audit myself. Quoted comments are verbatim from my own source. No client system is described.

Prices and model availability move faster than anything else in this article. Verify against the live pages before you commit spend, and note that several vendor consoles return HTTP 200 for paths that do not exist, so a URL that loads is not proof a page is real.

## Related reading

- [Your AI Vendor Lockout Continuity Plan](/blog/ai-vendor-lockout-continuity-plan/): what to do about the dramatic version of the cutoff, the one where the account gets flagged rather than capped.
- [Cut Your AI Bill in an Afternoon](/blog/blog-cut-ai-bill-in-an-afternoon/): the shorter, faster predecessor to this article, focused on the five checks that find waste in any recurring model spend.
- [Claude Code Multi-Model Routing](/blog/claude-code-multi-model-routing/): the routing argument at the level of a single developer's daily work rather than a pipeline.
- [One Cloudflare AI Gateway in Front of 24 Providers](/blog/cloudflare-ai-gateway-small-business/): the gateway path in depth, including the auth header inversion that causes most unexplained 401s.
- [Validate an AI Model Before You Trust It](/blog/blog-validate-ai-model-before-upgrade/): the golden-set method in more detail, for when you are about to route real traffic to something cheaper.
- Two free browser-side tools if you want to put numbers on your own spend before changing anything: the [Model-Tier AI Cost Calculator](/tools/model-tier-cost-calculator/) and the [AI Vendor Cost Reverse Calculator](/tools/ai-vendor-cost-reverse-calculator/). Both run entirely in your browser, no signup.

---

*This post is informational, not legal, financial, or procurement advice. Mentions of Anthropic, Moonshot AI, Alibaba Cloud, DeepSeek, Z.ai, OpenRouter, Groq, Together AI, Fireworks AI, Cerebras, Mistral, Amazon Web Services, Google Cloud, and Cloudflare are nominative fair use. No affiliation or endorsement is implied.*


---

Canonical HTML: https://jwatte.com/blog/diversify-ai-api-spend-across-providers/
RSS: https://jwatte.com/feed.xml
JSON Feed: https://jwatte.com/feed.json
Hero image: https://jwatte.com/images/diversify-ai-api-spend-across-providers.webp