# Together AI for a Small Business: What It Actually Costs, What Works, and Where It Is the Wrong Tool

Verified Together AI pricing, working code, and the two findings that change the retrieval advice: there is exactly one serverless embedding model with a 514-token context, and rerank is not on serverless at all.

Author: J.A. Watte
Published: August 6, 2026
Source: https://jwatte.com/blog/together-ai-for-small-business/

---

Together AI is one of the few places a small business can run serious open-weight models by the token without talking to a salesperson. It is also full of pricing units that look alike and are not, and two limits that quietly break the most common thing people want to build with it.

I pulled every number here off Together's own pricing page and documentation on August 5, 2026, then had a second pass attack each one. The corrections mattered. One unit error would have overstated dedicated GPU cost by sixty times. Another claimed a product did not exist when it does.

Here is the version that survived.

## The one thing to understand before you sign up

Together is **fully prepaid, with a $5 minimum, and no free trial.** From the billing docs:

> "Together AI does not currently offer free trials. Access to the Together platform requires a minimum $5 credit purchase."

> "Together AI is fully prepaid. You need a positive credit balance to use the platform... If your balance reaches zero, API access is suspended until you add credits."

An API key alone will not make a call succeed. That trips people on their first attempt, and it means a zero balance is a hard stop rather than a degraded mode. Set a calendar reminder or watch the balance, because there is no grace period.

## Getting a call working

The base URL is `https://api.together.ai/v1` and it is OpenAI-compatible. Note the package names differ between languages, which catches people out:

```bash
pip install together        # Python
npm install together-ai     # Node
```

```python
from together import Together

client = Together()   # reads TOGETHER_API_KEY from the environment

resp = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[{"role": "user", "content": "Summarise this support ticket in one sentence."}],
)
print(resp.choices[0].message.content)
```

Or through the OpenAI SDK with two changes:

```python
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["TOGETHER_API_KEY"],
    base_url="https://api.together.ai/v1",
)
```

Together's own documentation describes porting as "two changes: the API key and base URL." That is nearly true and the same page contradicts it under known incompatibilities. **Model strings are a mandatory third change.** Together IDs are namespaced, so `gpt-4o` or `text-embedding-3-large` return a 404. `logit_bias` is also unsupported on most models. Budget for a find-and-replace across every model string in your codebase, not a two-line diff.

## What the models actually cost

Per million tokens, read off the serverless catalogue on August 5, 2026. The cached column matters more than people expect, and I will come back to why.

| Model | Input | Cached | Output | Context |
|---|---|---|---|---|
| `openai/gpt-oss-120b` | $0.15 | none | $0.60 | 128,000 |
| `openai/gpt-oss-20b` | $0.05 | none | $0.20 | 128,000 |
| `deepseek-ai/DeepSeek-V4-Flash-0731` | $0.14 | $0.03 | $0.28 | 1,000,000 |
| `deepseek-ai/DeepSeek-V4-Pro` | $1.74 | $0.20 | $3.48 | 512,000 |
| `zai-org/GLM-5.2` | $1.40 | $0.26 | $4.40 | 262,144 |
| `Qwen/Qwen3.5-9B` | $0.17 | none | $0.25 | 262,144 |
| `Qwen/Qwen3.7-Plus` | $0.32 | none | $1.28 | 1,000,000 |
| `Qwen/Qwen2.5-7B-Instruct-Turbo` | $0.30 | none | $0.30 | 32,768 |
| `moonshotai/Kimi-K2.6` | $1.20 | $0.20 | $4.50 | 262,144 |
| `moonshotai/Kimi-K3` | $3.00 | $0.30 | $15.00 | 1,000,000 |
| `meta-llama/Llama-3.3-70B-Instruct-Turbo` | $1.04 | none | $1.04 | 131,072 |
| `LiquidAI/LFM2.5-8B-A1B` | $0.03 | none | $0.12 | 32,768 |

A caution on catalogue drift: the documentation's chat table carries 23 rows while the pricing page carries 30, and they disagree on which models have cached pricing. Qwen3.7-Max has a $0.13 cached rate on the pricing page and a blank cell in the docs. Check both pages for anything you plan to depend on.

**Prompt caching is automatic and unreliable by design.** There is no header or parameter to enable it. It is prefix-based, so only the longest matching prefix counts and everything after the first difference bills at full rate. And, in Together's words, it is "best-effort and short-lived: the serverless cache is shared across the fleet and entries are evicted as traffic shifts, so cache hits aren't guaranteed and there's no configurable retention window."

The part that matters for planning: **only 14 models have a published cached rate at all.** Models without one bill every input token at the standard rate. And `gpt-oss-120b`, the most portable model on the platform, is one of the models with no cached rate. So if your workload is a big stable prompt plus a small variable question, the cheapest sticker price is not the cheapest bill.

## The two findings that change the retrieval advice

If you came here to build search over your own documents, read this part twice, because the tidy pipeline everyone draws does not exist on serverless.

**There is exactly one serverless embedding model, and its context window is 514 tokens.** That is `intfloat/multilingual-e5-large-instruct` at $0.02 per million tokens, 1024 dimensions. Five hundred and fourteen tokens is roughly two paragraphs. Every design that assumed you could embed a page at a time needs rechunking, and chunk boundaries are where retrieval quality goes to die.

**There is no rerank model on serverless.** From the documentation: "There are currently no rerank models offered via serverless. Rerank models like `mixedbread-ai/mxbai-rerank-large-v2` are only available with dedicated model inference."

Rerank is the step that takes twenty mediocre retrieval hits and puts the three good ones on top. It is the cheapest quality win in retrieval, and on Together it is not a per-token line item. It is a dedicated endpoint, which means GPU-hours.

So the honest shape of a retrieval build on Together is: embeddings by the token for almost nothing, your own vector store, and either no rerank or a dedicated endpoint whose economics only work at real volume. That is not a reason to avoid Together. It is a reason to not draw the diagram with rerank in it and then discover the bill.

## Dedicated endpoints are priced per hour, not per minute

This is the sixtyfold trap. Two different tables on the same pricing page:

- **Dedicated Inference**, headed "All prices per gpu per hour": NVIDIA HGX H100 at $5.49, HGX B200 at $8.99 on demand. Larger GPUs are contact-sales.
- **GPU Clusters**, headed "All prices are per GPU per hour": HGX H100 at $3.99 on demand, dropping to $3.69 for 7 to 30 days, $3.45 for 31 to 90, and $3.19 for 91 to 180.

So "an H100 costs $5.49 an hour" and "an H100 costs $3.99 an hour" are both true, on the same page, for different products. Always name which table you mean.

**Provisioned Throughput** is the only per-minute product, at $0.05 per throughput unit per minute, and Together's own savings comparison assumes continuous 24/7 provisioning of roughly 43,800 minutes a month. Run that arithmetic before you assume it is cheaper.

For a small business the practical read is simple. A single H100 on dedicated inference at $5.49 an hour is about $4,000 a month if you leave it on. You need a lot of steady traffic before that beats per-token serverless, and "a lot" here means the kind of volume that comes with a real product, not an internal tool.

## Batch: real, but narrower than the headline

The batch page leads with "up to 50% lower cost." The enforceable mechanic is a list, and it is short. Exactly six model IDs get the discount:

`meta-llama/Llama-3.3-70B-Instruct-Turbo`, `meta-llama/Llama-3-70b-chat-hf`, `Qwen/Qwen2.5-7B-Instruct-Turbo`, `mistralai/Mixtral-8x7B-Instruct-v0.1`, `zai-org/GLM-4.5-Air-FP8`, and `openai/whisper-large-v3`.

Followed by: "Models not listed run at standard rates." Three of those six are no longer in the serverless chat catalogue at all, which narrows the practically useful list to about three. If your overnight bulk job runs on `gpt-oss-120b` or a DeepSeek model, batch buys you asynchronous processing and no discount whatsoever.

The limits are generous where it counts: up to 50,000 requests per batch, 100 MB per input file, 10 MB per line, and up to 30 billion tokens enqueued per model. The completion window is 24 hours, cannot be changed, and is described as a best-effort target rather than a guarantee. Batch also runs against a separate rate-limit pool from the real-time API, which is a genuine operational benefit even without the discount.

## Fine-tuning, and when it is worth it

Standard fine-tuning is priced per million training tokens, multiplied by epochs, plus any evaluation tokens:

| Base model size | SFT LoRA | SFT full | DPO LoRA | DPO full |
|---|---|---|---|---|
| Up to 16B | $0.48 | $0.54 | $1.20 | $1.35 |
| 17B to 69B | $1.50 | $1.65 | $3.75 | $4.12 |
| 70B to 100B | $2.90 | $3.20 | $7.25 | $8.00 |

**Every job carries a $4.00 minimum charge**, so tiny experiments cost $4 regardless. And specialised models cost far more than the table suggests: GLM-5 and GLM-5.1 are $40.00 for SFT LoRA with a $60.00 job minimum, and the Kimi K2 series is $15.00 with the same $60.00 floor.

Worked example. Fine-tuning a 14B model on 20,000 examples averaging 400 tokens, for three epochs, is 24 million training tokens. At $0.48 per million that is about $11.52, so call it $12. That is genuinely cheap.

The cost that gets people is not training, it is serving. A fine-tune runs on a **dedicated endpoint**, unconditionally: for LoRA, "after the job completes, deploy the merged model on a dedicated endpoint." So a $12 training run commits you to roughly $4,000 a month of GPU to serve it, unless you download the weights and host them elsewhere.

That reframes the decision entirely. Fine-tune when you have steady, high volume and a task where prompting genuinely plateaued. For a small business with bursty traffic, better prompting plus a cheap model with a long context will beat a fine-tune on total cost almost every time. Try the prompt first. It is free to iterate.

## What a small business should actually build

Three shapes work well and one does not.

**Classification and extraction at volume.** Tagging support tickets, routing inbound email, pulling fields out of documents. Low judgment, high volume, easy to verify. Use `gpt-oss-20b` at $0.05 in and $0.20 out, or `LFM2.5-8B-A1B` at $0.03 and $0.12. Rough arithmetic: 10,000 tickets a month at 800 input and 150 output tokens is 8 million input and 1.5 million output. On `gpt-oss-20b` that is about $0.40 plus $0.30, so under a dollar a month. This is where the value is, and it is not close.

**Summarising and condensing before an expensive model reads it.** Use a long-context cheap model to shrink the input, then send the summary to your best model. `DeepSeek-V4-Flash` at $0.14 in with a million-token context and a $0.03 cached rate is built for exactly this.

**Overnight bulk work**, if and only if your model is on the six-model batch list. Otherwise use batch for the separate rate-limit pool and ignore the discount.

**What does not work well** is a customer-facing chat product where response quality is the product. You are buying open-weight models at good prices, not a frontier model with a support contract. If a wrong answer costs you a customer, pay for the frontier model and route only the cheap work here.

Together also ships an **Evaluations** product I nearly reported as absent: LLM-as-judge with three job types, classify, score and compare, runnable from the CLI, the SDKs or the web console. If you are comparing a cheap model against your incumbent, that is the harness, and it means you do not have to build one.

## Free GitHub projects for the surrounding stack

Together sells inference. Everything around it, the gateway, the observability, the chat interface, the vector store, is something you would otherwise buy a subscription for. I checked each repository below against the GitHub API on August 5, 2026, reading the newest commit on the default branch rather than the repository's `pushed_at` field, because `pushed_at` counts pushes to fork pull request branches and makes a stalled project look busy. None is archived. Thirteen of the fifteen had a commit inside about two weeks, and the one real outlier is Portkey's gateway at ten weeks. The licences are what the repository actually grants, which is not always what the README implies.

| Project | Licence | Stars | What it replaces |
|---|---|---|---|
| [LiteLLM](https://github.com/BerriAI/litellm) | MIT core, separate commercial licence for `enterprise/` | 55.6k | A paid LLM gateway. Normalises many providers behind one interface and tracks per-key spend. The generous core here is unusual. |
| [Langfuse](https://github.com/langfuse/langfuse) | MIT core, paid licence for `ee/` directories | 32.6k | Hosted LLM observability and tracing seats. An honest open-core boundary. |
| [Helicone](https://github.com/Helicone/helicone) | Apache-2.0 | 6.0k | The same category, more permissively licensed. |
| [Portkey Gateway](https://github.com/Portkey-AI/gateway) | MIT | 12.7k | A commercial AI gateway. Genuinely MIT, and now a Palo Alto Networks company, which is a governance fact worth knowing. It is also the quietest repository in this table, with no commit on the default branch since May 25. For the component that sits in the path of every model call you make, check that yourself before adopting it. |
| [LibreChat](https://github.com/danny-avila/LibreChat) | MIT | 41.7k | Per-seat AI chat subscriptions. Point it at Together and pay by the token instead of by the person. |
| [Open WebUI](https://github.com/open-webui/open-webui) | Custom, BSD-3 plus a branding clause | 148.0k | The same, and the most popular option. **Read the licence:** you may not remove its branding unless your deployment has 50 or fewer users in a rolling 30 days. A small shop sits inside that carve-out. |
| [AnythingLLM](https://github.com/Mintplex-Labs/anything-llm) | MIT | 64.4k | A packaged document-chat product. |
| [pgvector](https://github.com/pgvector/pgvector) | PostgreSQL licence | 22.5k | A managed vector database. If you already run Postgres, this is the cheapest correct answer and it needs no new service. |
| [Qdrant](https://github.com/qdrant/qdrant) | Apache-2.0 | 33.8k | A managed vector database, when you have outgrown pgvector. |
| [Chroma](https://github.com/chroma-core/chroma) | Apache-2.0 | 29.0k | The same, with the gentlest on-ramp for prototypes. |
| [Ollama](https://github.com/ollama/ollama) | MIT | 177.9k | Per-token inference entirely, for work that fits on your own hardware. |
| [vLLM](https://github.com/vllm-project/vllm) | Apache-2.0 | 88.3k | A serving platform, if you rent GPUs and want throughput. |

Three that need a licence warning before you build on them. [Dify](https://github.com/langgenius/dify) (151.5k stars) is a modified Apache licence that forbids operating it as a multi-tenant service without written permission and forbids removing its branding, so it is fine internally and hostile if you intend to resell it. [Flowise](https://github.com/FlowiseAI/Flowise) (55.2k) is Apache for the bulk, with identity management specifically behind a commercial licence, so SSO is the paid part. And [n8n](https://github.com/n8n-io/n8n) (199.5k) is **not open source at all**: its Sustainable Use Licence restricts you to internal business purposes and forbids offering it to third parties as a hosted service. It is an excellent tool. It is not something you can build a client offering on.

The honest summary: **LiteLLM plus Langfuse plus pgvector is a genuinely good small-shop stack**, and all three are permissively licensed. The chat front-ends are worth it the moment you have more than about three people who would otherwise need their own seat. The rest of the list is enterprise tooling that will consume a two-person team rather than serve it.

And self-hosting is never free. A 2 vCPU, 4 GB virtual machine to run a couple of these is about $24 a month at DigitalOcean's published price. Somebody owns the upgrades, the backups and the restore, and that somebody is you. At the point where a paid tool costs under about $20 a month, self-hosting it is usually the more expensive choice once your own time is priced honestly.

## Where Together is the wrong answer

Be direct about this, because the enthusiastic version of this article would not be.

If your task needs the strongest available reasoning, use a frontier model and pay for it. If you need a fifty-percent batch discount on a specific model, check the six-model list first, because most models do not get it. If your workload is a large stable prompt with a small variable tail, check whether your chosen model has a cached rate at all, because most do not, and a provider with reliable caching may be cheaper at a worse sticker price. If your volume is genuinely tiny, the $5 minimum and the prepaid model add friction that a postpaid provider does not. And if what you actually need is a document-retrieval product rather than an inference bill, remember that rerank lives behind a dedicated endpoint here.

The strongest reason to use Together is not any individual price. It is that the models it serves are open-weight, so the same model string runs at several other providers at comparable rates. That portability is the real product, and it is the thing a proprietary API cannot sell you.

If you want the wider argument, my book *The $20 Dollar Agency* makes the case at length that most of what small businesses get quoted four figures for is now a config change and an afternoon of attention.

## Fact-check notes and sources

All prices, limits and quotes verified against Together AI's own pages and the GitHub API on **August 5, 2026**. Every figure survived a second adversarial pass; several first-pass numbers did not and were corrected before publication, including a per-minute-versus-per-hour unit error on dedicated GPUs that would have overstated that cost sixtyfold.

- **Serverless per-token prices, context windows and cached rates**: [Together serverless models catalogue](https://docs.together.ai/docs/serverless/models), cross-checked against [together.ai/pricing](https://www.together.ai/pricing). The two catalogues carry different row counts and disagree on some cached rates; check both for anything load-bearing.
- **Prompt caching being automatic, prefix-based and best-effort, and only some models having a cached rate**: [Together inference pricing documentation](https://docs.together.ai/docs/inference/pricing).
- **The single serverless embedding model, its $0.02 rate, 1024 dimensions and 514-token context**: [Together serverless models catalogue](https://docs.together.ai/docs/serverless/models).
- **No rerank on serverless, and rerank being dedicated-only**: [Together serverless models catalogue](https://docs.together.ai/docs/serverless/models) and [rerank documentation](https://docs.together.ai/docs/inference/embeddings/rerank).
- **Dedicated Inference at $5.49 per GPU per hour and GPU Clusters at $3.99, both per hour, and Provisioned Throughput at $0.05 per unit per minute**: [together.ai/pricing](https://www.together.ai/pricing). The per-hour headers are printed above each table; the per-minute unit belongs only to Provisioned Throughput.
- **Batch discount scope, the six eligible model IDs, "Models not listed run at standard rates", and the batch limits and 24-hour best-effort window**: [Together batch inference documentation](https://docs.together.ai/docs/batch-inference).
- **Fine-tuning rates, the $4.00 per-job minimum, specialised model rates, and dedicated-endpoint serving for fine-tunes**: [together.ai/pricing](https://www.together.ai/pricing) and Together's fine-tuning documentation.
- **The $5 minimum credit purchase, fully prepaid billing, and suspension at zero balance**: [Together billing and credits documentation](https://docs.together.ai/docs/billing-credits).
- **Base URL, SDK package names and the OpenAI-compatibility caveats including namespaced model IDs returning 404 and `logit_bias` being unsupported**: [Together quickstart](https://docs.together.ai/docs/quickstart) and its OpenAI-compatibility page.
- **The Evaluations product and its classify, score and compare job types**: Together evaluations documentation.
- **Every repository licence, star count, archived status and last push**: queried directly from the GitHub REST API on 2026-08-05. Where GitHub reports a licence as unrecognised, the repository's own licence file was read instead, which is why n8n appears as the Sustainable Use Licence and Open WebUI as a custom BSD variant rather than "Other."
- **The $24 monthly figure for a 2 vCPU / 4 GB virtual machine**: DigitalOcean's published pricing.
- Cost estimates in this article are arithmetic at published rates, shown so you can check them. They are not quotes, and they exclude your own time.

Model catalogues and prices on this platform move quickly, and the documentation and pricing pages do not always agree with each other. Verify before you commit spend.

## Related reading

- [Diversify Your AI API Spend Across Providers](/blog/diversify-ai-api-spend-across-providers/): why the same open-weight model at three providers is the actual insurance policy, and why a gateway forfeits the Anthropic batch discount.
- [Mintlify's Data Page on Agent Traffic](/blog/mintlify-data-agents-reading-documentation/): the same verification approach applied to a documentation vendor, including the free repositories that replace it.
- [DeepSeek V4 versus Kimi K2.6 Open Weight Showdown](/blog/blog-deepseek-v4-kimi-k26-open-weight-showdown/): how these model families compare on capability rather than price.
- [Local AI On-Prem versus Cloud](/blog/local-ai-on-prem-vs-cloud/): the arithmetic on running models yourself instead of renting them.
- [Validate an AI Model Before You Trust It](/blog/blog-validate-ai-model-before-upgrade/): the golden-set method, for before you route real traffic to something cheaper.
- Two free browser-side tools for putting numbers on your own spend: the [Model-Tier AI Cost Calculator](/tools/model-tier-cost-calculator/) and the [LLM Retrieval Cost Estimator](/tools/llm-retrieval-cost-estimator/). Both free, no signup, no email.

---

*This post is informational, not legal, financial or procurement advice. Mentions of Together AI, DigitalOcean, GitHub and the open-source projects named above are nominative fair use. No affiliation or endorsement is implied. Open-source licence summaries are my reading of the repositories on the date stated and are not legal advice; read the licence yourself before building a business on it.*


---

Canonical HTML: https://jwatte.com/blog/together-ai-for-small-business/
RSS: https://jwatte.com/feed.xml
JSON Feed: https://jwatte.com/feed.json
Hero image: https://jwatte.com/images/together-ai-for-small-business.webp