# When the Built-in LLM Won't Reach: Running Your Own Model in the Browsers Chrome Can't

Chrome's built-in LLM is Chrome-desktop-only. WebLLM and Transformers.js run full models on the visitor's own GPU in any WebGPU browser — Firefox and Safari included. How, and the tradeoffs.

Author: J.A. Watte
Published: July 14, 2026
Source: https://jwatte.com/blog/in-browser-llm-byo-model-cross-browser/

---

In the last post I walked through the free language model now sitting inside Chrome, and I was careful about the catch: the built-in Prompt API is Chrome and Edge desktop only, hardware-gated, and half of it is still an origin trial. If you wire a feature to it and stop there, every Firefox visitor, every Safari visitor, and everyone on a phone gets nothing. That is a lot of your audience staring at a dead button.

There is a second path to on-device AI that fixes the reach problem, and it does not wait on any one browser vendor shipping a model. Instead of borrowing the browser's built-in model, you bring your own: a model file that downloads once to the visitor's machine and runs on their own GPU, in any browser that supports WebGPU. And as of this year, that is nearly all of them.

## The thing that quietly changed: WebGPU is everywhere now

The whole approach rests on one browser capability, WebGPU, which lets a web page use the visitor's graphics chip for general computation, which is exactly what running a model needs. For years it was a Chrome-only story, which is why "AI in the browser" felt like a Chrome demo. That ended.

As of January 2026, WebGPU is what the web platform people call Baseline: it ships on by default in Chrome and Edge, in Firefox on Windows and Apple-silicon macOS, and in Safari 26 on macOS, iOS, and iPadOS ([web.dev](https://web.dev/blog/webgpu-supported-major-browsers), [caniuse](https://caniuse.com/webgpu)). Coverage sits around the low-to-mid 80s as a share of global users, with the remaining gaps mostly on Linux Firefox and older mobile GPUs ([GPU for the Web implementation status](https://github.com/gpuweb/gpuweb/wiki/Implementation-Status)). That is the shift that makes bring-your-own-model a real, cross-browser strategy instead of a party trick: the visitor's own graphics chip is now a compute target you can count on for the large majority of your traffic.

## Two ways to bring your own model

There are two mature, free, open-source libraries that do the heavy lifting, and they are good at different jobs.

### WebLLM, when you want an actual chat model

[WebLLM](https://github.com/mlc-ai/web-llm) (from the MLC project) runs full chat LLMs entirely in the browser on the visitor's GPU through WebGPU, with no server in the loop. It supports the model families you would expect for local work, Llama, Qwen, Gemma, Phi, and Mistral, in the small sizes that fit on consumer hardware, and the weights download once and cache in the browser so the second visit is instant.

The detail that makes it easy to adopt: WebLLM is built to be OpenAI-API compatible. It exposes the same `chat.completions` shape, with streaming, JSON mode, and structured output, that your code may already speak ([WebLLM docs](https://github.com/mlc-ai/web-llm)). If you have written anything against the OpenAI SDK, pointing it at a local WebLLM engine is close to a drop-in change rather than a rewrite. That is the difference between "someday" and "this afternoon."

### Transformers.js, when you want the wider toolbox

[Transformers.js](https://github.com/huggingface/transformers.js) (from Hugging Face, now on version 4) runs the broader family of models directly in the browser: embeddings, classification, vision, audio, and small language models, pulled straight from the Hugging Face Hub with no server. It runs on your CPU through WebAssembly by default and on the GPU when you ask for it with a `device: 'webgpu'` option, so the same code speeds up where the hardware allows and still works where it does not ([Transformers.js docs](https://huggingface.co/docs/transformers.js)). If your need is semantic search, a classifier, or a background-removal step rather than a chatbot, this is the reach-for library, and it is the one behind several of the tools coming later in this series.

## Why this matters if you run something small

Strip away the library names and here is what changes.

* **It covers the visitors Chrome's built-in model leaves out.** A Firefox or Safari user with a normal laptop can run a real local model through WebLLM, because it rides WebGPU, not a browser-specific API. Your on-device feature stops being a Chrome-only feature.
* **The inference is still free to you.** The model runs on the visitor's GPU. Whatever you would have paid per token through an API, you pay once in bandwidth for the model file and nothing after. That is the same floor the rest of this series keeps coming back to.
* **The data still never leaves the device, and it still works offline.** Prompts and answers stay in the tab; once the weights are cached the feature runs with the connection off. Privacy by architecture, no server to leak.
* **The switching cost is low.** Because WebLLM speaks the OpenAI shape, you are not betting a rewrite on it. You can gate it behind a feature check and fall through to your existing cloud call when the visitor cannot run it.

## The honest tradeoffs

This path buys reach, but it is not free of friction, and pretending otherwise gets you a slow, broken first impression.

* **The model download is real weight.** A built-in model is already on the machine; a bring-your-own model is not. A small chat model is on the order of hundreds of megabytes to a couple of gigabytes even quantized, so the first load is a genuine download. Trigger it only when the visitor reaches for the feature, show a progress bar, and let the browser cache it so the second visit is instant.
* **Quality is small-model quality.** These are the models that fit on a laptop GPU, not the frontier model in a datacenter. They are good for summarize, classify, draft, and extract; they are not going to match a top-tier cloud model on hard reasoning or long generation. Match the tool to the job.
* **WebGPU coverage is wide but not total.** Older phones, Linux Firefox, and some locked-down setups still lack it. When the GPU is missing, Transformers.js falls back to the CPU on its own, just slower; WebLLM needs the GPU, so there you fall back to the cloud. Either way, feature-detect first.
* **Memory is the real ceiling.** The model has to fit in the GPU's memory alongside everything else. On a machine with a small or shared GPU, a model that runs on a bigger one simply will not load. Pick model sizes conservatively and test on modest hardware.

## How you actually start

The pattern is: check for a GPU, load and cache the model once, then call it with a fallback ready for everyone who cannot run it. Here is the WebLLM shape, which is the one most people want because it is a chat model with a familiar API.

```js
import { CreateMLCEngine } from '@mlc-ai/web-llm';

// 1. No WebGPU, no local model. Old phone, Linux Firefox, locked-down setup.
if (!('gpu' in navigator)) return useCloudFallback();

// 2. Download + cache the model once; it runs on the visitor's GPU after that.
const engine = await CreateMLCEngine('Llama-3.2-3B-Instruct-q4f16_1-MLC', {
  initProgressCallback: (p) => showProgress(p.text), // load / download progress
});

// 3. The call is OpenAI-shaped. Existing SDK-style code moves over almost as-is.
const reply = await engine.chat.completions.create({
  messages: [{ role: 'user', content: 'Summarize in one sentence: ' + text }],
});
console.log(reply.choices[0].message.content); // generated on the visitor's chip
```

And the Transformers.js shape, for the non-chat jobs like on-device embeddings, where the CPU fallback is automatic:

```js
import { pipeline } from '@huggingface/transformers';

// Uses the GPU when present; falls back to CPU/WASM on its own when not.
const embed = await pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2', {
  device: 'webgpu',
});
const vector = await embed('how do I get a refund', { pooling: 'mean', normalize: true });
```

The exact model identifiers and options live in each project's docs, which is where you should copy from rather than from my sketch. The one line that matters in both is the guard at the top: check for the GPU, and have a fallback for the visitors who do not have one.

If you want to see which path your own browser can take before you write anything, that is exactly what the **[Browser AI Readiness Probe](/tools/browser-ai-readiness-probe/)** reports: it tells you whether WebGPU is available and names your GPU adapter, right alongside the built-in-API check, so you can see at a glance whether the bring-your-own-model route is open on your machine. It runs entirely in your browser.

## The bottom line

Chrome's built-in model is the easy win where it works, but it works in one place. Bringing your own model over WebGPU is the version that reaches Firefox, Safari, and the rest, because it depends on a capability nearly every modern browser now ships rather than on one vendor's model. It costs you a one-time download and a small-model quality ceiling, and it hands you a free, private, offline AI feature for the large majority of your visitors. Between this and the last post, you can now cover almost everyone: the built-in model for the Chrome desktop crowd, a bring-your-own model for everybody else, and a cloud call for the shrinking remainder who can run neither.

If you are trying to launch a real web presence full of features like this without a subscription stack quietly draining the budget, that lean, build-it-yourself approach is the whole argument of my book **The $97 Launch** (search the title on Amazon Kindle). The short version lives on this blog, in the posts below.

## Related reading

- [Your Browser Has a Free LLM Now](/blog/chrome-prompt-api-free-browser-llm/): the built-in Chrome model this post is the cross-browser counterweight to.
- [You Can Now Run Real AI in the Browser for Free](/blog/on-device-ai-browser-litert-js/): the vision, audio, and embedding side of on-device AI, and the post that started this series.
- [Local AI or Cloud APIs for a Small Business? The Honest Cost Math](/blog/blog-smb-ai-local-vs-cloud/): the on-device-versus-cloud tradeoff worked out in dollars.
- [The Chips Behind Browser AI](/blog/blog-xpu-chips-ai-hardware-ecosystem/): why the GPU and NPU story is what makes any of this fast.
- [The $50-a-Month AI Stack](/blog/blog-50-month-ai-stack-smb/): building a working AI toolkit on a tiny budget, where a free browser model earns its place.

## Fact-check notes and sources

* **WebGPU is Baseline across browsers**: as of January 2026 WebGPU ships by default in Chrome, Edge, Firefox (Windows and Apple-silicon macOS), and Safari 26 (macOS, iOS, iPadOS), with remaining gaps mainly on Linux Firefox and older mobile GPUs ([web.dev](https://web.dev/blog/webgpu-supported-major-browsers), [caniuse](https://caniuse.com/webgpu), [GPUweb implementation status](https://github.com/gpuweb/gpuweb/wiki/Implementation-Status)).
* **WebLLM runs full LLMs in-browser via WebGPU, OpenAI-compatible**: WebLLM brings LLM inference into the browser with WebGPU hardware acceleration and no server, is fully compatible with the OpenAI API (streaming, JSON mode, structured output), supports Llama, Qwen, Gemma, Phi, and Mistral families, and caches downloaded weights in the browser ([WebLLM, MLC AI](https://github.com/mlc-ai/web-llm)).
* **Transformers.js runs models in-browser with a WebGPU option**: Transformers.js (version 4) runs Hugging Face models directly in the browser with no server, using ONNX Runtime, on the CPU via WebAssembly by default and on the GPU via a `device: 'webgpu'` option ([Transformers.js docs](https://huggingface.co/docs/transformers.js), [GitHub](https://github.com/huggingface/transformers.js)).
* **On-device economics**: model weights download once and are cached in the browser, after which inference runs on the visitor's own hardware with no per-use API cost and no data leaving the device (same sources above).

---

*This post is informational, not engineering or legal advice. Mentions of Google, Chrome, Mozilla, Apple, Hugging Face, MLC, Meta, and other third parties are nominative fair use. No affiliation is implied. Library versions, model availability, and browser support change quickly; verify against the current project documentation before you build.*


---

Canonical HTML: https://jwatte.com/blog/in-browser-llm-byo-model-cross-browser/
RSS: https://jwatte.com/feed.xml
JSON Feed: https://jwatte.com/feed.json
Hero image: https://jwatte.com/images/in-browser-llm-byo-model-cross-browser.webp