Your Browser Has a Free LLM Now: What Chrome's Prompt A...

Last week I wrote about running real AI in the browser with Google's LiteRT.js, and I was careful to draw one line: that runtime is for vision, audio, and embeddings, not for a chat model. "Those live in datacenters for a reason," I said. That is still true for the frontier-scale model writing this sentence. But it is no longer the whole story, because there is a small language model already sitting inside Chrome, and any web page can now ask it questions without sending a byte to anyone's server.

It is called the Prompt API, and it runs Gemini Nano, a compact model Google ships with the browser. This is the exact piece LiteRT.js is not: a general text model you can prompt, on the visitor's own hardware, for zero per-token cost. It is genuinely useful and genuinely here. It also comes with fine print that most of the breathless posts skip, and the fine print is the difference between shipping something that works and shipping something that quietly fails for most of your visitors. So let me be precise about what this is, what it is not yet, and how to build with it honestly.

What actually shipped, and what did not

Here is the part to get right before you plan anything around it, because the word "stable" is doing a lot of uneven work in the coverage.

In a Chrome extension, the Prompt API is stable. If you build a browser extension, you can call the on-device model in released, stable Chrome today (Chrome for Developers).
On the open web, the Prompt API is an origin trial, not fully stable. A normal website can use it, but through Chrome's origin-trial mechanism: you register your site, drop in a token, and it works for your visitors while the trial runs. Google expects it to reach full stable in a later Chrome release, on the order of late 2026 into early 2027 (Chrome for Developers). Chrome 148 expanded it with image and audio input and moved the sampling controls into their own trial, but the core web API is still riding an origin trial, not shipped-and-forgotten.
The task-specific APIs are the ones that are stable on the web. Summarizer, Translator, and Language Detector have been stable for websites since Chrome 138 (built-in AI docs). If your need is "summarize this" or "translate this," you do not touch the Prompt API at all, and you do not need a trial token. That is its own post, coming next in this series.

So the honest headline is not "the browser shipped a stable free LLM." It is: "the browser has a free LLM you can build on right now, stable in extensions, in an origin trial on the web, with the narrower summarize-and-translate jobs already fully stable." That is a smaller claim than the hype and a much bigger one than the skeptics allow.

Why this matters if you run something small

Strip away the version numbers and the shape is the same argument I made for on-device vision, now applied to text.

The generation is free to you. The model runs on the visitor's chip, downloaded once to their machine, not metered to your account. Every summarize, classify, rewrite, or extract call that would tick a fraction of a cent through an API costs you nothing after the one-time model download. For a tool you want people to use a hundred times a day, that is the whole economics.
The text never leaves the device. The prompt and the answer stay in the tab. Google states no data is sent to it or any third party when the model runs (Prompt API docs). A "clean up this rough note" box or a "is this message spam" check never ships the user's words anywhere. That is not a privacy policy you have to write and defend; it is privacy by architecture.
It works offline. Once the model is on the machine, the feature runs with the connection off. No round trip, no outage to inherit.

What you can actually build with it

This is not a frontier reasoning engine, and pretending otherwise is how you end up disappointed. Gemini Nano is a small model, in the low single-digit billions of parameters, quantized to fit (Chrome AI overview). Matched to the right jobs, though, it is plenty.

Summarize and rewrite short text. Tidy a rough note, shorten a paragraph, turn a blob into bullet points, all in the tab.
Classify and route. Is this message urgent, positive, spam. Which of five buckets does this support ticket belong in. A first-pass classifier with zero API cost.
Extract structured fields. Pull a name, date, and amount out of messy pasted text, and force the shape with a JSON schema so the output is safe to use directly (more on that below).
Answer from context you hand it. Feed it a chunk of text and ask a question about that chunk. It is not a search engine over the whole web; it reasons over what you give it.
Take image and audio input. As of Chrome 148 the Prompt API accepts images and audio alongside text, so "describe this image" or "what was said in this clip" can run locally too, though audio input needs a GPU (Prompt API docs).

Notice what is missing: long-form generation, deep multi-step reasoning, anything where quality has to be frontier-grade. Those still belong in the cloud. The skill here, same as with LiteRT.js, is matching the small fast local model to the jobs it does well and routing the rest elsewhere.

The honest status, read this before you build

Three checks before you lean on this, because each one silently removes a chunk of your audience.

It is Chrome desktop only, and hardware-gated. The built-in model runs on Windows 10/11, macOS 13 and up, Linux, and Chromebook Plus, and it needs roughly 22 GB of free disk for the profile volume plus either a GPU with more than 4 GB of VRAM or 16 GB of RAM on the CPU path (hardware requirements). It does not run on Chrome for Android or iOS at all. A large share of your mobile traffic simply cannot use it.
There is no Firefox or Safari equivalent. This is a Chrome (and Chromium, so Edge) capability. Firefox and Safari visitors have no LanguageModel to call. If you want an in-browser LLM for those users, that is the bring-your-own-model path over WebGPU, which is the next post after the task-APIs one.
On the web you are in an origin trial. To run it on a live public site across your visitors, you register for the Prompt API origin trial and add the token; without it, the API is reachable only behind a browser flag on your own machine, which is fine for a prototype but not for the public (Chrome for Developers). In a Chrome extension you skip all of that.

The through-line: this is a real capability you can ship today, but it is a progressive enhancement, never a hard dependency. Feature-detect it, use it when it is there, and fall back cleanly when it is not. Which is exactly what the code does.

How you actually start

The shape is short: check if the model is available, create a session, prompt it, and always have a fallback for the visitors who cannot run it.

// 1. Is the built-in model even present in this browser?
if (!('LanguageModel' in self)) {
  // Firefox, Safari, older Chrome, or mobile. Use your cloud path.
  return useCloudFallback();
}

// 2. Can this specific device actually run it?
const status = await LanguageModel.availability();
// -> 'unavailable' | 'downloadable' | 'downloading' | 'available'
if (status === 'unavailable') return useCloudFallback();

// 3. Create a session. If the model needs downloading, show progress.
const session = await LanguageModel.create({
  monitor(m) {
    m.addEventListener('downloadprogress', (e) => {
      // e.loaded runs 0 -> 1
      showProgress(Math.round(e.loaded * 100));
    });
  },
});

// 4. Prompt it. This runs on the visitor's own chip. No network, no bill.
const summary = await session.prompt('Summarize in one sentence: ' + text);

// 5. Force a typed answer with a JSON schema when you need a safe shape.
const isUrgent = await session.prompt(message, {
  responseConstraint: { type: 'boolean' },
});

session.destroy(); // release the model when you are done

That is the whole pattern, simplified. The two lines that matter most are the two guards at the top: the 'LanguageModel' in self check and the availability() check. They are what turn "a cool Chrome demo" into "a feature that degrades gracefully for the people who cannot run it." The exact option names and the newer multimodal and structured-output details live in Google's Prompt API guide, which is where you should copy from rather than from my sketch.

Before you wire any of this in, it is worth knowing in one glance whether your own browser and machine can even run the model, and what your visitors will see. That is exactly what I built the next tool for.

Try the Browser AI Readiness Probe — it runs the real availability() checks in your browser, tells you which built-in AI APIs are present and whether the on-device model is ready, downloadable, or out of reach, and hands you a copy-paste feature-detect-and-fallback snippet to drop into your own code. Nothing is sent anywhere; it all runs in your tab.

The bottom line

For a growing set of text jobs, "call an API and pay per token" is no longer the only option. The browser itself now carries a small language model you can prompt for free, on the visitor's hardware, with their data never leaving the tab. The catch is not that it is fake; the catch is that it is Chrome-desktop-shaped and still half in an origin trial, so it is a bonus you layer on, not a floor you stand on. Build it as an enhancement, keep a fallback for everyone else, and you get a genuinely free tier of AI for the visitors who can run it.

If you are building tools or client features and trying to keep a metered AI bill from eating the margin, routing the cheap work to the visitor's own browser is a core move in that lean playbook, which is the whole argument of my book The $20 Dollar Agency (search the title on Amazon Kindle). The short version lives on this blog, in the posts below.

Fact-check notes and sources

Extensions stable, web in origin trial: the Prompt API is available in stable Chrome for extensions and, for web pages, through an origin trial rather than full stable, with broad stability expected in a later 2026/2027 release (Prompt API docs, built-in AI status).
Task APIs stable on the web since Chrome 138: Summarizer, Translator, and Language Detector are stable for websites and extensions from Chrome 138 (built-in AI docs).
Chrome 148 multimodal input: the Prompt API's implementation supports text, image, and audio inputs, with audio input requiring a GPU (Prompt API docs, New in Chrome 148).
Hardware and OS requirements: roughly 22 GB free disk on the Chrome profile volume, more than 4 GB VRAM on the GPU path or 16 GB RAM on the CPU path, on Windows 10/11, macOS 13+, Linux, and Chromebook Plus; not available on Chrome for Android or iOS (Prompt API docs).
On-device and private: the model downloads once per origin and runs locally, with no data sent to Google or any third party during inference (Prompt API docs).
Availability states and API surface: LanguageModel.availability() reports the model state, LanguageModel.create() opens a session with an optional download monitor, and session.prompt() accepts a responseConstraint JSON schema for structured output (Prompt API docs).

This post is informational, not engineering or legal advice. Mentions of Google, Chrome, Gemini Nano, and other third parties are nominative fair use. No affiliation is implied. API status and version numbers reflect Chrome's published documentation at the time of writing and change quickly; verify against the current docs before you build.

Your Browser Has a Free LLM Now: What Chrome's Prompt API Really Does to Your Per-Token Bill