Part of the extended model-selection series, alongside the Claude Code workflow, the Codex mini-series, multi-model routing, Qwen self-hosted, and Gemini CLI. This post covers the Google open-weights story.
Qwen gets most of the open-weights attention right now, and for good reason. Gemma is the alternative you should know about anyway — Google's open-weights family, released alongside (and behind the scenes, entangled with) the proprietary Gemini line. Different company, different release rhythm, different safety-tuning defaults, but the same job in your stack: a model whose weights you control.
Both are strong options. The differences are real enough to matter for some workloads. If you're picking one to install, Qwen-Coder has the broader coder-specialized variant lineup today; Gemma has Google's research heritage and cleaner integration with Google Cloud tooling. Most people who care about open weights end up running both over time.
What Gemma is
Google's open-weights LLM family, shipped parallel to the proprietary Gemini line. Google has iterated steadily since Gemma 1 launched in 2024 — by Q2 2026 the current generation is Gemma 3 or 4 depending on which variants have promoted to stable at the moment you read this. Version numbers move faster than blog posts; check Google AI's model page for the current releases.
Key attributes:
- Permissive license — redistributable, commercially usable, fine-tunable. Read the specific license on the checkpoint before committing to a product use.
- Size tiers — typically 2B, 7B, and 9B (and sometimes larger) parameter variants per generation. Smaller end runs on consumer hardware easily.
- Specializations — base models for general use, and in some generations specialized variants like Gemma-Code (coding-focused) and multimodal Gemma (image + text).
- Tuned with Google's safety stack — the default behavior leans conservative. Configurable but noticeable.
Access:
- Self-hosted via Ollama, LM Studio, vLLM, llama.cpp (same path as Qwen).
- HuggingFace hosted — quick-to-try inference API.
- Google AI Studio / Vertex AI — hosted options within Google's cloud.
When to pick Gemma over Qwen
1. Google research heritage matters for your work. Gemma shares architecture ancestry with Gemini. For research work that references Gemini in related ways — studying scaling laws, reproducing paper results, comparing open-weights behavior to proprietary-Google baselines — Gemma is the direct open analog.
2. Google Cloud is your deployment target. If your production inference runs on Vertex AI, Gemma is the natively-supported open-weights model. Qwen via Vertex is possible but less first-class than deploying Gemma directly.
3. Safety-tuned defaults align with your use case. Gemma's safety tuning is tighter than Qwen's out of the box — which can be a feature for user-facing products where refusing edge-case content is the right behavior, or a bug for research / internal tooling where you want more permissive responses.
4. Multilingual coverage on specific languages. Gemma's multilingual performance on certain European and Indic languages is often strong. Qwen's multilingual strength is similarly strong on CJK languages (Chinese, Japanese, Korean). Depending on your target language, one family leads.
5. Some specific multimodal scenarios. Gemma's multimodal variants (where released) have Google's image-understanding heritage. For visual-input tasks on open weights, Gemma's multimodal variants are often stronger than Qwen-VL.
When to pick Qwen over Gemma
1. Coder-specialized variants. Qwen-Coder is one of the strongest open-weights coder families. The Qwen 2.5-Coder releases specifically have been competitive with (and in some benchmarks exceeding) proprietary small models on code-specific tasks. Gemma's coder variants are capable but Qwen has more momentum here.
2. Broader size range. Qwen ships useful variants at 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B. Gemma tends to ship tighter generations (2B + 7B + 9B and occasionally larger). If you need a very small model (for CPU-only inference) or a very large one (72B), Qwen's range is deeper.
3. Community momentum for fine-tuning. The open-source fine-tuning community — LoRA adapters, specialized instruction-tuned variants, community benchmarks — has broader Qwen coverage today. If you plan to download community fine-tunes or contribute your own, Qwen has a more active ecosystem.
4. License friendliness for edge cases. Both are permissive, but specific use restrictions on the largest Gemma variants (check the current Google license) can bite some commercial uses. Qwen's license has its own specific terms but tends to be marginally more permissive for aggressive commercial uses. Always read the specific checkpoint's license.
Running Gemma at home
Same mechanics as self-hosting Qwen. Same hardware tiers, same quantization trade-offs, same tooling (Ollama, LM Studio, llama.cpp).
Quick Ollama walkthrough:
# Pull a Gemma model
ollama pull gemma3:9b # general-purpose
# or
ollama pull gemma3-code:7b # coder variant (if available)
# or
ollama pull gemma3:2b # small, CPU-friendly
# Run interactively
ollama run gemma3:9b
# Via OpenAI-compatible API
curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gemma3:9b",
"messages": [{"role": "user", "content": "Explain CAP theorem in three sentences."}]
}'
Swap the tag for your generation (Gemma 3 vs the latest 4 generation as they promote to stable). Same quantization considerations as Qwen: Q8 when memory allows, Q5/Q6 as the sweet-spot default, Q4 when you need to fit larger models on constrained hardware.
Which one should I actually install?
If you're adding one open-weights model to your stack for daily CLI coding work: Qwen-Coder, current generation, at the largest size your hardware tier supports. Qwen's coder specialization is the clearest open-weights win for the audience that's reading this.
If you're adding one open-weights model for general research, multilingual work, or Google Cloud deployments: Gemma, current generation, at an appropriate size. Safer default behavior and tighter Google ecosystem fit.
If you're adding both: Qwen for daily coding assistance, Gemma for general assistant work and multimodal tasks. They coexist cleanly on the same Ollama install.
Most developers who commit to running open-weights eventually settle into "Qwen-Coder for code work + something-else for general work." That something-else is often Gemma, sometimes Llama or Mistral, occasionally a domain-specific specialist.
Integrating with your CLI workflow
Same OpenAI-compatibility story as Qwen. Point your CLI tool at the Ollama endpoint, swap the model name, everything else stays the same.
# Use local Gemma for a task via Codex CLI (or any OpenAI-compatible tool)
export OPENAI_BASE_URL=http://localhost:11434/v1
export OPENAI_API_KEY=ollama
codex --model gemma3:9b "Summarize this design doc in 200 words"
The multi-model routing post covers the routing pattern in full. Gemma slots in as a substitute or complement to Qwen in any of those patterns.
Common pitfalls
Assuming Gemma and Gemini behave similarly. They don't. Same research heritage, same company, but Gemma is a trimmed-down open model and Gemini is the full proprietary system. Prompts that work well on Gemini don't automatically transfer.
Over-indexing on raw size. A well-tuned 9B Gemma often beats a poorly-tuned 32B alternative. Size matters; tuning matters more. Run real evaluations against your own task mix rather than trusting leaderboards.
Ignoring the license. Both Qwen and Gemma have permissive licenses with some specific conditions on the largest variants and some downstream-use clauses. For commercial products, read the actual license terms on the specific checkpoint. For hobbyist use, you're almost certainly fine, but knowing the terms matters if your project grows.
Treating open weights as a drop-in for frontier proprietary models. They aren't. Open weights shine on privacy, cost-at-volume, repeatability, and fine-tuning freedom. They trail on cutting-edge reasoning, agentic harness maturity, and specific capabilities (long context, best-in-class multimodal) that frontier proprietary models lead on. Match workload to model; don't expect parity everywhere.
What to try this week
- Pull current-generation Gemma via Ollama. Compare it against your current Qwen install on three tasks you do regularly. Note which wins where.
- If you have any multilingual work, test Gemma specifically on the languages you care about. Multilingual strengths are uneven across open-weights families.
- If you have Google Cloud as a deployment target, try Gemma via Vertex AI. The ecosystem integration is meaningful when you're already in Google's cloud.
- Commit to a decision after one week: Gemma in addition to Qwen, Gemma instead of Qwen, or Qwen-only. Running both indefinitely is fine; choosing is better.
Related reading
- AI Terminal Kickstart — install the base CLI stack.
- Qwen self-hosted at home — the sibling open-weights post; same hardware / tooling, different model family.
- Gemini CLI — when to use — Google's proprietary side of the same ecosystem.
- Multi-model routing — the overarching framing where Gemma fits as an open-weights option.
- Codex vs Claude Code — the proprietary comparison for context.
Fact-check notes and sources
- Google: Gemma model cards — canonical reference for current Gemma generations, variants, and license terms.
- HuggingFace: Google organization page — all public Gemma checkpoints with details per variant.
- Ollama: ollama.com/library — pull tags for current Gemma generations.
- Google Cloud: Vertex AI Gemma documentation — Vertex AI deployment patterns.
- Open LLM Leaderboard (HuggingFace) — community benchmarks comparing Gemma, Qwen, Llama, Mistral, and other open-weights families on standardized tasks.
- r/LocalLLaMA — active community discussion on current open-weights trade-offs per use case.
Informational, not engineering consulting advice. Gemma model generations, variants, licensing, and capabilities evolve with each release; verify current details on the Google AI site before committing to a deployment. Mentions of Google, Gemma, Gemini, Alibaba, Qwen, HuggingFace, Ollama, and linked publications are nominative fair use. No affiliation is implied.
