It's essentially a playbook written for AI

gemma-trainer's core is a SKILL.md file plus 5 ready-made Python script templates. It's not an app, and it's not a training library you're meant to call directly yourself.

SKILL.md is a blueprint written specifically for AI agents to read. It locks down the best practices for local fine-tuning — what value each parameter should take, which pitfalls to avoid — all in one place. Once your assistant has read it, it goes from "roughly knowing how fine-tuning works" to "knowing exactly which script to adjust on this machine, and which knob to turn to what setting." The 5 scripts in the repo each handle one job: one runs supervised fine-tuning, one runs preference alignment, one trains a scoring model, one validates data formatting, and one handles data distillation.

The skill isn't tied to any particular AI tool — it works with anything from Gemini to Claude, and also works with Google's own Antigravity CLI tool (agy). Setup is a one-liner: copy the gemma-trainer folder into your AI assistant's skills directory, and it immediately knows how to guide you.

What's more interesting is that the assistant doesn't just blindly execute — it also catches your mistakes based on the capability constraints written into the skill. One real example: a training request accidentally specified Gemma 4 31B for an audio task, and the assistant flagged it on the spot, pointing out that 31B only handles text and images and has no audio capability, suggesting E2B or 12B instead.

If you ask for audio fine-tuning on a model with no audio capability, the assistant will catch it and offer viable alternatives. Source: DEV / Google AI

Teach knowledge, tune preference, or train a scorer — how to choose

The skill splits fine-tuning into three methods, each aimed at a different goal. Once you're clear on what you want the model to learn, the right path becomes obvious.

DPO has one hard rule: you must complete SFT on your domain before doing DPO. Running DPO directly on a base model that's never seen your data will usually fail, or wreck the output format.

Learning rates differ by orders of magnitude across the three methods

Here's a detail that's easy to overlook but tells you a lot: the recommended learning rate (how far each step nudges the parameters) varies enormously across the three paths, and it's not simply "more teaching means bigger steps." DPO only lightly corrects style on top of what the model already knows — too big a step and it wrecks the formatting it already learned, so its learning rate has to be two hundred times smaller than SFT's. RM, by contrast, trains a brand-new scoring head from scratch, and ends up with the largest step size of the three.

Bars illustrate relative magnitude only, not a linear scale. Actual values: RM uses 1e-3, SFT uses 2e-4, DPO uses 1e-6 — DPO is two hundred times smaller than SFT.

Here's what the two data formats look like, straight from the skill's standard examples — you can model your own dataset on these.

{
  "messages": [
    {"role": "user", "content": "Tell me a joke."},
    {"role": "model", "content": "Why did the computer go to the doctor? It had a virus!"}
  ]
}

{
  "prompt": "Write a python function to compute factorial.",
  "chosen": "def factorial(n):\n    return 1 if n <= 1 else n * factorial(n - 1)",
  "rejected": "factorial is computed using recursion or loops. Just import math."
}

During training, it only learns "how to respond" — not your prompt

There's another subtlety buried in the scripts. What gets fed in is the full "user prompt plus model response" pair, but training doesn't learn from both halves equally. The script first locates where the model's response begins, and only computes loss on the response halfLoss is how the model gets penalized for wrong answers. Computing loss only on the response means the model learns "how to respond," without memorizing your prompt — the user's prompt is skipped entirely. That way the model learns "how to answer this kind of question," rather than memorizing your question itself.

No dataset on hand? Let a bigger model generate one for you

The thing fine-tuning is most often short on is data. The skill also includes a data distillation script, distill_dataset.py: the idea is to have a large "teacher" model (say, Gemma 4 31B) generate high-quality training data, then use that data to train a small "student" model (say, E2B). It offers two ways to generate data:

The teacher model can run locally through Ollama, so the whole process — from data generation to training — never has to leave your machine.

How to squeeze a large model onto your GPU

The biggest fear in local fine-tuning is running out of VRAM and having training crash outright (an out-of-memory error, or OOM). gemma-trainer relies on three things to fit large models onto consumer GPUs: LoRA, QLoRA, and a training engine called Unsloth.

The division of labor works like this: on a single GPU, default to Unsloth, which has native support for Gemma 4 and, per its own claims, cuts VRAM use by up to 70% and runs 2x faster than standard Hugging Face training; only in multi-GPU setups, or when Unsloth can't be installed, does it fall back to Hugging Face's TRL plus PEFT. Either way, it recommends enabling QLoRA (4-bit quantization)Compresses model parameters from the usual 16-bit precision down to 4 bits — roughly a quarter of the size and VRAM footprint, at the cost of a small amount of precision, which is the key to fitting Gemma variants as large as 12B or 31B onto consumer-grade VRAM.

One more pitfall to watch for: Gemma 4 natively supports context windows up to 256K, but don't get greedy with that locally — the skill recommends capping it at 2048 to 8192, or you'll blow out your VRAM easily.

What kind of machine do you actually need

With QLoRA and Unsloth enabled, the bar is lower than most people assume. Per Unsloth's own figures, fine-tuning different Gemma 4 sizes can start on a single consumer GPU, and the smallest model can even run on free Google Colab:

VRAM figures come from Unsloth's official Gemma 4 training docs, as reference values for QLoRA/LoRA fine-tuning; full fine-tuning (uncompressed) needs several times more and usually requires multiple GPUs.

A LoRA parameter recipe you can just copy

The skill hardcodes default values for the common LoRA parameters, so you can just fill them in when you're ready to train:

Once the parameters are set, running it is a single command. If you want to skip the AI assistant and drive the scripts yourself, that works too — for instance, running a supervised fine-tune on E2B with default settings:

python sft_train.py \
  --model google/gemma-4-E2B \
  --dataset your_data.json \
  --lora-r 16 --lora-alpha 32 \
  --epochs 3 --lr 2e-4

DPO and reward modeling have their own dpo_train.py and reward_train.py scripts, with the same parameter style. When Unsloth can't be installed, adding --force-hf falls back automatically to Hugging Face.

Teaching it to see and hear, then fitting it in your pocket

Gemma 4 is natively multimodal — it can already handle images and audio. Training it to see or hear takes a smaller change than you'd expect: just swap the content block's type from text to image or audio in the data format.

{
  "messages": [
    {"role": "user", "content": [
        {"type": "text", "text": "Describe this audio."},
        {"type": "audio", "url": "path/to/audio.wav"}
    ]},
    {"role": "assistant", "content": [
        {"type": "text", "text": "This is an audio file of a bird chirping."}
    ]}
  ]
}

Audio has a few quirks: it needs 16kHz mono sampling, and the scripts include a specific patch fixing an old bug where a mismatched audio data type caused training to error out, saving you from hitting it yourself. Which model sizes support image and audio fine-tuning is also spelled out clearly — audio only works on E2B, E4B, and 12B; 31B doesn't support it, which is exactly the reason the assistant flagged the earlier example.

After training, the model can travel all the way into your pocket. Using Unsloth's native export or llama.cpp, you can convert the model to a lightweight format like GGUF, then hand it off to LiteRT-LM to run on phones, the web, and IoT devices — with hardware acceleration on CPU, GPU, or NPU. "Fine-tune locally, deploy locally" becomes one continuous chain.

Hands-on: turning Gemma into an old Korean literature translator

Here's a real working example: fine-tuning Gemma 4 E2B into a translation specialist for The Tale of Hong Gildong, a classic of old Korean literature. The entire process boils down to a single sentence to the assistant.

Fine-tune Gemma 4 E2B on the dataset bebechien/HongGildongJeon.

From there, the assistant walks you through four steps: first it uses the validation script to check your data against the required format, then it picks a set of LoRA parameters that can teach the model subtle linguistic nuance without blowing out VRAM, then it kicks off training with resource-friendly defaults, and finally it evaluates the results and tunes parameters for another round based on performance. You can even have it write a custom evaluation script on the spot — say, one that checks transcription similarity. Once training finishes, the assistant hands you a full report laying out how this round performed and what to change in the next one.

Once training wraps up, the assistant produces a comprehensive report noting performance and what to improve in the next round. Source: DEV / Google AI

How do you get started? Copy the gemma-trainer folder into your AI assistant's skills directory, and it immediately knows how to guide you. The full gemma-skills repo is open source on GitHub: github.com/google-gemma/gemma-skills.

Google added gemma-trainer to its open-source repo. It's not another training app — it's a playbook written for AI assistants to read (with best practices and parameters all locked in), plus 5 ready-made scripts. You describe what you want in plain language, and the assistant picks the script, sets the parameters, and runs training.

But what exactly does the assistant do behind that one sentence? Here's a diagram that makes it clear.

Xiaohu just says "fine-tune Gemma 4 E2B on the Tale of Hong Gildong dataset," and the assistant reads the playbook and picks the path for him (teaching new knowledge, so SFT), then sets parameters, trains, and evaluates — one sentence from you, and it handles the script-picking, parameter-tuning, and error-avoiding. Three paths: teach new knowledge with SFT, tune tone with DPO, train a scorer with RM.

How does a home GPU costing a few thousand yuan manage to train a model this size? LoRA is the trick — instead of retraining the whole model, it clips on a small stack of trainable "sticky notes" beside it while the original model stays frozen; then it gets compressed to 4-bit precision (QLoRA) and handed to Unsloth, an engine purpose-built for single-GPU efficiency. Here's what that saves in VRAM (the memory on your GPU that training depends on — run out, and it crashes):

Once trained, the model can be converted to a lightweight format and run on phones and IoT devices; getting it to see images or hear audio just means swapping one field in the data from "text" to "image" or "audio" — fine-tune locally, deploy locally, one continuous chain.