Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions data/links.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ links:
gemma-terms:
url: /resources/terms
prohibited-use:
url: /resources/prohibited_use_policy
url: /resources/prohibited-use

# --- Weights and collections -----------------------------------------------------------
hf-gemma4:
Expand Down Expand Up @@ -122,7 +122,7 @@ links:
gemmaverse:
url: https://deepmind.google/models/gemma/gemmaverse/
gemma-cookbook:
url: https://github.com/google-gemma/gemma-cookbook
url: https://github.com/google-gemma/cookbook
gemma-skills-repo:
url: https://github.com/google-gemma/gemma-skills
awesome-gemma:
Expand Down
22 changes: 17 additions & 5 deletions data/models.yml
Original file line number Diff line number Diff line change
Expand Up @@ -304,15 +304,18 @@ gemma4:
hf: google/gemma-4-E2B-it
hf_base: google/gemma-4-E2B
hf_qat: google/gemma-4-E2B-it-qat-w4a16-ct
gguf: google/gemma-4-E2B-it-qat-q4_0-gguf
gguf: ggml-org/gemma-4-E2B-it-GGUF
gguf_qat: google/gemma-4-E2B-it-qat-q4_0-gguf
gguf_file: gemma-4-E2B-it-Q4_0.gguf
gguf_mmproj: mmproj-gemma-4-E2B-it-Q8_0.gguf
ollama: gemma4:e2b
mlx: mlx-community/gemma-4-E2B-it-qat-4bit
lmstudio: google/gemma-4-e2b
keras: gemma4_instruct_2b
keras_base: gemma4_2b
keras_assistant: gemma4_instruct_2b_assistant
litert: litert-community/gemma-4-E2B-it-litert-lm
litert_file: gemma-4-E2B-it.litertlm
memory: # GB, static weights + ~20% overhead
bf16: 11.4
sfp8: 5.7
Expand All @@ -335,15 +338,18 @@ gemma4:
hf: google/gemma-4-E4B-it
hf_base: google/gemma-4-E4B
hf_qat: google/gemma-4-E4B-it-qat-w4a16-ct
gguf: google/gemma-4-E4B-it-qat-q4_0-gguf

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this change?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This value is being used by llama.cpp page which is originally from https://ai.google.dev/gemma/docs/integrations/llamacpp and it was mixed up with our QAT weights -> https://gemma.ai/deployment/llamacpp/#model-repositories
I don't have any strong preference, but it makes sense to me to show their official gguf while showing the sample code.

gguf: ggml-org/gemma-4-E4B-it-GGUF
gguf_qat: google/gemma-4-E4B-it-qat-q4_0-gguf
gguf_file: gemma-4-E4B-it-Q4_0.gguf
gguf_mmproj: mmproj-gemma-4-E4B-it-Q8_0.gguf
ollama: gemma4:e4b
mlx: mlx-community/gemma-4-E4B-it-qat-4bit
lmstudio: google/gemma-4-e4b
keras: gemma4_instruct_4b
keras_base: gemma4_4b
keras_assistant: gemma4_instruct_4b_assistant
litert: litert-community/gemma-4-E4B-it-litert-lm
litert_file: gemma-4-E4B-it.litertlm
memory:
bf16: 17.9
sfp8: 8.9
Expand All @@ -366,8 +372,10 @@ gemma4:
hf: google/gemma-4-12B-it
hf_base: google/gemma-4-12B
hf_qat: google/gemma-4-12B-it-qat-w4a16-ct
gguf: google/gemma-4-12B-it-qat-q4_0-gguf
gguf: ggml-org/gemma-4-12B-it-GGUF
gguf_qat: google/gemma-4-12B-it-qat-q4_0-gguf
gguf_file: gemma-4-12B-it-Q4_0.gguf
gguf_mmproj: mmproj-gemma-4-12B-it-Q8_0.gguf
ollama: gemma4:12b
mlx: mlx-community/gemma-4-12B-it-qat-4bit
lmstudio: google/gemma-4-12b
Expand All @@ -391,8 +399,10 @@ gemma4:
ids: # verified repo/tag strings; see PLAN.md
hf: google/gemma-4-26B-A4B-it
hf_base: google/gemma-4-26B-A4B
gguf: google/gemma-4-26B-A4B-it-qat-q4_0-gguf
gguf: ggml-org/gemma-4-26B-A4B-it-GGUF
gguf_qat: google/gemma-4-26B-A4B-it-qat-q4_0-gguf
gguf_file: gemma-4-26B-A4B-it-Q4_0.gguf
gguf_mmproj: mmproj-gemma-4-26B-A4B-it-Q8_0.gguf
ollama: gemma4:26b
mlx: mlx-community/gemma-4-26B-A4B-it-qat-4bit
lmstudio: google/gemma-4-26b-a4b
Expand Down Expand Up @@ -420,8 +430,10 @@ gemma4:
hf: google/gemma-4-31B-it
hf_base: google/gemma-4-31B
hf_qat: google/gemma-4-31B-it-qat-w4a16-ct
gguf: google/gemma-4-31B-it-qat-q4_0-gguf
gguf: ggml-org/gemma-4-31B-it-GGUF
gguf_qat: google/gemma-4-31B-it-qat-q4_0-gguf
gguf_file: gemma-4-31B-it-Q4_0.gguf
gguf_mmproj: mmproj-gemma-4-31B-it-Q8_0.gguf
ollama: gemma4:31b
mlx: mlx-community/gemma-4-31B-it-qat-4bit
lmstudio: google/gemma-4-31b
Expand Down
6 changes: 4 additions & 2 deletions docs/deployment/huggingface.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ colab: https://colab.research.google.com/github/google-gemma/docs/blob/main/docs
Generating text, summarizing, and analysing content are just some of the tasks you can accomplish with Gemma open models. This tutorial shows you how to get started running Gemma using Hugging Face Transformers using both text and image input to generate text content. The Transformers Python library provides a API for accessing pre-trained generative AI models, including Gemma. For more information, see the [Transformers](https://huggingface.co/docs/transformers/en/index) documentation.

!!! Note
Gemma 3 and later models support input of both images and text. Earlier versions of Gemma only support text input, except for some variants, including [PaliGemma](../get-started/index.md).
Gemma 3 and later models support input of both images and text. Earlier versions of Gemma only support text input, except for some variants, including [PaliGemma](../models/paligemma/index.md).


### Install Python packages
Expand Down Expand Up @@ -143,6 +143,7 @@ You can include multiple images in your prompt by including additional `"type":
Do not use `<|image|>`{.token-multi}, `<start_of_image>`{.token-multi} or `<image_soft_token>`{.token-multi} tokens in text portion of a prompt template as this approach creates redundant tokens and processing errors.


<!-- notebook:requires-start audio -->
## Generate text from audio data

With [Gemma 4](../models/gemma-4/index.md) and [Gemma 3n](../models/gemma-3n/index.md), you can use audio data as part of your prompt. This section shows how to use the Transformers library to load and configure a Gemma model to use audio data and text input to generate text output.
Expand All @@ -151,7 +152,7 @@ With [Gemma 4](../models/gemma-4/index.md) and [Gemma 3n](../models/gemma-3n/ind

When generating content with audio, use a prompt template to structure your request. A prompt template allows you to specify input from specific roles, such as `user`{.token-role} or `model`{.token-role}, and is a required format for managing multi-turn chat interactions with Gemma models. The following example code shows how to constuct a prompt template for Gemma with audio data input:

```python { .notebook-cell }
```python { .notebook-cell data-requires="audio" }
from transformers import GenerationConfig
config = GenerationConfig.from_pretrained(MODEL_ID)
config.max_new_tokens = 512
Expand Down Expand Up @@ -183,6 +184,7 @@ You can include multiple audio files in your prompt by including additional `"ty
!!! Note
Do not use `<|audio|>`{.token-multi} or `<audio_soft_token>`{.token-multi} tokens in text portion of a prompt template as this approach creates redundant tokens and processing errors.

<!-- notebook:requires-end -->

## Next steps

Expand Down
2 changes: 1 addition & 1 deletion docs/deployment/keras.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ colab: https://colab.research.google.com/github/google-gemma/docs/blob/main/docs

Generating content, summarizing, and analysing content are just some of the tasks you can accomplish with Gemma open models. This tutorial shows you how to get started running Gemma using Keras, including generating text content with text and image input. [Keras](https://keras.io/) provides implementations for running Gemma and other models using JAX, PyTorch, and TensorFlow. If you're new to Keras, you might want to read [Getting started with Keras](https://keras.io/getting_started/) before you begin.

Gemma 3 and later models support text and image input. Earlier versions of Gemma only support text input, except for some variants, including [PaliGemma](../get-started/index.md).
Gemma 3 and later models support text and image input. Earlier versions of Gemma only support text input, except for some variants, including [PaliGemma](../models/paligemma/index.md).
!!! Note
This notebook won't run on T4 if you try with Gemma 4. Use L4 or higher GPUs.

Expand Down
36 changes: 12 additions & 24 deletions docs/deployment/llamacpp.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,41 +13,29 @@ most capable local runtime for Gemma 4 — **image, audio and video input all wo

## Install

=== "macOS / Linux"
=== "curl"

```bash
brew install llama.cpp
curl -LsSf https://llama.app/install.sh | sh
```

=== "Windows"
=== "powershell"

```bash
winget install llama.cpp
```

=== "conda"

```bash
conda install -c conda-forge llama.cpp
```

=== "Nix"

```bash
nix profile install nixpkgs#llama-cpp
```powershell
irm https://llama.app/install.ps1 | iex
```

## Run

```bash { .model-swap }
llama-cli -hf ggml-org/gemma-4-E2B-it-GGUF --prompt "Write a poem about the Kraken."
llama cli -hf ggml-org/gemma-4-E2B-it-GGUF --prompt "Write a poem about the Kraken."
```

`-hf` downloads the weights on first use, including the multimodal projector. With a system
prompt:

```bash { .model-swap }
llama-cli -hf ggml-org/gemma-4-E2B-it-GGUF -sys "You are Hong Gildong." -p "Who are you?"
llama cli -hf ggml-org/gemma-4-E2B-it-GGUF -sys "You are Hong Gildong." -p "Who are you?"
```

## Model repositories
Expand All @@ -64,21 +52,21 @@ For 4-bit, prefer Google's [QAT](../get-started/qat.md) weights — trained with
than compressed afterwards:

```bash { .model-swap }
llama-cli -hf google/gemma-4-12B-it-qat-q4_0-gguf -p "Explain PLE in one paragraph"
llama cli -hf google/gemma-4-E2B-it-qat-q4_0-gguf -p "Explain gravity in one paragraph"
```

Available as `google/gemma-4-{E2B,E4B,12B,26B-A4B,31B}-it-qat-q4_0-gguf`.

## Multimodal

```bash { .model-swap }
llama-mtmd-cli -hf ggml-org/gemma-4-E2B-it-GGUF
llama cli -hf ggml-org/gemma-4-E2B-it-GGUF --image "image.png" -p "Describe this image."
```

With local files, pass the projector explicitly:

```bash
llama-server -m gemma-4-12B-it-Q4_K_M.gguf --mmproj mmproj-gemma-4-12B-it.gguf
```bash { .model-swap }
llama cli -m gemma-4-E2B-it-Q4_0.gguf --mm mmproj-gemma-4-E2B-it-Q8_0.gguf
```

!!! note
Expand All @@ -88,7 +76,7 @@ llama-server -m gemma-4-12B-it-Q4_K_M.gguf --mmproj mmproj-gemma-4-12B-it.gguf
## Server

```bash { .model-swap }
llama-server -hf ggml-org/gemma-4-E2B-it-GGUF
llama server -hf ggml-org/gemma-4-E2B-it-GGUF
```

Browser UI at `http://localhost:8080`, OpenAI-compatible API at `http://localhost:8080/v1`.
Expand Down
41 changes: 19 additions & 22 deletions docs/deployment/unsloth.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,62 +86,59 @@ and both GGUF and MLX files are supported.

## llama.cpp

Build `llama.cpp` first — set `-DGGML_CUDA=OFF` for CPU-only machines and for Apple silicon,
where Metal is on by default:
A few options to get `llama.cpp` installed on your machine:

```bash
git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build \
-DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first \
--target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp
```
=== "curl"

```bash
curl -LsSf https://llama.app/install.sh | sh
```

=== "powershell"

```powershell
irm https://llama.app/install.ps1 | iex
```

Then pull and run a model straight from Hugging Face. There is no need to set a context length —
llama.cpp uses exactly what is required.

=== "E2B"

```bash
export LLAMA_CACHE="unsloth/gemma-4-E2B-it-GGUF"
./llama.cpp/llama-cli \
llama cli \
-hf unsloth/gemma-4-E2B-it-GGUF:Q8_0 \
--temp 1.0 --top-p 0.95 --top-k 64
```

=== "E4B"

```bash
export LLAMA_CACHE="unsloth/gemma-4-E4B-it-GGUF"
./llama.cpp/llama-cli \
llama cli \
-hf unsloth/gemma-4-E4B-it-GGUF:Q8_0 \
--temp 1.0 --top-p 0.95 --top-k 64
```

=== "12B"

```bash
export LLAMA_CACHE="unsloth/gemma-4-12B-it-GGUF"
./llama.cpp/llama-cli \
llama cli \
-hf unsloth/gemma-4-12b-it-GGUF:UD-Q4_K_XL \
--temp 1.0 --top-p 0.95 --top-k 64
```

=== "26B A4B"

```bash
export LLAMA_CACHE="unsloth/gemma-4-26B-A4B-it-GGUF"
./llama.cpp/llama-cli \
llama cli \
-hf unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_XL \
--temp 1.0 --top-p 0.95 --top-k 64
```

=== "31B"

```bash
export LLAMA_CACHE="unsloth/gemma-4-31B-it-GGUF"
./llama.cpp/llama-cli \
llama cli \
-hf unsloth/gemma-4-31B-it-GGUF:UD-Q4_K_XL \
--temp 1.0 --top-p 0.95 --top-k 64
```
Expand All @@ -158,7 +155,7 @@ hf download unsloth/gemma-4-26B-A4B-it-GGUF \
```

```bash
./llama.cpp/llama-cli \
llama cli \
--model unsloth/gemma-4-26B-A4B-it-GGUF/gemma-4-26B-A4B-it-UD-Q4_K_XL.gguf \
--mmproj unsloth/gemma-4-26B-A4B-it-GGUF/mmproj-BF16.gguf \
--temp 1.0 --top-p 0.95 --top-k 64
Expand All @@ -167,7 +164,7 @@ hf download unsloth/gemma-4-26B-A4B-it-GGUF \
### Server

```bash
./llama.cpp/llama-server \
llama server \
--model unsloth/gemma-4-26B-A4B-it-GGUF/gemma-4-26B-A4B-it-UD-Q4_K_XL.gguf \
--mmproj unsloth/gemma-4-26B-A4B-it-GGUF/mmproj-BF16.gguf \
--temp 1.0 --top-p 0.95 --top-k 64 \
Expand Down
4 changes: 2 additions & 2 deletions docs/get-started/capabilities/function-calling.md
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ if calls:
!!! Note
For optimal results, append the tool execution result to your message history using the specific format below. This ensures the chat template correctly generates the required token structure (e.g., `response:get_current_weather{temperature:15,weather:<|"|>sunny<|"|>}`).

```python { .notebook-cell }
```python
"tool_responses": [
{
"name": function_name,
Expand All @@ -275,7 +275,7 @@ if calls:

In case of multiple independent requests:

```python { .notebook-cell }
```python
"tool_responses": [
{
"name": function_name_1,
Expand Down
14 changes: 7 additions & 7 deletions docs/get-started/capabilities/image.md
Original file line number Diff line number Diff line change
Expand Up @@ -320,16 +320,16 @@ output = vqa_pipe(messages, return_full_text=False, generate_kwargs=gen_kwargs)
print(output[0]['generated_text'])

draw_results(output[0]['generated_text'])
```
```

```{ .no-copy .output }
```` { .no-copy .output }
```json
[
{"box_2d": [243, 252, 956, 415], "label": "person"},
{"box_2d": [356, 606, 654, 802], "label": "cat"}
]
```<turn|>
```
````

![png](../../assets/capabilities/output_b041486b_1.png)

Expand Down Expand Up @@ -409,14 +409,14 @@ print(output[0]['generated_text'])
draw_results(output[0]['generated_text'])
```

```{ .no-copy .output }
```` { .no-copy .output }
```json
[
{"box_2d": [454, 871, 533, 923], "label": "person"},
{"box_2d": [551, 316, 665, 628], "label": "car"}
]
```<turn|>
```
````

![png](../../assets/capabilities/output_bc2920e6_1.png)

Expand Down Expand Up @@ -506,7 +506,7 @@ plt.tight_layout()
plt.show()
```

```{ .no-copy .output }
```` { .no-copy .output }
Budget Size: 70
# of Image Tokens: 64
[transformers] You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
Expand Down Expand Up @@ -551,7 +551,7 @@ Budget Size: 560
{"box_2d": [404, 867, 520, 925], "label": "person"}
]
```<turn|>
```
````

![png](../../assets/capabilities/output_76f17518_3.png)

Expand Down
Loading
Loading