Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
a80a729
feat(image): prompt upsampling for flux.2 klein, krea 2 and qwen-imag…
asher Oct 17, 2026
0fa045c
feat(pull): qwen-image pulls fetch qwen's rewriter prompt and name it…
asher Oct 17, 2026
546764b
fix(image): upsampler replies keep the think tag the prompt opens, an…
asher Oct 17, 2026
22a4ae5
test(e2e): the upsample mode runs flux.2 klein beside flux.3 klein
asher Oct 17, 2026
f44b972
docs(image): prompt upsampling per family, upsample_prompt and the qw…
asher Oct 17, 2026
663c196
docs(image-api): the errors of a reply without a caption and a missin…
asher Oct 17, 2026
584fe53
feat(image): z-image prompt upsampling with the z-image-turbo space's…
asher Oct 17, 2026
34eb49c
fix(image): qwen-image rewritten_prompt with unescaped quotes, and a …
asher Oct 17, 2026
9b132e2
fix(image): krea 2 caption is the last paragraph of the reply
asher Oct 17, 2026
df26049
fix(image): upsampler replies skip a thinking block in every known th…
asher Oct 17, 2026
c860c9d
fix(image): pull fetches an upsampler prompt of another digest again …
asher Oct 17, 2026
31a213e
fix(image): qwen-image upsampler turn without a presence penalty
asher Oct 17, 2026
78e2a2b
fix(image): qwen-image reply with a json object but no rewritten_prom…
asher Oct 17, 2026
958123b
fix(image): pull leaves an upsampler prompt of another digest alone a…
asher Oct 17, 2026
6109357
fix(image): upsampler replies drop their reasoning as the chat termin…
asher Oct 17, 2026
4fb91a1
fix(image): loose rewritten_prompt read joins surrogate pairs, and kr…
asher Oct 17, 2026
c2c526c
docs(image): upsampler reply reading, the pull's handling of a prompt…
asher Oct 17, 2026
2e93e91
test(image): family_prices fixture recipes carry the prompt fields
asher Oct 17, 2026
b96c4a2
fix(image): upsampler reply that ends in reasoning has no caption, at…
asher Oct 17, 2026
0ca4075
docs(image): qwen-image replies without a brace, entries that keep na…
asher Oct 17, 2026
3ebb832
fix(sampling): presence and frequency penalties count the whole outpu…
asher Oct 17, 2026
c76ad08
fix(sampling): penalties take committed tokens from the verify rounds…
asher Oct 18, 2026
cd7d9ea
fix(serve): a tick-guard rebuild seeds fresh penalties with the deliv…
asher Oct 18, 2026
a7d0e13
fix(vendor): whole-output defaults in qwen3 omni and gmlx markers at …
asher Oct 18, 2026
8a7e7d8
docs(changelog): presence and frequency penalties under changed
asher Oct 18, 2026
9384666
feat(image): flux.3 and qwen-image upsamplers decode with their sourc…
asher Oct 18, 2026
dfa4f47
docs(image): the flux.3 upsample row names where its presence penalty…
asher Oct 18, 2026
c72e3b7
fix(spec): row processors anchor and commit only an output penalty, w…
asher Oct 18, 2026
f46c7b0
test(conftest): tests on local model files see this mac, live, not th…
asher Oct 18, 2026
2e4817f
fix(distill): gen writes the replies that land in one wake in prompt …
asher Oct 18, 2026
fc08ffc
test(serve): the build lock test clears the config with no models it …
asher Oct 18, 2026
b52a5b6
docs(changelog): distill gen keeps a reply that arrived before ctrl-c
asher Oct 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

- FLUX.3 klein rewrites each prompt with a chat model of your choice before
encoding it, set as the entry's `upsampler` or by `--upsampler`.
- FLUX.2 klein, Krea 2, Z-Image and Qwen-Image 2.1 take an `upsampler` too,
and `upsample_prompt` sets your own system prompt for any family.
- `gmlx pull`, `image generate`, `serve` and the ComfyUI pack run FLUX.3 klein
and FLUX.3 klein Base GGUFs, with edits from up to 10 reference images.
- `gmlx pull`, `image generate`, `serve` and the ComfyUI pack run FLUX.2 klein
Expand Down Expand Up @@ -40,9 +42,13 @@ adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
prefills faster. `GMLX_ARENA_SPLIT_MAX_SHARE` tunes the choice.
- `gmlx pull` of an image model reuses a VAE GGUF on disk packed for its
family from another source, and names it, where it downloaded the VAE again.
- `presence_penalty` and `frequency_penalty` count every generated token, as
vLLM does, where they counted the last 20 tokens, the prompt's included.

### Fixed

- `gmlx distill gen` stopped with Ctrl-C could leave out a reply that had
already arrived.
- A chat template with a `{% generation %}` block was refused by
`--chat-template` as invalid Jinja.
- `--thinking off` was silently ignored on a model with no thinking switch
Expand Down
4 changes: 3 additions & 1 deletion THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ in [`licenses/`](licenses/).
| [mflux](https://github.com/filipstrand/mflux) | MIT | The Z-Image transformer at commit `e066d98`, ported to `gmlx/image/families/z_image/model.py` with the fused qkv kept and modules named after the GGUF tensors, and its attention, timestep and RoPE helpers in `gmlx/image/ops/attention.py` and `gmlx/image/ops/embeds.py`. |
| [diffusers](https://github.com/huggingface/diffusers) | Apache-2.0 | Code at commit `e0abab8`, ported to MLX and modified for gmlx: the Z-Image forward of `transformer_z_image.py` in `gmlx/image/families/z_image/model.py`, its timestep embedding, RoPE tables and rotary product in `gmlx/image/ops/embeds.py` and `gmlx/image/ops/attention.py`, the RMSNorm rounding of `normalization.py` in `gmlx/image/ops/norms.py`, and the FLUX.1 autoencoder of `autoencoders/vae.py`, `resnet.py`, `unet_2d_blocks.py`, `downsampling.py`, `upsampling.py` and `attention_processor.py` in `gmlx/image/vae/flux1_ae.py`, the quant convs and BatchNorm statistics of `autoencoders/autoencoder_kl_flux2.py` in `gmlx/image/vae/flux2_ae.py`, the Qwen-Image 2.1 autoencoder of `autoencoders/autoencoder_kl_qwenimage21.py`, for one frame, in `gmlx/image/vae/qwen_image21_vae.py`, and the UniPC bh2 sampler of `schedulers/scheduling_unipc_multistep.py` in `gmlx/image/unipc.py`. Also the Qwen-Image 2.1 forward, token layout and RoPE of `transformer_qwenimage21.py` in `gmlx/image/families/qwen_image21/model.py` and `layout.py`, and the dynamic shift of `pipelines/qwenimage21/pipeline_qwenimage21.py` and `schedulers/scheduling_flow_match_euler_discrete.py` in `gmlx/image/schedule.py`, and the reference image sizes, the prompt template with image slots and the image branch of the encoder tap of that pipeline in `gmlx/image/families/qwen_image21/edit.py`. For Krea 2, at the same commit: the forward of `transformer_krea2.py` in `gmlx/image/families/krea2/model.py`, the Qwen-Image autoencoder of `autoencoders/autoencoder_kl_qwenimage.py`, for one frame, in `gmlx/image/vae/qwen_image_vae.py`, the prompt template, encoder layers and constants of `pipelines/krea2/pipeline_krea2.py` and `modular_pipelines/krea2/` in `gmlx/image/families/krea2/family.py`, and the schedule of that pipeline in `krea2_sigmas` of `gmlx/image/schedule.py`. |
| [flux-3-image-klein](https://github.com/black-forest-labs/flux-3-image-klein) | Apache-2.0 | BFL's FLUX.3 klein reference at commit `45e57c9`, ported to MLX and modified for gmlx: the DiT constants and tensor layout of `src/flux3/dit.py`, the image RoPE scale and schedule alpha of `src/flux3/pipeline.py` and the encoder layers, pad multiple, length, pad token and position and chat template of `src/flux3/text_encoder.py` in `gmlx/image/families/flux3_klein/family.py`, the DiT forward of `src/flux3/dit.py` with the position ids of `pipeline.py`, `autoencoder.py` and `text_encoder.py` in `gmlx/image/families/flux3_klein/model.py`, the schedule of `src/flux3/sampling.py` in `flux3_sigmas` of `gmlx/image/schedule.py`, the latent patchify and BatchNorm of `src/flux3/autoencoder.py` in `gmlx/image/vae/flux2_ae.py`, and the reference canvas, reference preparation and default size of `src/flux3/pipeline.py` with the reference choice and output crop of `src/flux3/generate.py` in `gmlx/image/families/flux3_klein/edit.py`; and at commit `bfc8889`, the upsampler's system prompts `src/flux3/pup_prompts_external_vlm.json`, shipped unmodified in `gmlx/image/families/flux3_klein/`, with the chat turn and sampling of `src/flux3/pup.py` in `gmlx/image/families/flux3_klein/family.py`. |
| [flux2](https://github.com/black-forest-labs/flux2) | Apache-2.0 | BFL's FLUX.2 reference at commit `50fe516`, ported to MLX and modified for gmlx: the DiT constants of `Klein4BParams`, `Klein9BParams` and `Flux2Params` and the tensor names and shapes of `Flux2` in `src/flux2/model.py`, the sampling defaults of `FLUX2_MODEL_INFO` in `src/flux2/util.py`, the prompt rules of `Qwen3Embedder` in `src/flux2/text_encoder.py`, the rounding of `denoise` and `denoise_cfg` and the reference image rules of `encode_image_refs` in `src/flux2/sampling.py` in `gmlx/image/families/flux2_klein/family.py`, the constants of `Klein9BParams`, the 9B rows of `FLUX2_MODEL_INFO` and the Qwen3-8B encoder of `load_qwen3_embedder` in `src/flux2/text_encoder.py` in `gmlx/image/families/flux2_klein_9b/family.py`, the `flux.2-klein-9b-kv` row of `FLUX2_MODEL_INFO` in `gmlx/image/families/flux2_klein_9b_kv/family.py` and the step of `denoise_cached` in `src/flux2/sampling.py` in `gmlx/image/families/flux2_klein_9b_kv/step.py`, the DiT forward of `src/flux2/model.py`, with `forward_kv_extract`, `forward_kv_cached` and `causal_attn_fn`, and the token ids of `prc_img`, `prc_txt` and `encode_image_refs` in `src/flux2/sampling.py` in `gmlx/image/families/flux2_klein/model.py`, and the schedule of `compute_empirical_mu` and `get_schedule` in `src/flux2/sampling.py` in `flux2_mu` and `flux2_sigmas` of `gmlx/image/schedule.py`, and the reference image preparation of `encode_image_refs` and `default_prep` in `src/flux2/sampling.py` in `gmlx/image/families/flux2_klein/edit.py`. |
| [flux2](https://github.com/black-forest-labs/flux2) | Apache-2.0 | BFL's FLUX.2 reference at commit `50fe516`, ported to MLX and modified for gmlx: the DiT constants of `Klein4BParams`, `Klein9BParams` and `Flux2Params` and the tensor names and shapes of `Flux2` in `src/flux2/model.py`, the sampling defaults of `FLUX2_MODEL_INFO` in `src/flux2/util.py`, the prompt rules of `Qwen3Embedder` in `src/flux2/text_encoder.py`, the rounding of `denoise` and `denoise_cfg` and the reference image rules of `encode_image_refs` in `src/flux2/sampling.py` in `gmlx/image/families/flux2_klein/family.py`, the constants of `Klein9BParams`, the 9B rows of `FLUX2_MODEL_INFO` and the Qwen3-8B encoder of `load_qwen3_embedder` in `src/flux2/text_encoder.py` in `gmlx/image/families/flux2_klein_9b/family.py`, the `flux.2-klein-9b-kv` row of `FLUX2_MODEL_INFO` in `gmlx/image/families/flux2_klein_9b_kv/family.py` and the step of `denoise_cached` in `src/flux2/sampling.py` in `gmlx/image/families/flux2_klein_9b_kv/step.py`, the DiT forward of `src/flux2/model.py`, with `forward_kv_extract`, `forward_kv_cached` and `causal_attn_fn`, and the token ids of `prc_img`, `prc_txt` and `encode_image_refs` in `src/flux2/sampling.py` in `gmlx/image/families/flux2_klein/model.py`, and the schedule of `compute_empirical_mu` and `get_schedule` in `src/flux2/sampling.py` in `flux2_mu` and `flux2_sigmas` of `gmlx/image/schedule.py`, and the reference image preparation of `encode_image_refs` and `default_prep` in `src/flux2/sampling.py` in `gmlx/image/families/flux2_klein/edit.py`; and the upsampler's text to image system message `SYSTEM_MESSAGE_UPSAMPLING_T2I` of `src/flux2/system_messages.py`, shipped unmodified as `gmlx/image/families/flux2_klein/system_message_upsampling_t2i.txt`, with the chat turn of `scripts/cli.py` in `gmlx/image/families/flux2_klein/family.py`. |
| [Z-Image-Turbo Space](https://huggingface.co/spaces/Tongyi-MAI/Z-Image-Turbo) | apache-2.0, from the Space's card; `pe.py` carries no license header | At revision `768cb50`, the prompt enhancer's system prompt `prompt_template` of `pe.py`, shipped as its string in `gmlx/image/families/z_image/pe_prompt_template.txt`, with its use in `app.py` in `gmlx/image/families/z_image/family.py`. |
| [krea-2](https://github.com/krea-ai/krea-2) | Apache-2.0 | At commit `db3984f`, the prompt expansion system prompt `docs/expansion.txt`, shipped unmodified in `gmlx/image/families/krea2/`, with its use of `docs/prompting.md` in `gmlx/image/families/krea2/family.py`. The Krea 2 weights are under their own license and do not ship with gmlx. |
| [ComfyUI-GGUF](https://github.com/city96/ComfyUI-GGUF) | Apache-2.0 | The model detection keys of `convert.py` at commit `6ea2651`, which `gmlx/image/fingerprint.py` uses to refuse other diffusion model layouts by name. |
| [misaki](https://github.com/hexgrad/misaki) | Apache-2.0 | Whole package vendored at `gmlx/_vendor/misaki` (commit `fba1236`): Kokoro's grapheme-to-phoneme front-end, used by the TTS service until upstream's next PyPI release. |
| [musl](https://musl.libc.org) | MIT | Statically linked into the guest entry binary `gmlx/container/guest/gmlx-entry` as its C library ([`licenses/musl-COPYRIGHT`](licenses/musl-COPYRIGHT)). |
Expand Down
6 changes: 5 additions & 1 deletion docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,11 @@ and their `*_context_size` companions, `enable_thinking`,
`thinking_budget`, and the OpenAI `reasoning` and `reasoning_effort`
controls. The three chat routes ignore `n`, `user`, `parallel_tool_calls`
and `metadata`: they return one choice, and the template decides how many
tool calls to make. The other fields differ by route:
tool calls to make. `presence_penalty` and `frequency_penalty` count every
token generated so far and never the prompt, as vLLM and SGLang do.
`presence_context_size` or `frequency_context_size` set to N counts only the
last N generated tokens, and 0 or no value counts all of them. The other
fields differ by route:

| Parameter | `/v1/chat/completions` | `/v1/responses` | `/v1/messages` | Notes |
|-----------|------------------------|-----------------|----------------|-------|
Expand Down
3 changes: 2 additions & 1 deletion docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -1068,8 +1068,9 @@ and `crop`, the left, top, right and bottom of the image in it.
| `--prompt TEXT` | Required, or `--prompt-file` | Generate from this prompt. |
| `--prompt-file FILE` | None | Read the prompt from this file. |
| `--negative-prompt TEXT` | Empty | Guide away from this prompt, or from the empty prompt without it. It is unused when `--guidance` is 1. |
| `--upsampler GGUF` | The entry's `upsampler` | On FLUX.3 klein, rewrite the prompt with this chat model, a GGUF or a `models` id, as [Prompt upsampling](images.md#prompt-upsampling) describes. |
| `--upsampler GGUF` | The entry's `upsampler` | Rewrite the prompt with this chat model, a GGUF or a `models` id, as [Prompt upsampling](images.md#prompt-upsampling) describes. |
| `--no-upsample` | Off | Run the prompt as typed, without the upsampler. |
| `--upsample-prompt FILE` | The entry's `upsample_prompt` | The upsampler takes the system prompt in this text file in place of the family's. Qwen-Image 2.1 ships none and needs one. |
| `--size WxH` | The init image's, else a FLUX.2 klein or FLUX.3 klein reference's, else the entry's `defaults.size`, else the variant's | Set the width and height, each a multiple of 32 on Qwen-Image 2.1 and of 16 on the others. Another size exits 2 and names the multiple. |
| `--steps N` | The variant's | Sample for N steps. |
| `--seed N` | Random, and printed | Seed the first image. |
Expand Down
3 changes: 2 additions & 1 deletion docs/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,8 @@ entries.
| <a id="image_modelsfamily"></a>`family` | detected | The image family, such as `z-image`, `qwen-image-2.1` or `flux-3-klein`. A file of another family is refused at load. |
| <a id="image_modelsvariant"></a>`variant` | detected | The DiT's variant, such as `turbo`, for a file whose weights match no known variant. A Krea 2 file of unknown weights needs `variant: turbo` to load. |
| <a id="image_modelsadapters"></a>`adapters` | none | LoRA files applied to every request, each a path or `{path, strength}` with strength 1 by default |
| <a id="image_modelsupsampler"></a>`upsampler` | none | The `models` id of the chat model that rewrites each FLUX.3 klein prompt, as [Prompt upsampling](images.md#prompt-upsampling) describes |
| <a id="image_modelsupsampler"></a>`upsampler` | none | The `models` id of the chat model that rewrites each prompt, as [Prompt upsampling](images.md#prompt-upsampling) describes |
| <a id="image_modelsupsample_prompt"></a>`upsample_prompt` | the family's | A text file with the system prompt for the `upsampler`, found in `model_dirs` when relative. Qwen-Image 2.1 needs one, which `gmlx pull qwen-image` fetches |
| <a id="image_modelsupsample"></a>`upsample` | `true` | Rewrite prompts with the `upsampler`. `false` runs them as typed, unless a request sends `prompt_upsampling: true`. |
| <a id="image_modelsdefaults"></a>`defaults` | the family's | Request defaults: `size` as `WxH`, `steps`, `guidance`, `sampler` (`euler` or `uni_pc_bh2`) and `negative_prompt` |
| <a id="image_modelsencoder_release"></a>`encoder_release` | `auto` | Release the text encoder after each encode: `always`, `never`, or `auto`, which keeps it while the whole model fits in 80 percent of the working budget |
Expand Down
3 changes: 2 additions & 1 deletion docs/image-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,8 @@ for event in stream:
| One of several FLUX.3 klein references is under 16 pixels on a side | 400 naming the image |
| A FLUX.2 klein reference is under 64 pixels on a side, or one side is more than 8 times the other | 400 naming the image |
| `prompt_upsampling: true` on an entry without an upsampler | 400 naming the `upsampler` setting |
| The upsampler writes an empty caption | 500 `server_error` |
| The upsampler writes an empty caption, or a reply its family's parser finds no caption in | 500 `server_error` |
| The entry's `upsample_prompt` file is missing, or a Qwen-Image 2.1 entry with an upsampler has none | 500 `server_error` saying where to get the file |
| The upsampler's context has no room for its system prompt and caption | 500 `server_error` naming the upsampler |
| More requests waiting than [`image.max_queue`](config.md#image-requests) | 429 with `Retry-After` |
| The client disconnects | The job stops at its next step, and the log marks the request `cancelled` |
Expand Down
Loading
Loading