Skip to content

Read cover images and model info embedded in safetensors headers - #1803

Open
deepratna-awale wants to merge 4 commits into
LykosAI:mainfrom
deepratna-awale:read-embedded-safetensors-covers
Open

deepratna-awale wants to merge 4 commits into
LykosAI:mainfrom
deepratna-awale:read-embedded-safetensors-covers

Conversation

@deepratna-awale

@deepratna-awale deepratna-awale commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1800

Problem

Stability Matrix only takes a local model's preview image and details from loose files next to the model (NAME.preview.jpeg, NAME.cm-info.json). A model that carries them inside its safetensors header, in the LoRA-Container format described in the issue, shows no image and no details in the Checkpoint Manager, so they are lost as soon as the model is moved, renamed or shared without its sidecar files.

The same header often has other useful metadata that the app ignores today: the base model and trigger words that ModelSpec and kohya-ss training write.

Changes

Reading the header

  • New SafetensorEmbeddedInfo reads, from a safetensors file's __metadata__:
    • Cover image: cover_1 (LoRA-Container), or modelspec.thumbnail if there is no cover_1. Following the spec, a file without cover_count has no covers, cover_N past the count is ignored, and a data:image/...;base64, prefix is accepted. GetCover(metadata, n) can read gallery covers too.
    • Connected model info: cm_info, the embedded copy of .cm-info.json.
    • Base model: modelspec.architecture, falling back to ss_base_model_version, mapped to the CivitAI base model names the app already uses. Only unambiguous architectures are mapped: SD 1.5, SDXL 1.0, SD 3, SD 3.5 Medium/Large, Flux.1 D and Flux.1 S. Kohya's flux1 and sd_v2 are left unmapped because they don't say which variant.
    • Trigger words: modelspec.trigger_phrase.
    • Original file hash: model_sha256_original.
  • SafetensorMetadata: header reading moved into an internal ReadMetadataDictionaryAsync (now cancellable) that both use. ParseAsync behaves as before.

Model index

  • After indexing, .safetensors models missing a preview image or a .cm-info.json have their header read once:
    • The cover is written to <temp>/StabilityMatrix/Cache/EmbeddedCovers/<sha256>.jpeg|png (the same temp cache root the image loader uses) and used only when there is no preview image file.
    • cm_info becomes the model's ConnectedModelInfo only when there is no .cm-info.json.
    • The base model and trigger words are stored on the model as EmbeddedBaseModel / EmbeddedTrainedWords.
  • Results are cached in memory per file path, size and last write time, so unchanged models aren't read again on later refreshes. Models that have both sidecar files are never opened. Read failures are logged at Debug.
  • The LiteDB ImportedAt truncation was moved into a helper so the embedded cm_info gets it too.

Where it shows up

  • LocalModelFile gets GetDisplayPreviewImageFullPath(), DisplayBaseModel and DisplayTrainedWords(String). These prefer the sidecar data and fall back to the embedded data. PreviewImageFullPathGlobal falls back to the embedded cover.
  • Checkpoint Manager: the card image uses the embedded cover. The base model badge, base model filter and sort, trigger words search and "Copy trigger words" and the card's trigger words line use the embedded base model and trigger words. The badge row now also shows for models with a base model from the header but no connected info. The model type badge stays hidden for those models.
  • Inference: the model pickers and the model picker dialog show the embedded cover and base model. The LoRA trigger words, LoRA base model filtering (ModelCompatChecker) and the base-model-based workflow and prompt hints use DisplayBaseModel / DisplayTrainedWords.
  • View Metadata dialog: cover_N and modelspec.thumbnail show as (embedded image, N KB) instead of their base64 text.

CivitAI lookup

  • Embedding images changes the file hash, so a lookup by hash finds nothing. When metadata import (single file or folder scan) can't find a safetensors model by its BLAKE3 hash, it retries with model_sha256_original from the header. ModelFinder.RemoteFindModel now matches the returned file by BLAKE3 or SHA256.

Everything is read-only. The model file is never modified. Rename, delete, move and the metadata editor still work only with the loose preview image file (GetPreviewImageFullPath is unchanged), so the cached cover is never copied or moved next to the model.

Screenshots

Rendered headless with the real Checkpoint Manager page and View Metadata dialog. The models folder holds five real LoRAs, with no sidecar files next to them:

  • niji_gloss, cb_lora_anima-000050, barska_ilxl and tosskant_ilxl: LoRAs downloaded from CivitAI whose trainers embedded a modelspec.thumbnail. The two _ilxl files carry ModelSpec's default SDXL thumbnail and kohya's ss_base_model_version. The two Anima LoRAs also have modelspec.trigger_phrase.
  • chaoslandscape: the LoRA-Container sample from the format's repository, linked in the issue, with six embedded covers and a cm_info.

Checkpoint Manager, before

Checkpoint Manager, after. Embedded covers and thumbnails, trigger words from cm_info or ModelSpec, and the base model badge where the architecture maps to one (Anima isn't in the mapping, so those two have no badge).

View Metadata for niji_gloss, before / after

Testing

  • New SafetensorEmbeddedInfoTests: reading a LoRA-Container file (cover, cm_info, original hash, base model), a file without metadata, gallery covers and cover_count bounds, data URI prefixes, missing cover_count, invalid base64, the modelspec.thumbnail fallback and its priority, the base model mapping table, trigger phrase splitting, invalid original hashes and invalid cm_info JSON.
  • SafetensorMetadataTests: embedded images are shown as placeholders.
  • LocalModelFileTests: the sidecar preview, base model and trigger words take priority over embedded ones, and the embedded ones are used when the sidecar has none.
  • MetadataImportServiceTests: a model not found by its BLAKE3 hash is found by model_sha256_original and its .cm-info.json is written.
  • Ran ModelIndexService.RefreshIndex() twice against a temp models folder with an embedded-cover-and-cm_info LoRA, a ModelSpec-only LoRA and a LoRA with both a sidecar and an embedded cm_info. Each got the expected preview, name, base model and trigger words, the sidecar won where present, and the second refresh served from the cache.
  • Took the screenshots above headless on this branch and on main, using a throwaway UI test that isn't part of the PR. This caught two bindings that still read ConnectedModelInfo directly: the card's trigger words line and the model picker dialog's base model badge. Both are fixed.
  • StabilityMatrix.Tests: all passed except the TestLicenseUrl / TestPreviewImageUri checks, which need outbound network access that my test environment blocks (unrelated to this change). StabilityMatrix.UITests: 8/8 passed.

Models can carry their cover image inside the safetensors header
metadata (cover_count and cover_1 ... cover_N, base64 JPEG), so the
preview survives moving, renaming or sharing the file without its
loose .preview image.

When indexing, safetensors models that have no preview image file are
checked for an embedded cover_1. It is written once to a cache folder
in the temp directory, named by its SHA-256, and used as the model's
preview in the Checkpoint Manager, the Inference model pickers and
prompt completion. A loose preview image always takes priority, the
model file is never modified, and results are cached per file size and
modification time so unchanged files are not re-read on later index
refreshes.

Fixes LykosAI#1800
Read the rest of the LoRA-Container metadata and the common ModelSpec
and kohya-ss keys while indexing safetensors models:

- cm_info: used as the model's connected model info when there is no
  .cm-info.json next to the model.
- modelspec.thumbnail: used as the preview when there is no cover_1.
- modelspec.architecture / ss_base_model_version: the base model, for
  models without one in their connected model info. Shown on the
  Checkpoint Manager card and used by its base model filter and sort,
  the Inference model pickers and LoRA base model filtering.
- modelspec.trigger_phrase: trigger words for models without any,
  shown and copyable in the Checkpoint Manager and Inference.
- model_sha256_original: when a CivitAI lookup by file hash finds
  nothing, metadata import retries with the original file hash, so
  models whose header was edited after download can still be matched.

Embedded images (cover_N, modelspec.thumbnail) are shown as a short
placeholder in the View Metadata dialog instead of their base64 text.
…l picker

The Checkpoint Manager card's trigger words line and the model picker
dialog's base model badge still read ConnectedModelInfo directly, so
trigger words and base models from the safetensors header were missing
there.
@deepratna-awale
deepratna-awale marked this pull request as ready for review November 19, 2026 01:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Read cover images and gallery embedded in the safetensors header

1 participant