Skip to content

✨ Add training - #1178

Draft
measty wants to merge 20 commits into
developfrom
add-training-draft
Draft

measty wants to merge 20 commits into
developfrom
add-training-draft

Conversation

@measty

@measty measty commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

This PR adds a training API and example notebooks to TIAToolbox.

It's main design choices are:

  • Assume annotations are provided as annotation stores, so we can provide various target-builder classes to turn annotation stores into targets like masks, boundary, gaussian centroid, and coverage-based classification targets to support a variety of training tasks
  • Provide dataset classes that take slide (or patch) + annotation store pairs, together with a target builder, and return an iterable over patch-target pairs suitable for training
  • Save the trained model with its configuration in the same format as that in pretrained_model.yaml, so model loading and WSI-level inference using TIAToolbox engines can be done in a unified way using the machinery already present for TIAToolbox pretrained models.

A good example of why this can be so powerful is shown in this code snippet for training a multi-headed segmentation model:

# this stacks a mask, boundary and gaussian target builder for each of conic's classes.
target_builder = CompositeTargetBuilder(
    {
        label.lower(): StackedTargetBuilder(
            {
                "mask": MaskTargetBuilder(
                    class_mapping=None,
                    default_label=0,
                ),
                "boundary": BoundaryTargetBuilder(line_width=1),
                "centroid": GaussianHeatmapTargetBuilder(sigma=2.0),
            },
            where=f'props["class"] == "{label}"',  # filter by the heads label
        )
        for label in conic_class_dict.values()
    }
)

# this creates a dataset of patches each associated with an annotation store, that will 
# output a patch and a structured dictionary of targets, one entry for each head, with each entry 
# being a stack of mask/boundary/centroid created from annotations only of a specific class.
dataset = PatchAnnotationDataset.from_dirs(
    patch_dir=images_dir,
    annotation_store_dir=stores_dir,
    target_builder=target_builder,
    preproc=KongNet.preproc,
)

We have defined, with relatively little code, a quite complex set of targets for a multi-head segmentation model such as KongNet, utilizing the AnnotationStore query functionality via the where=f'props["class"] == "{label}"' to limit the annotations read be each head to only the annotations of the appropriate class for that head, and using the 3 stacked target builders to create a 3-channel target for each head that combines mask, boundary and centroid supervison. We have also created a convenient dataset that will construct image-target pairs of that form directly from the annotation stores.

A typical training workflow for a new task would look like:

  1. Convert your training annotations into annotation stores
  2. Choose a target-builder appropriate to your task
  3. Pass image paths (wsi's or patches) and annotation paths together with the target-builder to the appropriate dataset class, and choose an augmentation preset
  4. Create a split and pass Training and validation dataloaders to the trainer
  5. Train, and export the final trained model
  6. The trained model will now load directly into TIAToolbox engines for WSI-level inference.

Some common cases are illustrated in the 3 example notebooks:

  • notebook 13 illustrates patch-level training from class-folder based dataset like kather100k
  • notebook 14 illustrates training a semantic segmentation task and performing WSI-level inference
  • notebook 15 illustrates a cell detection and instance segmentation training pipeline, with more complex stacked/composite targets (mask + centroid + boundary stack per class head) by training a KongNet model on a subset of the Conic dataset.

Most parts are fairly easy to replace with custom versions (for example custom targets or transforms) if needed.

More detailed documentation can be found in the training.rst docs page.

Currently a draft - it is a large PR so testing it out a bit would be useful.

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

try:
self.sample_slide_indices, self.sample_bounds = self._build_patch_index()
finally:
self.close()
)
)
finally:
self.close()
@codecov

codecov Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.93252% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.92%. Comparing base (58c0472) to head (e034581).

Files with missing lines Patch % Lines
tiatoolbox/models/training/trainer.py 99.51% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           develop    #1178     +/-   ##
==========================================
  Coverage    99.91%   99.92%             
==========================================
  Files          116      129     +13     
  Lines        13578    16489   +2911     
  Branches      1624     2155    +531     
==========================================
+ Hits         13567    16476   +2909     
  Misses           5        5             
- Partials         6        8      +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

failure = RuntimeError("read failed")
with pytest.raises(RuntimeError, match="read failed"), reader:
raise failure
handle.close.assert_called_once_with()
@shaneahmed shaneahmed changed the title Add training (draft) :New: Add training (draft) Oct 30, 2026
@shaneahmed shaneahmed added this to the Release v2.2.0 milestone Oct 30, 2026
@shaneahmed shaneahmed added the enhancement New feature or request label Oct 30, 2026
@shaneahmed shaneahmed changed the title :New: Add training (draft) ✨ Add training (draft) Oct 30, 2026
"""Return number of available pairs."""
return len(self.images)

def __getitem__(self: PatchMaskPairDataset, index: int) -> dict:
@measty measty changed the title ✨ Add training (draft) ✨ Add training Nov 12, 2026
@shaneahmed

Copy link
Copy Markdown
Member

Please can you rename the files to 14, 15, & 16?

@review-notebook-app

review-notebook-app Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

View / edit / reply to this conversation on ReviewNB

shaneahmed commented on 2026-11-13T13:35:57Z
----------------------------------------------------------------

can we add a hyperlink to 05 notebook?


@review-notebook-app

review-notebook-app Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

View / edit / reply to this conversation on ReviewNB

shaneahmed commented on 2026-11-13T13:35:57Z
----------------------------------------------------------------

Line #22.            "https://tiatoolbox.dcs.warwick.ac.uk/datasets/kather100k-validation-sample.zip",

Are these available on huggingface? If yes, can we use that link instead?


@review-notebook-app

review-notebook-app Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

View / edit / reply to this conversation on ReviewNB

shaneahmed commented on 2026-11-13T13:35:58Z
----------------------------------------------------------------

Line #30.            "https://tiatoolbox.dcs.warwick.ac.uk/sample_wsis/TCGA-3L-AA1B-01Z-00-DX1.8923A151-A690-40B7-9E5A-FCBEDFC2394F.svs",

same here, if this is available via huggingface.

I was also thinking if we should have an API to download TCGA images directly from TCGA?


@review-notebook-app

review-notebook-app Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

View / edit / reply to this conversation on ReviewNB

shaneahmed commented on 2026-11-13T13:35:59Z
----------------------------------------------------------------

Line #1.    fig, axes = plt.subplots(len(class_names), 4, figsize=(10, 14))

Can we print something above the figures to show what's being displayed? Figure title etc?


@review-notebook-app

review-notebook-app Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

View / edit / reply to this conversation on ReviewNB

shaneahmed commented on 2026-11-13T13:36:00Z
----------------------------------------------------------------

Line #16.        persistent_workers=True,

Move the comment  "Persistent workers avoid restarting worker processes every epoch." on line 16 if possible.


@review-notebook-app

review-notebook-app Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

View / edit / reply to this conversation on ReviewNB

shaneahmed commented on 2026-11-13T13:36:01Z
----------------------------------------------------------------

It would be good to add a markdown cell to explain what this cell is doing.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants