Feature/balanced detection batch sampling - #1405
Conversation
Optional train.positive_batch_fraction fixes annotated vs empty images per batch and subsamples large negative pools. Co-authored-by: Cursor <cursoragent@cursor.com>
DataLoader and Lightning require a BatchSampler instance when batch_sampler is set. Co-authored-by: Cursor <cursoragent@cursor.com>
…htning. PyTorch Lightning reinstantiates custom batch samplers with a distributed sampler; positive indices now flow through self.sampler. Co-authored-by: Cursor <cursoragent@cursor.com>
Replace per-image DataFrame equality scans with one groupby over image_path. Co-authored-by: Cursor <cursoragent@cursor.com>
…set sampler. Lightning reinstantiates batch samplers with a sampler over len(dataset); those indices must not index positive_indices. Shuffle positives with randperm instead. Document use_distributed_sampler for multi-GPU. Co-authored-by: Cursor <cursoragent@cursor.com>
…re-commit end-of-file-fixer)
PointDataset/PolygonDataset and shapefile-loaded BoxDataset annotations carry a geometry column instead of xmin/ymin/xmax/ymax, so the vectorized groupby raised KeyError. Derive row totals from geometry bounds when no box columns exist, mirroring annotations_for_path's existing branch.
|
@jveitchmichaelis this is ready, I'm not 100% happy with the code styling of |
| self._validate_labels() | ||
| if validate_coordinates: | ||
| self._validate_coordinates() | ||
| self.positive_indices, self.negative_indices = ( |
There was a problem hiding this comment.
I don't know if there's a significant penalty here, but this is always computed regardless of whether the indices are used.
|
|
||
| Distributed sharding of the positive pool is not applied. Multi-GPU runs | ||
| typically need ``Trainer(use_distributed_sampler=False)`` or equivalent | ||
| unless you accept duplicate positive coverage per rank. |
There was a problem hiding this comment.
Is this an issue in practice?
| validate_coordinates: if True, check annotation coordinates fall within image bounds | ||
| augmentations: augmentation configuration (str, list, or dict) | ||
| positive_batch_fraction: If set with shuffle=True, each batch contains | ||
| this fraction of annotated images and the rest hard negatives. |
There was a problem hiding this comment.
Are they actually hard negatives or empty? I understand hard negative mining to also include classes that are often mis-predicted, but not necessarily a blank image.
| batch_size=batch_size, | ||
| positive_batch_fraction=positive_batch_fraction, | ||
| ) | ||
| data_loader = torch.utils.data.DataLoader( |
There was a problem hiding this comment.
Style nit - move DataLoader out of the condition and set batch_sampler to None in the else?
There was a problem hiding this comment.
Maybe a test through deepforest.main to confirm that setting the balanced sampler in the config is fine + we can load samples (no need to actually predict).
2 participants
This PR allows users to balance empty and positive frames during detection training. On the BOEM dataset we found that 0.5 was fruitful. Default is null, normal random sampling.
https://www.comet.com/bw4sz/boem/compare?compareXAxis=step&experiment-tab=panels&experiments=9d437e52a91d464cb8a94a5ef478e576,e708632048074606aa3764131ec5003f,3c03784ccd32416f8ef4b75b67c54b35&showOutliers=true&smoothing=0&viewId=new&xAxis=step
Agent statement
I conceived of this idea and drafted a prototype alongside an agent. I then measured its performance through a series of comet experiments on the BOM dataset. When I was satisfied that it improved performance, I took over the pull request, edited the docs, reduced the test, and cleaned up the Python code.