Repository navigation
feat: more accurate background removal for clothes - #278
Open
TheBlackSide243 wants to merge 2 commits into
Open
TheBlackSide243 wants to merge 2 commits into
TheBlackSide243 wants to merge 2 commits into
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewer's GuideThe PR improves clothing background removal by defaulting to the baked-in birefnet-general-lite model, cleaning provider masks to recover garment fabric and remove stray background, and making retries and rotations preserve a correctly oriented original backup. Sequence diagram for cleaned background removalsequenceDiagram
participant ImageService
participant OriginalPhoto
participant Provider as BackgroundRemovalProvider
participant MaskCleanup as clean_cutout
participant Output
ImageService->>OriginalPhoto: Image.open(source_full)
ImageService->>Provider: remove(image)
Provider-->>ImageService: cutout
ImageService->>MaskCleanup: clean_cutout(image, cutout)
alt garment missing
MaskCleanup-->>ImageService: NoGarmentFound
ImageService-->>Output: Keep original photo
else valid garment
MaskCleanup-->>ImageService: cleaned cutout
ImageService->>Output: Composite onto background
end
Flow diagram for retrying background removal from the originalflowchart TD
A["Remove background requested"] --> B{"Backup original exists?"}
B -->|Yes| C["Use backup original"]
B -->|No| D["Use current original photo"]
D --> E["Create backup after processing starts"]
C --> F["Run provider and clean_cutout"]
E --> F
F --> G["Save cleaned cut-out"]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
github-actions
Bot
added
docker
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 6 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="backend/app/services/image_service.py" line_range="323" />
<code_context>
+ backup_full = self.storage_path / f"{image_path.rsplit('.', 1)[0]}_orig.jpg"
+ if backup_full.exists():
+ backup = _flatten_to_rgb(Image.open(backup_full)).rotate(angle, expand=True)
+ backup.save(backup_full, "JPEG", quality=95)
+
return self._save_all_sizes(rotated, image_path)
</code_context>
<issue_to_address>
**Backup and image orientations diverge**
When a backup exists and saving the rotated image or one of its variants fails after the backup save succeeds, `rotate_image` has already overwritten `backup_full`, while `_save_all_sizes` leaves the displayed image or a variant unchanged. A later restore or background-removal retry uses the rotated backup against the old image orientation.
Save the rotated backup only after `_save_all_sizes` succeeds, or restore its previous contents if saving the image or its variants fails.
</issue_to_address>
### Comment 2
<location path="backend/app/services/image_service.py" line_range="266" />
<code_context>
backup_full = self.storage_path / backup_path
+ # A second removal starts again from the true original, so redoing a bad
+ # cut-out does not cut a cut-out.
+ source_full = backup_full if backup_full.exists() else original_full
+
+ image = Image.open(source_full).convert("RGB")
</code_context>
<issue_to_address>
**Concurrent edits leave mismatched orientations**
When a background-removal request or worker overlaps a rotation for the same item, `remove_background` reads the source before `rotate_image` updates the image and backup, then saves a cut-out in the earlier orientation while the backup has the later one. Restoring the backup changes the item's orientation unexpectedly.
Serialize rotation and background removal per item so they cannot read or write images concurrently.
Also at `backend/app/services/image_service.py:320-323`.
</issue_to_address>
### Comment 3
<location path="backend/app/services/mask_cleanup.py" line_range="17" />
<code_context>
+MIN_COVERAGE = 0.02
+# Pieces smaller than this share of the biggest one are background specks. Generous
+# enough that the second shoe of a pair or a sock is always kept.
+KEEP_RATIO = 0.05
+# Holes up to this share of the garment are fabric the model missed; bigger ones are
+# real openings, such as the space inside a bag handle.
</code_context>
<issue_to_address>
**Valid garment parts disappear**
When a legitimate disconnected garment part has less than 5% of the largest component's mask area, `clean_cutout` uses `KEEP_RATIO` to drop that component along with background specks, making the legitimate garment part transparent in the saved cut-out.
Adjust the component-filtering policy so it preserves legitimate small garment parts while removing background specks.
Also at `backend/app/services/mask_cleanup.py:50-51`.
</issue_to_address>
### Comment 4
<location path="backend/app/services/mask_cleanup.py" line_range="43" />
<code_context>
+ pixels = np.array(cut, dtype=np.uint8)
+ alpha = pixels[..., 3]
+ solid = alpha >= OPAQUE
+ if solid.mean() < MIN_COVERAGE:
+ raise NoGarmentFound("No garment found in the photo; the original was kept")
+
</code_context>
<issue_to_address>
**Small garments are rejected**
When a valid garment occupies less than 2% of the image, `clean_cutout` compares `solid.mean()` with `MIN_COVERAGE` and raises `NoGarmentFound`, so background removal fails and leaves the original photo unchanged.
Avoid rejecting a mask solely because its coverage of the entire photo is below 2%.
</issue_to_address>
### Comment 5
<location path="backend/app/services/image_service.py" line_range="275" />
<code_context>
# First removal wins: a second removal must not overwrite the true
# original with an already-processed image
if not backup_full.exists():
shutil.copy2(original_full, backup_full)
- image = Image.open(original_full).convert("RGB")
</code_context>
<issue_to_address>
**Interrupted backup writes poison recovery**
When a process stops or a filesystem write fails during backup creation or rotation, `shutil.copy2` and `backup.save` write directly to the backup path, so a failed or interrupted write can leave a partial file that still exists. A later removal trusts that path as its source, or restore tries to open the damaged backup, leaving retries or restoration failing instead of recovering the original photo.
Write backups to temporary files, validate and flush them, then atomically replace the destination.
Also at `backend/app/services/image_service.py:323`.
</issue_to_address>
### Comment 6
<location path="backend/app/services/image_service.py" line_range="266" />
<code_context>
backup_full = self.storage_path / backup_path
+ # A second removal starts again from the true original, so redoing a bad
+ # cut-out does not cut a cut-out.
+ source_full = backup_full if backup_full.exists() else original_full
+
+ image = Image.open(source_full).convert("RGB")
</code_context>
<issue_to_address>
**Bulk redo skips processed items**
When a user selects an item that already has a background-removal backup for bulk removal, the bulk-remove endpoint excludes items with `original_image_path` from `to_queue`, so it never reaches `ImageService.remove_background` to use the saved original; users cannot redo removal through the bulk action.
Allow bulk removal to queue already-processed items now that `remove_background` reprocesses the backup, while preserving the backup path.
</issue_to_address>Sourcery assessment
Needs a human reviewer. 6 findings to address first, and a bad mask or model result can be saved as the clothing image, but the original photo is retained in a backup and can be restored or processed again. Reverting the code would not undo images already processed.
Blocking findings: backend/app/services/image_service.py:323, backend/app/services/image_service.py:266, backend/app/services/mask_cleanup.py:17, backend/app/services/mask_cleanup.py:43, backend/app/services/image_service.py:275, and 1 more
| backup_full = self.storage_path / f"{image_path.rsplit('.', 1)[0]}_orig.jpg" | ||
| if backup_full.exists(): | ||
| backup = _flatten_to_rgb(Image.open(backup_full)).rotate(angle, expand=True) | ||
| backup.save(backup_full, "JPEG", quality=95) |
There was a problem hiding this comment.
🟡 Medium · Backup and image orientations diverge
When a backup exists and saving the rotated image or one of its variants fails after the backup save succeeds, rotate_image has already overwritten backup_full, while _save_all_sizes leaves the displayed image or a variant unchanged. A later restore or background-removal retry uses the rotated backup against the old image orientation.
Save the rotated backup only after _save_all_sizes succeeds, or restore its previous contents if saving the image or its variants fails.
Prompt for AI agents
In `backend/app/services/image_service.py` at line 323:
**Backup and image orientations diverge**
When a backup exists and saving the rotated image or one of its variants fails after the backup save succeeds, `rotate_image` has already overwritten `backup_full`, while `_save_all_sizes` leaves the displayed image or a variant unchanged. A later restore or background-removal retry uses the rotated backup against the old image orientation.
Save the rotated backup only after `_save_all_sizes` succeeds, or restore its previous contents if saving the image or its variants fails.| backup_full = self.storage_path / backup_path | ||
| # A second removal starts again from the true original, so redoing a bad | ||
| # cut-out does not cut a cut-out. | ||
| source_full = backup_full if backup_full.exists() else original_full |
There was a problem hiding this comment.
🟠 High · Concurrent edits leave mismatched orientations
When a background-removal request or worker overlaps a rotation for the same item, remove_background reads the source before rotate_image updates the image and backup, then saves a cut-out in the earlier orientation while the backup has the later one. Restoring the backup changes the item's orientation unexpectedly.
Serialize rotation and background removal per item so they cannot read or write images concurrently.
Also at backend/app/services/image_service.py:320-323.
Prompt for AI agents
In `backend/app/services/image_service.py` at line 266:
**Concurrent edits leave mismatched orientations**
When a background-removal request or worker overlaps a rotation for the same item, `remove_background` reads the source before `rotate_image` updates the image and backup, then saves a cut-out in the earlier orientation while the backup has the later one. Restoring the backup changes the item's orientation unexpectedly.
Serialize rotation and background removal per item so they cannot read or write images concurrently.
Also at `backend/app/services/image_service.py:320-323`.| MIN_COVERAGE = 0.02 | ||
| # Pieces smaller than this share of the biggest one are background specks. Generous | ||
| # enough that the second shoe of a pair or a sock is always kept. | ||
| KEEP_RATIO = 0.05 |
There was a problem hiding this comment.
🟡 Medium · Valid garment parts disappear
When a legitimate disconnected garment part has less than 5% of the largest component's mask area, clean_cutout uses KEEP_RATIO to drop that component along with background specks, making the legitimate garment part transparent in the saved cut-out.
Adjust the component-filtering policy so it preserves legitimate small garment parts while removing background specks.
Also at backend/app/services/mask_cleanup.py:50-51.
Prompt for AI agents
In `backend/app/services/mask_cleanup.py` at line 17:
**Valid garment parts disappear**
When a legitimate disconnected garment part has less than 5% of the largest component's mask area, `clean_cutout` uses `KEEP_RATIO` to drop that component along with background specks, making the legitimate garment part transparent in the saved cut-out.
Adjust the component-filtering policy so it preserves legitimate small garment parts while removing background specks.
Also at `backend/app/services/mask_cleanup.py:50-51`.| pixels = np.array(cut, dtype=np.uint8) | ||
| alpha = pixels[..., 3] | ||
| solid = alpha >= OPAQUE | ||
| if solid.mean() < MIN_COVERAGE: |
There was a problem hiding this comment.
🟡 Medium · Small garments are rejected
When a valid garment occupies less than 2% of the image, clean_cutout compares solid.mean() with MIN_COVERAGE and raises NoGarmentFound, so background removal fails and leaves the original photo unchanged.
Avoid rejecting a mask solely because its coverage of the entire photo is below 2%.
Prompt for AI agents
In `backend/app/services/mask_cleanup.py` at line 43:
**Small garments are rejected**
When a valid garment occupies less than 2% of the image, `clean_cutout` compares `solid.mean()` with `MIN_COVERAGE` and raises `NoGarmentFound`, so background removal fails and leaves the original photo unchanged.
Avoid rejecting a mask solely because its coverage of the entire photo is below 2%.| # First removal wins: a second removal must not overwrite the true | ||
| # original with an already-processed image | ||
| if not backup_full.exists(): | ||
| shutil.copy2(original_full, backup_full) |
There was a problem hiding this comment.
🟡 Medium · Interrupted backup writes poison recovery
When a process stops or a filesystem write fails during backup creation or rotation, shutil.copy2 and backup.save write directly to the backup path, so a failed or interrupted write can leave a partial file that still exists. A later removal trusts that path as its source, or restore tries to open the damaged backup, leaving retries or restoration failing instead of recovering the original photo.
Write backups to temporary files, validate and flush them, then atomically replace the destination.
Also at backend/app/services/image_service.py:323.
Prompt for AI agents
In `backend/app/services/image_service.py` at line 275:
**Interrupted backup writes poison recovery**
When a process stops or a filesystem write fails during backup creation or rotation, `shutil.copy2` and `backup.save` write directly to the backup path, so a failed or interrupted write can leave a partial file that still exists. A later removal trusts that path as its source, or restore tries to open the damaged backup, leaving retries or restoration failing instead of recovering the original photo.
Write backups to temporary files, validate and flush them, then atomically replace the destination.
Also at `backend/app/services/image_service.py:323`.| backup_full = self.storage_path / backup_path | ||
| # A second removal starts again from the true original, so redoing a bad | ||
| # cut-out does not cut a cut-out. | ||
| source_full = backup_full if backup_full.exists() else original_full |
There was a problem hiding this comment.
🟡 Medium · Bulk redo skips processed items
When a user selects an item that already has a background-removal backup for bulk removal, the bulk-remove endpoint excludes items with original_image_path from to_queue, so it never reaches ImageService.remove_background to use the saved original; users cannot redo removal through the bulk action.
Allow bulk removal to queue already-processed items now that remove_background reprocesses the backup, while preserving the backup path.
Prompt for AI agents
In `backend/app/services/image_service.py` at line 266:
**Bulk redo skips processed items**
When a user selects an item that already has a background-removal backup for bulk removal, the bulk-remove endpoint excludes items with `original_image_path` from `to_queue`, so it never reaches `ImageService.remove_background` to use the saved original; users cannot redo removal through the bulk action.
Allow bulk removal to queue already-processed items now that `remove_background` reprocesses the backup, while preserving the backup path.- Backups are written through a temporary file and swapped in, so an interrupted write never leaves a truncated original to restore from. - Rotation turns the backup only after the photo and its sizes are saved. - Smaller detached parts (straps, hoods) are kept and small garments such as jewellery are no longer rejected as empty cut-outs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Open
8 of 15 tasks
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
u2netoften gets clothes wrong: a white print or a light patch on a shirt is taken forbackground and punched out, and bits of the bed or floor the garment lies on survive.
birefnet-general-lite(baked into the image likeu2netwas).On my NAS (x86, CPU only) it takes about 5 s per photo; it is heavier than
u2net, so on a Piit will be noticeably slower - happy to keep
u2netas the default and only document theoption if you prefer.
services/mask_cleanup.py, numpy/scipy, already dependencies throughimagehash): small holes inside the garment are filled back from the original photo, loose
specks of background are dropped (both shoes of a pair are kept), and a result that lost the
garment altogether raises instead of wiping the photo.
image, and the backup is rotated together with the photo.
This is one of a set of small, independent PRs (each branches from
mainand can be reviewed ordropped on its own); they only overlap in the nav/router registration lines.
Type of Change
Checklist
Testing
ruff check,ruff format --check, migration heads check and the full pytest suite on top of currentmain(2035 passed).npm run lint,i18n:check,vocab:check,tsc --noEmitand vitest (693 passed).Test Environment
🤖 Generated with Claude Code
Summary by Sourcery
Improve clothing background removal accuracy and protect original images during repeated processing and transformations.
New Features:
birefnet-general-litemodel by default and cleaning generated masks to preserve garment details while removing background artifacts.Bug Fixes:
Enhancements:
Documentation:
Tests: