Repository navigation
feat: log what you wore from a selfie - #275
TheBlackSide243 wants to merge 3 commits into
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewer's GuideAdds an end-to-end outfit-selfie feature: the backend sends a bounded, numbered wardrobe list and uploaded image to the vision model, defensively converts its response into item matches, and exposes clear failure statuses; the frontend lets users review and confirm those matches to log wear, with navigation, tests, and translations included. Sequence diagram for outfit selfie recognition and wear loggingsequenceDiagram
actor User
participant SelfiePage
participant SelfieAPI
participant VisionModel
participant WearAPI
User->>SelfiePage: choose(image)
SelfiePage->>SelfieAPI: POST /api/v1/selfie
SelfieAPI->>VisionModel: _call_with_fallback(photo, numbered wardrobe)
VisionModel-->>SelfieAPI: JSON matches
SelfieAPI->>SelfieAPI: parse_matches(content, count)
SelfieAPI-->>SelfiePage: SelfieResponse(matches, considered)
User->>SelfiePage: review and select matches
User->>SelfiePage: confirm()
loop selected items
SelfiePage->>WearAPI: POST /items/{id}/wear
end
WearAPI-->>SelfiePage: wear logged
Flow diagram for selfie recognition outcomesflowchart TD
A[Upload outfit selfie] --> B[Validate image and size]
B -->|valid| C[Load user's ready wardrobe]
B -->|invalid or too large| E[Return 400 or 413]
C --> D{AI available}
D -->|no| F[Return 503]
D -->|yes| G[match_selfie]
G --> H{Vision model succeeds}
H -->|yes| I[parse_matches]
I --> J[Return matched items]
H -->|no| K[Return 502]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
github-actions
Bot
added
javascript
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 10 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="backend/app/services/selfie_service.py" line_range="59" />
<code_context>
+ if not candidates:
+ return []
+ listing = "\n".join(f"{n}. {describe(item)}" for n, item in enumerate(candidates, start=1))
+ image_base64 = await asyncio.to_thread(ai._preprocess_image, photo)
+ messages = [
+ {"role": "system", "content": SELFIE_PROMPT},
</code_context>
<issue_to_address>
**Selfie uploads exhaust worker memory**
When an authenticated user uploads a compressed image above `max_image_megapixels` but within the byte-size limit, `match_selfie` calls `_preprocess_image` before enforcing the pixel ceiling, so decoding the full-resolution image allocates excessive memory and can exhaust API worker memory, making the service unavailable.
Enforce the configured decoded-pixel limit before preprocessing, or reject images that exceed it.
</issue_to_address>
### Comment 2
<location path="backend/app/api/selfie.py" line_range="50-51" />
<code_context>
+ detail="Invalid image file. Supported formats: JPEG, PNG, WebP, HEIC",
+ )
+ try:
+ ai = get_ai_service()
+ except AIDisabledError:
+ raise HTTPException(
+ status_code=status.HTTP_503_SERVICE_UNAVAILABLE, detail="The AI is turned off"
</code_context>
<issue_to_address>
**Disabled vision still receives photos**
When text AI is enabled while vision AI is disabled, `get_ai_service()` succeeds because text AI is enabled, and `match_selfie` then sends the photo through `_call_with_fallback` to the vision model despite the vision restriction, sending users’ photos to an AI provider against configuration.
Check that vision AI is enabled before sending the selfie to the AI service.
Also at `backend/app/services/selfie_service.py:73`.
</issue_to_address>
### Comment 3
<location path="frontend/app/dashboard/selfie/page.tsx" line_range="54" />
<code_context>
+ const confirm = async () => {
+ setSaving(true);
+ try {
+ for (const id of Array.from(selected)) await logWear.mutateAsync({ id });
+ setDone(true);
+ toast.success(t('saved', { count: selected.size }));
</code_context>
<issue_to_address>
**Retry creates duplicate wear logs**
When a wear request fails after one or more earlier requests in the confirmation loop succeeded, each `logWear.mutateAsync` request commits separately, but the catch leaves the full selection available and `done` false. Retrying logs the already-successful items again, creating duplicate history entries and inflating wear counts.
Make the selected wear updates atomic, or track successful IDs and omit them on retry.
</issue_to_address>
### Comment 4
<location path="backend/app/api/selfie.py" line_range="74" />
<code_context>
+ photo = Path(folder) / "selfie"
+ photo.write_bytes(content)
+ try:
+ matches = await match_selfie(ai, photo, items)
+ except Exception as e:
+ logger.warning(f"Selfie recognition failed: {e}")
</code_context>
<issue_to_address>
**Selfie calls exhaust database connections**
When concurrent selfie requests keep the AI service busy long enough to occupy the available database connections, the handler keeps its database session open while `match_selfie` waits on the AI service. Under enough concurrent slow requests, those sessions hold all available database connections, so unrelated database-backed requests block or time out.
End the database transaction and release its connection after loading the candidate items, before awaiting AI recognition.
</issue_to_address>
### Comment 5
<location path="backend/app/api/selfie.py" line_range="32" />
<code_context>
+ considered: int
+
+
+@router.post("", response_model=SelfieResponse)
+async def recognise_outfit(
+ db: DbSession,
</code_context>
<issue_to_address>
**Repeated requests consume AI capacity**
When an authenticated user repeatedly submits selfies while AI is enabled, `recognise_outfit` has no per-user rate limit and sends each accepted selfie to the AI service. An authenticated user can repeatedly trigger expensive inference, consuming provider quota or capacity for other users.
Apply a per-user rate limit to the selfie recognition endpoint before invoking the AI service.
</issue_to_address>
### Comment 6
<location path="backend/app/services/selfie_service.py" line_range="44" />
<code_context>
+ if not numbers:
+ numbers = re.findall(r"\d+", content)
+ seen: list[int] = []
+ for number in numbers:
+ try:
+ index = int(number) - 1
</code_context>
<issue_to_address>
**Malformed matches cause incorrect results**
When the model returns a non-list `matches` value or entries that are not integers, `parse_matches` iterates `matches` without validating its shape and coerces entries with `int()`, so a scalar raises `TypeError` and strings, booleans or fractional values can become indexes. `recognise_outfit` then returns a 502 or incorrect suggestions.
Validate that `matches` is a list of integers and handle malformed values without using them as indexes.
</issue_to_address>
### Comment 7
<location path="backend/app/services/selfie_service.py" line_range="41" />
<code_context>
+ numbers = json.loads(match.group(0)).get("matches", [])
+ except (ValueError, AttributeError):
+ numbers = []
+ if not numbers:
+ numbers = re.findall(r"\d+", content)
+ seen: list[int] = []
</code_context>
<issue_to_address>
**Prose numbers become matches**
When the model returns non-JSON commentary with unrelated numbers, or prose alongside a valid empty `matches` list, `parse_matches` falls back to extracting every digit, so unrelated numbers in the commentary become item references and can produce incorrect suggestions.
Only use the text fallback for explicitly supported item references, and do not fall back when valid JSON contains an empty `matches` list.
Also at `backend/app/services/selfie_service.py:42`.
</issue_to_address>
### Comment 8
<location path="backend/app/api/selfie.py" line_range="83" />
<code_context>
+ ) from None
+
+ return SelfieResponse(
+ matches=[ItemResponse.model_validate(i) for i in matches], considered=len(items)
+ )
</code_context>
<issue_to_address>
**Considered count includes unevaluated items**
When the user has more than 150 eligible wardrobe items, `match_selfie` sends only the first 150 items to the model, but the response reports `len(items)`, so callers see a considered count larger than the number actually evaluated.
Set `considered` to the number of items actually sent to the model.
Also at `backend/app/api/selfie.py:84`, `backend/app/services/selfie_service.py:55`.
</issue_to_address>
### Comment 9
<location path="backend/tests/test_selfie.py" line_range="51" />
<code_context>
+
+ async def fake_call(self, messages, task_name, **kwargs):
+ listing = messages[1]["content"][0]["text"]
+ wanted = [line.split(".")[0] for line in listing.splitlines()[1:] if "coat" not in line]
+ return json.dumps({"matches": [int(n) for n in wanted]}), None, None
+
</code_context>
<issue_to_address>
**Missing photo goes unnoticed**
When the model request stops including the uploaded selfie, `test_returns_the_recognised_items` at `test_selfie.py:51-52` builds its fake response only from the clothes listing and ignores the image part of the request, so the expected items still return even if the uploaded selfie is no longer sent to the model.
Assert that the mocked model call receives the uploaded image in its message payload.
</issue_to_address>
### Comment 10
<location path="backend/tests/test_selfie.py" line_range="91" />
<code_context>
+ files={"image": ("me.jpg", _jpeg(), "image/jpeg")},
+ headers=auth_headers,
+ )
+ assert response.status_code == 502
</code_context>
<issue_to_address>
**Unclear error response passes**
When the endpoint returns 502 without a useful error detail, `test_ai_failure_is_a_clear_error` at `test_selfie.py:91` checks only the status code, so a 502 with an empty or unrelated detail still passes even though the response gives the user no useful explanation.
Assert that the response body contains the expected clear error detail.
</issue_to_address>Sourcery assessment
Needs a human reviewer. 10 findings to address first, and the uploaded selfie is sent to the configured AI service for recognition, so a privacy mistake could expose a personal image outside the app. Reverting stops future uploads but cannot retract images already sent.
Blocking findings: backend/app/services/selfie_service.py:59, backend/app/api/selfie.py:51, frontend/app/dashboard/selfie/page.tsx:54, backend/app/api/selfie.py:74, backend/app/api/selfie.py:32, and 5 more
| if not candidates: | ||
| return [] | ||
| listing = "\n".join(f"{n}. {describe(item)}" for n, item in enumerate(candidates, start=1)) | ||
| image_base64 = await asyncio.to_thread(ai._preprocess_image, photo) |
There was a problem hiding this comment.
🟠 High · Selfie uploads exhaust worker memory
When an authenticated user uploads a compressed image above max_image_megapixels but within the byte-size limit, match_selfie calls _preprocess_image before enforcing the pixel ceiling, so decoding the full-resolution image allocates excessive memory and can exhaust API worker memory, making the service unavailable.
Enforce the configured decoded-pixel limit before preprocessing, or reject images that exceed it.
Prompt for AI agents
In `backend/app/services/selfie_service.py` at line 59:
**Selfie uploads exhaust worker memory**
When an authenticated user uploads a compressed image above `max_image_megapixels` but within the byte-size limit, `match_selfie` calls `_preprocess_image` before enforcing the pixel ceiling, so decoding the full-resolution image allocates excessive memory and can exhaust API worker memory, making the service unavailable.
Enforce the configured decoded-pixel limit before preprocessing, or reject images that exceed it.| ai = get_ai_service() | ||
| except AIDisabledError: |
There was a problem hiding this comment.
🟠 High · Disabled vision still receives photos
When text AI is enabled while vision AI is disabled, get_ai_service() succeeds because text AI is enabled, and match_selfie then sends the photo through _call_with_fallback to the vision model despite the vision restriction, sending users’ photos to an AI provider against configuration.
Check that vision AI is enabled before sending the selfie to the AI service.
Also at backend/app/services/selfie_service.py:73.
Prompt for AI agents
In `backend/app/api/selfie.py` at lines 50-51:
**Disabled vision still receives photos**
When text AI is enabled while vision AI is disabled, `get_ai_service()` succeeds because text AI is enabled, and `match_selfie` then sends the photo through `_call_with_fallback` to the vision model despite the vision restriction, sending users’ photos to an AI provider against configuration.
Check that vision AI is enabled before sending the selfie to the AI service.
Also at `backend/app/services/selfie_service.py:73`.| const confirm = async () => { | ||
| setSaving(true); | ||
| try { | ||
| for (const id of Array.from(selected)) await logWear.mutateAsync({ id }); |
There was a problem hiding this comment.
🟡 Medium · Retry creates duplicate wear logs
When a wear request fails after one or more earlier requests in the confirmation loop succeeded, each logWear.mutateAsync request commits separately, but the catch leaves the full selection available and done false. Retrying logs the already-successful items again, creating duplicate history entries and inflating wear counts.
Make the selected wear updates atomic, or track successful IDs and omit them on retry.
Prompt for AI agents
In `frontend/app/dashboard/selfie/page.tsx` at line 54:
**Retry creates duplicate wear logs**
When a wear request fails after one or more earlier requests in the confirmation loop succeeded, each `logWear.mutateAsync` request commits separately, but the catch leaves the full selection available and `done` false. Retrying logs the already-successful items again, creating duplicate history entries and inflating wear counts.
Make the selected wear updates atomic, or track successful IDs and omit them on retry.| photo = Path(folder) / "selfie" | ||
| photo.write_bytes(content) | ||
| try: | ||
| matches = await match_selfie(ai, photo, items) |
There was a problem hiding this comment.
🟡 Medium · Selfie calls exhaust database connections
When concurrent selfie requests keep the AI service busy long enough to occupy the available database connections, the handler keeps its database session open while match_selfie waits on the AI service. Under enough concurrent slow requests, those sessions hold all available database connections, so unrelated database-backed requests block or time out.
End the database transaction and release its connection after loading the candidate items, before awaiting AI recognition.
Prompt for AI agents
In `backend/app/api/selfie.py` at line 74:
**Selfie calls exhaust database connections**
When concurrent selfie requests keep the AI service busy long enough to occupy the available database connections, the handler keeps its database session open while `match_selfie` waits on the AI service. Under enough concurrent slow requests, those sessions hold all available database connections, so unrelated database-backed requests block or time out.
End the database transaction and release its connection after loading the candidate items, before awaiting AI recognition.| considered: int | ||
|
|
||
|
|
||
| @router.post("", response_model=SelfieResponse) |
There was a problem hiding this comment.
🟡 Medium · Repeated requests consume AI capacity
When an authenticated user repeatedly submits selfies while AI is enabled, recognise_outfit has no per-user rate limit and sends each accepted selfie to the AI service. An authenticated user can repeatedly trigger expensive inference, consuming provider quota or capacity for other users.
Apply a per-user rate limit to the selfie recognition endpoint before invoking the AI service.
Prompt for AI agents
In `backend/app/api/selfie.py` at line 32:
**Repeated requests consume AI capacity**
When an authenticated user repeatedly submits selfies while AI is enabled, `recognise_outfit` has no per-user rate limit and sends each accepted selfie to the AI service. An authenticated user can repeatedly trigger expensive inference, consuming provider quota or capacity for other users.
Apply a per-user rate limit to the selfie recognition endpoint before invoking the AI service.| if not numbers: | ||
| numbers = re.findall(r"\d+", content) | ||
| seen: list[int] = [] | ||
| for number in numbers: |
There was a problem hiding this comment.
🟡 Medium · Malformed matches cause incorrect results
When the model returns a non-list matches value or entries that are not integers, parse_matches iterates matches without validating its shape and coerces entries with int(), so a scalar raises TypeError and strings, booleans or fractional values can become indexes. recognise_outfit then returns a 502 or incorrect suggestions.
Validate that matches is a list of integers and handle malformed values without using them as indexes.
Prompt for AI agents
In `backend/app/services/selfie_service.py` at line 44:
**Malformed matches cause incorrect results**
When the model returns a non-list `matches` value or entries that are not integers, `parse_matches` iterates `matches` without validating its shape and coerces entries with `int()`, so a scalar raises `TypeError` and strings, booleans or fractional values can become indexes. `recognise_outfit` then returns a 502 or incorrect suggestions.
Validate that `matches` is a list of integers and handle malformed values without using them as indexes.| numbers = json.loads(match.group(0)).get("matches", []) | ||
| except (ValueError, AttributeError): | ||
| numbers = [] | ||
| if not numbers: |
There was a problem hiding this comment.
🟡 Medium · Prose numbers become matches
When the model returns non-JSON commentary with unrelated numbers, or prose alongside a valid empty matches list, parse_matches falls back to extracting every digit, so unrelated numbers in the commentary become item references and can produce incorrect suggestions.
Only use the text fallback for explicitly supported item references, and do not fall back when valid JSON contains an empty matches list.
Also at backend/app/services/selfie_service.py:42.
Prompt for AI agents
In `backend/app/services/selfie_service.py` at line 41:
**Prose numbers become matches**
When the model returns non-JSON commentary with unrelated numbers, or prose alongside a valid empty `matches` list, `parse_matches` falls back to extracting every digit, so unrelated numbers in the commentary become item references and can produce incorrect suggestions.
Only use the text fallback for explicitly supported item references, and do not fall back when valid JSON contains an empty `matches` list.
Also at `backend/app/services/selfie_service.py:42`.| ) from None | ||
|
|
||
| return SelfieResponse( | ||
| matches=[ItemResponse.model_validate(i) for i in matches], considered=len(items) |
There was a problem hiding this comment.
🟡 Medium · Considered count includes unevaluated items
When the user has more than 150 eligible wardrobe items, match_selfie sends only the first 150 items to the model, but the response reports len(items), so callers see a considered count larger than the number actually evaluated.
Set considered to the number of items actually sent to the model.
Also at backend/app/api/selfie.py:84, backend/app/services/selfie_service.py:55.
Prompt for AI agents
In `backend/app/api/selfie.py` at line 83:
**Considered count includes unevaluated items**
When the user has more than 150 eligible wardrobe items, `match_selfie` sends only the first 150 items to the model, but the response reports `len(items)`, so callers see a considered count larger than the number actually evaluated.
Set `considered` to the number of items actually sent to the model.
Also at `backend/app/api/selfie.py:84`, `backend/app/services/selfie_service.py:55`.|
|
||
| async def fake_call(self, messages, task_name, **kwargs): | ||
| listing = messages[1]["content"][0]["text"] | ||
| wanted = [line.split(".")[0] for line in listing.splitlines()[1:] if "coat" not in line] |
There was a problem hiding this comment.
🟡 Medium · Missing photo goes unnoticed
When the model request stops including the uploaded selfie, test_returns_the_recognised_items at test_selfie.py:51-52 builds its fake response only from the clothes listing and ignores the image part of the request, so the expected items still return even if the uploaded selfie is no longer sent to the model.
Assert that the mocked model call receives the uploaded image in its message payload.
Prompt for AI agents
In `backend/tests/test_selfie.py` at line 51:
**Missing photo goes unnoticed**
When the model request stops including the uploaded selfie, `test_returns_the_recognised_items` at `test_selfie.py:51-52` builds its fake response only from the clothes listing and ignores the image part of the request, so the expected items still return even if the uploaded selfie is no longer sent to the model.
Assert that the mocked model call receives the uploaded image in its message payload.| files={"image": ("me.jpg", _jpeg(), "image/jpeg")}, | ||
| headers=auth_headers, | ||
| ) | ||
| assert response.status_code == 502 |
There was a problem hiding this comment.
🟡 Medium · Unclear error response passes
When the endpoint returns 502 without a useful error detail, test_ai_failure_is_a_clear_error at test_selfie.py:91 checks only the status code, so a 502 with an empty or unrelated detail still passes even though the response gives the user no useful explanation.
Assert that the response body contains the expected clear error detail.
Prompt for AI agents
In `backend/tests/test_selfie.py` at line 91:
**Unclear error response passes**
When the endpoint returns 502 without a useful error detail, `test_ai_failure_is_a_clear_error` at `test_selfie.py:91` checks only the status code, so a 502 with an empty or unrelated detail still passes even though the response gives the user no useful explanation.
Assert that the response body contains the expected clear error detail.- No photo is sent while vision AI is off, and photos over the upload pixel ceiling are refused before they are decoded. - Ten selfie checks per user per minute. - The database connection is released before waiting on the model. - Only a JSON answer or a bare list of numbers is read; numbers in prose and malformed match lists are ignored. - 'considered' counts the items the model was actually shown. - A retry after a failed save skips the wears already logged. - Tests check the photo reaches the model and the error detail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings > AI lets each user pick their own endpoints and models; tagging, suggestions and pairings already honour them. This feature now does too, through services/user_ai.ai_service_for, with the server's AI_* settings as the fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 participants
Description
Adds an Outfit selfie page: the user takes a mirror selfie, the vision model picks the items of
their wardrobe that appear in it, and the user confirms them to log the wear in one go.
POST /api/v1/selfie(multipart image) returns the matched items; the model gets a numberedlist of up to 150 of the user's items and answers with JSON item numbers, parsed defensively
(
services/selfie_service.py)./dashboard/selfie, nav entry, translations for all 8 locales.This is one of a set of small, independent PRs (each branches from
mainand can be reviewed ordropped on its own); they only overlap in the nav/router registration lines.
Type of Change
Checklist
Testing
ruff check,ruff format --check, migration heads check and the full pytest suite on top of currentmain(2031 passed).npm run lint,i18n:check,vocab:check,tsc --noEmitand vitest (693 passed).Test Environment
🤖 Generated with Claude Code
Summary by Sourcery
Enable users to log an outfit from a selfie by matching visible garments to their wardrobe and confirming the results.
New Features:
Enhancements:
Tests: