Skip to content

fix(download): retry http 400 with browser user-agent - #325

Merged
soimkim merged 1 commit into
mainfrom
fix/download-http-400-user-agent
Nov 2, 2026
Merged

soimkim merged 1 commit into
mainfrom
fix/download-http-400-user-agent

Conversation

@soimkim

@soimkim soimkim commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Retry an HTTP download with the next User-Agent when the server returns 400, the same way 403 is already retried. maven.egovframe.go.kr answers 400 Request Blocked for curl and wget User-Agents and returns the jar for a browser User-Agent.
  • Send that browser User-Agent from the system wget fallback and its Content-Length probe.
  • A real 404 still fails on the first attempt.

Test plan

  • PYTHONPATH=src python -m pytest tests/test_download_user_agent.py
  • fosslight_download -s https://maven.egovframe.go.kr/maven/org/egovframe/rte/org.egovframe.rte.fdl.cmmn/4.1.0/org.egovframe.rte.fdl.cmmn-4.1.0.jar -t /tmp/egov-jar

Summary by CodeRabbit

  • Bug Fixes
    • Improved download compatibility with servers that reject the default User-Agent by retrying with a browser-style User-Agent after HTTP 400 or 403 responses.
    • System-wget downloads now use a browser-style User-Agent for both availability checks and downloads.

maven.egovframe.go.kr returns 400 Request Blocked for curl and wget
User-Agents. The retry only continued on 403, so the jar download
stopped on the first attempt.
@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 2caa78c and fe9d6e3.

Files selected for processing: 2
  • src/fosslight_util/download.py
  • tests/test_download_user_agent.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Walkthrough

Download probes and downloads now retry with a browser User-Agent for HTTP 400 and 403 responses. System-wget HEAD requests and commands now send the browser User-Agent. Tests cover these behaviors and confirm that HTTP 404 does not trigger a retry.

Changes: Download User-Agent handling

Layer / File(s) Summary
HTTP status retries
src/fosslight_util/download.py, tests/test_download_user_agent.py
A shared status check makes is_downloadable and download_file retry with a browser User-Agent for HTTP 400 and 403 responses. Tests check those retries and verify that HTTP 404 does not trigger one.
System-wget User-Agent
src/fosslight_util/download.py, tests/test_download_user_agent.py
The system-wget HEAD request and command now send the browser User-Agent. The test checks both.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Possibly related PRs

  • fosslight/fosslight_util#323: Adds version-level retries to the download flow; this PR changes the lower-level HTTP and system-wget attempts used by that flow.

Suggested reviewers: bjk7119

Merge Risk: Minimal · up to fe9d6

No merge-blocking risk is established from the supplied evidence.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: retrying HTTP 400 downloads with a browser User-Agent.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 9 functions across 2 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
Generate docstrings
  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@soimkim soimkim self-assigned this Nov 2, 2026
@soimkim soimkim added the chore [PR/Issue] Refactoring, maintenance the code label Nov 2, 2026
@soimkim
soimkim merged commit dd2cf2b into main Nov 2, 2026
7 of 8 checks passed
@soimkim
soimkim deleted the fix/download-http-400-user-agent branch November 2, 2026 05:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore [PR/Issue] Refactoring, maintenance the code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant