Skip to content

fix(api): resolve new pyrefly unreachable-except and unwrap diagnostics - #44053

Open
mesrop2008 wants to merge 18 commits into
langgenius:mainfrom
mesrop2008:fix/pyrefly-redundant-except-and-unwrap
Open

mesrop2008 wants to merge 18 commits into
langgenius:mainfrom
mesrop2008:fix/pyrefly-redundant-except-and-unwrap

Conversation

@mesrop2008

@mesrop2008 mesrop2008 commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #44038

The upcoming pyrefly 1.4 releases (reproduced with pyrefly==1.4.0.dev3) report 17 warnings in api/ and 10 errors in api/tests/unit_tests. This PR fixes all of them. Runtime behavior is unchanged: each simplified except still catches the same exceptions, and the removed branches could never run.

Unreachable except clauses (unreachable-except-clause)

ValueError already covers pydantic ValidationError, json.JSONDecodeError, UnicodeDecodeError, binascii.Error and DocumentIndexingError (BaseServiceError subclasses ValueError). These were removed from the exception tuples, along with imports that are no longer used:

  • controllers/console/app/workflow.py
  • controllers/console/datasets/rag_pipeline/rag_pipeline_workflow.py
  • core/plugin/plugin_service.py
  • core/provider_manager.py (2 sites)
  • core/workflow/file_reference.py
  • libs/oauth.py
  • repositories/data_source/credential_repository.py
  • repositories/knowledge/document_repository.py
  • services/knowledge/documents/adapters.py
  • services/tools/tools_transform_service.py

In controllers/console/explore/trial.py, AgentAppNotPublishedError subclasses AgentAppGeneratorError, and both already mapped to AppUnavailableError, so the clause is now just except AgentAppGeneratorError.

Unreachable code (unreachable)

  • libs/token.py: _unauthorized() always raises, so it is now annotated -> NoReturn. The raise after it, which existed only for the type checker, is removed.
  • services/knowledge/documents/adapters.py: removed the indexing_technique is required check. KnowledgeConfig.indexing_technique is a required Literal["high_quality", "economy"], and config always comes from KnowledgeConfig.model_validate, so the condition could never be true.
  • services/knowledge/dataset_service.py: removed the pre_processing_rules id is required check, because PreProcessingRule.id is a required, non-empty Literal. Also removed the matching test in tests/unit_tests/services/document_service_validation.py. That test skipped validation with model_construct(id=None) to reach the check, and pytest doesn't collect the file anyway because its name doesn't match test_*.py.

unwrap argument errors (bad-argument-type)

When setup_required is stacked on top of another ParamSpec decorator, pyrefly 1.4 cannot resolve the method's signature and infers (ParamSpec(Unknown)) -> Unknown. As suggested in the issue thread, the tests now cast the decorated method before unwrapping it. Production decorator types are unchanged:

  • tests/unit_tests/controllers/console/workspace/test_skills.py: cast to the concrete return type each test uses (Response, or tuple[dict[str, str], int]).
  • tests/unit_tests/controllers/files/test_plugin_file_upload.py: the seven identical unwrap calls now go through a single _post_upload() helper.
  • tests/unit_tests/controllers/inner_api/app/test_file_grants.py: cast to Callable[..., object] before inspect.unwrap.

Testing

  • pyrefly 1.4.0.dev3 (the version that reproduces the issue): 17 warnings and 10 errors before; no diagnostics after, for both the api/ project and tests/unit_tests/pyrefly.toml.
  • pyrefly 1.3.2 (pinned in uv.lock, used by CI): still no diagnostics for the api/, tests/unit_tests and tests/test_containers_integration_tests configs.
  • ruff: ruff format --check and ruff check pass on all changed files, and the pre-commit hooks pass.
  • CI guards: check_no_new_test_doubles, check_no_new_getattr, check_no_new_controller_sqlalchemy, check_no_new_session_mock and check_no_spec_mock pass against current main.
  • Unit tests: 1760 passed. The run covered the changed controllers (console/app/test_workflow.py, console/datasets/rag_pipeline, console/explore, console/workspace/test_skills.py, files/test_plugin_file_upload.py, inner_api/app/test_file_grants.py), plus libs/test_oauth_*.py, libs/test_token.py, core/test_provider_manager.py, repositories/data_source, repositories/knowledge, services/knowledge and services/tools/test_tools_transform_service.py.
  • New tests for the simplified except ValueError branches (97 passed across these files):
    • core/test_provider_manager.py: a load-balancing config with malformed stored JSON is skipped while valid configs are kept and cached, and malformed provider credentials return {}.
    • core/workflow/test_file_reference.py (new): canonical references round-trip, and invalid base64 or JSON payloads fall back to using the raw value as the record_id.
    • repositories/data_source/test_credential_repository.py: a stored document with unparsable data_source_info does not resolve a credential.
    • services/knowledge/test_dataset_adapters.py: change_status maps both ValueError and DocumentIndexingError to DocumentInvalidActionError.

Note: pyrefly 1.4.0.dev4, released after the issue was filed, adds 27 method-assign errors in unit tests and 1 redundant-cast warning in services/workflow_draft_variable_service.py. They are not part of #44038 and are left for a follow-up.

Screenshots

Before After
N/A (backend type-check cleanup, no UI change) N/A

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've verified the change and added or updated tests where meaningful regression risk justifies coverage.
  • I've updated the documentation accordingly.
  • I ran make lint && make type-check (backend) and vp staged (frontend) to appease the lint gods

pyrefly 1.4 flags except clauses that can never match and fails to
resolve the signature of methods stacked under setup_required.

- Collapse except tuples whose members are already covered by
  ValueError (pydantic ValidationError, JSONDecodeError,
  UnicodeDecodeError, binascii.Error, DocumentIndexingError) or by
  AgentAppGeneratorError.
- Annotate the CSRF _unauthorized helper as NoReturn and drop the
  trailing raise that existed only for the type checker.
- Remove dead validation branches guarded by required Literal fields
  (KnowledgeConfig.indexing_technique, PreProcessingRule.id) and the
  model_construct-based test that targeted one of them.
- Cast decorated controller methods at the test boundary before
  unwrapping them.

Fixes langgenius#44038
@github-actions github-actions Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Oct 14, 2026
@ghfind-review ghfind-review Bot added the review: medium ghfind author score; see https://ghfind.com label Oct 14, 2026
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Pyrefly Diff

base → PR
--- /tmp/pyrefly_base.txt	2026-11-03 02:37:13.909384149 +0000
+++ /tmp/pyrefly_pr.txt	2026-11-03 02:37:00.933304384 +0000
@@ -184,6 +184,8 @@
    --> providers/vdb/vdb-tidb-on-qdrant/src/dify_vdb_tidb_on_qdrant/tidb_service.py:153:35
 ERROR Argument `Any | None` is not assignable to parameter `cluster_response` with type `Mapping[str, Any]` in function `TidbService.extract_qdrant_endpoint` [bad-argument-type]
    --> providers/vdb/vdb-tidb-on-qdrant/src/dify_vdb_tidb_on_qdrant/tidb_service.py:154:75
+ERROR Could not find name `binascii` [unknown-name]
+   --> libs/oauth.py:101:13
 ERROR Argument `Literal['normal']` is not assignable to parameter `status` with type `ConversationStatus | SQLCoreOperations[ConversationStatus]` in function `models.model.Conversation.__init__` [bad-argument-type]
   --> tests/e2e_tests/containers/controllers/console/app/test_chat_conversation_status_count_api.py:75:16
 ERROR Argument `Literal['app-run']` is not assignable to parameter `triggered_from` with type `SQLCoreOperations[WorkflowRunTriggeredFrom] | WorkflowRunTriggeredFrom` in function `models.workflow.WorkflowRun.__init__` [bad-argument-type]
@@ -1981,7 +1983,7 @@
 ERROR Object of class `NoneType` has no attribute `pre_processing_rules` [missing-attribute]
     --> tests/helpers/unit/services/document_service_validation.py:1388:9
 ERROR Object of class `NoneType` has no attribute `segmentation` [missing-attribute]
-    --> tests/helpers/unit/services/document_service_validation.py:1464:9
+    --> tests/helpers/unit/services/document_service_validation.py:1437:9
 ERROR Cannot index into `list[Unknown]` [bad-index]
    --> tests/helpers/unit/services/hit_service.py:423:20
 ERROR Cannot index into `object` [bad-index]
@@ -4540,8 +4542,20 @@
    --> tests/unit_tests/core/test_provider_manager.py:569:37
 ERROR Argument `list[LoadBalancingModelConfig]` is not assignable to parameter `load_balancing_model_configs` with type `list[_LoadBalancingModelConfigCacheEntry] | None` in function `core.provider_manager.ProviderManager._to_model_settings` [bad-argument-type]
    --> tests/unit_tests/core/test_provider_manager.py:570:42
+ERROR Argument `Literal['llm']` is not assignable to parameter `model_type` with type `ModelType | SQLCoreOperations[ModelType]` in function `models.provider.ProviderModelSetting.__init__` [bad-argument-type]
+   --> tests/unit_tests/core/test_provider_manager.py:587:20
+ERROR Argument `Literal['llm']` is not assignable to parameter `model_type` with type `ModelType | SQLCoreOperations[ModelType]` in function `models.provider.LoadBalancingModelConfig.__init__` [bad-argument-type]
+   --> tests/unit_tests/core/test_provider_manager.py:597:24
+ERROR Argument `Literal['llm']` is not assignable to parameter `model_type` with type `ModelType | SQLCoreOperations[ModelType]` in function `models.provider.LoadBalancingModelConfig.__init__` [bad-argument-type]
+   --> tests/unit_tests/core/test_provider_manager.py:606:24
+ERROR Argument `Literal['llm']` is not assignable to parameter `model_type` with type `ModelType | SQLCoreOperations[ModelType]` in function `models.provider.LoadBalancingModelConfig.__init__` [bad-argument-type]
+   --> tests/unit_tests/core/test_provider_manager.py:615:24
+ERROR Argument `list[ProviderModelSetting]` is not assignable to parameter `provider_model_settings` with type `list[_ProviderModelSettingCacheEntry] | None` in function `core.provider_manager.ProviderManager._to_model_settings` [bad-argument-type]
+   --> tests/unit_tests/core/test_provider_manager.py:635:37
+ERROR Argument `list[LoadBalancingModelConfig]` is not assignable to parameter `load_balancing_model_configs` with type `list[_LoadBalancingModelConfigCacheEntry] | None` in function `core.provider_manager.ProviderManager._to_model_settings` [bad-argument-type]
+   --> tests/unit_tests/core/test_provider_manager.py:636:42
 WARN Identity comparison `False is True` is always False [unnecessary-comparison]
-    --> tests/unit_tests/core/test_provider_manager.py:1309:30
+    --> tests/unit_tests/core/test_provider_manager.py:1387:30
 ERROR Class member `DummyTool.get_runtime_parameters` overrides a member in a parent class but is missing an `@override` decorator [missing-override-decorator]
   --> tests/unit_tests/core/tools/__base/test_tool.py:71:9
 ERROR Object of type `ToolInvokeMessage.BlobChunkMessage | ToolInvokeMessage.BlobMessage | ToolInvokeMessage.FileMessage | ToolInvokeMessage.JsonMessage | ToolInvokeMessage.LogMessage | ToolInvokeMessage.RetrieverResourceMessage | ToolInvokeMessage.TextMessage | ToolInvokeMessage.VariableMessage | None` has no attribute `text` [missing-attribute]

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Pyrefly Type Coverage

Metric Base PR Delta
Type coverage 82.88% 82.88% +0.00%
Strict coverage 82.54% 82.54% +0.00%
Typed symbols 85,474 85,481 +7
Untyped symbols 17,725 17,726 +1
Modules 4373 4373 0

@codecov

codecov Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.75%. Comparing base (992f7dc) to head (8635d5c).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #44053      +/-   ##
==========================================
+ Coverage   73.31%   82.75%   +9.44%     
==========================================
  Files        6239     6193      -46     
  Lines      390948   383074    -7874     
  Branches    79291    78820     -471     
==========================================
+ Hits       286615   317006   +30391     
+ Misses      98235    60301   -37934     
+ Partials     6098     5767     -331     
Flag Coverage Δ
api 77.50% <100.00%> (-12.14%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

mesrop2008 and others added 15 commits October 14, 2026 15:20
Add unit tests for the except ValueError paths simplified in the
previous commit so the patch is fully covered:

- ProviderManager skips load-balancing configs and returns empty
  provider credentials when the stored JSON is malformed.
- parse_file_reference falls back to the raw value for invalid base64
  or JSON payloads, and round-trips canonical references.
- The datasource credential repository rejects stored documents whose
  data_source_info cannot be parsed.
- Document change_status maps ValueError and DocumentIndexingError to
  DocumentInvalidActionError.
…nt-except-and-unwrap

# Conflicts:
#	api/controllers/console/app/workflow.py
#	api/controllers/console/datasets/rag_pipeline/rag_pipeline_workflow.py
…nt-except-and-unwrap

# Conflicts:
#	api/controllers/console/app/workflow.py
#	api/controllers/console/datasets/rag_pipeline/rag_pipeline_workflow.py
#	api/controllers/console/explore/trial.py
#	api/tests/unit_tests/controllers/files/test_plugin_file_upload.py
#	api/tests/unit_tests/core/test_provider_manager.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review: medium ghfind author score; see https://ghfind.com size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

some new type errors

2 participants