Skip to content

Commit eff77e8

Browse files
author
luk384090-cloud
committed
refactor(shared): update shared module docs and migrate subpackage implementations
- Update _shared module documentation to clarify its role as state shared between the two stacks - Migrate the agentstudio subpackage implementation to dashscope.v2.agentstudio - Add a compatibility layer for agentstudio to preserve historical import paths - Migrate aigc, app, assistants, audio, cli, common, embeddings and other subpackage implementations to their corresponding legacy_1x modules - Add backward-compatible import shims for each subpackage - Remove finetune/reinforcement-related demo and parser components - Update the import structure of the finetune/reinforcement module
1 parent f7d05bf commit eff77e8

373 files changed

Lines changed: 6684 additions & 3362 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎MANIFEST.in‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
include dashscope/resources/qwen.tiktoken
1+
include dashscope/v2/tokenizers/qwen.tiktoken
22
include dashscope/py.typed
33
include dashscope/__init__.pyi
44
include NOTICE

‎dashscope/_shared/__init__.py‎

Lines changed: 23 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,28 @@
22
# Copyright (c) Alibaba, Inc. and its affiliates.
33
"""Neutral leaf layer shared by the 1.x API and the 2.0 implementation.
44
5-
Modules here hold state or values that must exist exactly once per
6-
installed distribution (the process-wide session id, the version
7-
number). Copying them into either stack would silently split that single
8-
source of truth, so both stacks import from here instead.
5+
What belongs here is anything the two stacks must agree on exactly once
6+
per installed distribution: the process-wide session id, the mutable
7+
client tag, the exception base class that ``except`` clauses on both
8+
sides match against, and the validators that keep a request pointed at
9+
``*.maas.aliyuncs.com``. Copying one of those into either stack would
10+
silently split a single source of truth, so both stacks import from
11+
here instead.
912
10-
Nothing in this package may import ``dashscope`` or any of its
11-
subpackages: it sits below both stacks and is the anchor that keeps them
12-
from forming an import cycle. Stdlib only.
13+
The only thing this package reaches outside itself is
14+
``dashscope.version``, read defensively by ``_sdk_ident``. That module
15+
is a bare constant with no imports of its own, and it deliberately stays
16+
where it is: ``setup.py`` and ``docs/source/conf.py`` ``exec()`` the
17+
file to read the version *without* importing the package, and turning
18+
it into a re-export of something here would drag the whole SDK and its
19+
runtime dependencies into any build environment that only wanted a
20+
string.
21+
22+
What this package must never import is the 1.x business surface
23+
(``client``, ``common``, ``finetune``, ``audio`` and the rest).
24+
``common/utils`` reaches ``_sdk_ident`` while ``dashscope/__init__`` is
25+
still executing, so an import back into that surface would close a cycle
26+
through a partially initialized root. ``dashscope.version`` is safe
27+
precisely because it has no imports of its own; the business modules
28+
all have plenty.
1329
"""

‎dashscope/_shared/_errors.py‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# -*- coding: utf-8 -*-
2+
# Copyright (c) Alibaba, Inc. and its affiliates.
3+
"""The one exception base class both stacks inherit from.
4+
5+
``except dashscope.legacy_1x.common.error.DashScopeException`` has to catch errors
6+
raised by the 1.x API classes *and* by the AgentStudio sub-SDK, which the
7+
2.0 client mounts directly. That only works if every subclass in both
8+
stacks derives from the same class object, so the base lives here rather
9+
than in either stack: copying it would silently split the hierarchy and
10+
let errors escape a caller's ``except`` clause.
11+
12+
``dashscope.legacy_1x.common.error`` re-exports this name, so the 1.x import path
13+
keeps working and keeps the identical class.
14+
"""
15+
16+
17+
class DashScopeException(Exception):
18+
"""Base class for every error the dashscope SDK raises."""
19+
20+
21+
__all__ = ["DashScopeException"]

‎dashscope/_shared/_sdk_ident.py‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
# -*- coding: utf-8 -*-
2+
# Copyright (c) Alibaba, Inc. and its affiliates.
3+
"""SDK identification headers shared by the 1.x API and the 2.0 package.
4+
5+
``x-dashscope-sdk-client`` tells the backend which client and which
6+
functional area issued a request, and ``x-dashscope-sdk-session-id``
7+
groups every request one interpreter process makes.
8+
9+
The client segment is process-wide *mutable* state: entry points that
10+
embed the SDK overwrite it (the ``dashscope`` CLI marks itself as
11+
``python-cli`` via ``set_sdk_client``), and every caller in both stacks
12+
must see the same value. Duplicating it would silently drop the override
13+
for whichever stack held the copy, so it lives here -- one source of
14+
truth, same reason as the session id in ``_shared._sdk_session``.
15+
16+
The 2.0 transport builds its own header in
17+
``dashscope.v2.transport._ident`` because it appends a trailing
18+
generation marker (``aigc`` -> ``aigc2``) that this unmarked 1.x shape
19+
must not carry. Sub-SDKs that predate the marker -- AgentStudio, whose
20+
``agentstudio`` tag the backend already reports on -- keep using this
21+
module.
22+
23+
``get_user_agent`` belongs here for the same reason as the two headers:
24+
it is the other half of the request identity, and it must agree with
25+
``get_sdk_headers`` on the version segment. It is the *1.x shape*
26+
(``dashscope/<version>; python/...; platform/...; processor/...``) and is
27+
deliberately not shared with the 2.0 transport's own
28+
``dashscope.v2.transport.http.default_user_agent()``
29+
(``dashscope-v2-python/<version>``). Those are two distinct backend
30+
contracts, not two spellings of one value, so they must not be merged.
31+
32+
Only the stdlib is imported at module level; ``dashscope.version`` is
33+
pulled in lazily and defensively so this leaf stays below both stacks.
34+
"""
35+
36+
import os
37+
import platform
38+
from typing import Dict
39+
40+
from dashscope._shared._sdk_session import sdk_session_id
41+
42+
_SDK_CLIENT = "python-sdk"
43+
44+
# Shared with the 2.0 transport so one process reports one
45+
# x-dashscope-sdk-session-id across both stacks (see _shared._sdk_session;
46+
# acli deliberately keeps its own, it is a one-way sync copy).
47+
_SDK_SESSION_ID = sdk_session_id()
48+
49+
__all__ = ["get_sdk_headers", "get_user_agent", "set_sdk_client"]
50+
51+
52+
def _sdk_version() -> str:
53+
try:
54+
from dashscope.version import __version__
55+
except Exception: # pragma: no cover - defensive import guard
56+
return "0.0.0"
57+
return __version__
58+
59+
60+
# Read once at import so the value is stable for the process lifetime.
61+
_VERSION = _sdk_version()
62+
63+
64+
def set_sdk_client(client: str) -> None:
65+
"""Override the client identifier in the x-dashscope-sdk-client header.
66+
67+
Process-wide, used by entry points that embed the SDK (e.g. the
68+
``dashscope`` CLI marks itself as ``python-cli``).
69+
"""
70+
global _SDK_CLIENT # pylint: disable=global-statement
71+
if client:
72+
_SDK_CLIENT = client
73+
74+
75+
def get_sdk_headers(module: str = "") -> Dict[str, str]:
76+
"""Structured SDK identification headers for backend statistics.
77+
78+
Set DASHSCOPE_DISABLE_SDK_HEADERS=1 to omit them.
79+
80+
Args:
81+
module: Optional module name (e.g. agentstudio, rl, app).
82+
"""
83+
if os.environ.get("DASHSCOPE_DISABLE_SDK_HEADERS"):
84+
return {}
85+
parts = [_SDK_CLIENT, _VERSION]
86+
if module:
87+
parts.append(module)
88+
return {
89+
"x-dashscope-sdk-client": "/".join(parts),
90+
"x-dashscope-sdk-session-id": _SDK_SESSION_ID,
91+
}
92+
93+
94+
def get_user_agent() -> str:
95+
"""The 1.x-shaped ``user-agent`` value.
96+
97+
Kept byte-identical to the pre-move implementation: the version
98+
segment is ``_VERSION`` here rather than a second read of
99+
``dashscope.version``, so it can never drift from the one in
100+
``get_sdk_headers``.
101+
"""
102+
try:
103+
platform_ = (
104+
platform.platform().replace("\n", "").replace("\r", "").strip()
105+
)
106+
except Exception:
107+
platform_ = "unknown"
108+
109+
try:
110+
processor_ = (
111+
platform.processor().replace("\n", "").replace("\r", "").strip()
112+
)
113+
except Exception:
114+
processor_ = "unknown"
115+
116+
ua = (
117+
f"dashscope/{_VERSION}; python/{platform.python_version()}; "
118+
f"platform/{platform_}; "
119+
f"processor/{processor_}"
120+
)
121+
return ua

‎dashscope/_shared/_sdk_session.py‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,18 +5,18 @@
55
The ``x-dashscope-sdk-session-id`` header lets server-side access logs
66
group every request one SDK session makes. A process can emit requests
77
through both stacks at once -- the 1.x API classes and the 2.0 transport
8-
-- and the 2.0 client even delegates some calls (rl, agentstudio)
9-
straight into 1.x. If each stack minted its own id, one logical session
10-
would split across several ids and the grouping breaks.
8+
-- and the sub-SDKs owned by 2.0 (AgentStudio, agentic RL) build their
9+
request headers from ``_shared._sdk_ident`` rather than from the v2
10+
transport. If each stack minted its own id, one logical session would
11+
split across several ids and the grouping breaks.
1112
1213
This leaf module is the one source of truth both import. It depends only
1314
on the stdlib so it can be pulled in from anywhere without dragging
1415
``dashscope/__init__`` (and its v2 surface) into a cycle.
1516
1617
The embedded ``acli`` deliberately keeps its own id: it is a one-way
17-
sync copy of the agenticCLI repo (design doc §3.2), so wiring it to this
18-
module would be silently reverted by the next sync and cannot be
19-
upstreamed as written.
18+
sync copy of the agenticCLI repo, so wiring it to this module would be
19+
silently reverted by the next sync and cannot be upstreamed as written.
2020
"""
2121

2222
from __future__ import annotations

‎dashscope/_shared/_validation.py‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# -*- coding: utf-8 -*-
2+
# Copyright (c) Alibaba, Inc. and its affiliates.
3+
"""Host-label validation shared by the 1.x API and the 2.0 implementation.
4+
5+
``workspace_id`` and ``region`` are interpolated into the host of the MaaS
6+
endpoint, so both stacks must reject the same malformed values with the
7+
same rule. Keeping the check in one leaf module means a value that is
8+
safe for one stack is safe for the other, and neither stack has to import
9+
the other to get it.
10+
11+
Stdlib only, like the rest of ``dashscope._shared``.
12+
"""
13+
14+
import re
15+
16+
# workspace_id and region become DNS labels of the MaaS endpoint host, so
17+
# they must not contain characters (/, ?, #, @, :, whitespace, ...) that
18+
# could break out of the host and redirect requests elsewhere.
19+
# \A...\Z rather than ^...$: $ also matches before a trailing newline.
20+
_WORKSPACE_ID_PATTERN = re.compile(r"\A[A-Za-z0-9][A-Za-z0-9_-]{0,63}\Z")
21+
# RFC 1123 hostname label: 1-63 chars, no leading/trailing hyphen. Upper
22+
# case is allowed because DNS is case-insensitive and httpx lowercases the
23+
# host, so rejecting it would break working callers for no security gain.
24+
_REGION_PATTERN = re.compile(
25+
r"\A[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?\Z",
26+
)
27+
28+
__all__ = ["validate_region", "validate_workspace_id"]
29+
30+
31+
def validate_workspace_id(workspace_id):
32+
"""Validate that workspace_id is safe to use as a URL subdomain.
33+
34+
Raises:
35+
ValueError: If workspace_id contains characters outside
36+
[A-Za-z0-9_-], is empty, or exceeds 64 characters.
37+
"""
38+
if not workspace_id or not _WORKSPACE_ID_PATTERN.fullmatch(
39+
workspace_id,
40+
):
41+
raise ValueError(
42+
f"Invalid workspace_id {workspace_id!r}: must match "
43+
"^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$",
44+
)
45+
return workspace_id
46+
47+
48+
def validate_region(region):
49+
"""Validate that region is safe to use as a URL subdomain.
50+
51+
Deliberately a hostname-label check and not a MAAS_REGIONS whitelist:
52+
rejecting an unknown-but-well-formed region would break callers as soon
53+
as a subpackage supports a region this list has not caught up with.
54+
55+
Raises:
56+
ValueError: If region is not a valid hostname label.
57+
"""
58+
if not region or not _REGION_PATTERN.fullmatch(region):
59+
raise ValueError(
60+
f"Invalid region {region!r}: must be a hostname label "
61+
"(1-63 chars of [A-Za-z0-9-], not starting or ending "
62+
"with '-')",
63+
)
64+
return region

‎dashscope/acli/ui/tts.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
44
Supports both streaming playback (low-latency, audio starts before synthesis
55
finishes) and one-shot synthesis. Uses
6-
``dashscope.audio.tts_v2.SpeechSynthesizer``.
6+
``dashscope.legacy_1x.audio.tts_v2.SpeechSynthesizer``.
77
"""
88
# pylint: disable=too-many-return-statements,too-many-branches
99
# pylint: disable=too-many-statements
@@ -284,7 +284,7 @@ def _synthesize_and_play(self, text: str) -> Optional[str]:
284284
dashscope.api_key = self.api_key
285285

286286
try:
287-
from dashscope.audio.tts_v2 import (
287+
from dashscope.legacy_1x.audio.tts_v2 import (
288288
AudioFormat,
289289
ResultCallback,
290290
SpeechSynthesizer,

‎dashscope/acli/ui/voice.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,10 @@ class _RealtimeASR:
5858

5959
def __init__(self, api_key: str, model: str):
6060
os.environ.setdefault("DASHSCOPE_API_KEY", api_key)
61-
from dashscope.audio.asr import Recognition, RecognitionCallback
61+
from dashscope.legacy_1x.audio.asr import (
62+
Recognition,
63+
RecognitionCallback,
64+
)
6265

6366
self._model = model
6467
self._Recognition = Recognition

0 commit comments

Comments
 (0)