fix(client): point the dataset-run helpers at the v4 read path in docs and logs

[passionworkeer]

fix(client): point the dataset-run helpers at the v4 read path in docs and logs

Fixes #1906.

What is wrong

On a Langfuse v4 deployment the three dataset-run read helpers cannot work: the
endpoints behind them are rejected with 404 in events_only mode. Two things made
that a bad experience rather than a merely inconvenient one.

The docstrings said nothing about v4. So the only signal a caller got was a
404 whose Python str() begins with the response headers — the server's own
explanation is only reachable through exc.body.

The error handlers were dead code. All three helpers were written as:

except Error as e:
    handle_fern_exception(e)
    raise e

where Error is langfuse.api.Error, the fern-generated base class. The errors
the SDK actually raises derive from langfuse.api.core.api_error.ApiError:

>>> from langfuse.api import Error, NotFoundError
>>> from langfuse.api.core.api_error import ApiError
>>> issubclass(NotFoundError, ApiError)
True
>>> issubclass(NotFoundError, Error)
False

So the clause never fired for a server rejection, and handle_fern_exception —
which only logs — was never called either.

What this changes

1. Each of the three docstrings now states that the endpoint is unavailable on v4,
   names client.api.experiments.list() / list_items() as the replacement, notes
   that the dataset_run_id returned by run_experiment() is the same value those
   return, and links the migration guide.

2. The three clauses now catch ApiError, the type actually raised, instead of the
   fern base class that never matched.

3. All three delegate to _handle_dataset_run_error(e), which decides once per
   failure what to do: a recognized v4 rejection logs the actionable WARNING
   naming the replacement, an ordinary 404 logs nothing, and every other status
   continues through handle_fern_exception as originally intended. delete_dataset_run
   passes its own hint, because there is no delete counterpart on
   client.api.experiments and pointing the caller at the read path could lead them
   to conclude the run had been removed. The exception type is unchanged, so
   anything already catching NotFoundError keeps working.

Routing the helpers themselves onto the v4 read path is deliberately not part of
this change. It is not a drop-in: DatasetRunItem.id has no experiment counterpart,
dataset_name needs a datasets.list() lookup, metadata needs fields=metadata,
startTime/endTime are clipped to the requested from_start_time window, and
pagination is cursor-based instead of page-based. That mapping is a product decision
and belongs to a maintainer.

Scope note

client.py contained eight except Error clauses with this latent bug. The
three dataset-run helpers are fixed here; the five left alone are get_dataset,
auth_check, create_dataset, create_dataset_item and create_prompt. Widening
them alters error-logging behaviour across unrelated methods, so it is left for a
separate decision.

404 is not routed through the error logger

handle_fern_exception maps a bare status onto a generic message, and its 404 entry
reads "Internal error occurred. This is an unusual occurrence and we are monitoring it
closely." These three helpers raise NotFoundError routinely — asking for a run that
does not exist is an ordinary outcome — so routing 404 through it would ship a false
claim at ERROR level, which is also what error alerting keys on. A v4 refusal and a 404
therefore log the actionable warning (or nothing), while every other status still goes
through handle_fern_exception as originally intended.

Verification

Against a real self-hosted Langfuse v4 events_only deployment (langfuse-web
4.42.0) on base 0bc5897b:

• run_experiment() returns a populated dataset_run_id with item_count=2 and a
  dataset_run_url; the same client then fails to read that run back through
  get_dataset_run() with the v4 404.
• Before: get_dataset_run, get_dataset_runs and delete_dataset_run all raised
  NotFoundError with no guidance reachable, and the error-log handler never ran.
• After: all three emit the v4 guidance, and client.api.experiments.list(...) — the
  replacement the guidance names — returns ExperimentsResponse on the same
  deployment.

New unit tests in tests/unit/test_dataset_run_v4_guidance.py:

• the three docstrings name the replacement and the migration guide;
• the v4 guidance fires on the real server body (pinned verbatim) and stays quiet on an
  ordinary 404, on missing bodies and on non-dict bodies;
• a refused delete does not receive the read hint — there is no delete counterpart on
  client.api.experiments, so pointing the caller at the read path could lead them to
  conclude the run had been removed;
• the ApiError / Error class relationship that caused the dead handlers;
• behavioural tests that drive each helper through a stand-in generated client and assert
  a v4 refusal logs the guidance and never "Internal error", an ordinary 404 logs neither,
  and a non-404 API error still reaches the original logger.

The dataset_run_id == experiment id claim was checked against the live deployment:
the experiment returned by experiments.list() carries the same id, and it resolves both
by id and by name.

The focused guidance suite recorded 25 passed during the initial implementation. The
correction at 353c00c4 records 32 passed across
test_dataset_run_v4_guidance.py and test_datasets.py, with ruff check,
ruff format --check and mypy passing. That correction scopes the migration hint to the
dataset, guards a missing status_code before the integer comparison, and formats the
helper signature. client.py has 563 pre-existing ruff findings, identical before and
after this change.

These are historical local and contributor results. The current-head GitHub Actions runs
are action_required with no jobs executed; upstream CI has not yet run on this revision.

Confidence Score: 4/5

The runtime change appears safe to merge, but the repository’s import-placement requirement must be satisfied first.

Summary

The PR adds v4 migration guidance to three legacy dataset-run helpers and catches generated API errors so a recognized v4 rejection produces an actionable warning. It also adds focused regression tests.

_{Reviews (1) · Last reviewed commit: "fix(client): point the dataset-run helpe..."}

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[greptile-apps]

Run name lacks dataset scope The old lookup uses both dataset_name and run_name, but this guidance suggests matching an experiment by name alone. If two datasets use the same run name, that match is ambiguous and a caller could select the wrong run. Advise callers to include dataset_id when matching by name, or to use the run ID.

Knowledge Base Used: Dataset and run management

Prompt To Fix With AI

This is a comment left during a code review.
Path: langfuse/_client/client.py
Line: 2608-2611

Comment:
**Run name lacks dataset scope** The old lookup uses both `dataset_name` and `run_name`, but this guidance suggests matching an experiment by name alone. If two datasets use the same run name, that match is ambiguous and a caller could select the wrong run. Advise callers to include `dataset_id` when matching by name, or to use the run ID.

**Knowledge Base Used:** [Dataset and run management](https://app.greptile.com/personal-org-4986/-/custom-context/knowledge-base/langfuse/langfuse-python/-/docs/dataset-and-run-management.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[passionworkeer]

Good catch — fixed in 353c00c.

The hint now names dataset_id explicitly, says why a bare name match is ambiguous when two datasets reuse a run name, and points at id as the reliable key (it is the same value run_experiment() returns as dataset_run_id). This also lines up get_dataset_run with get_dataset_runs, which already passed dataset_id.

Two unrelated things CI caught in the same file while I was in there: _is_not_found handed a possibly-None status_code to int() (mypy rejects it), and _handle_dataset_run_error's signature fits on one line per ruff format. Both fixed in the same commit.