Skip to content

fix(helpers): reshape 1-D float32 audio in LocalAudioPlayer - #3590

Open
hsusul wants to merge 2 commits into
openai:mainfrom
hsusul:fix/local-audio-player-float32-reshape
Open

hsusul wants to merge 2 commits into
openai:mainfrom
hsusul:fix/local-audio-player-float32-reshape

Conversation

@hsusul

@hsusul hsusul commented Aug 10, 2026

Copy link
Copy Markdown
Contributor
  • I understand that this repository is auto-generated and my pull request may not be merged

Changes being requested

LocalAudioPlayer (in src/openai/helpers/local_audio_player.py) mishandles 1-D float32 NumPy arrays.

Both play() and play_stream() reshape int16 input to (-1, channels), but pass float32 input through without reshaping. A 1-D float32 array — e.g. a mono waveform loaded via soundfile/librosa, and an explicitly declared input type of play() (npt.NDArray[np.float32]) — then breaks:

  • play() raises IndexError: tuple index out of range, because the still-1-D array is used as channels=audio_content.shape[1] when constructing the sounddevice.OutputStream.
  • play_stream() raises a broadcasting error inside the audio callback when assigning a 1-D slice into the 2-D outdata buffer.

int16 input already works because it is reshaped; only the float32 branch was missing the reshape.

Repro (play())

import asyncio
import numpy as np
from openai.helpers import LocalAudioPlayer

# a mono float32 waveform, e.g. from soundfile / librosa
samples = np.array([0.1, 0.2, 0.3, -0.4, -0.5], dtype=np.float32)
asyncio.run(LocalAudioPlayer().play(samples))
# IndexError: tuple index out of range   (at channels=audio_content.shape[1])

Fix

Reshape float32 input to (-1, self.channels) in both code paths, mirroring the existing int16 handling. Already-2-D input (e.g. the (-1, 1) buffer produced from a streamed TTS response) is unaffected — the reshape is a no-op.

             elif input.dtype == np.float32:
-                audio_content = cast("npt.NDArray[np.float32]", input)
+                audio_content = cast("npt.NDArray[np.float32]", input).reshape(-1, self.channels)
                         if current_buffer.dtype == np.int16 and self.dtype == np.float32:
                             current_buffer = (current_buffer.astype(np.float32) / 32767.0).reshape(-1, self.channels)
+                        elif current_buffer.dtype == np.float32:
+                            current_buffer = current_buffer.reshape(-1, self.channels)

Why this is a hand-maintained file (not generated)

src/openai/helpers/local_audio_player.py has no File generated from our OpenAPI spec by Stainless header and is not part of the OpenAPI surface — it is one of the hand-written helpers under helpers/. The change is a self-contained correctness fix in that helper; it touches no generated code.

Tests

Added tests/lib/test_local_audio_player.py with a lightweight, hardware-free fake for sounddevice.OutputStream that drives the real callback flow from a background thread:

  • test_play_accepts_one_dimensional_float32 — reproduces the IndexError (fails before the fix) and asserts the played frames match the input.
  • test_play_accepts_one_dimensional_int16 — regression guard for the existing int16 path.
  • test_play_stream_accepts_one_dimensional_float32 — covers the streaming path.

Each test fails on main for the exact reason described above and passes with the fix. No network, no API key, no audio device.

Validation (run locally)

  • pytest tests/lib/test_local_audio_player.py — 3 passed (and confirmed red on main: IndexError for play, broadcast/hang for play_stream).
  • pytest tests/lib/ — 259 passed.
  • ruff format --check / ruff check . — clean.
  • scripts/run-pyright -p pyproject.toml — 0 errors.
  • mypy . — Success: no issues.

Additional context & links

  • Compatibility: no public API change; only fixes a previously-crashing input shape. int16 and already-2-D float32 behavior is unchanged.
  • local_audio_player.py carries # mypy: ignore-errors, so mypy does not type-check it; the change is pyright-clean.

`LocalAudioPlayer.play()` and `play_stream()` reshape int16 input to
`(-1, channels)` but left float32 input untouched. A 1-D float32 array
(e.g. a mono waveform from soundfile/librosa, which is a valid input per
the `play()` type signature) then broke:

- `play()` raised `IndexError: tuple index out of range` from
  `channels=audio_content.shape[1]` when building the output stream.
- `play_stream()` raised a broadcasting error in the callback when
  assigning a 1-D slice into the 2-D `outdata` buffer.

Reshape float32 input to `(-1, channels)` in both paths, mirroring the
existing int16 handling. Already-2-D input is unaffected (no-op reshape).
@hsusul
hsusul requested a review from a team as a code owner August 10, 2026 05:17

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7603230061

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

audio_content = (input.astype(np.float32) / 32767.0).reshape(-1, self.channels)
elif input.dtype == np.float32:
audio_content = cast("npt.NDArray[np.float32]", input)
audio_content = cast("npt.NDArray[np.float32]", input).reshape(-1, self.channels)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve multi-channel float32 playback

When play() receives a 2-D float32 array such as (frames, 2) stereo audio, self.channels is hardcoded to 1, so this reshape flattens interleaved left/right samples into a single channel with twice as many frames. The subsequent OutputStream therefore plays corrupted mono audio instead of the stereo stream supported by the previous audio_content.shape[1] behavior. Reshape only 1-D input, while preserving the channel dimension of existing 2-D arrays.

AGENTS.md reference: AGENTS.md:L5-L8

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 472919a. The float32 reshape now only applies to 1-D input, so a 2-D array keeps its channel dimension and play() still opens the stream with shape[1] channels. The same guard is in play_stream. Added test_play_preserves_two_dimensional_float32_channels, which fails without the guard.

@jnohclee-rgb

Copy link
Copy Markdown

AI-assisted offline comparison at head 7603230061873c3c692069d0b7807246f4a5c3cf against parent 0c09a3fe815184f0a46fbf18b1aba84a467c854e, using Python 3.14.7 and NumPy 2.5.3. I replaced the module's sounddevice binding with a fake OutputStream that invokes the real callback on zero-filled output arrays; no audio devices or network were accessed.

Four play() cases observed:

Input / player setting Parent PR head
float32 [1,2,3,4], default mono IndexError before opening stream mono samples preserved, tail zero-filled
float32 [[1,10],[2,20]], explicitly set player.channels = 2 two-channel frames preserved same frames preserved
same two-dimensional array, default LocalAudioPlayer() two-channel frames preserved one channel containing [1,10,2,20]
float32 [1,2,3], explicitly set player.channels = 2 IndexError before opening stream ValueError before opening stream

The one-dimensional fix is confirmed in this harness. The third case may deserve an explicit test or documented contract: the new reshape also flattens already two-dimensional float32 input when its channel dimension differs from self.channels. If existing array shape is meant to determine playback channels, limiting the reshape to one-dimensional inputs would preserve that behavior; if the player setting should be authoritative, the changed behavior should be intentional.

Minimal array for the third case:

player = LocalAudioPlayer()  # channels defaults to 1
samples = np.array([[1, 10], [2, 20]], dtype=np.float32)
await player.play(samples)

Values are fictional markers, not recorded audio. Setting player.channels above is an explicit attribute mutation, not a constructor option. This checks play() only, not threaded native playback, play_stream(), device behavior, or the full suite.

A 2-D float32 array already carries its channel dimension, which play()
uses to open the output stream. Reshaping it to self.channels flattened
stereo input into mono.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-19T01:47:00.727013Z 472919a New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants