Skip to content

fix(core): let JacksonJsonCodec read back every string it can write - #3468

Open
wahllllll wants to merge 2 commits into
agentscope-ai:mainfrom
wahllllll:fix/json-codec-read-constraints
Open

wahllllll wants to merge 2 commits into
agentscope-ai:mainfrom
wahllllll:fix/json-codec-read-constraints

Conversation

@wahllllll

Copy link
Copy Markdown
Contributor

Symptom

An AgentState carrying inline Base64 media (the report's case: a 25 MB VideoBlock, 33,563,888 Base64 characters) saves successfully and then cannot be loaded. The load happens before the model is called, so every later turn of that session fails too — text-only turns included. The session is bricked by its own persisted state.

Reported in #3465 against OssAgentStateStore, with a standalone repro that needs no OSS, video or model.

Root cause

The store is not where the bug lives. Every AgentStateStore — JSON file, JDBC, Redis (Jedis/Redisson), MongoDB, OSS, COS and the Service JPA store — persists through JsonUtils.getJsonCodec(), i.e. the single default JacksonJsonCodec. Its createDefaultObjectMapper() is a bare new ObjectMapper(), so it inherits Jackson's default StreamReadConstraints: a single string value may not exceed 20,000,000 characters on read. Serialization has no such cap.

That is a round-trip asymmetry inside one codec: toJson accepts a state that the same codec's fromJson rejects with

StreamConstraintsException: String value length (…) exceeds the maximum allowed (20000000, from StreamReadConstraints.getMaxStringLength())

A Base64 payload crosses 20 M characters at ~15 MB of source bytes; an OpenAI-size image limit (20 MB) already does.

Fix

Build the default ObjectMapper from a JsonFactory whose read constraints set maxStringLength to Integer.MAX_VALUE, so the reader accepts whatever the writer emitted. Nesting depth, number length and name length keep their Jackson defaults, and a caller-supplied ObjectMapper (new JacksonJsonCodec(mapper)) is left untouched — only the framework's own default changes.

JsonFactory factory =
        JsonFactory.builder()
                .streamReadConstraints(
                        StreamReadConstraints.builder()
                                .maxStringLength(Integer.MAX_VALUE)
                                .build())
                .build();
ObjectMapper mapper = new ObjectMapper(factory);

Why the reader cap, not the writer

Jackson's read cap exists to bound memory from hostile input. Strings of this size only ever reach this codec from the framework's own persisted state: model output and tool arguments are bounded by provider output limits (well under 1 M characters), and HTTP bodies are bounded at the transport layer. So lifting the per-string cap on the default codec does not weaken an untrusted boundary — it removes a limit that only ever fired on data the framework itself wrote. Any finite ceiling below what toJson can emit re-creates the asymmetry for the next larger payload; if you would rather keep one, the single number to change is in createDefaultObjectMapper().

Offloading media out of AgentState (#2867) remains the right long-term answer; this PR restores the round-trip invariant the stores already rely on today.

Tests

Added roundTripsStringsLongerThanJacksonDefaultReadLimit to JacksonJsonCodecTest: a 20,000,001-character string (one over the old cap) through toJson → fromJson, compared back.

Red — on current main:

JacksonJsonCodecTest.roundTripsStringsLongerThanJacksonDefaultReadLimit <<< ERROR!
  Caused by: com.fasterxml.jackson.core.exc.StreamConstraintsException: String value length (20000001)
    exceeds the maximum allowed (20000000, from StreamReadConstraints.getMaxStringLength())
Tests run: 21, Failures: 0, Errors: 1

Green — with the fix:

JacksonJsonCodecTest: Tests run: 21, Failures: 0, Errors: 0 (20 existing + the new one)
io.agentscope.core.util.** : Tests run: 59, Failures: 0, Errors: 0 -- BUILD SUCCESS
agentscope-core full suite (minus the Windows symlink env test): Tests run: 2581, Failures: 0, Errors: 0 -- BUILD SUCCESS

spotless:apply applied. No configuration change and no database migration.

Fixes #3465

@codecov

codecov Bot commented Oct 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@wahllllll
wahllllll force-pushed the fix/json-codec-read-constraints branch from 39d84ae to 1e0171a Compare October 10, 2026 01:58
Jackson's default StreamReadConstraints reject any single string value above
20,000,000 characters on read, while serialization imposes no cap at all. Every
AgentStateStore -- JSON file, JDBC, Redis, MongoDB, OSS, COS, the Service JPA
store -- persists through JsonUtils.getJsonCodec(), i.e. this codec's default
ObjectMapper, and an AgentState carrying inline Base64 media (a 25 MB video is
33.5 million characters) saved successfully and then failed to load on the very
next turn. Because the load happens before the model is called, every later
turn of that session failed too, text-only ones included: a session bricked by
its own persisted state.

Configure the default ObjectMapper's JsonFactory with maxStringLength set to
Integer.MAX_VALUE so the reader accepts whatever the writer emitted. Nesting
depth, number length and name length keep their Jackson defaults; a caller-
supplied ObjectMapper is left untouched. Strings this large only ever reach the
codec from the framework's own persisted state -- model output and tool
arguments are bounded far below the old limit -- so no untrusted boundary
loses protection.

Offloading media out of AgentState (agentscope-ai#2867) remains the right long-term answer;
this restores the round-trip invariant the stores already rely on.

Fixes agentscope-ai#3465
@wahllllll
wahllllll force-pushed the fix/json-codec-read-constraints branch from 1e0171a to d853f25 Compare October 10, 2026 02:37
@wahllllll

Copy link
Copy Markdown
Contributor Author

The build (windows-latest) reds here are this repo's known timing flake, not this change. Three consecutive runs failed on three different tests:

run failing test
38012415750 SharedWorkspaceSkillStagingE2ETest.concurrentAgentsSharingWorkspaceBothComplete (2 agents × 50 concurrent calls, await(120s) timed out)
38015240237 ChatSessionsTest.pendingExternalRequestSurvivesRestartAndContinuesSameTurn (15 s await → Timed out: [Turn … status=running])
38017641209 ChatSessionsTest.interruptedTaskKeepsQueuedWorkUntilExplicitContinuation — the same test that fails on main in run 37902215993, with the identical Timed out: [Turn[turnId=7d47e214-… from the same 15 s await helper

Supporting evidence:

  • main is 8/12 green on Java CI with Maven; all four reds include build (windows-latest), failing on this same class of timing tests (ChatSessionsTest, ReActAgentTest.testAgentIdOverride). Over 26 recent failed runs, ReActAgentTest.testAgentIdOverride failed 8× and ChatSessionsTest 9× across 4 different methods — none of them related to this diff.
  • build (windows-latest) total time is unchanged: 31.8 min on main's last green run vs 31.9 / 33.4 min here, so this is not a uniformly slower runner.
  • JacksonJsonCodecTest passes 21/21 on Windows in all three runs, new test included.
  • The ChatSessionsTest failures land on :281 (Timed out: … status=running). That helper throws at :274 whenever snapshot.executionError() is non-null, so nothing was raised while reloading the persisted state — it simply did not finish within 15 s. build (ubuntu-latest) is green in all three runs.

Nothing in this diff touches the failing paths. I'm happy to re-run again, but given main's own streak I'd suggest treating the Windows job as the flake it appears to be rather than blocking on it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: OssAgentStateStore can persist large Base64 media but cannot reload the resulting AgentState

1 participant