fix: numeric fields with @JsonFormat(shape=STRING) produced invalid JSON (#7734) - #7779
fix: numeric fields with @JsonFormat(shape=STRING) produced invalid JSON (#7734)#7779fudianchn wants to merge 1 commit into
Conversation
wenshao
left a comment
There was a problem hiding this comment.
Not explored to full depth (tool budget reached): "This PR fixes issue #7734: a BigDecimal field annotated…": verify whether JSONB + BeanToArray is actually a supported/reachable combination (would confirm reachability of finding 1).; "This PR fixes issue #7734: a BigDecimal field annotated…": verify whether JSONWriterUTF16.writeDecimal(BigDecimal, long, DecimalFormat) merges the features param with context features (the BigDecimal ASM branch's writer…; "This PR fixes issue #7734: a BigDecimal field annotated…": dynamic/static JIT inlining measurement of the grown FieldWriterDouble/Float.writeValue (estimated growth ~30 bytes; method likely exceeded MaxInlineSize both b…; "This PR fixes issue #7734: a BigDecimal field annotated…": verify whether JSONB + BeanToArray is actually a supported/reachable combination (confirms reachability of the finding above); "This PR fixes issue #7734: a BigDecimal field annotated…": verify whether JSONWriterUTF16/UTF8.writeDecimal(BigDecimal, long, DecimalFormat) merges the explicit features parameter with context features (writer-level b…, and 7 more.
— qwen3.8-max via Qwen Code /review (v0.21.11)
| * the formatted-string branch and emitted the raw pattern text unquoted. The | ||
| * fix keeps {@code WriteNonStringValueAsString} (set for shape=STRING) in charge | ||
| * and never builds a {@code DecimalFormat} from the sentinel, in every place a | ||
| * {@code DecimalFormat} is constructed from a field format. |
There was a problem hiding this comment.
[Critical] List<BigDecimal>/List<Double>/List<Float> fields annotated @JsonFormat(shape = STRING) still serialize to invalid JSON after this PR — probe with the real Jackson annotation on this head: {"amounts":[string200,string7]} / {"values":[string2,string2]} (unparseable, the exact defect class of #7734). The sentinel reaches new DecimalFormat(format) through FieldWriterList.getItemWriter → ObjectWriterProvider.getObjectWriter(Type, String, Locale) (ObjectWriterProvider.java:461-472), the one DecimalFormat-from-field-format site the new guard does not cover. The leak is pre-existing (both files are untouched here), but the claim in this Javadoc — "in every place a DecimalFormat is constructed from a field format" — is what makes it actionable in this PR.
Suggested fix:
Either extend the sentinel exclusion to ObjectWriterProvider.getObjectWriter(Type, String, Locale) (treat "string" like a null format for Double/Float/BigDecimal, returning the INSTANCE writers, and add a List regression case), or narrow this claim to the sites this PR patches.
— qwen3.8-max via Qwen Code /review (v0.21.11)
There was a problem hiding this comment.
Confirmed on the previous head: List<BigDecimal> -> {"amounts":[string200,string7]}, List<Double> -> {"values":[string2,string2]}. The sentinel reached new DecimalFormat(format) through FieldWriterList.getItemWriter -> ObjectWriterProvider.getObjectWriter(Type, String, Locale), the one remaining DecimalFormat-from-field-format site (the others are covered by the FieldWriter constructor and the static getObjectWriter guards; ObjectWriterImplDouble builds one only from the context date format when it contains #).
Fixed by returning the format-less INSTANCE writers for the "string" and "millis" sentinels in ObjectWriterProvider.getObjectWriter: the list loop passes the field features down to the item writers, so writeDecimal / ObjectWriterImplDouble.write see WriteNonStringValueAsString and quote the items. Added listBigDecimalShapeStringQuoted / listDoubleShapeStringQuoted / listFloatShapeStringQuoted asserting exact JSON, plus a JSONB round-trip. Both leaks are pre-existing at the merge base; included here because the same sentinel guards are touched.
|
|
||
| DecimalFormat decimalFormat = null; | ||
| if (format != null | ||
| && !"string".equals(format) |
There was a problem hiding this comment.
[Critical] The sibling sentinel "millis" reproduces the same defect through this new guard: BeanUtils.processJacksonJsonFormat sets format = "millis" unconditionally for Jackson @JsonFormat(shape = NUMBER) (BeanUtils.java:2944-2948, no field-type check), and it reaches new DecimalFormat("millis") at the exact sites this diff edits. Probe on this head: BigDecimal field → {"amount":millis200}, Double/Float field → {"value":millis2} — invalid JSON for a standard Jackson annotation. Pre-existing behavior (identical at the merge base), but the same defect class this PR eliminates, flowing through these lines. Note "millis" is a legitimate fastjson2 date format, so the fix belongs in the DecimalFormat-construction guards — which only ever see float/double/BigDecimal field classes — not in removing the sentinel.
Suggested fix:
Extend the exclusion at all three patched sites: && !"string".equals(format) && !"millis".equals(format) here, and format == null || format.isEmpty() || "string".equals(format) || "millis".equals(format) in the static getObjectWriter BigDecimal/BigDecimal[] branches — or guard on field type in BeanUtils.processJacksonJsonFormat so shape=NUMBER only assigns "millis" to date/time fields. A probe with the guard extended produced valid output without disturbing the shape=STRING fix.
— qwen3.8-max via Qwen Code /review (v0.21.11)
There was a problem hiding this comment.
Confirmed on the previous head: BigDecimal -> {"amount":millis200}, Double / Float -> {"value":millis2} (new DecimalFormat("millis").format(200) returns millis200). Pre-existing at the merge base, same defect class.
Fixed with the first suggested variant: the exclusion now covers "millis" at every construction site touched here (FieldWriter constructor, static getObjectWriter BigDecimal / BigDecimal[] branches, ObjectWriterProvider.getObjectWriter), so numeric fields with shape=NUMBER serialize as plain numbers ({"amount":200}, {"value":2.0}). "millis" stays untouched for date fields: these guards only apply to float / double / BigDecimal classes and FieldWriterDate keeps handling it. The second variant (type guard in BeanUtils.processJacksonJsonFormat so shape=NUMBER only assigns "millis" to date/time fields) also works and is the root-cause fix; I can switch to it or split it into a follow-up if you prefer that direction.
| if (BigDecimal[].class == valueClass) { | ||
| if (format == null || format.isEmpty()) { | ||
| if (format == null || format.isEmpty() || "string".equals(format)) { | ||
| return new ObjectWriterArrayFinal(BigDecimal.class, null); |
There was a problem hiding this comment.
[Suggestion] This changed branch is reachable — probe: an Object/Number field with format="string" holding a runtime BigDecimal[] routes here via FieldWriterObject and now emits valid quoted ["200","6.56"] — but nothing in core/src/test combines BigDecimal[] with shape=STRING. Pre-fix, this branch built new ObjectWriterArrayFinal(BigDecimal.class, new DecimalFormat("string")) and emitted invalid per-element output; if a future refactor re-drops the "string" check only here, the #7734 bug silently returns for array values while the entire suite stays green.
Suggested fix:
Add a test mirroring the polymorphic ones: @JsonFormat(shape = STRING) public Object value; holding new BigDecimal[]{new BigDecimal("1.23")}, asserting the exact serialized JSON.
— qwen3.8-max via Qwen Code /review (v0.21.11)
There was a problem hiding this comment.
Added polymorphicObjectBigDecimalArrayShapeString: an Object field annotated @JsonFormat(shape = STRING) holding new BigDecimal[]{1.23, 4.56} asserts {"amount":["1.23","4.56"]}. Also added polymorphicObjectBigDecimalShapeNumber for the "millis" side of the same static branch.
| public void floatShapeStringJsonbRoundTrip() { | ||
| FloatBean bean = new FloatBean(); | ||
| bean.value = 1.5F; | ||
| byte[] jsonb = JSON.toJSONBytes(bean); |
There was a problem hiding this comment.
[Suggestion] floatShapeStringJsonbRoundTrip and bigDecimalShapeStringJsonbRoundTrip claim JSONB coverage in name and comment, but JSON.toJSONBytes(Object) uses JSONWriter.ofUTF8 (JSON.java:3426) and produces text UTF-8 — probe: first byte 0x7b ({), versus 0xa6 for JSONB.toBytes on the same bean. The file has no JSONB import, so the suite has zero actual JSONB coverage while appearing to have it. That gap is material: JSONB writers diverge in exactly this feature area (JSONWriterJSONB.writeDouble/writeFloat ignore WriteNonStringValueAsString; writeDecimal ignores features and format), so a JSONB-side regression in the changed writeValue paths would pass this suite undetected.
Suggested fix:
Use JSONB.toBytes(bean) / JSONB.parseObject(bytes, ...) in these two tests, or rename them to reflect that they cover UTF-8 JSON bytes.
— qwen3.8-max via Qwen Code /review (v0.21.11)
There was a problem hiding this comment.
Correct: JSON.toJSONBytes uses JSONWriter.ofUTF8, so those two tests covered UTF-8 text bytes, not JSONB. Both now use JSONB.toBytes / JSONB.parseObject, and a new listBigDecimalShapeStringJsonbRoundTrip covers the collection path in JSONB as well.
| } else { | ||
| jsonWriter.writeDouble(doubleValue); | ||
| long features = this.features | jsonWriter.getFeatures(); | ||
| if ((features & Feature.WriteNonStringValueAsString.mask) != 0) { |
There was a problem hiding this comment.
[Suggestion] This new branch honors writer-level features (| jsonWriter.getFeatures()), but the ASM BeanToArray twin bakes only field features, and JSONWriterJSONB.writeDouble/writeFloat never consult the feature. Probe with the feature set at writer level only (JSONB + BeanToArray, unannotated Double/Float field): the reflection creator writes JSONB string elements ["1.5","2.25"], while the default ASM creator writes binary BP_DOUBLE/BP_FLOAT — same input, different bytes, and before this diff both creators wrote binary, so the divergence is introduced here. Impact is bounded (both encodings round-trip through JSONB.parseObject; trigger = non-default reflection creator + JSONB + BeanToArray + writer-level-only feature), hence Suggestion.
Suggested fix:
Make the creators agree: have the ASM path OR in runtime writer features (precedent for emitting jsonWriter.getFeatures() exists in the same file), or have JSONWriterJSONB.writeDouble/writeFloat consult the feature like the UTF8/UTF16 writers; at minimum add a JSONB BeanToArray test across both creators pinning the intended encoding.
— qwen3.8-max via Qwen Code /review (v0.21.11)
There was a problem hiding this comment.
Confirmed the divergence: reflection wrote 95 4c 31 2e 35 (array + string "1.5") while ASM wrote 95 b5 3f f8 00 00 00 00 00 00 (array + binary double), and before this change both wrote binary.
Went with the delegating variant: the ASM fast path cannot see writer-level features at class-generation time (supportDirectWrite already excludes the feature when it is known at bean or field level, so the gap is runtime-only). The generated writeArrayMappingJSONB now starts with a guard that delegates to ObjectWriterAdapter.writeArrayMappingJSONB, the reflective writeValue loop which merges writer features, when the runtime writer features contain WriteNonStringValueAsString. Both creators then produce the same bytes through the same code. jsonbBeanToArrayWritersAgreeOnWriteAsStringFeature pins byte-equality across creators with and without the feature.
The other variant, making JSONWriterJSONB.writeDouble / writeFloat consult the feature, would change the encoding for every writeDouble caller (root values, arrays) in JSONB, so I did not take it; happy to rework in that direction if you prefer it.
8618be4 to
c1fc57e
Compare
wenshao
left a comment
There was a problem hiding this comment.
Reviewed. Suggestions are inline.
Not explored to full depth (tool budget reached): "agent 1c": static bytecode-size measurement (JIT tier 1) of the grown FieldWriterDouble.writeValue / FieldWriterFloat.writeValue — not performed; the worktree index is r…; "agent 1c": dynamic reproduction of the finding above (running JSONB.toBytes with both creators for a serializeFeatures -annotated Double field under BeanToArray)..
— qwen3.8-max via Qwen Code /review (v0.21.11)
| // "string" (Jackson shape=STRING) and "millis" (shape=NUMBER) are sentinel | ||
| // formats, not DecimalFormat patterns; quoting for "string" is driven by | ||
| // WriteNonStringValueAsString which callers merge into the item features | ||
| boolean sentinelFormat = "string".equals(format) || "millis".equals(format); |
There was a problem hiding this comment.
[Suggestion] The "string"/"millis" sentinel predicate is hand-inlined at four sites added by this diff (the FieldWriter constructor, the static FieldWriter.getObjectWriter BigDecimal/BigDecimal[] branches, and here) with no shared helper tying them to the producer, BeanUtils.processJacksonJsonFormat — Concrete cost: the next sentinel value (or a numeric type added to one of the guarded branches) requires editing all four sites by hand; missing one reproduces this PR's exact bug class — a sentinel fed to new DecimalFormat emits invalid JSON like {"amount":string200} — with no compiler or test signal. This PR's own review history demonstrates the risk: the round-1 blocker was exactly a missed DecimalFormat-from-field-format site in this very method.
Suggested fix — extract one shared predicate and call it from all four sites:
static boolean isSentinelFormat(String format) {
return "string".equals(format) || "millis".equals(format);
}— qwen3.8-max via Qwen Code /review (v0.21.11)
There was a problem hiding this comment.
Done. Extracted as a package-private static predicate on FieldWriter (three of the four sites live in that class, ObjectWriterProvider is in the same package), javadoc naming the producer BeanUtils.processJacksonJsonFormat:
static boolean isSentinelFormat(String format) {
return "string".equals(format) || "millis".equals(format);
}All four sites now call it: the FieldWriter constructor, both the BigDecimal and BigDecimal[] branches of the static FieldWriter.getObjectWriter, and ObjectWriterProvider.getObjectWriter(Type, String, Locale).
| jsonWriter.writeDouble(doubleValue); | ||
| long features = this.features | jsonWriter.getFeatures(); | ||
| if ((features & Feature.WriteNonStringValueAsString.mask) != 0) { | ||
| jsonWriter.writeString(Double.toString(doubleValue)); |
There was a problem hiding this comment.
[Suggestion] This re-implements the existing JSONWriter.writeString(double) overload (JSONWriter.java:2664, literally writeString(Double.toString(value))); the FieldWriterFloat twin does the same with writeString(float), while this PR's own ASM hunk generates calls to those very overloads — one operation expressed two ways within one PR — Concrete cost: if the helper's semantics ever change (formatting or escaping), the reflective and ASM BeanToArray paths silently drift — the exact creator-disagreement class this PR exists to eliminate. No JSONWriter subclass overrides either overload, so the substitution is provably behavior-identical today.
| jsonWriter.writeString(Double.toString(doubleValue)); | |
| jsonWriter.writeString(doubleValue); |
— qwen3.8-max via Qwen Code /review (v0.21.11)
There was a problem hiding this comment.
Done. FieldWriterDouble.writeValue now calls jsonWriter.writeString(doubleValue) and FieldWriterFloat.writeValue calls jsonWriter.writeString(floatValue), the same overloads the ASM hunk targets. Verified no JSONWriter subclass overrides either scalar overload (the only overrides are the array overloads writeString(float[]) / writeString(double[]) in JSONWriterJSONB), so the substitution is behavior-identical.
The writeString(Double.toString(...)) occurrences in writeDoubleValue / writeFloatValue predate this PR and are left unchanged here; unifying them can be a separate follow-up if preferred.
| && !"string".equals(format) | ||
| && !"millis".equals(format) |
There was a problem hiding this comment.
[Suggestion] Untested guard branch (location 1 of 2): the sentinel guards cover primitive double and boxed-array field classes (double.class, Double[].class, Float[].class, BigDecimal[].class), but no test in core exercises any of these shapes. Failure scenario: a mutation skipping double.class here makes @JsonFormat(shape = STRING) public double value build new DecimalFormat("string") and emit {"value":string1.5} — invalid JSON shipping with the suite green; a shape=STRING Float[]/Double[] field would emit unquoted pattern text per item via ObjectWriterArrayFinal. A probe sweep at this head shows all these shapes currently behave correctly — tests would lock that in: primitive double shape=STRING -> {"value":"1.5"}; Double[]/Float[] shape=STRING -> {"values":["1.5","2.0"]}; BigDecimal[] shape=STRING -> {"values":["1.23","4.56"]}.
Suggested tests: primitive double scalar and Double[]/Float[]/BigDecimal[] declared fields with shape=STRING and shape=NUMBER, asserting exact JSON.
— qwen3.8-max via Qwen Code /review (v0.21.11)
There was a problem hiding this comment.
Done. Added 8 cases to Issue7734 for the previously untested guard shapes, each asserting exact JSON:
primitiveDoubleShapeStringQuoted{"value":"1.5"}/primitiveDoubleShapeNumberPlain{"value":1.5}boxedDoubleArrayShapeStringQuoted{"values":["1.5","2.0"]}/boxedDoubleArrayShapeNumberPlain{"values":[1.5,2.0]}boxedFloatArrayShapeStringQuoted/boxedFloatArrayShapeNumberPlain(same shape with1.5F,2.0F)boxedBigDecimalArrayShapeStringQuoted{"values":["1.23","4.56"]}/boxedBigDecimalArrayShapeNumberPlain{"values":[1.23,4.56]}
Expected values were taken from a probe on the previous head, where all these shapes already behaved correctly, so the tests lock the guards in rather than document new behavior. Issue7734: 32 cases green; full core suite: 7997 tests, 0 failures.
| if (format == null || format.isEmpty() || "string".equals(format) || "millis".equals(format)) { | ||
| return new ObjectWriterArrayFinal(BigDecimal.class, null); |
There was a problem hiding this comment.
[Suggestion] Untested guard branch (location 2 of 2): the "millis" clause of this BigDecimal[] branch has no test — "millis" is only exercised against scalar BigDecimal and "string" against BigDecimal[]. Failure scenario: deleting || "millis".equals(format) from only this branch makes a polymorphic @JsonFormat(shape = NUMBER) Object holder of BigDecimal[] resolve new ObjectWriterArrayFinal(BigDecimal.class, new DecimalFormat("millis")) and emit {"amount":[millis200,millis7]} — invalid JSON, undetected by the current suite. Probe at this head: Object holding BigDecimal[] with shape=NUMBER -> {"amount":[200,7]} (correct today, untested).
Suggested test: a polymorphic shape=NUMBER Object holder with new BigDecimal[]{200} expecting {"amount":[200]}.
— qwen3.8-max via Qwen Code /review (v0.21.11)
There was a problem hiding this comment.
Done. polymorphicObjectBigDecimalArrayShapeNumber reuses the existing PolyNumberShapeBean (shape = NUMBER on an Object field) holding new BigDecimal[]{200} and asserts {"amount":[200]}, covering the millis clause of the BigDecimal[] branch that was previously exercised only against scalar BigDecimal (and string only against BigDecimal[]).
c1fc57e to
3a76010
Compare
| * WriteNonStringValueAsString set at writer level is not visible when this | ||
| * method is generated (only bean and field features are); delegate to the | ||
| * reflective loop, which writes each field with the merged features |
There was a problem hiding this comment.
[Critical] This JSONB guard only intercepts the WRITER-level feature — jsonWriter.getFeatures() & WriteNonStringValueAsString.mask — but Jackson @JsonFormat(shape = STRING) sets the feature at FIELD level (features |= WriteNonStringValueAsString.mask in the FieldWriter constructor), which the guard cannot see. Boxed Double/Float fields then route through gwObjectJSONB → ObjectWriterPrimitiveImpl.writeArrayMappingJSONB, which discards its features argument (writeJSONB(jsonWriter, object, null, null, 0)), so JSONWriterJSONB.writeDouble writes binary unconditionally. The reflective creator writes strings because this same PR's FieldWriterDouble/Float.writeValue change merges this.features | jsonWriter.getFeatures() — so this diff newly creates a creator divergence for the canonical Jackson case (the removed jsonWriter.writeDouble(doubleValue) line shows both creators agreed on binary before the change). Concretely: a bean with @JsonFormat(shape = STRING) Double amount = 1.5 and @JsonFormat(shape = STRING) double prim = 2.5 serialized via JSONB.toBytes(order, JSONWriter.Feature.BeanToArray) encodes the boxed Double as a binary double under the default ASM creator while the identically-annotated primitive double sibling in the same output is a string; the reflective creator emits strings for both. The new jsonbBeanToArrayWritersAgreeOnWriteAsStringFeature test only pins the writer-level feature on an unannotated bean, so it does not catch this.
Probe on this head:
bean: @JsonFormat(shape=STRING) Double amount=1.5; @JsonFormat(shape=STRING) double prim=2.5
JSONB.toBytes(bean, Feature.BeanToArray):
ObjectWriterCreator -> [-106, 76,'1','.','5', 76,'2','.','5'] both elements strings
ObjectWriterCreatorASM -> [-106, 0xB5 <IEEE754 double 1.5>, 76,'2','.','5'] boxed Double binary, primitive string
control (writer-level WriteNonStringValueAsString): both creators agree, both strings
Fix options: broaden this guard to also delegate when any field's fieldWriter.features carries WriteNonStringValueAsString.mask (computable at generation time), or make ObjectWriterPrimitiveImpl.writeArrayMappingJSONB/writeArrayMapping propagate their features argument instead of hardcoding 0; then extend the agreement test to the annotated field-level case.
— qwen3.8-max via Qwen Code /review (v0.21.11)
There was a problem hiding this comment.
Confirmed and fixed. Reproduced the probe byte-for-byte on the previous head: reflective [-106, 76,'1','.','5', 76,'2','.','5'], ASM [-106, 0xB5 <IEEE754 1.5>, 76,'2','.','5'].
One path detail: for public fields the leak is the direct-write group, not gwObjectJSONB -> ObjectWriterPrimitiveImpl.writeArrayMappingJSONB. writeFieldValueDirectJSONB emits JSONB.IO.writeDouble(bytes, off, Double, features), which only reads the null-related masks, and the FEATURES local carries the runtime context features only. Propagating the features argument through ObjectWriterPrimitiveImpl would not fix that route, so I took the first suggested variant: when any fieldWriter.features carries WriteNonStringValueAsString.mask, knowable at generation time, genMethodWriteArrayMappingJSONB now emits only the delegate to ObjectWriterAdapter.writeArrayMappingJSONB (beans without such fields keep the runtime guard and the direct path).
jsonbBeanToArrayFieldLevelShapeStringWritersAgree extends the agreement test to the annotated field-level case: 9 vs 14 bytes on the previous head, byte-equal now. Issue7734: 35 cases green, full core suite: 8000 tests, 0 failures.
| static boolean isSentinelFormat(String format) { | ||
| return "string".equals(format) || "millis".equals(format); | ||
| } |
There was a problem hiding this comment.
[Suggestion] The sentinel set is recognized correctly here, but the producer BeanUtils.processJacksonJsonFormat still assigns format = "millis" for Jackson shape = NUMBER unconditionally regardless of field type, and one non-DecimalFormat consumer mis-reads it: int/Integer fields route to FieldWriterInt32.writeInt32, whose format != null branch calls JSONWriter.writeInt32(int, String) → writeString(String.format(format, value)) — String.format("millis", 5) has no conversion specifier, so the value is silently replaced by the sentinel text: @JsonFormat(shape = NUMBER) int f = 5 serializes as {"f":"millis"} on both creators. This all predates the PR, but the test javadoc added here claims shape=NUMBER on numeric fields serializes as plain numbers instead of pattern text, which the int case falsifies. If the producer-side fix (only assign the sentinels to field types that consume them semantically) is too invasive for this PR, extending sentinel recognition to JSONWriter.writeInt32(int, String) (fall back to plain writeInt32 when FieldWriter.isSentinelFormat(format)) and tracking the remaining consumers in a follow-up also closes the corner this javadoc warns about.
Probe on this head:
int f=5 -> {"f":"millis"} (both creators)
Integer f=5 -> {"f":5}
long f=200 -> {"f":200} (routes to FieldWriterMillis, numerically harmless)
— qwen3.8-max via Qwen Code /review (v0.21.11)
There was a problem hiding this comment.
Confirmed: {"f":"millis"} on both creators (int only; Integer and long were plain). Went with the producer-side variant: processJacksonJsonFormat gained a fieldClass parameter (the writer call sites pass field.getType() / the method return type / the Kotlin creator parameter type) and assigns "millis" for shape=NUMBER only to non-numeric fields, so the sentinel stops reaching numeric pattern consumers rather than writeInt32 growing a sentinel fallback. The reader call sites keep the type-less overload: readers ignore the format for numeric fields (verified {"i":5,"gi":7,"l":200} parses unchanged), so read behavior is untouched. FieldWriter.isSentinelFormat stays as the consumer-side guard for the paths that still see sentinels (polymorphic Object / Number fields, collection items).
intShapeNumberPlain locks {"value":5} (failed on the previous head). dateShapeNumberEpochMillis locks the date side: it fails if the producer stops assigning "millis" entirely. The javadoc claim is corrected to say "millis" is assigned only to non-numeric fields.
3a76010 to
d240702
Compare
…SON (alibaba#7734) A BigDecimal field annotated with Jackson @jsonformat(shape = STRING) was serialized as invalid JSON like {"amount":string200} instead of {"amount":"200"}: the "string" sentinel set for shape=STRING was fed to new DecimalFormat(...) as a pattern, so writeDecimal took the formatted branch and emitted the raw pattern text unquoted. The sentinel is now excluded in every place a DecimalFormat is constructed from a field format: - FieldWriter constructor: skip building a DecimalFormat for the "string" and "millis" sentinels -> writeDecimal reaches writeAsString logic - FieldWriter.getObjectWriter BigDecimal / BigDecimal[]: treat the sentinels like a null format, returning the no-format writers - ObjectWriterProvider.getObjectWriter(Type, String, Locale), reached for collection items via FieldWriterList.getItemWriter: return the format-less writers for the sentinels, so List items holding Double / Float / BigDecimal no longer emit pattern text All four checks share one package-private predicate, FieldWriter.isSentinelFormat, tied in its javadoc to the producer BeanUtils.processJacksonJsonFormat. The "millis" sentinel is assigned for shape=NUMBER only to non-numeric fields (the writer annotation processing passes the field / method / Kotlin creator-parameter type into BeanUtils.processJacksonJsonFormat; the reader call sites keep the type-less overload so read behavior is unchanged): it selects the date epoch-millis mode, while on a numeric field it is meaningless and pattern consumers emit it verbatim ({"amount":millis200} through DecimalFormat, {"f":"millis"} through String.format in JSONWriter.writeInt32). The same applies to the bean-level format inherited by every field: BeanUtils.inheritBeanFormat does not pass the sentinel to numeric fields at the field, getter and ASM creator copy sites, so a class-level @jsonformat(shape = NUMBER) no longer emits {"f":"millis"} from an int field either. The sentinel checks live in BeanUtils (isSentinelFormat / isNumericType), with FieldWriter.isSentinelFormat delegating to them. Date fields keep the sentinel, so @jsonformat(shape = NUMBER) on date fields still serializes epoch millis and on numeric fields serializes plain numbers (in plain JSONB a long field is now an int64 instead of a timestamp value; text is unchanged and both encodings parse back to the same value). The float / double BeanToArray paths now honor WriteNonStringValueAsString: FieldWriterFloat.writeValue / FieldWriterDouble.writeValue (reflection, through the existing JSONWriter.writeString(float) / writeString(double) overloads), and the ASM-generated writeArrayMappingJSONB delegates to the reflective loop when the feature is enabled at writer level at runtime (not visible at class-generation time) or carried by any field's features (visible at generation time; the direct-write path drops per-field features), so both creators produce the same JSONB encoding. Integer / int fields with shape=STRING were already quoted (separate toString flag, no decimalFormat). Genuine DecimalFormat patterns (e.g. @jsonformat(pattern = "0.00")) and the default (no shape=STRING) path are unchanged. Signed-off-by: 付典 <fudianchn@gmail.com>
d240702 to
28442d7
Compare
|
Self-audit follow-up after the replies: class-level Also disclosed in the description: in plain JSONB a New cases in |
AI disclosure: this change was prepared with AI coding agents, reviewed and revised line by line by me.
What this PR does / why we need it?
A
BigDecimalfield annotated with Jackson@JsonFormat(shape = JsonFormat.Shape.STRING)was serialized as invalid JSON like{"amount":string200}instead of{"amount":"200"}. Closes #7734.The emitted text (
string200, no quotes) is not parseable JSON, so any consumer (Jackson, fastjson itself, browsers) fails to read it. Reported as a regression since 2.0.61 / 2.0.63.Supersedes #7772 (closed after its branch was force-pushed into a state GitHub could no longer reconcile). It carries the same fix, extended to every path the reviews on #7772 and on this PR surfaced.
Summary of your change
For
shape = STRING,BeanUtilsstores the sentinelfieldInfo.format = "string"(and theFieldWriterconstructor sets theWriteNonStringValueAsStringfeature, the intended "write as string" mechanism). Forshape = NUMBERit stores the sibling sentinel"millis"for non-numeric fields only. Either sentinel reachingnew DecimalFormat(...)(orString.format) as a pattern made the numeric writers emit the raw pattern text unquoted (string200,millis200,"millis").Changes (each mirrors an existing project pattern):
FieldWriterconstructor: skip building aDecimalFormatwhen the format is the"string"or"millis"sentinel ->writeDecimalreaches the existingwriteAsStringlogic.FieldWriter.getObjectWriterBigDecimal / BigDecimal[]: treat the sentinels like a null format -> returns the no-format writer, so polymorphicObject/Numberfields no longer emit pattern text.ObjectWriterProvider.getObjectWriter(Type, String, Locale)(reached for collection items throughFieldWriterList.getItemWriter): return the format-less writers for the sentinels, soListfields holdingDouble/Float/BigDecimalserialize quoted instead of emitting[string200,string7].FieldWriterFloat.writeValue/FieldWriterDouble.writeValue(reflection BeanToArray): honorWriteNonStringValueAsStringthrough the existingwriteString(float)/writeString(double)overloads, mirroringwriteFloatValue/writeDoubleValue.ObjectWriterCreatorASM.gwObject(ASM BeanToArray for text writers, the default creator): mirror the existinggwValuewriteAsStringpattern (writeAsString ? "writeString" : "writeDouble"/"writeFloat").ObjectWriterCreatorASM.genMethodWriteArrayMappingJSONB(ASM BeanToArray for JSONB): delegate toObjectWriterAdapter.writeArrayMappingJSONBwhenWriteNonStringValueAsStringis enabled at writer level at runtime (not visible at class-generation time) or carried by any field's features (visible at generation time; the direct-write path drops per-field features), so the reflective and ASM creators produce the same encoding.BeanUtils.processJacksonJsonFormat: gained afieldClassparameter (the writer call sites pass the field / method-return / Kotlin creator-parameter type) and assigns"millis"forshape = NUMBERonly to non-numeric fields: it selects the date epoch-millis mode, while on a numeric field it is meaningless and pattern consumers emit it verbatim ({"f":"millis"}viaString.formatinJSONWriter.writeInt32). The reader call sites keep the type-less overload, so read behavior is unchanged.BeanUtils.inheritBeanFormat: the bean-level format is no longer inherited by numeric fields at the field, getter and ASM creator copy sites (fieldInfo.format = beanInfo.format), so a class-level@JsonFormat(shape = NUMBER)no longer emits{"f":"millis"}from anintfield either;Datefields keep the epoch-millis mode.BeanUtils(isSentinelFormat/isNumericType), with the package-privateFieldWriter.isSentinelFormatdelegating to them; all guard sites share the one predicate.Genuine
DecimalFormatpatterns (e.g.@JsonFormat(pattern = "0.00")) are unaffected. Date fields are unaffected:"millis"is still assigned forshape = NUMBERon date/time fields (epoch millis) and theDecimalFormatguards only apply to float / double / BigDecimal classes.Integer/intfields withshape = STRINGwere already quoted (separatetoStringflag, nodecimalFormat); withshape = NUMBERanintfield previously emitted the literal{"f":"millis"}(declared or inherited from a class-level annotation) and now serializes a plain number. One observable encoding change is intentional: in plain JSONB alongfield withshape = NUMBERis now an int64 instead of a timestamp value (a plainlongfield never had date semantics); the text encoding is unchanged and both encodings parse back to the same value. The default (noshape = STRING) serialization path is unchanged.Addresses the #7772 review findings (polymorphic
Object/Number,float/doubleBeanToArray, broader coverage, exact assertion), this PR review's findings (List items reachingObjectWriterProvider, the sibling"millis"sentinel, aBigDecimal[]test, real JSONB tests, creator agreement in JSONB BeanToArray), its re-review's suggestions (the sharedisSentinelFormatpredicate, thewriteString(double)/writeString(float)overloads instead of re-implementing them, tests for the primitivedoubleandDouble[]/Float[]/BigDecimal[]guard shapes and theBigDecimal[]millisclause), its third review's findings (the field-levelWriteNonStringValueAsStringgap in the JSONB BeanToArray guard, closed by delegating whenever any field carries the feature; the"millis"sentinel reachingwriteInt32'sString.formatonintfields, closed at the producer by assigning"millis"only to non-numeric fields), and a self-audit follow-up (the same sentinel still reaching numeric fields through the bean-format inheritance for class-level annotations, closed byBeanUtils.inheritBeanFormat).Please indicate you've done the following:
Testing:
Issue7734(@Tag("regression"), 38 cases) covers declaredBigDecimal, primitivedoubleand boxedDouble[]/Float[]/BigDecimal[]fields undershape = STRINGandshape = NUMBER, polymorphicObject/NumberholdingBigDecimalandBigDecimal[],List/List<Double>/List<Float>items,shape = NUMBERon scalar (int, getter-annotatedint, class-level annotatedint/long/Dateepoch millis,longwith text + JSONB int64 encoding), and collection fields,float/doubleBeanToArray via bothTestUtils.writerCreators()(reflection + ASM), primitivefloatBeanToArray, JSONB round-trips (JSONB.toBytes/JSONB.parseObject), and cross-creator JSONB BeanToArray encoding agreement with writer-level and field-level (annotated)WriteNonStringValueAsString. Verified the reported cases fail before the fix ({"amount":string200},{"amounts":[string200,string7]},{"amount":millis200},{"value":"millis"}, the class-level{"f":"millis"}, and a 9-vs-14-byte creator divergence in JSONB BeanToArray for annotatedDouble/doublefields) and pass after. Fullcoresuite: 8003 tests, 0 failures.