MOD-16608 Object memory improvement - pack header, u32 hash table, elide table for small objects

[AvivDavid23]

Object memory reduction (~20–40% per object container)

• Pack IObject header 16B → 8B: {len:usize, cap:usize} → single u64 (32b len / 32b cap). Flat −8B per object.
• Hash-table index usize → u32: table slots hold item indices (≤cap < 2^32), so 4B suffices. Halves table: 10·cap → 5·cap bytes.
• Small-object no-table: objects with cap ≤ 8 store no hash table; lookups linear-scan the items array. Drops the table entirely for the common small object.
• Net: −40% on tiny objects (table elided), −20% on larger ones (table halved), applied per-container down the whole tree.

API — fallible allocation

• IObject mutators (with_capacity, insert, reserve, entry, …) now return Result<_, IJsonError>, mirroring IArray. Allocation failure and the 32-bit limit are recoverable instead of panicking.
• Extend/FromIterator/From<HashMap|BTreeMap> → TryExtend/TryFromIterator/TryFrom.
• Moved the shared TryExtend/TryFromIterator/TryCollect traits into new src/convert.rs; pub use re-exports keep existing paths (ijson::array::TryExtend) working.
• Infallible-context .unwrap()s (Clone, Drop/dealloc, layout, defrag, obj[k]=v) - they preserve the prior panic-on-OOM contract.

Compatibility

• Breaking: a single IObject is now capped at 2^32−1 (~4.29B) entries (was usize) — the tradeoff for the packed 8B header + halved table.

---

Note

High Risk
Large breaking API surface plus unsafe object hash-table and layout changes; incorrect small-object vs table-mode logic could corrupt maps or lose keys.

Overview
IObject memory layout is reworked for smaller containers: the header is a single 8-byte u64 (32-bit len/cap), hash buckets store u32 item indices instead of usize, and objects with cap ≤ 8 skip the hash table and use linear scan. Layout, resize, and Robin-Hood insert/remove paths are updated for both modes.

Breaking API: IObject allocation and mutation (with_capacity, insert, reserve, entry, …) now return Result<_, IJsonError>; Extend/FromIterator/From<HashMap|BTreeMap> become TryExtend/TryFromIterator/TryFrom. Shared fallible iterator traits move to src/convert.rs and are re-exported from the crate root (and from array.rs).

Call-site updates: JSON/CBOR deserialize, serde serialize, macros, and CBOR map building propagate allocation errors instead of ignoring them. IArray internal len/cap stay u32 with casts at public usize boundaries; fuzz_object_ops is added to CI to fuzz insert/remove/get against a HashMap oracle across small/large table behavior.

^{Reviewed by Cursor Bugbot for commit 23c23af. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 3 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 1fface9. Configure here.}

[galcohen-redislabs]

Now that all sizes and capacities are 32 bits why keep functions with usizes (in args and/or return value) around at all?

[AvivDavid23]

This will require a bigger change that we might can change in future,, but I'm against since:

1. In some places we can't change the usize(serde, other std/public interfaces)
2. Changing to u32 might cause a lot more casting changes in the upstream usage

[galcohen-redislabs]

Ideally the castings are only done at the edges (e.g., when you must call a function outside the scope of our code, like allocations, etc). Other places should stick to u32 - it will make the code less confusing and will benefit from Rust's nazi compiler's type checks.

[AvivDavid23]

Maybe so. But anyway I prefer to do so in a separate PR(which will have more breaking API changes also in array.rs

[AvivDavid23]

@galcohen-redislabs check last commit, changed to u32 in most cases(apart for some public APIs whihc are called from serde/etc (to handle the casting in the array/object side)

[galcohen-redislabs]

Beautiful