Repository navigation
Hash only the bytes read from a stream in Bytes.getMD5 - #16543
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 3.3 #16543 +/- ##
============================================
- Coverage 60.90% 60.89% -0.01%
- Complexity 15 11756 +11741
============================================
Files 1953 1953
Lines 89280 89274 -6
Branches 13476 13474 -2
============================================
- Hits 54376 54365 -11
- Misses 29323 29326 +3
- Partials 5581 5583 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
md5.testfile.txt is 12 bytes while getMD5(File) reads through an 8 KiB
buffer. The old getMD5 copied the whole buffer into the digest on every
iteration, so the expectation in this test was the digest of the file
followed by ~8180 zero bytes rather than the digest of the file.
Now that only the bytes actually read are hashed, the expectation has to
be the real digest ("/D/5joxqDTCH1RXARz+Gdw=="). Assert it against
getMD5(byte[]) as well, so the test states the intent instead of pinning
whatever the buffer handling happens to produce.
|
Fixed the failing test — the expectation was stale, not the implementation.
The other overloads are unaffected: |
EarthChen
left a comment
There was a problem hiding this comment.
LGTM. Confirmed the bug: the old loop called md.update(buf) with the whole 8 KiB buffer instead of the bytes actually read, so any stream whose length is not a multiple of 8192 produced a wrong MD5. The available() > 0 loop condition also returned the empty-input digest for streams that report 0 available. Reading until -1 and hashing buf[0..read) fixes both. BytesTest passes locally. In this repo only getMD5(String) is called from production code, so no compatibility impact.
Non-blocking: the thread-local MessageDigest is not reset if read throws (pre-existing); consider md.reset() in the failure path.
What is the purpose of the change
Bytes.getMD5(InputStream)returns a wrong MD5 whenever the stream length is not a multiple of the internal 8 KiB buffer, and returns the digest of an empty input for streams that reportavailable() == 0while data is still readable. Fixes #16451.Brief changelog
getMD5(InputStream, int)had two defects:md.update(buf), hashing the entire buffer instead of thetotalbytes actually read. The last chunk of any stream whose length is not a multiple ofbstherefore hashed stale tail bytes from the previous chunk, producing a wrong digest. This is the normal case forFileInputStreamon file systems that short-read (network mounts, encrypted volumes), which is exactly whatgetMD5(File)forwards to.while (is.available() > 0).available()reports the bytes readable without blocking, not the remaining length, so a stream that momentarily reports0(pipelined/socket-backed/deflated streams) terminated the loop before any byte was consumed.Both are fixed by the canonical digest loop:
The buffer-filling inner loop becomes unnecessary once the digest consumes exactly the bytes read.
Verifying this change
Added two regressions to
BytesTest, both comparing againstgetMD5(byte[]):testMD5ForStreamWithShortFinalRead: a random8192 + 17byte stream (short final chunk).testMD5ForStreamWithZeroAvailable: a stream whoseavailable()always returns0but whoseread()serves data.Both fail on the previous implementation (verified locally) and pass with this change.
mvn -f dubbo-common/pom.xml test -Dtest=BytesTestandspotless:checkpass.Follow this checklist to request gating of a pull request:
./mvnw -T 1C clean install -DskipTests=false -DskipIntegrationTests=falsepasses locally — not run locally in full; focused module tests and spotless pass (module-levelmvn -f dubbo-common/pom.xml testandspotless:check).