Skip to content

Release shared reference when Bytes is advanced to its end - #856

Open
shoemoney wants to merge 1 commit into
tokio-rs:masterfrom
shoemoney:fix/bytes-advance-to-end-releases-shared-ref
Open

shoemoney wants to merge 1 commit into
tokio-rs:masterfrom
shoemoney:fix/bytes-advance-to-end-releases-shared-ref

Conversation

@shoemoney

Copy link
Copy Markdown

<Bytes as Buf>::advance only calls inc_start, which moves the start pointer and shrinks len without touching the shared reference:

bytes/src/bytes.rs

Lines 703 to 705 in 7930d93

unsafe {
self.inc_start(cnt);
}

bytes/src/bytes.rs

Lines 651 to 656 in 7930d93

unsafe fn inc_start(&mut self, by: usize) {
// should already be asserted, but debug assert for tests
debug_assert!(self.len >= by, "internal: inc_start out of bounds");
self.len -= by;
self.ptr = self.ptr.add(by);
}

So advancing all the way to the end leaves an empty Bytes that still holds a reference to the parent buffer. A Bytes length can never grow again, so that reference can only keep the allocation alive without ever being useful. It blocks reuse and reclamation for as long as the empty handle is kept around, which is exactly the surprise reported in #840 for clear() / truncate(0).

This is the sibling case @abonander pointed out in #840 (comment) while #840 was being fixed. The credit for spotting it is theirs; #842 fixed truncate(0) and left advance as it was.

The inconsistency is now visible inside a single type:

  • truncate(0) replaces self with an empty Bytes and releases the reference (Release shared reference on zero truncate #842)
  • split_to(self.len()) replaces self with an empty Bytes and releases the reference
  • advance(self.len()) retains it

The fix mirrors what split_to(self.len()) already does: when cnt == self.len(), replace self with Bytes::new_empty_with_ptr(self.ptr.wrapping_add(cnt)) and drop the old value. The non-empty path is untouched.

Reproduction

Added as a regression test in tests/test_bytes.rs, modelled on the truncate_to_zero_releases_shared_reference test from #842:

let mut bytes = BytesMut::from(&b\"hello\"[..]);
drop(bytes.split_off(bytes.len()));
let mut advanced = bytes.freeze();
let remaining = advanced.clone();

advanced.advance(5);

// fails on master: the advanced handle still holds a reference
let mut remaining = remaining.try_into_mut().unwrap();

On master that unwrap panics with Err(b\"hello\"). With this change it succeeds, and the second half of the test pins the unchanged behaviour for a partial advance, where the remaining handle must still fail to upgrade.

Verification

  • New test fails on the unmodified tree and passes with the change.
  • cargo test: full suite green, 1306 tests across all targets.
  • cargo test --all-features: green.
  • MIRIFLAGS=-Zmiri-strict-provenance cargo miri test --test test_bytes: 119 passed, 1 ignored, including the new test. The replacement pointer is one past the end, the same pointer split_to(self.len()) already constructs, and new_empty_with_ptr strips its provenance.
  • cargo fmt --check clean, and cargo clippy --all-targets reports no new warnings.

Bytes::advance only increments the start pointer, so advancing all the
way to the end leaves an empty Bytes that still holds a reference to the
parent buffer. Because the length of a Bytes can never grow again, that
reference can only keep the allocation alive without ever being useful,
preventing reuse or reclamation.

truncate(0) and split_to(self.len()) already replace self with an empty
Bytes so the shared reference is dropped. Do the same in advance when
cnt equals the current length.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant