Skip to content

clean resources (sockets/files)#5335

Open
ajozwik wants to merge 1 commit into
softwaremill:masterfrom
ajozwik:clean-resources
Open

clean resources (sockets/files)#5335
ajozwik wants to merge 1 commit into
softwaremill:masterfrom
ajozwik:clean-resources

Conversation

@ajozwik

@ajozwik ajozwik commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

As a result of investigating the unstable tests, the following issues were discovered:

  • After tests on linux the some sockets are waiting for close(even 10 minutes after run):

netstat -nap | grep tcp | grep ps ax | grep sbt | grep java | awk '{print $1}'
tcp6      32      0 127.0.0.1:36046         127.0.0.1:39005         CLOSE_WAIT  28290/java
tcp6       0      0 127.0.0.1:52832         127.0.0.1:41039         CLOSE_WAIT  28290/java
tcp6       0      0 127.0.0.1:37852         127.0.0.1:33087         CLOSE_WAIT  28290/java

Tested on ubuntu 25 and slackware 15.

But it failed unpredictably - so solution was removed from this PR

After adding ServerWebSocketTests.scala waitForMessage the problem disappeared.

  • For other if StreamMaxLengthExceededException/EntityStreamSizeException (or related) is thrown the created file was not deleted. It is risky - this poses a risk of a DDOS attack (disk full).

@ajozwik ajozwik force-pushed the clean-resources branch 2 times, most recently from fa49449 to d46e904 Compare June 20, 2026 11:40
@adamw

adamw commented Jun 22, 2026

Copy link
Copy Markdown
Member

Code Review

Overview: This PR addresses a real resource-leak / DoS vector: when a request body exceeds the configured max length while being streamed to a temp file, the partially-written file was left on disk. The fix adds file.delete() on that exception path consistently across all file-writing server backends (akka-http, pekko-http, jdkhttp, netty core/cats/zio, zio-http). 👍 for covering the whole surface uniformly.

Strengths

  • Correct, consistent fix across every backend.
  • Delete-then-rethrow preserves the original error, so the client still gets the size-exceeded failure.
  • For netty core, deletion happens after fileChannel.close() — correct ordering.

Suggestions

1. Narrow exception scope leaves other leaks (most important). Only the max-length exception triggers cleanup. Any other mid-stream failure (connection reset, client abort, I/O error) still leaves the temp file behind — the same concern, different trigger. Consider cleaning up on any failure of writeToFile (e.g. onError / catchAll / ensuring) rather than matching one exception type per backend.

2. netty-cats: unidiomatic handleError with throw.

.handleError {
  case e: StreamMaxLengthExceededException => file.delete(); throw e
  case th => throw th
}

handleError is meant to recover to a value; throwing inside it works only because IO re-captures the throw. onError runs a side effect and preserves the original error without the case th => throw th boilerplate:

.onError { case _: StreamMaxLengthExceededException => Sync[F].delay { file.delete(); () } }

3. Side effects not suspended (FP). In the cats and zio handlers, file.delete() runs as a raw side effect inside the error-handling function rather than wrapped in F.delay / ZIO.attempt. It executes at the right time so it's functionally OK, but it's impure and inconsistent with the surrounding code.

4. Ignored delete() result is inconsistent. netty core uses val _ = path.toFile.delete(); others discard the boolean implicitly. Minor, but worth one convention — and a debug log on a failed delete would aid future leak hunts.

5. No test coverage. The fix adds no test verifying the file is actually removed after a limit breach. A deterministic assertion (send oversized body → assert temp dir has no leftover file) would lock in the behavior and guard against regression.

6. Unrelated noise mixed in. Cosmetic changes ride along (e.getCause()e.getCause, new StreamMaxLengthExceededException(...)StreamMaxLengthExceededException(...), and a pure reformat of the zio-http MultipartBody case). Harmless but they widen the diff.

Risk

Low functional risk — changes are additive on existing error paths; the happy path is untouched. Main residual gap is that cleanup is exception-specific, so temp files from non-size-limit failures remain a leak.

Verdict: Solid, well-targeted fix applied consistently. Before merge I'd suggest: (a) broaden cleanup to all failure modes (or note why only the size limit matters), (b) switch netty-cats to onError, and (c) add at least one deterministic file-cleanup test.

@ajozwik

ajozwik commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

For file deletion the method
deleteFile: TapirFile=>F[Unit] was used.
For netty generic implementation was used writeToFile.
Now the type of exception is ignored - for all deleteFile is called.
testPayloadTooLarge was extended - it now checks if the the temp file exist and contain marker tooLargeMarker.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants