[chore][PLAT-1440] Raise termination grace periods so in-flight SLFs can finish - #365
Merged
nataliechen-pixel merged 3 commits intoAug 20, 2026
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
|
| Filename | Overview |
|---|---|
| charts/retool/templates/deployment_backend.yaml | Raises the rendered backend termination grace-period fallback from 136 to 391 seconds. |
| charts/retool/templates/deployment_code_executor.yaml | Adds a configurable termination grace period with a 391-second fallback. |
| charts/retool/templates/deployment_js_executor.yaml | Adds a configurable termination grace period with a 376-second fallback. |
| charts/retool/values.yaml | Publishes the updated lifecycle defaults for backend, DB connector, code executor, and JS executor. |
| values.yaml | Keeps the repository-level values copy synchronized with the packaged chart defaults. |
| charts/retool/Chart.yaml | Increments the chart version from 6.11.20 to 6.11.21. |
Reviews (2): Last reviewed commit: "[chore][PLAT-1440] Bump chart version to..." | Re-trigger Greptile
nataliechen-pixel
requested review from
andoliveyou,
mako-taco and
mertbozfakioglu
August 20, 2026 21:48
nataliechen-pixel
marked this pull request as ready for review
August 20, 2026 21:49
mako-taco
approved these changes
Aug 20, 2026
| {{ toYaml .Values.ui.labels | indent 8 }} | ||
| {{- end }} | ||
| spec: | ||
| terminationGracePeriodSeconds: {{ .Values.backend.terminationGracePeriodSeconds | default 136 }} |
Contributor
There was a problem hiding this comment.
is it possible to add comments in the YAML to explain how all of these magic numbers were arrived at?
Contributor
There was a problem hiding this comment.
ah I see... they're documented in the default values. Do we need the | still, considering we have default values checked in?
Match the shared 391s grace used by the other SLF-serving services instead of carrying a separate 376s value for the JS executor. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related:
Standardize and increase
terminationGracePeriodSecondsacross backend, dbconnector, codeExecutor, and jsExecutor deployments to better accommodate in-flight requests during pod shutdown.What changed?
backendterminationGracePeriodSecondsincreased from 136s (2m 15s + 1s) to 391s (6m 30s + 1s)dbconnectorterminationGracePeriodSecondschanged from a commented-out 960s default to an active value of 391scodeExecutornow has an explicitterminationGracePeriodSecondsof 391s (6m in-flight + 30s preStop buffer + 1s), previously unsetjsExecutornow has an explicitterminationGracePeriodSecondsof 391s (6m in-flight SLF + 30s drain buffer + 1s), previously unsetHow to test?
Deploy the chart and verify that pods for the backend, dbconnector, codeExecutor, and jsExecutor reflect the updated
terminationGracePeriodSecondsvalues in their pod specs. During a rolling update, confirm that in-flight requests are allowed to complete before pods are forcibly terminated.Why make this change?
The previous default grace period for the backend (136s) was based on a 2-minute query timeout, which no longer reflects the actual in-flight request duration. Aligning all services to a 6-minute baseline (plus appropriate drain buffers) ensures pods have sufficient time to finish processing requests before Kubernetes forcibly kills them, reducing disruption during updates and rescheduling events.