Add workload-level profiling (draft — provisory) - #140
Draft
artursarlo wants to merge 1 commit into
Draft
artursarlo wants to merge 1 commit into
artursarlo wants to merge 1 commit into
Conversation
Sync Pinterest's workload-level profiling feature forward onto upstream as a coherent, self-consistent unit so it builds and runs against intel/gprofiler-performance-studio master. Backend: - Heartbeat workload inventory sync (diff/upsert instead of delete+insert) into structured inventory tables, avoiding sequence write amplification. - GET /profiling/workload_status endpoint with server-side pagination, filtering, and a ~30s precomputed snapshot store for scale. - Supporting dynamic-profiling utilities, metrics/filters models, and DB manager methods. Frontend: - Profiling Status console view (workload-aware inventory) with filter deep-links, partial-match filtering, and pagination wired to the workload_status endpoint. Schema & ops: - Postgres migrations for structured workload inventory, workload precompute store, and heartbeat sequence cache tuning; fresh-install schema updated to match. - periodic_tasks refresh_workload_snapshot job to rebuild the snapshot. Testing: - Local e2e harness and Kubernetes sandbox (Spark-on-k8s workloads, per-thread Java profiling) plus backend/integration/spec tests for the workload feature. The backend requirements add bitmath; date-fns is already declared for the frontend locale helper pulled in transitively. Verified: full backend import graph is internally consistent (py_compile clean, import closure converged) and the frontend import closure resolves. Co-authored-by: Lucas <lpenhademoura@pinterest.com> Co-authored-by: ashokchatharajupalli <achatharajupalli@pinterest.com> Co-authored-by: prashantpatel <prashantpatel@pinterest.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings Pinterest's workload-level profiling feature onto upstream as a single, self-consistent commit.
Backend
GET /profiling/workload_statusendpoint with server-side pagination, filtering, and a ~30s precomputed snapshot store for scale.Frontend
workload_statusendpoint.Schema & ops
periodic_tasksrefresh_workload_snapshotjob to rebuild the snapshot.Testing
Backend requirements add
bitmath;date-fnsis already declared for the frontend locale helper pulled in transitively.Verified: backend import graph internally consistent (py_compile clean, import closure converged); frontend import closure resolves.
Co-authored with Pinterest contributors (see commit trailers).