Skip to content

Add workload-level profiling (draft — provisory) - #140

Draft
artursarlo wants to merge 1 commit into
masterfrom
workload_level_profiling
Draft

artursarlo wants to merge 1 commit into
masterfrom
workload_level_profiling

Conversation

@artursarlo

Copy link
Copy Markdown
Contributor

⚠️ Draft / provisory — not yet ready for review. Opening early to run more precise end-to-end tests against upstream before requesting review. Scope, test plan, and this description will be refined.

Brings Pinterest's workload-level profiling feature onto upstream as a single, self-consistent commit.

Backend

  • Heartbeat workload inventory sync (diff/upsert instead of delete+insert) into structured inventory tables, avoiding sequence write amplification.
  • GET /profiling/workload_status endpoint with server-side pagination, filtering, and a ~30s precomputed snapshot store for scale.
  • Supporting dynamic-profiling utilities, metrics/filters models, and DB manager methods.

Frontend

  • Profiling Status console view (workload-aware inventory) with filter deep-links, partial-match filtering, and pagination wired to the workload_status endpoint.

Schema & ops

  • Postgres migrations for structured workload inventory, workload precompute store, and heartbeat sequence cache tuning; fresh-install schema updated to match.
  • periodic_tasks refresh_workload_snapshot job to rebuild the snapshot.

Testing

  • Local e2e harness and Kubernetes sandbox (Spark-on-k8s workloads, per-thread Java profiling) plus backend/integration/spec tests.

Backend requirements add bitmath; date-fns is already declared for the frontend locale helper pulled in transitively.

Verified: backend import graph internally consistent (py_compile clean, import closure converged); frontend import closure resolves.

Co-authored with Pinterest contributors (see commit trailers).

Sync Pinterest's workload-level profiling feature forward onto upstream as
a coherent, self-consistent unit so it builds and runs against
intel/gprofiler-performance-studio master.

Backend:
- Heartbeat workload inventory sync (diff/upsert instead of delete+insert)
  into structured inventory tables, avoiding sequence write amplification.
- GET /profiling/workload_status endpoint with server-side pagination,
  filtering, and a ~30s precomputed snapshot store for scale.
- Supporting dynamic-profiling utilities, metrics/filters models, and DB
  manager methods.

Frontend:
- Profiling Status console view (workload-aware inventory) with filter
  deep-links, partial-match filtering, and pagination wired to the
  workload_status endpoint.

Schema & ops:
- Postgres migrations for structured workload inventory, workload
  precompute store, and heartbeat sequence cache tuning; fresh-install
  schema updated to match.
- periodic_tasks refresh_workload_snapshot job to rebuild the snapshot.

Testing:
- Local e2e harness and Kubernetes sandbox (Spark-on-k8s workloads,
  per-thread Java profiling) plus backend/integration/spec tests for the
  workload feature.

The backend requirements add bitmath; date-fns is already declared for the
frontend locale helper pulled in transitively. Verified: full backend
import graph is internally consistent (py_compile clean, import closure
converged) and the frontend import closure resolves.

Co-authored-by: Lucas <lpenhademoura@pinterest.com>
Co-authored-by: ashokchatharajupalli <achatharajupalli@pinterest.com>
Co-authored-by: prashantpatel <prashantpatel@pinterest.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant