Skip to content

feat(pipeline): replace run_daily.sh with local control-plane - #2591

Merged
twoimo merged 2 commits into
mainfrom
data
Aug 17, 2026
Merged

twoimo merged 2 commits into
mainfrom
data

Conversation

@twoimo

@twoimo twoimo commented Aug 17, 2026

Copy link
Copy Markdown
Owner

Summary

Promote the pipeline-control cutover from data to main.

  • Operator/CI entry is python3 -m backend.pipeline_control.worker.
  • Local-first async control-plane API + worker. Overlay compose has no Postgres.
  • Admin BFF/dashboard talks to the control-plane API.
  • Legacy run_daily.sh / run_local_heavy.sh remain only as regression fixtures.

Upstream: #2590 (develop → data) merged as 53eecd0c3.

Promotion path

Serialized PR: data -> main. Requires CODEOWNERS / branch-protection approval. Do not bypass.

This source tree does not claim hosted pipeline_control was applied. Hosted project writes remain forbidden.

Test plan

twoimo and others added 2 commits August 18, 2026 03:11
Operator and CI entry is now python3 -m backend.pipeline_control.worker.
Jobs, locks, and audit live in local schema pipeline_control plus a
process-shared FileStore; overlay compose has no Postgres. Admin BFF
and dashboard talk to the API, not Grafana. Legacy shims stay only as
regression fixtures.
feat(pipeline): replace run_daily.sh with local control-plane
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@twoimo
twoimo merged commit 3d2c398 into main Aug 17, 2026
22 of 32 checks passed
@gitguardian

gitguardian Bot commented Aug 17, 2026

Copy link
Copy Markdown

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
33514022 Triggered Generic Password e6c7f2a .github/workflows/daily-crawler.yml View secret
33514022 Triggered Generic Password 53eecd0 .github/workflows/daily-crawler.yml View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant