Skip to content

Read a workload's recorded creation failures and its namespace's quotas and LimitRanges (#411 Q1, core) - #524

Merged
hellices merged 13 commits into
mainfrom
feat/523-creation-core
Oct 30, 2026
Merged

hellices merged 13 commits into
mainfrom
feat/523-creation-core

Conversation

@hellices

Copy link
Copy Markdown
Owner

Closes #523

Summary

The core half of slice Q1 of #411, following the approved design (docs/superpowers/specs/2026-10-30-creation-constraints-design.md, #516). This is pure Python in core/ with no UI change; the view is the next slice.

  • k8s/events.py: deprecatedLastTimestamp and deprecatedFirstTimestamp now each come right after their core name in the time precedence, so an older controller's event served through events.k8s.io/v1 is dated by its last occurrence.
  • core/creation_quantities.py:
    • normalises cpu, memory and ephemeral-storage to their requests.* names;
    • groups quota resources as compute, storage, object counts or other;
    • compares used against hard as exact Decimals; an unparsable value is not established.
  • core/creation_policies.py:
    • projects ResourceQuotas: values verbatim, entries grouped and ordered, scopes and scope selectors kept;
    • marks a quota stale when its status is missing, its status.hard lags spec.hard, or status.used lacks a key. A stale quota has no entry "at its limit";
    • projects LimitRanges, keeping each constraint verbatim.
  • core/creation_failures.py:
    • classifies a message only by the fixed quota and LimitRanger phrasings, in the order it states them;
    • names a quota only as the message does; a LimitRange is never named;
    • keeps the message verbatim, up to 500 characters;
    • turns FailedCreate events into failures by UID, and counts earlier incarnations;
    • takes the count from series.count, then deprecatedCount, then count;
    • turns a ReplicaFailure condition into a failure only when its status is True and its reason is FailedCreate.
  • core/creation_workloads.py:
    • covers the six kinds: Deployment, ReplicaSet, StatefulSet, DaemonSet, Job and CronJob;
    • keeps each kind's header counters and the template demand Q2 needs: containers and sidecars in order, priority class, deadline, runtime class, Pod-level resources, and StatefulSet claims with an omitted storage class kept apart from an empty one;
    • keeps no environment value, command, image, volume source or annotation;
    • keeps children only when the workload is their controller.
  • core/creation_constraints.py: explain_creation reads the workload, its children, the Events, the quotas and the LimitRanges together under one token. It returns a CreationReport with a baseline:
    • an AVAILABLE quota list with a stale quota is downgraded to PARTIAL/STALE_STATUS; an already partial list keeps its own basis;
    • a denied workload read still matches events by the selected UID;
    • an absent workload has no failures.
  • Plan: docs/superpowers/plans/2026-10-30-creation-constraints-core.md, whose "Departures from the design" section covers the denied-read fallback and the failure order.

Verification

  • Tests were written to fail first for each module: tests/core/test_creation_{quantities,policies,failures,workloads,constraints}.py and tests/k8s/test_events.py. They include:
    • equality across units, such as 1Gi against 1073741824 and 0.5 against 500m;
    • every classifier phrase, including multi-violation LimitRanger messages and webhook messages;
    • events.k8s.io, core and legacy event shapes;
    • earlier-incarnation counting;
    • FailedDelete and False conditions;
    • stale-status variants, including a stale quota in a truncated list;
    • CronJob, StatefulSet, recreated, denied, absent and unserved cases;
    • the literal-value repr checks.
  • Mutation-checked every new condition; each mutation failed a test. One mutation survived because it is equivalent: the explicit None check before parse_quantity, which raises on None anyway, so the check is kept for clarity.
  • Local gate, run with the venv's tools behind the corporate mirror:
    • scripts/check_source_size.py, ruff check src tests, ruff format --check src tests, mypy src tests/core tests/k8s/test_events.py (334 files) and tach check all passed;
    • pytest -n 8: 15121 passed, 40 skipped, 1 failed. The failure is test_device_login_prefixes::test_the_measured_release_is_the_installed_one, which fails only locally because the venv has litellm 1.100.3 and can't re-sync behind the mirror; CI has 1.100.4.

Checklist

  • This closes an accepted issue in this repository that is assigned to me.
  • Tests cover the change, written to fail first where behavior changed.
  • make check passes locally.
  • User-facing docs and the README match the change, or nothing user-facing changed.
  • No security invariant is weakened: writes still need a keystroke approval
    and a fail-closed audit entry, and sensitive reads are still masked.

hellices and others added 7 commits October 30, 2026 18:15
An older controller's event served through events.k8s.io/v1 carries its
core times as deprecatedLastTimestamp and deprecatedFirstTimestamp. Read
each right after its core name, so such an event is dated by its last
occurrence rather than its creation.

Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
core/creation_quantities.py normalises short compute names, groups quota
resources as compute, storage, object counts or other, and compares used
against hard as exact decimals; an unparsable value is not established.

Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
core/creation_policies.py keeps quota values verbatim, grouped and compared
exactly, with scopes and scope selectors, and marks a quota stale when its
status is missing, lags spec.hard or lacks a used amount, so none of its
resources is called at its limit. LimitRanges keep each constraint
verbatim.

Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
core/creation_failures.py turns FailedCreate events about a workload or its
children, matched by UID, and FailedCreate ReplicaFailure conditions into
failures, newest first, and counts events about earlier incarnations. A
message is classified only by the fixed quota and LimitRanger phrasings, in
the order it states them, names a quota only as the message does, and is
kept verbatim up to 500 characters.

Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
core/creation_workloads.py reads the six workload kinds the view opens on
into identity, header counters, a FailedCreate ReplicaFailure condition and
the template's demand, and keeps only the ReplicaSets or Jobs the workload
controls. No environment value, command, image, volume source or
annotation leaves the projection.

Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
core/creation_constraints.py resolves and reads, under one run token, the
workload with its selected UID, a Deployment's ReplicaSets or a CronJob's
Jobs, the namespace's Events and its ResourceQuotas and LimitRanges, and
reports projected failures, children and policies with a baseline. A stale
quota downgrades an available quota list to PARTIAL/STALE_STATUS; a denied
workload read still matches events by the selected UID.

Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 30, 2026 09:29
@hellices hellices added this to the v0.7.0 milestone Oct 30, 2026
@hellices hellices added enhancement New feature or request area: k8s Kubernetes clients, resources, operations, and cluster behavior labels Oct 30, 2026
@hellices hellices self-assigned this Oct 30, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Quadratic failure matching can block the event loop, while malformed policies, UID-less events, and webhook denials can produce misleading conclusions.

4 open issues
What changed in this PR

Implements the pure-Python core for workload creation diagnostics in #523, building on the shared evidence readers for #411. It prepares reports for the forthcoming view without adding UI changes or cluster mutations.

Changes:

  • Projects six workload kinds, owned children, UID-matched creation failures, ResourceQuotas, and LimitRanges.
  • Combines bounded namespace reads under one run token, preserving evidence completeness and stale-status information.
  • Adds legacy event timestamp handling, projection tests, and optional-dependency import checks.
File Description
tests/​test_optional_extras.py Checks that new modules import without optional extras.
tests/​k8s/​test_events.py Tests deprecated event timestamp precedence.
tests/​core/​test_creation_workloads.py Tests workload demand, counters, ownership, and excluded fields.
tests/​core/​test_creation_quantities.py Tests resource grouping and exact quantity comparisons.
tests/​core/​test_creation_policies.py Tests policy projections and stale quota status.
tests/​core/​test_creation_failures.py Tests classification, UID matching, counts, and conditions.
tests/​core/​test_creation_constraints.py Tests report assembly, read budgets, and evidence states.
src/​korvid/​k8s/​events.py Adds deprecated timestamps to event time selection.
src/​korvid/​core/​creation_workloads.py Projects workload identity, demand, counters, and children.
src/​korvid/​core/​creation_quantities.py Groups quota resources and compares quantities.
src/​korvid/​core/​creation_policies.py Projects quotas and LimitRanges with freshness checks.
src/​korvid/​core/​creation_failures.py Classifies and projects recorded creation failures.
src/​korvid/​core/​creation_constraints.py Coordinates evidence reads and builds creation reports.
docs/​superpowers/​plans/​2026-10-30-creation-constraints-core.md Documents implementation steps and design departures.

Review effort: Balanced

Comment thread src/korvid/core/creation_constraints.py Resolved
Comment thread src/korvid/core/creation_failures.py Resolved
Comment thread src/korvid/core/creation_failures.py Resolved
Comment thread src/korvid/core/creation_failures.py Resolved
…d policies

- An admission webhook's denial is classified as OTHER before any policy
  phrase, so its text cannot be attributed to a quota or LimitRange.
- The ratio phrase is anchored at a word's start, so a long unbroken
  message is classified in linear time.
- An event without a UID is ignored, not counted as an earlier
  incarnation.
- An available quota or LimitRange list from which the projection drops an
  unnamed item becomes PARTIAL/MALFORMED.

Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Unresolved issues can make incomplete observations appear complete, misclassify admission denials, and cause malformed quantities to abort analysis.

No open issues 🎉

5 previously missed issues

In code that hasn't changed since last review

Medium severity Downgrade observation for malformed owned children

src/​korvid/​core/​creation_constraints.py:308

If a listed child is controlled by this workload but lacks a name or UID, project_children drops it while children_lists.observation remains AVAILABLE. The baseline can then be available despite losing that child's conditions and any events that require its UID. Track malformed owned children and downgrade the children observation to PARTIAL/MALFORMED when an otherwise available list cannot project them.

Medium severity Classify policy-authored admission denials as OTHER

src/​korvid/​core/​creation_failures.py:112

ValidatingAdmissionPolicy denials also contain policy-authored messages, wrapped as ValidatingAdmissionPolicy '...' with binding '...' denied request: .... If such a message contains exceeded quota: compute, or a LimitRange phrase, it is incorrectly attributed to that standard policy. Treat this explicit admission-policy denial as OTHER before matching phrases, as is done for webhooks.

Medium severity Mark observations partial for malformed policy fields

src/​korvid/​core/​creation_policies.py:104

Malformed policy fields are silently discarded while the observation can remain complete: a quota's services: null constraint disappears, and a named LimitRange with spec.limits: "x" becomes an empty policy. Propagate projection validity so unreadable constraints make an otherwise available list PARTIAL/MALFORMED, preserving the basis of an already partial list. The same issue occurs in _limit_range at lines 193–198.

Medium severity Treat missing quota status as stale

src/​korvid/​core/​creation_policies.py:116

A quota with an empty spec.hard and no status is marked fresh: both maps are empty, so _stale returns false. Consequently an available quota list is not downgraded even though the required status was never observed. Check for missing or empty status independently of the hard-map comparison.

Medium severity Catch DecimalException during quantity parsing

src/​korvid/​core/​creation_quantities.py:87

parse_quantity can also raise a decimal.DecimalException: for example, 1e999999999999999999999999 raises InvalidOperation. A malformed quota value like this therefore aborts explain_creation instead of leaving the comparison unestablished. Catch decimal arithmetic errors as well as ValueError.

4 resolved since last review

Review effort: Balanced

hellices and others added 2 commits October 30, 2026 19:03
The two truncated-list tests repeated one continue token, which the reader
rightly rejects as malformed, so they were not testing truncation at all.
The fake now serves a page sequence and both tests assert
PARTIAL/TRUNCATED, which also pins the stale-status precondition.

Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- A quota or LimitRange with a constraint the projection cannot read, and
  an owned child without a name or UID, make an available list
  PARTIAL/MALFORMED.
- A quota with no status is stale even when spec.hard is empty.
- A ValidatingAdmissionPolicy denial is classified as OTHER, like a
  webhook's.
- A quantity decimal arithmetic cannot represent is not established
  instead of aborting the run.

Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@hellices

Copy link
Copy Markdown
Owner Author

The five "previously missed" issues from Copilot's aafe2f3 review are addressed in b87d878, each with a failing test first:

  • Malformed owned children: project_children now returns ChildrenProjection(children, unprojected). A child the workload controls but that has no name or UID makes an AVAILABLE children list PARTIAL/MALFORMED. Tests: test_owned_children_without_a_name_or_uid_are_counted_as_unprojected and test_an_owned_child_the_projection_cannot_name_makes_the_children_inconclusive.

  • ValidatingAdmissionPolicy denials: ValidatingAdmissionPolicy '…' with binding '…' denied request is now classified OTHER before any policy phrase, like a webhook denial. Test: test_a_validating_admission_policy_denial_is_never_attributed_to_a_policy.

  • Malformed policy fields: quotas and LimitRanges now carry complete. Each of these makes a policy incomplete, and an incomplete policy makes an AVAILABLE list PARTIAL/MALFORMED:

    • a constraint that isn't a quantity, or a hard that isn't a map;
    • a scope that isn't text, or scopes that isn't a list;
    • a scope expression without a name or operator, or with non-text values;
    • limits that isn't a list of maps of quantities.

    A list that is already partial keeps its basis. Tests: test_a_quota_with_an_unreadable_constraint_is_incomplete[*], test_a_limit_range_with_an_unreadable_constraint_is_incomplete[*] and test_a_policy_with_an_unreadable_constraint_makes_its_list_inconclusive[*].

  • Missing quota status: a quota with no status is stale even when spec.hard is empty. I had removed that check as redundant in an earlier commit, which was wrong. Test: test_a_quota_with_no_hard_limits_and_no_status_is_stale.

  • DecimalException: quantity parsing now also catches ArithmeticError (decimal.InvalidOperation), so 1e999999999999999999999999 is not established instead of aborting the run. Test: test_a_quantity_too_large_for_decimal_arithmetic_is_not_established.

Separately, in 1ac9a95: the Windows failure on aafe2f3 was a bad test, not a platform issue. The two truncated-list tests repeated one continue token, which the reader correctly rejects as malformed, so they never tested truncation. The fake now serves a real page sequence, and both tests assert PARTIAL/TRUNCATED.

@my-reviewer-agent my-reviewer-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이슈 6건을 인라인 코멘트로 남겼습니다.

Comment thread src/korvid/core/creation_policies.py Resolved
Comment thread src/korvid/core/creation_workloads.py Resolved
Comment thread tests/core/test_creation_constraints.py Resolved
Comment thread tests/core/test_creation_constraints.py Resolved
Comment thread tests/core/test_creation_constraints.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An empty target UID can abort diagnostics, and unreadable quota or LimitRange policies can be incorrectly reported as conclusive.

5 open issues

Review effort: Balanced

Comment thread src/korvid/core/creation_constraints.py Resolved
Comment thread src/korvid/core/creation_policies.py Resolved
Comment thread src/korvid/core/creation_policies.py Resolved
Comment thread src/korvid/core/creation_policies.py Resolved
Comment thread src/korvid/core/creation_policies.py Resolved
- A quota without a readable spec or scopeSelector, and a LimitRange
  without spec.limits or with a limit that has no text type, are
  incomplete, so their list cannot rule anything out.
- CreationTarget treats an empty UID as none, so a summary without one
  does not abort the run.
- The fake reader records continuation tokens, max_bytes and full targets,
  and the read test checks every call exactly once.

Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Potential issues in policy validation, quantity precision, and event attribution warrant human assessment.

No open issues 🎉

5 resolved since last review

Review effort: Balanced

@my-reviewer-agent my-reviewer-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이슈 3건을 인라인 코멘트로 남겼습니다.

Comment thread src/korvid/core/creation_workloads.py Resolved
Comment thread tests/core/test_creation_constraints.py Resolved
Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@my-reviewer-agent my-reviewer-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이슈 2건을 인라인 코멘트로 남겼습니다.

Comment thread src/korvid/core/creation_workloads.py Resolved
Comment thread tests/core/test_creation_policies.py Resolved
Part of #523

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@my-reviewer-agent my-reviewer-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이슈 1건을 인라인 코멘트로 남겼습니다.

Comment thread src/korvid/core/creation_workloads.py Resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: k8s Kubernetes clients, resources, operations, and cluster behavior enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Read a workload's recorded creation failures and its namespace's quotas and LimitRanges (#411 Q1, core)

2 participants