Skip to content

fix(scanner): remove compute-rule false positives/negatives (#268) - #272

Open
parthrohit22 wants to merge 1 commit into
openshield-org:devfrom
parthrohit22:fix/issue-268-compute-rule-quality
Open

fix(scanner): remove compute-rule false positives/negatives (#268)#272
parthrohit22 wants to merge 1 commit into
openshield-org:devfrom
parthrohit22:fix/issue-268-compute-rule-quality

Conversation

@parthrohit22

Copy link
Copy Markdown
Member

Summary

Issue #268 claimed three compute rules relied on weak signals that produce false positives or
false negatives. Read the current code first (per the issue's own instruction to verify rather
than assume) and confirmed two of the three claims, fixed both, and found the third already
resolved.

Root cause per rule

  • AZ-CMP-001 (scanner/rules/az_cmp_001.py): only checked nic.network_security_group.
    A VM whose NIC had no NSG of its own but whose subnet carried one -- a common, valid Azure
    pattern -- was flagged as a false-positive HIGH finding.
  • AZ-CMP-003 (scanner/rules/az_cmp_003.py): only checked whether a recognised
    endpoint-protection extension name was present (KNOWN_EP_EXTENSIONS), with no health check.
    An installed-but-failed EP extension read as a silent compliant pass -- a false negative.
  • AZ-CMP-004 (scanner/rules/az_cmp_004.py): claimed to use a weak patch signal. Verified
    against current code -- it already checks patch_mode == "AutomaticByPlatform" for both
    Windows and Linux (in addition to the Windows enable_automatic_updates flag). Already
    correct, no change made.

What changed

  • Added AzureClient.get_subnet(subnet_id) (scanner/azure_client.py), following the
    get_network_interface/get_disk pattern: parses resource group/vnet/subnet name out of the
    subnet resource ID, returns Optional[Any], logs and returns None on any failure.
  • az_cmp_001.scan(): a VM is now non-compliant only when it has a public IP and no NIC-level
    NSG and no subnet-level NSG. An unresolvable subnet is treated as "not protected," never as
    protection, so this only removes false positives -- it introduces no new false negatives.
    Metadata now records nic_nsg_attached / subnet_nsg_attached for transparency.
  • az_cmp_003.scan(): when a recognised EP extension is present, its provisioning_state (where
    the API exposes it) is now checked. "Succeeded" or missing/unknown state keeps the existing
    name-based pass (data doesn't support a stricter check, so the bar isn't raised past what's
    confirmable). Any other state (Failed, etc.) now surfaces as an indeterminate LOW finding
    instead of a silent pass, mirroring AZ-CMP-002's existing metadata.determination convention.
    The name-based check remains the primary signal -- a genuinely missing extension still flags
    HIGH exactly as before.
  • az_cmp_004.py: unchanged, see above.
  • Tests: extended tests/test_rules_compute.py with subnet-NSG-compliant,
    no-NIC-NSG-no-subnet-NSG, and unresolvable-subnet cases for AZ-CMP-001, and
    healthy/unhealthy/no-provisioning-state-data cases for AZ-CMP-003. Added a direct
    AzureClient.get_subnet() unit test to tests/test_azure_client_management.py. Extended
    tests/helpers/mock_azure.py with set_subnet/get_subnet.

Tested

  • python -m pytest tests/test_rules_compute.py -q -- 22 passed
  • python -m pytest tests/ -q -k "rule or compute" -- 250 passed, 1 skipped (pre-existing skip,
    unrelated)
  • ruff check scanner/rules/az_cmp_001.py scanner/rules/az_cmp_003.py scanner/azure_client.py tests/test_rules_compute.py tests/helpers/mock_azure.py tests/test_azure_client_management.py
    -- all checks passed
  • No changes to scanner/rules/az_cmp_002.py, identity/enterprise rules, or the frontend.

Fixes #268

…01/003)

AZ-CMP-001: a VM's NIC lacking its own NSG is no longer flagged if the
NIC's subnet carries a protecting NSG instead (a valid, common Azure
pattern). Added AzureClient.get_subnet() to resolve the subnet referenced
by a NIC's ip_configuration; an unresolvable subnet is still treated as
unprotected so this only removes false positives, never introduces false
negatives.

AZ-CMP-003: a recognised endpoint-protection extension whose
provisioning_state is present and not "Succeeded" is no longer read as a
silent pass. It now surfaces as an indeterminate/LOW finding instead,
mirroring AZ-CMP-002's existing determination convention. Extensions with
no provisioning_state data (not exposed by the API) still fall back to
the prior name-based check to avoid inventing new false positives.

AZ-CMP-004: verified against current code -- it already checks
patch_mode == "AutomaticByPlatform" for both Windows and Linux, so the
weak-signal claim in openshield-org#268 does not apply here. No change made.

Fixes openshield-org#268

Signed-off-by: PARTH J ROHIT <parthrohit60@gmail.com>
@parthrohit22
parthrohit22 force-pushed the fix/issue-268-compute-rule-quality branch from 21001ff to 465d07b Compare August 13, 2026 12:27

@TFT444 TFT444 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@parthrohit22 Good improvements, but this does not yet complete issue #268. Defender health and real patch-assessment evidence are still missing, and an inaccessible subnet can currently be treated as confirmed unprotected and produce a HIGH finding. Please either implement the remaining scope and return an unknown result for unreadable subnet evidence, or narrow the PR and linked issue to the work actually completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(scanner): compute rule quality pass — subnet NSGs, Defender health, real patch state

2 participants