fix(scanner): remove compute-rule false positives/negatives (#268) - #272
Open
parthrohit22 wants to merge 1 commit into
Open
fix(scanner): remove compute-rule false positives/negatives (#268)#272parthrohit22 wants to merge 1 commit into
parthrohit22 wants to merge 1 commit into
Conversation
parthrohit22
requested review from
SHAURYAKSHARMA24,
TFT444,
Vishnu2707 and
ritiksah141
as code owners
August 13, 2026 12:21
…01/003) AZ-CMP-001: a VM's NIC lacking its own NSG is no longer flagged if the NIC's subnet carries a protecting NSG instead (a valid, common Azure pattern). Added AzureClient.get_subnet() to resolve the subnet referenced by a NIC's ip_configuration; an unresolvable subnet is still treated as unprotected so this only removes false positives, never introduces false negatives. AZ-CMP-003: a recognised endpoint-protection extension whose provisioning_state is present and not "Succeeded" is no longer read as a silent pass. It now surfaces as an indeterminate/LOW finding instead, mirroring AZ-CMP-002's existing determination convention. Extensions with no provisioning_state data (not exposed by the API) still fall back to the prior name-based check to avoid inventing new false positives. AZ-CMP-004: verified against current code -- it already checks patch_mode == "AutomaticByPlatform" for both Windows and Linux, so the weak-signal claim in openshield-org#268 does not apply here. No change made. Fixes openshield-org#268 Signed-off-by: PARTH J ROHIT <parthrohit60@gmail.com>
parthrohit22
force-pushed
the
fix/issue-268-compute-rule-quality
branch
from
August 13, 2026 12:27
21001ff to
465d07b
Compare
TFT444
requested changes
Aug 14, 2026
TFT444
left a comment
Collaborator
There was a problem hiding this comment.
@parthrohit22 Good improvements, but this does not yet complete issue #268. Defender health and real patch-assessment evidence are still missing, and an inaccessible subnet can currently be treated as confirmed unprotected and produce a HIGH finding. Please either implement the remaining scope and return an unknown result for unreadable subnet evidence, or narrow the PR and linked issue to the work actually completed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Issue #268 claimed three compute rules relied on weak signals that produce false positives or
false negatives. Read the current code first (per the issue's own instruction to verify rather
than assume) and confirmed two of the three claims, fixed both, and found the third already
resolved.
Root cause per rule
scanner/rules/az_cmp_001.py): only checkednic.network_security_group.A VM whose NIC had no NSG of its own but whose subnet carried one -- a common, valid Azure
pattern -- was flagged as a false-positive HIGH finding.
scanner/rules/az_cmp_003.py): only checked whether a recognisedendpoint-protection extension name was present (
KNOWN_EP_EXTENSIONS), with no health check.An installed-but-failed EP extension read as a silent compliant pass -- a false negative.
scanner/rules/az_cmp_004.py): claimed to use a weak patch signal. Verifiedagainst current code -- it already checks
patch_mode == "AutomaticByPlatform"for bothWindows and Linux (in addition to the Windows
enable_automatic_updatesflag). Alreadycorrect, no change made.
What changed
AzureClient.get_subnet(subnet_id)(scanner/azure_client.py), following theget_network_interface/get_diskpattern: parses resource group/vnet/subnet name out of thesubnet resource ID, returns
Optional[Any], logs and returnsNoneon any failure.az_cmp_001.scan(): a VM is now non-compliant only when it has a public IP and no NIC-levelNSG and no subnet-level NSG. An unresolvable subnet is treated as "not protected," never as
protection, so this only removes false positives -- it introduces no new false negatives.
Metadata now records
nic_nsg_attached/subnet_nsg_attachedfor transparency.az_cmp_003.scan(): when a recognised EP extension is present, itsprovisioning_state(wherethe API exposes it) is now checked.
"Succeeded"or missing/unknown state keeps the existingname-based pass (data doesn't support a stricter check, so the bar isn't raised past what's
confirmable). Any other state (
Failed, etc.) now surfaces as an indeterminate LOW findinginstead of a silent pass, mirroring AZ-CMP-002's existing
metadata.determinationconvention.The name-based check remains the primary signal -- a genuinely missing extension still flags
HIGH exactly as before.
az_cmp_004.py: unchanged, see above.tests/test_rules_compute.pywith subnet-NSG-compliant,no-NIC-NSG-no-subnet-NSG, and unresolvable-subnet cases for AZ-CMP-001, and
healthy/unhealthy/no-provisioning-state-data cases for AZ-CMP-003. Added a direct
AzureClient.get_subnet()unit test totests/test_azure_client_management.py. Extendedtests/helpers/mock_azure.pywithset_subnet/get_subnet.Tested
python -m pytest tests/test_rules_compute.py -q-- 22 passedpython -m pytest tests/ -q -k "rule or compute"-- 250 passed, 1 skipped (pre-existing skip,unrelated)
ruff check scanner/rules/az_cmp_001.py scanner/rules/az_cmp_003.py scanner/azure_client.py tests/test_rules_compute.py tests/helpers/mock_azure.py tests/test_azure_client_management.py-- all checks passed
scanner/rules/az_cmp_002.py, identity/enterprise rules, or the frontend.Fixes #268