fix(resolution): validate rule priority + harden parent-deny immutability against unsat bias#3169
Conversation
🤖 AI Agent: docs-sync-checker — Docs Sync
Docs Sync
|
🤖 AI Agent: breaking-change-detector — API Compatibility
API Compatibility
|
🤖 AI Agent: code-reviewer — View details
TL;DR: 0 blockers, 1 warning. The PR improves robustness and security but introduces a minor backward compatibility concern.
Action items:
Warnings:
No issues found. Clean change. |
🤖 AI Agent: contributor-guide — View details
Welcome, and thank you for your detailed and well-documented contribution! What you did well:
Actionable items before merge:
Let us know if you need any assistance! |
🤖 AI Agent: test-generator — View details
Test coverage looks good. No gaps identified. |
🤖 AI Agent: security-scanner — View details
No security issues found. |
PR Review Summary
Verdict: AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims. |
MohammadHaroonAbuomar
left a comment
There was a problem hiding this comment.
Priority validation is correct (int/float, bool rejected, placed before the single-doc early return). The ignore_left_unsatisfiable flag is threaded through all four recursive arms with a safe default, so the second caller at merge.py:231 is unaffected.
One required rewrite: tests/test_merge_deny.py:101-107 (test_conditions_overlap_protects_parent_against_left_unsat_bias) uses two identical condition dicts. _conditions_overlap returns True at merge.py:294 via _condition_key(parent) == _condition_key(child) before the new code path is reached. The test passes unchanged on main and exercises none of the new logic. Please rewrite with a parent condition that _condition_unsatisfiable actually classifies as unsatisfiable (for example {"and": [{"field":"x","operator":"in","value":[]}, {"field":"tool","operator":"eq","value":"shell"}]}) versus a non-identical child, so the test fails on main and passes here.
None of the other three tests are RED on main either (they pin existing behaviour). Fine to keep as regression guards, but the PR needs at least one RED-to-GREEN test to substantiate the unsat-bias claim, or the description should frame the change as defense-in-depth.
Minor: merge.py:260 _flag = ignore_left_unsatisfiable alias slightly hurts grep-ability; non-blocking.
…soft#3169) @MohammadHaroonAbuomar のレビュー対応: - 【必須】test_conditions_overlap_..._unsat_bias は同一 condition 2つで _condition_key 一致の早期 return を踏むだけで、main でも pass=新ロジックを 検証できていなかった。レビュアー提案の具体例で書き直し: 親 deny = {and:[{x in []}, {tool eq shell}]}(_condition_unsatisfiable が unsatisfiable と分類するが tool==shell で子と重なる)vs 非同一の子 allow。 → main では子 allow が生き残る(fail-open)= RED、本ブランチでは drop = GREEN を実機で確認済み。これにより本指摘が defense-in-depth ではなく実在の fail-open であることも実証。 - _condition_key 早期 return の sanity guard を別テストに分離。 - minor: _flag エイリアスを廃し ignore_left_unsatisfiable を直接渡す(grep性)。 検証: test_merge_deny.py + test_manifest_resolution.py 58 passed。 新テストは main で RED / 本ブランチで GREEN を確認。 Refs microsoft#3137 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VT2CfGX3tmPsQaofuuWwED
|
@MohammadHaroonAbuomar good catch — you're right that the original test only exercised the
So this also settles the framing: it is a genuine fail-open, not just defense-in-depth — I'd been overly cautious in the description. Kept a separate |
imran-siddique
left a comment
There was a problem hiding this comment.
The two fixes are correct and the test suite is solid after the 8238dd8 rewrite.\n\nPriority validation\n\nisinstance(priority, bool) or not isinstance(priority, (int, float)) is the right guard -- bool subclasses int so the explicit bool check must come first. The validation runs inside the pre-sort loop, before both sort paths (single-doc early return and the multi-doc merge path), so neither sort can receive a non-numeric key.\n\nDeny-immutability hardening\n\nThe ignore_left_unsatisfiable flag is threaded through all four recursive arms (left_or, right_or, left_and, right_and) and defaults to False, so the second call site inside _condition_unsatisfiable (which checks intra-conjunct disjointness, not deny-path logic) is unaffected. The _conditions_overlap caller passes ignore_left_unsatisfiable=True, which is the correct scope.\n\nTests\n\ntest_unsat_classified_parent_deny_still_protects_overlapping_child uses the shape MohammadHaroonAbuomar requested ({and: [{x in []}, {tool eq shell}]} vs {tool eq shell}), which is genuinely RED on main and GREEN here -- confirming this is a real fail-open fix, not just defense-in-depth. The _condition_key fast-path sanity test is correctly kept separate.\n\nAll required CI gates pass. No issues blocking merge.
8238dd8 to
901028a
Compare
…ity against unsat bias Addresses the manifest_resolution/merge.py findings from review microsoft#3137. - merge_documents sorted on r.get("priority", 0); an explicit priority: null or a non-numeric value raised a bare TypeError. Validate priority as a non-bool int/float up front and raise ResolutionError otherwise. - Deny-immutability (ADR-0014): _conditions_disjoint short-circuited to "disjoint" when either side was classified unsatisfiable. On the deny path the parent deny is the left operand, so this could neutralise a parent deny. _conditions_overlap now passes ignore_left_unsatisfiable so the parent's unsatisfiability cannot declare "no overlap"; the child side is still honored. Adds tests/test_merge_deny.py, including a test that is RED on main (the child allow survives, neutralising the deny) and GREEN here, demonstrating the fix closes a real fail-open. Verified with the resolution suite. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VT2CfGX3tmPsQaofuuWwED Signed-off-by: Himanaraba <kayaharuka@hotmail.com>
901028a to
885e3ba
Compare
Addresses the
manifest_resolution/merge.pyfindings from the robustness review #3137 (per @imran-siddique's request to track fixes as separate per-file PRs).Unhandled
TypeErroron non-numericprioritymerge_documentssorts withkey=lambda r: r.get("priority", 0)..get(..., 0)only substitutes0when the key is absent — an explicitpriority: null(→None) or a typo (priority: high→str) reaches the sort andraises a bare
TypeErrorcomparing against ints. That is not aResolutionError,so it escapes the documented contract and may surface as a crash instead of a
fail-closed deny (ADR-0013).
Fix: validate
priorityas numeric (non-boolint/float) in the upfrontrule-validation loop and raise
ResolutionError.invalid_governanceotherwise.Deny-immutability hardening (ADR-0014)
_conditions_disjointshort-circuits to "disjoint" when either side isclassified unsatisfiable. Because the parent deny condition is the
leftoperand on the deny-immutability path, a false positive in the unsatisfiability
heuristic would declare the parent deny disjoint from a child
allow, so_conditions_overlapreturnsFalseand the child allow is kept — silentlyneutralising a parent deny.
Fix:
_conditions_overlap(used only by the deny-immutability check) nowpasses
ignore_left_unsatisfiable=True, so the parent (left) side'sunsatisfiability classification can never establish "no overlap". The child
(right) side's unsatisfiability is still honored (an unsatisfiable child allow
never fires, so it overrides nothing).
Honest scope note
_condition_unsatisfiablelooks sound today (itsand/or/in []cases allclassify only genuinely-unsatisfiable conditions, and a genuinely-unsatisfiable
deny is inert), so I could not exhibit a concrete exploitable fail-open — the
empty-
orrepro from the review is an inert deny, not a true neutralisation.This change is therefore defensive hardening: it removes the structural
dependence of a security-critical check on the heuristic being false-positive
free. For sound classifications it is behavior-preserving and only ever more
conservative (stronger deny immutability), as the regression tests confirm.
Happy to drop it if maintainers prefer to keep merge semantics untouched.
Tests
Adds
tests/test_merge_deny.py: non-numericpriority→ResolutionError,priority sort ordering, and deny-immutability behavior (overlapping child allow
dropped, disjoint child allow survives, satisfiable compound parent deny still
protects).
Verified locally against real OPA 0.70.0 (WSL):
Refs #3137
🤖 Generated with Claude Code