Conditional Access Policy Investigation - Instructions

Purpose

This skill investigates Conditional Access (CA) policy changes in correlation with sign-in failures to detect:

• Legitimate troubleshooting (authorized policy changes to resolve access issues)
• Security control bypass (unauthorized policy modifications to circumvent blocks)
• Privilege abuse (users with admin rights weakening security controls)

The key distinction is whether policy changes were authorized and necessary vs self-service bypass of security controls.

📑 TABLE OF CONTENTS

1. Critical Investigation Rules - Mandatory workflow steps
2. Common Error Codes - Sign-in failure reference
3. CA Policy States - Understanding policy modes
4. 5-Step Investigation Workflow - KQL queries and analysis
5. Real-World Example - Complete walkthrough
6. Critical Mistakes - What NOT to do
7. Security Recommendations - Remediation guidance

Critical Investigation Rules

When investigating sign-in failures (error codes 53000, 50074) with CA policy correlation:

⚠️ MANDATORY STEPS - DO NOT SKIP:

1. Query ALL CA policy changes in chronological order (±2 days from failure time)
2. Parse policy state transitions from the JSON (enabled → disabled → report-only)
3. Compare failure timeline with policy change timeline
4. Verify logical consistency: Ask "does this make sense?"

Key Questions to Answer:

• Was the user blocked BEFORE the policy change?
• Did the policy change resolve the block?
• Who initiated the policy change? (same user = suspicious)
• What was the business justification?

Common Error Codes

Error Code Investigation Priority

CA Policy State Meanings

State Transition Risk Assessment

Investigation Workflow Pattern

Step 1: Identify Sign-In Failures

Query sign-in failures with CA context:

// Get failures with CA context
union isfuzzy=true SigninLogs, AADNonInteractiveUserSignInLogs
| where TimeGenerated between (datetime(<START>) .. datetime(<END>))
| where UserPrincipalName =~ '<UPN>'
| where ResultType != '0'
| where AppDisplayName has '<APPLICATION>'  // e.g., "Visual Studio Code"
| project TimeGenerated, IPAddress, Location, ResultType, ResultDescription, 
    ConditionalAccessStatus, UserAgent
| order by TimeGenerated asc

What to Look For:

• ResultType values: 53000, 50074, 530032, 53003
• ConditionalAccessStatus: "failure", "notApplied"
• Pattern of repeated failures followed by success

Step 2: Query ALL CA Policy Changes in Timeframe

CRITICAL: Query ±2 days from the first failure time

let failure_time = datetime(<FIRST_FAILURE_TIME>);
let start = failure_time - 2d;
let end = failure_time + 2d;
AuditLogs
| where TimeGenerated between (start .. end)
| where OperationName has_any ("Conditional Access", "policy")
| where Identity =~ '<UPN>' or tostring(InitiatedBy) has '<UPN>'
| extend InitiatorUPN = tostring(parse_json(InitiatedBy).user.userPrincipalName)
| extend InitiatorIPAddress = tostring(parse_json(InitiatedBy).user.ipAddress)
| extend TargetName = tostring(parse_json(TargetResources)[0].displayName)
| project TimeGenerated, OperationName, Result, InitiatorUPN, InitiatorIPAddress, 
    TargetName, CorrelationId
| order by TimeGenerated asc  // CRITICAL: Chronological order

Critical Analysis Points:

• InitiatorUPN: Who made the change? Same user as blocked = suspicious
• TargetName: Which policy was modified?
• TimeGenerated: Did change occur AFTER sign-in failures?
• Order: Always chronological (oldest first) to see cause/effect

Step 3: Parse Policy State Changes

For each CorrelationId from Step 2, get detailed changes:

// Get detailed property changes for a specific policy modification
AuditLogs
| where CorrelationId == "<CORRELATION_ID>"
| extend ModifiedProperties = parse_json(TargetResources)[0].modifiedProperties
| mv-expand ModifiedProperties
| extend PropertyName = tostring(ModifiedProperties.displayName)
| extend OldValue = tostring(ModifiedProperties.oldValue)
| extend NewValue = tostring(ModifiedProperties.newValue)
| project TimeGenerated, PropertyName, OldValue, NewValue

Key Properties to Extract:

• Look for "state" property in the JSON
• Parse OldValue and NewValue for state transitions
• Document: enabled → disabled → enabledForReportingButNotEnforced

Step 4: Extract Policy State from JSON

Manual JSON Parsing:

The OldValue and NewValue fields contain JSON. Look for the "state" field:

{
  "state": "enabled",
  "conditions": { ... },
  "grantControls": { ... }
}

Build the Timeline:

1. Extract "state" from each OldValue and NewValue
2. Create chronological list: enabled → disabled → enabledForReportingButNotEnforced
3. Correlate with sign-in failure timeline

Step 5: Security Assessment

Compare timelines and assess intent:

Risk Escalation Criteria:

Real-World Example Analysis

Scenario: User blocked by device compliance policy, then modifies policy

Timeline

Analysis

1. ✅ Policy was correctly blocking non-compliant device

   • Device compliance policy was enforcing as intended
   • User's device failed compliance checks (not enrolled or failing policy)

2. 🚨 User disabled security control to bypass block

   • Same user who was blocked made the policy change
   • Change occurred within 4 minutes of repeated failures
   • No approval or change request documented

3. ⚠️ User partially reversed by enabling report-only

   • Shows some awareness that disabling was too aggressive
   • But report-only still defeats the purpose (doesn't block)

4. ❌ Report-only mode is NOT a valid security posture

   • Logs violations but allows non-compliant access
   • Creates false sense of security (policy "exists" but doesn't protect)

Assessment

Recommendations

1. Immediate Actions:

   • Restore policy to enabled state
   • Verify user's device compliance status
   • Document incident for security review

2. User-Specific:

   • Enroll user's device in Intune
   • Verify device meets compliance requirements
   • Review if user needs Security Administrator role

3. Process Improvements:

   • Implement approval workflow for CA policy changes
   • Create alert for policy state changes (enabled → disabled/report-only)
   • Review all users with permission to modify CA policies
   • Consider PIM for Security Administrator role

Critical Mistakes to Avoid

❌ DON'T:

✅ DO:

Security Recommendations

When CA Policy Changes Are Detected

1. Determine Legitimacy

• Was the policy change authorized?
• Was there a valid business reason?
• Did the user have approval to make this change?
• Is there a change request ticket?

2. Assess Impact

• How many users affected by policy change?
• What applications/resources are now unprotected?
• How long was the policy disabled/weakened?
• Are there compliance implications (regulatory requirements)?

3. Remediation Actions

4. Long-Term Improvements

Prerequisites

Required MCP Servers

This skill requires:

1. Microsoft Sentinel MCP - For KQL queries against SigninLogs and AuditLogs
   • mcp_sentinel-data_query_lake: Execute KQL queries
   • mcp_sentinel-data_search_tables: Discover table schemas

Required Data Sources

• SigninLogs - Interactive sign-in events with CA status
• AADNonInteractiveUserSignInLogs - Non-interactive sign-in events
• AuditLogs - CA policy modification events

Required Permissions

To view CA policy changes in AuditLogs, ensure:

• Sentinel workspace has AuditLogs ingestion enabled
• User has appropriate RBAC to query the workspace

Integration with Other Skills

CA Policy Investigation often follows a user-investigation:

1. Run user-investigation skill → Identifies sign-in failures
2. Notice CA-related error codes → 53000, 50074, 530032
3. Run ca-policy-investigation skill → Correlate failures with policy changes
4. Document findings → Security assessment with remediation recommendations

Key Integration Points:

• Sign-in failure data comes from user investigation
• CA policy changes are NEW queries specific to this skill
• Assessment combines timeline correlation with policy state analysis