Conditional Access policies – do you backup them ALL?

This will be a short post about a recent finding: AzureAD Conditional Access policies created from template may miss from your backups if not using Graph API beta endpoint.

TL;DR
– When you create a Conditional Access policy using the “New policy from template (Preview)” button, the policy will not show when querying policies using the “traditional tools”
– This may also apply anytime you have preview features set in your policy
– You may want to check if all your policies are backed up
– To switch to the beta endpoint in Microsoft Graph PowerShell, use the Select-MgProfile -Name “beta” command

Explained

I was creating a new Conditional Access policy for a customer where I have my own script running as a scheduled task to backup these settings. When there is a change I get notified and it became strange that this time I did not receive such alert. I started to investigate the issue, no errors, but even the “full” backup did not notice the new policy.

Time to reproduce it in sandbox environment. Here are 4 policies, one is created from template:

When querying the policies this one is missing:

Same results with Graph PowerShell:

Now it is time to open up F12 developer tools to see what is the trick. Opening the policy let me find the policy ID:

Now if I try to query the policy by ID, I get the following error message:

Get-AzureADMSConditionalAccessPolicy -PolicyId '09981539-1959-4b4a-8543-1f71bc34217d'
Get-AzureADMSConditionalAccessPolicy : Error occurred while executing GetAzureADMSConditionalAccessPolicy
Code: BadRequest
Message: 1037: The policy you requested contains preview features. Use the Beta endpoint to retrieve this policy.
InnerError:
  RequestId: 1016ffe3-6338-4eaf-b334-4595a6023a6c
  DateTimeStamp: Tue, 28 Feb 2023 16:47:18 GMT
HttpStatusCode: BadRequest
HttpStatusDescription: Bad Request
HttpResponseStatus: Completed
At line:1 char:1
+ Get-AzureADMSConditionalAccessPolicy -PolicyId '09981539-1959-4b4a-85 ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Get-AzureADMSConditionalAccessPolicy], ApiException
    + FullyQualifiedErrorId : Microsoft.Open.MSGraphV10.Client.ApiException,Microsoft.Open.MSGraphV10.PowerShell.GetAzureADMSConditionalAccessPolicy

Same error with Graph PowerShell:

Get-MgIdentityConditionalAccessPolicy -ConditionalAccessPolicyId '09981539-1959-4b4a-8543-1f71bc34217d'                                                                                                                     Get-MgIdentityConditionalAccessPolicy : 1037: The policy you requested contains preview features. Use the Beta
endpoint to retrieve this policy.
At line:1 char:1
+ Get-MgIdentityConditionalAccessPolicy -ConditionalAccessPolicyId '099 ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: ({ ConditionalAc...ndProperty =  }:<>f__AnonymousType35`3) [Get-MgIden
   tityC...cessPolicy_Get1], RestException`1
    + FullyQualifiedErrorId : BadRequest,Microsoft.Graph.PowerShell.Cmdlets.GetMgIdentityConditionalAccessPolicy_Get1

I don’t know how to use the beta endpoint in AzureAD PowerShell, but since it is being deprecated, I will rely on the Graph Powershell cmdlet which has the Select-MgProfile option to switch to the beta endpoint (link):

Select-MgProfile -Name "beta"

And here we go, now these policies are retrieved too:

So if you have a policy backup solution in place, take a look on those backups if you are using templates or other preview features.