Secure your Azure DevOps Release Management Azure VM deployments with NSGs and PowerShell

Note: since originally authoring this piece Microsoft has built secure deployments into the platform using Deployment Groups which you should go and read about. Read on if you're interested in how you can do this without that feature.

One of the neat features of Azure Pipelines Release capability is the ability to deploy to Virtual Machine hosted in Azure (amongst other environments) which I previously walked through setting up.

One thing that you need to configure when you use this deployment approach is an open TCP port to the Virtual Machines to allow remote access to PowerShell and WinRM on the target machines from Azure DevOps.

In Azure this means we need to define a Network Security Group (NSG) inbound rule to allow the traffic (sample shown below). As we are unable to limit the source address (i.e. where Azure Pipelines Release Management will call from) we are stuck creating a rule with a Source of "Any" which is less than ideal, even with the connection being TLS-secured. This would probably give security teams a few palpitations when they look at it too!

We might be able to determine a source address based on monitoring traffic, but there is no guarantee that the Release Management host won't change at some point which would mean our rule blocks that traffic and our deployment breaks.

So how do we fix this in an automated way with Azure Pipelines Release Management and provide a secured environment?

Let's take a look.

The Fix

The fix is actually quite straightforward it turns out.

As the first step you should go to the existing NSG and flip the inbound rule from "Allow" to "Deny". This will stop the great unwashed masses from being able to hit TCP port 5986 on your Virtual Machines immediately.

As a side note... if you think nobody is looking for your VMs and open ports, try putting a VM up in Azure and leaving RDP (3389) open to "Any" and see how long it takes before you start seeing authentication failures in your Security event log due to account enumeration attempts.

Modify Project Being Deployed

We're going to leverage an existing Release Management capability to solve this issue, but first we need to provide a custom PowerShell script that we can use to manipulate the NSG that contains the rule we are currently using to block inbound traffic.

This PowerShell script is just a simple wrapper that combines Azure PowerShell Cmdlets to allow us to a) read the NSG b) update the rule we need c) update the NSG, which commits the change back to Azure.

[CmdletBinding()]
Param(
    [Parameter(Mandatory=$True)]
    [string]$resourceGroupName,
    [Parameter(Mandatory=$True)]
    [string]$networkSecurityGroupName,
    [Parameter(Mandatory=$True)]
    [string]$securityRuleName,
    [Parameter(Mandatory=$True)]
    [string]$allowOrDeny,
    [Parameter(Mandatory=$True)]
    [int]$priority
)

$nsg = Get-AzureRmNetworkSecurityGroup -ResourceGroupName $resourceGroupName -Name $networkSecurityGroupName
Set-AzureRmNetworkSecurityRuleConfig -Name $securityRuleName `
                                     -NetworkSecurityGroup $nsg `
                                     -Access $allowOrDeny `
                                     -Protocol Tcp `
                                     -SourcePortRange * `
                                     -DestinationPortRange 5986 `
                                     -SourceAddressPrefix * `
                                     -Priority $priority `
                                     -Direction Inbound `
                                     -DestinationAddressPrefix *
Set-AzureRmNetworkSecurityGroup -NetworkSecurityGroup $nsg

I usually include this script in a Folder called "Deploy" in my project and set the build action to "Copy always". As a result the file will be copied to the Artefacts folder at build time which means we have access to it in Release Management.

You should run a build with this included file so that it is available in your

Modify Release Management Definition

Note that in order to complete this step you must have a connection between Azure DevOps and your target Azure Subscription already configured as a Service Endpoint. Typically this needs to be done by a user with sufficient rights in both Azure DevOps and the Azure Subscription.

Now we are going to modify our existing Release Management definition to make use of this new script.

The way we are going to enable this is by using the existing Azure PowerShell Task that we have available in both Build and Release Management environments in Azure DevOps.

I've shown a sample where I've added this Task to an existing Release Management definition.

There is a reason this Task is added twice - once to change the NSG rule to be "Allow" and then once, at the end, to switch it back to "Deny". Ideally we want to do the "Allow" early in the process flow to allow time for the NSG to be updated prior to our RM deployment attempting to access the machine(s) remotely.

The Open NSG Task is configured as shown.

The Script Arguments should match those given in the sample script above. As sample we might have:

-resourceGroupName MyTestResourceGroup -networkSecurityGroupName vnet01-nsg
-securityRuleName custom-vsts-deployments -allowOrDeny Allow -priority 3010

The beauty of our script is that the Close NSG Task is effectively the same, but instead of "Allow" we put "Deny" which will switch the rule to blocking traffic!

Make sure you set the "Close" Task to "Always run". This way if any other component in the Definition fails we will at least close up the NSG again.

Additionally, if you have a Resource Group Lock in place (and you should for all production workloads) this approach will still work because we are only modifying an existing rule, rather than trying to add / remove it each time.

That's it!

You can now benefit from Azure DevOps remote deployments while at the same time keeping your environment locked down.

Happy days 🙂