Tag Archives: Security

Secure your VSTS Release Management Azure VM deployments with NSGs and PowerShell

One of the neat features of VSTS’ Release Management capability is the ability to deploy to Virtual Machine hosted in Azure (amongst other environments) which I previously walked through setting up.

One thing that you need to configure when you use this deployment approach is an open TCP port to the Virtual Machines to allow remote access to PowerShell and WinRM on the target machines from VSTS.

In Azure this means we need to define a Network Security Group (NSG) inbound rule to allow the traffic (sample shown below). As we are unable to limit the source address (i.e. where VSTS Release Management will call from) we are stuck creating a rule with a Source of “Any” which is less than ideal, even with the connection being TLS-secured. This would probably give security teams a few palpitations when they look at it too!

Network Security Group

We might be able to determine a source address based on monitoring traffic, but there is no guarantee that the Release Management host won’t change at some point which would mean our rule blocks that traffic and our deployment breaks.

So how do we fix this in an automated way with VSTS Release Management and provide a secured environment?

Let’s take a look.

The Fix

The fix is actually quite straightforward it turns out.

As the first step you should go to the existing NSG and flip the inbound rule from “Allow” to “Deny”. This will stop the great unwashed masses from being able to hit TCP port 5986 on your Virtual Machines immediately.

As a side note… if you think nobody is looking for your VMs and open ports, try putting a VM up in Azure and leaving RDP (3389) open to “Any” and see how long it takes before you start seeing authentication failures in your Security event log due to account enumeration attempts.

Modify Project Being Deployed

We’re going to leverage an existing Release Management capability to solve this issue, but first we need to provide a custom PowerShell script that we can use to manipulate the NSG that contains the rule we are currently using to block inbound traffic.

This PowerShell script is just a simple wrapper that combines Azure PowerShell Cmdlets to allow us to a) read the NSG b) update the rule we need c) update the NSG, which commits the change back to Azure.

I usually include this script in a Folder called “Deploy” in my project and set the build action to “Copy always”. As a result the file will be copied to the Artefacts folder at build time which means we have access to it in Release Management.

Project Setup

You should run a build with this included file so that it is available in your

Modify Release Management Defintion

Note that in order to complete this step you must have a connection between VSTS and your target Azure Subscription already configured as a Service Endpoint. Typically this needs to be done by a user with sufficient rights in both VSTS and the Azure Subscription.

Now we are going to modify our existing Release Management definition to make use of this new script.

The way we are going to enable this is by using the existing Azure PowerShell Task that we have available in both Build and Release Management environments in VSTS.

I’ve shown a sample where I’ve added this Task to an existing Release Management definition.

Release Management Definition

There is a reason this Task is added twice – once to change the NSG rule to be “Allow” and then once, at the end, to switch it back to “Deny”. Ideally we want to do the “Allow” early in the process flow to allow time for the NSG to be updated prior to our RM deployment attempting to access the machine(s) remotely.

The Open NSG Task is configured as shown.

Allow Script

The Script Arguments should match those given in the sample script above. As sample we might have:

-resourceGroupName MyTestResourceGroup -networkSecurityGroupName vnet01-nsg 
-securityRuleName custom-vsts-deployments -allowOrDeny Allow -priority 3010

The beauty of our script is that the Close NSG Task is effectively the same, but instead of “Allow” we put “Deny” which will switch the rule to blocking traffic!

Make sure you set the “Close” Task to “Always run”. This way if any other component in the Definition fails we will at least close up the NSG again.

Additionally, if you have a Resource Group Lock in place (and you should for all production workloads) this approach will still work because we are only modifying an existing rule, rather than trying to add / remove it each time.

That’s it!

You can now benefit from VSTS remote deployments while at the same time keeping your environment locked down.

Happy days 🙂

Tagged , , , , ,

Azure Functions: Access KeyVault Secrets with a Cert-secured Service Principal

Azure Functions is one of those services in Azure that is seeing a massive amount of uptake. People are using it for so many things, some of which require access to sensitive information at runtime.

At time of writing this post there is a pending Feature Request for Functions to support storing configuration items in Azure KeyVault. If you can’t wait for that Feature to drop here’s how you can achieve this today.

Step 1: Create a KeyVault and Register Secrets

I’m not going to step through doing this in detail as the documentation for KeyVault is pretty good, especially for adding Secrets or Keys. For our purposes we are going to store a password in a Secret in KeyVault and have the most recent version of it be available from this URI:

https://mytestvault.vault.azure.net/secrets/remotepassword

Step 2: Setup a Cert-secured Service Principal in Azure AD

a. Generate a self-signed certificate

This certificate will be used for our Service Principal to authorise itself when calling into KeyVault. You’ll notice that I’m putting a -1 day “start of” validity period into this certificate. This allows us to deal with the infrastructure running at UTC (which my location isn’t) and avoid not being able to access the certificate until UTC matches our local timezone.

b. Create Service Principal with Cert Authentication

This step requires you to log into an Azure Subscription that is tied to the target Azure AD instance in which you wish to register the Service Principal. Your user must also have sufficient privileges to create new users in Azure AD – if it doesn’t this step will fail.

At this point we now have a Vault, a Secret, and a Service Principal that has permissions to read Secrets from our Vault.

Step 3: Add Cert to App Service

In order for our Function App(s) to utilise this Service Principal and its certificate to access KeyVault we need to upload the PFX file we created in 2.a above into the App Service in which our Functions live. This is just as you would do if this App Service was running a Web App but without the need to bind it to anything. The official Azure documentation on uploading certs is good so I won’t duplicate the instructions here.

Watch out – Gotcha!

Once you’ve uploaded your certificate you do need to do one item to ensure that your Function code can read the certificate from store. You do this by adding an Application Setting “WEBSITE_LOAD_CERTIFICATES” and either specify just the thumbprint of your certificate or put “*” to specify any certificate held in the store.

Step 4: Function App KeyVault and Service Principal Setup

a. Nuget Packages

Accessing KeyVault with a Service Principal in Functions requires us to load some Nuget packages that contain the necessary logic to authenticate with Azure AD and to call KeyVault. We do this by adding the following to our Function App’s project.json.

b. KeyVault Client CSX
Now let’s go ahead and drop in our KeyVault “client” that wraps all code for accessing KeyVault in a single CSX (note that this is mostly inspired by other code that shows you how to do this for Web Apps).

Step 5: Use in a Function

As we’ve encapsulated everything to do with KeyVault into a CSX we can retrieve a secret from KeyVault in a Function using a single call once we’ve imported our client code.

Happy (Secure) Days!

Tagged , , ,

Azure Security Fundamentals: Azure SQL Database

I’m continuing my Kloud blog series on the security fundamentals for various Microsoft Azure services with my most recent post being for Azure SQL Database.

Please go and have a read!

Tagged , ,

Azure Security Fundamentals: Moving Co-Admins to RBAC

I’ve been working on some engagements recently where we’ve been reviewing the current state of security capabilities in Azure, and with the recent General Availability of RBAC support in Azure I thought I’d write a quick overview on Kloud’s blog of how you can move from the traditional Administrator / Co-Adminstrator setup to leveraging RBAC.

Tagged , ,