cmatskas

Secure Azure ARM templates with Key Vault and VSTS

Christos Matskas -

Azure ARM templates are the recommended way for standardising and automating resource deployments to Azure. The resource manager engine that drives the current portal and is also responsible for managing your infrastructure, where everything is a resource (VMs, WebApps, CosmosDB etc). ARM templates are JSON files that describe what your infrastructure looks like and comes with some great benefits:

  • Infrastructure as Code
  • Idempotent
  • Source Control
  • Tool flexibility
  • Tighter security and control

For this post, I would like to focus a bit on security and show you how to leverage some of the built-in Azure features to improve your overall Azure DevOps story. In summary:

ARM + Service Principals + Key Vault = awesomeness

So how do we achieve this awesomeness? There are 3 parts to the story.

1. Configure a Service Principal

You want to avoid using your own account for deployments at all costs. This is because, in most cases, your account will be an "Owner" on the subscription. Owners can not only manage resources but can also provision accounts and even wipe out a subscription (by accident or malicious intent). If your account credentials are passed around in scripts, then you're creating a security hole.

In addition, VSTS requires a Service Principal by default in order to deploy ARM templates. If you don't have one, VSTS will end up creating one for you anyway. I prefer to use a predefined one that I know exactly which permissions have been assigned. You can apply custom RBAC permissions to your Service Principal to only allow the provision of a subset of resources. This is great both from a security and cost management perspective. And this is what makes Service Principals so appealing.

To setup a Service Principal, you can follow the instructions in [this blog post](GHOST_URL/service-principals-in-microsoft-azure/" target="_blank).

2. Set up an Azure Key Vault

[Azure Key Vault](https://azure.microsoft.com/en-gb/services/key-vault/" target="_blank) is a SaaS solution that allows you to store keys (SSL) and secrets (strings) in a decoupled manner, consequently improving the integrity of your services, infrastructure and code. I've blogged before about its [benefits](GHOST_URL/azure-key-vault-the-new-security-service-from-microsoft/" target="_blank) and [how to set it up.](https://www.simple-talk.com/cloud/platform-as-a-service/setting-up-and-configuring-an-azure-key-vault/" target="_blank). Key Vault is now been integrated with a lot more services within Azure such as SQL Server, Disk Encryption, Storage etc, but one of the best-known secrets is its interoperability with ARM templates.

Once you've provisioned your Key Vault, you need to configure the Service Principal you created in the previous step (#1) to allow it to pull secrets at deployment time. There are a couple of small but important steps you'll need perform.

  1. Log in on the Azure Portal
  2. Navigate to the Key Vault you want to associate with your Service Principal
  3. Open the Access Policies blade
  4. Click on Add New to add your Service Principal
  5. Click on Select Principal and find the one you wish to add
  6. Configure the following permissions under Secrets
    1. List
    2. Get
  7. Make sure you click OK
  8. Press Save in the end to ensure that the Service Principal is saved against the Key Vault. In the end, you should end up with something that looks like this:

    In this example, I've configured the cm-demo1 Service Principal.
  9. Open the Advanced Access Policies blade
  10. Select the Enable access to Azure Resource Manager for template deployment. This is required to ensure that the Key Vault can be accessed during an ARM template deployment. Without this configured, you will get the following deployment error in VSTS (or any other tool, such as PowerShell or the Azure CLI).

This error will occur if you haven't configured the Service Principal correctly or you failed to provision the Advanced Access policy. If you come across this error, just go back and follow the steps carefully.

3. Add Key Vault secrets to you ARM templates

You can configure your ARM templates to use the secrets defined in Key Vault, either in the parameters file or the template file itself. The choice is down to you, though I prefer to use a parameters file as this allows me to easily target various deployment environments. A typical parameters.json file looks like this:

{
  "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "sqlAdministratorLogin": {
      "value": "sqladmin"
    },
    "sqlAdministratorLoginPassword": {		
      "value": "Test123456789!"		
    }		
  }
}

To take advantage of Key Vault, we could change it to this:

{
  "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "sqlAdministratorLogin": {
      "reference": {
          "keyVault": {
            "id": "/subscriptions/25abd97b-44a3-4092-8337-df6b4bee83c2/resourceGroups/mykeyfault/providers/Microsoft.KeyVault/vaults/cmkvtest"
          },
          "secretName": "sqlusername"
        }
    },
    "sqlAdministratorLoginPassword": {		
      "reference": {
          "keyVault": {
            "id": "/subscriptions/25abd97b-44a3-4092-8337-df6b4bee83c2/resourceGroups/mykeyvault/providers/Microsoft.KeyVault/vaults/cmkvtest"
          },
          "secretName": "sqlpassword"
        }		
    }		
  }
}

The 2 important bits are:

  • keyvault:id -> defines which Key Vault we should point to
  • keyvault:secret -> defines which secret within Key Vault to access

You need to define these secrets prior to using them in your ARM templates. However, you'll notice straight away that I didn't have to disclose any access keys or anything else to allow my templates to work with Key Vault. It just works (TM).

Note: any ARM template parameters that need to access Key Vault, need to be defined as "securestring" inside the template.

In this example, my ARM template parameters are defined as securestring as per the example below:

"sqlAdministratorLogin": {
      "type": "securestring",
      "metadata": {
        "description": "The admin user of the SQL Server"
      }
    },
    "sqlAdministratorLoginPassword": {
      "type": "securestring",
      "metadata": {
        "description": "The password of the admin user of the SQL Server"
      }
    }

Now, if you plan to deploy this ARM template, you need to make sure you use the right Service Principal or your deployment will fail. Security FTW!

4. CI/CD with ARM templates and Visual Studio Team Services (VSTS)

Steps 1-3 work in isolation and they work OK. You can use PowerShell or the CLI to deploy your templates. But where it all shines and makes all these bits to come together and work in harmony is within VSTS's Build steps. Assuming that you commit your ARM templates in Source Control (if you don't make sure you do this!), you can use VSTS to deploy your templates as part of your CI/CD, in the same way you do for your code. In the end, you need some kind of infrastructure to run your code, right?

In this part, we'll set up VSTS to deploy our ARM templates using Key Vault and the SP defined in step #1.

  1. In VSTS -> Build -> Add a new Azure Resource Group Deployment step to your build definition
  2. Edit the settings on the right-hand pane
  3. Under Azure Subscription click on the Settings (little cog-icon on the right-hand side)
  4. Configure a new Endpoint or edit an existing one. The important bit to configure here is the Service Principal to ensure you use the one from step #1
  5. Click on Update Service Configuration
  6. Provide the appropriate values as per the image below
  7. Make sure you validate the settings by clicking on Verify connection in the end.
  8. If all works as expected, you should click OK to complete the setup.
  9. Return to the Build step configuration, select the newly defined endpoint for your Azure Subscription and complete the rest of the steps.
  10. At the end of the Build step configuration, click on Save & Queue to verify that everything works end-to-end

Putting all these moving parts together may require a few extra configuration steps, but getting a successful built knowing that you managed to eliminated hard-coded, clear text passwords from your infrastructure deployment process is a massive accomplishment. If you're using ARM templates, I would encourage you to integrate Key Vault and start using Service Principals to manage your deployment pipeline. As always, feel free to leave a comment below if you have any questions.