Introduction
Are you an experienced DevOps engineer or managing cloud resources on Azure, or about to jump into Azure resource management?
While you are creating, updating or deleting resources on Azure, you must have worked with Azure Resource Manager that keeps all resource definitions known as ARM templates. In other words, you have been using those ARM templates implicitly and/or explicitly. But are you sure you use ARM templates properly? This article is going to discuss what you should consider DOs and DON’Ts to build ARM templates.
In order to create or update Azure resources, ARM templates are used, which can be written in many different ways, but not all of them are considered as “Best Practice”. ARM template is a convenient tool to manage Azure resources but, at the same time, can be difficult and time-consuming to craft, unless it is efficiently written.
Recently, a post has been published on the MVP blog site, which is Best Practices For Using Azure Resource Manager Templates. While it contains many handy insights on how to write ARM templates, you will be able to see complementary DOs and DON’Ts here in this article, from the lifecycle management point of view – writing, testing, deploying and managing.
Here is a brief overview of this article:
- Writing:
- Testing:
- Deploying:
- Managing:
Consider YAML
An ARM template is a massive size of JSON object. It consists of so many small JSON objects with various depths. As a data carrying format, JSON is very good, but it is not human readable with depths of depths.
This is a typical example of ARM template. There are many curly brackets and square brackets with different depths. As they are all closing brackets, it is very hard to find their corresponding opening brackets. Some Azure resources take more than hundred lines to define themselves so finding a bracket pair requires endless scrolling up and down over and over again.
Unfortunately, ARM templates only support JSON format. However, if you can use YAML for ARM template writing, it will make your lives easier. Why YAML over JSON? YAML is more human-readable than JSON is. YAML supports comments while JSON does not, which makes a huge difference for developers.
If you use Visual Studio Code, there is a great extension called, json2yaml.
Behold the magic!!! pic.twitter.com/BAaXzy0nUo
— Darrel Miller (@darrel_miller) February 1, 2018
Alternatively, you can clone this repository called yarm
and run it as a local Azure Function instance to convert JSON to YAML or vice versa. yarm
is still in an alpha version, but at least it is working on your local machine, so there is no harm to use this.
The author is the maintainer of
yarm
.
Using either json2yaml
or yarm
will give you much freedom from wrestling with JSON object. Once you finish writing an ARM template with YAML, simply convert it to JSON and commit to your source code control. If you deploy yarm
to your Azure Function instance, you don’t even need to store JSON ARM template to the source control. Instead, write and store ARM template in YAML then convert it to JSON within the CI/CD pipeline, when necessary, by calling the Azure Function endpoint.
Use variables
over parameters
An ARM template consists of four different sections – parameters
, variables
, resources
and outputs
, and two metadata properties – $schema
and contentVersion
.
In the resources
section, many DevOps engineers use both parameters
and variables
at the same time. There is no harm using both in the resources
section to define Azure resources. However, from the maintainability point of view, mixing both is certainly not a good idea. The parameters
section accepts values from outside the ARM template by interacting with users or CI/CD pipelines, while the variables
section defines variables within itself for resource provisioning. Therefore, all parameters
should be handled within the variables
section and the resources
section should only deal with the variables
section. So should the outputs
section.
Let’s bring up an example. webapp-sales-dev-ase
consists of four parameters – webapp
as a resource identifier, sales
as a business unit, dev
as a running environment, and ase
as a resource location (Australia Southeast). In order to enable this, your ARM template should accept at least four parameters. They can be directly used in the resources
section, but processing them within the variables
section and referring the variables
section from the resources
section results in much more consistency.
If something happens during the deployment, the variables
section is the first place to look at, rather than scrolling up to parameters
and down to variables
over and over again. Do all dirty jobs handling parameters
in the variables
section and just use the processed variables
within the resources
section.
Test Behaviours on Deployment
Testing ARM templates is not that easy, but once it is done, it will be rewarding. Azure PowerShell has a cmdlet, Test-AzureRmResourceGroupDeployment
. Azure CLI has also a command like az group deployment validate
. With this, a basic ARM template testing can be covered. However, it does not capture whether an Azure resource is created with expected properties. For example, the cmdlet validates whether the ARM template can be successfully run or not. However, that does not necessarily mean the ARM template correctly deploys expected resources. Resources might be deployed with different name, different SKUs or different configurations, especially when number of parameters have been introduced.
In order to test the ARM template deployment behaviour, Pester becomes handy. As it is a BDD based test runner, once you set up acceptance criteria and translate it into test cases using Pester, the test runner will easily be able to capture whether proper configuration is set or not. Have a look at the test script below. It writes typical BDD style test cases – when running a test, it should expect something
.
Therefore, integrate this test script in your CI/CD pipelines and your ARM template becomes more robust. More details around ARM template testing with Pester can be found at our Mexia blog.
Avoid Linked Template Unless Necessary
Using linked ARM template is one of the best practices. It is true from one sense. But actually it is not true in many cases. Rather, it is recommended to avoid using the linked ARM template.
The idea behind of using linked ARM template is to split individual Azure resources into each corresponding ARM template and bunch them together, to keep single responsibility. At the same time, each ARM template should be independently working. Then why not deploying individual ARM templates within the CI/CD pipeline, rather than running as linked template?
In order to use the linked ARM template approach, all ARM templates MUST be publicly accessible or at least the CI/CD pipeline MUST be accessible to a private place with access token like a SAS token. This is the problem. When an ARM template is updated, it has to be uploaded to somewhere so that the CI/CD pipeline can pick it up during the deployment process.
Too complicated.
Why should it be uploaded to somewhere other than the repository for deployment?
Another issue that the linked ARM template approach imposes is, it only accepts either parameter URI or parameter object. There is no combination of both. However, many ARM template deployment cases use both parameter file and parameter object at the same time to handle sensitive information. In order to handle sensitive information with linked ARM template, parameter URI cannot be used, unless using a hacky workaround.
Therefore, unless you have a specific reason, it is recommended not to use linked ARM template. Instead, create individual ARM templates for each resource to increase reusability and let the CI/CD pipeline take care of them.
Manage Deployment History
Have you been aware that each Azure Resource Group only contains the deployment history of maximum 800? If you deploy ARM template under a certain resource group more than 800 times, the deployment will fail. The number, 800, would not be enough, especially when you integrate CI/CD pipelines with your ARM template deployment. In other words, those deployment history should be constantly monitored, backed up and deleted.
Even though this is not related to ARM template itself but is related to management or monitoring activity, this should be considered when using ARM templates for your Azure resource management.
Azure PowerShell has a cmdlet, Remove-AzureRmResourceGroupDeployment
and Azure CLI has a command, az group deployment delete
that delete an individual deployment history. Therefore, as soon as you complete ARM template deployment within the CI/CD pipeline, just delete the history. Nice and easy. Alternatively, you can regularly iterate the entire deployment history to delete them all at once.
There is a problem by the way. You might have to keep the deployment history for auditing purpose. Therefore, before deleting, you should copy the deployment history to somewhere like Azure Table Storage. If you use an Azure Logic App, copying each deployment history and deleting it becomes really easy. The Logic App periodically checks whether they exists or not, and if exists back them up and delete them. This blog post, Managing Excessive ARM Deployment Histories with Logic Apps explains how to manage deployment history using Azure Logic Apps.
Conclusion
So far, you have learnt what you need to DO and DON’T while writing ARM templates from four different perspectives:
- Writing:
- Testing:
- Deploying:
- Managing:
Hopefully, this article helps your ARM template experience become more robust and efficient.
This has been cross-posted to Dev Kimchi.