Replace/Fix Unicode characters created by ConvertTo-Json in PowerShell for ARM Templates

2 May

Often there will be instances where you may want to make a programmatic change to an ARM template using a scripting language like PowerShell. Parsing XML or JSON type files in the past wasn’t always the easiest thing via some scripting languages. Those of us who remember the days of parsing XML files using VBScript still occasionally wake up in the middle of the night in a cold sweat.

Thankfully, PowerShell makes this relatively easy for even non-developers through the use of some native conversion cmdlets that allow you to quickly take a JSON file and convert it to and from a custom PowerShell object that you can easily traverse and modify. You can do this with the ConvertFrom-Json cmdlet like so:

$myJSONobject = Get-Content -raw "c:\myJSONfile.txt" | ConvertFrom-Json

Without digging into all the different ways that you can use this to tweak an ARM template to your heart’s satisfaction, it’s reasonable to assume that at some point you’d like to convert your newly modified object back into a plain ole’ JSON file with the intention of submitting it back into Azure, and that’s when things go awry. When you use the intuitively named ConvertTo-Json cmdlet to convert your object back into a flat JSON text block, your gorgeous block of JSON content that once looked like this:

Now looks like this:

See the problem? Here it is again with some helpful highlighting:

All your special characters get replaced by Unicode escape characters. While PowerShell’s ConvertFrom-Json cmdlet and other parsers handle these codes just fine when converting from text, Azure will have none of this nonsense and your template will fail to import/deploy/etc. Fortunately, the fix is very simple. The Regex class in the .NET framework has an “unescape” method:

https://msdn.microsoft.com/en-us/library/system.text.regularexpressions.regex(v=vs.110).aspx

So instead of using:

$myOutput = $myJSONobject | ConvertTo-Json -Depth 50

Use:

$myOutput = $myJSONobject | ConvertTo-Json -Depth 50 | % { [System.Text.RegularExpressions.Regex]::Unescape($_) }

The output from the second command should generate a perfectly working ARM template. That is, of course, assuming that you correctly built your ARM template in the first place, but I take no responsibility for that. 😉

Robert Henry

Robert is a Cloud Architect on Avanade’s Global Azure Cloud Enablement (ACE) team. He has been with Avanade for 11 years delivering Microsoft technology and cloud solutions for large enterprise customers.