Azure AD Authentication (Connect-AzureAD) in Azure Automation

Microsoft Azure

It is now (has been for a while) possible to modify Azure AD via the Azure Automation. The example below uses the Run As Automation Account to first Connect to Azure AD and then run the appropriate commands. You can also create a dedicated Run As account if you want, as well as use a username and password (less secure).

Before you write your code make sure that you:

  • Add the “AzureAD” module to the Automation Account
  • Give the Azure Automation Run As account the appropriate permission as show at the end of this article

Automation Code example (list all the groups in AD):

Give the Azure Automation Run As account the appropriate permissions:

  • Go to Azure Active Directory -> App registrations -> The Run Ass Account.
  • Then go to the API access as show:

  • Give the appropriate access, example below:

Don’t forget to click grant permissions!

Azure ASR Error- 78052 Master target contains different types of scsi controllers.

Microsoft Azure

This is a bit of a self-explanatory one, but I thought I would mention it anyway. When you build an ASR Master Target server make sure if you have more than one SCSI controller that they are of the same type, it doesn’t matter what type they are (LSI Logic SAS, VMware Paravirtual, ect..) but they both need to be the same or you will get the following error on the Azure portal when you attempt to fall back the machine to On-premeses.

 

Azure ASR Error- 90068 disks specified not present

Microsoft Azure

Quick fyi for anyone using Azure ASR, make sure if you are protecting a virtual machine located in Azure to unselect the temp drive disk D when you are adding the machine to ASR protection. If you try and protect the disk to on-premises, you will get the below error message. If you do you will need to delete the protection and reprotect without drive D. The below error only occurs when you try and reprotect to on-premises, it seem to work fine if you reprotect to another azure location.

 

Traffic Manager Endpoint monitor and ADFS /adfs/probe

Microsoft Azure, Windows

Microsoft has a very nice post on how to setup Traffic manager in front of an ADFS farm for high availability, where both sites are in Azure but in different GEO locations or one in Azure and one on premises. The Article is located here: https://docs.microsoft.com/en-us/azure/active-directory/active-directory-adfs-in-azure-with-azure-traffic-manager. What the article lacks is how to setup proper ADFS monitoring, which monitors both tte WAP and the ADFS service, at the moment the article only goes into details which monitor the WAP service.

So this post will go over how to configure your environment so the health point will report the status of both WAP and ADFS.

Some info before we begin:

  • The solutions is achieved by monitoring the /adfs/probe/ on the ADFS server via the WAP proxy
  • The solution will report failure if the WAP proxy is not forwarding or the ADFS service is down. So we are monitoring the whole solution.
  •  It will work if you have an external load balancer in front of the WAP servers and an internal one in front of the ADFS servers, for simplicity I will outline how it’s done on the non-load-balanced solution but it’s the same procedure for both.
  • You can’t monitor /adfs/probe on the WAP server as that will only give you the status of the WAP server
  • You can create a rule on the WAP server to redirect /adfs/probe to the ADFS server, but it will get ignored and show you the status of the WAP server.
  • I tested this on Server 2016 but it will work for 2012 R2 as well
  • If you are using 2012 R2 make sure you update your WAP to the latest version so you can forward HTTP traffic
  • We use HTTP as this prevents certificate problems and because Traffic manager does not support SNI.
  • You can’t monitor the “/federationmetadata/2007-06/federationmetadata.xml” because the way you set this up for Traffic manager means you are monitoring the ADFS on a different DNS so the request will not be forwarded.

Essentially this is what we are doing

adfs_probe_check

Once you setup the environment as per Microsofts Article above we need to do the following:

The variables for my test environment:

  • ADFS URL and Federation Service Name – test123.blah.local
  • Traffic Manager DNS – adfstest.trafficmanager.net
  • WAP server public IP dns (this can be replaced by a load balancer) – http://mytestadfsa.westeurope.cloudapp.azure.com
  • Custom monitor path (you can choose anything but the default which is /adfs/) –  /adfsprobe/

The Steps:

  • Change the Traffic Manager Configuration to point to our custom monitor path for the endpoint monitoring

configuration-microsoft-azure

  • Create an HTTP rule on the WAP server in the Remote Access Management Console to forward (via Pass- through) the WAP DNS + our custom monitor path to the ADFS server. I assume that your WAP server host file has been modified to point the ADFS URL to the ADFS internal IP or load balancer IP

wap-rule

iis-url-rewrite

  • The rule to be created is Reverse Proxy with the following settings:

arp-rule

  • And finally change your Public DNS record and create a CName for your ADFS URL (test123.blah.local) to point to the traffic manager DNS name (adfstest.trafficmanager.net)

And you are done.

HDInsight Cluster Scaling

Microsoft Azure, Powershell

I recently had to come up with options for scaling a Azure HDInsight cluster out (adding worker nodes) or in (removing worker nodes) based on a time based schedule.    At the time of writing HDInsight doesn’t allow you to pause or stop a cluster – you have to delete the cluster if you do not want to incur costs. One way of potentially reducing costs is to use workload specific clusters with Azure Blob Storage and Azure SQL Database for the metastore, then scaling down the cluster outside of core hours when there is no processing to be done.

There are a number of ways of doing this including:

  • Using Azure Resource Manager (ARM) template that has a parameter for the number of worker nodes; then simply running the template at a scheduled time with the    appropriate number of worker nodes.
  • Using the PowerShell module to scale the cluster with Set-AzureHDInsightCluster cmdlet and either running the script as a scheduled task from a VM or using Azure Automation

This post is going to show how to use a script that can be run either via Azure Automation or via a scheduled task running on a VM, the solution consists of:

  • An Azure Storage Account containing XML configuration files that includes information on the cluster, the subscription within which it resides, the number of     worker nodes when scaling out the cluster, the number of worker nodes when scaling in the cluster, a list of email addresses of people that are to be notified when a   scaling operation takes place
  • A PowerShell script that uses the Azure PowerShell module to scale the cluster out and send email notifications
  • Optionally an Azure Automation account from which to schedule the script to run

Azure Storage

Create a storage account (a LRS storage account is sufficient) and a private container, in the container for each cluster store a XML configuration file of the following format:

 <ClusterConfiguration>

     <SubscriptionName>MySubscriptionName</SubscriptionName>

     <ResourceGroupName>MY-RG-0001</ResourceGroupName>

     <ClusterName>vjt-hdi1</ClusterName>

     <MinWorkers>1</MinWorkers>

     <MaxWorkers>3</MaxWorkers>

     <Notify>hadoopsupport@acme.com,team@acme.com</Notify>

 </ClusterConfiguration>

Where:

  • <SubscriptionName> is the subscription containing the cluster to be scaled out/in
  • <ResourceGroupName> is the resource group containing the HDInsight cluster to scale
  • <ClusterName> is the name of the cluster to scale
  • <MinWorkers> is the number of worker nodes in the cluster when performing a scale IN operation
  • <MaxWorkers> is the number of worker nodes in the cluster when performing a scale OUT operation
  • <Notify> is a comma separated list of email addresses to which notifications are to be sent

NOTE: It is important that the XML configuration file is in UTF-8 format.

While one XML configuration file could conceivably contain the information for every cluster there are a number of benefits to having one configuration file per cluster:

  • No need to write logic to parse the file and find the relevant part for the target cluster
  • I can keep each cluster configuration in version control and separately modify them
  • A mistake in the file is less likely to affect all clusters

PowerShell script

The HDInsight cluster scaling PowerShell script is available in my GitHub repository.

The script will need to be modified to suit your environment but the key elements of the script are described here. Since the script can be either run from Azure Automation or via a scheduled task from a Windows server, it supports sending emails from an internal mail server. The script assumes no authentication is required for the internal mail server (but it is trivial to add authentication support for the internal mail server).

Email Providers

The script supports sending emails via SendGrid, Office365 or via internal / on-premise mail servers. At present the latter is supported via hard-coded mail server details but these could be   provided via a XML configuration file or parameters to the script.

Storing credentials on disk

There are other ways of doing this but when the script runs via a scheduled task from a VM it expects that the credentials for the SendGrid / Office365 email account to be stored in encrypted from on disk in an XML file. This is achieved using the Windows Data Protection API.
     1. First open a PowerShell console *as the user under which the scheduled task will run*. E.g.
         runas /user:SVC-RTE-PSAutomation powershell.exe
     2. Then read the password in
         $Password = Read-Host -assecurestring "Please enter your password"
         $Password | ConvertFrom-SecureString
     3. Paste it into the password tag of the XML file
      <credentials>
         <credential>
             <username>emailaccount@acme.onmicrosoft.com</username>
             <password>ReplaceWithActualPassword</password>
         </credential>
      </credentials>

Key Elements of the script

Since the script can be executed via Azure Automation or via a scheduled task, if the $PSPrivateMetadata.JobId property exists then the script is running in Azure Automation. following snippet checks whether the job is running in Azure Automation:
If( -not($PSPrivateMetadata.JobId) )
Next the script contains two try catch blocks one that authenticates using the “Classic” Azure PowerShell module, and the other with the ARM based Azure PowerShell module.
The Get-ClusterScalingConfigurationFile function uses the classic Azure PowerShell module cmdlets to retrieve the storage account key and creates an Azure Storage Account context which is then used to download the blob/file from the storage accoun uses the classic Azure PowerShell module cmdlets to retrieve the storage account key and creates an Azure Storage Account context   which is then used to download the blob/file from the storage account. Note it is important the the cluster configuration file is in UTF-8 format else when we try to parse the contents of the file as XML it will fail (especially if it’s UTF-8 WITH BOM).
 [byte[]] $myByteArray = New-Object -TypeName byte[] -ArgumentList ($BlobReference.Length)
 $null = $BlobReference.ICloudBlob.DownloadToByteArray($myByteArray,0)
 [xml] $BlobContent = [xml] ([System.Text.Encoding]::UTF8.GetString($myByteArray))
The Get-HDInsightQuota function uses the Get-AzureRmHDInsightProperties cmdlet to obtain the number of cores used and available for HDInsight, the script then uses information when scaling out the cluster.
First we use the Get-AzureRmHDInsightCluster cmdlet to get the cluster’s Azure Resource ID, then pass this to the Get-AzureRmResource cmdlet to then obtain the VM sizes currently in use by the cluster – this information is not provided by the Get-AzureRmHDInsightCluster.
 $HDInsightClusterDetails = Get-AzureRmHDInsightCluster | Where-Object {
    $_.Name -eq $ClusterName
  }
  $ClusterSpec = Get-AzureRmResource -ResourceId $HDInsightClusterDetails.Id |
  Select-Object -ExpandProperty Properties |
  Select-Object -ExpandProperty computeProfile

Catching Azure Automation Child runbook Errors

Microsoft Azure

Today I had a very fun time trying to figure out where errors were coming from my child runbook when it ran fine on it’s own. Its seems Microsoft error handling from inline calls to child runbooks is a bit buggy and cost me a day of looking through working code to figure out the problem. It reports weird errors out of which not all are actually true or correct. You may get errors like:

If you do look at the code the foreach/if loop error is referring to as the problem is most likely coming from a function or a method inside the loop. I found deleting (not commenting as it doesn’t work) all code below the if statement or foreach loop gives a better and more precise message exactly where the error is coming from.

New-Object PSObject Problem with Azure Automation

Microsoft Azure

I found a weird behaviour in Azure automation, if you use “New-Object PSObject” it runs perfectly fine when the script is executed directly (via the portal or the Powershell command “Start-AzureRmAutomationRunbook”). However if you try and do an inline call from another workbook it fails and errors out with something like:

the code i used initially was (snippet):

It seems like this command is no longer supported or runs correctly when the child workbook is called by another workbook. To get around this i recommend using the Powershell v3 method which works perfectly:

 

Google Domain Verification Azure DNS

Microsoft Azure

Before you can use your domain with G Suite/Google Cloud, you need to verify that you own it. This can be achieved simply by adding a txt verification record to your domain’s settings. Although Google do provide instructions for a lot of DNS providers such as AWS, GoDaddy etc…However I did notice that Azure was not included in this list.

For convenience I have outlined steps below if you are using Azure to manage the DNS Zone.

Get Verification Code

  1. Open a new browser window or tab, and sign in to your Google Admin console.
  2. Select the option to Verify Domain.
  3. Select Other from the provider list
  4. Select the Add a domain host record (TXT or CNAME) verification method.
  5. Copy the complete contents of the Value/Answer/Destination field. The contents should have “google-site-verification=” at the beginning. This is the verification record that you will submit to your domain host.

Add Verification Record to Azure DNS

  1. Open a new browser window or tab, and sign in to your Azure Portal.
  2. Open up DNS Zones
  3. Select the Domain
  4. Click add Record Set
  5. It will already prefix with your domain, so either leave blank or use @. However if you are using a subdomain enter the subdomain such as test.contoso.com. Although the @ should cover the complete domain, I noticed the google verification process was looking for the exact subdomain record
  6. Select TXT from the Type dropdown
  7. Leave TTL as 1 Hour
  8. In the Value field paste the google site verification string
  9. Click Ok to create the record

Now return back to google and finish the verification process. It may take up to 72hr for some changes to take effect.

Using nslookup, you can verify to see if the TXT record is returned.

From command prompt:

nslookup
set type=txt
subdomain.yourdomain.com

 

 

 

Azure Public and Internal Load balancer for the same availability set

Microsoft Azure

This is one people keep asking me which Microsoft hasn’t published officially. Currently Microsoft Azure allows you to have a public and private load balancer in front of the same availability set.

You can also link both load balancer to the same machines in the availability set. This will enable your servers to have a load balanced private and public IP, which comes in real handy.

You can also link the public to some of the machines and the private to other machines in the availability set.

 

OMS and Power BI Integration Schedule

Microsoft Azure

This is more of a gotcha than anything else but when you’re setting up a schedule to export OMS queries/logs to Power BI please be aware that the initial schedule you setup will also be the historical data it retrieves. What I mean by that is, if I setup a schedule for 24 hours (which is currently the max) in its initial sync it will export the data from that point back 24 hours. So don’t be surprised if you only have the last x amount of data, once the schedule is running it will keep exporting every x hours or minute whatever you have setup so your data over time will grow beyond the schedule you initially setup.

This does raise some concerns and questions:

  • What happens if the sync fails, most likely you will lose some data and it will not be consistent
  • How do I export all the data and sync I? For the moment you can export it to CSV and then import it in PowerBI.
  • You could cheat and set a schedule for 24 hours let it sync and then change it to an hour, this will at least get you the 24 hours rather than just an hour of historical data
  • Is this going to change as this feature is still in preview?