Tags: azure


TABLE OF CONTENTS


Overview


When Kublr creates an Azure cluster it will create separate Virtual Machines or Virtual Machine Scale Sets depending on the group type.


When updating the group parameters that affect the virtual machine parameters (e.g. the VM image, disk size, instance types etc), Azure may or may not be able to automatically change the existing Virtual Machines depending on the group type and the required change.


Refer to Azure documentation for more information on allowed VM and VMSS changes:


This article describes some strategies and constraints related to VM parameters updates.

Updating Node Groups of Virtual Machine Type


Node groups created by Kublr as a set of individual Virtual Machines include VirtualMachine, AvailabilitySet, and AvailabilitySetLegacy (see Kublr Azure reference documentation for more information).


Azure has limitation on which existing VM properties can be changed, therefore if you change VM parameters (such as instance type, image and some others) in the Kublr cluster specification and try to update the cluster without preparations, the cluster update may fail in Kublr due to failing Azure Deployment.


If it is still necessary to change the group definition, and the required changes cannot be made automatically by Kublr, one of the following approaches can be used:


1. Group replacement: create a new group and remove the old one


1. create a new node group with required parameters in Kublr UI and update the cluster

2. wait for the nodes of the new node group to become available and the cluster update to complete

3. drain nodes in the old node group

4. remove the old node group in Kublr UI and update the cluster


Note: this method will result in a group with a new name. Sometimes it is not desirable or impossible (e.g. for the master group), so a different method should be used.


2. In-place group update with manual VM removal


1. Delete Virtual Machine, OS Disk and NIC resources belonging to the group in Azure portal or using Azure CLI or API.

   The required resources may be identified by their names:

   - Virtual Machines: <cluster-name>-<group-name><node#>

   - OS Disk: <cluster-name>-<group-name><node#>

   - NIC: <cluster-name>-<group-name>Nic<node#>

2. Change the group parameters in Kublr cluster spec and run the cluster update

   The group virtual machines and other resources will be automatically recreated with the new parameters.


The benefit of this method is simplicity, while the downside is that it requires full group downtime, which, for the master group, means full cluster downtime.


Use the following method for most critical groups (e.g. master) where full downtime is not desired.


3. In-place group update with manual VM update


1. Stop/deallocate VM #0 in Azure portal

2. Change the VM size or other required properties in Azure portal

3. Start the VM in Azure portal

4. (if needed) Wait for the VM to come online completely and connect to the cluster
5. repeat steps 1-4 for other VMs in the group
6. Change the group instance type and/or other properties to the required values in Kublr cluster spec and run the cluster update.


Updating Node Groups of VMSS Type


Node groups created by Kublr as Virtual Machine Scale Sets include VirtualMachineScaleSet (see Kublr Azure reference documentation for more information).


The VMSS is created by Kublr with upgrade policy mode Automatic by default, therefore when VM parameters are changed, Azure will automatically restart and reinitialize instances in that VMSS with new parameters.


The VMSS update policy can be customized and changed to less or more conservative option then the default by adjusting VMSS upgradePolicy properties:


spec:
  nodes:
    - ...
      locations:
        - ...
          azure:
            ...
            armTemplateExtras:
              scaleSet:
                properties:
                  upgradePolicy:
                    mode: Rolling
                    rollingUpgradePolicy:
                      maxBatchInstancePercent: 10
                      maxUnhealthyInstancePercent: 30


Refer to Azure VMSS upgrade policy documentation and VMSS documentation for more details.


Note that even though VMSS allow changing some VM parameters, it can still be impossible to change others. In particular, for example image cannot be changed to another image with different generation.

Refer to Azure documentation for more details on VMSS Properties with restrictions on modification.


References