Tags: azure, network


TABLE OF CONTENTS


1. Overview


By default Kublr and Kubernetes on Azure allocate public IP addresses for the cluster and for the applications running in the cluster from the global pool of Azure public IP addresses by just creating a PublicIPAddress objects.

Allocating IP addresses this way does not allow using predictable IP ranges.


To allow allocating IP addresses from a predictable range of addresses Azure supports PublicIPPrefix objects, that allow allocating pools of public IP addresses.


This article describes using Azure public IP prefixes in Azure Kublr deployment.


2. Constraints and limitations


  • PublicIPPrefix object and PublicIPAddress object using this prefix must be in the same Azure subscription (but may be in different resource groups)
  • The prefix size can only be in the range from 28 to 30, which means that the prefix may include from 2 to 16 individual addresses.
  • Kubernetes Azure cloud controller does not support specifying public IP prefix in a Kubernetes service; the only way to associate a Service of type LoadBalancer with a public IP with a specific prefix is to create the public IP resource manually and specify the public IP in the Service spec.


3. Defining public IP address prefix in Kublr cluster spec


Azure public IP prefix resource can be created manually via Azure portal, Azure CLI, or included in the Kublr cluster specification as an Azure "external" resource.


The following cluster spec snippet can be used as an example of defining the prefix resource in the cluster spec:


spec:
  locations:
    - azure:
        armTemplateResourcesExtra:
          - apiVersion: '2020-05-01'
            type: Microsoft.Network/publicIPPrefixes
            comments: Static Public IP address CIDR for Kublr Infra
            location: '[parameters(''region'')]'

            # specify the name of the resource here;
            # it will be used in other resources for a reference
            name: Kublr-Infra-PublicIPs

            properties:

              # specify prefix length here
              prefixLength: 30

              publicIPAddressVersion: IPv4
            sku:
              name: Standard
              tier: Regional
            tags:

              # specify Kublr cluster name here
              KubernetesCluster: my-cluster-name


Please note: at least properties.prefixLenght and name and tags.KubernetesCluster fields need to be set accordingly.


4. Using public IP address prefix for cluster IP addresses


There are usually three categories of public IP addresses created in a Kublr Kubernetes cluster:


  1. Public IP address for the cluster Kubernetes master API load balancer endpoint
  2. Public IP address for the cluster NAT gateway
  3. Public IP addresses created by Kubernetes for Services of type LoadBalancer


Using public IP prefix for addresses from each of this categories is described in the following sections.


4.1. NAT Gateway


spec:
  locations:
    - azure:
        armTemplateExtras:
          natGateway:
            properties:
              publicIpPrefixes:
                - id: '[concat(''/subscriptions/'', subscription().subscriptionId, ''/resourceGroups/rgCommon/providers/Microsoft.Network/publicIPPrefixes/IPCommon'')]'
          natGatewayPublicIP:
            dependsOn:
              - '[resourceId(''Microsoft.Network/publicIPPrefixes'', ''PublicIPs'')]'
            properties:
              publicIPPrefix:
                id: '[resourceId(''Microsoft.Network/publicIPPrefixes'', ''PublicIPs'')]'

4.2. Master LoadBalancer


The following cluster spec snippet shows using the prefix includes in the Kublr cluster spec:


spec:
  locations:
    - azure:
        armTemplateExtras:
          loadBalancerPublicIP:

            # this dependency specification is only necessary if the prefix
            # resource is included in the cluster spec
            dependsOn:
              - '[resourceId(''Microsoft.Network/publicIPPrefixes'', ''PublicIPs-Prefix-Name'')]'

            properties:
              publicIPPrefix:

                # Here the id of the IP prefix resource should be specified;
                # It can be either the expression for the prefix included in
                # the cluster spec, or a literal ID of a prefix resource created
                # outside of the Kublr cluster spec
                id: '[resourceId(''Microsoft.Network/publicIPPrefixes'', ''PublicIPs-Prefix-Name'')]'


The following is an example of using a prefix resource not included in the Kublr cluster spec and from a different resource group:


spec:
  locations:
    - azure:
        armTemplateExtras:
          loadBalancerPublicIP:
            properties:
              publicIpPrefixes:
                - id: '[concat(''/subscriptions/'', subscription().subscriptionId, ''/resourceGroups/my-resource-group/providers/Microsoft.Network/publicIPPrefixes/MyPrefixName'')]'


5. Reference