Tags: azure, loadbalancer, network


TABLE OF CONTENTS


Overview


Kublr configures a public and a private load balancers for each Azure Kubernetes cluster. The load balancers are used for Kubernetes Services of type LoadBalancer as well as the entry points for the Kubernetes cluster API.


By default Kublr configures a regular load balancer rule for Kubernetes API endpoint, which may limit certain ports usage in some rare scenarios.


For example when a master-only cluster is configured, or master nodes need to be allowed to tun user applications and serve incoming user applications traffic, certain limitations of Azure regular load balancer rules will prevent creating Kubernetes LoadBalancer Services with the same listener port as the port used by Kubernetes API (even though each such service allocates its own separate public IP address).


One way to overcome this limitation is using Floating IP load balancer rule for Kubernetes API.


This article describes how to configure a Kublr Kubernetes cluster to use Floating IP load balancer rules for Kubernetes API endpoints.


Configure FloatingIP use via Kublr cluster specification


The following modifications in the cluster specification will configure Floating IP for Kubernetes API:


spec:
  locations:
    - azure:
        armTemplateExtras:

          # modify public LB K8S API rule FloatingIP property in the ARM template
          loadBalancerPublicRule:
            properties:
              enableFloatingIP: true

          # modify private LB K8S API rule FloatingIP property in the ARM template
          loadBalancerPrivateRule:
            properties:
              enableFloatingIP: true

          # add a network security rule enabling K8S API traffic in the vnet
          resources:
            - name: "[concat(variables('k8sSecurityGroup'),'/allow_kube_tls2')]"
              type: Microsoft.Network/networkSecurityGroups/securityRules
              apiVersion: '2022-01-01'
              properties:
                description: Allow kube-apiserver (tls) traffic to master
                protocol: Tcp
                sourcePortRange: '*'
                sourceAddressPrefix: '*'
                destinationPortRange: 6443-6443
                access: Allow
                priority: 201
                direction: Inbound
                destinationAddressPrefixes:
                  - "[concat(reference(variables('k8sMasterIP')).ipAddress,'/32')]"
                  - "[concat(reference(variables('k8sLoadBalancerPrivate')).frontendIpConfigurations[0].properties.privateIPAddress,'/32')]"

  master:
    kublrAgentConfig:
      kublr:
        setup:
          cmd:

            # on masters add public and private LB addresses to the loopback interface
            # (this guest OS configuration is required for Azure Floating IP to work)
            before:
            - '/bin/sh'
            - '-c'
            - >-
              ip addr replace ${k8sMasterPublicEndpointAddress}/32  dev lo:0 label lo:0:kpub ;
              ip addr replace ${k8sMasterPrivateEndpointAddress}/32 dev lo:0 label lo:0:kpri


References