Tags: ingress, aws, azure, google, vmware, vcd, baremetal, onprem


TABLE OF CONTENTS


Overview


When a cluster is created in Kublr with ingress feature enabled, Kublr installs a standard Kubernetes ingress controller using its standard helm chart. It can therefore be customized using this chard configuration parameters.


Kublr users have full control over this chart configuration via the cluster specification, so it is possible to specify and/or override any parameters for the ingress controller chart.


In particular it is possible to customize both Nginx configuration and Kubernetes deployment objects of the ingress controller, such as its Service load balancer.


Nginx ingress controller helm chart values can be customized in the Kublr cluster specification as follows:


spec:
  features:
    ingress:
      values:
        nginx-controller:
          ...


Some examples of the ingress controller customization are listed in the "Configuration" section of the chart documentation


Full list of parameters available for configuration is also available in the helm chart values.yaml file.


All parameters available for customization via the values can also be found in the chart documentation chart documentation


Some common customizations are shown below.


Disable/customize HTTP/HTTPS ports


This can be done via chart parameters controller.service.enableHttp , controller.service.enableHttps etc as shown below in Kublr cluster specification snippet:


spec:
  features:
    ingress:
      values:
        nginx-ingress:
          controller:
            service:
              enableHttp: false
              enableHttps: true
              ports:
                http: 80
                https: 443


Customize default SSL certificate secret


This can be done via chart parameters controller.extraArgs etc as shown below in Kublr cluster specification snippet:

spec:
  features:
    ingress:
      values:
        nginx-ingress:
          controller:
            extraArgs:
              default-ssl-certificate: kube-system/custom-secret-name-tls

Ingress ELB Customization in AWS Clusters


Ingress ELB in AWS is created by Kubernetes AWS cloud provider in response to the ingress controller chart creating a Service of LoadBalancer type. Therefore any required customization of the ELB  must be done via Kubernetes Service annotations that in turn can be specified in the ingress controller chart's values in the Kublr cluster specification.


Various annotation that can be specified on the Service are described in the Kubernetes documentation at https://kubernetes.io/docs/concepts/services-networking/service/#internal-load-balancer (pick AWS tab for AWS or other tabs for other clouds).


Additional information about AWS cloud provide service annotations may be found here: https://cloud-provider-aws.sigs.k8s.io/service_controller/


For example, a load balancer type (ELB or NLB) can be selected via service.beta.kubernetes.io/aws-load-balancer-type annotation in Kublr cluster specification as follows:


spec:
  features:
    ingress:
      values:
        nginx-ingress:
          controller:
            service:
              enableHttp: false
              annotations:
                service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
                service.beta.kubernetes.io/aws-load-balancer-security-groups: sg-ab32hc33


This snippet disables HTTP on the ingress entry point, selects NLB as the type for the load balancer and instructs Kubernetes to associate the specified security group with the load balancer.


Specify Ingress ELB S3 Bucket Policy in AWS Clusters


AWS ELB access logs can be enabled using the annotations described in https://kubernetes.io/docs/concepts/services-networking/service/#elb-access-logs-on-aws


The use the following Kublr cluster specification snipped as an example:


spec:
  features:
    ingress:
      values:
        nginx-ingress:
          controller:
            service:
              annotations:
                service.beta.kubernetes.io/aws-load-balancer-access-log-enabled: "true"
                service.beta.kubernetes.io/aws-load-balancer-access-log-emit-interval: "60" # can be either "5" or "60"
                service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-name: "my-bucket"
                service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix: "my-bucket-prefix/prod"