Tags: https, ingress, ssl, tls, security

TABLE OF CONTENTS

• Ciphers used by Kubernetes components and etcd
• Ciphers used by Ngninx ingress controller

Ciphers used by Kubernetes components and etcd

TLS ciphers have had a number of known vulnerabilities and weaknesses, which can reduce the protection provided by them. By default Kubernetes supports a number of TLS ciphersuites including some that have security concerns, weakening the protection provided. The set of TLS ciphers used by Kubernetes components may be adjusted via Kublr cluster specification as follows (note the different parameters used for kubelet for pre- and post-1.18 Kublr:

spec:
  kublrAgentConfig:
    kublr:
      kubelet_flag: # use with Kublr before 1.18
        tls_cipher_suites: '--tls-cipher-suites=...'
      kubelet_config:  # use with Kublr 1.18 and later
        tlsCipherSuites: 
          - '...'
          - '...'
          - '...'
      kube_api_server_flag:
        tls_cipher_suites: '--tls-cipher-suites=...'
      kube_controller_manager_flag:
        tls_cipher_suites: '--tls-cipher-suites=...'
      etcd_flag:
        cipher_suites: '--cipher-suites=...'

The list of ciphers is a comma-separated list of names found at https://golang.org/pkg/crypto/tls/#pkg-constants

The following cipher lists were tested and found to be compatible with Kublr control plane:

1. all ciphers supported by RHEL minus two ciphers using DES

TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_GCM_SHA384

2. ciphers supported by RHEL that are not included in HTTP/2 blacklist (https://http2.github.io/http2-spec/#BadCipherSuites)

TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

NB! Over time, the list of strong ciphers may change. Up-to-date information can be found at the following link: SSL and TLS Deployment Best Practices

Ciphers used by Ngninx ingress controller

The following Kublr cluster specification snippet shows how the set of ciphers used by the default Kublr Nginx ingress controller can be adjusted:

spec:
  features:
    ingress:
      values:
        nginx-ingress:
          controller:
            config:
              ssl-ciphers: ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384

              # This parameter can be used to limit the supported SSL protocols
              # ssl-protocols: "TLSv1 TLSv1.1 TLSv1.2 TLSv1.3"

              # The namespace and the name of the secret with DH key necessary for the DHE-* ciphers to work
              # ssl-dh-param: "kube-system/lb-dhparam"

Note, that unlike with etcd and Kubernetes components, the cipher list is colon-separated, rather than comma-separated.

Please refer to Nginx ingress controller documentation for more information about enabling different ciphers:

https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#ssl-ciphers

https://kubernetes.github.io/ingress-nginx/user-guide/tls/#default-tls-version-and-ciphers

Note additionally that enabling some ciphers may require additional configuration. In particular DHE-* ciphers require an additional K8S object (secret) created and referred to in the configuration parameter ssl-dh-param as described in the ingress controller documentation and as shown in the example above.

The following script can be used to create the secret (check the mentioned support and documentation articles for more details):

# generate DH key

KEY=$(openssl dhparam 4096 | base64 -w0)
echo "$KEY"

# create a K8S secret with the key

kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: lb-dhparam
  namespace: kube-system
data:
  dhparam.pem: "$KEY"
EOF

NB! the actual set of enabled ciphers may be smaller then the specified list depending on the version of the ingress controller.

The actual set of ciphers exposed on the ingress controller endpoints can be verified using openssl client with build in support for the required ciphers as described in the following SSL/TLS/HTTPS troubleshooting support article.