Tags: security, ssl, tls, letsencrypt, dns


TABLE OF CONTENTS


Overview


By default Kublr uses an HTTP solver in Certmanager Letsencrypt configuration (https://cert-manager.io/docs/configuration/acme/http01/). Default clusterissuer named letsencrypt is created automatically on Certmanager installation.

In some cases you cannot use HTTP solver (firewall rules for ingress, HTTPS-only policy, air-gaped environment, etc). Certmanager and Letsencrypt provide DNS01 challenge solver configuration (https://cert-manager.io/docs/configuration/acme/dns01/).

Also we have two different scenarios in terms of where Kublr cluster is hosted: 1) cluster hosted in AWS 2) cluster hosted on other platform (Azure, GCP,vSpere, etc).


Prerequisites

  • Access to Cluster specification for editing
  • AWS Route53 access account and AWS AIM policies applaying permissions


Letsencrypt DNS validation with AWS Route53


AWS-Hosted Kublr Cluster


You should use this approach whether Kublr cluster and Route53 are in the same AWS account or in different ones. Let's call Route53 AWS account  DnsAccount and call Kublr cluster AWS account KublrAccount.


  1. Create an IAM snap-policy in DnsAccount to provide minimal access to Route53 (https://cert-manager.io/docs/configuration/acme/dns01/route53/#set-up-an-iam-role). Let's call policy dns-policy*.
    This policy can be added via the Cluster Specification thus avoiding the need to do it manually as described in the next steps 2-6. Kublr will create and manage the policy automatically.
    The following is a cluster spec snipped for the policy for Kublr cluster and Route53 in the same AWS:
      locations:
    - aws:
       ...
        iamRoleMasterCloudFormationExtras:
          Properties:
            Policies:
              - PolicyDocument:
                  Statement:
                    - Action:
                        - 'route53:GetChange'
                      Effect: Allow
                      Resource: 'arn:aws:route53:::change/*'
                    - Action:
                        - 'route53:ChangeResourceRecordSets'
                        - 'route53:ListResourceRecordSets'
                      Effect: Allow
                      Resource: 'arn:aws:route53:::hostedzone/{ROUTE_53_HOSTED_ZONE_ID}'
                  Version: '2012-10-17'
                PolicyName: dns-policy

    If KublrAccount and DNSAccount are different, the policy in the cluster specification should look as follows:

      locations:
    - aws:
        ...
        iamRoleMasterCloudFormationExtras:
          Properties:
            Policies:
              - PolicyDocument:
                  Statement:
                    - Action:
                        - 'sts:AssumeRole'
                      Effect: Allow
                      Resource:  {{ DNS-MANAGER_ROLE_ARN }}
                  Version: '2012-10-17'
                PolicyName: dns-policy

    Alternatively, if you want to configure the policy manually, follow the steps 2-6:

  2. Go to DnsAccount  and create IAM role in DnsAccount and attach dns-policy. Let's call this role dns-manager. see https://cert-manager.io/docs/configuration/acme/dns01/route53/#cross-account-access).
  3. Go to KublrAccount and create policy (Let's call it cert-policy). This policy should allow Assume role for dns-manager role. So as a resource in this policy spec please use dns-manager role ARN.  See details here  https://cert-manager.io/docs/configuration/acme/dns01/route53/#cross-account-access.
  4. Go to KublrAccount, and attach cert-policy from step 3 to roles both for cluster Master EC2 instances and for Worker EC2 instances. Also get EC2s roles' ARNs to use them on step 5.
  5. Go to DnsAccount, open dns-manager role and add role ARNs from previous step to Trusted membersip. Get dns-manager role ARN to use it below in cluster spec as DNS-MANAGER_ROLE_ARN.
  6. Go to DnsAccount  and get AWS Route53 HostedZoneID for your domain. Use it as ROUTE_53_HOSTED_ZONE_ID in Kublr spec below.


GCP/Azure/Other-Hosted Kublr Cluster (non AWS clusters)


In this case we have AWS account with Route53. All steps below should be done in this account.

  1. Create IAM policy to provide minimal access to Route53 (https://cert-manager.io/docs/configuration/acme/dns01/route53/#set-up-an-iam-role). Let's call policy dns-policy.
  2. Create IAM user (see https://cert-manager.io/docs/configuration/acme/dns01/route53/#credentials), attach created dns-policy to this IAM user and get accessKeyID (AWS_ACCESS_KEY_ID) and secretAccessKey (IAM_USER_SECRET_KEY) for created IAM user.
  3. Create AWS Secret in the target Kubernetes cluster in the CertManager namespace (usually kube-system) for CertManager to be able to access AWS Route53

  4. $ kubectl create secret -n kube-system generic aws-route-53-access-key --from-literal=secret-access-key='{{IAM_USER_SECRET_KEY}}'

  5. You can create this secret via Kublr Cluster spec or use plaintext secret in clusterissuer.

  6. Get AWS Route53 HostedZoneID for your domain. Use it as ROUTE_53_HOSTED_ZONE_ID in Kublr spec below.


Kublr Cluster Specification Adjustments to Use Letsencrypt DNS solver


Note that the following spec is not a full cluster specification, it only includes excerpts that have to be added to a full cluster specification in order to set up Let's Encrypt DNS validation.


Spec for AWS-hosted Kublr cluster 

spec:
...
  packages:
    route-53-issuer:
      releaseName: route-53-issuer
      namespace: kube-system
      helmVersion: v3.5.2
      chart:
        name: raw
        repoUrl: 'https://charts.helm.sh/incubator/packages'
        version: 0.2.5
      values:
        resources:
          - apiVersion: cert-manager.io/v1
            kind: ClusterIssuer
            metadata:
              name: letsencrypt-route53
            spec:
              acme:
                email: {{ ACME_ACCOUNT_EMAIL }}
                privateKeySecretRef:
                  name: letsencrypt-route53
                server: 'https://acme-v02.api.letsencrypt.org/directory'
                solvers:
                  - dns01:
                      route53:
                        hostedZoneID: {{ ROUTE_53_HOSTED_ZONE_ID }}
                        region: global
                        role: {{ DNS-MANAGER_ROLE_ARN }}

Spec for NON AWS-hosted Kublr cluster

spec:
...
  packages:
    route-53-issuer:
      releaseName: route-53-issuer
      namespace: kube-system
      helmVersion: v3.5.2
      chart:
        name: raw
        repoUrl: 'https://charts.helm.sh/incubator/packages'
        version: 0.2.5
      values:
        resources:
          - apiVersion: cert-manager.io/v1
            kind: ClusterIssuer
            metadata:
              name: letsencrypt-route53
            spec:
              acme:
                email: {{ ACME_ACCOUNT_EMAIL }}
                privateKeySecretRef:
                  name: letsencrypt-route53
                server: 'https://acme-v02.api.letsencrypt.org/directory'
                solvers:
                  - dns01:
                      route53:
                        accessKeyID: {{ AWS_ACCESS_KEY_ID }}
                        hostedZoneID: {{ ROUTE_53_HOSTED_ZONE_ID }}
                        region: global
                        secretAccessKeySecretRef:
                          name: aws-route-53-access-key
                          key: secret-access-key
         # Optionally, if you create secret manually on step 2
          - apiVersion: v1
            kind: Secret
            type: Opaque
            metadata:
              name: aws-route-53-access-key
            data:
              secret-access-key: {{BASE64_ENCODED_ACCESS_KEY }}

Change default issuer for certmanager:

spec:
...
  features:
    ingress:
      values:
        certmanager:
          dns01RecursiveNameservers: kube-dns.kube-system.svc.{{.spec.network.dnsDomain}}:53,8.8.8.8:53
          dns01RecursiveNameserversOnly: true
          ingressShim:
            defaultIssuerName: letsencrypt-route53
...

How to use different cert solvers in the same cluster

You can manually define which cert issuer to use for specific ingress rules using annotations:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-route53
    kubernetes.io/tls-acme: "true"
...


DNS solver:

cert-manager.io/cluster-issuer: letsencrypt-route53


HTTP Solver:

cert-manager.io/cluster-issuer: letsencrypt