TABLE OF CONTENTS


Tags: aws, persistence, security, storage


Overview


Kublr supports cluster master and worker node EC2 instances with encrypted EBS volumes - both for root volumes and master etcd data volumes.


Implementation plan

  1. Create or ensure a KMS key (default AWS account KMS EBS key may be used)
  2. Configure the key permissions to allow use by Kublr, AWS EC2, AWS Autoscaling
  3. Configure Kublr cluster to use the KMS key


Additional references:


KMS Key Configuration


The KMS key policy should allow the key use by Kublr, AWS EC2 instances created by Kublr, and by AWS Autoscaling service.


While there are multiple ways of achieving this, this document describes one method that requires the least configuration when creating multiple clusters; refer to AWS documentation for more details on various options available to customize this approach. 


User can choose to use the EBS KMS key that AWS creates by default for each AWS account, or use a different Customer Master Key (CMK).


Note, that not all CMK can be used for EBS encryption, in particular asymmetric keys are not currently supported for EBS.


After the key is created, the following additional policies and permissions should be configured for the key:

  • permission for Kublr IAM account to use the key
  • permission for EBS service to use the key when attached to EC2 VMs
  • permission for Autoscaling service to use the key when starting EC2 VMs


KMS Key Policy - Kublr IAM account permissions


The following snippet shows a policy statement that should be added to the KMS key policy to allow Kublr IAM account to use the key:

NB! put correct AWS account ID and the IAM user name in the statement


{
  "Version": "2012-10-17",
  "Statement": [
    ...
    {
      "Sid": "Allow use of the key by Kublr",
      "Effect": "Allow",
      "Principal": { "AWS": "arn:aws:iam::<aws-account-id>:user/<kublr-iam-user-name>" },
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:DescribeKey"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Allow attachment of persistent resources by Kublr",
      "Effect": "Allow",
      "Principal": { "AWS": "arn:aws:iam::<aws-account-id>:user/<kublr-iam-user-name>" },
      "Action": [
        "kms:CreateGrant",
        "kms:ListGrants",
        "kms:RevokeGrant"
      ],
      "Resource": "*",
      "Condition": { "Bool": { "kms:GrantIsForAWSResource": "true" } }
    },
    ...
  ]
}


KMS Key Policy - EBS service permissions


The following snippet shows a policy statement that should be added to the KMS key policy to allow EBS service to access the key when attaching EBS to EC2 VMS:


NB! put correct AWS account ID and the region in the statement


{
  "Version": "2012-10-17",
  "Statement": [
    ...
    {
      "Sid": "Allow access through EBS for all principals in the account that are authorized to use EBS",
      "Effect": "Allow",
      "Principal": { "AWS": "*" },
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:CreateGrant",
        "kms:DescribeKey"
      ],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "kms:CallerAccount": "<aws-account-id>",
          "kms:ViaService": "ec2.<region>.amazonaws.com"
        }
      }
    },
    ...
  ]
}


KMS Key Policy - Autoscaling service permissions


The following snippet shows a policy statement that should be added to the KMS key policy to allow AWS Autoscaling service to access the key when attaching EBS to EC2 VMS:


NB! put correct AWS account ID and the region in the statement


{
  "Version": "2012-10-17",
  "Statement": [
    ...
    {
      "Sid": "Allow use of the key by AutoScaling",
      "Effect": "Allow",
      "Principal": { "AWS": "arn:aws:iam::<aws-account-id>:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling" },
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:DescribeKey"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Allow attachment of persistent resources by AutoScaling",
      "Effect": "Allow",
      "Principal": { "AWS": "arn:aws:iam::<aws-account-id>:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling" },
      "Action": [
        "kms:CreateGrant",
        "kms:ListGrants",
        "kms:RevokeGrant"
      ],
      "Resource": "*",
      "Condition": {
        "Bool": { "kms:GrantIsForAWSResource": "true" }
      }
    }
    ...
  ]
}


Kublr cluster configuration


Use KMS for the cluster master and worker nodes root and etcd data volumes


The following cluster specification snippet shows cluster specification properties related to encrypted EBS use for worker and master root volumes, and master etcd volumes:

spec:
  # For master instance group volume encryption can be
  # turned on and off separately.
  master:
    locations:
      - aws:

          # rootVolume section sets up the instance's root volume properties
          rootVolume:
            encrypted: true
            # AWS KMS key ID may be specified for the volume encryption
            kmsKeyId: 'alias/my-kms-key'

          # masterVolume section sets up the master etcd volume properties
          # This section can only be used in the master instance group
          masterVolume:
            encrypted: true
            # if kmsKeyId is omitted, the default AWS account KMS key is used
            kmsKeyId: 'alias/my-kms-key'

  # For each worker instance group volume encryption can be
  # turned on and off separately.
  nodes:
    - locations:
        - aws:

            # rootVolume section sets up the instance's root volume properties
            rootVolume:
              encrypted: true


kmsKeyId property may contain key ID, key alias, key ARN, or alias ARN as described in the AWS documentation.


Use KMS for the default cluster StorageClass


The following cluster specification snippet shows how to enable encryption for the default EBS storage class created by Kublr in AWS clusters:

spec:
  ...
  features:
    system:
      values:
        storageClass:
          aws:
            parameters:

              # note that the value of 'encrypted' key must be a sting 'true' or 'false'
              encrypted: 'true'

              # If 'kmsKeyId' parameter is omitted, then the default AWS account KMS key is used by AWS
              # kmsKeyId: '...'