By default, Kublr automatically creates all AWS resources required to run a secure Kubernetes cluster in a single AWS CloudFormation stack.


This includes a set of standard IAM roles and instance profiles for master and worker nodes of the cluster.


At the same time, sometimes it is necessary to use IAM user roles and instance profiles already created in AWS to implement complex or extended use-cases.


The following resources were provisioned in AWS and are referenced in the example below:

  1. IAM policies for master and worker nodes = RoleMaster_testIAMrole; RoleNode_testIAMrole
  2. IAM roles = KublrIAMtest-RoleMaster; KublrIAMtest-RoleNode
  3. IAM instance profiles = KublrIAMtest-RoleMaster-IP; KublrIAMtest-RoleNode-IP


As a prerequisite to this scenario, AWS user account should be provisioned with required permissions.

The following example IAM policy could be used for such scenario:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetUser",
                "iam:GetRole",
                "iam:PassRole",
                "iam:ListRoles",
                "iam:PutRolePolicy",
                "iam:DeleteRolePolicy",
                "iam:DeleteRole",
                "iam:RemoveRoleFromInstanceProfile",
                "iam:DeleteInstanceProfile",
                "iam:AddRoleToInstanceProfile",
                "cloudformation:CreateUploadBucket",
                "cloudformation:ListStacks",
                "cloudformation:CancelUpdateStack",
                "cloudformation:DescribeStackResources",
                "cloudformation:SignalResource",
                "cloudformation:UpdateTerminationProtection",
                "cloudformation:DescribeStackResource",
                "cloudformation:CreateChangeSet",
                "cloudformation:DeleteChangeSet",
                "cloudformation:GetTemplateSummary",
                "cloudformation:DescribeStacks",
                "cloudformation:ContinueUpdateRollback",
                "cloudformation:GetStackPolicy",
                "cloudformation:DescribeStackEvents",
                "cloudformation:CreateStack",
                "cloudformation:GetTemplate",
                "cloudformation:DeleteStack",
                "cloudformation:UpdateStack",
                "cloudformation:DescribeChangeSet",
                "cloudformation:ExecuteChangeSet",
                "cloudformation:ValidateTemplate",
                "cloudformation:ListChangeSets",
                "cloudformation:ListStackResources",
                "autoscaling:DescribeAutoScalingInstances",
                "autoscaling:DescribeAutoScalingGroups",
                "autoscaling:DescribePolicies",
                "autoscaling:DescribeTags",
                "autoscaling:DescribeLaunchConfigurations",
                "autoscaling:DescribeLoadBalancerTargetGroups",
                "autoscaling:DescribeLoadBalancers",
                "autoscaling:DescribeScalingActivities",
                "autoscaling:UpdateAutoScalingGroup",
                "autoscaling:TerminateInstanceInAutoScalingGroup",
                "autoscaling:SetDesiredCapacity",
                "autoscaling:AttachLoadBalancerTargetGroups",
                "autoscaling:CreateOrUpdateTags",
                "autoscaling:DeleteAutoScalingGroup",
                "autoscaling:CreateAutoScalingGroup",
                "autoscaling:AttachLoadBalancers",
                "autoscaling:CreateLaunchConfiguration",
                "autoscaling:DeleteLaunchConfiguration",
                "ec2:*",
                "tag:GetResources",
                "tag:GetTagValues",
                "tag:GetTagKeys",
                "s3:*",
                "logs:DescribeLogGroups",
                "logs:CreateLogGroup",
                "logs:PutRetentionPolicy",
                "logs:DeleteLogGroup"
            ],
            "Resource": "*"
        }
    ]
}

(Note above, that there are no permissions for "iam:CreateRole" and "iam:CreateInstanceProfile")


Please, see below the required permissions for using existing IAM roles for Kublr cluster creation.


KublrIAMtest-RoleMaster:

Trusted Entities:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "ec2.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

RoleMaster_testIAMrole (IAM policy)
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "ec2:*",
                "s3:*",
                "logs:*",
                "elasticloadbalancing:*",
                "iam:CreateServiceLinkedRole",
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability",
                "ecr:GetDownloadUrlForLayer",
                "ecr:GetRepositoryPolicy",
                "ecr:DescribeRepositories",
                "ecr:ListImages",
                "ecr:BatchGetImage",
                "autoscaling:DescribeTags",
                "autoscaling:DescribeAutoScalingGroups",
                "autoscaling:DescribeLaunchConfigurations",
                "autoscaling:DescribeAutoScalingInstances",
                "autoscaling:SetDesiredCapacity",
                "autoscaling:TerminateInstanceInAutoScalingGroup",
                "rds:DescribeDBInstances",
                "cloudformation:DescribeStackResources",
                "cloudformation:DescribeStacks",
                "iam:CreateServiceLinkedRole",
                "kms:DescribeKey"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}


KublrIAMtest-RoleNode:

Trusted Entities:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "ec2.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

RoleNode_testIAMrole (IAM policy)
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "ec2:Describe*",
                "s3:*",
                "logs:*",
                "ec2:CreateSnapshot",
                "ec2:CreateTags",
                "ec2:DeleteSnapshot",
                "ec2:AttachVolume",
                "ec2:DetachVolume",
                "ec2:AssociateAddress",
                "ec2:DisassociateAddress",
                "ec2:ModifyInstanceAttribute",
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability",
                "ecr:GetDownloadUrlForLayer",
                "ecr:GetRepositoryPolicy",
                "ecr:DescribeRepositories",
                "ecr:ListImages",
                "ecr:BatchGetImage",
                "autoscaling:DescribeTags",
                "autoscaling:DescribeAutoScalingGroups",
                "autoscaling:DescribeLaunchConfigurations",
                "autoscaling:DescribeAutoScalingInstances",
                "autoscaling:SetDesiredCapacity",
                "autoscaling:TerminateInstanceInAutoScalingGroup",
                "rds:DescribeDBInstances",
                "cloudformation:DescribeStackResources",
                "cloudformation:DescribeStacks"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}



The example below shows how IAM roles and instance profiles created outside Kublr can be used.


During cluster creation steps, you may populate IAM roles and instance profiles for master and worker nodes in the Kublr UI.


CLUSTER -> Advanced Options -> AWS IAM for Master nodes; AWS IAM for Worker Nodes



Also note below how custom IAM roles definition(s) can be included in the cluster specification and referred to:

kind: Cluster
metadata:
  name: cluster-1659946822
spec:
  locations:
    - aws:
        awsApiAccessSecretRef: test-iamtest
        enableTerminationProtection: false
        iamInstanceProfileMasterPathName: null
        iamInstanceProfileNodePathName: null
        iamInstanceProfileMasterPathName: KublrIAMtest-RoleMaster-IP
        iamInstanceProfileNodePathName: KublrIAMtest-RoleNode-IP
        iamRoleMasterPathName: KublrIAMtest-RoleMaster
        iamRoleNodePathName: KublrIAMtest-RoleNode
        region: us-west-2
        vpcCidrBlock: null
        vpcId: null
      name: aws1
  master:
    kublrAgentConfig:
      kublr:
        psp:
          default_clusterrole: 'psp:privileged'
    kublrVariant: aws-ubuntu-20.04
    locations:
      - aws:
          availabilityZones:
            - us-west-2a
          instanceMonitoring: false
          instanceType: t3.large
          rootVolume:
            size: 40
          sshKey: test-aws-us-west-2
          subnetIds:
            - ''
        locationRef: aws1
    minNodes: 1
  network:
    apiServerSecurePort: 6443
  nodes:
    - autoscaling: false
      kublrVariant: aws-ubuntu-20.04
      locations:
        - aws:
            availabilityZones:
              - us-west-2a
              - us-west-2b
            instanceMonitoring: false
            instanceType: t3.xlarge
            rootVolume:
              size: 40
            sshKey: test-aws-us-west-2
            subnetIds:
              - ''
              - ''
          locationRef: aws1
      minNodes: 2
      name: group1


As a reference, please, consult with official Kublr documentation: