By default, Kublr automatically creates all AWS resources required to run a secure Kubernetes cluster in a single AWS CloudFormation stack.
This includes a set of standard IAM roles and instance profiles for master and worker nodes of the cluster.
At the same time, sometimes it is necessary to use IAM user roles and instance profiles already created in AWS to implement complex or extended use-cases.
The following resources were provisioned in AWS and are referenced in the example below:
- IAM policies for master and worker nodes = RoleMaster_testIAMrole; RoleNode_testIAMrole
- IAM roles = KublrIAMtest-RoleMaster; KublrIAMtest-RoleNode
- IAM instance profiles = KublrIAMtest-RoleMaster-IP; KublrIAMtest-RoleNode-IP
As a prerequisite to this scenario, AWS user account should be provisioned with required permissions.
The following example IAM policy could be used for such scenario:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:GetUser",
"iam:GetRole",
"iam:PassRole",
"iam:ListRoles",
"iam:PutRolePolicy",
"iam:DeleteRolePolicy",
"iam:DeleteRole",
"iam:RemoveRoleFromInstanceProfile",
"iam:DeleteInstanceProfile",
"iam:AddRoleToInstanceProfile",
"cloudformation:CreateUploadBucket",
"cloudformation:ListStacks",
"cloudformation:CancelUpdateStack",
"cloudformation:DescribeStackResources",
"cloudformation:SignalResource",
"cloudformation:UpdateTerminationProtection",
"cloudformation:DescribeStackResource",
"cloudformation:CreateChangeSet",
"cloudformation:DeleteChangeSet",
"cloudformation:GetTemplateSummary",
"cloudformation:DescribeStacks",
"cloudformation:ContinueUpdateRollback",
"cloudformation:GetStackPolicy",
"cloudformation:DescribeStackEvents",
"cloudformation:CreateStack",
"cloudformation:GetTemplate",
"cloudformation:DeleteStack",
"cloudformation:UpdateStack",
"cloudformation:DescribeChangeSet",
"cloudformation:ExecuteChangeSet",
"cloudformation:ValidateTemplate",
"cloudformation:ListChangeSets",
"cloudformation:ListStackResources",
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribePolicies",
"autoscaling:DescribeTags",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeLoadBalancerTargetGroups",
"autoscaling:DescribeLoadBalancers",
"autoscaling:DescribeScalingActivities",
"autoscaling:UpdateAutoScalingGroup",
"autoscaling:TerminateInstanceInAutoScalingGroup",
"autoscaling:SetDesiredCapacity",
"autoscaling:AttachLoadBalancerTargetGroups",
"autoscaling:CreateOrUpdateTags",
"autoscaling:DeleteAutoScalingGroup",
"autoscaling:CreateAutoScalingGroup",
"autoscaling:AttachLoadBalancers",
"autoscaling:CreateLaunchConfiguration",
"autoscaling:DeleteLaunchConfiguration",
"ec2:*",
"tag:GetResources",
"tag:GetTagValues",
"tag:GetTagKeys",
"s3:*",
"logs:DescribeLogGroups",
"logs:CreateLogGroup",
"logs:PutRetentionPolicy",
"logs:DeleteLogGroup"
],
"Resource": "*"
}
]
}(Note above, that there are no permissions for "iam:CreateRole" and "iam:CreateInstanceProfile")
Please, see below the required permissions for using existing IAM roles for Kublr cluster creation.
KublrIAMtest-RoleMaster:
Trusted Entities:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
RoleMaster_testIAMrole (IAM policy)
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:*",
"s3:*",
"logs:*",
"elasticloadbalancing:*",
"iam:CreateServiceLinkedRole",
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:GetRepositoryPolicy",
"ecr:DescribeRepositories",
"ecr:ListImages",
"ecr:BatchGetImage",
"autoscaling:DescribeTags",
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup",
"rds:DescribeDBInstances",
"cloudformation:DescribeStackResources",
"cloudformation:DescribeStacks",
"iam:CreateServiceLinkedRole",
"kms:DescribeKey"
],
"Resource": "*",
"Effect": "Allow"
}
]
}KublrIAMtest-RoleNode:
Trusted Entities:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
RoleNode_testIAMrole (IAM policy)
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:Describe*",
"s3:*",
"logs:*",
"ec2:CreateSnapshot",
"ec2:CreateTags",
"ec2:DeleteSnapshot",
"ec2:AttachVolume",
"ec2:DetachVolume",
"ec2:AssociateAddress",
"ec2:DisassociateAddress",
"ec2:ModifyInstanceAttribute",
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:GetRepositoryPolicy",
"ecr:DescribeRepositories",
"ecr:ListImages",
"ecr:BatchGetImage",
"autoscaling:DescribeTags",
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup",
"rds:DescribeDBInstances",
"cloudformation:DescribeStackResources",
"cloudformation:DescribeStacks"
],
"Resource": "*",
"Effect": "Allow"
}
]
}The example below shows how IAM roles and instance profiles created outside Kublr can be used.
During cluster creation steps, you may populate IAM roles and instance profiles for master and worker nodes in the Kublr UI.
CLUSTER -> Advanced Options -> AWS IAM for Master nodes; AWS IAM for Worker Nodes
Also note below how custom IAM roles definition(s) can be included in the cluster specification and referred to:
kind: Cluster
metadata:
name: cluster-1659946822
spec:
locations:
- aws:
awsApiAccessSecretRef: test-iamtest
enableTerminationProtection: false
iamInstanceProfileMasterPathName: null
iamInstanceProfileNodePathName: null
iamInstanceProfileMasterPathName: KublrIAMtest-RoleMaster-IP
iamInstanceProfileNodePathName: KublrIAMtest-RoleNode-IP
iamRoleMasterPathName: KublrIAMtest-RoleMaster
iamRoleNodePathName: KublrIAMtest-RoleNode
region: us-west-2
vpcCidrBlock: null
vpcId: null
name: aws1
master:
kublrAgentConfig:
kublr:
psp:
default_clusterrole: 'psp:privileged'
kublrVariant: aws-ubuntu-20.04
locations:
- aws:
availabilityZones:
- us-west-2a
instanceMonitoring: false
instanceType: t3.large
rootVolume:
size: 40
sshKey: test-aws-us-west-2
subnetIds:
- ''
locationRef: aws1
minNodes: 1
network:
apiServerSecurePort: 6443
nodes:
- autoscaling: false
kublrVariant: aws-ubuntu-20.04
locations:
- aws:
availabilityZones:
- us-west-2a
- us-west-2b
instanceMonitoring: false
instanceType: t3.xlarge
rootVolume:
size: 40
sshKey: test-aws-us-west-2
subnetIds:
- ''
- ''
locationRef: aws1
minNodes: 2
name: group1
As a reference, please, consult with official Kublr documentation: