By default Kublr automatically creates all AWS resources required to run a secure Kubernetes cluster in a single AWS CloudFormation stack.
This includes a set of standard VPC, subnets, CIDRs for master and worker nodes of the cluster.
At the same time, sometimes it is necessary to use existing VPC, subnets, CIDRs, S3 bucket already created in AWS to implement complex or extended use-cases.
The following resources were pre-created in AWS and are referenced in the example below:
- VPC = vpc-0e4def1bc72b9d01d (VPC IP CIDR block: 10.0.0.0/23)
- Subnet for master nodes = subnet-0fd907786669ac382 (IPv4 CIDR: 10.0.0.0/27)
- Subnet for worker nodes = subnet-01d3c9315c0ca79d1 (IPv4 CIDR: 10.0.1.0/27)
- S3 bucket = cluster-1658917632-secrets-xxxxxxx
Kublr supports it via specification of existing VPC, subnets, CIDRs, S3 bucket.
As a prerequisite to this scenario, AWS user account should be provisioned with required permissions.
The following example IAM policy could be used for such scenario:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:GetUser",
"iam:GetRole",
"iam:PassRole",
"iam:ListRoles",
"iam:CreateRole",
"iam:PutRolePolicy",
"iam:DeleteRolePolicy",
"iam:DeleteRole",
"iam:CreateInstanceProfile",
"iam:RemoveRoleFromInstanceProfile",
"iam:DeleteInstanceProfile",
"iam:AddRoleToInstanceProfile",
"cloudformation:CreateUploadBucket",
"cloudformation:ListStacks",
"cloudformation:CancelUpdateStack",
"cloudformation:DescribeStackResources",
"cloudformation:SignalResource",
"cloudformation:UpdateTerminationProtection",
"cloudformation:DescribeStackResource",
"cloudformation:CreateChangeSet",
"cloudformation:DeleteChangeSet",
"cloudformation:GetTemplateSummary",
"cloudformation:DescribeStacks",
"cloudformation:ContinueUpdateRollback",
"cloudformation:GetStackPolicy",
"cloudformation:DescribeStackEvents",
"cloudformation:CreateStack",
"cloudformation:GetTemplate",
"cloudformation:DeleteStack",
"cloudformation:UpdateStack",
"cloudformation:DescribeChangeSet",
"cloudformation:ExecuteChangeSet",
"cloudformation:ValidateTemplate",
"cloudformation:ListChangeSets",
"cloudformation:ListStackResources",
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribePolicies",
"autoscaling:DescribeTags",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeLoadBalancerTargetGroups",
"autoscaling:DescribeLoadBalancers",
"autoscaling:DescribeScalingActivities",
"autoscaling:UpdateAutoScalingGroup",
"autoscaling:TerminateInstanceInAutoScalingGroup",
"autoscaling:SetDesiredCapacity",
"autoscaling:AttachLoadBalancerTargetGroups",
"autoscaling:CreateOrUpdateTags",
"autoscaling:DeleteAutoScalingGroup",
"autoscaling:CreateAutoScalingGroup",
"autoscaling:AttachLoadBalancers",
"autoscaling:CreateLaunchConfiguration",
"autoscaling:DeleteLaunchConfiguration",
"ec2:AllocateAddress",
"ec2:AssociateIamInstanceProfile",
"ec2:AttachNetworkInterface",
"ec2:AttachVolume",
"ec2:CreateLaunchTemplate",
"ec2:CreateLaunchTemplateVersion",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DeleteLaunchTemplate",
"ec2:DeleteLaunchTemplateVersions",
"ec2:DeleteTags",
"ec2:DeleteVolume",
"ec2:DescribeVolumes",
"ec2:DetachVolume",
"ec2:DisassociateAddress",
"ec2:DisassociateIamInstanceProfile",
"ec2:ModifyInstanceAttribute",
"ec2:ModifyLaunchTemplate",
"ec2:ModifyNetworkInterfaceAttribute",
"ec2:ModifyVolumeAttribute",
"ec2:RebootInstances",
"ec2:ReleaseAddress",
"ec2:ReplaceIamInstanceProfileAssociation",
"ec2:RunInstances",
"ec2:StartInstances",
"ec2:StopInstances",
"ec2:TerminateInstances",
"ec2:DescribeImages",
"ec2:DescribeAvailabilityZones",
"ec2:CreateSecurityGroup",
"ec2:describeAddresses",
"ec2:DescribeSecurityGroups",
"ec2:DeleteSecurityGroup",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:RevokeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupEgress",
"ec2:RevokeSecurityGroupIngress",
"ec2:DescribeLaunchTemplates",
"ec2:DescribeLaunchTemplateVersions",
"ec2:DescribeKeyPairs",
"tag:GetResources",
"tag:GetTagValues",
"tag:GetTagKeys",
"s3:ListAllMyBuckets",
"s3:PutBucketPolicy",
"s3:CreateBucket",
"s3:DeleteBucket",
"s3:GetObject",
"logs:DescribeLogGroups",
"logs:CreateLogGroup",
"logs:PutRetentionPolicy",
"logs:DeleteLogGroup"
],
"Resource": "*"
},
{
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::cluster-1658917632-secrets-xxxxxxx",
"arn:aws:s3:::cluster-1658917632-secrets-xxxxxxx/*"
],
"Effect": "Allow"
}
]
}(The S3 bucket was also created and required permissions to access it are defined)
The example below shows how VPC, subnets, CIDRs created outside Kublr can be used.
This approach has certain limitations, in particular the VPC, subnets, CIDRs must already exist before the cluster is created or updated.
During cluster creation steps, you may populate VPC, VPC CIDR, subnets in the Kublr UI.
CLUSTER -> Advanced Options -> VPC: "VPC ID"; "VPC IP CIDR block"
INSTANCES -> Master Configuration -> Advanced Options -> Subnets
INSTANCES -> Workers Instance Group ("group 1" example) -> Advanced Options -> Subnets
Also note below how custom VPC, subnets, CIDRs, S3 bucket definition(s) can be included in the cluster specification and referred to:
kind: Cluster
metadata:
name: test-vpc-subnet-cidrs
spec:
locations:
- aws:
# each AWS instance group (workers or master) in Kublr runs
# in a specified set of AWS availability zones
availabilityZones:
- us-west-2a
- us-west-2b
- us-west-2c
# CIDRs are defined as cidrBlocks for each Master and Worker nodes
cidrBlocks:
masterPublic:
- 10.0.0.0/27
nodePublic:
- 10.0.1.0/27
region: us-west-2
# The VPC CIDR can also be specified as vpcCidrBlock
vpcCidrBlock: 10.0.0.0/23
# Existing VPC should be defined by its VPC id
vpcId: vpc-0e4def1bc72b9d01d
name: aws1
master:
maxNodes: 1
minNodes: 1
locations:
- aws:
availabilityZones:
- us-west-2a
imageId: ami-0aab355e1bfa1e72e
overrideImageId: ami-0aab355e1bfa1e72e
# subnetIds property allows to specify subnets to be used
# by the nodes of this instance group in each availability zone.
subnetIds:
- subnet-0fd907786669ac382
locationRef: aws1
name: master
nodes:
maxNodes: 2
minNodes: 2
locations:
- aws:
availabilityZones:
- us-west-2a
imageId: ami-0aab355e1bfa1e72e
overrideImageId: ami-0aab355e1bfa1e72e
sshKey: mykey-aws-us-west-2
# subnetIds property allows to specify subnets to be used
# by the nodes of this instance group in each availability zone.
subnetIds:
- subnet-01d3c9315c0ca79d1
locationRef: aws1
name: group2
secretStore:
awsS3:
locationRef: aws1
s3BucketName: cluster-1658917632-secrets-xxxxxxx
As a reference, please, consult with official Kublr documentation: