Tags: aws, cost


TABLE OF CONTENTS


Overview


By default Kublr creates all master and worker instance groups using regular AWS on-demand instances and autoscaling groups with launch templates with a single instance type specified in the cluster specification.

At the same time, Kublr allows customizing instance group configuration to implement complex or extended use-cases.


Spot instances with groups of type 'asg-mip'


The following cluster specification snippet shows an example of such customization with a node group using spot instances and mixed instance policy (the example shows master group customization, but any worker groups can be customized in the same manner):


spec:
  nodes:
    - name: group1
      locations:
        - aws:
            # the main instance type
            instanceType: t3.medium

            # group type "asg-mip" means that Kublr will create an ASG with
            # a launch template and a mixed instance policy
            groupType: asg-mip

            # cloud formation mixed instance type section customizations
            mixedInstancesPolicyCloudFormationExtras:
              InstancesDistribution:
                # on-demand percentage set to 0 means that only spot
                # instances will be used
                OnDemandPercentageAboveBaseCapacity: 0
              LaunchTemplate:
                # other instance types to be used if the main
                # one is unavailable
                Overrides:
                  - InstanceType: t3.medium
                  - InstanceType: t2.medium


Spot instances with groups of type 'asg-lt'


The following example shows launch template customization to use spot instances without mixed instance type:


spec:
  nodes:
    - name: group1
      locations:
        - aws:
            # the main instance type
            instanceType: t3.medium

            # group type "asg-lt" means that Kublr will create an ASG with
            # a launch template (no mixed instance policy)
            groupType: asg-lt

            # launch template customization - use spot instances
            launchTemplateDataCloudFormationExtras:
              InstanceMarketOptions:
                MarketType: spot
                SpotOptions:
                  # this is required as the spot instances are started as a part
                  # of an auto-scaling group
                  SpotInstanceType: one-time


Kublr pre-1.24.0 deprecation notice


In Kublr 1.24.0 cluster specification properties mixedInstancePolicyCloudFormationExtras and launchTemplateDataCloudFormationExtras were deprecated in favor of more consistent data structure.


Configuration migration to the new specification format is performed automatically.

The examples above will look as follows in Kublr 1.24.0+.


Spot instances for groups of type asg-mip:


spec:
  nodes:
    - name: group1
      locations:
        - aws:
            # the main instance type
            instanceType: t3.medium

            # group type "asg-mip" means that Kublr will create an ASG with
            # a launch template and a mixed instance policy
            groupType: asg-mip

            # cloud formation mixed instance type section customizations
            cloudFormationExtras:
              autoScalingGroup:
                Properties:
                  MixedInstancesPolicy:
                    InstancesDistribution:
                      OnDemandPercentageAboveBaseCapacity: 0
                    LaunchTemplate:
                      Overrides:
                        - InstanceType: t3.medium
                        - InstanceType: t2.medium


Spot instances for groups of type asg-lt:


spec:
  nodes:
    - name: group1
      locations:
        - aws:
            # the main instance type
            instanceType: t3.medium

            # group type "asg-lt" means that Kublr will create an ASG with
            # a launch template (no mixed instance policy)
            groupType: asg-lt

            # launch template customization - use spot instances
            cloudFormationExtras:
              launchTemplate:
                Properties:
                  LaunchTemplateData:
                    InstanceMarketOptions:
                      MarketType: spot
                      SpotOptions:
                        SpotInstanceType: one-time