TABLE OF CONTENTS

Overview and preparations

Velero provide a wide range of features from simple backup and restore to disaster recovery and cluster migration. It is a flexible and pretty universal tool for creation backups and restore whole kubernetes clusters with persistence storage.

Straight forward instruction from Velero side required additional user with wide range permissions. Here we are figure out how to do all necessary steps for Velero install with current cluster permissions and policies throughout Kublr Cluster specification. 


No need to create IAM user, this method allow use cluster's permissions  and policies of AWS for Velero


  • Step 1: Prerequisites

    • Kublr 1.18+ 

    • Access to AWS throughout CLI or UI for S3 bucket creation

    • Permissions for edit / applying Cluster specification in KCP

    • Possibility to run in Terminal app kubectl and velero commands

  • Step 2: Create AWS S3 bucket

Create S3 bucket in AWS via cli or UI interface. Velero requires an object storage bucket to store backups in, preferably unique to a single Kubernetes cluster. Create an S3 bucket, replacing placeholders appropriately:


aws s3api create-bucket \
    --bucket $AWS_BUCKET_NAME \
    --region $AWS_REGION_NAME

  • Step 3: Change configuration for Master Nodes in the Cluster specification. In this case Velero will be installed on Master Nodes, please ensure that the master nodes have enough resources and make sure that you monitor them. That will allow managing and extending Worker Nodes seamlessly:

  locations:
    - aws:
        iamRoleMasterCloudFormationExtras:
          Properties:
            Policies:
              - PolicyDocument:
                  Statement:
                    - Action:
                        - 's3:GetObject'
                        - 's3:DeleteObject'
                        - 's3:PutObject'
                        - 's3:AbortMultipartUpload'
                        - 's3:ListMultipartUploadParts'
                      Effect: Allow
                      Resource: 'arn:aws:s3:::<AWS_BUCKET_NAME>/*'
                    - Action:
                        - 's3:ListBucket'
                      Effect: Allow
                      Resource: 'arn:aws:s3:::<AWS_BUCKET_NAME>'
                  Version: '2012-10-17'
                PolicyName: <AWS-POLICY-NAME>
        natMode: multi-zone


packages:
  velero:
    chart:
      name: velero
      url: 'https://github.com/vmware-tanzu/helm-charts/releases/download/velero-2.30.1/velero-2.30.1.tgz' #current stable version of velero
    helmVersion: v3.5.2 
    namespace: velero-backup
    releaseName: velero
    values:
      backupsEnabled: true
      configuration:
        backupStorageLocation:
          accessMode: ReadWrite
          bucket: <AWS_BUCKET_NAME>
          config:
            region: <AWS_REGION_NAME>
          provider: aws
        logFormat: json
        logLevel: debug #optional, can be remove in UAT/Prod env's
        namespace: velero-backup
        provider: aws
        volumeSnapshotLocation:
          config:
            region: <AWS_REGION_NAME>
          provider: aws
      credentials:
        useSecret: false
      deployRestic: false
      dnsPolicy: ClusterFirst
      initContainers:
        - image: 'velero/velero-plugin-for-aws:v1.4.1' #current version of AWS plugin for Velero
          imagePullPolicy: IfNotPresent
          name: velero-plugin-for-aws
          volumeMounts:
            - mountPath: /target
              name: plugins
      limits: # must be changed accoding to cluster size and topology
        cpu: 1000m 
        memory: 512Mi
      nodeSelector:
        kublr.io/node-group: master #this setting annotate that Velero run in Master Nodes only
      resources: # must be changed accoding to cluster size and topology
        requests:
          cpu: 500m
          memory: 128Mi
      snapshotsEnabled: true
      tolerations:
        - effect: NoSchedule
          operator: Exists
        - effect: NoExecute
          operator: Exists
      upgradeCRDs: true

Validate and Update cluster specification with all this changes

  • Step 5: Install Velero client and check Velero functionality

Install Velero CLI according to Instruction for your platform

  • Step 6: Checking Velero functions

As example create Velero backup in Terminal App with command:

velero backup create whole-cluster-backup -n velero-backup


Look at created backup with command: 

velero backup describe whole-cluster-backup -n velero-backup

As example delete namespace via kubectl and restore it via velero restore command:


kubectl delete namespaces kubernetes-dashboard
velero restore create --from-backup whole-cluster-backup -n velero-backup