Bala's Blog
Automate Your AWS EC2 Backup: A Guide to EBS Snapshot Creation

Automate Your AWS EC2 Backup: A Guide to EBS Snapshot Creation

Bala's photo
Bala
·

9 min read

Securing your AWS EC2 instances is paramount, and a crucial component of this strategy is implementing robust backup practices. Leveraging AWS Elastic Block Store (EBS) snapshots is a key step in fortifying your data recovery and protection mechanisms.

These snapshots, capturing both root volumes and additional volumes, provide a valuable point-in-time backup.

They offer a cost-effective solution without compromising on the essential requirements of a backup strategy.

Streamlining the backup process for your EC2 instances is a cost-effective and straightforward task with AWS EBS Snapshots. This tutorial explores three efficient methods to automate EBS snapshots, ensuring the resilience of your data:

The three ways are as follows:

  1. AWS EBS Lifecycle Manager

  2. AWS CloudWatch Events

  3. AWS Lambda Functions

So lets gets started with the first efficient method to automate EBS snapshots,

AWS EBS Lifecycle Manager:

The AWS EBS Lifecycle Manager offers a seamless and native solution for the automated management of EBS volumes and snapshots. This approach is swift and user-friendly, relying on AWS tags to categorize EBS volumes.

Step 1: Tag your ec2 instance and volumes

EC2 EBS snapshots with the life cycle manager work with the instance & volume tags. It requires instances and volumes to be tagged to identify the snapshot candidate.

You can use the following tag in the instances and volumes that needs automated snapshots.

Key = Backup 
Value = True

Step 2: Go to the EBS life cycle manager to create a snapshot lifecycle policy.

Head over to the EC2 dashboard and select “Lifecycle Manager” option under ELASTIC BLOCK STORE category as shown below.

You will be taken to the life cycle manager dashboard. Click “Create Lifecycle Policy” button.

Step 3: Select Policy type as default or custom

For this case we will select custom policy since we are setting it manually with custom value.

Step 4: Add EBS snapshot life cycle policy rules

Enter the EBS snapshot policy details as shown below. Ensure that you select the right tags for the volumes you need the snapshot.

Note: You can add multiple tags to target specific Volumes

We will select default IAM role here.

Also select “enable policy” for the policy to be active immediately after creation.

Enter EBS snapshot schedule details based on your requirements. You can choose retention type for both count & age.

For regular ec2 ebs backups, count is the ideal way.

Also apply proper tags to identify the snapshots.

Click create policy.

Now the policy manager will automatically create snapshots based on the schedules you have added.

Create EBS Volume Snapshots With Cloudwatch Events

Utilizing AWS CloudWatch Events, you can schedule and trigger custom actions based on specified events or time intervals

Follow the steps outlined below to set up scheduled EBS snapshots.

First we need to create a Cloudwatch Schedule.

Head over to cloudwatch service and click create a rule under the event options as shown below.

Click on create rule, give a name for the rule and select rule type as schedule as shown below.

Click on continue to create rule.

You can choose either a fixed schedule or a cron expression. Under targets, select the “EBS Create Snapshot” option.

Get the Volume ID from the EBS volume information, apply it to the Volume ID field and select create a new role.

Create more targets if you want to take snapshot of more volumes.
Click create rule.

Thats it. Based on the cloudwatch schedules, the snapshots will be created.

Automate EBS snapshot Creation and Deletion With Lambda Function

When the AWS EC2 Lifecycle Manager doesn't fully meet specific requirements, Lambda-based snapshot creation becomes a versatile alternative, particularly for unscheduled activities.

Consider scenarios where snapshots need to be taken just before updating or upgrading stateful systems. In such use cases, an automation setup could involve triggering a Lambda function to execute the snapshot action seamlessly.

Getting Started With Lambda Based EBS snapshot

We will use Python 3.11 scripts, lambda, IAM role, and cloud watch event schedule for this setup.

For this lambda function to work, you need to create a tag named “backup” with the value true for all the instances for which you need a backup.

For setting up a lambda function for creating automated snapshots, you need to do the following.

  1. A snapshot creation python script with the necessary parameters.

  2. An IAM role with snapshot create, modify, and delete access.

  3. A lambda function.

Configure Python Script for EBS Snapshot

Following python code will create snapshots on all the instance which have a tag named “backup.”

Note: You can get all the code from here 

import boto3
import collections
import datetime

ec = boto3.client('ec2')

def lambda_handler(event, context):
    reservations = ec.describe_instances(
        Filters=[
            {'Name': 'tag-key', 'Values': ['backup', 'Backup']},
        ]
    ).get(
        'Reservations', []
    )

    instances = sum(
        [
            [i for i in r['Instances']]
            for r in reservations
        ], [])

    print("Found %d instances that need backing up" % len(instances))

    to_tag = collections.defaultdict(list)

    for instance in instances:
        try:
            retention_days = [
                int(t.get('Value')) for t in instance['Tags']
                if t['Key'] == 'Retention'][0]
        except IndexError:
            retention_days = 10

        for dev in instance['BlockDeviceMappings']:
            if dev.get('Ebs', None) is None:
                continue
            vol_id = dev['Ebs']['VolumeId']
            print("Found EBS volume %s on instance %s" % (
                vol_id, instance['InstanceId']))

            snap = ec.create_snapshot(
                VolumeId=vol_id,
            )

            to_tag[retention_days].append(snap['SnapshotId'])

            print("Retaining snapshot %s of volume %s from instance %s for %d days" % (
                snap['SnapshotId'],
                vol_id,
                instance['InstanceId'],
                retention_days,
            ))

    for retention_days in to_tag.keys():
        delete_date = datetime.date.today() + datetime.timedelta(days=retention_days)
        delete_fmt = delete_date.strftime('%Y-%m-%d')
        print("Will delete %d snapshots on %s" % (len(to_tag[retention_days]), delete_fmt))
        ec.create_tags(
            Resources=to_tag[retention_days],
            Tags=[
                {'Key': 'DeleteOn', 'Value': delete_fmt},
                {'Key': 'Name', 'Value': "LIVE-BACKUP"}
            ]
        )

Also, you can decide on the retention time for the snapshot.

By default, the code sets the retention days as 10. If you want to reduce or increase the retention time, you can change the following parameter in the code.

retention_days = 10

The python script will create a snapshot with a tag key “Deletion” and “Date” as the value that is calculated based on the retention days. This will help in deleting the snapshots which are older than the retention time.

Lambda Function To Automate Snapshot Creation

Now that we have our python script ready for creating snapshots, it has to deployed as a Lambda function.

Triggering the Lambda function totally depends on your use case.

For demo purposes, we will set up cloudwatch triggers to execute the lambda function whenever a snapshot is required.

Follow the steps given below for creating a lambda function.

Step 1: Head over to lambda service page and select “create lambda function”.

Step 2: Choose “Author from Scratch” and python 3.11 runtime. Also, select an exiting IAM role with snapshot create permissions.

Click “Create Function” function button after filling up the details

Step 3: On the next page, if you scroll down, you will find the function code editor. Copy the python script from the above section to the editor and deploy it.

Once saved, click the “Test” button. It will open an evet pop up. Just enter an event name and click create it.

Click “Test” button again and you will see the code getting executed and its logs as shown below. As per the code, it should create snapshots of all volumes if a instance has a tag named “Backup:True”.

Step 4: Now you have a Lamda function ready to create snapshots.

You have to decide what triggers you need to invoke the lambda function. If you click the “Add Trigger” Button from the function dashboard, it will list all the possible trigger options as shown below. You can configure one based on your use case. It can be API gateway call or a cloudwatch event trigger like I explained above.

For this example, I will choose cloudwatch event trigger, It will look like the following.

EBS Snapshot Deletion Automation Using Lambda

We have seen how to create a lambda function and create EBS snapshots of ec2 instances tagged with a “backup” tag. We cannot keep the snapshots piling up over time. That’s the reason we used the retention days in the python code. It tags the snapshot with the deletion date.

The deletion python script scans for snapshots with a tag with a value that matches the current date. If a snapshot matches the requirement, it will delete that snapshot. This lambda function runs every day to remove the old snapshots.

Create a lambda function with the cloudwatch event schedule as one day. You can follow the same steps I explained above for creating the lambda function.

Here is the python code for snapshot deletion.

import boto3
import re
import datetime

ec = boto3.client('ec2')
iam = boto3.client('iam')

def lambda_handler(event, context):
    account_ids = list()
    try:
        """
        You can replace this try/except by filling in `account_ids` yourself.
        Get your account ID with:
        > import boto3
        > iam = boto3.client('iam')
        > print iam.get_user()['User']['Arn'].split(':')[4]
        """
        iam.get_user()
    except Exception as e:
        # use the exception message to get the account ID the function executes under
        account_ids.append(re.search(r'(arn:aws:sts::)([0-9]+)', str(e)).groups()[1])

    delete_on = datetime.date.today().strftime('%Y-%m-%d')
    filters = [
        {'Name': 'tag-key', 'Values': ['DeleteOn']},
        {'Name': 'tag-value', 'Values': [delete_on]},
    ]
    snapshot_response = ec.describe_snapshots(OwnerIds=account_ids, Filters=filters)

    for snap in snapshot_response['Snapshots']:
        print("Deleting snapshot %s" % snap['SnapshotId'])
        ec.delete_snapshot(SnapshotId=snap['SnapshotId'])

To Restore EBS Snapshot

You can restore a snapshot in two ways.

  1. Restore the EBS Volume from the snapshot.

  2. Restore EC2 Instance from a snapshot

You can optionally change following while restoring a snapshot

  1. Volume Size

  2. Disk Type

  3. Availability Zone

Restore EBS Volume from a EBS Snapshot

Step 1: Head over to snapshots, select the snapshot you want to restore, select the “Actions” dropdown, and click create volume from snapshot.

Step 2: Fill in the required details and click “create volume” option.

That’s it. Your volume will be created. You can mount this volume to the required instance to access its data.

Restore EC2 Instance From EBS Snapshot

You can restore an ec2 instance from a EBS snapshot with two simple steps.

  1. Create an AMI (ec2 machine Image) from the snapshot.

  2. Launch an instance from the AMI created from the snapshot.

Follow the below steps.

Step 1: Head over to snapshots, select the snapshot you want to restore, select the “Actions” dropdown, and click create image from snapshot.

Step 2: Enter the AMI name, description, and modify the required parameters. Click “Create Image” to register the AMI.

Step 3: Now, select AMIs from the left panel menu, select the AMI, and from the “Actions” drop-down, select launch.

It will take you to the generic instance launch wizard. You can launch the VM as you normally do with any ec2 instance creation.

In conclusion, this article has presented three distinct methods for automating EBS snapshot processes in AWS.

Whether you opt for AWS EBS Lifecycle Manager, CloudWatch Events, or Lambda-based solutions, the choice ultimately depends on your project requirements and aligning with your organization's security standards.

Whether you're seeking a straightforward snapshot solution or considering a transition to the Snapshot Lifecycle Manager, these methods offer versatility and adaptability.

Feel free to share your thoughts or specific use cases in the comments below. Your feedback and insights contribute to a collaborative learning environment for the AWS community.

##

AWSebs snapshotslambda#CloudWatchec2