In this article, we will create an AWS cloud Ec2 instances backup solution using Python script, CloudWatch, and AWS Lambda functions that will take instance backups on a regular basis.
AWS Lambda is an event-driven Serverless compute service provided by AWS that lets you run your code without provisioning and managing infrastructure.
We will create two lambda functions with Python to create a backup AMI and delete it after the retention period and use Cloudwatch to automate this task so that we get a new backup AMI every fifteen days with the previous one automatically deleted.
Step1: First, we need to create an IAM Role with the required privileges
- Go to IAM->Policies->Create Policy->JSON.
- Paste the following code there
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:*"
],
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": "ec2:*",
"Resource": "*"
}
]
}
- Click Next, add Tags.
- Click Next, and give the Name and description to the policy.
- Click on create policy.
- Go to Roles->Create Roles.
- Select AWS service as the trusted entity type and Lambda in the use case.
- Click Next. Enter the name of the policy in the search bar, select it, and hit next.
- Give name and description to Role and click on Create Role.
Step 2: Now we need to create AWS lambda functions
- Go to the Lambda service console.
- Click on Create Function.
- Select Author from scratch, give the function name, and select Python 3.9 as runtime.
- Select the architecture. Change the default execution role and select the role which we have created.
- Click on Create Function. First we will create lambda function for creating AMI.
- Copy the following python script and paste it into the Code section.
import boto3
import os
import datetime
import time
import collections
def lambda_handler(event, context):
retention_days=15
aws_account_id = context.invoked_function_arn.split(":")[4]
delete_date = datetime.date.today() + datetime.timedelta(days=retention_days)
delete_fmt = delete_date.strftime('%m-%d-%Y-%H')
create_time = datetime.datetime.now()
create_fmt = create_time.strftime('%m-%d-%Y-%H')
ec2_resource = boto3.client('ec2')
reservations = ec2_resource.describe_instances(
Filters=[
{'Name': 'tag:key', 'Values': ['Backup']}
]
).get(
'Reservations', []
)
instances = sum(
[
[i for i in r['Instances']]
for r in reservations
], [])
ami_list = []
for instance in instances:
name = "Lambda" + "-" + instance['InstanceId']+ "-" + create_fmt
amiid = ec2_resource.create_image(
Description="Lambda AMI",
DryRun=False,
InstanceId=instance['InstanceId'],
Name= name,
NoReboot=True
)
ec2_resource.create_tags(
Resources=[amiid['ImageId']],
Tags=[
{'Key': 'Delete', 'Value': delete_fmt},
{'Key': 'Backup', 'Value': 'True'},
{'Key': 'Name', 'Value': name}
]
)
ami_list.append(amiid['ImageId'])
time.sleep(10)
for ami in ami_list:
snapshots = ec2_resource.describe_snapshots(
DryRun=False,
OwnerIds=[
aws_account_id
],
Filters=[{
'Name': 'description',
'Values': [ '*'+ami+'*']
}] ).get('Snapshots', [] )
delete_date = datetime.date.today() + datetime.timedelta(days=retention_days)
delete_fmt = delete_date.strftime('%m-%d-%Y-%H')
snap_tag = ami + "-" + "Lambda"
for snapshot in snapshots:
ec2_resource.create_tags(
Resources=[snapshot['SnapshotId']],
Tags=[
{'Key': 'DeleteOn', 'Value': delete_fmt},
{'Key': 'Backup', 'Value': 'True'},
{'Key': 'Name', 'Value': snap_tag},
]
)
- Go to Configuration-> Edit General Configuration. Increase the timeout to 15 seconds.
- Click on Test to create AMI.
- Now create another lambda function to delete the AMI. Copy the following python script in Code and increase the timeout to 15 seconds in Configuration.
import boto3
import datetime
import os
import time
ec2_resource = boto3.client('ec2')
def lambda_handler(event, context):
aws_account_id = context.invoked_function_arn.split(":")[4]
image_resource = ec2_resource.describe_images(
DryRun=False,
Owners=[aws_account_id],
Filters=[
{'Name': 'tag:Backup', 'Values': ['True']}
]
).get(
'Images', []
)
amiList = []
current_date = datetime.datetime.now().strftime('%m-%d-%Y-%H:%M:%S')
time.sleep(5)
for image in image_resource:
deleteon = ''
for tag in image['Tags']:
if tag['Key'] == 'DeleteOn':
deleteon = tag['Value']
break
if deleteon == '':
continue
if deleteon > currentDate:
ec2_resource.deregister_image(
DryRun=False,
ImageId=image['ImageId']
)
amiList.append(image['ImageId'])
snapshots = ec2_resource.describe_snapshots(
DryRun=False,
OwnerIds=[
aws_account_id
],
Filters=[
{
'Name': 'description',
'Values': [ '*'+image['ImageId']+'*' ]
} ]
).get(
'Snapshots', []
)
for snapshot in snapshots:
time.sleep(5)
ec2_resource.delete_snapshot(
DryRun = False,
SnapshotId = snapshot['SnapshotId']
)
Step 3: Create a Trigger for our Lambda Functions
- Select our lambda function for AMI backup.
- Go to Configuration->Triggers->Add Triggers.
- Select EventBridge(CloudWatch Events).
- Enter your cron expression in Schedule Expression.
- This will run our function on the 1st and 15th of every month at 8 AM.
- Click on Add Trigger.
- Now repeat the above steps and create a trigger for our other lambda function for deleting AMI.
- Add the following cron expression to delete our old AMI on the 1st and 15th of every month at 11 AM.
Author info
This blog is written by Checkmate Management Consulting cloud engineering team. Please write to our technical to hire fully managed cloud engineer to transform entire cloud infrastructure and IT Staffing Services in India.
