We will create two lambda functions with Python to create a backup AMI and delete it after the retention period and use Cloudwatch to automate this task so that we get a new backup AMI every fifteen days with the previous one automatically deleted.<\/p>\n
Step1:<\/strong> First, we need to create an IAM Role with the required privileges<\/strong><\/h3>\n\n- Go to IAM->Policies->Create Policy->JSON.<\/li>\n
- Paste the following code there<\/li>\n<\/ul>\n
{\r\n\"Version\": \"2012-10-17\",\r\n\"Statement\": [\r\n{\r\n\"Effect\": \"Allow\",\r\n\"Action\": [\r\n\"logs:*\"\r\n],\r\n\"Resource\": \"arn:aws:logs:*:*:*\"\r\n},\r\n{\r\n\"Effect\": \"Allow\",\r\n\"Action\": \"ec2:*\",\r\n\"Resource\": \"*\"\r\n}\r\n]\r\n}\r\n<\/pre>\n\n- Click Next, add Tags.<\/li>\n<\/ul>\n
![\"\"](\"http:\/\/checkmateq.com\/blog\/wp-content\/uploads\/2022\/05\/Screenshot-2022-05-21-171346-300x53.png\")
<\/p>\n
\n- Click Next, and give the Name and description to the policy.<\/li>\n
- Click on create policy.<\/li>\n
- Go to Roles->Create Roles.<\/li>\n
- Select AWS service as the trusted entity type and Lambda in the use case.<\/li>\n<\/ul>\n
![\"Lambda](\"http:\/\/checkmateq.com\/blog\/wp-content\/uploads\/2022\/05\/Screenshot-2022-05-21-171905-300x118.png\")
<\/p>\n
\n- Click Next. Enter the name of the policy in the search bar, select it, and hit next.<\/li>\n
- Give name and description to Role and click on Create Role.<\/li>\n<\/ul>\n
Step 2: Now we need to create AWS lambda functions<\/strong><\/h3>\n\n- Go to the Lambda service console.<\/li>\n
- Click on Create Function.<\/li>\n
- Select Author from scratch, give the function name, and select Python 3.9 as runtime.<\/li>\n<\/ul>\n
![\"\"](\"http:\/\/checkmateq.com\/blog\/wp-content\/uploads\/2022\/05\/Screenshot-2022-05-21-173744-300x121.png\")
<\/p>\n
\n- Select the architecture. Change the default execution role and select the role which we have created.<\/li>\n<\/ul>\n
![\"\"](\"http:\/\/checkmateq.com\/blog\/wp-content\/uploads\/2022\/05\/Screenshot-2022-05-21-174014-300x130.png\")
<\/p>\n
\n- Click on Create Function. First we will create lambda function for creating AMI.<\/li>\n
- Copy the following python script and paste it into the Code section.<\/li>\n<\/ul>\n
import boto3\r\nimport os\r\nimport datetime\r\nimport time\r\nimport collections\r\n\r\ndef lambda_handler(event, context):\r\n retention_days=15\r\n aws_account_id = context.invoked_function_arn.split(\":\")[4]\r\n delete_date = datetime.date.today() + datetime.timedelta(days=retention_days)\r\n delete_fmt = delete_date.strftime('%m-%d-%Y-%H')\r\n create_time = datetime.datetime.now()\r\n create_fmt = create_time.strftime('%m-%d-%Y-%H')\r\n\r\n ec2_resource = boto3.client('ec2')\r\n\r\n reservations = ec2_resource.describe_instances(\r\n Filters=[\r\n {'Name': 'tag:key', 'Values': ['Backup']}\r\n ]\r\n ).get(\r\n 'Reservations', []\r\n )\r\n\r\n instances = sum(\r\n [\r\n [i for i in r['Instances']]\r\n for r in reservations\r\n ], [])\r\n\r\n\r\n ami_list = []\r\n for instance in instances:\r\n name = \"Lambda\" + \"-\" + instance['InstanceId']+ \"-\" + create_fmt\r\n amiid = ec2_resource.create_image(\r\n Description=\"Lambda AMI\",\r\n DryRun=False,\r\n InstanceId=instance['InstanceId'],\r\n Name= name,\r\n NoReboot=True\r\n )\r\n\r\n ec2_resource.create_tags(\r\n Resources=[amiid['ImageId']],\r\n Tags=[\r\n {'Key': 'Delete', 'Value': delete_fmt},\r\n {'Key': 'Backup', 'Value': 'True'},\r\n {'Key': 'Name', 'Value': name}\r\n ]\r\n )\r\n\r\n ami_list.append(amiid['ImageId'])\r\n time.sleep(10)\r\n\r\n for ami in ami_list:\r\n snapshots = ec2_resource.describe_snapshots(\r\n DryRun=False,\r\n OwnerIds=[\r\n aws_account_id\r\n ],\r\n Filters=[{\r\n 'Name': 'description',\r\n 'Values': [ '*'+ami+'*']\r\n }] ).get('Snapshots', [] )\r\n\r\n delete_date = datetime.date.today() + datetime.timedelta(days=retention_days)\r\n delete_fmt = delete_date.strftime('%m-%d-%Y-%H')\r\n snap_tag = ami + \"-\" + \"Lambda\"\r\n \r\n for snapshot in snapshots:\r\n ec2_resource.create_tags(\r\n Resources=[snapshot['SnapshotId']],\r\n Tags=[\r\n {'Key': 'DeleteOn', 'Value': delete_fmt},\r\n {'Key': 'Backup', 'Value': 'True'},\r\n {'Key': 'Name', 'Value': snap_tag},\r\n ]\r\n )\r\n<\/pre>\n\n- Go to Configuration-> Edit General Configuration. Increase the timeout to 15 seconds.<\/li>\n<\/ul>\n
![\"\"](\"http:\/\/checkmateq.com\/blog\/wp-content\/uploads\/2022\/05\/Screenshot-2022-05-21-174702-300x136.png\")
<\/p>\n
\n- Click on Test to create AMI.<\/li>\n<\/ul>\n
![\"Lambda](\"http:\/\/checkmateq.com\/blog\/wp-content\/uploads\/2022\/05\/Screenshot-2022-05-21-175037-300x70.png\")
<\/p>\n
\n- Now create another lambda function to delete the AMI. Copy the following python script in Code and increase the timeout to 15 seconds in Configuration.<\/li>\n<\/ul>\n
import boto3\r\nimport datetime\r\nimport os\r\nimport time\r\n\r\nec2_resource = boto3.client('ec2')\r\ndef lambda_handler(event, context):\r\n aws_account_id = context.invoked_function_arn.split(\":\")[4]\r\n image_resource = ec2_resource.describe_images(\r\n DryRun=False,\r\n Owners=[aws_account_id],\r\n Filters=[\r\n {'Name': 'tag:Backup', 'Values': ['True']}\r\n ]\r\n ).get(\r\n 'Images', []\r\n )\r\n amiList = []\r\n current_date = datetime.datetime.now().strftime('%m-%d-%Y-%H:%M:%S')\r\n time.sleep(5)\r\n for image in image_resource:\r\n deleteon = ''\r\n for tag in image['Tags']:\r\n if tag['Key'] == 'DeleteOn':\r\n deleteon = tag['Value']\r\n break\r\n if deleteon == '':\r\n continue\r\n if deleteon > currentDate:\r\n ec2_resource.deregister_image(\r\n DryRun=False,\r\n ImageId=image['ImageId']\r\n )\r\n amiList.append(image['ImageId'])\r\n snapshots = ec2_resource.describe_snapshots(\r\n DryRun=False,\r\n OwnerIds=[\r\n aws_account_id\r\n ],\r\n Filters=[\r\n {\r\n 'Name': 'description',\r\n 'Values': [ '*'+image['ImageId']+'*' ]\r\n } ]\r\n ).get(\r\n 'Snapshots', []\r\n )\r\n for snapshot in snapshots:\r\n time.sleep(5)\r\n ec2_resource.delete_snapshot(\r\n DryRun = False,\r\n SnapshotId = snapshot['SnapshotId']\r\n )\r\n<\/pre>\nStep 3: Create a Trigger for our Lambda Functions<\/strong><\/h3>\n\n- Select our lambda function for AMI backup.<\/li>\n
- Go to Configuration->Triggers->Add Triggers.<\/li>\n
- Select EventBridge(CloudWatch Events).<\/li>\n
- Enter your cron expression in Schedule Expression.<\/li>\n<\/ul>\n
![\"Lambda](\"http:\/\/checkmateq.com\/blog\/wp-content\/uploads\/2022\/05\/Screenshot-2022-05-21-185819-300x88.png\")
<\/p>\n
<\/p>\n
\n- This will run our function on the 1st and 15th of every month at 8 AM.<\/li>\n
- Click on Add Trigger.<\/li>\n
- Now repeat the above steps and create a trigger for our other lambda function for deleting AMI.<\/li>\n
- Add the following cron expression to delete our old AMI on the 1st and 15th of every month at 11 AM.<\/li>\n<\/ul>\n
![\"\"](\"http:\/\/checkmateq.com\/blog\/wp-content\/uploads\/2022\/05\/Screenshot-2022-05-21-184459-300x41.png\")
<\/p>\n
Author info<\/p>\n
This blog is written by Checkmate Management Consulting cloud engineering team. Please write<\/a> to our technical to hire fully managed cloud engineer<\/a> to transform entire cloud infrastructure and IT Staffing Services in India<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"In this article, we will create an AWS cloud Ec2 instances backup solution using Python script, CloudWatch, and AWS Lambda functions that will take instance backups on a regular basis. AWS Lambda is an event-driven Serverless compute service provided by AWS that lets you run your code without provisioning and managing infrastructure. We will create … Continue reading “Automating AMI Backup Using Lambda and CloudWatch”<\/span><\/a><\/p>\n","protected":false},"author":3,"featured_media":518,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[4,3,7,11,8,6],"_links":{"self":[{"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/posts\/389"}],"collection":[{"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/comments?post=389"}],"version-history":[{"count":20,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/posts\/389\/revisions"}],"predecessor-version":[{"id":4630,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/posts\/389\/revisions\/4630"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/media\/518"}],"wp:attachment":[{"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/media?parent=389"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/categories?post=389"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/tags?post=389"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
{\r\n\"Version\": \"2012-10-17\",\r\n\"Statement\": [\r\n{\r\n\"Effect\": \"Allow\",\r\n\"Action\": [\r\n\"logs:*\"\r\n],\r\n\"Resource\": \"arn:aws:logs:*:*:*\"\r\n},\r\n{\r\n\"Effect\": \"Allow\",\r\n\"Action\": \"ec2:*\",\r\n\"Resource\": \"*\"\r\n}\r\n]\r\n}\r\n<\/pre>\n- \n
- Click Next, add Tags.<\/li>\n<\/ul>\n<\/p>\n
- \n
- Click Next, and give the Name and description to the policy.<\/li>\n
- Click on create policy.<\/li>\n
- Go to Roles->Create Roles.<\/li>\n
- Select AWS service as the trusted entity type and Lambda in the use case.<\/li>\n<\/ul>\n<\/p>\n
- \n
- Click Next. Enter the name of the policy in the search bar, select it, and hit next.<\/li>\n
- Give name and description to Role and click on Create Role.<\/li>\n<\/ul>\n
Step 2: Now we need to create AWS lambda functions<\/strong><\/h3>\n
- \n
- Go to the Lambda service console.<\/li>\n
- Click on Create Function.<\/li>\n
- Select Author from scratch, give the function name, and select Python 3.9 as runtime.<\/li>\n<\/ul>\n<\/p>\n
- \n
- Select the architecture. Change the default execution role and select the role which we have created.<\/li>\n<\/ul>\n<\/p>\n
- \n
- Click on Create Function. First we will create lambda function for creating AMI.<\/li>\n
- Copy the following python script and paste it into the Code section.<\/li>\n<\/ul>\n
import boto3\r\nimport os\r\nimport datetime\r\nimport time\r\nimport collections\r\n\r\ndef lambda_handler(event, context):\r\n retention_days=15\r\n aws_account_id = context.invoked_function_arn.split(\":\")[4]\r\n delete_date = datetime.date.today() + datetime.timedelta(days=retention_days)\r\n delete_fmt = delete_date.strftime('%m-%d-%Y-%H')\r\n create_time = datetime.datetime.now()\r\n create_fmt = create_time.strftime('%m-%d-%Y-%H')\r\n\r\n ec2_resource = boto3.client('ec2')\r\n\r\n reservations = ec2_resource.describe_instances(\r\n Filters=[\r\n {'Name': 'tag:key', 'Values': ['Backup']}\r\n ]\r\n ).get(\r\n 'Reservations', []\r\n )\r\n\r\n instances = sum(\r\n [\r\n [i for i in r['Instances']]\r\n for r in reservations\r\n ], [])\r\n\r\n\r\n ami_list = []\r\n for instance in instances:\r\n name = \"Lambda\" + \"-\" + instance['InstanceId']+ \"-\" + create_fmt\r\n amiid = ec2_resource.create_image(\r\n Description=\"Lambda AMI\",\r\n DryRun=False,\r\n InstanceId=instance['InstanceId'],\r\n Name= name,\r\n NoReboot=True\r\n )\r\n\r\n ec2_resource.create_tags(\r\n Resources=[amiid['ImageId']],\r\n Tags=[\r\n {'Key': 'Delete', 'Value': delete_fmt},\r\n {'Key': 'Backup', 'Value': 'True'},\r\n {'Key': 'Name', 'Value': name}\r\n ]\r\n )\r\n\r\n ami_list.append(amiid['ImageId'])\r\n time.sleep(10)\r\n\r\n for ami in ami_list:\r\n snapshots = ec2_resource.describe_snapshots(\r\n DryRun=False,\r\n OwnerIds=[\r\n aws_account_id\r\n ],\r\n Filters=[{\r\n 'Name': 'description',\r\n 'Values': [ '*'+ami+'*']\r\n }] ).get('Snapshots', [] )\r\n\r\n delete_date = datetime.date.today() + datetime.timedelta(days=retention_days)\r\n delete_fmt = delete_date.strftime('%m-%d-%Y-%H')\r\n snap_tag = ami + \"-\" + \"Lambda\"\r\n \r\n for snapshot in snapshots:\r\n ec2_resource.create_tags(\r\n Resources=[snapshot['SnapshotId']],\r\n Tags=[\r\n {'Key': 'DeleteOn', 'Value': delete_fmt},\r\n {'Key': 'Backup', 'Value': 'True'},\r\n {'Key': 'Name', 'Value': snap_tag},\r\n ]\r\n )\r\n<\/pre>\n- \n
- Go to Configuration-> Edit General Configuration. Increase the timeout to 15 seconds.<\/li>\n<\/ul>\n<\/p>\n
- \n
- Click on Test to create AMI.<\/li>\n<\/ul>\n<\/p>\n
- \n
- Now create another lambda function to delete the AMI. Copy the following python script in Code and increase the timeout to 15 seconds in Configuration.<\/li>\n<\/ul>\n
import boto3\r\nimport datetime\r\nimport os\r\nimport time\r\n\r\nec2_resource = boto3.client('ec2')\r\ndef lambda_handler(event, context):\r\n aws_account_id = context.invoked_function_arn.split(\":\")[4]\r\n image_resource = ec2_resource.describe_images(\r\n DryRun=False,\r\n Owners=[aws_account_id],\r\n Filters=[\r\n {'Name': 'tag:Backup', 'Values': ['True']}\r\n ]\r\n ).get(\r\n 'Images', []\r\n )\r\n amiList = []\r\n current_date = datetime.datetime.now().strftime('%m-%d-%Y-%H:%M:%S')\r\n time.sleep(5)\r\n for image in image_resource:\r\n deleteon = ''\r\n for tag in image['Tags']:\r\n if tag['Key'] == 'DeleteOn':\r\n deleteon = tag['Value']\r\n break\r\n if deleteon == '':\r\n continue\r\n if deleteon > currentDate:\r\n ec2_resource.deregister_image(\r\n DryRun=False,\r\n ImageId=image['ImageId']\r\n )\r\n amiList.append(image['ImageId'])\r\n snapshots = ec2_resource.describe_snapshots(\r\n DryRun=False,\r\n OwnerIds=[\r\n aws_account_id\r\n ],\r\n Filters=[\r\n {\r\n 'Name': 'description',\r\n 'Values': [ '*'+image['ImageId']+'*' ]\r\n } ]\r\n ).get(\r\n 'Snapshots', []\r\n )\r\n for snapshot in snapshots:\r\n time.sleep(5)\r\n ec2_resource.delete_snapshot(\r\n DryRun = False,\r\n SnapshotId = snapshot['SnapshotId']\r\n )\r\n<\/pre>\nStep 3: Create a Trigger for our Lambda Functions<\/strong><\/h3>\n
- \n
- Select our lambda function for AMI backup.<\/li>\n
- Go to Configuration->Triggers->Add Triggers.<\/li>\n
- Select EventBridge(CloudWatch Events).<\/li>\n
- Enter your cron expression in Schedule Expression.<\/li>\n<\/ul>\n<\/p>\n
<\/p>\n
- \n
- This will run our function on the 1st and 15th of every month at 8 AM.<\/li>\n
- Click on Add Trigger.<\/li>\n
- Now repeat the above steps and create a trigger for our other lambda function for deleting AMI.<\/li>\n
- Add the following cron expression to delete our old AMI on the 1st and 15th of every month at 11 AM.<\/li>\n<\/ul>\n<\/p>\n
Author info<\/p>\n
This blog is written by Checkmate Management Consulting cloud engineering team. Please write<\/a> to our technical to hire fully managed cloud engineer<\/a> to transform entire cloud infrastructure and IT Staffing Services in India<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"
In this article, we will create an AWS cloud Ec2 instances backup solution using Python script, CloudWatch, and AWS Lambda functions that will take instance backups on a regular basis. AWS Lambda is an event-driven Serverless compute service provided by AWS that lets you run your code without provisioning and managing infrastructure. We will create … Continue reading “Automating AMI Backup Using Lambda and CloudWatch”<\/span><\/a><\/p>\n","protected":false},"author":3,"featured_media":518,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[4,3,7,11,8,6],"_links":{"self":[{"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/posts\/389"}],"collection":[{"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/comments?post=389"}],"version-history":[{"count":20,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/posts\/389\/revisions"}],"predecessor-version":[{"id":4630,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/posts\/389\/revisions\/4630"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/media\/518"}],"wp:attachment":[{"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/media?parent=389"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/categories?post=389"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/tags?post=389"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- Now create another lambda function to delete the AMI. Copy the following python script in Code and increase the timeout to 15 seconds in Configuration.<\/li>\n<\/ul>\n
- Click on Test to create AMI.<\/li>\n<\/ul>\n
- Go to Configuration-> Edit General Configuration. Increase the timeout to 15 seconds.<\/li>\n<\/ul>\n
- Select the architecture. Change the default execution role and select the role which we have created.<\/li>\n<\/ul>\n