{"id":3009,"date":"2022-09-15T12:29:57","date_gmt":"2022-09-15T12:29:57","guid":{"rendered":"https:\/\/www.checkmateq.com\/blog\/?p=3009"},"modified":"2023-08-04T14:19:11","modified_gmt":"2023-08-04T14:19:11","slug":"apache-airflow","status":"publish","type":"post","link":"https:\/\/www.checkmateq.com\/blog\/apache-airflow","title":{"rendered":"How to set up Apache Airflow Environment on AWS"},"content":{"rendered":"<p><strong>Amazon <a href=\"https:\/\/www.checkmateq.com\/cloud\">cloud<\/a> Managed Workflows for Apache Airflow (MWAA)<\/strong> is a managed service by AWS which makes it simpler to set up and run end-to-end data pipelines in the cloud at scale. An open-source application called <strong>Apache Airflow<\/strong> is used to programmatically author, schedule, and keep track of &#8220;workflows,&#8221; which are collections of processes and tasks. <strong>Using Python and Airflow, you can build processes using Amazon MWAA without having to worry about scalability, availability, or security of the underlying infrastructure<\/strong>. To assist you to get quick access to your data while maintaining security, <strong>Amazon MWAA integrates with AWS security services<\/strong> and extends its workflow execution capacity automatically to meet your demands.<\/p>\n<p>In this blog, we will see how to set up an Amazon MWAA environment.<\/p>\n<h3>Step1: Create an S3 bucket<\/h3>\n<ul>\n<li>\u00a0We require an Amazon S3 bucket to store Apache Airflow Directed Acyclic Graphs (DAGs), custom plugins, and Python dependencies.<\/li>\n<li>Before creating a bucket make sure that it is in the same region, in which you will set up MWAA.<\/li>\n<li>Go to the S3 console and click Create bucket.<\/li>\n<li>Select a unique name for your bucket with<strong> &#8216;airflow-&#8216;<\/strong> prefix.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3026\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Hire-full-stack-development-engineer-1-300x121.png\" alt=\"\" width=\"751\" height=\"303\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Hire-full-stack-development-engineer-1-300x121.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Hire-full-stack-development-engineer-1-1024x412.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Hire-full-stack-development-engineer-1-768x309.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Hire-full-stack-development-engineer-1-1200x483.png 1200w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Hire-full-stack-development-engineer-1.png 1239w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<ul>\n<li>Block all public access and enable versioning.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3027\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-offshore-it-service-1-300x79.png\" alt=\"\" width=\"748\" height=\"197\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-offshore-it-service-1-300x79.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-offshore-it-service-1-1024x270.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-offshore-it-service-1-768x202.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-offshore-it-service-1.png 1177w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<ul>\n<li>Click <strong>Create bucket<\/strong>.<\/li>\n<li>Now go to your bucket and create a dags folder to store DAG files.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3028\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-software-development-Team-300x66.png\" alt=\"\" width=\"745\" height=\"164\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-software-development-Team-300x66.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-software-development-Team-1024x224.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-software-development-Team-768x168.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-software-development-Team-1200x262.png 1200w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-software-development-Team.png 1239w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<h3>Step2: Create Airflow environment<\/h3>\n<ul>\n<li>Go to Managed Apache Airflow dashboard.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3029\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-analytics-services-300x106.png\" alt=\"\" width=\"747\" height=\"264\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-analytics-services-300x106.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-analytics-services-1024x363.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-analytics-services-768x272.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-analytics-services-1536x544.png 1536w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-analytics-services-1200x425.png 1200w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-analytics-services.png 1668w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<ul>\n<li>Click Create environment.<\/li>\n<li>Select name and Airflow version.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3030\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-Analytic-service-300x119.png\" alt=\"\" width=\"741\" height=\"294\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-Analytic-service-300x119.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-Analytic-service-1024x406.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-Analytic-service-768x305.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-Analytic-service.png 1198w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<ul>\n<li>Next select S3 bucket and dag folder.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3031\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-application-development-services-300x130.png\" alt=\"\" width=\"741\" height=\"321\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-application-development-services-300x130.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-application-development-services-1024x442.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-application-development-services-768x332.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-application-development-services.png 1197w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<ul>\n<li>Click<strong> Next<\/strong>.<\/li>\n<li>Now click<strong> Create MWAA VPC<\/strong>. You will be redirected to Cloudformation. Click on <strong>Create Stack<\/strong>.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3034\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-migration-services-1-300x144.png\" alt=\"\" width=\"744\" height=\"357\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-migration-services-1-300x144.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-migration-services-1-1024x493.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-migration-services-1-768x370.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-migration-services-1-1200x578.png 1200w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-migration-services-1.png 1440w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<ul>\n<li>Select the VPC.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3037\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-blockchain-development-services-1-300x139.png\" alt=\"Apache Airflow\" width=\"749\" height=\"347\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-blockchain-development-services-1-300x139.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-blockchain-development-services-1-1024x474.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-blockchain-development-services-1-768x356.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-blockchain-development-services-1.png 1186w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<ul>\n<li>Under web server access select your preferred access mode.<\/li>\n<li>With a private network, you can limit the access to Apache Airflow to the users within your VPC, that have granted IAM permission.<\/li>\n<li>With public network, Apache Airflow can be accessed over internet.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3038\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-AWS-cloud-services-300x77.png\" alt=\"Apache Airflow\" width=\"740\" height=\"190\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-AWS-cloud-services-300x77.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-AWS-cloud-services-1024x262.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-AWS-cloud-services-768x196.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-AWS-cloud-services-1200x307.png 1200w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-AWS-cloud-services.png 1221w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<ul>\n<li>Under Security group select <strong>Create new security group<\/strong>.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3039\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-web-app-developer-300x88.png\" alt=\"Apache Airflow\" width=\"740\" height=\"217\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-web-app-developer-300x88.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-web-app-developer-1024x300.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-web-app-developer-768x225.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-web-app-developer.png 1191w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<ul>\n<li>Select Environment class, maximum and minimum worker count, and scheduler count.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3040\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-qa-tester-1-300x106.png\" alt=\"Apache Airflow\" width=\"750\" height=\"265\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-qa-tester-1-300x106.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-qa-tester-1-1024x362.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-qa-tester-1-768x272.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-qa-tester-1-1200x424.png 1200w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/hire-qa-tester-1.png 1222w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3041\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-software-development-company-300x123.png\" alt=\"Apache Airflow\" width=\"739\" height=\"303\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-software-development-company-300x123.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-software-development-company-1024x419.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-software-development-company-768x314.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-software-development-company.png 1159w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<ul>\n<li>Under<strong> permissions<\/strong>, select <strong>Create a new role<\/strong>.<\/li>\n<li><img loading=\"lazy\" class=\"alignnone wp-image-3043\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-engineering-services-300x108.png\" alt=\"\" width=\"764\" height=\"275\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-engineering-services-300x108.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-engineering-services-1024x368.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-engineering-services-768x276.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-data-engineering-services.png 1105w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/li>\n<li>Click<strong> Next<\/strong>, review and click <strong>Create<\/strong>.<\/li>\n<li>It will take 15 to 20 minutes for the Apache Airflow environment to be created.<\/li>\n<li>Click on the Airflow UI link to access the UI.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3045\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-mobile-app-development-company-1-300x91.png\" alt=\"Apache Airflow\" width=\"762\" height=\"231\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-mobile-app-development-company-1-300x91.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-mobile-app-development-company-1-1024x311.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-mobile-app-development-company-1-768x233.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-mobile-app-development-company-1-1536x466.png 1536w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-mobile-app-development-company-1-1200x364.png 1200w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-mobile-app-development-company-1.png 1912w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<ul>\n<li>Click Sign in With AWS Management Console.<\/li>\n<li>Now upload your DAG file in the dag folder of your S3 bucket.<\/li>\n<li>Wait for a few seconds and refresh airflow and you will see your DAG on UI.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3046\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-engineering-services-300x98.png\" alt=\"\" width=\"741\" height=\"242\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-engineering-services-300x98.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-engineering-services-1024x335.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-engineering-services-768x251.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-engineering-services-1536x503.png 1536w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-engineering-services-1200x393.png 1200w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-engineering-services.png 1906w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<ul>\n<li>Click run to trigger your DAG.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3047\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-solutions-company-300x154.png\" alt=\"Apache Airflow\" width=\"738\" height=\"379\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-solutions-company-300x154.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-solutions-company-1024x526.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-solutions-company-768x395.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-solutions-company-1200x616.png 1200w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-cloud-solutions-company.png 1425w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3048\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-google-cloud-services-300x113.png\" alt=\"Apache Airflow\" width=\"741\" height=\"279\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-google-cloud-services-300x113.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-google-cloud-services-1024x384.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-google-cloud-services-768x288.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-google-cloud-services-1536x577.png 1536w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-google-cloud-services-1200x450.png 1200w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Checkmate-google-cloud-services.png 1849w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<p><img loading=\"lazy\" class=\"alignnone wp-image-3049\" src=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Screenshot-2022-09-15-175332-300x100.png\" alt=\"Apache Airflow\" width=\"765\" height=\"255\" srcset=\"https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Screenshot-2022-09-15-175332-300x100.png 300w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Screenshot-2022-09-15-175332-1024x343.png 1024w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Screenshot-2022-09-15-175332-768x257.png 768w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Screenshot-2022-09-15-175332-1536x514.png 1536w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Screenshot-2022-09-15-175332-1200x402.png 1200w, https:\/\/www.checkmateq.com\/blog\/wp-content\/uploads\/2022\/09\/Screenshot-2022-09-15-175332.png 1828w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<p><a href=\"https:\/\/www.checkmateq.com\/\">Please contact<\/a> our DevOps engineering team if you have anything related to cloud infrastructure to be discussed.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Amazon cloud Managed Workflows for Apache Airflow (MWAA) is a managed service by AWS which makes it simpler to set up and run end-to-end data pipelines in the cloud at scale. An open-source application called Apache Airflow is used to programmatically author, schedule, and keep track of &#8220;workflows,&#8221; which are collections of processes and tasks. &hellip; <a href=\"https:\/\/www.checkmateq.com\/blog\/apache-airflow\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;How to set up Apache Airflow Environment on AWS&#8221;<\/span><\/a><\/p>\n","protected":false},"author":3,"featured_media":3023,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[3,2,70,7,11,6],"_links":{"self":[{"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/posts\/3009"}],"collection":[{"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/comments?post=3009"}],"version-history":[{"count":13,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/posts\/3009\/revisions"}],"predecessor-version":[{"id":4236,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/posts\/3009\/revisions\/4236"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/media\/3023"}],"wp:attachment":[{"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/media?parent=3009"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/categories?post=3009"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.checkmateq.com\/blog\/wp-json\/wp\/v2\/tags?post=3009"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}