AWS Data Engineering Labs

This hands-on course is designed for individuals familiar with AWS to enhance their skills in data engineering. Students should have a basic understanding of Python, SQL, and database concepts. However, even beginners to data engineering can follow along and learn. The course is minimal on theory, focusing instead on practical aspects of data engineering on AWS. Participants will gain practical experience through a series of labs covering essential AWS services such as Glue, Lambda, Kinesis, S3, Redshift, EventBridge, and more. While the labs provide practical exercises, participants are encouraged to refer to AWS documentation for a full understanding of concepts. This course will also give you practical experience to aid in your preparation for the Data Engineering certification (DEA-C01)

AWS Data Engineering Labs :

• Creating a data catalog in Glue and viewing data in Athena
• Running an ETL job using Glue
• Triggering SNS Notification for S3 Upload Event using EventBridge
• Orchestrating Lambda functions with Step Functions State Machine
• ETL Workflow Orchestration with AWS Glue Lambda EventBridge Step Functions
• Storing and Retrieving Data from a Kinesis Data Stream Using AWS CLI
• Kinesis Data Stream Python Boto3 Producer & Consumer
• Writing simulated weather data from a Kinesis Stream to S3 with AWS Lambda
• Running Spark transformation jobs using Amazon EMR on EC2
• Creating a Data Warehouse on S3 data using Amazon Redshift

Prerequisites:

• Basic understanding of AWS
• Basic knowledge of Python, SQL, Spark and database concepts

Note: Even if you are a beginner to data engineering, you can still follow and learn from this course.