Curriculum
14 Sections
61 Lessons
26 Weeks
Expand all sections
Collapse all sections
Module 1: Introduction to Data Engineering
4
1.1
Lesson 1.1: What is Data Engineering? Roles & Responsibilities
1.2
Lesson 1.2: Data Engineer vs Data Scientist vs Data Analyst
1.3
Lesson 1.3: Modern Data Stack Overview (ETL, ELT, Streaming)
1.4
Lesson 1.4: Career Paths and Salaries in Data Engineering
Module 2: Linux & Shell Scripting for Data Engineers
4
2.1
Lesson 2.1: Introduction to Linux OS and Basic Commands
2.2
Lesson 2.2: Working with Files and Directories
2.3
Lesson 2.3: Bash Scripting Essentials
2.4
Lesson 2.4: Automating Data Tasks with Shell Scripts
Module 3: SQL for Data Analysis & Transformation
5
3.1
Lesson 3.1: Introduction to SQL Databases (PostgreSQL/MySQL)
3.2
Lesson 3.2: Data Retrieval – SELECT, WHERE, ORDER BY
3.3
Lesson 3.3: Joins (Inner, Left, Right, Full) Explained
3.4
Lesson 3.4: Subqueries, CTEs, Window Functions
3.5
Lesson 3.5: Writing Complex Analytical Queries
Module 4: Python for Data Engineering
5
4.1
Lesson 4.1: Python Basics: Variables, Data Types, Functions
4.2
Lesson 4.2: Working with CSV, JSON, and APIs
4.3
Lesson 4.3: Data Manipulation with Pandas
4.4
Lesson 4.4: Error Handling and Logging in Python
4.5
Lesson 4.5: Python Scripts for ETL Operations
Module 5: ETL Pipelines & Data Transformation
4
5.1
Lesson 5.1: Understanding ETL and ELT Concepts
5.2
Lesson 5.2: Building ETL Pipelines with Python and SQL
5.3
Lesson 5.3: Data Cleaning, Validation, and Transformation
5.4
Lesson 5.4: Scheduling ETL Jobs (Cron Jobs / Airflow Intro)
Module 6: Apache Airflow
5
6.1
Lesson 6.1: What is Airflow? Core Concepts (DAGs, Tasks)
6.2
Lesson 6.2: Setting Up and Configuring Airflow Locally
6.3
Lesson 6.3: Writing and Scheduling Workflows (DAGs)
6.4
Lesson 6.4: Sensors, Operators, Hooks
6.5
Lesson 6.5: Best Practices for Airflow Pipeline Management
Module 7: Data Warehousing & Modeling
5
7.1
Lesson 7.1: Introduction to Data Warehousing Concepts
7.2
Lesson 7.2: Star Schema vs Snowflake Schema
7.3
Lesson 7.3: Designing Fact and Dimension Tables
7.4
Lesson 7.4: Building a Data Warehouse with Redshift / BigQuery
7.5
Lesson 7.5: Slowly Changing Dimensions (SCD Types)
Module 8: Apache Spark & Big Data Processing
5
8.1
Lesson 8.1: Introduction to Big Data and Apache Spark
8.2
Lesson 8.2: Spark Architecture and Components
8.3
Lesson 8.3: Working with Spark DataFrames and RDDs
8.4
Lesson 8.4: Data Transformations and Actions
8.5
Lesson 8.5: PySpark for ETL Pipelines
Module 9: Apache Kafka & Real-Time Streaming
4
9.1
Lesson 9.1: What is Kafka? Use Cases in Real-Time Data
9.2
Lesson 9.2: Kafka Topics, Partitions, Brokers
9.3
Lesson 9.3: Kafka Producers and Consumers (Python Client)
9.4
Lesson 9.4: Streaming Data Pipelines using Kafka + Spark Streaming
Module 10: Cloud Data Engineering (AWS/GCP Focus)
5
10.1
Lesson 10.1: Introduction to Cloud Platforms for Data Engineers
10.2
Lesson 10.2: AWS Services (S3, Redshift, Glue, Lambda)
10.3
Lesson 10.3: GCP Services (BigQuery, Pub/Sub, Dataflow)
10.4
Lesson 10.4: Building a Cloud-Native ETL Pipeline
10.5
Lesson 10.5: Serverless Architectures for Data Engineering
Module 11: DataOps & CI/CD for Data Pipelines
5
11.1
Lesson 11.1: Introduction to DataOps Principles
11.2
Lesson 11.2: Version Control using Git for Data Projects
11.3
Lesson 11.3: Containerization with Docker
11.4
Lesson 11.4: Automating ETL Deployment with Jenkins
11.5
Lesson 11.5: Monitoring Data Pipelines
Module 12: Capstone Project
5
12.1
Lesson 12.1: Problem Statement and Dataset Introduction
12.2
Lesson 12.2: Data Ingestion (Batch and Real-Time)
12.3
Lesson 12.3: Data Processing and Cleaning
12.4
Lesson 12.4: Building Data Warehouse and Visualization
12.5
Lesson 12.5: Final Presentation and Submission
🎯 Bonus Lessons (Career Support)
• Resume Writing for Data Engineers • Preparing for Data Engineering Interviews • Mock Interviews with Mentors • LinkedIn Profile Building • Salary Negotiation Tips
5
13.1
• Resume Writing for Data Engineers
13.2
• Preparing for Data Engineering Interviews
13.3
• Mock Interviews with Mentors
13.4
• LinkedIn Profile Building
13.5
• Salary Negotiation Tips
🛠️ Tools and Technologies Covered:
✅ Python ✅ SQL (PostgreSQL/MySQL) ✅ Apache Airflow ✅ Apache Spark ✅ Apache Kafka ✅ AWS (S3, Redshift, Glue) ✅ GCP (BigQuery, Dataflow, Pub/Sub) ✅ Docker & Git ✅ Linux Shell Scripting
0
IIQMC Data Engineering Career Accelerator
Search
This content is protected, please
login
and enroll in the course to view this content!
Login with your site account
Lost your password?
Remember Me
Not a member yet?
Register now
Register a new account
Are you a member?
Login now
Modal title
Main Content