Apache Spark: Design & Execute ETL Pipelines Hands-On
Build practical data engineering skills by learning how to design, develop, and execute end-to-end ETL (Extract, Transform, Load) pipelines using Apache Spark. In this hands-on course, you will begin by setting up a Spark development environment, installing and configuring PySpark, Hadoop, and MySQL, organizing ETL project structures, and exploring real-world datasets. As you progress, you will implement complete and incremental ETL workflows using Apache Spark. You'll integrate Spark with MySQL through JDBC, apply data transformation logic with Spark SQL, perform business-rule filtering, and
Skills you'll learn
We may earn a commission if you enroll through our links — it never affects the price you pay.
Similar courses
AWS: Database
AWS: Databases Course is the fourth course of Exam Prep DVA-C02: AWS Certified Developer Associate Specialization. This course assists learners in learning the core concepts of NoSQL, contrasting it with traditional SQL databases. It covers key aspects of DynamoDB, including compute capacity, read consistency, transactions, and streams. The course also explores other NoSQL options like Amazon MemoryDB for Redis, covering cluster setup, data loading, and its comparison with ElastiCache for Redis. Finally, it provides an overview of Amazon RDS, a managed relational database service. This cours
All levels
Shipping Rust
Take a Rust crate from a "works on my machine" laptop build to a production-grade artifact that ships through a gate no one can bypass. The course walks through a real 3-crate ETL workspace — etl-core, etl-cli, and etl-bench — built around clap derive macros, a tuned `[profile.release]`, and a multi-stage Dockerfile that drops a 1.8 GB rust:latest image to a 6 MB scratch+musl container with no shell to attack. You then wire pmat, bashrs, forjar, and pv onto the standard fmt + clippy + test + 100% coverage + audit + deny stack — because a green build badge is misleading when an agent wrote half
All levels
Data Engineering with Delta Lake on Databricks
Build production-ready data pipelines using Delta Live Tables and the Medallion Architecture on Databricks. This hands-on course teaches you to design, implement, and monitor ETL workflows that transform raw data into reliable, business-ready datasets through a structured bronze-silver-gold layering pattern. This course is primarily aimed at first- and second-year undergraduates interested in engineering or science, along with professionals with an interest in programming. You will start by mastering DLT fundamentals — declarative pipeline syntax in both SQL and Python, streaming ingestion w
All levels
Data Cleaning, Transformation, and Manipulation
In Data Cleaning, Transformation, and Manipulation, you’ll learn to turn messy data into analysis- and modeling-ready datasets using Python (pandas) and SQL. This is a skill-based path organized around real workplace tasks. Each module mirrors responsibilities you see in job descriptions and focuses on the exact steps you’ll perform on the job. You’ll begin with a quick skills check, then personalize your journey: double down on new topics, or skip what you already know. For each skill, you’ll review concise lessons curated from expert instructors with explanations and demos for filtering and
All levels