multi-cloud-data-pipeline

Project information

A production-ready data pipeline that ingests data from public API endpoints, validates and processes it through a medallion architecture (Raw → Bronze → Gold) using Apache Airflow orchestration, AWS Glue transformations, and S3 Data Lake storage. Features robust error handling, circuit breaker patterns, and schema evolution management.

Amazon Web S... apache-airflow datalake

12 Commits
1 Branch
0 Tags

README

Created on

July 24, 2025

Loading