D

data-engineering

Fundamental theory and practice in Machine Learning (ML) and Data Science (DS). 🧮

Solución end-to-end para la migración y análisis de datos utilizando Python, FastAPI, Kafka y PostgreSQL. Implementa un pipeline de datos asíncrono y una API RESTful para analíticas, todo completamente containerizado con Docker Compose para un despliegue fácil y reproducible.

End-to-end design of a Hadoop-based ecosystem for healthcare data at scale (50 TB, IoT streams, medical imaging). Proposed a 10-node cluster architecture integrating HDFS, Spark, Hive, NiFi, Kafka, and Docker with HIPAA-compliant security (Kerberos, TLS, Apache Ranger). Delivered a proof-of-concept Docker deployment and professional proposal document.

A collection of Airflow operators, hooks, and utilities to elevate dbt to a first-class citizen of Airflow.

See the documentation at: https://airflow-dbt-python.readthedocs.io/