D

Data Engineering

ELT pipeline orchestrated with Apache Airflow, extracting YouTube data via the YouTube Data API v3, loading it into PostgreSQL (staging & core schemas), with data quality checks using Soda. Containerized with Docker Compose, tested with Pytest, and automated via GitLab CI/CD.

This repository holds code for the AskAnna Backend. Our backend stack primarily uses Django and the Django REST Framework.

Cloud-native data engineering + ML POC: ingest Reddit images, run OCR, store results in BigQuery/Cloud Storage, and serve analytics via FastAPI + a React dashboard.

Quick and dirty pipeline to prepare a dataset for mobility studies, starting from a collection of CDRs.

Funnel provide an easy to use, easy to read framework to create very complex data selections over pandas DataFrames

Live data source that can be used for data engineering, data warehouse and etl development.