Project information
- Category: Course Job
- Client: Alura
- Project date: 12 August, 2022
- Client Details: Alura's web site
Twitter ETL Description
Twitter ETL was a course project where I had the opportunity to build each pipeline's step and practice the top Data Engineering technologies on the process.
In the course, we explored a datalake structure and simulated the three data layers (bronze, silver and gold) as folders on my local machine. It was important to see data moving between these layers to improve my knowledge of data quality.
Technologies
- Airflow
- Spark
- Python