Data Engineering Archives

Data Warehouse vs Data Lake vs Data Lakehouse

"Data Warehouse vs Data Lake vs Data Lakehouse" is a topic that often confuses many people. In this blog, we will cover everything you need to know about the architecture…

Ansam Yousry August 9, 2024

Data Engineering

What are Transactional Databases?

How Does a Transactional Database Work?Transactional Database ExamplesBenefits of Transactional DatabasesDisadvantages and Limits of Transactional DatabasesDifferences Between Transactional Databases and Analytical DatabasesTransactional DatabasesAnalytical DatabasesWhat are non-transactional databases?Conclusion Transactional databases are…

Ansam Yousry July 29, 2024

Data Engineering

Understanding DAG Scheduling in Apache Airflow

Hey there! Ready to dive into the world of Apache Airflow and make your workflows run like a charm? Let’s explore how to schedule your Directed Acyclic Graphs (DAGs) effectively.…

Ansam Yousry July 4, 2024

Data Engineering

What are the differences between ETL, ELT, and ELTP?

The differences between ETL, ELT, and ELTP methodologies highlight the evolution of data integration strategies over the past few decades. These advancements have given rise to three primary methodologies: ETL…

Ansam Yousry June 10, 2024

Data Engineering

Apache Airflow Tutorial: Architecture, Concepts, and How to Run Airflow Locally With Docker

Apache Airflow is an open-source platform that provides a way to programmatically author, schedule, and monitor workflows. It is widely used in the industry to manage complex data pipelines and automate…

Ansam Yousry May 20, 2024

How Uber use Apache Kafka and Presto for Data Analysis

Data Engineering

How Uber Harnesses the Power of Presto and Apache Kafka for Data Analysis

At Uber, data is at the heart of everything we do. From optimizing routes for drivers to predicting rider demand, our ability to process and analyze vast amounts of data…

Ansam Yousry May 15, 2024

Data Engineering

How Uber Uses Kafka in Its Dynamic Pricing Model

In the fast world of ride-sharing, Uber stands out because of its large network, easy-to-use app, and smart dynamic pricing, known as "surge pricing." This system changes ride prices based…

Ansam Yousry May 15, 2024

Data Engineering

Is It Still Smart to Learn Data Engineering in 2024?

Current State of Data EngineeringMarket Demand for Data EngineersSkills Required in 2024Challenges Facing Data EngineersConclusion As we look at the tech world in 2024, data engineering is a field full of…

Ansam Yousry April 26, 2024

what the differences are between Data Warehouse , Data Mart, and Data Lake

Data Engineering

Data Warehouse vs. Data Mart vs. Data Lake: Understanding Architecture and Use Cases

What is a Data Warehouse?Data Warehouse ArchitectureData Warehouse Use CasesWhat is a Data Mart?Data Mart vs Data WarehouseData Mart ArchitectureData Mart Use CasesWhat is a Data Lake?Data Lake vs Data…

Ansam Yousry April 25, 2024

Data Engineering

What is an Analytics Engineer and What Do They Do?

Analytics Engineering is a highly in-demand role that focuses on using data to make informed decisions. In this article, we will explore the evolution of analytics engineering, its importance in…

Ansam Yousry March 6, 2024