Data on carlos zansavio

My data platform diary - Week 1

Sat, 18 Jul 2026 00:00:00 +0000

Some of you already know I run a bunch of self-hosted services out of a homelab I’ve been building for over two years now. It started as a place to store personal files, but it’s grown into something a few people in my local community actually rely on.

I work in data engineering for a living, but I’ve never had my own data platform. At some point that stopped making sense to me — I have the infrastructure, I have the knowledge, so why not build one?

Apache Spark exercises to test and improve your knowledge

Sat, 11 Jul 2026 00:00:00 +0000

You’ve read the docs. You can write queries using SparkSQL or PySpark. You’re pretty sure you understand how Spark works under the hood. Then an interviewer asks you to actually reason through a shuffle, and that confidence gets shaky fast.

Been there.

What actually got these concepts to stick for me was working through math problems tied to Spark’s internals. You have to understand the mechanics or the numbers don’t add up. So I put together a set of exercises built around exactly that.

Building a CDC Streaming Pipeline locally

Sun, 05 Jul 2026 00:00:00 +0000

Introduction Link to heading

Want to understand what a Change Data Capture (CDC) streaming pipeline is and how to build one locally with Docker? Let me help you. This is a POC project to understand how things work in real life.

You can find the repository here.

What’s CDC Streaming pipeline? Link to heading

CDC stands for Change Data Capture. In simple terms, it streams database update logs to a topic (if using Kafka), so other systems can stay in sync with the source database in near real-time. This approach is especially useful for feeding data into Online Analytical Processing (OLAP) platforms, where analysts need fresh data to work with.