Data Engineering

Presenters Roman Khavronenko Source PromCon EU 2025 🕵️‍♀️ Unmasking the Truth: Debugging Queries with Row Data Samples 🚀 Hey everyone! 👋 As a tech blogger, I love diving deep into the nitty-gritty of how systems work. Today, we’re tackling a surprisingly complex issue: the discrepancy between what you see in monitoring tools like Prometheus and Victoria Metrics and what’s actually happening in your database. Let’s explore how viewing row samples can be your secret weapon for troubleshooting. ...

Presenters Israel Blancas Source PromCon EU 2025 🚨 Unmasking Telemetry Trouble: Preventing Data Loss & Bottlenecks in OpenTelemetry 🚀 Let’s be honest – OpenTelemetry is amazing. The promise of automatic instrumentation, a unified observability solution – it’s almost too good to be true. But as Coral Logix’s Israel brilliantly demonstrated, chasing that “magical” automatic approach without a solid strategy can lead to a whole heap of problems. This presentation wasn’t about how to use OpenTelemetry, it was about how not to – specifically, how to avoid the common pitfalls that can turn your observability investment into a frustrating data graveyard. 💀 ...

Presenters Bartek Protka Source PromCon EU 2025 🤖 The Trolley Problem of Metrics: Navigating Collection Resiliency 🚀 Hey everyone! 👋 Let’s talk about a surprisingly relevant thought experiment that’s impacting how we manage our data pipelines – the Trolley Problem. It might sound a bit heavy, but it perfectly illustrates a critical challenge in collection resiliency and how we’re tackling it. 🤯 The Trolley Problem in Data Collection The classic Trolley Problem asks: if you can divert a runaway trolley to save five people, but doing so will kill one, what do you do? In the world of data collection, we’re facing a similar dilemma. Imagine your data collection system – think Prometheus, OpenTelemetry collectors, or any other metric pipeline – is nearing its memory limit. You’re staring down a massive influx of data, potentially high-cardinality data (meaning lots of unique values), and you know that processing it could lead to a crash. ...

Presenters Björn “Beorn” Rabenstein Source PromCon EU 2025 Rate Smoothing & Anchoring: A Deep Dive into Prometheus Rate Function Evolution 🚀 Hey everyone! 👋 Let’s talk about something that’s been buzzing in the monitoring world: the Prometheus rate function. It’s a fundamental piece of the puzzle for calculating rates from time series data, but it’s also a surprisingly complex and often misunderstood component. Today, we’re going to break down the recent changes and explore the new approaches of “anchored” and “smooth” rates – and why they matter for your observability stack. ...

Presenters Vivek Yadav Source InfoQ podcast Stripe’s Secret Weapon: How Apache Spark Fuels Bulletproof Payment Systems 🚀 Ever wondered how companies like Stripe, which handle a mind-boggling amount of financial transactions every second, ensure their systems are absolutely rock-solid? It’s not just about clever coding; it’s about innovative testing strategies that go above and beyond. In a recent illuminating presentation, Vivek Yadav, an Engineering Manager at Stripe, pulled back the curtain on a rather unconventional yet incredibly effective use of Apache Spark – not for its usual analytical wizardry, but for something far more critical: large-scale regression testing and “what-if” scenario analysis for their core payment systems. Prepare to be amazed as we dive into how Stripe is leveraging this powerful tool to achieve unparalleled system integrity. ...