Finish Slime #13

Data Engineering, Analytics. No ML, no AI. The weekly dose of the data content you actually want to read!

Want to share anything with me? Hit me up on Twitter @sbalnojan or Linkedin.

News

61 data observability use cases

Data observability is still an obscure trend. This article highlights 61 different use cases of observability, increasing revenue, saving time, and many others.

Enterprise Data Platform @ Compass

The data team at Compass built their new data platform on top of Databricks. They faced several challenges, including multiple overlapping analytics systems, inefficient ad hoc reporting processes, and the need for a unified machine learning infrastructure—and, of course, scale.

How Instacart Ads Modularized Data Pipelines With Lakehouse Architecture and Spark

Instacarts ad system is a complex ecosystem that generates data, powering performance metrics and insights for ad campaigns. To overcome challenges with their previous ETL stack, Instacart implemented modular pipelines in Spark, enhancing testing, monitoring, and integrating streaming for incremental processing capabilities.

Building up a semantic layer with dbt Metrics, Cube and Droughty

Let’s build a semantic layer with dbt Metrics, cube, and Droughty (paid, optional).

Fresh Data Is The New Real-Time Data

This article challenges the idea of real-time data and proposes that fresh data instead of real-time data is valuable. Indeed it is even precious to the data engineer because it leads to faster development cycles and more robust data flows.

Tools

What’s the hype DuckDB?

Matt offers his perspective on DuckDB: DuckDB has the potential to be a cost-effective alternative to managed OLAP systems. It could serve as a staging layer for data transformation or provide a platform for tutorials and learning in the data engineering field.

Polars cookbook

Polars is the new pandas. The polars cookbook is a great place to start experiencing this new technology.

Mosaic: An Extensible Framework for Linking Databases and Interactive Views

Mosaic is an extensible architecture for linking data visualizations, tables, input widgets, and other data-driven components, leveraging a backing database for scalable static and interactive views processing. With Mosaic, you can visualize and explore millions and even billions of data points at interactive rates.