DuckDB Labs recently released DuckLake 1.0, a data lake format that stores table metadata in a SQL database rather than ...

Mukul Garg is the Head of Support Engineering at PubNub, which powers apps for virtual work, play, learning and health. In my journey through data engineering, one of the most remarkable shifts I’ve ...

Organizations today are constantly seeking ways to optimize their operations and gain a competitive edge. Ashmin Swain, a seasoned data engineering professional and an Information Management graduate ...

Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with content, and download exclusive resources. Dany Lepage discusses the architectural ...

Data engineering is the gritty, often unglamorous work that underpins every AI model, every dashboard, and every strategic data driven decision. For years, we treated our data lakes like giant, messy ...

Though the AI era conjures a futuristic, tech-advanced image of the present, AI fundamentally depends on the same data standards that have been around forever. These data standards—such as being clean ...

Four key considerations to keep in mind when you need a database designed for analytical queries of vast quantities of time series data. SQL often struggles when it comes to managing massive amounts ...