Duckdb, Iceberg, and Upsert

A New Frontier

This has been a long time coming. I work with Duckdb nearly everyday to test and process small and medium datasets. I also work a lot with Apache Iceberg. Where these 2 generally do not jive though is when I need to run incremental merges to an Iceberg table a.k.a an “upsert”.

Upsert - An atomic operation on a table to update and insert date in a single…