DuckLake: The Lakehouse That's Just SQL & Parquet [Talk Python to Me, Ep. 562] How many files does your query read before it reads any data? On some data lakes, you go through JSON and metadata files first, just to learn which Parquet files matter. #DuckLake asks one SQL question instead. The metadata lives in a real database. The data stays in plain Parquet. That's the entire format. With Quack as the catalog, DuckLake handles 200 transactions a second under heavy contention. No other open table format comes close. In this episode of @TalkPython, host @mkennedy talks with @holanda_pe, lead developer for DuckLake at DuckLabs, and Guillermo Sanchez Dionis, who works on both DuckLake and the new Quack protocol – together they discuss all the details of DuckLake, Quack, and more. Watch the episode here or listen wherever you get your podcasts: duckdb.org/library/ducklake-…

DuckDB 2.0 alpha reads Parquet and CSV from S3 2x to 3x faster, with zero query changes. 1.5: each worker downloads, waits, decodes, repeats. 2.0: a download pool fetches ahead, workers only decode. Same laptop, same query: 18.8 s to 7.7 s. On by default. Full numbers, plus recursive CTEs and VARIANT: motherduck.com/blog/why-duck…

The new DuckDB async I/O seems to be working really well. 3,000 CSV files / 163 GB → Onelake Iceberg in just 109 seconds is pretty damn impressive. 😁 That's about 1.49 GB/s of end-to-end processing #onelake #apacheiceberg #python #dataengineering #duckdb

🦆🤓 When Denis Hirn implemented DuckDB’s first recursive CTE operator in 2020, correctness determined the design: evaluate the non-recursive term once, then evaluate the recursive term until the next working table is empty. That established the right semantic contract, but reusable runtime state was scoped too narrowly. In the upcoming DuckDB v2.0 release, we assign those scopes explicitly. #DuckDB’s recursive CTE engine now treats recursion as one long-lived computation: it retains eligible epoch-invariant state, chooses execution modes from exact frontier cardinalities and physical work, probes keyed state directly and gives `USING KEY ... UNION` changed-key semantics. Read more in this preview of the forthcoming v2.0 release: duckdb.org/2026/08/25/how-du…

Today, @ducklabs_com is joining @awscloud. The move is expected to be completed by early September. Joining AWS gives DuckLabs the resources and reach to bring DuckDB, DuckLake, and the Quack protocol to many more developers and organizations – and to pursue ideas at a scale that would have been difficult for us to reach alone. The DuckLabs team will remain together in Amsterdam and will continue working for the Duck Stack community as an AWS subsidiary. Most importantly, #DuckDB and the other open source components of the Duck Stack will remain free and open source under the MIT license, with the non-profit DuckDB Foundation continuing its stewardship of the projects. This is a significant moment for DuckLabs and the Duck Stack community. It marks the end of one chapter that we are immensely proud of, and the beginning of another that we believe will take DuckDB much further. Read the full announcement here: ducklabs.com/ducklabs-is-joi… #AWS

Quack, quack! A DuckDB v2.0 preview is here! 🦆 🦆⚡ In this post, #DuckDB co-creators Mark Raasveldt and Hannes Mühleisen preview the headline features of DuckDB v2.0 (expected to arrive this autumn): • DuckDB as a server • Triggers • The VARIANT type • Asynchronous I/O • A new SQL parser • A new storage format • and much more! 🤩 Dive into the details: duckdb.org/2026/08/17/duckdb…

New blog post alert! 🦆 🦆 ⚡ ` Asynchronous I/O in DuckDB: Work, Thread, Work ` by @holanda_pe It doesn't matter how fast query operators are in a database system if we can't pull in the data quickly. For most of DuckDB's history, however, this problem was largely avoided by pruning data early. By pushing down filters and projections, we could ensure that we only read what we actually needed. This worked particularly well because DuckDB primarily ran locally, with its main use case being as a quick-draw database engine for querying data directly from your machine's SSD. But, as usual, things changed. Starting with DuckDB v2.0 – scheduled for fall 2026 – DuckDB will support asynchronous reads of Parquet and CSV files. This significantly speeds up queries when synchronous I/O does not saturate the available bandwidth, as is typical in EC2/S3 compute-storage setups. Read the rest here: duckdb.org/2026/07/31/asynch… #DuckDB

Ducklake with duckdb is really really good. Tell your agent to use it if you ask it to import a lot of data ( like health data from Fitbit ). If you use Hermes, just go “/learn duckdb and ducklake and use it” and all your data asks will get better.

Summary of my last month: piped.video/watch?v=1V_rT1Ye…

Today, we are releasing DuckDB v1.5.3. This is not an ordinary patch release – it packs many new features as extensions. First, DuckDB v1.5.3 ships our new Quack client-server protocol as an autoloadable core extension. Second, Quack can now also work as a DuckLake catalog. And finally, the AWS and Iceberg extensions made huge strides, along with changes in the HTTPS extension for better proxy support. We wrote a blog post about all these changes and improvements: link in the thread 🧵

Serverless DuckLake on @CloudflareDev Durable Objects via the new Quack protocol of @duckdb

Brazil's courts have ruled that dogs and cats are legally recognized as sentient beings, not property. The Sansão Law increased penalties for animal abuse against dogs and cats to 2 to 5 years in prison, along with fines and a lifetime ban on owning animals for offenders.

Ducklake already supports quack, and it will be THE DuckLake catalog in the long run. I'm already working on server-side retrials to skip unnecessary round-trip retrials under contention (i.e., gotta get more tx/s).

DuckDB just released Quack: a client/server protocol over HTTP. Any DuckDB can serve (quack_serve()), any DuckDB can attach to it. DuckDB-Wasm included since it's plain HTTP on port 9494. Some fun stuff : - A browser tab talking directly to a DuckDB server somewhere on the internet - Local-first apps with a remote target that speaks the exact same dialect - Notebook-to-notebook query forwarding Curious what people will build with this. We've got a few ideas brewing at MotherDuck too :) And while we're at it, can we finally retire the "DuckDB has no multi-writer support" take? There are plenty of options out there now, just depends how you want to slice and dice.

Hannes Mühleisen just revealed the Next Big Thing for DuckDB at AI Council 2026: Quack, a protocol that turns DuckDB into a client-server database. True to DuckDB's philosophy, Quack is simple and fast. You can set it up in seconds and it delivers high-performance remote access, turning DuckDB into a full-fledged general-purpose database system. We are very excited to see the possibilities Quack will unlock, from serving multiple concurrent writers to creative architectures with several DuckDB processes talking to each other. The ingenuity of our community never ceases to amaze us and we're certain that we'll see many use cases we did not even think of. For now – follow the link in the thread, give it a spin and let us know what you think. Happy quacking!

DuckLake v1.0 has a feature called "inlining". This allows you to stream directly to DuckLake without creating a file for each INSERT. The row data is stored in the metadata database first and referenced when queried. You then use CHECKPOINT to flush the data to a Parquet file.

DuckLake connector in @ApacheDataFusio contrib: github.com/datafusion-contri… is an interesting alternative for people looking for faster performance and don't need the current (and ever growing) feature set of Apache Iceberg

The Delta and Unity Catalog extensions in the latest DuckDB release come with a fresh set of features and have shed their experimental labels. In today's blog post, Ben Fleis walks you through the key improvements: ✍️ You can now write Delta tables with DuckDB. Multiple inserts within a transaction produce a single atomic version in the Delta table. 🤝 The Unity Catalog unlocks multi-writer access. DuckDB and other clients such as Spark can now perform writes alongside each other with the catalog handling concurrency control. ⏪ You can use the coolest feature of data lake formats: time travel. This lets you query any Delta table at a specific historical version. Thanks to incremental snapshot loading, this is fast even across large Delta logs. Read the full blog post for more – link in the thread 🧵

We're excited to announce the program of DuckCon #7 Amsterdam! The event will start with the “State of the Duck” talk, covering DuckLake v1.0 and the super-secret next big thing for DuckDB – something we cannot tell you more about yet as it will be released next week! We'll continue with six regular talks covering use cases, technologies built with DuckDB and DuckLake, and a talk on how Spotify uses DuckDB to analyze their listening history. The program also features six lightning talks, including one on how the Toyota Gazoo Racing World Rally Team built an application with DuckDB. Finally, we'll have drinks and snacks so you'll have ample time to socialize with the flock! For the full program and registration, follow the link in the thread.