Legacy database protocols emerged in an era before massive data volumes became routine. Many continue to transmit results one row at a time, duplicating metadata such as column names and data types alongside each row. According to Guillermo Sánchez Dionis, product team lead at DuckLabs, this approach generates "quite a bloated message." The cumulative effect of this overhead becomes significant when transferring large result sets across networks.
DuckDB's response is Quack, an HTTP-based protocol engineered to stream query results as batches of columnar data as the database engine produces them. The protocol was shown in preview during May and reached production status in October as part of DuckDB 2.0.0 LTS. Sánchez Dionis discussed the protocol's design during a recent episode of The New Stack podcast, recorded at DuckLabs' Amsterdam office, highlighting how it enables developers to begin processing results before query execution completes.
Understanding Database Protocols
A database protocol establishes the formal rules and message structures governing communication between client applications and database servers. It functions as a specialized interface allowing software to transmit SQL queries securely, retrieve structured data, and coordinate transactions across network connections. The protocol manages connection establishment, user authentication, query formatting, data serialization, and error handling.
Sánchez Dionis notes that most existing protocols were designed during the pre-big-data era for different use cases, relying on serialized rows rather than columns. They also introduce unnecessary overhead by including column types and names with every row. "It becomes quite a bloated message. That already makes the protocol not only not good for analytical systems, but it's also quite heavy," he states. While later protocols adapted for analytical workloads by adopting columnar approaches, "they were still doing things like serializing one value at a time and other things that are really not efficient. Modern systems are trying to converge on using vectorized execution to be able to serialize more values at a time and trying to use more efficient representations of columnar data formats to send over the wire."
Quack's Production Release and Features
Quack reached production status on October 21 with DuckDB 2.0.0, marking its transition from preview to general availability. The release introduces several capabilities designed for developer convenience:
- The
CONNECTstatement enables developers to attach to and query remote Quack servers using standard SQL syntax. - A new
-serveCLI flag launches a Quack server directly from the command line, while-connectestablishes connections to running instances. - Multiple clients can simultaneously write to the same database, with transaction isolation managed through DuckDB's multi-version concurrency control (MVCC) mechanism.
Efficiency Through Streaming and Vectorization
Quack leverages DuckDB's internal columnar data representation, known as data chunks, combined with efficient serialization enabled by the vectorized execution engine. The protocol's distinguishing feature is its streaming approach to result delivery. "We also make sure that we send the results in a streaming fashion," Sánchez Dionis explains. "One of the most important things is tightly coupling the engine. Once you start producing results, instead of waiting for the query execution to end and then serializing all of the results at once, you serialize and send results as they are emitted by the engine. Therefore, the client can keep consuming results without having to stop the engine."
This architecture eliminates the need for developers to implement special handling. "You run a query from the comfort of your laptop, and you just get streaming results back." Sánchez Dionis highlights the practical benefits: . The flexibility extends to deployment scenarios—Quack can be installed and run locally, and multiple clients on the same machine can communicate through the Quack extension without additional infrastructure.
Scaling Quack deployments depends entirely on organizational requirements. The server-side resource consumption remains minimal, representing Quack's most significant efficiency improvement to date. DuckLabs plans additional enhancements in upcoming DuckDB 2.0 patch releases, focusing on server-side stability improvements.