Learning to optimize database joins
Teaching PostgreSQL join operators to learn during query execution and share information across operators.
Yong-Hwan Lee · Arash Termehchy · Huazheng Wang
Published · Updated
Abstract
It is crucial to provide real-time performance in many applications, such as interactive and exploratory data analysis. It is challenging to deliver such results over large datasets for relational operators over multiple relations, such as join. Join algorithms usually spend a long time scanning and attempting to join parts of relations that may not generate any result. Current solutions usually require lengthy and frequent preprocessing, which is resource-intensive and may not be possible to do in many settings. Also, they often support restricted types of joins. We outline a novel approach for achieving efficient join processing in which join operators learn during query execution the portions of their inputs that are most likely to satisfy their join predicates. We improve this approach by overlapping learning and sharing information across join operators to speed up learning and query execution. Our empirical studies using standard benchmarks and real-world datasets indicate that this approach outperforms similar methods considerably.
Figure 1: Example of Learning Query Processing Strategy Online
Publication status
Submitted to Proceedings of the VLDB Endowment (PVLDB) in 2025.