Pushing ML Predictions Into DBMSs

Abstract

In the past decade, many approaches have been suggested to execute ML workloads on a DBMS. However, most of them have looked at in-DBMS ML from a training perspective, whereas ML inference has been largely overlooked. We think that this is an important gap to fill for two main reasons: (1) in the near future, every application will be infused with some sort of ML capability; (2) behind every web page, application, and enterprise there is a DBMS, whereby in-DBMS inference is an appealing solution both for efficiency (e.g., less data movement), performance (e.g., cross-optimizations between relational operators and ML) and governance. In this article, we study whether DBMSs are a good fit for prediction serving. We introduce a technique for translating trained ML pipelines containing both featurizers (e.g., one-hot encoding) and models (e.g., linear and tree-based models) into SQL queries, and we compare in-DBMS performance against popular ML frameworks such as Sklearn and ml.net. Our experiments show that, when pushed inside a DBMS, trained ML pipelines can have performance comparable to ML frameworks in several scenarios, while they perform quite poorly on text featurization and over (even simple) neural networks.

Keywords: MLOPs; SQL; machine learning.

Fig. 1.

A typical ML workflow. Rectangles are used to identify data artifacts (e.g., input data, or trained models); ellipses determine computations (e.g., data preparation and serving).

Fig. 2.

$\mathsf{MASQ}$ applied to an ML predictive pipeline.

Fig. 3.

Parsing of the pipeline of Example 1. The pipeline (top) is parsed on a container DAG (bottom). Each container stores a reference to the operator, its signature and extractor.

Fig. 4.

Scaling and linear model in SQL.

Fig. 5.

One-hot encoding in SQL.

Fig. 6.

SQL workflow for the one-hot encoding sparse implementation.

Fig. 7.

Pipeline with OHE followed by a linear regression executed in $\mathsf{MASQ}$ with TRO and partitioning.

Fig. 8.

How tree ensembles over triplet are translated in $\mathsf{MASQ}$.

Fig. 9.

Throughput for Sklearn, ml.net (on CSV, MySQL and SQL Server) and $\mathsf{MASQ}$ (on MySQL and SQL Server).

Fig. 10.

Scalability of the different frameworks, over MySQL, as we change the batch size.

Fig. 11.

Latency numbers over a single record (MySQL).

Fig. 12.

Operator breakdown for P7 and P8.

Fig. 13.

Operator breakdown for P9 ($\mathsf{Criteo}$) and P10 ($\mathsf{Flight}\mathsf{Delay}$). For P9 we also compare GBDT vs SDCA.

Fig. 14.

Latency breakdown for $\mathsf{MASQ}$, ml.net (ML.) and Sklearn (SK.), for pipelines P7, P8, P9 and P10. The time spent is divided into three buckets: load, computation, and write.

Fig. 15.

Performance comparison with indexing (MySQL).

Fig. 16.

Operator fusion (OHE + GBDT) for P9 and P10.

Fig. 17.

Comparison of different tree implementation methods (MySQL).

Fig. 18.

Comparison of single intermediate data and multi-way join strategy for OHE + linear models.

Fig. 19.

Comparison of tree ensembles performance with variable number of leaves on P8.

Fig. 20.

Left hand-side: Sentiment Analysis over textual features. Right hand-side: an MLP model applied on $\mathsf{Credit}\mathsf{Card}$.