Translating synthetic natural language to database queries with a polyglot deep learning framework

Abstract

The number of databases as well as their size and complexity is increasing. This creates a barrier to use especially for non-experts, who have to come to grips with the nature of the data, the way it has been represented in the database, and the specific query languages or user interfaces by which data are accessed. These difficulties worsen in research settings, where it is common to work with many different databases. One approach to improving this situation is to allow users to pose their queries in natural language. In this work we describe a machine learning framework, Polyglotter, that in a general way supports the mapping of natural language searches to database queries. Importantly, it does not require the creation of manually annotated data for training and therefore can be applied easily to multiple domains. The framework is polyglot in the sense that it supports multiple different database engines that are accessed with a variety of query languages, including SQL and Cypher. Furthermore Polyglotter supports multi-class queries. Good performance is achieved on both toy and real databases, as well as a human-annotated WikiSQL query set. Thus Polyglotter may help database maintainers make their resources more accessible.

Figure 1

Overview of the random query generation method used by Polyglotter to obtain training datasets for the sequence-to-sequence models. Classes and attributes are added through a random walk, and constraint operators and values are also selected randomly.

Figure 2

Graphical summary of the overall workflow for Polyglotter.

Figure 3

Test set performance as a function of training dataset size for HumanMine, MySQL and Neo4j (rows 1, 2 and 3 respectively). Left: overall test set performance allowing any of the top 1, 3 or 5 predictions from each model to match the corresponding test. Right: test set performance separately for each of the elements forming a query (attributes, classes and constraints) when using just the top prediction. The error bars show the standard deviation across ten independent datasets.

Figure 4

HumanMine performance as a function of number of classes in the query, as the number of training items varies, with $k=1$. Note that queries containing five classes were absent from the training set.