Natural Language Processing is a computer science and artificial intelligence topic concerned with computer-human language interactions and how computers are designed for processing and exploring a variety of natural language data, in particular. The Structured Query Language for non-expert users is usually a challenging database storage, they may not know the database structure. For database applications to improve the interaction between database and user, a new intelligent interface is therefore necessary. The concept of utilizing a natural language instead of a structured query language has led to the creation of the natural language interface to database systems as a new form of processing procedure. The aim of this research is to build a query generating process using an algorithm for the machine learning to represent information according to user's demands for answering query and obtaining information. For the conversion of Natural Language Query into Structured Query, we utilized a lowercase conversion, removing escaped words, tokenization, PoS tagging, word similarity, Jaro-Winklar matching algorithm, and the method Naive Bayes.

1. 3rd INTERNATIONAL CONFERENCE ON SUSTAINABLE
TECHNOLOGIES FOR INDUSTRY 4.0
Date: 18 - 19 December 2021
Natural Language Query to SQL conversion
using Machine Learning Approach
Minhazul Arefin, Kazi Mojammel Hossen and Mohammed Nasir Uddin

2. 2
Overview
Introduction
Problem Description
Objective of this Paper
Proposed Methodology
Result
Conclusion
Future Works

3. 3
• Natural language or ordinary
language is any language that
has evolved naturally in human brain.
• It can take different forms, such as
Natural Language
 Speech
 Signing

4. • SQL stands for Structured Query
Language
• It is used to communicate with a
database.
• Standard SQL commands
4
Structured Query Langage
 Select
 Insert
 Update
 Delete

5. 5
Natural Language
Processing (NLP)
Structured Query Language (SQL)
Machine Learning
Algorithms
• NLIDB stands for Natural Language Interface with Database
Systems.
Introduction

6. Problem Description
• Asking questions in natural language to get answers from
databases is a very convenient and easy method of data
access.
• For non-expert user, it is necessary to compile the natural
language to structured query language (SQL)
• Filling a form in internet that has many fields can be tedious
for navigate through the screen, to scroll, to look up the
scroll box values.
6

7. Objective
The main objectives of this research work are:
 To provide algorithms for converting Natural Language to
Structured Query Language (SQL)
 To propose a general framework for efficient processing of
natural language query
 To extract information from the database.
7

8. Contributions
The main contributions of this research work are:
 Designing algorithms for this machine translation system
 Implementing the proposed translation algorithm and comparing
the performance of our approach with the state-of-the-art works.
Our findings show that machine learning approach can outperform
other existing systems.
 Using simple algorithms increase the performance as well as
reducing the time complexity.
8

9. Methodology
9

10. 1. Text Preprocessing
10
Tokenization
Escape word
Parts of Speech Tagger
Word Similarity

11. 1.1. Tokenization
• Tokenization is the process of converting a sequence of
characters into a sequence of tokens.
• This tokenize function performs the following steps:
 treat most punctuation characters
 split off commas and single quotes, followed by
whitespace
 separate periods that appear at the end of line
Input Text : “get names of all students”
Output After tokenization:
11

12. 1.2. Escape Words
• The escape word is a set of words which contains
the list of unnecessary words that occur in the
given text.
• It mainly contains
 Auxiliaries verb
 Articles
12
Input from Tokenization Step :
Output After Removing Escape Words:

13. 1.2. Escape Words
13

14. 1.3. Part-Of-Speech Tagger
• Parts-of-speech(PoS) tagging used to classify
words into their parts-of-speech
• Input Get from Tokenization Step :
14
Here,
VB -> Verb, base form
NNS -> noun, common, plural
DT -> Determiner, article
Output After tokenization:

15. 1.3. Part-Of-Speech Tagger
15

16. 1.4. Word Similarity
• In this step we get all the synonyms of all words after we
remove escape words from the given text.
• For word similarity, we use WordNet database .
• For example all synonym of phone is:
16
For example Similarity between ‘telephone’ & ‘phone’:

17. 1.4. Word Similarity
17

18. 2. Attribute Extraction
• In this section at first we get the synonym of words from
tokenization step
• Then we match the type with one to another by Jaro -
Winkler algorithm
• It computes the similarity between two strings, and the
returned value lies in the interval [0.0, 1.0]
• The distance is computed as:
simw = simj + (lp(1- simj))
18

19. 19
2. Attribute Extraction

20. 2. Attribute Extraction
20
Input Text :
Output After attribute extraction:

21. 2. Attribute Extraction(Continue)
21
Input Text :
“get all telephone number, address & name of the
students”
Output After attribute extraction:

22. 3. Table Extraction
• This step only works if the previous step gets no
attribute from the given text.
• At first, this step find all table names from the
existing database.
• Then, it will go to the next step.
22
• For example:
Input Text : “show all”
Output After Table Extraction:

23. 3. Table Extraction
23

24. 4. Command Extraction
• Here we use Naive Bayes classifier for detecting SQL command
• Using Bayes' theorem, the conditional probability can be described
as:
𝑷(𝑨|𝑩) =
𝑷 𝑩 𝑨 × 𝑷(𝑨)
𝑷(𝑩)
• In our case, suppose we wantP(select | get names of all students). So
using this theorem we can get the conditional probability:
P(select |get names of all students) =
P (get names of all students | select) × 𝑷(𝒔𝒆𝒍𝒆𝒄𝒕)
P (get names of all students| select)
24

25. 4. Command Extraction
25
Input Text : “get names of all students”
Output command:
Sentence Result Select Insert Delete Update
Get names of all students Select 86.97 0.26 12.36 0.39
Result of command Extraction:

26. • We used decision tree classier for extract condition from
the given input
• It find the specific condition appropriate for the given input
text
26
5. Condition Extraction

27. 6. Query Generation
Input Text : “get names of all students”
Output After Query Generation:
27
Attributes FROM Table Name WHERE Condition
Operation
• In this step we will start to build the query.

28. 28
7. Executing the code
• In this step we run the SQL query which we get
from query generation step
Input Text : “get names of all students”
Output After Query Generation:
Input Text : “get all phone number, address, name of the students”
Output After Query Generation:

29. Result
29
Input Text : “get names of all students”
Output After Building Query:
Input Text : “SELECT names FROM students”
Output After Running Query:
Jakir, Minhaz, Jisan, Rana, Imran

30. Comparison
30
Sl. Model / Performance Factor Accuracy (%) Error Rate (%) Run Time (s)
1 Generic Model 73.14 26.86 5.29
2 NLIDB for RDBMS 83.6 16.4 7.8
3 Our Study 88.17 11.83 2.929
o We mainly focus on attributes to build an SQL query

31. Comparison
31

32. • This research has a substantial import on 4th industrial
revolution.
• As every automation system or IoT devices has a data store
that can be manipulated via plain text.
• Furthermore, existing database can be used as a knowledge
base for AI powered chat bots.
• Which may be used as Virtual assistants.
32
Conclusion

33. Future Works
In future, we intend to work on
 joining table
 develop an efficient algorithm using other mechanism for
better performance.
 provide a deep learning solution for this problem
33

34. Thanks!
Any questions?
34

35. Why use Naive Bayes
▷ Relatively less number of training samples are sufficient for
training with Naive Bayes algorithm
▷ variance tradeoff. Spam/sentiment type data are often
noisy and usually high-dimensional (more predictors than
samples, n « p. The naive assumption that predictors are
independent of one another is a strong, high-bias, one.
▷ By assuming independence of predictors we're saying that
covariance matrix of our model only has non-zero entries
on the diagonal.
35

36. Why use Jaro-Winkler?
▷ Jaro-Winkler gives a matching score between 0.0 to 1.0.
▷ The Jaro algorithm is a measure of characters in common,
being no more than half the length of the longer string in
distance, with consideration for transpositions.
▷ It gives high-accuracy
36

37. Jaro-Winkler
• Here we use Jaro-Winkler algorithm for attribute extraction
• We match all similar word with attributes by Jaro-Winkler algorithm and
detect the necessary attribute for the specific query.
• Jaro-Winkler is a string edit distance that was developed in the area of
record linkage (duplicate detection)
• It computes the similarity between two strings, and the returned value lies
in the interval [0.0, 1.0]
• The distance is computed as:
simw = simj + (lp(1- simj))
Where:
 Simj is the Jaro similarity for given strings s1 and s2
 l is the length of common prefix at the start of the string up to a
maximum of four characters
 p is a constant scaling factor for how much the score is adjusted
upwards for having common prefixes.
 The Jaro-Winkler distance dw is defined as dw = 1- simw
37