SlideShare a Scribd company logo
SID:XXXXXXX MOD002691
1
QR Secure: A hybrid approach using Machine
Learning and Security Validation Functions to
prevent interaction with Malicious QR codes.
Word Count: 10000
Richford, A.
SID:XXXXXXX
MOD002691 Final Project
Final Project Report
BSc Cyber Security
Submitted: 24/03/2024
SID:XXXXXXX MOD002691
2
Abstract
QR codes are becoming an increasingly used attack vector for cybercriminal to obtain users
confidential information resulting in both financial and identity theft. This study has been
formulated with the intent to discover how effective a hybrid approach of machine learning and
programming validation functions are at determining if a QR code derived Uniform Resource
Locator (URL) is malicious in nature. The first section of this study details why this question is
necessary and what threats are faced from malicious QR codes. In addition to this background
information on QR codes, machine learning (ML), Public Key Infrastructure (PKI) certificates
and URLs has been detailed. Next a literature review on several related academic papers has been
conducted to obtain a problem statement for the paper. From this the methodology has been
defined for the planning, creation and implementation of a system which uses ML, a URL format
validation function, and a PKI certificate validation function to determine if a QR code is
malicious in nature. Finally, the implementation section details the creation of the system from
the development to testing. The results show the effectiveness of a hybrid approach to addressing
if a URL derived from a QR code is malicious, this has been fostered by a highly accurate and
efficient ML model in conjunction with the programming validation functions, the discussion and
conclusion section of this study details these findings.
SID:XXXXXXX MOD002691
3
Acknowledgements
Firstly, I would like to acknowledge the significant support provided to me by my family, who
have always supported me in both relation to my studies and personal endeavours. In addition, I
would also like to acknowledge the support of the faculty residing at Anglia Ruskin University.
In specific I would like to acknowledge my supervisor and Personal Development Tutor (PDT)
Muhammad Ali. Who has provided exceptional support and time investment into me
throughout this development project and my university career.
SID:XXXXXXX MOD002691
4
Table of Contents
Acknowledgements.................................................................................................................. 3
1.0 Introduction....................................................................................................................... 8
1.1 Problem Statement ........................................................................................................ 9
1.2 Aims of the study ........................................................................................................... 9
1.3 Contribution................................................................................................................... 9
1.4 Structure...................................................................................................................... 10
2.0 Background on QR codes.................................................................................................. 11
3.0 Background on ML ........................................................................................................... 18
4.0 Literature Review............................................................................................................. 21
4.1 Critical Analysis ............................................................................................................ 29
5.0 Proposed Work ................................................................................................................ 31
5.1 Methodology................................................................................................................ 31
5.2 Machine Learning model to detect malicious URLs. ...................................................... 33
5.2.1 Collection of data .................................................................................................. 33
5.2.2 Cleaning and preparation of dataset...................................................................... 35
5.2.3 Feature Engineering............................................................................................... 36
5.2.4 Test classifier algorithms against model to determine the most appropriate
algorithm. ...................................................................................................................... 37
5.3 Validating if URL has a valid PKI certificate.................................................................... 38
5.4 Validating if URL format is valid. ................................................................................... 39
5.5 Creation of QR code reader and system GUI................................................................. 41
6.0 Implementation and Results............................................................................................. 43
6.1 Implementation of ML model to detect malicious URLs. ............................................... 46
6.1.1 Testing model predictions against known malicious and safe URLs. ....................... 51
6.2 Implementation of URL PKI certificate validation.......................................................... 53
6.2.1 Testing function against known valid and invalid certificates.................................. 54
6.3 Implementation of URL format validation..................................................................... 55
6.3.1 Testing function against known valid and invalid format URLs................................ 57
6.4 Implementation of the QR code scanner ...................................................................... 58
6.5 Implementation of Graphical user interface ................................................................. 61
7.0 Testcases ......................................................................................................................... 65
8.0 Discussion ........................................................................................................................ 71
9.0 Conclusion ....................................................................................................................... 73
References............................................................................................................................. 74
SID:XXXXXXX MOD002691
5
Appendix I: Interim report...................................................................................................... 82
Appendix II: Ethics Certificate of Completion.......................................................................... 87
Appendix III: Project Poster.................................................................................................... 88
Appendix IV: Presentation of source code.............................................................................. 89
SID:XXXXXXX MOD002691
6
Table of Figures
Figure 1: Generated QR code containing link 'https://QRSecure.com' .................................... 11
Figure 2: Generated QR code containing link 'http:/MalWARE.cog' ........................................ 12
Figure 3: CIA triad (IBM, 2023) ............................................................................................... 13
Figure 4: Certificate chain (The SSL Store, n.d.)....................................................................... 14
Figure 5: Accuracy Comparison (Adapted from Pawar et al, 2022).......................................... 21
Figure 6: Evaluating security performance of QR code scanners (Adapted from Rafsanjani et al,
2023) ..................................................................................................................................... 24
Figure 7: Testing results (Adapted from Xuan et al, 2020)....................................................... 27
Figure 8: Proposed architecture of system function. .............................................................. 32
Figure 9: URL Dataset entries. ................................................................................................ 34
Figure 10: ratio of good and bad URLs in dataset.................................................................... 34
Figure 11: Observing no NULL values in dataset. .................................................................... 35
Figure 12: Figure to show testing and training set split........................................................... 36
Figure 13: Illustration of certificate validation. ....................................................................... 38
Figure 14: Validators source code (Adapted from validators, n.d.).......................................... 39
Figure 15: Illustration of URL validation.................................................................................. 41
Figure 16: Navigation map of system. .................................................................................... 42
Figure 17: Wireframe diagram of GUI (iPhone template adapted from unblast, n.d.) ............. 42
Figure 18: Imported libraries for ML model............................................................................ 46
Figure 19: โ€˜urldataโ€™ dataset manipulation............................................................................... 46
Figure 20: splitting dataset into input and output set. ............................................................ 47
Figure 21: Vectorizing data with TF-IDF .................................................................................. 47
Figure 22: Splitting data for testing and training..................................................................... 48
Figure 23: NB, RF, SVM, LR and DT model reports. ................................................................. 48
Figure 24: Confusion matrix for LR. ........................................................................................ 50
Figure 25: ML model prediction function................................................................................ 50
Figure 26: PKI Certificate validation function import. ............................................................. 53
Figure 27: URL PKI Certificate validation function................................................................... 54
Figure 28: Tested good and bad certificates. .......................................................................... 54
Figure 29: URL format validation function import................................................................... 55
Figure 30: URL format validation function. ............................................................................. 56
Figure 31: QR code scanner imports....................................................................................... 58
Figure 32: QR code scanner working example. ....................................................................... 59
Figure 33: QR code scanner code. .......................................................................................... 60
Figure 34: GUI imports........................................................................................................... 61
Figure 35: GUI code segment 1 .............................................................................................. 62
Figure 36: GUI code segment 2. ............................................................................................. 63
Figure 37: GUI code segment 3 .............................................................................................. 64
Figure 38: GUI code segment 4. ............................................................................................. 64
Figure 39: Certificate of Completion CPD Course.................................................................... 87
Figure 40: Project Poster........................................................................................................ 88
SID:XXXXXXX MOD002691
7
Table of Tables
Table 1: Components of valid URL.......................................................................................... 16
Table 2: Components of example URL.................................................................................... 17
Table 3: Classification algorithms for model testing................................................................ 19
Table 4: Details on validator source code ............................................................................... 40
Table 5: Utilized python libraries............................................................................................ 45
Table 6: Accuracy of algorithms summary.............................................................................. 49
Table 7: ML model predictions of provided URLs.................................................................... 52
Table 8: Testing PKI Certificate validation function................................................................. 55
Table 9: Testing URL format validation function ..................................................................... 57
Table 10: Testcase 1............................................................................................................... 65
Table 11: Testcase 2............................................................................................................... 66
Table 12: Testcase 3............................................................................................................... 67
Table 13: Testcase 4............................................................................................................... 68
Table 14: Testcase 5............................................................................................................... 69
Table 15: Testcase 6............................................................................................................... 70
SID:XXXXXXX MOD002691
8
1.0 Introduction
One of the most notorious attack vectors used by cyber criminals today is Phishing, this attempts
to lure a target individual into providing confidential or sensitive information and will often direct
a user to a malicious webpage where malicious activities such as data theft are inflicted on the
victim (Phishing.org, n.d.). It is estimated that 3.4 billion phishing emails are sent per day
(Griffiths, 2023). However, due to the constant changing technology landscape, threat actors are
finding new ways to lure individuals into unknowingly providing their confidential information.
One of the emerging attack vectors is known as Quishing. Quishing, also known as QR code
phishing is where an attacker lures a victim into scanning a malicious QR code which then
redirects the victim to a malicious URL in attempts to infect them with malware or acquiring the
victimโ€™s confidential information (sosafe, n.d.). In the month of September 2023 QR code
phishing attacks saw a rise of 51% compared to the combined known attacks from January to
August 2023 (Security Staff, 2023). In addition to the recent rise in QR code phishing attacks,
the overall cyber security attack posture has QR code phishing attacks as 22% of all phishing
attacks within the month of October 2023 (Alder, 2023). This data suggests that QR code phishing
attacks are being increasingly used by threat actors to conduct both cyber enabled crime such as
identity theft and fraud, in addition to cyber dependent crimes such as system hacking and
malware infections. This recent change in the threat landscape is what inspired the creation of a
system that can be used to scan QR codes and determine if the derived URL is malicious in nature.
Such a system would be able to mitigate QR code phishing attacks and therefore decrease the
viability of QR codes as an attack vector. This report has been formulated to detail the research,
planning, creation, and testing of such a system I created in efforts to achieve this goal.
SID:XXXXXXX MOD002691
9
1.1 Problem Statement
This study plans to answer the question: Can a hybrid approach using ML, and programming
validation functions successfully be used to identify malicious URLs derived from scanned QR
codes in both an accurate and efficient fashion?
1.2 Aims of the study
The aims of this study are to detail the research, planning, and creation of a system which prevents
interactions with malicious QR codes, ideally this report will:
โ€ข Provide research on previously used methods to detect malicious content within
URLs derived from QR codes.
โ€ข Develop a hybrid solution to identify malicious URLs derived from QR codes
that uses both ML and programming language functions which concern both the
validity of the URLโ€™s PKI certificate state and the URL format.
โ€ข Explore multiple ML classification algorithms against a model to determine
which prospers the most accurate and efficient result and is therefore most suited
to the system.
1.3 Contribution
My proposed system can be used to efficiently and accurately identify malicious QR codes, as a
result mitigate any unsafe interactions with them. As a result, attacks such as Quishing will be
significantly reduced and therefore the threat landscape to users will be pronouncedly smaller.
SID:XXXXXXX MOD002691
10
1.4 Structure
Chapter 1 details the introduction to the study, the research question, and aims.
Chapter 2 and 3 detail related background information and concepts.
Chapter 4 consists of a literature review on several academic papers related to my research
question.
Chapter 5 details the proposed work and methodology that will be followed for implementation.
Chapter 6 details the implementation of the system. In addition, conducts testing to determine the
accuracy and integrity of the solutions.
Chapter 7 conducts testcases on the complete system.
Chapter 8 consists of a detailed discussion on the results of the study.
Lastly chapter 9 concludes upon the study and determines if the aims have been achieved.
SID:XXXXXXX MOD002691
11
2.0 Background on QR codes
This section is formulated to provide background information of the concepts used within the
study, and their relevance to the research question.
QR Codes
Vishrut Sharma notes, that QR (Quick Response) codes are a two-dimensional barcode which
was first created in 1994. QR codes were first used in attempts to identify cars within car
manufacturing processes. However, due to the fast readability of these codes in conjunction with
the relatively large storage capacity, QR codes are now extremely popular in all aspects and
domains of life. With the only barrier of entry being the need for a smartphone camera which is
rather ubiquitous today (Sharma, 2012).
QR codes can be encoded with either numeric or alphanumeric information, this information is
often related to a URL. According to Jessica Scapati:
โ€œA URL (Uniform Resource Locator) is a unique identifier used to locate a resource on the
internet.โ€ (Scarpati, 2021).
From this it can be understood that a URL is used in efforts to navigate the internet by acting as
an address of a websites. QR codes can have URLs encoded within them to direct users to a
specified website. An example of a QR code encoded with a URL can be seen below.
Figure 1: Generated QR code containing link 'https://QRSecure.com'
SID:XXXXXXX MOD002691
12
Threat actors can use QR codes as an attack vector by encoding a QR code with a Phishing URL.
This could be a mimicking login of a bank in attempts to enumerate a targets bank information.
Or, in addition, have an encoded URL which has a malware download on the website. Although
these attack vectors exist, there is no obvious way to determine if the encoded content of a QR
code is safe, as a QR code is only a representation of encoded data, no sanitation of that data is
conducted. For instance, the below QR code has a malformed URL and has malicious indicators
such as the key word โ€˜Malwareโ€™.
The QR code seen in the figure above has an invalid URL format of โ€˜http:/โ€™ where this should be
โ€˜https://โ€™ which is the correct format for a secure URL. In addition, it contains the keyword
โ€˜MalWAREโ€™. Although the content is seemingly malicious, the visual representation is like the
โ€˜safeโ€™ QR code seen in the previous figure, this comparison demonstrates how a victim could
easily scan a malicious QR code believing it is legitimate and safe.
As there is no simple way to identify malicious QR codes, the interaction with them can be
extremely dangerous. With the projected smartphone QR scans rising to 99.6 million in the US
alone by 2025 (Cherisien, 2024), the need to ensure safe interaction is paramount. In addition to
the rise in QR code scans, a study indicated 80% of respondents had used QR codes for payment
transactions (Cherisien, 2024), this ubiquity and trust in the technology fosters huge concern for
security and safety as a popular technique in phishing is to overlay a legitimate QR code with a
Figure 2: Generated QR code containing link 'http:/MalWARE.cog'
SID:XXXXXXX MOD002691
13
malicious one to trick an individual into interaction with it. This highlights the importance to be
able to identify malicious QR codes and in tandem the importance of this study.
PKI Certificates
As there is no specific way to identify malicious QR codes the QR code must be decoded to reveal
the data. As discussed previously, the encoded data typically will be a URL. One way to identify
if a URL is likely safe is to ensure it has a valid PKI certificate.
Public Key Infrastructure (PKI) Certificates are digital certificates which are used to authenticate
users and encrypt connections across networks (Comodo, n.d.). A PKI certificate uses Transport
Layer Security (TLS) which is a protocol used to provide encrypted and authenticated
communications. Lawrence E. Hughes notes, prior to being named TLS it was known as Secure
Socket Layer (SSL) which is now been deprecated for over two decades, however the terms are
often still used interchangeably (Hughes, 2022).
PKI certificates ensure both Confidentiality of the data via encryption, and integrity due to the
authentication of the certificate user, which are two of the three fundamental pillars within the
Confidentiality, Integrity, and Availability (CIA) triad, as seen in the below figure.
Figure 3: CIA triad (IBM, 2023)
SID:XXXXXXX MOD002691
14
PKI certificates are used within PKI, comodo notes, PKI is a fundamental component of the
current internet, it works via a hierarchy of trust that starts from Certificate Authorities (CA)
which upon validating parties, can issue digital certificates to them. At the top of the hierarchy is
the Root CA which has the highest level of authentication as this is the entity from which
certificates are issued. Below root CAs are Intermediate CAs which are used to decrease the
workload from root CAs and distribute certificates for use, such as for a browser connection (The
SSL Store, n.d.). A visual representation of this can be seen in the below figure.
PKI is fundamentally used to ensure that certificates are issued to the correct entities to allow
trust and secure connections between users online. Without a PKI Certificate there is no verified
trust within that entity. This means that a connection to a website lacking a PKI certificate could
potentially be unsecure and lack the implementation of TLS resulting in no encryption or integrity
between the parties. This is common behaviour in websites that have malicious intent as an
illegitimate website may struggle to obtain, or not want to obtain a PKI certificate. The lack of a
certificate allows threat actors to steal information upon a connection to one of their sites, as there
is no security protocol implemented, which can result in targets personal information being stolen
from the session.
Figure 4: Certificate chain (The SSL Store, n.d.)
SID:XXXXXXX MOD002691
15
From this it can be understood that PKI certificates are used to ensure that users have
confidentiality and integrity when online and is an essential part in any website or internet
connection, as such, it is essential that a URL derived from a presented QR code, has a certificate
check to ensure that the connection is secured.
Valid URL format
David Naylor et al notes, HTTP (Hyper Text Transfer Protocol) is a foundational component in
using the internet, it is an essential part of loading webpages on computer systems (Naylor et al,
n.d.). However, it is not secure, its alternative HTTPS (Hyper Text Transfer Protocol Secure) is
in fact secure, and it is the standard for navigating the internet securely today, taking advantage
of SSL/TLS Certificates detailed in the above section PKI certificates is extremely important to
ensure security when navigating the internet. URLs are mostly used with the internet protocol
HTTP/HTTPS and therefore will be used to explain the components of a URL and how to ensure
a URL is valid.
IBM notes, that a URL must possess certain components for it to be valid for use on the internet.
These being:
SID:XXXXXXX MOD002691
16
URL Component Description
Scheme A scheme is the protocol identified within the URL.
Host A host is the address of the resource. This can be a host name relating
to an Internet Protocol (IP) address. Or can alternatively be a domain
name related to an IP address such as an A record for IPv4. In addition,
host names can include the port number appended to the host.
Path A path being the path to the resource that is being accessed, such as a
webpage.
Query strings In the event a query string is used this must be specified in efforts to
allow the resource information to perform an action. (IBM, 2021)
Table 1: Components of valid URL
An example of a complete HTTPS URL would look like:
https://www.ExampleURL.com/thePath/recource.html
each section of the above example address can be seen detailed in the table below.
SID:XXXXXXX MOD002691
17
Section from example Component
https:// Scheme
www.ExampleURL.com Host
/thePath Path
Table 2: Components of example URL
As seen in the table above, URLs follow a specific format to ensure that they are all uniformed.
Defined by RFC 1738. (RFC, 1994.) As seen scheme is followed by :// and / are used to separate
the components of the URL.
A threat actor may deliberately malform a URL for malicious purposes. For instance, the example
URL below is malformed, however at first glance, many will not see any issue.
https:/ExampleURL.com
The above example URLs scheme is malformed resulting in the URL not using the HTTPS
protocol as it has a missing /. If a URL is malformed, it is an indication that it is malicious and
could be a malicious embedded download and not a webpage.
Due to the possible risk within malformed URLs, this is why a URL validation function will be
implemented into the development project, to ensure any URLs derived from QR codes have
legitimately formatted URLs.
SID:XXXXXXX MOD002691
18
3.0 Background on ML
For one of the fundamental aspects of this study, machine learning has been used to detect
malicious URLs derived from a provided QR code. According to Jafar Alzubi et al:
โ€œMachine Learning (ML) is a category of artificial intelligence that enables computers to think
and learn on their ownโ€ (Jafar, et al., 2018).
From this it can be understood that ML allows computers to make intelligent decisions based
upon learned behaviour. For a machine to perform this type of learning and decision making, an
algorithm must be implemented to a model specific to the type of problem you wish to solve.
There are a few variations of ML that can be used to apply to a problem. Reinforcement learning
can be used to learn a series of actions without any predefined data, unsupervised ML uses
unlabelled data and identifies patterns within the data. And lastly, supervised ML uses labelled
data to calculate an outcome (Kumar, 2020) supervised ML is most suited in relation to this
project as a prediction based on previous data needs to be determined. The problem faced in this
study is a classification problem, this is often thought of as a problem in which the answer resides
as โ€˜yes or โ€˜noโ€™ (Jafar, et al, 2018). The question being, is the related URL from the provided QR
code safe? Yes, or no? To make this decision a specific algorithm type can be applied named a
classification algorithm. Classification algorithms excel in problems where the prediction must
be categorised (MonkeyLearn, n.d.), for example category 1: Good, category 2: Bad. There are
several viable classification algorithms used today, these have been detailed below:
SID:XXXXXXX MOD002691
19
Algorithm Description
Support Vector Machine (SVM) Batta Mahesh notes, SVM is a widely used technique. SVM can
perform non linier classification by utilising the kernel trick, which
allows for minimization of classification errors (Mahesh, 2020).
Naรฏve Bayes (NB) Batta Mahesh notes, NB is a classification algorithm that is based on
Bayes Theorem, NB assumes that features are independent to other
features when computing (Mahesh, 2020).
Decision Tree (DT) Batta Mahesh notes, DT represents choices in a tree form, the tree
has decision nodes which lead to branches, this makes predictions in
a conditional manner (Mahesh, 2020).
Random Forest (RM) IBM notes, RM is a common algorithm that combines multiple DT
output to compute its prediction (IBM, n.d.).
Logistic Regression (LG) IBM notes, LG works by estimating the likelihood of an event
occurring, the prediction is found between binary values 0 and 1, this
is useful for classification problems where the result tends to be yes
or no.
Table 3: Classification algorithms for model testing
SID:XXXXXXX MOD002691
20
A classification algorithm can use provided data to intelligently make a prediction of โ€˜yesโ€™ or โ€˜noโ€™
on a provided value and have previously been very effective when used in the security domain to
detect malicious values (Scispace, n.d.). in relation to this study, datasets containing known โ€˜safeโ€™
and โ€˜maliciousโ€™ URLs will be used by an algorithm to predict if a provided URL is โ€˜safeโ€™ or
โ€˜maliciousโ€™ As a URL can only be defined as โ€˜safeโ€™ or โ€˜maliciousโ€™ for the scope of this study, a
classification algorithm is essential for the accuracy of the ML model predictions.
However, to allow the algorithm to determine its prediction from the data, natural language
processing (NLP) must first be applied which allows the algorithm to understand context within
the data. This is done by encoding the human readable strings into numerical form which the
algorithm can understand. This process is known as vectorization (Jha, 2023).
Machine learning is greatly suited to this type of project as it can make predictions instead of
searching for a matching value within a dataset. Meaning when a user provides a QR code to the
system, the machine learning model can intelligently make a prediction on that URL. This is
significantly more effective at stopping interactions with a โ€˜maliciousโ€™ URL as a traditional
database search method would have no data to provide a result if the scanned malicious URL has
not previously been identified, new malicious URLs are created constantly so archaic techniques
such as this are not effective in todayโ€™s cyber landscape. ML models donโ€™t need to match a value,
instead it decides upon a probability of a provided URL being โ€˜maliciousโ€™ or โ€˜safeโ€™ and returns
the prediction.
However, there is a problem concerning this type of implementation of machine learning which
is how accurate the prediction is. To ensure the predictions are of a high accuracy, a model must
be trained on data until it is providing a satisfactory level of accuracy. A model being the
programme that can recognise the patterns within the data to make a prediction (Microsoft, 2023).
This is why ensuring a ML model has a high volume of quality data is essential to the ML process.
SID:XXXXXXX MOD002691
21
4.0 Literature Review
This literature review will consist of the analysis and review of several published academic
studies which closely align with the proposed concept of my system. I will identify what the
papers were intended for, and both the strengths and weaknesses of their proposed solutions. In
addition, I will conduct a critical analysis upon the literature, to detail what it has overlooked with
regards to their solutions. This will help me identify a problem statement for my system.
Secure QR Code Scanner to Detect Malicious URL using Machine Learning
This paper formulated by Pawar, et al, created a system which used machine learning to identify
malicious URLs derived from QR codes. multiple classification algorithms have been tested
against an ML model to determine what algorithm produces the highest accuracy at detecting
malicious URLs derived from QR codes. Each applied algorithm was explained in detail and the
results of each were recorded. The highest accuracy was 83.79% from a Bidirectional Long Short-
Term Memory (BI-LSTM) algorithm which is a type of recurrent neural network (RNN) which
can process the provided data in both a forward and backwards direction (Anishnama, 2023).
Other tested algorithms were Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and
Random Forest (RF). which all resulted in accuracy between 55% and 65%. as can be seen in the
below figure:
The ML model used three feature groups to achieve the resulting accuracy. The first feature group
was lexical, these include word length, frequency, and language style (Liu, 2022). The next
Figure 5: Accuracy Comparison (Adapted from Pawar et al, 2022)
SID:XXXXXXX MOD002691
22
feature group was Host-based which derives information from the webpage content, and the final
being correlated which is the total value of values such as URL length. The dataset used for the
training was comprised of a few large datasets however the specific value of URLs is unspecified.
However, it can be gathered that the size was sufficient (Pawar, et al, 2022).
This study's strengths lie in the significant background information regarding both the study
concepts and each of the applied algorithms. In addition, the application of each algorithm has
been detailed with evidence to support the proclaimed accuracies. However, there are weaknesses
identified within the study. Firstly, although the model accuracies are of an acceptable percentage,
the model accuracy could be greatly improved. In addition, only four algorithms were applied
within this study. Another significant improvement would be to apply more algorithms to ensure
that the best accuracy could be identified.
Detecting Malicious URLs Using Machine Learning Techniques: Review and
Research Directions
This paper formulated by Aljabri et al, conducts extensive research on preexisting literature
concerning the detection of malicious URLs with ML. In addition to English language URLs, this
paper conducts further analysis on the accuracy of ML algorithms specifically against Arabic
language URLs. From the 47 papers research it was discovered that the most used machine
learning algorithm to detect malicious URLs were either SVM or RF classifier algorithms. In
addition, the least used algorithm was Deep Belief Networks (DBN). Due to the range of sources
used for this study, the datasets ranged, however, the most common dataset used were PhishTank
and Alexa (Aljabri et al, 2017). PhishTank notes, that the PhishTank dataset is comprised of
known Phishing websites (PhishTank, n.d.). Papers with code notes, Alexa Domains dataset is
comprised of the most common benign URLs (Paperswithcode, n.d.).
This paper did extensive testing to determine the most common and effective ML classifier
algorithm for detecting malicious URLs. However, the paper did no primary testing of the
algorithms on a model. resulting in all the statistics being drawn straight from other literature.
SID:XXXXXXX MOD002691
23
This being a weakness of the paper as replication of the model accuracy would suggest more
legitimacy of the statistics presented.
Malicious URL Detection: A Comparative Study
This paper by Shantanu, et al consists of the creation and testing of an ML model that predicts if
a provided URL is malicious. The paper covers both the background information relating to the
used concepts and the implementation of the applied algorithms in detail. The model was applied
with 7 different classification algorithms which were, Logistic Regression (LG), KKN, Naive
Bayes (NB), Decision Tree (DT), RF, SVM and Stochastic Gradient descent (SGD). The highest
accuracy algorithm was RF with a 92.6% accuracy when applied with the OpenPhish dataset. The
paper supported these findings with evidence for each model implementation and detailed
information regarding the dataset used which had a total value of 450,000 URLs both malicious
and benign (Shantanu, et al, 2021).
This study's testing of seven different classification algorithms is a significantly strong point of
the report, the extensive testing allowed the researchers to determine the best accuracy model and
therefore get the best result for the final model. In addition to this, each model and algorithm has
been detailed extensively with visual evidence of the implementation. Another positive aspect of
this study was that a dataset of adequate size was used which ensures the models foster the best
results possible. However, the study's weaknesses are that only the model applied with the RF
algorithm has had the model accuracy detailed. There is no detail on the other six applied
algorithms to gather an understanding of how well they performed. This is in addition to the
accuracy of the model which could be greatly improved.
SID:XXXXXXX MOD002691
24
QsecR: Secure QR Code Scanner According to a Novel Malicious URL Detection
Framework
This paper formulated by Rafsanjani et al, presents an Android application named QsecR which
is a QR code scanner designed to stop the interaction with malicious QR codes. The application
relies on a ML model that was tested with multiple classifier algorithm consisting of NB, SVM,
LR, KNN, and DT. The model used these classification algorithms with a range of feature groups
consisting of lexical, host based, content based and blacklist which checks to see if a provided
URL is known to be malicious, the final model implementation produced an accuracy of 93.80%
using a data set of 4000 URLs combined from PhishTank and Google Safe Browsing.
The report went on to compare the accuracy of the model to other known QR code scanners and
demonstrated that the accuracy was superior to the other tested scanner such as Gamma-Play,
InShot-Inc and Trend-Micro scanners, As seen in the figure below when presented with known
malicious QR codes QsecR preformed significantly better (Rafsanjani et al, 2023).
This report produced a sufficient detection system and covered the research and implementation
in detail. In addition, the GUI portion of the application again was implemented well granting a
high-level user experience. However, the ML model accuracy could have been improved and
additional approaches to the QR detection could have been included. For instance, additional
programming functions to validate if the URL is โ€˜safeโ€™, such as validating the URLโ€™s PKI
certificate.
Figure 6: Evaluating security performance of QR code scanners (Adapted from Rafsanjani et al, 2023)
SID:XXXXXXX MOD002691
25
Classification of Malicious URLs Using Machine Learning
This study by Abad et al, evaluates the effectiveness of using ML to identify malicious URLs
when the model is applied with different instance selection techniques, which were random
selection, DRLSH, and BPLSH. Random selection helps make the training process of the model
faster by selecting a subset of the data for training. Data Reduction based on Locality-Sensitive
Hashing (DRLSH) and Border Point Extraction based on Locality-Sensitive Hashing (BPLSH)
are also used to increase the efficiency of the model.
The study tested four different classification algorithms against the model with RF fostering the
highest accuracy of 92.18% The study detailed the background information, relevant algorithms,
and methodology extensively which allows the reader to gain a holistic understanding of the study
and its findings (Abad et al, 2023).
The obvious strength of this study is the computational effectiveness that is fostered by the
application of random selection, DRLSH and BPLSH which resulted in the model training for
RF being between 71 and 82 seconds. This allows the model to have significant efficiency in
training and prediction.
However, there are identified weaknesses in the study. Firstly, the highest accuracy achieved was
92.18%, ideally, this accuracy should be improved to ensure a more accurate and reliable model.
In addition, there was no testing done without the applied instance selection, therefore the
comparison in training time cannot be quantified by the reader which due to the nature of the
study is an important data point to detail.
SID:XXXXXXX MOD002691
26
Malicious URL Detection and Identification
This paper formulated by Sayamber A., and Dixit A., created a method to detect malicious URLs
via a machine learning model which used the NB classifier algorithm. Upon testing it was found
to have a higher accuracy than when the model used the SVM algorithm. The model used the
following features to assist in the prediction: Lexical, Link popularity, webpage content, and DNS
features. The dataset was comprised of several dataset sources, including datasets such as
PhishTank and Yahoo!โ€™s directory (Sayamber A., and Dixit A., 2014).
The model used within this paper has significant use of features that increase the integrity within
the modelโ€™s prediction, in addition, the study explains clearly to the reader how the model
classifies data using multiple flow charts and diagrams.
The primary downfall of this paper is the lack of detail of the accuracy of the model. The report
fails to detail exactly what accuracy was produced from the model and what errors regarding
False positives were produced. The testing was restricted to only two classification algorithms
which additional testing of other classification algorithms may have found the model to be more
accurate. Lastly, the detection method of the resource focuses on only a ML model and no
external methods of detection.
Malicious URL Detection based on Machine Learning
This paper formulated by Xuan et al, produced a machine learning model to predict if a URL is
malicious or benign. This model used three feature groups to increase the accuracy of the model.
These three being lexical, Host-based, and correlated. The model uses two algorithms which are
the SVM and RF classifier algorithms. The dataset used for training consists of a total 470,000
URLs, 70,000 or 14.89% of which are known malicious URLs, the other 400,000 or 85.11%
being benign URLs. As seen in the figure below, the RF algorithm had the best accuracy of 96%
over 100 iterations, the SVM algorithm having a 90% accuracy over 100 iterations (Xuan et al,
2020).
SID:XXXXXXX MOD002691
27
This paper conducted significant testing on the ML model used. In addition. The feature groups
used within the model were comprehensive in their respected features. The oversights of this
study are that not many classification algorithms were tested to identify the most accurate
algorithm for the model. This implementation could have improved the accuracy.
QR Code Security โ€“ How Secure and Usable Apps Can Protect Users Against
Malicious QR Codes
This paper formulated by Krombholz et al, consists of a comprehensive look at QR codes and
how they can be used as an attack vector by threat actors. This paper tackles the problem in a
holistic view, considering both ML and externals security validation techniques. The paper
suggests the implementation of Digital signatures to ensure the integrity of the QR codes and
applying pre display analysis to analyse the full URL in the case a URL shortener has been applied
to presented URLs (Krombholz et al, 2013).
This paper outlines the threat of malicious QR codes extremely well, supported by primary
research of demographic likelihood of malicious QR code interaction, and secondary research
indicating to lack of secure QR code scanners. This literature also describes innovative techniques
to provide security, such as modifying the QR code to allow detection of errors with a technique
called masking. Although this paper presented some very innovative ideas on how to secure QR
code scanners, no implementation for the ideas was attempted which would have demonstrated
if the proposed ideas were viable solutions.
Figure 7: Testing results (Adapted from Xuan et al, 2020)
SID:XXXXXXX MOD002691
28
Secure Real-Time Artificial Intelligence System against Malicious QR Code Links
This paper formulated by Al-Zahrani et al, implemented a ML model to detect malicious QR
codes. The model itself was tested with a range of algorithms consisting of NB, SVM, LR, KNN
and DT where it was discovered that DT had the best accuracy rating. The model was trained of
a dataset of 100000 malicious and benign URLs and used one feature group consisting of lexical
properties. The research produced an application named BarAI which had a final accuracy of
90.243%. In addition to the implementation, the report detailed many types of attack vectors used
within QR codes, such as detailing how threat actors can use a โ€˜barcode-in-barcode attackโ€™ to get
victims to interact with malicious URLs (Al-Zahrani et al, 2021).
The literature researched the related concepts of QR code security well and conducted a
significant amount of testing on different classification algorithms against the model to determine
the best to use. In addition, the data was derived from relevant and recent sources increasing the
accuracy of the model in current times. However, the final accuracy of the ML model could have
been improved to foster a more reliable system. In addition, the dataset used for training was
relatively small in comparison, this could have potentially hindered the accuracy of the final
model.
Secure Real-Time Computational Intelligence System Against Malicious QR Code
Links
This paper formulated by Heider Wahsheh and Mohammed Al-Zahrani, consisted of the
implementation of ML using a multilayer perception artificial neural network (MLP-ANN)
algorithm. In addition, fuzzing logic was applied in attempts to detect malicious URLs derived
from QR codes. The model used a dataset of 90,000 benign and malicious URLs. The model
produced a real-time detection accuracy of 82.9%. real-time in the sense of this ML model means
the model is using live data instead of offline historic data. The model used a feature group of
lexical properties. The dataset had equal halves of malicious and benign URLs each being 45,000
URLs (Wahsheh, H., and Al-Zahrani, M, 2021).
SID:XXXXXXX MOD002691
29
The literature strengths lie within its testing of the programme. The programme was tested against
known scanners such as Kaspersky and Norton to see how its security features compared. In
addition, its approach to ML was decidedly unique in that it opted to use a real-time artificial
intelligence approach instead of a traditional batch model approach. The primary downfall of the
implementation was the amount of data. A dataset of 90,000 is relatively small for this type of
classification problem and a larger dataset may have produced a higher model accuracy and
model integrity.
4.1 Critical Analysis
The above literature review was composed of analysing several academic papers which closely
follow the concept of my proposed project. The covered papers range in their detail and
comprehensiveness. However, All the above sources decided that a critical part of detecting
malicious URLs derived from QR codes was a machine learning model. Higher accuracy
percentages mostly were dependent on the size of the used dataset, and the testing of multiple
algorithms.
The primary oversights from most of the papers were the depth of testing conducted. Many papers
when determining the algorithm to use, only tested a few algorithms. This is something I intend
to remediate when training my model, as testing a range of algorithms will discover which
algorithm produces the best accuracy, and therefore making my ML model more effective and
capable of achieving its required goal.
Secondly, many of the models used had insufficient sized datasets with little detail on the cleaning
and preparation of the data. Again, this is something I intent to remediate by using a sufficient
sized dataset and ensuring that the data is of good quality, this will ensure my model achieves the
best accuracy it is capable of.
Moreover, a significant oversight for most papers was the lack of additional validation of the
URL outside of the ML model. For example, no online validations such as ensuring a URL has a
SID:XXXXXXX MOD002691
30
valid certificate were present, in addition, none of the models implemented additional functions
to ensure that a valid protocol was being used for the presented URL such as HTTPS. This is a
feature I intent to implement into my system.
From this analysis it can be observed that there is significant oversight within the observed
literature, I intend to implement the discussed solutions by taking a hybrid approach to the
problem. This will use ML as much of the literature used, however, ML alone is not enough to
identify malicious QR codes, this is because ML models can be wrong in their predictions, so
additional methods should be used in tandem to ensure the integrity of a prediction, to do this,
online URL validation will be implemented within my system. These being, PKI certificate
validation, and URL format validation, these solutions specifically are important as they ensure
real time security validation such as if the URL is using secured protocols such as HTTPS and
have a valid certificate for session security and integrity.
SID:XXXXXXX MOD002691
31
5.0 Proposed Work
For the proposed solution to be created, the three main components must be designed to
effectively achieve there aims. For this to be achieved, a ML model must be fostered that can
detect malicious URLs. In addition, a function to identify if a URL has a valid PKI certificate
must be created. And lastly, a function to validate a URLs format must be created. These
components then need to be implemented into a hybrid system that can be used by an end user.
5.1 Methodology
This section has been formulated to detail the methodology of the proposed system and detail
all the stages related to the implementation. The system serves the function of detecting
malicious content within QR codes. The architecture of the system function can be seen in the
below figure.
SID:XXXXXXX MOD002691
32
Figure 8: Proposed architecture of system function.
SID:XXXXXXX MOD002691
33
This methodology will detail how the sections of the system architecture will provide the
desired outcomes. The following steps have been adopted in my approach:
1. Machine Learning model to detect malicious URLs.
1.1 Collection of data
1.2 Cleaning and preparation of dataset
1.3 Feature engineering
1.4 Test classifier algorithms against model to determine the most appropriate
algorithm.
2. Validating if URL has a valid PKI certificate.
3. Validating if URL format is valid.
4. Creation of QR code reader and system GUI
5.2 Machine Learning model to detect malicious URLs.
The first step of the implementation will be the programming of the ML model from which a
prediction will be derived. The ML implementation will follow the below steps.
5.2.1 Collection of data
The first step is to gather the data that the ML model will use to train. I discovered a dataset on
Kaggle that was aligned with the requirements for my ML model, this dataset named Url
Dataset (Teseract, 2017), consisted of 420,464 URLs, either assigned a value of good or bad
which correspond to benign or malicious. Eugene Dorfman notes, a ML model should apply the
10-time rule to have a sufficient dataset (Dorfman, 2022). Meaning, the dataset should have 10
time the amount of input data as there are parameters within the dataset. As this dataset only has
two parameters being the URL and the assigned URL state value, the 10-time rule would
SID:XXXXXXX MOD002691
34
require an input set of 20 entries, this dataset far exceeds the minimum requirement, thus giving
it ample data to produce accurate predictions. The figure below shows some example data from
the dataset.
344,821 or 82.01% of the dataset URLs were assigned the value of good. With the remaining
75,643 or 17.99% being assigned the value of bad. As seen in the below figure.
Figure 9: URL Dataset entries.
Figure 10: ratio of good and bad URLs in dataset.
SID:XXXXXXX MOD002691
35
5.2.2 Cleaning and preparation of dataset
After acquiring the dataset, the next stage is too โ€˜Cleanโ€™ the dataset. Kirsten Barkevd notes,
cleaning data is the process of modifying or removing data that is incorrect or not relevant to
the dataset, not cleaning data can negatively impact the accuracy of a ML model (Barkevd,
2022). Upon analysis of the dataset, it was observed that no cleaning was needed. The below
figure show that the dataset had no NULL values, meaning all URLs had either good or bad
assigned, if a value had a NULL value, True would be displayed.
The next step is preparing the data by splitting the data set into a training set and a testing set.
Javatpoint notes, that splitting the data set into testing and training is an essential element of
data preparation, the training set is used to train the model and then the test set is used as test
data when testing. It is important that the datasets are kept separate as testing a model on the
training set will provide inaccurate results as the model is aware of the data pretesting. A
common split for the dataset is 80:20 where 20 is the testing set, this is due to the model
benefiting from a larger training set as it allows more data for computations, and the testing set
can be smaller due to it being a subset of the original dataset for testing. (Javatpoint, n.d.). For
my model, I will follow the 80:20 split for the dataset as represented in the figure below.
Figure 11: Observing no NULL values in dataset.
SID:XXXXXXX MOD002691
36
Figure 12: Figure to show testing and training set split.
5.2.3 Feature Engineering
Feature Engineering is an essential part of the ML process as it allows the algorithm to work
efficiently with the dataset and enhance the performance of the model (Rosencrance, n.d.). For
the proposed ML model, the feature engineering consists of vectorising the dataset for NLP.
This allows the model to identify how important specific words are within a URL (Karbhari,
2019).
Dremio notes, NLP is the process of converting natural language, such as sentences into
numerical data that the ML model can use for analysis (Dremio, n.d.). The specific technique
that will be used is Term frequency โ€“ inverse document frequency (TF-IDF). Fatih Karabiber
notes, TF-IDF measures the importance of a natural language string. This will be used to
identify malicious or benign indicators within a URL. This happens by multiplying a natural
language words Term frequency (TF) with the inverse Document Frequency (IDF). TF is equal
to the count of times a term is within the data/document, divided by the total number of
data/document words. And IDF is used to discover the importance of a word by identifying the
number of documents commonly thought of as a โ€˜bag of wordsโ€™ in the larger set of data known
as a corpus and dividing this over the total number of documents within the corpus containing
Dataset
Training Set Testing Set
SID:XXXXXXX MOD002691
37
the word (Karabiber, n.d.). The formular Adapted from Fatih Karabiber can be seen below
(Karabiber, n.d.).
๐‘‡๐น ๐ผ๐ท๐น = ๐‘‡๐น โˆ— ๐ผ๐ท๐น
๐‘‡๐น =
๐‘๐‘ข๐‘š๐‘๐‘’๐‘Ÿ ๐‘œ๐‘“ ๐‘ก๐‘–๐‘š๐‘’๐‘  ๐‘Ž ๐‘ก๐‘’๐‘Ÿ๐‘š ๐‘Ž๐‘๐‘๐‘’๐‘Ž๐‘Ÿ๐‘  ๐‘–๐‘› ๐‘‘๐‘Ž๐‘ก๐‘Ž
๐‘‡๐‘œ๐‘ก๐‘Ž๐‘™ ๐‘›๐‘ข๐‘š๐‘๐‘’๐‘Ÿ ๐‘œ๐‘“ ๐‘ก๐‘’๐‘Ÿ๐‘š๐‘  ๐‘–๐‘› ๐‘‘๐‘Ž๐‘ก๐‘Ž
๐ผ๐ท๐น = log (
๐‘๐‘ข๐‘š๐‘๐‘’๐‘Ÿ ๐‘œ๐‘“ ๐‘กโ„Ž๐‘’ ๐‘‘๐‘œ๐‘๐‘ข๐‘š๐‘’๐‘›๐‘ก๐‘  ๐‘–๐‘› ๐‘กโ„Ž๐‘’ ๐‘๐‘œ๐‘Ÿ๐‘๐‘ข๐‘ 
๐‘๐‘ข๐‘š๐‘๐‘’๐‘Ÿ ๐‘œ๐‘“ ๐‘‘๐‘œ๐‘๐‘ข๐‘š๐‘’๐‘›๐‘ก๐‘  ๐‘ค๐‘–๐‘กโ„Ž๐‘–๐‘› ๐‘กโ„Ž๐‘’ ๐‘๐‘œ๐‘Ÿ๐‘๐‘ข๐‘  ๐‘๐‘œ๐‘›๐‘ก๐‘Ž๐‘–๐‘›๐‘–๐‘›๐‘” ๐‘กโ„Ž๐‘’ ๐‘ก๐‘’๐‘Ÿ๐‘š
)
5.2.4 Test classifier algorithms against model to determine the most appropriate
algorithm.
The next stage will be testing different classification algorithms to train the model and
determine which of the tested algorithms produce the best accuracy. This is to determine which
classification algorithm is most appropriate for my modelโ€™s problem. The algorithms that will
be tested will be SVM, NB, DT, RM, and LG which have all been detailed in the background
information.
SID:XXXXXXX MOD002691
38
5.3 Validating if URL has a valid PKI certificate.
The second stage of the implementation will be the security validation function that will
determine if a URL has a valid certificate. This will be achieved by creating a function that
takes advantage of the Python library Requests. A function will be created which sends a HTTP
request to the provided URL, the Requests import then determines if the URL has a valid
certificate, if it is a response of 200 will be returned. Umbraco notes, a returning of 200 equal
the status code โ€˜OKโ€™, meaning the request was successful (Umbraco, n.d.). The function will
only return 200 if it validated a certificate, if not the function will return a SSLError (Pypi,
n.d.). The below figure illustrates the functionality of the programme.
Figure 13: Illustration of certificate validation.
SID:XXXXXXX MOD002691
39
5.4 Validating if URL format is valid.
The third stage of the implementation will be the second security validation function. The
purpose of this function is to validate that the URL is formatted correctly. This utilizes the
Python library validators. This import searches a provided URL for specific parameters to
ensure it is valid. Utilizing validators python library to determine if URL components are
properly formatted. Afzaal Ahmad Zeeshan notes, validators achieve this by ensuring that the
URL has a valid protocol such as HTTP or HTTPS and has a resource associated with the
address. This is in accordance with RFC 1738 (Zeeshan, 2022). An adapted section of the
validators.url source code can be seen below and detailed in the below table:
Figure 14: Validators source code (Adapted from validators, n.d.)
SID:XXXXXXX MOD002691
40
Section of Source Code Description
# protocol identifier As seen in the top section of the code, the code identifies if the URL is using
a valid protocol such as HTTPS or File transfer Protocol (FTP) which is a
host-to-host file transferring protocol (Fortinet, n.d.).
# IP address exclusion Below this we can see that the code is checking the URL is not resolving to
a private address from the classes A (10.0.0.0 โ€“ 10.255.255.255), B
(172.16.0.0 โ€“ 172.31.255.255) or C (192.168.0.0 โ€“ 192.168.255.255) (Avast,
n.d.). and is within the public address space.
# Resource path The final action of code seen ensures that the URL has a valid resource that
the user is navigated to.
Table 4: Details on validator source code
This will be used within a function to determine if a URL is valid or invalid, an illustration on
how the function will determine this can be seen in the below figure.
SID:XXXXXXX MOD002691
41
5.5 Creation of QR code reader and system GUI
The last step of the implementation will consist of the user interface and scanner. The
methodology applied to build the QR code scanner will be to adopt the Kivy library and take
advantage of its features which allow interaction with the device camera. From this the input
can be decoded, and a derived URL can be found. For the GUI, Kivy will again be adopted for
its cross-platform capabilities allowing it to be used on any device, The system will follow a
simple design to increase usability and efficiency of the system. Below can be seen a navigation
map to which the system user interface (UI) will follow.
Figure 15: Illustration of URL validation.
SID:XXXXXXX MOD002691
42
For a visual representation of what the final system GUI will look like, the below wireframe
diagram can be seen.
Figure 16: Navigation map of system.
Figure 17: Wireframe diagram of GUI (iPhone template adapted from
unblast, n.d.)
SID:XXXXXXX MOD002691
43
6.0 Implementation and Results
Development environment
To programme the proposed system, an integrated development environment (IDE) will be used
to aid in the development process, the development environment of choice is Visual Studio Code.
Microsoft notes, that Visual Studio Code is a powerful and comprehensive development
environment (Microsoft, 2023). The reason I have selected Visual Studio Code for this project is
due to my personal familiarity with the software.
In addition to using Visual Studio Code. JupyterLab will be used to aid in the development of the
machine learning code. Jupyter notes, jupyter notebook allows for configuration and arranging
of workflows in data science (jupyter, n.d.). Meaning, jupyter notebook can be used to test and
configure the developed machine learning code in a dedicated environment.
Python
For the programming language used to build this system, Python was selected. Python is a high-
level programming language that is extremely versatile in its functionality. Python can be used in
multiple cyber security related domains, ranging from malware analysis to penetration testing
(CyberWarrior, 2023). Due to this it is a highly sought after skill in cyber security professionals.
Forbes notes, Python as the number one in demand programming language of 2023 (Forbes,
2023). Due to the high demand in Python programming ability, I decided that the Python language
would be a suitable language to create the system with. Not only will using Python increase my
ability within the language. Buit in addition, the vast array of Python imports and library allow
additional functionality to the system such as the ability to build cross platform GUIs. This is in
addition to the range of cyber security and network security imports that will assist in building
this system.
SID:XXXXXXX MOD002691
44
Python Libraries
Python allows users to import Python libraries. According to docs.python.org, libraries:
โ€œProvide standardized solutions for many problems that occur in everyday programming.โ€
(docs.python.org, n.d.)
From this it can be understood that Python Libraries are predefined useful functions that
mitigate the need to rewrite commonly used code.
Within the development of my system, a range of libraries will be imported to assist in the
development of the code. The most important ones to the development are listed in the below
table:
SID:XXXXXXX MOD002691
45
Library Description
Sklearn Scikit-Learn.org notes, that sklearn is a python library which allows users to build machine
learning programmes with Python (Scikit Learn, n.d.). Sklearn will be used for the
development of the projects machine learning programme to predict malicious URLs.
Kivy Kivy notes, that the Kivy python library allows for the development of cross platform
applications programmed in Python (Kivy, n.d.). Kivy is essential for the development of
my system as it allows cross platform functionality and GUI creation.
Validators Read the Docs notes, that the validator collection is a Python library that can be used to
validate the type and contents from a provided input value (Read the Docs, n.d.). I will be
using the validators library within my programme to ensure that a provided URL derived
from a provided QR code is correctly formatted.
Requests Pypi.org notes, that the requests Python library is used to send HTTP requests (pypi.org,
n.d.). I will be using the request library to send a HTTP request to a URL derived from a
provided QR code. I will use the provided response to determine of the URL has a valid
PKI Certificate.
Table 5: Utilized python libraries.
SID:XXXXXXX MOD002691
46
6.1 Implementation of ML model to detect malicious URLs.
The first stage of the ML section of the programme was importing the necessary libraries. The
libraries utilised mainly consisted of Sklearn derivatives, consisting of all the algorithms that
were tested and imports that allow the model to be constructed and trained. In addition, other
imports such as pandas, matplotlib and numpy were used for data manipulation and
visualization, the imports can be seen in the below figure.
After importing all necessary libraries, the next step was to access the โ€˜urldataโ€™ dataset
explained in the methodology, this was accessed via a panda function as seen in the figure
below.
Figure 18: Imported libraries for ML model.
Figure 19: โ€˜urldataโ€™ dataset manipulation.
SID:XXXXXXX MOD002691
47
Upon completion of the data cleaning, the data next needed to be prepared for training. This
was done by splitting the dataset into an input and output set. The input set consisting of the
โ€˜urlโ€™ values and the output set consisting of the โ€˜labelโ€™ values containing either good or bad. The
sets are named in this way as the input set is the feature we wish to predict and the output set
contains the outcomes of an input value (Spark code hub, n.d.). y is used to denote the output
set and X is used for the input set; however, the input set must be vectorized so the input set is
stored in the variable โ€˜urlsโ€™ The data splitting can be seen in the below figure.
After splitting the dataset into the input and output set, the data must be vectorised for feature
engineering via NLP as it is in string format. To do this we apply TF-IDF vectorization to the
data as explained in the methodology, this allows our data to be computed. Once the
tfidfVectorizer() function has been implemented, this can be applied to the input set as seen
below.
Now the dataset has been prepared and NLP has been applied, the next step is to split the input
and output set in to testing and training set, as explained in the methodology, this is to ensure
that the model can be trained to a high accuracy with good integrity. As seen in the below
figure, the input and output sets have both been split into testing and training sets via the
train_test_split() function, with the testing sets being 20% of the dataset and the training set
having 80%, the raindom_state has been applied to ensure that the data is randomised and
doesnโ€™t produce false accuracy from a class imbalance problem (Pramoditha, 2022).
Figure 20: splitting dataset into input and output set.
Figure 21: Vectorizing data with TF-IDF
SID:XXXXXXX MOD002691
48
At this stage the data has been cleaned and prepared and is now ready to be applied to a model
for training. The first model used the Naรฏve Bayes algorithm. The model was first defined, and
the algorithm was applied, next the fit method was applied to train the model with the training
datasets. Once the model had been trained, the model predictions via the predict function from
the input set were stored in the y_pred variable. Once complete, the classification_report
function was used with the testing output set and the y_pred set to test the modelโ€™s accuracy.
This function tests the modelโ€™s accuracy on a range of variables to determine an accuracy
rating. This applied method was used for all the algorithms and resulting in the following
accuracy ratings seen in the below figure.
Figure 22: Splitting data for testing and training.
Figure 23: NB, RF, SVM, LR and DT model reports.
SID:XXXXXXX MOD002691
49
The below table summarises the accuracy of the different tested algorithms.
Classification Algorithm Accuracy Percentage
SVM 98%
DT 97%
LR 96%
NB 95%
RF 82%
Table 6: Accuracy of algorithms summary
From the testing conducted it was discovered that the highest accuracy was produced by the
model applied with the SVM algorithm. However, the algorithm ultimately chosen for the
programme was LR with 96% accuracy. This was due to the following factors, SVM while
producing a very high accuracy, took a substantial amount of time to predict, which would not
be efficient and would discourage interaction with the system, DT while again had high
accuracy, had a lower precision than LR when determining โ€˜badโ€™ URLs. Which is significant as
this system needs to be as risk averse as possible when predicting malicious URLs. Due to the
stated reasons the model utilising LR has been used for the final model implementation which
had the highest true positive (TP) accuracy at identifying malicious URLs of the top three
algorithms, this is illustrated in the below confusion matrix.
SID:XXXXXXX MOD002691
50
Once the final model was implemented a function was defined to allow a URL to be passed as
an argument, the URL is then vectorised for NLP and predicted against the model. The model
would then return a value for the output set being โ€˜goodโ€™ or โ€˜badโ€™. Once the value was returned
an โ€˜IFโ€™ statement would return either โ€˜Clearโ€™ or โ€˜Maliciousโ€™ from the function depending on the
prediction. This function can be seen in the below figure.
Figure 24: Confusion matrix for LR.
Figure 25: ML model prediction function.
SID:XXXXXXX MOD002691
51
6.1.1 Testing model predictions against known malicious and safe URLs.
Open Phish is a website that collect known malicious URLs. (OpenPhish, n.d.) ten of these
URL have been predicted by my ML model. As can be seen, from the provided known bad
URLs the model identified all of them correctly, However, with the known good URLs, the
model identified one of them incorrectly, giving this test a 95% accuracy.
Disclaimer: The malicious URLs presented in the below table should only be accessed in a safe
environment. I the author of this report hold no responsibility for the damages caused by a
reader accessing the detailed URLs.
SID:XXXXXXX MOD002691
52
URL Prediction Type Correct /
Incorrect
https://clainmask.com Malicious Malicious Correct
https://login-dana-id.giixzg.me/ Malicious Malicious Correct
http://havkeye14.wixsite.com/my-site-1/ Malicious Malicious Correct
https://dovzzt.n0c.world/bpost2/ Malicious Malicious Correct
https://cg96358.tw1.ru/?return_url=https://www.orange.fr/portail&_Auth Malicious Malicious Correct
https://accshmesrvc0log.github.io/ Malicious Malicious Correct
https://one.link/annushka_almazova Malicious Malicious Correct
https://dev-kejomokkeflushah.pantheonsite.io/att/att.html Malicious Malicious Correct
http://havkeye14.wixsite.com/my-site-1/ Malicious Malicious Correct
https://innovative-divya.github.io/Netflix-clone Malicious Malicious Correct
https://en.m.wikipedia.org Clear Clear Correct
https://stackoverflow.com/questions/58356254/machine-learning Clear Clear Correct
https://www.loveholidays.com/sem/cheap.html?WT.mc_id=pgo-35492155817 Malicious Safe Incorrect
https://www.autotrader.co.uk/car-search?postcode=i77777&price-from=10000&price-
to=15000&make=Mercedes-Benz&advertising-location=at_cars&page=1
Clear Clear Correct
https://teams.microsoft.com Clear Clear Correct
https://sts.anglia.ac.uk/adfs/ls/idpinitiatedsignon.aspx?SAMLRequest Clear Clear Correct
https://500px.com/photo/49283436/chicago-looking-up-by-alex-dibrova Clear Clear Correct
https://kids.nationalgeographic.com/animals Clear Clear Correct
https://id.cisco.com/ Clear Clear Correct
https://twitter.com/?lang=en Clear Clear Correct
Table 7: ML model predictions of provided URLs.
SID:XXXXXXX MOD002691
53
6.2 Implementation of URL PKI certificate validation.
The first step in the implementation of the URL PKI certificate validation function was to
import the relevant libraries. As detailed in the methodology the requests import will be used to
send a HTTP GET request to a provided URL. The requests import can be seen in the below
figure.
Once the relevant library was imported a function was defined to pass a URL as an argument.
This function first defined an empty variable. This was to allow the variable to be accessed
outside of the try except scope which was implemented after a persistent connection error was
found. Implementing the try except allowed the function to complete as needed. In the try
clause the response variable uses the requests.get() function to determine if the URL has a valid
certificate. The function will only return OK or <Response [200]> if a valid certificate is
present, if not it will return an SSLError. The except method was to prevent the connection
error from stopping the function and then passes the response variable to the next section. Here
the variable is turned into a string and an โ€˜IFโ€™ statement determine what the response is. If the
string is exactly equal to <Response [200]> the function will return โ€˜Clearโ€™ as a valid certificate
is present, if an error is returned, the function will return โ€˜Invalidโ€™ as no certificate was found.
Figure 26: PKI Certificate validation function import.
SID:XXXXXXX MOD002691
54
6.2.1 Testing function against known valid and invalid certificates.
Badssl is a website that hosts invalid certificates for testing purposes (badssl, n.d.). This
function was tested against six known bad URLs and six known good URLs as seen in the
below figure.
Figure 27: URL PKI Certificate validation function.
Figure 28: Tested good and bad certificates.
SID:XXXXXXX MOD002691
55
The below table details the responses form testing the URLs. As can be observed, the function
has a 100% accuracy on the presented testbed.
URL Response Type Correct / Incorrect
Invalid 1 SSLError Expired Correct
Invalid 2 SSLError Wrong Host Correct
Invalid 3 SSLError Self-Signed Correct
Invalid 4 SSLError Untrusted Correct
Invalid 5 SSLError Revoked Correct
Invalid 6 SSLError Pinning-test Correct
Valid 1 <Response [200]> Valid Certificate Correct
Valid 2 <Response [200]> Valid Certificate Correct
Valid 3 <Response [200]> Valid Certificate Correct
Valid 4 <Response [200]> Valid Certificate Correct
Valid 5 <Response [200]> Valid Certificate Correct
Valid 6 <Response [200]> Valid Certificate Correct
Table 8: Testing PKI Certificate validation function.
6.3 Implementation of URL format validation
The first step in the implementation of the URL format validation function was to import the
necessary libraries. As detailed in the methodology, the validators import will be utilized within
a function to validate the format of a provided URL. The import can be seen in the below
figure.
Figure 29: URL format validation function import.
SID:XXXXXXX MOD002691
56
Upon importing of the library, a function was defined that takes a URL as an argument, the
validators.url() function which validates the URL is then applied to the passed URL and stored
in a variable, Within the variable is a Boolean value of either True or False, the variable output
is then changed into a string on which an โ€˜IFโ€™ statement is conducted to determine if the output
is True which means the URL format is valid. Or False, which means the URL format is
invalid. If the URL format is valid the function will return โ€˜Clearโ€™ else the function will return
โ€˜Invalidโ€™. The function can be seen in the below figure.
Figure 30: URL format validation function.
SID:XXXXXXX MOD002691
57
6.3.1 Testing function against known valid and invalid format URLs.
The below table details the responses form testing the URLs. As can be observed, the function
has a 100% accuracy on the presented testbed.
URL Format Result Correct /
Incorrect
https:/autocars.com Invalid Invalid Correct
https://www.google. Invalid Invalid Correct
httpb://github.com Invalid Invalid Correct
https://www.youtube/com Invalid Invalid Correct
http:||www.udemy.com Invalid Invalid Correct
https;//www.linkedin.com Invalid Invalid Correct
https://www.youtube.com Valid Valid Correct
https://www.amazon.co.uk Valid Valid Correct
https://www.gymshark.com Valid Valid Correct
https://whois.is Valid Valid Correct
https://teams.microsoft.com Valid Valid Correct
https://wordcounter.net Valid Valid Correct
Table 9: Testing URL format validation function
SID:XXXXXXX MOD002691
58
6.4 Implementation of the QR code scanner
The first stage of the QR code scanner implementation was to import the related libraries, as
defined in the methodology, the Kivy library has been used to allow this system to work cross
platform, in addition Kivy also allows access to a device camera through branches of the
library. To be able to effectively recognise and decode a provide QR code, the pyzbar library
has been imported also. Pypi notes, pyzbar allows reading of barcodes and QR codes (pypi,
n.d.). As can be seen below:
After the necessary imports have been implemented, the QR code scanner itself can be created.
Firstly, a template for the scanner is created by creating a new class which inherits the argument
App which allows it access to the Kivy library functionality. Once the class is defined, a new
function is created that uses the kivy builder function to load the output stored in the variable
Scanner. The scanner variable consists of a multi-line string that imports the needed libraries
and defines the layout for the camera window under MDBoxLayout. In addition the ZBarCam
object is also defined here which uses the id:qrcodecam to load the native device camera and
allows QR codes to be recognised. Below this the ZBarSymbol is used to define the types of
codes the scanner can recognise. Lastly an object that allows decoding of a QR code has been
defined which is calling a function defined below and is calling all the function arguments.
The function in question is below the builder function and firstly checks to make sure that a QR
code is present. If a QR code is present the function passes the output to the next function which
firstly defines a variable as global allowing global access, then stores the decoded data within
this variable using the decode() function. The variable is made global to allow access to the
decoded URL throughout the code. As can be seen in the below figure the class allows the
camera to be used to scan and decode QR codes.
Figure 31: QR code scanner imports.
SID:XXXXXXX MOD002691
59
Figure 32: QR code scanner working example.
SID:XXXXXXX MOD002691
60
For the purposes of the above figure, a QR code was generated with the URL
https://ExampleURL.com, as can be observed, the programme accessed the device camera and
printed the decoded data as output, proving that the scanner can identify and decode QR codes.
The main class for the QR code scanner with its related functions can be seen in the figure
below.
Figure 33: QR code scanner code.
SID:XXXXXXX MOD002691
61
6.5 Implementation of Graphical user interface
The first step of the GUI implementation was to import the necessary libraries. Kivy, as before
has been utilized significantly for its cross-platform GUI capabilities, a range of Kivy derivative
have been used such as Gridlayout features for the GUI layout and button features to allow
button functionality for the โ€˜Continueโ€™ and โ€˜Returnโ€™ buttons. In addition to the Kivy modules,
the import Webbrowser has been used which allows the programme to open up a web browser
(docs.python, n.d.). In this case, this will be used to open a URL after scanning. Lastly, as the
GUI utilises all the main components of the system, the three main components have been
imported to this file, these being the URL format validation function, PKI Certificate validation
function, and lastly the ML model prediction function. The imports can be seen in the below
figure.
After importing the relevant modules and libraries, the first step was to store the output link
from the scanner by calling the print_global_link() function from the QR code scanner. The
output was then stored in the variable ScanThisURL. Now that the link has been stored in a
variable, the three main component functions can be imported, and the link can be passed to
each function as an argument to allow the individual scans to be run on the provided link. After
Figure 34: GUI imports.
SID:XXXXXXX MOD002691
62
each scan has been finished the retuned values are turned into strings and stored in variables as
seen in the below figure.
Now that the returned values from each component have been stored in variables, the window
to display the output needs to be created. By utilizing the Kivy Popup() function, a popup
window was defined to display after a QR code is scanned. Within this popup window the Kivy
GridLayout function was utilized to arrange the GUI components on the screen. TopGrid was
defined with one column, this allowed for the title and passed URL to be displayed at the top
centre of the GUI. Next another Grid was defined name EmbbeddedGrid with two columns and
was embedded into the first grid, this allowed the second grid to have two columns without
effecting the objects within the TopGrid. Within EmbeddedGrid, the first column consisted of
the names of each scan and the second grid is where the returned values from each component
have been displayed. This can be seen in the below figure.
Figure 35: GUI code segment 1
SID:XXXXXXX MOD002691
63
At this point the GUI can display the title, scanned URL, and results of the scan. The next part
of the implementation was to define two buttons which can be used to either return to the
scanner or continue to the scanned URL. In addition, if the scans determine that a URL is
malicious a warning should be applied to the screen.
First, the Continue button was defined by using the Kivy Button() function, once the design
elements were applied the button was bind to the con() function, which used the open_new()
function to open the past URL argument in a web browser if the button is pressed. Once the
button was bind to the function, the button was displayed on the GUI with the add_widget()
function. In addition, a variable named โ€˜CWarningโ€™ was appended to the button text, this
variable contains a warning dependent on if the scans were all clear. The Continue button
related code can be seen in the below figure.
Figure 36: GUI code segment 2.
SID:XXXXXXX MOD002691
64
Lastly, the Return button was implemented, this button followed the same design as the first
however it accessed the Popup up function and bind the dismiss function to the button if the
button was pressed. In addition, instead of applying the โ€˜CWarningโ€™ variable to the text, the
Return button has the โ€˜RWarningโ€™ variable applied. The code for the return button can be seen
below.
Figure 37: GUI code segment 3
Figure 38: GUI code segment 4.
SID:XXXXXXX MOD002691
65
7.0 Testcases
The below testcases are testing the ability of the complete system. is important to note that not
all combination of output have been tested as this is not practical. Such as the combination of a
valid certificate with an invalid URL will not be produced.
Testcase 1
Description Pass / Fail
URL provided has: Valid Certificate, Valid URL format and is safe.
The application is expected to return Clear, Clear, and Clear
respectively
Pass
Evidence
QR Code:
https://en.wikipedia.org/wiki/QR_code
Application response
Table 10: Testcase 1
SID:XXXXXXX MOD002691
66
Testcase 2
Description Pass / Fail
URL provided has: Valid Certificate, Valid URL format and is
Malicious.
The application is expected to return Clear, Clear, and Malicious
respectively
Pass
Evidence
QR Code:
http://maliciouswebsitetest.com
Application response
Table 11: Testcase 2
SID:XXXXXXX MOD002691
67
Testcase 3
Description Pass / Fail
URL provided has: Invalid Certificate, invalid URL format and is
Malicious.
The application is expected to return Invalid, Invalid, and Malicious
respectively
Pass
Evidence
QR Code:
https://ExampleBadURL,com
Application response
Table 12: Testcase 3
SID:XXXXXXX MOD002691
68
Testcase 4
Description Pass / Fail
URL provided has: Invalid Certificate, Valid URL format and is
malicious.
The application is expected to return Invalid, Clear, and malicious
respectively
Pass
Evidence
QR Code:
https://expired.badssl.com
Application response
Table 13: Testcase 4
SID:XXXXXXX MOD002691
69
Testcase 5
Description Pass / Fail
GUI Continue button is expected to open derived URL in native web
browser
Pass
Evidence
Application response
Table 14: Testcase 5
SID:XXXXXXX MOD002691
70
Testcase 6
Description Pass / Fail
GUI โ€˜Returnโ€™ button is expected to return user to QR code scanner. Pass
Evidence
Application response
Table 15: Testcase 6
SID:XXXXXXX MOD002691
71
8.0 Discussion
In this study I have discovered how best to identify malicious QR codes accurately and efficiently
in efforts to prosper an effective and usable system which can be used to prevent interaction with
malicious QR codes. This was achieved by conducting research and analysis on the current
literature to identify the best identification methods and in addition what weaknesses were present
in the current solutions. From this I identified how to address the oversight to produce a superior
system in both the ML accuracy and efficiency. In addition, implementing a hybrid approach
which utilized additional programming function to ensure additional prediction integrity outside
the ML model. Once the methodology was identified, the system was implemented into an
operational system. Extensive testing was conducted to ensure the usability, accuracy, and
efficiency of the system.
The completed system achieved all specified requirements defined from the original research
question. In addition, managed to effectively improve upon all oversights identified in the current
literature. From this a highly effective system at identifying malicious URLs derived from QR
codes has been created. This systems hybrid approach to identifying malicious URLs allows for
a more accurate and holistic prediction opposed to soul reliance on a ML model. Therefore,
producing a more suitable solution than anything found within the current literature.
It can be observed from the results that the system achieved great prediction accuracy. The ML
component of the system boasts a 96% accuracy with a significantly high TP accuracy of 97%
ensuring the likelihood of a malicious URL not being identified is extremally low. The model
accuracy is higher than any identified within the covered literature, which was in part due to the
extensive testing of different classification algorithms to determine which was the most accurate
and effective at solving the problem. In addition to the ML, the functions that ensure valid PKI
certificates and URL format prospered 100% accuracy against the test bed. These solutions work
SID:XXXXXXX MOD002691
72
together to produce an exceptionally high prediction integrity. In addition to the hybrid solution
testing, multiple testcases were conducted on the system ensuing the subsystems integrated
together correctly and that the GUI worked as expected. From this it was observed that the system
was both accurate and efficient at the defined task.
From this it can be observed that the system is an extremally viable solution to the original
research question and is not just effective in its ability to identify malicious QR codes, but in
addition, at being an efficient and usable system by any level of technical ability.
However, I do believe there are improvements that could be introduced to the system in the future.
In specifically the ML model accuracy and integrity could be further improved. As this project
was my first introduction to machine learning there are certain lack of complexities which would
have benefited the ML model in its predictions. More advanced feature engineering and selection
could be implemented to increase the accuracy of the model, for example, implementing
extensive feature groups that identify many aspects of the URL. Moreover, although the
programming validation functions are significantly effective, additional function could be
implemented, such as a function to check a URL against known databases of malicious URLs for
improved prediction integrity.
Overall, it can be observed that although there is scope for future improvement, the current system
is fit for purpose in all aspects of its function and has achieved all aims of this study and addressed
all problems identified.
SID:XXXXXXX MOD002691
73
9.0 Conclusion
It can be concluded from the discussion that this development project has achieved all aims and
requirements originally defined at the beginning of the study. Due to this, I believe that the
developed system has real value to the cyber security space as it can prevent a range of malicious
cyber security attacks which utilize QR codes as an attack vector. The extent to which each aim
of the study has been achieved is detailed below:
Aim one was to provide research on the current methods which are being utilized to identify
malicious URLs derived from QR codes. As can be observed from chapters 1-4, extensive
research and analysis has been conducted upon the current methods used to address this problem,
in addition the weaknesses and oversights of the current literature have been identified and
mitigation to the issues have been identified. From this it can be concluded that aim one has been
successfully achieved.
Aim two of the study was to develop a hybrid solution to the research question. It can be
concluded from this study content that this aim was successfully achieved. The created system is
a superior solution to the current one-dimensional approaches covered in the current literature.
The last aim was to conduct extensive testing of different classification alogrithms accuracy when
applied to the ML model. Five different algorithms have been tested and detailed to identify the
most appropriate algorithm for the model. From this it can be concluded that aim three was
achieved.
SID:XXXXXXX MOD002691
74
References
Abad, S., et al, 2023, Classification of Malicious URLs Using Machine Learning (pdf) Available
at: <https://www.mdpi.com/1424-8220/23/18/7760 > [Accessed on 24 February 2024].
Alder, S., 2023, QR Codes Increasingly Used in Phishing Attacks (online) Available at
<https://www.hipaajournal.com/qr-codes-increasingly-used-in-phishing-attacks/#:>
[Accessed on 4 December 2023].
Aljabri et al, 2017, Detecting Malicious URLs Using Machine Learning Techniques: Review and
Research Directions (pdf) Available at: <
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9950508> [Accessed on 13
December 2023].
Al-Zahrani, M., Wahsheh, H., Alsaade, F., 2021, Secure Real-Time Artificial Intelligence System
against Malicious QR Code Links (pdf) Available at: <
https://www.hindawi.com/journals/scn/2021/5540670/> [Accessed on 14 December
2023].
Anishnama, 2023, Understanding Bidirectional LSTM for Sequential Data Processing (online)
Available at: < https://medium.com/@anishnama20/understanding-bidirectional-lstm-
for-sequential-data-processing-b83d6283befc#> [Accessed on 24 February 2024].
Avast, n.d., Public vs. Private IP Addresses: Whatโ€™s the Difference? (online) Available at:
<https://www.avast.com/c-ip-address-public-vs-private> [Accessed on 19 December
2023].
Badssl.com, n.d., badssl.com (online) Available at: <https://badssl.com/> [Accessed on 3 January
2024].
Barkeved, K., 2022, Data Cleaning: The Most Important Step in Machine Learning (online)
Available at: < https://www.obviously.ai/post/data-cleaning-in-machine-learning >
[Accessed on 18 December 2023].
SID:XXXXXXX MOD002691
75
Cherisien, W., 2024, 17 Creative Ways to Use QR Codes (online) Available at: <
https://mention.com/en/blog/creative-ways-to-use-qr-codes/#> [Accessed on 23
February 2024].
Comodo, n.d., What is a PKI Certificate? (online) Available at <
https://comodosslstore.com/resources/what-is-a-pki-certificate/> [Accessed on 9
December 2023].
CyberWarrior, 2023, Is Python Good for Cybersecurity? (online) Available at
<https://www.cyberwarrior.com/is-python-good-for-cybersecurity/#:> [Accessed on 4
December 2023].
Docs.python,org, n.d., The Python Standard Library (online) Available at
<https://docs.python.org/3/library/index.html> [Accessed on 11 December 2023].
Dorfman, E., 2022, How Much Data Is Required for Machine Learning? (online) Available at: <
https://postindustria.com/how-much-data-is-required-for-machine-learning/#: >
[Accessed on 18 December 2023].
Dremio, n.d., Vectorization in NLP (online) Available at:
<https://www.dremio.com/wiki/vectorization-in-nlp/> [Accessed on 19 December
2023].
Forbes, 2023, Partner Should Know: The Top Programming Languages Of 2023 (online)
Available at <https://www.forbes.com/sites/forbestechcouncil/2022/12/28/what-your-
software-partner-should-know-the-top-programming-languages-of-2023/> [Accessed on
4 December 2023].
Fortinet, n.d., File Transfer Protocol (FTP) Meaning and Definition (online) Available at:
<https://www.fortinet.com/resources/cyberglossary/file-transfer-protocol-ftp-meaning>
[Accessed on 19 December 2023].
SID:XXXXXXX MOD002691
76
Griffiths, C., 2023, The Latest 2023 Phishing Statistics (Updates December 2023) (online)
Available at: < https://aag-it.com/the-latest-phishing-statistics/#:> [Accessed on 2
December 2023].
Hughes, L., 2022, SSL and TLS (online) Available at <
https://link.springer.com/chapter/10.1007/978-1-4842-7486-6_11> [Accessed on 9
December 2023].
IBM, 2021, The components of a URL (online) Available at <https://www.ibm.com/docs/en/cics-
ts/5.1?topic=concepts-components-url> [Accessed on 9 December 2023].
IBM, n.d., What is logistic regression? (online) Available at: <
https://www.ibm.com/topics/logistic-regression> [Accessed on 19 December 2023].
IBM, n.d., What is random forest? (online) Available at: < https://www.ibm.com/topics/random-
forest> [Accessed on 19 December 2023].
Jafar a, et al, 2018, Machine Learning from Theory to Algorithms: An Overview (pdf) Available
at <https://iopscience.iop.org/article/10.1088/1742-6596/1142/1/012012/pdf>
[Accessed on 5 December 2023].
Javatpoint, n.d., Train and Test dataset in Machine Learning (online) Available at:
<https://www.javatpoint.com/train-and-test-datasets-in-machine-learning> [Accessed
on 19 December 2023].
Jha, A., 2023, Vectorization Techniques ion NLP [Guide] (online) Available at <
https://neptune.ai/blog/vectorization-techniques-in-nlp-guide> [Accessed on 9
December 2023].
Jupyter, n.d., jupyter (online) Available at < https://jupyter.org/ > [Accessed on 11 December
2023].
SID:XXXXXXX MOD002691
77
Karabiber, F., n.d., TF-IDF โ€“ Term Frequency โ€“ Inverse Document Frequency (online) Available
at: <https://www.learndatasci.com/glossary/tf-idf-term-frequency-inverse-document-
frequency/#:> [Accessed on 19 December 2023].
Karbhari, V., 2019, What is TF-IDF in Feature Engineering? (online) Available at:
<https://medium.com/acing-ai/what-is-tf-idf-in-feature-engineering-7f1ba81982bd#>
[Accessed on 23 February 2024].
Kivy, n.d., Kivy: The Open Source Python App Development Framework (online) Available at
<https://kivy.org/index.html> [Accessed on 11 December 2023].
Krombholz K., Fruhwirt, P., Rieder, T., Kapsalis, I., Ullrich, J., Weippl E., 2013, QR Code
Security โ€“ How Secure and Usable Apps Can Protect Users Against Malicious QR Codes
(pdf) Available at: <
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7299920> [Accessed on 14
December 2023].
Kumar, S., 2020, Supervised vs Unsupervised vs Reinforcement (online) Available at:
<https://www.aitude.com/supervised-vs-unsupervised-vs-reinforcement/#> [Accessed
on 23 March 2024.].
Liu, J., 2022, Lexical Features of Economic Legal Policy and News in China Since the COVID-
19 Outbreak (online) Available at: < https://www.frontiersin.org/journals/public-
health/articles/10.3389/fpubh.2022.928965/full> [Accessed on 24 February 2024].
Mahesh, B., 2020, Machine Learning Algorithms โ€“ A Review (pdf) Available at: <
https://www.researchgate.net/profile/Batta-
Mahesh/publication/344717762_Machine_Learning_Algorithms_-
A_Review/links/5f8b2365299bf1b53e2d243a/Machine-Learning-Algorithms-A-
Review.pdf> [Accessed on 19 December 2023].
SID:XXXXXXX MOD002691
78
McAfee, n.d., What is Typosquatting? (online) Available at
<https://www.mcafee.com/learn/what-is-typosquatting/#:> [Accessed on 11 December
2023].
Microsoft, 2023, What is a machine learning model? (online) Available at <
https://learn.microsoft.com/en-us/windows/ai/windows-ml/what-is-a-machine-learning-
model > [Accessed on 9 December 2023].
Microsoft, 2023, What is Visual Studio? (online) Available at <https://learn.microsoft.com/en-
us/visualstudio/get-started/visual-studio-ide?view=vs-2022> [Accessed on 11 December
2023].
MonkeyLearn, n.d., Machine Learning (online) Available at:
<https://monkeylearn.com/blog/classification-algorithms/#> [Accessed on 23 February
2024].
Naylor, D., n.d., The Cost of the โ€œSโ€ in HTTPS (pdf) Available at
<https://dl.acm.org/doi/pdf/10.1145/2674005.2674991> [Accessed on 9 December
2023].
OpenPhish, n.d., OpenPhish (online) Available at: <https://openphish.com/> [Accessed on 3
January 2024].
OSIbeyond, 2023, QR Code Scams: Think Before You Scan (online) Available at:
<https://www.osibeyond.com/blog/qr-code-scams/> [Accessed on 26 February 2024].
Pawar, A., et al, 2022, Secure QR Code Scanner to Detect Malicious URL using Machine
Learning (pdf) Available at: < https://ieeexplore.ieee.org/Xplore/home.jsp> [Accessed
on 24 February 2024].
Phising.org, n.d., What Is Phishing? (online) Available at: < https://www.phishing.org/what-is-
phishing> [Accessed on 2 December 2023].
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.

More Related Content

Similar to QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.

Guide on the use of Artificial Intelligence-based tools by lawyers and law fi...
Guide on the use of Artificial Intelligence-based tools by lawyers and law fi...Guide on the use of Artificial Intelligence-based tools by lawyers and law fi...
Guide on the use of Artificial Intelligence-based tools by lawyers and law fi...
Massimo Talia
ย 
Master_Thesis
Master_ThesisMaster_Thesis
Master_Thesis
Kieran Flesk
ย 
WebIT2 Consultants Proposal
WebIT2 Consultants ProposalWebIT2 Consultants Proposal
WebIT2 Consultants Proposal
Sarah Killey
ย 
Mobile d
Mobile dMobile d
Mobile d
franks90
ย 
FINAL PROJECT REPORT
FINAL PROJECT REPORTFINAL PROJECT REPORT
FINAL PROJECT REPORT
Abdul Samad
ย 
An evaluation of Dockerโ€™s security
An evaluation of Dockerโ€™s securityAn evaluation of Dockerโ€™s security
An evaluation of Dockerโ€™s security
Ade Ajasa
ย 
Thesis Nha-Lan Nguyen - SOA
Thesis Nha-Lan Nguyen - SOAThesis Nha-Lan Nguyen - SOA
Thesis Nha-Lan Nguyen - SOA
Nha-Lan Nguyen
ย 
Milan_thesis.pdf
Milan_thesis.pdfMilan_thesis.pdf
Milan_thesis.pdf
kanaka vardhini
ย 
Abrek_Thesis
Abrek_ThesisAbrek_Thesis
Abrek_Thesis
Natascha Abrek
ย 
Telecottage_Handbook__How_to_Establish_and_Run_a_Successful_Telecentre
Telecottage_Handbook__How_to_Establish_and_Run_a_Successful_TelecentreTelecottage_Handbook__How_to_Establish_and_Run_a_Successful_Telecentre
Telecottage_Handbook__How_to_Establish_and_Run_a_Successful_Telecentre
Yuri Misnikov
ย 
Integrating developing countriesโ€™ SMEs into Global Value Chain.
Integrating developing countriesโ€™ SMEs into Global Value Chain.Integrating developing countriesโ€™ SMEs into Global Value Chain.
Integrating developing countriesโ€™ SMEs into Global Value Chain.
Ira Tobing
ย 
VeraCode State of software security report volume5 2013
VeraCode State of software security report volume5 2013VeraCode State of software security report volume5 2013
VeraCode State of software security report volume5 2013
Cristiano Caetano
ย 
Technical Communication 14th Edition Lannon Solutions Manual
Technical Communication 14th Edition Lannon Solutions ManualTechnical Communication 14th Edition Lannon Solutions Manual
Technical Communication 14th Edition Lannon Solutions Manual
IgnaciaCash
ย 
A.R.C. Usability Evaluation
A.R.C. Usability EvaluationA.R.C. Usability Evaluation
A.R.C. Usability Evaluation
JPC Hanson
ย 
THE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKS
THE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKSTHE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKS
THE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKS
Debashish Mandal
ย 
Blockchain in Education. Alexander Grech & Anthony F. Camilleri. Editor Andre...
Blockchain in Education. Alexander Grech & Anthony F. Camilleri. Editor Andre...Blockchain in Education. Alexander Grech & Anthony F. Camilleri. Editor Andre...
Blockchain in Education. Alexander Grech & Anthony F. Camilleri. Editor Andre...
eraser Juan Josรฉ Calderรณn
ย 
DM_DanielDias_2020_MEI.pdf
DM_DanielDias_2020_MEI.pdfDM_DanielDias_2020_MEI.pdf
DM_DanielDias_2020_MEI.pdf
Muthusankaranarayana1
ย 
Ict in africa education fullreport
Ict in africa education fullreportIct in africa education fullreport
Ict in africa education fullreport
Stefano Lariccia
ย 
My PhD Thesis
My PhD Thesis My PhD Thesis
My PhD Thesis
Suman Srinivasan
ย 
Requirements engineering by elizabeth hull, ken jackson, jeremy dick (z lib.org)
Requirements engineering by elizabeth hull, ken jackson, jeremy dick (z lib.org)Requirements engineering by elizabeth hull, ken jackson, jeremy dick (z lib.org)
Requirements engineering by elizabeth hull, ken jackson, jeremy dick (z lib.org)
DagimbBekele
ย 

Similar to QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes. (20)

Guide on the use of Artificial Intelligence-based tools by lawyers and law fi...
Guide on the use of Artificial Intelligence-based tools by lawyers and law fi...Guide on the use of Artificial Intelligence-based tools by lawyers and law fi...
Guide on the use of Artificial Intelligence-based tools by lawyers and law fi...
ย 
Master_Thesis
Master_ThesisMaster_Thesis
Master_Thesis
ย 
WebIT2 Consultants Proposal
WebIT2 Consultants ProposalWebIT2 Consultants Proposal
WebIT2 Consultants Proposal
ย 
Mobile d
Mobile dMobile d
Mobile d
ย 
FINAL PROJECT REPORT
FINAL PROJECT REPORTFINAL PROJECT REPORT
FINAL PROJECT REPORT
ย 
An evaluation of Dockerโ€™s security
An evaluation of Dockerโ€™s securityAn evaluation of Dockerโ€™s security
An evaluation of Dockerโ€™s security
ย 
Thesis Nha-Lan Nguyen - SOA
Thesis Nha-Lan Nguyen - SOAThesis Nha-Lan Nguyen - SOA
Thesis Nha-Lan Nguyen - SOA
ย 
Milan_thesis.pdf
Milan_thesis.pdfMilan_thesis.pdf
Milan_thesis.pdf
ย 
Abrek_Thesis
Abrek_ThesisAbrek_Thesis
Abrek_Thesis
ย 
Telecottage_Handbook__How_to_Establish_and_Run_a_Successful_Telecentre
Telecottage_Handbook__How_to_Establish_and_Run_a_Successful_TelecentreTelecottage_Handbook__How_to_Establish_and_Run_a_Successful_Telecentre
Telecottage_Handbook__How_to_Establish_and_Run_a_Successful_Telecentre
ย 
Integrating developing countriesโ€™ SMEs into Global Value Chain.
Integrating developing countriesโ€™ SMEs into Global Value Chain.Integrating developing countriesโ€™ SMEs into Global Value Chain.
Integrating developing countriesโ€™ SMEs into Global Value Chain.
ย 
VeraCode State of software security report volume5 2013
VeraCode State of software security report volume5 2013VeraCode State of software security report volume5 2013
VeraCode State of software security report volume5 2013
ย 
Technical Communication 14th Edition Lannon Solutions Manual
Technical Communication 14th Edition Lannon Solutions ManualTechnical Communication 14th Edition Lannon Solutions Manual
Technical Communication 14th Edition Lannon Solutions Manual
ย 
A.R.C. Usability Evaluation
A.R.C. Usability EvaluationA.R.C. Usability Evaluation
A.R.C. Usability Evaluation
ย 
THE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKS
THE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKSTHE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKS
THE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKS
ย 
Blockchain in Education. Alexander Grech & Anthony F. Camilleri. Editor Andre...
Blockchain in Education. Alexander Grech & Anthony F. Camilleri. Editor Andre...Blockchain in Education. Alexander Grech & Anthony F. Camilleri. Editor Andre...
Blockchain in Education. Alexander Grech & Anthony F. Camilleri. Editor Andre...
ย 
DM_DanielDias_2020_MEI.pdf
DM_DanielDias_2020_MEI.pdfDM_DanielDias_2020_MEI.pdf
DM_DanielDias_2020_MEI.pdf
ย 
Ict in africa education fullreport
Ict in africa education fullreportIct in africa education fullreport
Ict in africa education fullreport
ย 
My PhD Thesis
My PhD Thesis My PhD Thesis
My PhD Thesis
ย 
Requirements engineering by elizabeth hull, ken jackson, jeremy dick (z lib.org)
Requirements engineering by elizabeth hull, ken jackson, jeremy dick (z lib.org)Requirements engineering by elizabeth hull, ken jackson, jeremy dick (z lib.org)
Requirements engineering by elizabeth hull, ken jackson, jeremy dick (z lib.org)
ย 

Recently uploaded

Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
ssuser1915fe1
ย 
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
FIDO Alliance
ย 
Finetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and DefendingFinetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and Defending
Priyanka Aash
ย 
Types of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technologyTypes of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technology
ldtexsolbl
ย 
Redefining Cybersecurity with AI Capabilities
Redefining Cybersecurity with AI CapabilitiesRedefining Cybersecurity with AI Capabilities
Redefining Cybersecurity with AI Capabilities
Priyanka Aash
ย 
Semantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software DevelopmentSemantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software Development
Baishakhi Ray
ย 
Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)
Debmalya Biswas
ย 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
SynapseIndia
ย 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
SynapseIndia
ย 
It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...
Zilliz
ย 
Girls call Kolkata ๐Ÿ‘€ XXXXXXXXXXX ๐Ÿ‘€ Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata ๐Ÿ‘€ XXXXXXXXXXX ๐Ÿ‘€ Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata ๐Ÿ‘€ XXXXXXXXXXX ๐Ÿ‘€ Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata ๐Ÿ‘€ XXXXXXXXXXX ๐Ÿ‘€ Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
ย 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
shanihomely
ย 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
ย 
Step-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From ScratchStep-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From Scratch
softsuave
ย 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
maigasapphire
ย 
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and CitiesThe Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
Arpan Buwa
ย 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
janagijoythi
ย 
Improving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning ContentImproving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning Content
Enterprise Knowledge
ย 
Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024
siddu769252
ย 
The Path to General-Purpose Robots - Coatue
The Path to General-Purpose Robots - CoatueThe Path to General-Purpose Robots - Coatue
The Path to General-Purpose Robots - Coatue
Razin Mustafiz
ย 

Recently uploaded (20)

Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
ย 
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
ย 
Finetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and DefendingFinetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and Defending
ย 
Types of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technologyTypes of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technology
ย 
Redefining Cybersecurity with AI Capabilities
Redefining Cybersecurity with AI CapabilitiesRedefining Cybersecurity with AI Capabilities
Redefining Cybersecurity with AI Capabilities
ย 
Semantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software DevelopmentSemantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software Development
ย 
Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)Gen AI: Privacy Risks of Large Language Models (LLMs)
Gen AI: Privacy Risks of Large Language Models (LLMs)
ย 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
ย 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
ย 
It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...
ย 
Girls call Kolkata ๐Ÿ‘€ XXXXXXXXXXX ๐Ÿ‘€ Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata ๐Ÿ‘€ XXXXXXXXXXX ๐Ÿ‘€ Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata ๐Ÿ‘€ XXXXXXXXXXX ๐Ÿ‘€ Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata ๐Ÿ‘€ XXXXXXXXXXX ๐Ÿ‘€ Rs.9.5 K Cash Payment With Room Delivery
ย 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
ย 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
ย 
Step-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From ScratchStep-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From Scratch
ย 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
ย 
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and CitiesThe Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
ย 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
ย 
Improving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning ContentImproving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning Content
ย 
Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024
ย 
The Path to General-Purpose Robots - Coatue
The Path to General-Purpose Robots - CoatueThe Path to General-Purpose Robots - Coatue
The Path to General-Purpose Robots - Coatue
ย 

QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.

  • 1. SID:XXXXXXX MOD002691 1 QR Secure: A hybrid approach using Machine Learning and Security Validation Functions to prevent interaction with Malicious QR codes. Word Count: 10000 Richford, A. SID:XXXXXXX MOD002691 Final Project Final Project Report BSc Cyber Security Submitted: 24/03/2024
  • 2. SID:XXXXXXX MOD002691 2 Abstract QR codes are becoming an increasingly used attack vector for cybercriminal to obtain users confidential information resulting in both financial and identity theft. This study has been formulated with the intent to discover how effective a hybrid approach of machine learning and programming validation functions are at determining if a QR code derived Uniform Resource Locator (URL) is malicious in nature. The first section of this study details why this question is necessary and what threats are faced from malicious QR codes. In addition to this background information on QR codes, machine learning (ML), Public Key Infrastructure (PKI) certificates and URLs has been detailed. Next a literature review on several related academic papers has been conducted to obtain a problem statement for the paper. From this the methodology has been defined for the planning, creation and implementation of a system which uses ML, a URL format validation function, and a PKI certificate validation function to determine if a QR code is malicious in nature. Finally, the implementation section details the creation of the system from the development to testing. The results show the effectiveness of a hybrid approach to addressing if a URL derived from a QR code is malicious, this has been fostered by a highly accurate and efficient ML model in conjunction with the programming validation functions, the discussion and conclusion section of this study details these findings.
  • 3. SID:XXXXXXX MOD002691 3 Acknowledgements Firstly, I would like to acknowledge the significant support provided to me by my family, who have always supported me in both relation to my studies and personal endeavours. In addition, I would also like to acknowledge the support of the faculty residing at Anglia Ruskin University. In specific I would like to acknowledge my supervisor and Personal Development Tutor (PDT) Muhammad Ali. Who has provided exceptional support and time investment into me throughout this development project and my university career.
  • 4. SID:XXXXXXX MOD002691 4 Table of Contents Acknowledgements.................................................................................................................. 3 1.0 Introduction....................................................................................................................... 8 1.1 Problem Statement ........................................................................................................ 9 1.2 Aims of the study ........................................................................................................... 9 1.3 Contribution................................................................................................................... 9 1.4 Structure...................................................................................................................... 10 2.0 Background on QR codes.................................................................................................. 11 3.0 Background on ML ........................................................................................................... 18 4.0 Literature Review............................................................................................................. 21 4.1 Critical Analysis ............................................................................................................ 29 5.0 Proposed Work ................................................................................................................ 31 5.1 Methodology................................................................................................................ 31 5.2 Machine Learning model to detect malicious URLs. ...................................................... 33 5.2.1 Collection of data .................................................................................................. 33 5.2.2 Cleaning and preparation of dataset...................................................................... 35 5.2.3 Feature Engineering............................................................................................... 36 5.2.4 Test classifier algorithms against model to determine the most appropriate algorithm. ...................................................................................................................... 37 5.3 Validating if URL has a valid PKI certificate.................................................................... 38 5.4 Validating if URL format is valid. ................................................................................... 39 5.5 Creation of QR code reader and system GUI................................................................. 41 6.0 Implementation and Results............................................................................................. 43 6.1 Implementation of ML model to detect malicious URLs. ............................................... 46 6.1.1 Testing model predictions against known malicious and safe URLs. ....................... 51 6.2 Implementation of URL PKI certificate validation.......................................................... 53 6.2.1 Testing function against known valid and invalid certificates.................................. 54 6.3 Implementation of URL format validation..................................................................... 55 6.3.1 Testing function against known valid and invalid format URLs................................ 57 6.4 Implementation of the QR code scanner ...................................................................... 58 6.5 Implementation of Graphical user interface ................................................................. 61 7.0 Testcases ......................................................................................................................... 65 8.0 Discussion ........................................................................................................................ 71 9.0 Conclusion ....................................................................................................................... 73 References............................................................................................................................. 74
  • 5. SID:XXXXXXX MOD002691 5 Appendix I: Interim report...................................................................................................... 82 Appendix II: Ethics Certificate of Completion.......................................................................... 87 Appendix III: Project Poster.................................................................................................... 88 Appendix IV: Presentation of source code.............................................................................. 89
  • 6. SID:XXXXXXX MOD002691 6 Table of Figures Figure 1: Generated QR code containing link 'https://QRSecure.com' .................................... 11 Figure 2: Generated QR code containing link 'http:/MalWARE.cog' ........................................ 12 Figure 3: CIA triad (IBM, 2023) ............................................................................................... 13 Figure 4: Certificate chain (The SSL Store, n.d.)....................................................................... 14 Figure 5: Accuracy Comparison (Adapted from Pawar et al, 2022).......................................... 21 Figure 6: Evaluating security performance of QR code scanners (Adapted from Rafsanjani et al, 2023) ..................................................................................................................................... 24 Figure 7: Testing results (Adapted from Xuan et al, 2020)....................................................... 27 Figure 8: Proposed architecture of system function. .............................................................. 32 Figure 9: URL Dataset entries. ................................................................................................ 34 Figure 10: ratio of good and bad URLs in dataset.................................................................... 34 Figure 11: Observing no NULL values in dataset. .................................................................... 35 Figure 12: Figure to show testing and training set split........................................................... 36 Figure 13: Illustration of certificate validation. ....................................................................... 38 Figure 14: Validators source code (Adapted from validators, n.d.).......................................... 39 Figure 15: Illustration of URL validation.................................................................................. 41 Figure 16: Navigation map of system. .................................................................................... 42 Figure 17: Wireframe diagram of GUI (iPhone template adapted from unblast, n.d.) ............. 42 Figure 18: Imported libraries for ML model............................................................................ 46 Figure 19: โ€˜urldataโ€™ dataset manipulation............................................................................... 46 Figure 20: splitting dataset into input and output set. ............................................................ 47 Figure 21: Vectorizing data with TF-IDF .................................................................................. 47 Figure 22: Splitting data for testing and training..................................................................... 48 Figure 23: NB, RF, SVM, LR and DT model reports. ................................................................. 48 Figure 24: Confusion matrix for LR. ........................................................................................ 50 Figure 25: ML model prediction function................................................................................ 50 Figure 26: PKI Certificate validation function import. ............................................................. 53 Figure 27: URL PKI Certificate validation function................................................................... 54 Figure 28: Tested good and bad certificates. .......................................................................... 54 Figure 29: URL format validation function import................................................................... 55 Figure 30: URL format validation function. ............................................................................. 56 Figure 31: QR code scanner imports....................................................................................... 58 Figure 32: QR code scanner working example. ....................................................................... 59 Figure 33: QR code scanner code. .......................................................................................... 60 Figure 34: GUI imports........................................................................................................... 61 Figure 35: GUI code segment 1 .............................................................................................. 62 Figure 36: GUI code segment 2. ............................................................................................. 63 Figure 37: GUI code segment 3 .............................................................................................. 64 Figure 38: GUI code segment 4. ............................................................................................. 64 Figure 39: Certificate of Completion CPD Course.................................................................... 87 Figure 40: Project Poster........................................................................................................ 88
  • 7. SID:XXXXXXX MOD002691 7 Table of Tables Table 1: Components of valid URL.......................................................................................... 16 Table 2: Components of example URL.................................................................................... 17 Table 3: Classification algorithms for model testing................................................................ 19 Table 4: Details on validator source code ............................................................................... 40 Table 5: Utilized python libraries............................................................................................ 45 Table 6: Accuracy of algorithms summary.............................................................................. 49 Table 7: ML model predictions of provided URLs.................................................................... 52 Table 8: Testing PKI Certificate validation function................................................................. 55 Table 9: Testing URL format validation function ..................................................................... 57 Table 10: Testcase 1............................................................................................................... 65 Table 11: Testcase 2............................................................................................................... 66 Table 12: Testcase 3............................................................................................................... 67 Table 13: Testcase 4............................................................................................................... 68 Table 14: Testcase 5............................................................................................................... 69 Table 15: Testcase 6............................................................................................................... 70
  • 8. SID:XXXXXXX MOD002691 8 1.0 Introduction One of the most notorious attack vectors used by cyber criminals today is Phishing, this attempts to lure a target individual into providing confidential or sensitive information and will often direct a user to a malicious webpage where malicious activities such as data theft are inflicted on the victim (Phishing.org, n.d.). It is estimated that 3.4 billion phishing emails are sent per day (Griffiths, 2023). However, due to the constant changing technology landscape, threat actors are finding new ways to lure individuals into unknowingly providing their confidential information. One of the emerging attack vectors is known as Quishing. Quishing, also known as QR code phishing is where an attacker lures a victim into scanning a malicious QR code which then redirects the victim to a malicious URL in attempts to infect them with malware or acquiring the victimโ€™s confidential information (sosafe, n.d.). In the month of September 2023 QR code phishing attacks saw a rise of 51% compared to the combined known attacks from January to August 2023 (Security Staff, 2023). In addition to the recent rise in QR code phishing attacks, the overall cyber security attack posture has QR code phishing attacks as 22% of all phishing attacks within the month of October 2023 (Alder, 2023). This data suggests that QR code phishing attacks are being increasingly used by threat actors to conduct both cyber enabled crime such as identity theft and fraud, in addition to cyber dependent crimes such as system hacking and malware infections. This recent change in the threat landscape is what inspired the creation of a system that can be used to scan QR codes and determine if the derived URL is malicious in nature. Such a system would be able to mitigate QR code phishing attacks and therefore decrease the viability of QR codes as an attack vector. This report has been formulated to detail the research, planning, creation, and testing of such a system I created in efforts to achieve this goal.
  • 9. SID:XXXXXXX MOD002691 9 1.1 Problem Statement This study plans to answer the question: Can a hybrid approach using ML, and programming validation functions successfully be used to identify malicious URLs derived from scanned QR codes in both an accurate and efficient fashion? 1.2 Aims of the study The aims of this study are to detail the research, planning, and creation of a system which prevents interactions with malicious QR codes, ideally this report will: โ€ข Provide research on previously used methods to detect malicious content within URLs derived from QR codes. โ€ข Develop a hybrid solution to identify malicious URLs derived from QR codes that uses both ML and programming language functions which concern both the validity of the URLโ€™s PKI certificate state and the URL format. โ€ข Explore multiple ML classification algorithms against a model to determine which prospers the most accurate and efficient result and is therefore most suited to the system. 1.3 Contribution My proposed system can be used to efficiently and accurately identify malicious QR codes, as a result mitigate any unsafe interactions with them. As a result, attacks such as Quishing will be significantly reduced and therefore the threat landscape to users will be pronouncedly smaller.
  • 10. SID:XXXXXXX MOD002691 10 1.4 Structure Chapter 1 details the introduction to the study, the research question, and aims. Chapter 2 and 3 detail related background information and concepts. Chapter 4 consists of a literature review on several academic papers related to my research question. Chapter 5 details the proposed work and methodology that will be followed for implementation. Chapter 6 details the implementation of the system. In addition, conducts testing to determine the accuracy and integrity of the solutions. Chapter 7 conducts testcases on the complete system. Chapter 8 consists of a detailed discussion on the results of the study. Lastly chapter 9 concludes upon the study and determines if the aims have been achieved.
  • 11. SID:XXXXXXX MOD002691 11 2.0 Background on QR codes This section is formulated to provide background information of the concepts used within the study, and their relevance to the research question. QR Codes Vishrut Sharma notes, that QR (Quick Response) codes are a two-dimensional barcode which was first created in 1994. QR codes were first used in attempts to identify cars within car manufacturing processes. However, due to the fast readability of these codes in conjunction with the relatively large storage capacity, QR codes are now extremely popular in all aspects and domains of life. With the only barrier of entry being the need for a smartphone camera which is rather ubiquitous today (Sharma, 2012). QR codes can be encoded with either numeric or alphanumeric information, this information is often related to a URL. According to Jessica Scapati: โ€œA URL (Uniform Resource Locator) is a unique identifier used to locate a resource on the internet.โ€ (Scarpati, 2021). From this it can be understood that a URL is used in efforts to navigate the internet by acting as an address of a websites. QR codes can have URLs encoded within them to direct users to a specified website. An example of a QR code encoded with a URL can be seen below. Figure 1: Generated QR code containing link 'https://QRSecure.com'
  • 12. SID:XXXXXXX MOD002691 12 Threat actors can use QR codes as an attack vector by encoding a QR code with a Phishing URL. This could be a mimicking login of a bank in attempts to enumerate a targets bank information. Or, in addition, have an encoded URL which has a malware download on the website. Although these attack vectors exist, there is no obvious way to determine if the encoded content of a QR code is safe, as a QR code is only a representation of encoded data, no sanitation of that data is conducted. For instance, the below QR code has a malformed URL and has malicious indicators such as the key word โ€˜Malwareโ€™. The QR code seen in the figure above has an invalid URL format of โ€˜http:/โ€™ where this should be โ€˜https://โ€™ which is the correct format for a secure URL. In addition, it contains the keyword โ€˜MalWAREโ€™. Although the content is seemingly malicious, the visual representation is like the โ€˜safeโ€™ QR code seen in the previous figure, this comparison demonstrates how a victim could easily scan a malicious QR code believing it is legitimate and safe. As there is no simple way to identify malicious QR codes, the interaction with them can be extremely dangerous. With the projected smartphone QR scans rising to 99.6 million in the US alone by 2025 (Cherisien, 2024), the need to ensure safe interaction is paramount. In addition to the rise in QR code scans, a study indicated 80% of respondents had used QR codes for payment transactions (Cherisien, 2024), this ubiquity and trust in the technology fosters huge concern for security and safety as a popular technique in phishing is to overlay a legitimate QR code with a Figure 2: Generated QR code containing link 'http:/MalWARE.cog'
  • 13. SID:XXXXXXX MOD002691 13 malicious one to trick an individual into interaction with it. This highlights the importance to be able to identify malicious QR codes and in tandem the importance of this study. PKI Certificates As there is no specific way to identify malicious QR codes the QR code must be decoded to reveal the data. As discussed previously, the encoded data typically will be a URL. One way to identify if a URL is likely safe is to ensure it has a valid PKI certificate. Public Key Infrastructure (PKI) Certificates are digital certificates which are used to authenticate users and encrypt connections across networks (Comodo, n.d.). A PKI certificate uses Transport Layer Security (TLS) which is a protocol used to provide encrypted and authenticated communications. Lawrence E. Hughes notes, prior to being named TLS it was known as Secure Socket Layer (SSL) which is now been deprecated for over two decades, however the terms are often still used interchangeably (Hughes, 2022). PKI certificates ensure both Confidentiality of the data via encryption, and integrity due to the authentication of the certificate user, which are two of the three fundamental pillars within the Confidentiality, Integrity, and Availability (CIA) triad, as seen in the below figure. Figure 3: CIA triad (IBM, 2023)
  • 14. SID:XXXXXXX MOD002691 14 PKI certificates are used within PKI, comodo notes, PKI is a fundamental component of the current internet, it works via a hierarchy of trust that starts from Certificate Authorities (CA) which upon validating parties, can issue digital certificates to them. At the top of the hierarchy is the Root CA which has the highest level of authentication as this is the entity from which certificates are issued. Below root CAs are Intermediate CAs which are used to decrease the workload from root CAs and distribute certificates for use, such as for a browser connection (The SSL Store, n.d.). A visual representation of this can be seen in the below figure. PKI is fundamentally used to ensure that certificates are issued to the correct entities to allow trust and secure connections between users online. Without a PKI Certificate there is no verified trust within that entity. This means that a connection to a website lacking a PKI certificate could potentially be unsecure and lack the implementation of TLS resulting in no encryption or integrity between the parties. This is common behaviour in websites that have malicious intent as an illegitimate website may struggle to obtain, or not want to obtain a PKI certificate. The lack of a certificate allows threat actors to steal information upon a connection to one of their sites, as there is no security protocol implemented, which can result in targets personal information being stolen from the session. Figure 4: Certificate chain (The SSL Store, n.d.)
  • 15. SID:XXXXXXX MOD002691 15 From this it can be understood that PKI certificates are used to ensure that users have confidentiality and integrity when online and is an essential part in any website or internet connection, as such, it is essential that a URL derived from a presented QR code, has a certificate check to ensure that the connection is secured. Valid URL format David Naylor et al notes, HTTP (Hyper Text Transfer Protocol) is a foundational component in using the internet, it is an essential part of loading webpages on computer systems (Naylor et al, n.d.). However, it is not secure, its alternative HTTPS (Hyper Text Transfer Protocol Secure) is in fact secure, and it is the standard for navigating the internet securely today, taking advantage of SSL/TLS Certificates detailed in the above section PKI certificates is extremely important to ensure security when navigating the internet. URLs are mostly used with the internet protocol HTTP/HTTPS and therefore will be used to explain the components of a URL and how to ensure a URL is valid. IBM notes, that a URL must possess certain components for it to be valid for use on the internet. These being:
  • 16. SID:XXXXXXX MOD002691 16 URL Component Description Scheme A scheme is the protocol identified within the URL. Host A host is the address of the resource. This can be a host name relating to an Internet Protocol (IP) address. Or can alternatively be a domain name related to an IP address such as an A record for IPv4. In addition, host names can include the port number appended to the host. Path A path being the path to the resource that is being accessed, such as a webpage. Query strings In the event a query string is used this must be specified in efforts to allow the resource information to perform an action. (IBM, 2021) Table 1: Components of valid URL An example of a complete HTTPS URL would look like: https://www.ExampleURL.com/thePath/recource.html each section of the above example address can be seen detailed in the table below.
  • 17. SID:XXXXXXX MOD002691 17 Section from example Component https:// Scheme www.ExampleURL.com Host /thePath Path Table 2: Components of example URL As seen in the table above, URLs follow a specific format to ensure that they are all uniformed. Defined by RFC 1738. (RFC, 1994.) As seen scheme is followed by :// and / are used to separate the components of the URL. A threat actor may deliberately malform a URL for malicious purposes. For instance, the example URL below is malformed, however at first glance, many will not see any issue. https:/ExampleURL.com The above example URLs scheme is malformed resulting in the URL not using the HTTPS protocol as it has a missing /. If a URL is malformed, it is an indication that it is malicious and could be a malicious embedded download and not a webpage. Due to the possible risk within malformed URLs, this is why a URL validation function will be implemented into the development project, to ensure any URLs derived from QR codes have legitimately formatted URLs.
  • 18. SID:XXXXXXX MOD002691 18 3.0 Background on ML For one of the fundamental aspects of this study, machine learning has been used to detect malicious URLs derived from a provided QR code. According to Jafar Alzubi et al: โ€œMachine Learning (ML) is a category of artificial intelligence that enables computers to think and learn on their ownโ€ (Jafar, et al., 2018). From this it can be understood that ML allows computers to make intelligent decisions based upon learned behaviour. For a machine to perform this type of learning and decision making, an algorithm must be implemented to a model specific to the type of problem you wish to solve. There are a few variations of ML that can be used to apply to a problem. Reinforcement learning can be used to learn a series of actions without any predefined data, unsupervised ML uses unlabelled data and identifies patterns within the data. And lastly, supervised ML uses labelled data to calculate an outcome (Kumar, 2020) supervised ML is most suited in relation to this project as a prediction based on previous data needs to be determined. The problem faced in this study is a classification problem, this is often thought of as a problem in which the answer resides as โ€˜yes or โ€˜noโ€™ (Jafar, et al, 2018). The question being, is the related URL from the provided QR code safe? Yes, or no? To make this decision a specific algorithm type can be applied named a classification algorithm. Classification algorithms excel in problems where the prediction must be categorised (MonkeyLearn, n.d.), for example category 1: Good, category 2: Bad. There are several viable classification algorithms used today, these have been detailed below:
  • 19. SID:XXXXXXX MOD002691 19 Algorithm Description Support Vector Machine (SVM) Batta Mahesh notes, SVM is a widely used technique. SVM can perform non linier classification by utilising the kernel trick, which allows for minimization of classification errors (Mahesh, 2020). Naรฏve Bayes (NB) Batta Mahesh notes, NB is a classification algorithm that is based on Bayes Theorem, NB assumes that features are independent to other features when computing (Mahesh, 2020). Decision Tree (DT) Batta Mahesh notes, DT represents choices in a tree form, the tree has decision nodes which lead to branches, this makes predictions in a conditional manner (Mahesh, 2020). Random Forest (RM) IBM notes, RM is a common algorithm that combines multiple DT output to compute its prediction (IBM, n.d.). Logistic Regression (LG) IBM notes, LG works by estimating the likelihood of an event occurring, the prediction is found between binary values 0 and 1, this is useful for classification problems where the result tends to be yes or no. Table 3: Classification algorithms for model testing
  • 20. SID:XXXXXXX MOD002691 20 A classification algorithm can use provided data to intelligently make a prediction of โ€˜yesโ€™ or โ€˜noโ€™ on a provided value and have previously been very effective when used in the security domain to detect malicious values (Scispace, n.d.). in relation to this study, datasets containing known โ€˜safeโ€™ and โ€˜maliciousโ€™ URLs will be used by an algorithm to predict if a provided URL is โ€˜safeโ€™ or โ€˜maliciousโ€™ As a URL can only be defined as โ€˜safeโ€™ or โ€˜maliciousโ€™ for the scope of this study, a classification algorithm is essential for the accuracy of the ML model predictions. However, to allow the algorithm to determine its prediction from the data, natural language processing (NLP) must first be applied which allows the algorithm to understand context within the data. This is done by encoding the human readable strings into numerical form which the algorithm can understand. This process is known as vectorization (Jha, 2023). Machine learning is greatly suited to this type of project as it can make predictions instead of searching for a matching value within a dataset. Meaning when a user provides a QR code to the system, the machine learning model can intelligently make a prediction on that URL. This is significantly more effective at stopping interactions with a โ€˜maliciousโ€™ URL as a traditional database search method would have no data to provide a result if the scanned malicious URL has not previously been identified, new malicious URLs are created constantly so archaic techniques such as this are not effective in todayโ€™s cyber landscape. ML models donโ€™t need to match a value, instead it decides upon a probability of a provided URL being โ€˜maliciousโ€™ or โ€˜safeโ€™ and returns the prediction. However, there is a problem concerning this type of implementation of machine learning which is how accurate the prediction is. To ensure the predictions are of a high accuracy, a model must be trained on data until it is providing a satisfactory level of accuracy. A model being the programme that can recognise the patterns within the data to make a prediction (Microsoft, 2023). This is why ensuring a ML model has a high volume of quality data is essential to the ML process.
  • 21. SID:XXXXXXX MOD002691 21 4.0 Literature Review This literature review will consist of the analysis and review of several published academic studies which closely align with the proposed concept of my system. I will identify what the papers were intended for, and both the strengths and weaknesses of their proposed solutions. In addition, I will conduct a critical analysis upon the literature, to detail what it has overlooked with regards to their solutions. This will help me identify a problem statement for my system. Secure QR Code Scanner to Detect Malicious URL using Machine Learning This paper formulated by Pawar, et al, created a system which used machine learning to identify malicious URLs derived from QR codes. multiple classification algorithms have been tested against an ML model to determine what algorithm produces the highest accuracy at detecting malicious URLs derived from QR codes. Each applied algorithm was explained in detail and the results of each were recorded. The highest accuracy was 83.79% from a Bidirectional Long Short- Term Memory (BI-LSTM) algorithm which is a type of recurrent neural network (RNN) which can process the provided data in both a forward and backwards direction (Anishnama, 2023). Other tested algorithms were Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Random Forest (RF). which all resulted in accuracy between 55% and 65%. as can be seen in the below figure: The ML model used three feature groups to achieve the resulting accuracy. The first feature group was lexical, these include word length, frequency, and language style (Liu, 2022). The next Figure 5: Accuracy Comparison (Adapted from Pawar et al, 2022)
  • 22. SID:XXXXXXX MOD002691 22 feature group was Host-based which derives information from the webpage content, and the final being correlated which is the total value of values such as URL length. The dataset used for the training was comprised of a few large datasets however the specific value of URLs is unspecified. However, it can be gathered that the size was sufficient (Pawar, et al, 2022). This study's strengths lie in the significant background information regarding both the study concepts and each of the applied algorithms. In addition, the application of each algorithm has been detailed with evidence to support the proclaimed accuracies. However, there are weaknesses identified within the study. Firstly, although the model accuracies are of an acceptable percentage, the model accuracy could be greatly improved. In addition, only four algorithms were applied within this study. Another significant improvement would be to apply more algorithms to ensure that the best accuracy could be identified. Detecting Malicious URLs Using Machine Learning Techniques: Review and Research Directions This paper formulated by Aljabri et al, conducts extensive research on preexisting literature concerning the detection of malicious URLs with ML. In addition to English language URLs, this paper conducts further analysis on the accuracy of ML algorithms specifically against Arabic language URLs. From the 47 papers research it was discovered that the most used machine learning algorithm to detect malicious URLs were either SVM or RF classifier algorithms. In addition, the least used algorithm was Deep Belief Networks (DBN). Due to the range of sources used for this study, the datasets ranged, however, the most common dataset used were PhishTank and Alexa (Aljabri et al, 2017). PhishTank notes, that the PhishTank dataset is comprised of known Phishing websites (PhishTank, n.d.). Papers with code notes, Alexa Domains dataset is comprised of the most common benign URLs (Paperswithcode, n.d.). This paper did extensive testing to determine the most common and effective ML classifier algorithm for detecting malicious URLs. However, the paper did no primary testing of the algorithms on a model. resulting in all the statistics being drawn straight from other literature.
  • 23. SID:XXXXXXX MOD002691 23 This being a weakness of the paper as replication of the model accuracy would suggest more legitimacy of the statistics presented. Malicious URL Detection: A Comparative Study This paper by Shantanu, et al consists of the creation and testing of an ML model that predicts if a provided URL is malicious. The paper covers both the background information relating to the used concepts and the implementation of the applied algorithms in detail. The model was applied with 7 different classification algorithms which were, Logistic Regression (LG), KKN, Naive Bayes (NB), Decision Tree (DT), RF, SVM and Stochastic Gradient descent (SGD). The highest accuracy algorithm was RF with a 92.6% accuracy when applied with the OpenPhish dataset. The paper supported these findings with evidence for each model implementation and detailed information regarding the dataset used which had a total value of 450,000 URLs both malicious and benign (Shantanu, et al, 2021). This study's testing of seven different classification algorithms is a significantly strong point of the report, the extensive testing allowed the researchers to determine the best accuracy model and therefore get the best result for the final model. In addition to this, each model and algorithm has been detailed extensively with visual evidence of the implementation. Another positive aspect of this study was that a dataset of adequate size was used which ensures the models foster the best results possible. However, the study's weaknesses are that only the model applied with the RF algorithm has had the model accuracy detailed. There is no detail on the other six applied algorithms to gather an understanding of how well they performed. This is in addition to the accuracy of the model which could be greatly improved.
  • 24. SID:XXXXXXX MOD002691 24 QsecR: Secure QR Code Scanner According to a Novel Malicious URL Detection Framework This paper formulated by Rafsanjani et al, presents an Android application named QsecR which is a QR code scanner designed to stop the interaction with malicious QR codes. The application relies on a ML model that was tested with multiple classifier algorithm consisting of NB, SVM, LR, KNN, and DT. The model used these classification algorithms with a range of feature groups consisting of lexical, host based, content based and blacklist which checks to see if a provided URL is known to be malicious, the final model implementation produced an accuracy of 93.80% using a data set of 4000 URLs combined from PhishTank and Google Safe Browsing. The report went on to compare the accuracy of the model to other known QR code scanners and demonstrated that the accuracy was superior to the other tested scanner such as Gamma-Play, InShot-Inc and Trend-Micro scanners, As seen in the figure below when presented with known malicious QR codes QsecR preformed significantly better (Rafsanjani et al, 2023). This report produced a sufficient detection system and covered the research and implementation in detail. In addition, the GUI portion of the application again was implemented well granting a high-level user experience. However, the ML model accuracy could have been improved and additional approaches to the QR detection could have been included. For instance, additional programming functions to validate if the URL is โ€˜safeโ€™, such as validating the URLโ€™s PKI certificate. Figure 6: Evaluating security performance of QR code scanners (Adapted from Rafsanjani et al, 2023)
  • 25. SID:XXXXXXX MOD002691 25 Classification of Malicious URLs Using Machine Learning This study by Abad et al, evaluates the effectiveness of using ML to identify malicious URLs when the model is applied with different instance selection techniques, which were random selection, DRLSH, and BPLSH. Random selection helps make the training process of the model faster by selecting a subset of the data for training. Data Reduction based on Locality-Sensitive Hashing (DRLSH) and Border Point Extraction based on Locality-Sensitive Hashing (BPLSH) are also used to increase the efficiency of the model. The study tested four different classification algorithms against the model with RF fostering the highest accuracy of 92.18% The study detailed the background information, relevant algorithms, and methodology extensively which allows the reader to gain a holistic understanding of the study and its findings (Abad et al, 2023). The obvious strength of this study is the computational effectiveness that is fostered by the application of random selection, DRLSH and BPLSH which resulted in the model training for RF being between 71 and 82 seconds. This allows the model to have significant efficiency in training and prediction. However, there are identified weaknesses in the study. Firstly, the highest accuracy achieved was 92.18%, ideally, this accuracy should be improved to ensure a more accurate and reliable model. In addition, there was no testing done without the applied instance selection, therefore the comparison in training time cannot be quantified by the reader which due to the nature of the study is an important data point to detail.
  • 26. SID:XXXXXXX MOD002691 26 Malicious URL Detection and Identification This paper formulated by Sayamber A., and Dixit A., created a method to detect malicious URLs via a machine learning model which used the NB classifier algorithm. Upon testing it was found to have a higher accuracy than when the model used the SVM algorithm. The model used the following features to assist in the prediction: Lexical, Link popularity, webpage content, and DNS features. The dataset was comprised of several dataset sources, including datasets such as PhishTank and Yahoo!โ€™s directory (Sayamber A., and Dixit A., 2014). The model used within this paper has significant use of features that increase the integrity within the modelโ€™s prediction, in addition, the study explains clearly to the reader how the model classifies data using multiple flow charts and diagrams. The primary downfall of this paper is the lack of detail of the accuracy of the model. The report fails to detail exactly what accuracy was produced from the model and what errors regarding False positives were produced. The testing was restricted to only two classification algorithms which additional testing of other classification algorithms may have found the model to be more accurate. Lastly, the detection method of the resource focuses on only a ML model and no external methods of detection. Malicious URL Detection based on Machine Learning This paper formulated by Xuan et al, produced a machine learning model to predict if a URL is malicious or benign. This model used three feature groups to increase the accuracy of the model. These three being lexical, Host-based, and correlated. The model uses two algorithms which are the SVM and RF classifier algorithms. The dataset used for training consists of a total 470,000 URLs, 70,000 or 14.89% of which are known malicious URLs, the other 400,000 or 85.11% being benign URLs. As seen in the figure below, the RF algorithm had the best accuracy of 96% over 100 iterations, the SVM algorithm having a 90% accuracy over 100 iterations (Xuan et al, 2020).
  • 27. SID:XXXXXXX MOD002691 27 This paper conducted significant testing on the ML model used. In addition. The feature groups used within the model were comprehensive in their respected features. The oversights of this study are that not many classification algorithms were tested to identify the most accurate algorithm for the model. This implementation could have improved the accuracy. QR Code Security โ€“ How Secure and Usable Apps Can Protect Users Against Malicious QR Codes This paper formulated by Krombholz et al, consists of a comprehensive look at QR codes and how they can be used as an attack vector by threat actors. This paper tackles the problem in a holistic view, considering both ML and externals security validation techniques. The paper suggests the implementation of Digital signatures to ensure the integrity of the QR codes and applying pre display analysis to analyse the full URL in the case a URL shortener has been applied to presented URLs (Krombholz et al, 2013). This paper outlines the threat of malicious QR codes extremely well, supported by primary research of demographic likelihood of malicious QR code interaction, and secondary research indicating to lack of secure QR code scanners. This literature also describes innovative techniques to provide security, such as modifying the QR code to allow detection of errors with a technique called masking. Although this paper presented some very innovative ideas on how to secure QR code scanners, no implementation for the ideas was attempted which would have demonstrated if the proposed ideas were viable solutions. Figure 7: Testing results (Adapted from Xuan et al, 2020)
  • 28. SID:XXXXXXX MOD002691 28 Secure Real-Time Artificial Intelligence System against Malicious QR Code Links This paper formulated by Al-Zahrani et al, implemented a ML model to detect malicious QR codes. The model itself was tested with a range of algorithms consisting of NB, SVM, LR, KNN and DT where it was discovered that DT had the best accuracy rating. The model was trained of a dataset of 100000 malicious and benign URLs and used one feature group consisting of lexical properties. The research produced an application named BarAI which had a final accuracy of 90.243%. In addition to the implementation, the report detailed many types of attack vectors used within QR codes, such as detailing how threat actors can use a โ€˜barcode-in-barcode attackโ€™ to get victims to interact with malicious URLs (Al-Zahrani et al, 2021). The literature researched the related concepts of QR code security well and conducted a significant amount of testing on different classification algorithms against the model to determine the best to use. In addition, the data was derived from relevant and recent sources increasing the accuracy of the model in current times. However, the final accuracy of the ML model could have been improved to foster a more reliable system. In addition, the dataset used for training was relatively small in comparison, this could have potentially hindered the accuracy of the final model. Secure Real-Time Computational Intelligence System Against Malicious QR Code Links This paper formulated by Heider Wahsheh and Mohammed Al-Zahrani, consisted of the implementation of ML using a multilayer perception artificial neural network (MLP-ANN) algorithm. In addition, fuzzing logic was applied in attempts to detect malicious URLs derived from QR codes. The model used a dataset of 90,000 benign and malicious URLs. The model produced a real-time detection accuracy of 82.9%. real-time in the sense of this ML model means the model is using live data instead of offline historic data. The model used a feature group of lexical properties. The dataset had equal halves of malicious and benign URLs each being 45,000 URLs (Wahsheh, H., and Al-Zahrani, M, 2021).
  • 29. SID:XXXXXXX MOD002691 29 The literature strengths lie within its testing of the programme. The programme was tested against known scanners such as Kaspersky and Norton to see how its security features compared. In addition, its approach to ML was decidedly unique in that it opted to use a real-time artificial intelligence approach instead of a traditional batch model approach. The primary downfall of the implementation was the amount of data. A dataset of 90,000 is relatively small for this type of classification problem and a larger dataset may have produced a higher model accuracy and model integrity. 4.1 Critical Analysis The above literature review was composed of analysing several academic papers which closely follow the concept of my proposed project. The covered papers range in their detail and comprehensiveness. However, All the above sources decided that a critical part of detecting malicious URLs derived from QR codes was a machine learning model. Higher accuracy percentages mostly were dependent on the size of the used dataset, and the testing of multiple algorithms. The primary oversights from most of the papers were the depth of testing conducted. Many papers when determining the algorithm to use, only tested a few algorithms. This is something I intend to remediate when training my model, as testing a range of algorithms will discover which algorithm produces the best accuracy, and therefore making my ML model more effective and capable of achieving its required goal. Secondly, many of the models used had insufficient sized datasets with little detail on the cleaning and preparation of the data. Again, this is something I intent to remediate by using a sufficient sized dataset and ensuring that the data is of good quality, this will ensure my model achieves the best accuracy it is capable of. Moreover, a significant oversight for most papers was the lack of additional validation of the URL outside of the ML model. For example, no online validations such as ensuring a URL has a
  • 30. SID:XXXXXXX MOD002691 30 valid certificate were present, in addition, none of the models implemented additional functions to ensure that a valid protocol was being used for the presented URL such as HTTPS. This is a feature I intent to implement into my system. From this analysis it can be observed that there is significant oversight within the observed literature, I intend to implement the discussed solutions by taking a hybrid approach to the problem. This will use ML as much of the literature used, however, ML alone is not enough to identify malicious QR codes, this is because ML models can be wrong in their predictions, so additional methods should be used in tandem to ensure the integrity of a prediction, to do this, online URL validation will be implemented within my system. These being, PKI certificate validation, and URL format validation, these solutions specifically are important as they ensure real time security validation such as if the URL is using secured protocols such as HTTPS and have a valid certificate for session security and integrity.
  • 31. SID:XXXXXXX MOD002691 31 5.0 Proposed Work For the proposed solution to be created, the three main components must be designed to effectively achieve there aims. For this to be achieved, a ML model must be fostered that can detect malicious URLs. In addition, a function to identify if a URL has a valid PKI certificate must be created. And lastly, a function to validate a URLs format must be created. These components then need to be implemented into a hybrid system that can be used by an end user. 5.1 Methodology This section has been formulated to detail the methodology of the proposed system and detail all the stages related to the implementation. The system serves the function of detecting malicious content within QR codes. The architecture of the system function can be seen in the below figure.
  • 32. SID:XXXXXXX MOD002691 32 Figure 8: Proposed architecture of system function.
  • 33. SID:XXXXXXX MOD002691 33 This methodology will detail how the sections of the system architecture will provide the desired outcomes. The following steps have been adopted in my approach: 1. Machine Learning model to detect malicious URLs. 1.1 Collection of data 1.2 Cleaning and preparation of dataset 1.3 Feature engineering 1.4 Test classifier algorithms against model to determine the most appropriate algorithm. 2. Validating if URL has a valid PKI certificate. 3. Validating if URL format is valid. 4. Creation of QR code reader and system GUI 5.2 Machine Learning model to detect malicious URLs. The first step of the implementation will be the programming of the ML model from which a prediction will be derived. The ML implementation will follow the below steps. 5.2.1 Collection of data The first step is to gather the data that the ML model will use to train. I discovered a dataset on Kaggle that was aligned with the requirements for my ML model, this dataset named Url Dataset (Teseract, 2017), consisted of 420,464 URLs, either assigned a value of good or bad which correspond to benign or malicious. Eugene Dorfman notes, a ML model should apply the 10-time rule to have a sufficient dataset (Dorfman, 2022). Meaning, the dataset should have 10 time the amount of input data as there are parameters within the dataset. As this dataset only has two parameters being the URL and the assigned URL state value, the 10-time rule would
  • 34. SID:XXXXXXX MOD002691 34 require an input set of 20 entries, this dataset far exceeds the minimum requirement, thus giving it ample data to produce accurate predictions. The figure below shows some example data from the dataset. 344,821 or 82.01% of the dataset URLs were assigned the value of good. With the remaining 75,643 or 17.99% being assigned the value of bad. As seen in the below figure. Figure 9: URL Dataset entries. Figure 10: ratio of good and bad URLs in dataset.
  • 35. SID:XXXXXXX MOD002691 35 5.2.2 Cleaning and preparation of dataset After acquiring the dataset, the next stage is too โ€˜Cleanโ€™ the dataset. Kirsten Barkevd notes, cleaning data is the process of modifying or removing data that is incorrect or not relevant to the dataset, not cleaning data can negatively impact the accuracy of a ML model (Barkevd, 2022). Upon analysis of the dataset, it was observed that no cleaning was needed. The below figure show that the dataset had no NULL values, meaning all URLs had either good or bad assigned, if a value had a NULL value, True would be displayed. The next step is preparing the data by splitting the data set into a training set and a testing set. Javatpoint notes, that splitting the data set into testing and training is an essential element of data preparation, the training set is used to train the model and then the test set is used as test data when testing. It is important that the datasets are kept separate as testing a model on the training set will provide inaccurate results as the model is aware of the data pretesting. A common split for the dataset is 80:20 where 20 is the testing set, this is due to the model benefiting from a larger training set as it allows more data for computations, and the testing set can be smaller due to it being a subset of the original dataset for testing. (Javatpoint, n.d.). For my model, I will follow the 80:20 split for the dataset as represented in the figure below. Figure 11: Observing no NULL values in dataset.
  • 36. SID:XXXXXXX MOD002691 36 Figure 12: Figure to show testing and training set split. 5.2.3 Feature Engineering Feature Engineering is an essential part of the ML process as it allows the algorithm to work efficiently with the dataset and enhance the performance of the model (Rosencrance, n.d.). For the proposed ML model, the feature engineering consists of vectorising the dataset for NLP. This allows the model to identify how important specific words are within a URL (Karbhari, 2019). Dremio notes, NLP is the process of converting natural language, such as sentences into numerical data that the ML model can use for analysis (Dremio, n.d.). The specific technique that will be used is Term frequency โ€“ inverse document frequency (TF-IDF). Fatih Karabiber notes, TF-IDF measures the importance of a natural language string. This will be used to identify malicious or benign indicators within a URL. This happens by multiplying a natural language words Term frequency (TF) with the inverse Document Frequency (IDF). TF is equal to the count of times a term is within the data/document, divided by the total number of data/document words. And IDF is used to discover the importance of a word by identifying the number of documents commonly thought of as a โ€˜bag of wordsโ€™ in the larger set of data known as a corpus and dividing this over the total number of documents within the corpus containing Dataset Training Set Testing Set
  • 37. SID:XXXXXXX MOD002691 37 the word (Karabiber, n.d.). The formular Adapted from Fatih Karabiber can be seen below (Karabiber, n.d.). ๐‘‡๐น ๐ผ๐ท๐น = ๐‘‡๐น โˆ— ๐ผ๐ท๐น ๐‘‡๐น = ๐‘๐‘ข๐‘š๐‘๐‘’๐‘Ÿ ๐‘œ๐‘“ ๐‘ก๐‘–๐‘š๐‘’๐‘  ๐‘Ž ๐‘ก๐‘’๐‘Ÿ๐‘š ๐‘Ž๐‘๐‘๐‘’๐‘Ž๐‘Ÿ๐‘  ๐‘–๐‘› ๐‘‘๐‘Ž๐‘ก๐‘Ž ๐‘‡๐‘œ๐‘ก๐‘Ž๐‘™ ๐‘›๐‘ข๐‘š๐‘๐‘’๐‘Ÿ ๐‘œ๐‘“ ๐‘ก๐‘’๐‘Ÿ๐‘š๐‘  ๐‘–๐‘› ๐‘‘๐‘Ž๐‘ก๐‘Ž ๐ผ๐ท๐น = log ( ๐‘๐‘ข๐‘š๐‘๐‘’๐‘Ÿ ๐‘œ๐‘“ ๐‘กโ„Ž๐‘’ ๐‘‘๐‘œ๐‘๐‘ข๐‘š๐‘’๐‘›๐‘ก๐‘  ๐‘–๐‘› ๐‘กโ„Ž๐‘’ ๐‘๐‘œ๐‘Ÿ๐‘๐‘ข๐‘  ๐‘๐‘ข๐‘š๐‘๐‘’๐‘Ÿ ๐‘œ๐‘“ ๐‘‘๐‘œ๐‘๐‘ข๐‘š๐‘’๐‘›๐‘ก๐‘  ๐‘ค๐‘–๐‘กโ„Ž๐‘–๐‘› ๐‘กโ„Ž๐‘’ ๐‘๐‘œ๐‘Ÿ๐‘๐‘ข๐‘  ๐‘๐‘œ๐‘›๐‘ก๐‘Ž๐‘–๐‘›๐‘–๐‘›๐‘” ๐‘กโ„Ž๐‘’ ๐‘ก๐‘’๐‘Ÿ๐‘š ) 5.2.4 Test classifier algorithms against model to determine the most appropriate algorithm. The next stage will be testing different classification algorithms to train the model and determine which of the tested algorithms produce the best accuracy. This is to determine which classification algorithm is most appropriate for my modelโ€™s problem. The algorithms that will be tested will be SVM, NB, DT, RM, and LG which have all been detailed in the background information.
  • 38. SID:XXXXXXX MOD002691 38 5.3 Validating if URL has a valid PKI certificate. The second stage of the implementation will be the security validation function that will determine if a URL has a valid certificate. This will be achieved by creating a function that takes advantage of the Python library Requests. A function will be created which sends a HTTP request to the provided URL, the Requests import then determines if the URL has a valid certificate, if it is a response of 200 will be returned. Umbraco notes, a returning of 200 equal the status code โ€˜OKโ€™, meaning the request was successful (Umbraco, n.d.). The function will only return 200 if it validated a certificate, if not the function will return a SSLError (Pypi, n.d.). The below figure illustrates the functionality of the programme. Figure 13: Illustration of certificate validation.
  • 39. SID:XXXXXXX MOD002691 39 5.4 Validating if URL format is valid. The third stage of the implementation will be the second security validation function. The purpose of this function is to validate that the URL is formatted correctly. This utilizes the Python library validators. This import searches a provided URL for specific parameters to ensure it is valid. Utilizing validators python library to determine if URL components are properly formatted. Afzaal Ahmad Zeeshan notes, validators achieve this by ensuring that the URL has a valid protocol such as HTTP or HTTPS and has a resource associated with the address. This is in accordance with RFC 1738 (Zeeshan, 2022). An adapted section of the validators.url source code can be seen below and detailed in the below table: Figure 14: Validators source code (Adapted from validators, n.d.)
  • 40. SID:XXXXXXX MOD002691 40 Section of Source Code Description # protocol identifier As seen in the top section of the code, the code identifies if the URL is using a valid protocol such as HTTPS or File transfer Protocol (FTP) which is a host-to-host file transferring protocol (Fortinet, n.d.). # IP address exclusion Below this we can see that the code is checking the URL is not resolving to a private address from the classes A (10.0.0.0 โ€“ 10.255.255.255), B (172.16.0.0 โ€“ 172.31.255.255) or C (192.168.0.0 โ€“ 192.168.255.255) (Avast, n.d.). and is within the public address space. # Resource path The final action of code seen ensures that the URL has a valid resource that the user is navigated to. Table 4: Details on validator source code This will be used within a function to determine if a URL is valid or invalid, an illustration on how the function will determine this can be seen in the below figure.
  • 41. SID:XXXXXXX MOD002691 41 5.5 Creation of QR code reader and system GUI The last step of the implementation will consist of the user interface and scanner. The methodology applied to build the QR code scanner will be to adopt the Kivy library and take advantage of its features which allow interaction with the device camera. From this the input can be decoded, and a derived URL can be found. For the GUI, Kivy will again be adopted for its cross-platform capabilities allowing it to be used on any device, The system will follow a simple design to increase usability and efficiency of the system. Below can be seen a navigation map to which the system user interface (UI) will follow. Figure 15: Illustration of URL validation.
  • 42. SID:XXXXXXX MOD002691 42 For a visual representation of what the final system GUI will look like, the below wireframe diagram can be seen. Figure 16: Navigation map of system. Figure 17: Wireframe diagram of GUI (iPhone template adapted from unblast, n.d.)
  • 43. SID:XXXXXXX MOD002691 43 6.0 Implementation and Results Development environment To programme the proposed system, an integrated development environment (IDE) will be used to aid in the development process, the development environment of choice is Visual Studio Code. Microsoft notes, that Visual Studio Code is a powerful and comprehensive development environment (Microsoft, 2023). The reason I have selected Visual Studio Code for this project is due to my personal familiarity with the software. In addition to using Visual Studio Code. JupyterLab will be used to aid in the development of the machine learning code. Jupyter notes, jupyter notebook allows for configuration and arranging of workflows in data science (jupyter, n.d.). Meaning, jupyter notebook can be used to test and configure the developed machine learning code in a dedicated environment. Python For the programming language used to build this system, Python was selected. Python is a high- level programming language that is extremely versatile in its functionality. Python can be used in multiple cyber security related domains, ranging from malware analysis to penetration testing (CyberWarrior, 2023). Due to this it is a highly sought after skill in cyber security professionals. Forbes notes, Python as the number one in demand programming language of 2023 (Forbes, 2023). Due to the high demand in Python programming ability, I decided that the Python language would be a suitable language to create the system with. Not only will using Python increase my ability within the language. Buit in addition, the vast array of Python imports and library allow additional functionality to the system such as the ability to build cross platform GUIs. This is in addition to the range of cyber security and network security imports that will assist in building this system.
  • 44. SID:XXXXXXX MOD002691 44 Python Libraries Python allows users to import Python libraries. According to docs.python.org, libraries: โ€œProvide standardized solutions for many problems that occur in everyday programming.โ€ (docs.python.org, n.d.) From this it can be understood that Python Libraries are predefined useful functions that mitigate the need to rewrite commonly used code. Within the development of my system, a range of libraries will be imported to assist in the development of the code. The most important ones to the development are listed in the below table:
  • 45. SID:XXXXXXX MOD002691 45 Library Description Sklearn Scikit-Learn.org notes, that sklearn is a python library which allows users to build machine learning programmes with Python (Scikit Learn, n.d.). Sklearn will be used for the development of the projects machine learning programme to predict malicious URLs. Kivy Kivy notes, that the Kivy python library allows for the development of cross platform applications programmed in Python (Kivy, n.d.). Kivy is essential for the development of my system as it allows cross platform functionality and GUI creation. Validators Read the Docs notes, that the validator collection is a Python library that can be used to validate the type and contents from a provided input value (Read the Docs, n.d.). I will be using the validators library within my programme to ensure that a provided URL derived from a provided QR code is correctly formatted. Requests Pypi.org notes, that the requests Python library is used to send HTTP requests (pypi.org, n.d.). I will be using the request library to send a HTTP request to a URL derived from a provided QR code. I will use the provided response to determine of the URL has a valid PKI Certificate. Table 5: Utilized python libraries.
  • 46. SID:XXXXXXX MOD002691 46 6.1 Implementation of ML model to detect malicious URLs. The first stage of the ML section of the programme was importing the necessary libraries. The libraries utilised mainly consisted of Sklearn derivatives, consisting of all the algorithms that were tested and imports that allow the model to be constructed and trained. In addition, other imports such as pandas, matplotlib and numpy were used for data manipulation and visualization, the imports can be seen in the below figure. After importing all necessary libraries, the next step was to access the โ€˜urldataโ€™ dataset explained in the methodology, this was accessed via a panda function as seen in the figure below. Figure 18: Imported libraries for ML model. Figure 19: โ€˜urldataโ€™ dataset manipulation.
  • 47. SID:XXXXXXX MOD002691 47 Upon completion of the data cleaning, the data next needed to be prepared for training. This was done by splitting the dataset into an input and output set. The input set consisting of the โ€˜urlโ€™ values and the output set consisting of the โ€˜labelโ€™ values containing either good or bad. The sets are named in this way as the input set is the feature we wish to predict and the output set contains the outcomes of an input value (Spark code hub, n.d.). y is used to denote the output set and X is used for the input set; however, the input set must be vectorized so the input set is stored in the variable โ€˜urlsโ€™ The data splitting can be seen in the below figure. After splitting the dataset into the input and output set, the data must be vectorised for feature engineering via NLP as it is in string format. To do this we apply TF-IDF vectorization to the data as explained in the methodology, this allows our data to be computed. Once the tfidfVectorizer() function has been implemented, this can be applied to the input set as seen below. Now the dataset has been prepared and NLP has been applied, the next step is to split the input and output set in to testing and training set, as explained in the methodology, this is to ensure that the model can be trained to a high accuracy with good integrity. As seen in the below figure, the input and output sets have both been split into testing and training sets via the train_test_split() function, with the testing sets being 20% of the dataset and the training set having 80%, the raindom_state has been applied to ensure that the data is randomised and doesnโ€™t produce false accuracy from a class imbalance problem (Pramoditha, 2022). Figure 20: splitting dataset into input and output set. Figure 21: Vectorizing data with TF-IDF
  • 48. SID:XXXXXXX MOD002691 48 At this stage the data has been cleaned and prepared and is now ready to be applied to a model for training. The first model used the Naรฏve Bayes algorithm. The model was first defined, and the algorithm was applied, next the fit method was applied to train the model with the training datasets. Once the model had been trained, the model predictions via the predict function from the input set were stored in the y_pred variable. Once complete, the classification_report function was used with the testing output set and the y_pred set to test the modelโ€™s accuracy. This function tests the modelโ€™s accuracy on a range of variables to determine an accuracy rating. This applied method was used for all the algorithms and resulting in the following accuracy ratings seen in the below figure. Figure 22: Splitting data for testing and training. Figure 23: NB, RF, SVM, LR and DT model reports.
  • 49. SID:XXXXXXX MOD002691 49 The below table summarises the accuracy of the different tested algorithms. Classification Algorithm Accuracy Percentage SVM 98% DT 97% LR 96% NB 95% RF 82% Table 6: Accuracy of algorithms summary From the testing conducted it was discovered that the highest accuracy was produced by the model applied with the SVM algorithm. However, the algorithm ultimately chosen for the programme was LR with 96% accuracy. This was due to the following factors, SVM while producing a very high accuracy, took a substantial amount of time to predict, which would not be efficient and would discourage interaction with the system, DT while again had high accuracy, had a lower precision than LR when determining โ€˜badโ€™ URLs. Which is significant as this system needs to be as risk averse as possible when predicting malicious URLs. Due to the stated reasons the model utilising LR has been used for the final model implementation which had the highest true positive (TP) accuracy at identifying malicious URLs of the top three algorithms, this is illustrated in the below confusion matrix.
  • 50. SID:XXXXXXX MOD002691 50 Once the final model was implemented a function was defined to allow a URL to be passed as an argument, the URL is then vectorised for NLP and predicted against the model. The model would then return a value for the output set being โ€˜goodโ€™ or โ€˜badโ€™. Once the value was returned an โ€˜IFโ€™ statement would return either โ€˜Clearโ€™ or โ€˜Maliciousโ€™ from the function depending on the prediction. This function can be seen in the below figure. Figure 24: Confusion matrix for LR. Figure 25: ML model prediction function.
  • 51. SID:XXXXXXX MOD002691 51 6.1.1 Testing model predictions against known malicious and safe URLs. Open Phish is a website that collect known malicious URLs. (OpenPhish, n.d.) ten of these URL have been predicted by my ML model. As can be seen, from the provided known bad URLs the model identified all of them correctly, However, with the known good URLs, the model identified one of them incorrectly, giving this test a 95% accuracy. Disclaimer: The malicious URLs presented in the below table should only be accessed in a safe environment. I the author of this report hold no responsibility for the damages caused by a reader accessing the detailed URLs.
  • 52. SID:XXXXXXX MOD002691 52 URL Prediction Type Correct / Incorrect https://clainmask.com Malicious Malicious Correct https://login-dana-id.giixzg.me/ Malicious Malicious Correct http://havkeye14.wixsite.com/my-site-1/ Malicious Malicious Correct https://dovzzt.n0c.world/bpost2/ Malicious Malicious Correct https://cg96358.tw1.ru/?return_url=https://www.orange.fr/portail&_Auth Malicious Malicious Correct https://accshmesrvc0log.github.io/ Malicious Malicious Correct https://one.link/annushka_almazova Malicious Malicious Correct https://dev-kejomokkeflushah.pantheonsite.io/att/att.html Malicious Malicious Correct http://havkeye14.wixsite.com/my-site-1/ Malicious Malicious Correct https://innovative-divya.github.io/Netflix-clone Malicious Malicious Correct https://en.m.wikipedia.org Clear Clear Correct https://stackoverflow.com/questions/58356254/machine-learning Clear Clear Correct https://www.loveholidays.com/sem/cheap.html?WT.mc_id=pgo-35492155817 Malicious Safe Incorrect https://www.autotrader.co.uk/car-search?postcode=i77777&price-from=10000&price- to=15000&make=Mercedes-Benz&advertising-location=at_cars&page=1 Clear Clear Correct https://teams.microsoft.com Clear Clear Correct https://sts.anglia.ac.uk/adfs/ls/idpinitiatedsignon.aspx?SAMLRequest Clear Clear Correct https://500px.com/photo/49283436/chicago-looking-up-by-alex-dibrova Clear Clear Correct https://kids.nationalgeographic.com/animals Clear Clear Correct https://id.cisco.com/ Clear Clear Correct https://twitter.com/?lang=en Clear Clear Correct Table 7: ML model predictions of provided URLs.
  • 53. SID:XXXXXXX MOD002691 53 6.2 Implementation of URL PKI certificate validation. The first step in the implementation of the URL PKI certificate validation function was to import the relevant libraries. As detailed in the methodology the requests import will be used to send a HTTP GET request to a provided URL. The requests import can be seen in the below figure. Once the relevant library was imported a function was defined to pass a URL as an argument. This function first defined an empty variable. This was to allow the variable to be accessed outside of the try except scope which was implemented after a persistent connection error was found. Implementing the try except allowed the function to complete as needed. In the try clause the response variable uses the requests.get() function to determine if the URL has a valid certificate. The function will only return OK or <Response [200]> if a valid certificate is present, if not it will return an SSLError. The except method was to prevent the connection error from stopping the function and then passes the response variable to the next section. Here the variable is turned into a string and an โ€˜IFโ€™ statement determine what the response is. If the string is exactly equal to <Response [200]> the function will return โ€˜Clearโ€™ as a valid certificate is present, if an error is returned, the function will return โ€˜Invalidโ€™ as no certificate was found. Figure 26: PKI Certificate validation function import.
  • 54. SID:XXXXXXX MOD002691 54 6.2.1 Testing function against known valid and invalid certificates. Badssl is a website that hosts invalid certificates for testing purposes (badssl, n.d.). This function was tested against six known bad URLs and six known good URLs as seen in the below figure. Figure 27: URL PKI Certificate validation function. Figure 28: Tested good and bad certificates.
  • 55. SID:XXXXXXX MOD002691 55 The below table details the responses form testing the URLs. As can be observed, the function has a 100% accuracy on the presented testbed. URL Response Type Correct / Incorrect Invalid 1 SSLError Expired Correct Invalid 2 SSLError Wrong Host Correct Invalid 3 SSLError Self-Signed Correct Invalid 4 SSLError Untrusted Correct Invalid 5 SSLError Revoked Correct Invalid 6 SSLError Pinning-test Correct Valid 1 <Response [200]> Valid Certificate Correct Valid 2 <Response [200]> Valid Certificate Correct Valid 3 <Response [200]> Valid Certificate Correct Valid 4 <Response [200]> Valid Certificate Correct Valid 5 <Response [200]> Valid Certificate Correct Valid 6 <Response [200]> Valid Certificate Correct Table 8: Testing PKI Certificate validation function. 6.3 Implementation of URL format validation The first step in the implementation of the URL format validation function was to import the necessary libraries. As detailed in the methodology, the validators import will be utilized within a function to validate the format of a provided URL. The import can be seen in the below figure. Figure 29: URL format validation function import.
  • 56. SID:XXXXXXX MOD002691 56 Upon importing of the library, a function was defined that takes a URL as an argument, the validators.url() function which validates the URL is then applied to the passed URL and stored in a variable, Within the variable is a Boolean value of either True or False, the variable output is then changed into a string on which an โ€˜IFโ€™ statement is conducted to determine if the output is True which means the URL format is valid. Or False, which means the URL format is invalid. If the URL format is valid the function will return โ€˜Clearโ€™ else the function will return โ€˜Invalidโ€™. The function can be seen in the below figure. Figure 30: URL format validation function.
  • 57. SID:XXXXXXX MOD002691 57 6.3.1 Testing function against known valid and invalid format URLs. The below table details the responses form testing the URLs. As can be observed, the function has a 100% accuracy on the presented testbed. URL Format Result Correct / Incorrect https:/autocars.com Invalid Invalid Correct https://www.google. Invalid Invalid Correct httpb://github.com Invalid Invalid Correct https://www.youtube/com Invalid Invalid Correct http:||www.udemy.com Invalid Invalid Correct https;//www.linkedin.com Invalid Invalid Correct https://www.youtube.com Valid Valid Correct https://www.amazon.co.uk Valid Valid Correct https://www.gymshark.com Valid Valid Correct https://whois.is Valid Valid Correct https://teams.microsoft.com Valid Valid Correct https://wordcounter.net Valid Valid Correct Table 9: Testing URL format validation function
  • 58. SID:XXXXXXX MOD002691 58 6.4 Implementation of the QR code scanner The first stage of the QR code scanner implementation was to import the related libraries, as defined in the methodology, the Kivy library has been used to allow this system to work cross platform, in addition Kivy also allows access to a device camera through branches of the library. To be able to effectively recognise and decode a provide QR code, the pyzbar library has been imported also. Pypi notes, pyzbar allows reading of barcodes and QR codes (pypi, n.d.). As can be seen below: After the necessary imports have been implemented, the QR code scanner itself can be created. Firstly, a template for the scanner is created by creating a new class which inherits the argument App which allows it access to the Kivy library functionality. Once the class is defined, a new function is created that uses the kivy builder function to load the output stored in the variable Scanner. The scanner variable consists of a multi-line string that imports the needed libraries and defines the layout for the camera window under MDBoxLayout. In addition the ZBarCam object is also defined here which uses the id:qrcodecam to load the native device camera and allows QR codes to be recognised. Below this the ZBarSymbol is used to define the types of codes the scanner can recognise. Lastly an object that allows decoding of a QR code has been defined which is calling a function defined below and is calling all the function arguments. The function in question is below the builder function and firstly checks to make sure that a QR code is present. If a QR code is present the function passes the output to the next function which firstly defines a variable as global allowing global access, then stores the decoded data within this variable using the decode() function. The variable is made global to allow access to the decoded URL throughout the code. As can be seen in the below figure the class allows the camera to be used to scan and decode QR codes. Figure 31: QR code scanner imports.
  • 59. SID:XXXXXXX MOD002691 59 Figure 32: QR code scanner working example.
  • 60. SID:XXXXXXX MOD002691 60 For the purposes of the above figure, a QR code was generated with the URL https://ExampleURL.com, as can be observed, the programme accessed the device camera and printed the decoded data as output, proving that the scanner can identify and decode QR codes. The main class for the QR code scanner with its related functions can be seen in the figure below. Figure 33: QR code scanner code.
  • 61. SID:XXXXXXX MOD002691 61 6.5 Implementation of Graphical user interface The first step of the GUI implementation was to import the necessary libraries. Kivy, as before has been utilized significantly for its cross-platform GUI capabilities, a range of Kivy derivative have been used such as Gridlayout features for the GUI layout and button features to allow button functionality for the โ€˜Continueโ€™ and โ€˜Returnโ€™ buttons. In addition to the Kivy modules, the import Webbrowser has been used which allows the programme to open up a web browser (docs.python, n.d.). In this case, this will be used to open a URL after scanning. Lastly, as the GUI utilises all the main components of the system, the three main components have been imported to this file, these being the URL format validation function, PKI Certificate validation function, and lastly the ML model prediction function. The imports can be seen in the below figure. After importing the relevant modules and libraries, the first step was to store the output link from the scanner by calling the print_global_link() function from the QR code scanner. The output was then stored in the variable ScanThisURL. Now that the link has been stored in a variable, the three main component functions can be imported, and the link can be passed to each function as an argument to allow the individual scans to be run on the provided link. After Figure 34: GUI imports.
  • 62. SID:XXXXXXX MOD002691 62 each scan has been finished the retuned values are turned into strings and stored in variables as seen in the below figure. Now that the returned values from each component have been stored in variables, the window to display the output needs to be created. By utilizing the Kivy Popup() function, a popup window was defined to display after a QR code is scanned. Within this popup window the Kivy GridLayout function was utilized to arrange the GUI components on the screen. TopGrid was defined with one column, this allowed for the title and passed URL to be displayed at the top centre of the GUI. Next another Grid was defined name EmbbeddedGrid with two columns and was embedded into the first grid, this allowed the second grid to have two columns without effecting the objects within the TopGrid. Within EmbeddedGrid, the first column consisted of the names of each scan and the second grid is where the returned values from each component have been displayed. This can be seen in the below figure. Figure 35: GUI code segment 1
  • 63. SID:XXXXXXX MOD002691 63 At this point the GUI can display the title, scanned URL, and results of the scan. The next part of the implementation was to define two buttons which can be used to either return to the scanner or continue to the scanned URL. In addition, if the scans determine that a URL is malicious a warning should be applied to the screen. First, the Continue button was defined by using the Kivy Button() function, once the design elements were applied the button was bind to the con() function, which used the open_new() function to open the past URL argument in a web browser if the button is pressed. Once the button was bind to the function, the button was displayed on the GUI with the add_widget() function. In addition, a variable named โ€˜CWarningโ€™ was appended to the button text, this variable contains a warning dependent on if the scans were all clear. The Continue button related code can be seen in the below figure. Figure 36: GUI code segment 2.
  • 64. SID:XXXXXXX MOD002691 64 Lastly, the Return button was implemented, this button followed the same design as the first however it accessed the Popup up function and bind the dismiss function to the button if the button was pressed. In addition, instead of applying the โ€˜CWarningโ€™ variable to the text, the Return button has the โ€˜RWarningโ€™ variable applied. The code for the return button can be seen below. Figure 37: GUI code segment 3 Figure 38: GUI code segment 4.
  • 65. SID:XXXXXXX MOD002691 65 7.0 Testcases The below testcases are testing the ability of the complete system. is important to note that not all combination of output have been tested as this is not practical. Such as the combination of a valid certificate with an invalid URL will not be produced. Testcase 1 Description Pass / Fail URL provided has: Valid Certificate, Valid URL format and is safe. The application is expected to return Clear, Clear, and Clear respectively Pass Evidence QR Code: https://en.wikipedia.org/wiki/QR_code Application response Table 10: Testcase 1
  • 66. SID:XXXXXXX MOD002691 66 Testcase 2 Description Pass / Fail URL provided has: Valid Certificate, Valid URL format and is Malicious. The application is expected to return Clear, Clear, and Malicious respectively Pass Evidence QR Code: http://maliciouswebsitetest.com Application response Table 11: Testcase 2
  • 67. SID:XXXXXXX MOD002691 67 Testcase 3 Description Pass / Fail URL provided has: Invalid Certificate, invalid URL format and is Malicious. The application is expected to return Invalid, Invalid, and Malicious respectively Pass Evidence QR Code: https://ExampleBadURL,com Application response Table 12: Testcase 3
  • 68. SID:XXXXXXX MOD002691 68 Testcase 4 Description Pass / Fail URL provided has: Invalid Certificate, Valid URL format and is malicious. The application is expected to return Invalid, Clear, and malicious respectively Pass Evidence QR Code: https://expired.badssl.com Application response Table 13: Testcase 4
  • 69. SID:XXXXXXX MOD002691 69 Testcase 5 Description Pass / Fail GUI Continue button is expected to open derived URL in native web browser Pass Evidence Application response Table 14: Testcase 5
  • 70. SID:XXXXXXX MOD002691 70 Testcase 6 Description Pass / Fail GUI โ€˜Returnโ€™ button is expected to return user to QR code scanner. Pass Evidence Application response Table 15: Testcase 6
  • 71. SID:XXXXXXX MOD002691 71 8.0 Discussion In this study I have discovered how best to identify malicious QR codes accurately and efficiently in efforts to prosper an effective and usable system which can be used to prevent interaction with malicious QR codes. This was achieved by conducting research and analysis on the current literature to identify the best identification methods and in addition what weaknesses were present in the current solutions. From this I identified how to address the oversight to produce a superior system in both the ML accuracy and efficiency. In addition, implementing a hybrid approach which utilized additional programming function to ensure additional prediction integrity outside the ML model. Once the methodology was identified, the system was implemented into an operational system. Extensive testing was conducted to ensure the usability, accuracy, and efficiency of the system. The completed system achieved all specified requirements defined from the original research question. In addition, managed to effectively improve upon all oversights identified in the current literature. From this a highly effective system at identifying malicious URLs derived from QR codes has been created. This systems hybrid approach to identifying malicious URLs allows for a more accurate and holistic prediction opposed to soul reliance on a ML model. Therefore, producing a more suitable solution than anything found within the current literature. It can be observed from the results that the system achieved great prediction accuracy. The ML component of the system boasts a 96% accuracy with a significantly high TP accuracy of 97% ensuring the likelihood of a malicious URL not being identified is extremally low. The model accuracy is higher than any identified within the covered literature, which was in part due to the extensive testing of different classification algorithms to determine which was the most accurate and effective at solving the problem. In addition to the ML, the functions that ensure valid PKI certificates and URL format prospered 100% accuracy against the test bed. These solutions work
  • 72. SID:XXXXXXX MOD002691 72 together to produce an exceptionally high prediction integrity. In addition to the hybrid solution testing, multiple testcases were conducted on the system ensuing the subsystems integrated together correctly and that the GUI worked as expected. From this it was observed that the system was both accurate and efficient at the defined task. From this it can be observed that the system is an extremally viable solution to the original research question and is not just effective in its ability to identify malicious QR codes, but in addition, at being an efficient and usable system by any level of technical ability. However, I do believe there are improvements that could be introduced to the system in the future. In specifically the ML model accuracy and integrity could be further improved. As this project was my first introduction to machine learning there are certain lack of complexities which would have benefited the ML model in its predictions. More advanced feature engineering and selection could be implemented to increase the accuracy of the model, for example, implementing extensive feature groups that identify many aspects of the URL. Moreover, although the programming validation functions are significantly effective, additional function could be implemented, such as a function to check a URL against known databases of malicious URLs for improved prediction integrity. Overall, it can be observed that although there is scope for future improvement, the current system is fit for purpose in all aspects of its function and has achieved all aims of this study and addressed all problems identified.
  • 73. SID:XXXXXXX MOD002691 73 9.0 Conclusion It can be concluded from the discussion that this development project has achieved all aims and requirements originally defined at the beginning of the study. Due to this, I believe that the developed system has real value to the cyber security space as it can prevent a range of malicious cyber security attacks which utilize QR codes as an attack vector. The extent to which each aim of the study has been achieved is detailed below: Aim one was to provide research on the current methods which are being utilized to identify malicious URLs derived from QR codes. As can be observed from chapters 1-4, extensive research and analysis has been conducted upon the current methods used to address this problem, in addition the weaknesses and oversights of the current literature have been identified and mitigation to the issues have been identified. From this it can be concluded that aim one has been successfully achieved. Aim two of the study was to develop a hybrid solution to the research question. It can be concluded from this study content that this aim was successfully achieved. The created system is a superior solution to the current one-dimensional approaches covered in the current literature. The last aim was to conduct extensive testing of different classification alogrithms accuracy when applied to the ML model. Five different algorithms have been tested and detailed to identify the most appropriate algorithm for the model. From this it can be concluded that aim three was achieved.
  • 74. SID:XXXXXXX MOD002691 74 References Abad, S., et al, 2023, Classification of Malicious URLs Using Machine Learning (pdf) Available at: <https://www.mdpi.com/1424-8220/23/18/7760 > [Accessed on 24 February 2024]. Alder, S., 2023, QR Codes Increasingly Used in Phishing Attacks (online) Available at <https://www.hipaajournal.com/qr-codes-increasingly-used-in-phishing-attacks/#:> [Accessed on 4 December 2023]. Aljabri et al, 2017, Detecting Malicious URLs Using Machine Learning Techniques: Review and Research Directions (pdf) Available at: < https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9950508> [Accessed on 13 December 2023]. Al-Zahrani, M., Wahsheh, H., Alsaade, F., 2021, Secure Real-Time Artificial Intelligence System against Malicious QR Code Links (pdf) Available at: < https://www.hindawi.com/journals/scn/2021/5540670/> [Accessed on 14 December 2023]. Anishnama, 2023, Understanding Bidirectional LSTM for Sequential Data Processing (online) Available at: < https://medium.com/@anishnama20/understanding-bidirectional-lstm- for-sequential-data-processing-b83d6283befc#> [Accessed on 24 February 2024]. Avast, n.d., Public vs. Private IP Addresses: Whatโ€™s the Difference? (online) Available at: <https://www.avast.com/c-ip-address-public-vs-private> [Accessed on 19 December 2023]. Badssl.com, n.d., badssl.com (online) Available at: <https://badssl.com/> [Accessed on 3 January 2024]. Barkeved, K., 2022, Data Cleaning: The Most Important Step in Machine Learning (online) Available at: < https://www.obviously.ai/post/data-cleaning-in-machine-learning > [Accessed on 18 December 2023].
  • 75. SID:XXXXXXX MOD002691 75 Cherisien, W., 2024, 17 Creative Ways to Use QR Codes (online) Available at: < https://mention.com/en/blog/creative-ways-to-use-qr-codes/#> [Accessed on 23 February 2024]. Comodo, n.d., What is a PKI Certificate? (online) Available at < https://comodosslstore.com/resources/what-is-a-pki-certificate/> [Accessed on 9 December 2023]. CyberWarrior, 2023, Is Python Good for Cybersecurity? (online) Available at <https://www.cyberwarrior.com/is-python-good-for-cybersecurity/#:> [Accessed on 4 December 2023]. Docs.python,org, n.d., The Python Standard Library (online) Available at <https://docs.python.org/3/library/index.html> [Accessed on 11 December 2023]. Dorfman, E., 2022, How Much Data Is Required for Machine Learning? (online) Available at: < https://postindustria.com/how-much-data-is-required-for-machine-learning/#: > [Accessed on 18 December 2023]. Dremio, n.d., Vectorization in NLP (online) Available at: <https://www.dremio.com/wiki/vectorization-in-nlp/> [Accessed on 19 December 2023]. Forbes, 2023, Partner Should Know: The Top Programming Languages Of 2023 (online) Available at <https://www.forbes.com/sites/forbestechcouncil/2022/12/28/what-your- software-partner-should-know-the-top-programming-languages-of-2023/> [Accessed on 4 December 2023]. Fortinet, n.d., File Transfer Protocol (FTP) Meaning and Definition (online) Available at: <https://www.fortinet.com/resources/cyberglossary/file-transfer-protocol-ftp-meaning> [Accessed on 19 December 2023].
  • 76. SID:XXXXXXX MOD002691 76 Griffiths, C., 2023, The Latest 2023 Phishing Statistics (Updates December 2023) (online) Available at: < https://aag-it.com/the-latest-phishing-statistics/#:> [Accessed on 2 December 2023]. Hughes, L., 2022, SSL and TLS (online) Available at < https://link.springer.com/chapter/10.1007/978-1-4842-7486-6_11> [Accessed on 9 December 2023]. IBM, 2021, The components of a URL (online) Available at <https://www.ibm.com/docs/en/cics- ts/5.1?topic=concepts-components-url> [Accessed on 9 December 2023]. IBM, n.d., What is logistic regression? (online) Available at: < https://www.ibm.com/topics/logistic-regression> [Accessed on 19 December 2023]. IBM, n.d., What is random forest? (online) Available at: < https://www.ibm.com/topics/random- forest> [Accessed on 19 December 2023]. Jafar a, et al, 2018, Machine Learning from Theory to Algorithms: An Overview (pdf) Available at <https://iopscience.iop.org/article/10.1088/1742-6596/1142/1/012012/pdf> [Accessed on 5 December 2023]. Javatpoint, n.d., Train and Test dataset in Machine Learning (online) Available at: <https://www.javatpoint.com/train-and-test-datasets-in-machine-learning> [Accessed on 19 December 2023]. Jha, A., 2023, Vectorization Techniques ion NLP [Guide] (online) Available at < https://neptune.ai/blog/vectorization-techniques-in-nlp-guide> [Accessed on 9 December 2023]. Jupyter, n.d., jupyter (online) Available at < https://jupyter.org/ > [Accessed on 11 December 2023].
  • 77. SID:XXXXXXX MOD002691 77 Karabiber, F., n.d., TF-IDF โ€“ Term Frequency โ€“ Inverse Document Frequency (online) Available at: <https://www.learndatasci.com/glossary/tf-idf-term-frequency-inverse-document- frequency/#:> [Accessed on 19 December 2023]. Karbhari, V., 2019, What is TF-IDF in Feature Engineering? (online) Available at: <https://medium.com/acing-ai/what-is-tf-idf-in-feature-engineering-7f1ba81982bd#> [Accessed on 23 February 2024]. Kivy, n.d., Kivy: The Open Source Python App Development Framework (online) Available at <https://kivy.org/index.html> [Accessed on 11 December 2023]. Krombholz K., Fruhwirt, P., Rieder, T., Kapsalis, I., Ullrich, J., Weippl E., 2013, QR Code Security โ€“ How Secure and Usable Apps Can Protect Users Against Malicious QR Codes (pdf) Available at: < https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7299920> [Accessed on 14 December 2023]. Kumar, S., 2020, Supervised vs Unsupervised vs Reinforcement (online) Available at: <https://www.aitude.com/supervised-vs-unsupervised-vs-reinforcement/#> [Accessed on 23 March 2024.]. Liu, J., 2022, Lexical Features of Economic Legal Policy and News in China Since the COVID- 19 Outbreak (online) Available at: < https://www.frontiersin.org/journals/public- health/articles/10.3389/fpubh.2022.928965/full> [Accessed on 24 February 2024]. Mahesh, B., 2020, Machine Learning Algorithms โ€“ A Review (pdf) Available at: < https://www.researchgate.net/profile/Batta- Mahesh/publication/344717762_Machine_Learning_Algorithms_- A_Review/links/5f8b2365299bf1b53e2d243a/Machine-Learning-Algorithms-A- Review.pdf> [Accessed on 19 December 2023].
  • 78. SID:XXXXXXX MOD002691 78 McAfee, n.d., What is Typosquatting? (online) Available at <https://www.mcafee.com/learn/what-is-typosquatting/#:> [Accessed on 11 December 2023]. Microsoft, 2023, What is a machine learning model? (online) Available at < https://learn.microsoft.com/en-us/windows/ai/windows-ml/what-is-a-machine-learning- model > [Accessed on 9 December 2023]. Microsoft, 2023, What is Visual Studio? (online) Available at <https://learn.microsoft.com/en- us/visualstudio/get-started/visual-studio-ide?view=vs-2022> [Accessed on 11 December 2023]. MonkeyLearn, n.d., Machine Learning (online) Available at: <https://monkeylearn.com/blog/classification-algorithms/#> [Accessed on 23 February 2024]. Naylor, D., n.d., The Cost of the โ€œSโ€ in HTTPS (pdf) Available at <https://dl.acm.org/doi/pdf/10.1145/2674005.2674991> [Accessed on 9 December 2023]. OpenPhish, n.d., OpenPhish (online) Available at: <https://openphish.com/> [Accessed on 3 January 2024]. OSIbeyond, 2023, QR Code Scams: Think Before You Scan (online) Available at: <https://www.osibeyond.com/blog/qr-code-scams/> [Accessed on 26 February 2024]. Pawar, A., et al, 2022, Secure QR Code Scanner to Detect Malicious URL using Machine Learning (pdf) Available at: < https://ieeexplore.ieee.org/Xplore/home.jsp> [Accessed on 24 February 2024]. Phising.org, n.d., What Is Phishing? (online) Available at: < https://www.phishing.org/what-is- phishing> [Accessed on 2 December 2023].