hskdhfs

1. Query Conversion Service
Securonix
A training report
Submitted in partial fulfillment of the requirements for the award of degree of
Bachelor of Technology
(Computer Science and Engineering)
Submitted to
LOVELY PROFESSIONAL UNIVERSITY
PHAGWARA, PUNJAB
From 03/25/21 to till date
SUBMITTED BY
Name of student: Aryan Agarwal Submitted to:
Registration Number: 11707334 Name of Supervisor: Dr. Parampreet Kaur
Signature of the student:
Designation

2. Securonix India Private Limited.
Corporate Office: “The HUB”, Ground Floor, Sy No. 8&8/2, Ambalipura Village, Varthur Hobli Sarjapura Main Road, Bengaluru – 560 103
Pune : Beta 1 Building, 2nd Floor, Gigaspace IT Park, Viman Nagar, Pune – 41101
Security Intelligence Delivered.
Annexure-IX (c): Declaration by the supervisors
To whom so ever it may concern
This is to certify that Aryan Agarwal, 11707334 from Lovely Professional University, Phagwara, Punjab, is working as an Intern
at Securonix on “ Spotter Query Parser” under my supervision from March, 2021 to till date. It is further stated that the work
carried out by the student is a recordof original work to the best of my knowledge for the partial fulfillment of the
requirements for the award of the degree, degree name.
Mallikarjuna Reddy Gondireddy
Name of External Supervisor Name of Internal Supervisor
Senior Software Engineer
Designation of the External Supervisor Designation of the Internal Supervisor
Signature of the external Supervisor Signature of the Internal Supervisor
Dated:11-08-2021. Dated:

3. To whom so ever it may concern
I, Aryan Agarwal, 11707334, hereby declare that the work done byme on “Spotter
Query Parser” from March, 2021 to till date, under the supervision of Mallikarjuna
Reddy Gondireddy, Senior Software Engineer, Securonix and Dr. Parampreet
Kaur, Designation, Lovely professional University, Phagwara, Punjab, is a record of
original work for the partial fulfillment of the requirements for the award of the degree,
B.Tech. Computer Science and Engineering.
Aryan Agarwal (11707334)
Signature of the student:
Dated: 12/08/2021

4. C. ACKNOWLEDGEMENT
This report is the overview of my work as Intern at Securonix. This work would not be possible
without the guidance and supervision of people who has helped me throughout my internship.
I would like to thanks my supervisor and team at Securonix who helped me and guided me in
the work. It was a fun and learning experience for me.
I would also like to thanks my mentor at Lovely Professional University for their supervision.
Lastly, I would like to thanks Lovely Professional University for providing me this opportunity
to excel in my career, for the development of my future.

5. LIST OF FIGURES
1.1 Securonix Logo ……………………………………………………………………….2
1.2 Securonix a leader in Gartner Magic Quadrant………………………………………..4
1.3 Executives of Securonix……………………………………………………………….6
1.4 Services by Securonix…………………………………………………………………7
2.1 SNYPR by Securonix………………………………………………………………...12
2.2 Spotter Interface……………………………………………………………………...17
3.1 ANTLR……………………………………………………………………………….23
3.2 Sample ANTLR Grammar…………………………………………………………...25
3.3 Parse Tree for above grammar……………………………………………………….26
3.4 Working of Antlr……………………………………………………………………..27
3.5 SLF4J…………………………………………………………………………………29
3.6 Log4J…………………………………………………………………………………30
3.7 JUnit 5………………………………………………………………………………..32
3.8 JaCoCo……………………………………………………………………………….33

6. LIST OF ABBREVIATIONS
1. SIEM: Security Information and Event Management
2. UEBA: User and Entity Behavior Analytics
3. SOAR: Security Orchestration Automation and Response
4. NDR: Network Detection and Response
5. SDL: Security Data Lake
6. XDR: Extended Detection and Response
7. AWS: Amazon Web Service
8. PDF: Portable Document File
9. XML: Extensible Markup Language
10. CSV: Comma Separated Values
11. RTF: Rich Text File
12. ANTLR: Another Tool for Language Recognition
13. JDK: Java Development Kit
14. JaCoCo: Java Code Coverage

7. 1
INDEX
INDEX .........................................................................................................................................1
1. INTRODUCTION OF THE COMPANY ...................................................................................2
1.1 COMPANY SERVICES ..................................................................................................7
1.2 COMPANY SOLUTIONS...............................................................................................9
1.3 SECURONIX MISSION AND VALUES..........................................................................10
1.4 MORE INFORMATIONS ............................................................................................11
2. INTRODUCTION OF THE PROJECT UNDERTAKEN.............................................................12
2.1 SNYPR.......................................................................................................................12
2.2 SPOTTER...................................................................................................................17
3. WORK DONE.....................................................................................................................22
3.1 LIBRARIES, FRAMEWORKS & TOOLS........................................................................23
3.1.1 ANTLR...............................................................................................................23
3.1.2 LOGGING FRAMEWORK...................................................................................28
3.1.3 JUNIT 5 .............................................................................................................31
3.1.4 JaCoCo..............................................................................................................33
4. CONCLUSION....................................................................................................................34
5. REFERENCES.....................................................................................................................35

8. 2
1. INTRODUCTION OF THE COMPANY
Securonix is a privately held solution provider based in Addison, Texas, USA.
Established in 2007 by a team of experts with information on data security, risk
management and ownership compliant, the company brought its first product to
market in 2011 and has been growing firmly from there. The company currently has
more than 300 employees in North America, EMEA and APJ and a large global
partner network. With a strong focus on building healthy technologies ecosystem,
Securonix offers a large number of integrations with various security solutions as
well maintains strategic partnerships with major consultants and consulting
companies.
1.1 Securonix Logo
As modern corporate networks become less and more integrated, this leaving
them open to new types of complex cyber-attacks, both from external and malicious
characters insiders. Unfortunately, traditional security solutions are no longer
compatible with a very large number of security incidents found, many of which are
false or otherwise it doesn't matter. However, because it is not possible to
differentiate without a wide range (and especially handmade) forensic analysis, even
advanced security analysts can no longer detect and reduce security breach within
the prescribed period. In recent years, this has led to a severe shortage of employees
who have the skills to run Corporate Operations Centers for companies, even for
their own companies their budget. The industry's response to this major problem is

9. 3
next-generation Security Analytics solutions that focus on real-time analysis and
integration of security events across the company network, to find out export stores
and other surprises and thus identified potentially dangerous activities. These
products are affordable eliminate false benefits and provide security analysts with a
small number of possible warnings developed with additional knowledge of the
context of forensic analysis and clearly defined scores. Combined with a high level
of automated workflow and highly improved reporting skills, they are able to
significantly reduce the time required for analysis and reduce cyber threats.
Securonix offers an impressive portfolio of various security statistics products based
on standard. A sub-platform for Security Analytics for data collection, analysis and
visualization details (and more). In our previous look at Executive View, we saw
the Securonix solution as one the most advanced use of the Real-Time Security
Intelligence (RTSI) concept. However, the platform could be based on Big Data
technology and is therefore ubiquitous among other similar solutions, such as is not
intended to be used as a storage solution for long-term security events. In February
2017, the company launched the next generation of SNYPR Security Analytics
Platform, Big Backend data security analytics solution based on Apache Hadoop
and Kafka platforms. New the product removes the limit of long-term storage and
provides customers with an end-to-end solution log management, security
information and event management (SIEM) and user and business conduct statistics
(UEBA) on one platform. The previous generation platform is still available to
customers looking for an analytics solution that complements their existing SIEM
platforms, while SNYPR the platform offers a full-fledged leg of leg and event
management infrastructure.
Securonix provides the Next Generation Security and Information Event
Management (SIEM) solution. As a recognized leader in the SIEM
industry, Securonix helps some of largest organizations globally to detect
sophisticated cyberattacks and rapidly respond to these attacks within minutes. With
the Securonix SNYPR platform, organizations can collect billions of events each
day and analyze them in near real time to detect advanced persistent threats (APTs),
insider threats, privilege account misuses and online fraud. Securonix pioneered the
User and Entity Behavior Analytics (UEBA) market and holds patents in the use of

10. 4
behavioural algorithms to detect malicious activities. The Securonix SNYPR
platform is built on big data Hadoop technologies and is infinitely scalable. Our
platform is used by some of the largest organizations in the financial, healthcare,
pharmaceutical, manufacturing, and federal sectors.
1.2 Securonix a leader in Gartner Magic Quadrant
Securonix provides the Next Generation Security and Information Event
Management (SIEM) solution. As a recognized leader in the SIEM industry,
Securonix helps some of largest organizations globally to detect sophisticated
cyberattacks and rapidly respond to these attacks within minutes. With the
Securonix SNYPR platform, organizations can collect billions of events each day
and analyze them in near real time to detect advanced persistent threats (APTs),
insider threats, privilege account misuses and online fraud. Securonix pioneered the
User and Entity Behavior Analytics (UEBA) market and holds patents in the use of
behavioral algorithms to detect malicious activities. The Securonix SNYPR
platform is built on big data Hadoop technologies and is infinitely scalable. Our
platform is used by some of the largest organizations in the financial, healthcare,
pharmaceutical, manufacturing, and federal sectors.

11. 5
The Securonix platform delivers positive security outcomes with zero infrastructure
to manage. It provides analytics-driven next-generation SIEM, UEBA, and security
data lake capabilities as a pure cloud solution, without compromise. Built on an
open big data platform, Securonix NextGen SIEM provides unlimited scalability
and log management, behavior analytics-based advanced threat detection, and
automated incident response on a single platform. Customers use it to address their
insider threat, cyber threat, cloud security, and application security monitoring
requirements. Securonix UEBA leverages sophisticated machine learning and
behavior analytics to analyze and correlate interactions between users, systems,
applications, IP addresses, and data.
Light, nimble, and quick to deploy, it detects advanced insider threats, cyber threats,
fraud, cloud data compromise, and non-compliance. Built-in automated response
playbooks and customizable case management workflows allow security teams to
respond to threats quickly and accurately. Securonix Security Data Lake is a
massively scalable, fault-tolerant, open data platform that ingests massive amounts
of data per day and supports reliable, economical, long term data retention.
It transforms raw log data into meaningful security insights using super-enriched
data, blazing fast search, and elegant visualizations to uncover comprehensive,
actionable insights into your organization’s security posture.
SNYPR integrates directly with sources of event information enterprises already
have in place. It ingests limitless volumes of data, normalizes, enriches and
processes data at lightning speed, and then analyzes it in real-time using a
combination of user and entity behavior analytics (UEBA), unsupervised deep
learning and applied threat models to deliver true predictive threat detection.
SNYPR is not only the most sophisticated threat detection capability ever released,
it is also steering the entire industry toward a big data analytics approach to
enterprise security.

12. 6
1.3 Executives of Securonix
“SNYPR completely revolutionizes how enterprise organizations discover and
manage cyber threats, and we are honored that this award from respected journalists
recognizes our significant innovation,” said Tanuj Gulati, CTO, Securonix.
“SNYPR delivers a completely new visualization of the enterprise security posture,
harnesses the power of big data and puts actionable intelligence into the hands of
security leaders, enabling them to combat cyber threats and mitigate risk to their
organization with fewer resources and lower costs.”
Securonix is working to radically transform all areas of data security with actionable
security intelligence. Its purpose is to build advanced security analytics technology
mines, enriches, analyzes, scores and visualizes customer data into actionable
intelligence on the highest risk threats from within and outside their environment.
Using signature-less anomaly detection techniques that track users, account and
system behavior, Securonix is able to detect the most advanced data security, insider
threats and fraud attacks automatically and accurately. Globally customers are using
Securonix to address the most basic and complex needs around advanced persistent
threat detection and monitoring, high privileged activity monitoring, enterprise and
web fraud detection, application risk monitoring and access risk management.

13. 7
1.1 COMPANY SERVICES
Securonix offers various services as listed below.
1.4 Services by Securonix
1. Next-Gen Security Information and Event Management (SIEM)
Legacy, signature based SIEMs aren’t effective at detecting advanced
threats. The only way to catch a sophisticated attacker in time is to leverage
advanced analytics within your SIEM. Stay ahead of the attackers by using
technology such as machine learning to give your security team better
insights and less false positives.
Built on big data, Securonix Next-Gen SIEM combines log management;
user and entity behavior analytics (UEBA); and security orchestration,
automation, and response into a complete, end-to-end security operations
platform. It collects massive volumes of data in real time, uses patented
machine learning algorithms to detect advanced threats, and provides
artificial intelligence-based security incident response capabilities for fast
remediation.
2. User and Entity Behavior Analytics (UEBA)
Today, many attacks are specifically built to evade traditional signature-
based defenses, such as file hash matching and malicious domain lists. They
use low and slow tactics, such as dormant or time triggered malware, to

14. 8
infiltrate their targets. The market is flooded with security products that
claim to use advanced analytics or machine learning for better detection and
response. The truth is that all analytics are not created equal.
Securonix UEBA leverages sophisticated machine learning and behavior
analytics to analyze and correlate interactions between users, systems,
applications, IP addresses, and data. Light, nimble, and quick to deploy,
Securonix UEBA detects advanced insider threats, cyber threats, fraud,
cloud data compromise, and non-compliance. Built-in automated response
playbooks and customizable case management workflows allow your
security team to respond to threats quickly, accurately, and efficiently.
3. Security Orchestration Automation and Response (SOAR)
As the attack surface expands, there is a shortage of skilled security
personnel to secure businesses and keep the attackers at bay. Rapid response
is essential to mitigate the risks of cybersecurity threats, but disparate
security tools are cumbersome for security teams to manage, costing time
and effort.
Securonix Security Orchestration, Automation, and Response (SOAR) helps
security operations teams improve their incident response times by providing
automation that adds context and suggesting playbooks and next steps to
guide analysts. SOAR optimizes orchestration by streamlining incident
response with built-in case management, integrations covering over 275
applications, and seamless access to your SIEM, UEBA, and network
detection and response (NDR) solutions in a single pane of glass.
4. Network Detection and Response (NDR)
Network systems have evolved over time. Legacy network protection tools
and firewalls are unable to provide adequate visibility into application traffic
due to factors such as encryption, browser emulation, and advanced evasion
techniques. The traditional methods of detection are labor intensive and
manual, resulting in limited visibility and information overload. Securonix
Network Detection and Response (NDR) gives you the visibility your
security team needs to detect and respond to network-borne threats.

15. 9
Securonix NDR uses analytics powered by machine learning to analyze
network events and alert analysts to anomalies arising from interactions
between users, applications, servers, and network components.
5. Security Data Lake
The SDL, therefore, is a critical component of a next generation SIEM
platform. It provides the scale and storage that enables modern security
solutions. However, some data lake solutions are built on legacy, outdated
technology. One example are data lakes that use relational databases for
storage, which make it impossible for those solutions to deliver the above
capabilities efficiently.
The Securonix Security Data Lake is the core of the Securonix platform,
providing scalability, data security, and searchability. It is a robust, modern
data lake architecture that is fault tolerant, secure, scalable, economical, and
open.
6. Extended Detection and Response (XDR)
Securonix Open XDR provides you with a comprehensive security fabric
that combines the core components required for fast and effective threat
detection and response. Using advanced behavior analytics powered by an
industry-pioneering user and entity behavior analytics (UEBA), Securonix
Open XDR continuously delivers threat detection content aligned to the
MITRE ATT&CK framework. Seamlessly integrated automated response
capabilities, powered by pre-built connectors and playbooks, mitigate
identified threats quickly and efficiently.
1.2 COMPANY SOLUTIONS
Securonix offers various solutions as listed below:
• Application Security
• AWS security monitoring
• Azure security monitoring

16. 10
• Cloud SIEM
• Securonix for Crowdstrike
• Securonix for EMR applications
• Fraud prevention
• Securonix for Healthcare
• Identity analytics and intelligence
• Insider threat
• Securonix for PTC Windchill
• Cloud Security Monitoring
• GCP Security Monitoring
• Office 365 Security Monitoring
• Securonix for Okta
• SAP Security Monitoring[2]
1.3 SECURONIX MISSION AND VALUES
Securonix ongoing mission is to monitor the constantly-shifting threat
landscape, conducting security investigations and developing detection
methods for the latest real-world cyberattacks. It provides advanced security
expertise for the customer’s security operations, including threat hunting and
incident response. It also shares their expertise with the wider community
through Threat Research Reports in order to help you better understand,
detect, and protect yourself against the latest real-world cyberattacks.
Securonix values:
- Customers First: Securonix believe customer’s long-term success is vital
to it long-term success. It collaborates closely with their customers to
understand and provide sustainable value to customer business in order
to ensure both immediate and ongoing success.

17. 11
- Visionary: Securonix is opposed to the status quo — and it is obsessed
with innovating its way forward. That’s what led it to build the first
signature-less user behavior analytics solution and led them to transform
it into a complete security analytics and operations platform to help
organizations detect and respond to advanced threats.
- Collaborative: Securonix believe that no single organization can do it all.
It collaborates with their customers and partners to develop the best in
breed solution to combat advanced threats.
- Pragmatic: Securonix see things as they are. It believes the best way to
build a better security analytics platform is to harness the power of
machine learning on Hadoop. All to deliver unlimited scale, resilience,
and cost-effectiveness as well as the power to predict, detect, and respond
to advanced threats.
- Authoritative: Securonix is writing the rules to deliver on the promise of
next generation SIEM — it has pioneered and is leading the market.[2]
1.4 MORE INFORMATIONS
• Headquarters: Addison, Texas
• Founded in: 2007
• Company Size: 501-1000 employees
• Website: https://www.securonix.com

18. 12
2. INTRODUCTION OF THE PROJECT UNDERTAKEN
The project undertaken is Spotter Query Parser which translate the query entered
by users in spotter service of Snypr platform.
2.1 SNYPR
SNYPR(TM) is a security analytics platform that transforms Big Data into
actionable security intelligence. It delivers the proven power of Securonix
analytics with the speed, scale, and affordable, long-term storage of Hadoop in
a single, out-of-the box solution.
SNYPR ingests petabytes of data generated in large organizations, processes it
and analyzes it in real-time using a combination of user and entity behavior
analytics (UEBA), unsupervised Deep Learning, and threat modeling to deliver
true predictive threat detection and unprecedented historical investigation
capabilities.
2.1 SNYPR by Securonix
SNYPR runs the Securonix technology and all its features natively on Hadoop
and uses Hadoop both as its distributed security analytics engine and long-term
data retention engine. The more data to be ingested and analyzed, the more
Hadoop nodes to be added, the solution scales horizontally as needed.
SNYPR comes as a prepackaged bundle that includes the latest Securonix 5.0
technology and the Cloudera Enterprise. For enterprises, Snypr is a holistic
enterprise security analytics platform that marries best-of-breed Big Data and
analytics technologies. It detects the most sophisticated advanced persistent

19. 13
threats and “low and slow” attacks over extended periods of time. All historical,
security-relevant data is available for investigation.
Securonix SNYPR is the next generation of the company's Security Analytics
Platform, namely the technical basis of the company’s product portfolio.
Advanced security analytics technology designed from scratch to be large,
flexible, and capable of supporting a wide range of data sources across the
business. An important distinction of the Securonix SNYPR platform its
flexibility and extension; provides a wide range of pre-defined threat models and
more than 350 out-of-the-box connectors for identity management and security
data collection tools, access and rights, and duties and infringements arising
from existing company infrastructure.
This allows the product to support almost any data source within the company's
IT infrastructure including networks, devices, applications and cloud services.
For each supported data source, the platform automatically works for relevant
behavior models and statistics. It is also possible to explain custom analysis
models for specific data sources and customer needs. So, the same the platform
is capable of dealing with a wide range of use cases ranging from cyberthreats
and malicious intruders, compliance or fraudulent detection. A large number of
industry-specific business cases can also be supported.
SNYPR's Apache Hadoop- and Kafka-powered backend are the basis of its Big
Data pool that supports large data collection and storage. SNYPR Security Data
Lake is based on open data a model that provides long-term storage of terabytes
of security event data in a traditional way - even data from third-party
applications - that is available for real-time search and analysis at any time.
The only way to deal with this new approach is to increase the hardware
requirements, namely make the first investment in setting up Big Data
infrastructure. Still, it offers customers being able to measure large amounts of
data is much easier and provides more reliability as well instead of saving an
existing log management solution.
The Securonix platform is flexible enough to accommodate retrospective
options, allowing companies to they have limited data requirements and want to

20. 14
maintain their long-term log storage space to choose from a non-Hadoop
backend, with the option to upgrade to later Big Data building. Moving to
SNYPR is specific to existing customers and allows them to store existing data
as well configuration. Both solutions share the same front end, however the new
backend adds several notable one’s development similar to the SPOTTER
search engine, which offers native language, real-time search across a large data
pool.
All information used by the Securonix platform is enhanced with additional
content attributes, which can be automatically downloaded using over 100
defined functions or custom rules. Also, the key functionality is a grant of
ownership - each incoming event is automatically linked to an upcoming ID not
just from corporate user directories, but from other proprietary sources such as
HR programs. Speaking Potential violations of privacy regulations, Securonix
includes a number of privacy enhancements enters the platform, including
encryption and encryption to keep employee details anonymous, geographical
policies, access control by granularity, and a designated privacy officer role,
which are the only ones allowed to disclose the activity involved in a security
incident.
Real-time integration engine in SNYPR is able to bind each security event to a
business within business, be it a user, a device or an organization unit. The basis
for automatic behavior established and maintained for each such business. The
new release offers much better improved behavioral statistics, including 200
new models that threaten integration and analysis security events from users,
devices, apps and other assets and getting better performance over the long term
Attacks on legacy solutions that will not identify you at all. To conduct a forensic
investigation, the remedy includes a special Specialbench Workbench provides
visibility of communication between users, IP addresses, systems, tasks and
more relevant data in the event. Naturally, new data pool technology can greatly
simplify the analyst function by providing real-time access to all security
information collected both in its native format and developed with rich
contextual information. The new native language search engine supports

21. 15
detection and movement between businesses. Each search can be saved as a
dashboard or sent to a variety format. A number of built-in reports and standard
dashboards in recent releases were available extended too.
A number of debugging skills are also used, such as disabling the user account
in Active Directory of the company or blocking the IP of the device in the
company firewall. These activities depend on integration with third-party
security tools, IAM systems, SIEM solutions and other products. Recently,
Securonix has added its own Threat Model Exchange service, which allows
customers access the latest innovations made by the company's research team
and the delivery of new threat models with one. The company also promotes
mass production, by allowing customers to share threatening models and other
information. Naturally, the platform also supports integration with external
threatened intelligence providers.
The Securonix Security Analytics Platform provides truly advanced security
analytics technology collect, analyze and visualize various business and security
information and modify it in practical wisdom. What positions Securonix other
than many other players in this market are platform expansion, a complete set of
out-of-the-box content, and a wide range of connectors and integration with third
party management and security products.
Ability to collect and integrate security events across all IT systems, applications
and even the cloud services, impressive power enrichment capabilities and a
powerful free integration engine customization to ensure that the platform is able
to provide the most complete security analyst incident investigation tools. This
is further enhanced by the built-in privacy controls approved by trade unions in
several countries. Unfortunately, the power to fix the solution works limited
comparisons, relying heavily on custom integration with third-party tools.
With their latest release based on the open and standard Big Data model, the
company has addressed the need for reduction and long-term retention of
companies seeking distribution Solution as an end-to-end solution for log
management, SIEM and Security Analytics. He learned something new backend

22. 16
comes with increased hardware requirements, speaker flexibility allows
Securonix to continue to provide a previous generation solution to customers
who want to complete an existing log the SIEM management platform, at the
same time provides a straightforward approach to development if needed.
Key features of Snypr are:
• Data Enrichment:
All the data ingested by SNYPR is normalized, summarized, and
enriched at time of ingestion with contextually relevant information such as
user, third-party intelligence, and geolocation data.
• Distributed Behavior Analytics:
Leveraging Hadoop’s distributed and scalable nature, SNYPR performs
distributed real-time anomaly detection regardless of the amount of data
coming into the platform.
• Historical Investigation:
With SPOTTER, the investigators can go back in time and understand
who was doing what, when, and why, with all the relevant contextual
information needed to be effective.
• Scalability:
Fully distributed and scalable architecture for data ingestion, processing,
and analytics of petabytes of data with the affordable long-term storage of
Hadoop.

23. 17
• Data Redundancy:
All machine data ingested, processed, and analyzed by SNYPR is
automatically replicated across Hadoop Distributed File System (HDFS)
data nodes to provide fault tolerance.
• Enterprise Management:
With the pre-packaged Cloudera OEM version of SNYPR, use Cloudera
Manager to manage all your Hadoop components from a single pane of glass.
2.2 SPOTTER
Spotter is a lightning fast, natural language search engine that uses normalized
search syntax and visualization techniques to provide threat hunters the tools
they need to investigate current threats and trends, and track advanced persistent
threats over long periods of time. Spotter is built on Apache Lucene™, a java-
based, high-performance text search engine that provides powerful, efficient,
and accurate search capabilities.[1]
2.2 Spotter Interface

24. 18
From the Spotter start screen, you can search for and view threats using various
search filters. You can specify the report format to display information in tables,
as bar charts, bubble charts, and time charts, or view a geographical map.
The Spotter search language encompasses all the search operators and their
functions, arguments, and clauses. Search operators tell SNYPR what to do to
the events you retrieved from the executed search. For example, you will use an
operator to filter unwanted information, evaluate new fields, calculate statistics,
extract more information, or create a chart.
Many search operators have functions and arguments associated with them.
These functions and their arguments are used to specify how the operators act
on your results and which fields they act on. For example, functions can be used
to format the data within a chart, describe the calculated statistics, or specify the
fields to modify and evaluate.
As you search in Spotter, you will begin to identify information and recognize
patterns that can be useful as search-able fields. You can configure Spotter to
recognize new fields or you can create new fields as you search. When you learn
to identity this information and recognize the patterns, you'll be able to search
more efficiently and build more detailed reports.
Spotter support two kind of searches including:
• Raw event searches
• Transforming searches
Raw event searches retrieve events from an index or indexes. This search is used
to analyze a problem or find specific information within your data. Examples of
a raw event search can include:
• Checking error codes

25. 19
• Correlating events
• Investigating security issues
• Analyzing failures
Raw event searches do not usually include search commands, and the results are
typically a list of the event cards.
Transforming searches perform a statistical calculation against a set of results.
These are searches where you first retrieve events from an index, and then pass
the events into one or more search operators. This search requires fields and at
least one set of statistical operators. Examples of a transforming search can
include:
• Getting a count of error events
• Counting the number of times, a user logged in
• Formulating a chart to display the data in various way
Whether you are retrieving raw events or building a report, be mindful of if
you are looking for sparse or dense information:
Sparse: This search looks for single or multiple events that rarely occur
within a large data set. Examples include querying for a unique IP address
or error code.
Dense: This search scans through events and reports on events. Examples
include finding all events associated with an entity or counting all the emails
a user has sent in a given time.

26. 20
Operators tell SNYPR what to do with the data retrieved from the relevant search
executed. The Spotter search language uses eight categories to describe the
search operators:
• Streaming
• Non-streaming
• Distributable streaming
• Centralized streaming
• Transforming
• Generating
• Orchestrating
• Data processing
These categories can fit one or multiple operators. For example, the STATS
operator only fits into the Transforming category, while other operators can fit
into the Streaming and Generating category at the same time.
Command Type queries: There are eight types for all of the search commands,
including:
• Distributable Streaming
• Centralized Streaming
• Transforming
• Generating
• Orchestrating

27. 21
• Data Set Processing
These types are not mutually exclusive. A command might be streaming or
transforming, and also generating.
You can export Spotter search results in several file formats. Some of the
supported file formats are: PDF, XML, CSV, XLS, RTF, TEXT, DOCX, XLSX.

28. 22
3. WORK DONE
In my internship at Securonix for duration of 6 Months from March to
September, I am working on a query parsing service for Spotter which is a
lightning fast, natural language search engine that help in searching the data,
analyzing and generating report of events created in Snypr Platform.
In this project, I was required to create a parser which can take spotter queries
which is entered by the user as an input then convert those queries into other
languages like MySQL, PostgreSQL and Solr queries which can be executed by
system in Snypr applications. The spotter queries are based on natural language
which is very closed the simple English language. Although these queries are
easier for user to learn and use, these queries cannot be directly processed by the
system to generate the response and need to be translated into technical language
which have well defined syntax. This kind of translation of the queries from
spotter language to other well-defined language is what my work is based on.
Before the actual translation of the query there are lots of prechecks required
to validate the query entered by users is correct and there is no syntax error in
spotter query. I have to also take care of all the fields that user was searching for
is actually present in the Snypr system and have to throw the appropriate error
messages if something wrong happens so that user can be notified what went
wrong which prevented the query from executing. This error message was in
simple language so that user can easily understand the problem and rectify the
input query.
The spotter query parser service was created in Java programming language
as a jar library which can be imported into Snypr application and other classes
can use it. I have made use of various java frameworks and library to develop
this project which I will discuss below.

29. 23
3.1 LIBRARIES, FRAMEWORKS & TOOLS
Below is the library used for the development of the project.
3.1.1 ANTLR
ANTLR (ANother Tool for Language Recognition) is a powerful parser
generator for reading, processing, executing, or translating structured text or
binary files. It's widely used to build languages, tools, and frameworks. From a
grammar, ANTLR generates a parser that can build and walk parse trees. [6]
3.1 Antlr
ANTLR is a powerful parser generator that you can use to read, process, execute,
or translate structured text or binary files. It’s widely used in academia and
industry to build all sorts of languages, tools, and frameworks. Twitter search
uses ANTLR for query parsing, with over 2 billion queries a day. The languages
for Hive and Pig, the data warehouse and analysis systems for Hadoop, both use
ANTLR. Lex Machina uses ANTLR for information extraction from legal texts.
Oracle uses ANTLR within SQL Developer IDE and their migration tools.
NetBeans IDE parses C++ with ANTLR. The HQL language in the Hibernate
object-relational mapping framework is built with ANTLR.
Aside from these big-name, high-profile projects, you can build all sorts of
useful tools like configuration file readers, legacy code converters, wiki markup
renderers, and JSON parsers. I’ve built little tools for object-relational database
mappings, describing 3D visualizations, injecting profiling code into Java
source code, and have even done a simple DNA pattern matching example for a
lecture.

30. 24
From a formal language description called a grammar, ANTLR generates a
parser for that language that can automatically build parse trees, which are data
structures representing how a grammar matches the input. ANTLR also
automatically generates tree walkers that you can use to visit the nodes of those
trees to execute application-specific code.[5]
There are thousands of ANTLR downloads a month and it is included on all
Linux and OS X distributions. ANTLR is widely used because it's easy to
understand, powerful, flexible, generates human-readable output, comes with
complete source under the BSD license, and is actively supported.
ANTLR has contributed to the theory and practice of parsing including:
• linear approximate lookahead
• semantic and syntactic predicates
• ANTLRWorks
• tree parsing
• LL(*)
• Adaptive LL(*) in ANTLR v4
Terence Parr is the person behind ANTLR and has been working on ANTLR
since 1989. He is a professor of computer science at the University of San
Francisco.
ANTLR is really two things: a tool that translates your grammar to a parser/lexer
in Java (or other target language) and the runtime needed by the generated
parsers/lexers. Even if you are using the ANTLR Intellij plug-in or
ANTLRWorks to run the ANTLR tool, the generated code will still need the
runtime library.[3]

31. 25
For computer-based language recognition, ANTLR (called antler), or ANother
Tool for Language Recognition, is a parser generator that uses LL (*) to
distinguish. ANTLR is a follower of the Purdue Compiler Construction Tool Set
(PCCTS), which was first developed in 1989, and is under active development.
Its curator is Professor Terence Parr of the University of San Francisco.
3.2 Sample ANTLR Grammar
АNTLR tаkes аs input а grаmmаr thаt sрeсifies а lаnguаge аnd generаtes
аs оutрut sоurсe соde fоr а reсоgnizer оf thаt lаnguаge. While Versiоn 3
suрроrted generаting соde in the рrоgrаmming lаnguаges Аdа95,
АсtiоnSсriрt, С, С#, Jаvа, JаvаSсriрt, Оbjeсtive-С, Рerl, Рythоn, Ruby,
аnd Stаndаrd ML, the сurrent releаse аt рresent оnly tаrgets Jаvа, С#,
С++, JаvаSсriрt, Рythоn, Swift, аnd Gо [4]

32. 26
3.3 Parse Tree for above grammar
АNTLR саn generаte lexers, раrsers, tree раrsers, аnd соmbined lexer-
раrsers. Раrsers саn аutоmаtiсаlly generаte раrse trees оr аbstrасt syntаx
trees, whiсh саn be further рrосessed with tree раrsers. АNTLR рrоvides
а single соnsistent nоtаtiоn fоr sрeсifying lexers, раrsers, аnd tree раrsers.
By defаult, АNTLR reаds а grаmmаr аnd generаtes а reсоgnizer fоr the
lаnguаge defined by the grаmmаr (i.e., а рrоgrаm thаt reаds аn inрut
streаm аnd generаtes аn errоr if the inрut streаm dоes nоt соnfоrm tо the
syntаx sрeсified by the grаmmаr). If there аre nо syntаx errоrs, the defаult
асtiоn is tо simрly exit withоut рrinting аny messаge. In оrder tо dо
sоmething useful with the lаnguаge, асtiоns саn be аttасhed tо grаmmаr
elements in the grаmmаr. These асtiоns аre written in the рrоgrаmming
lаnguаge in whiсh the reсоgnizer is being generаted. When the reсоgnizer
is being generаted, the асtiоns аre embedded in the sоurсe соde оf the
reсоgnizer аt the аррrорriаte роints. Асtiоns саn be used tо build аnd
сheсk symbоl tаbles аnd tо emit instruсtiоns in а tаrget lаnguаge, in the
саse оf а соmрiler.

33. 27
3.4 Working of Antlr
Оther thаn lexers аnd раrsers, АNTLR саn be used tо generаte tree
раrsers. These аre reсоgnizers thаt рrосess аbstrасt syntаx trees, whiсh
саn be аutоmаtiсаlly generаted by раrsers. These tree раrsers аre unique
tо АNTLR аnd helр рrосessing аbstrасt syntаx trees.
These tree parsers can traverse the tree in two different forms, as a visitor or as
a listener. While this parser is walking over the tree, we can process the input as
we required.
Antlr is used in wide range of popular projects. Some of them are:
• Groovy
• Hibernate
• OpenJDK Compiler Grammar project
• Apex, Salesforce.com's programming language.
• The expression evaluator in Numbers, Apple's spreadsheet.
• Twitter's search query language.
• Apache Cassandra
• MySQL Workbench

34. 28
3.1.2 LOGGING FRAMEWORK
Logging is the activity of recording log messages at the time of execution of the
program into a centralized location. This logging enables you to report and
continue error and warning messages as well as information messages (e.g.,
turnaround times) so that messages can be retrieved and analyzed.
The object which performs the logging in applications is typically just called
Logger.
There are various levels at which log can be taken. The log levels define the
severity of a message. The Level class is used to define which messages should
be written to the log. There are following log level with severity higher to lower:
1. Fatal
2. Error
3. Warning
4. Info
5. Debug
6. Trace
Last two levels are debug level logging.
For this project I have used SLF4J and Log4J logging framework. Let discuss
few things about it below:
SLF4J:
The Simple Logging Facade for Java (SLF4J) serves as a simple facade or
abstraction for various logging frameworks, such as java.util.logging, logback
and log4j. SLF4J allows the end-user to plug in the desired logging framework
at deployment time. Note that SLF4J-enabling your library/application implies
the addition of only a single mandatory dependency, namely slf4j-api-2.0.0-
alpha3.jar.[7]

35. 29
Simple Logging Facade for Java (abbreviated SLF4J) – acts as a facade for
different logging frameworks (e.g. java.util.logging, logback, Log4j). It offers a
generic API making the logging independent of the actual implementation.
3.5 SLF4J
This allows for different logging frameworks to coexist. It also helps migrate
from one framework to another. Finally, apart from standardized API, it also
offers some “syntactic sugar”.
Following are the advantages of SLF4J:
• Using SLF4J framework, you can migrate to the desired logging framework
at the time of deployment.
• Slf4J provides bindings to all popular logging frameworks such as log4j,
JUL, Simple logging and, NOP. Therefore, you can switch to any of these
popular frameworks at the time of deployment.
• SLF4J provides support to parameterized logging messages irrespective of
the binding you use.
• Since SLF4J decouples application and logging framework, you can easily
write applications independent of logging frameworks. You need not bother
about the logging framework being used to write an application.
• SLF4J provides a simple Java tool known as migrator. Using this tool, you
can migrate existing projects, which use logging frame works like Jakarta
Commons Logging (JCL) or, log4j or, Java.util.logging (JUL) to SLF4J.

36. 30
LOG4J:
Apache Log4j is a Java-based logging utility. It was originally written by Ceki
Gülcü and is part of the Apache Logging Services project of the Apache Software
Foundation. Log4j is one of several Java logging frameworks.[9]
3.6 Log4J
Log4j is a reliable, fast and flexible logging framework (APIs) written in Java,
which is distributed under the Apache Software License. log4j is a popular logging
package written in Java. log4j has been ported to the C, C++, C#, Perl, Python,
Ruby, and Eiffel languages.
Inserting log statements into code is a low-tech method for debugging it. It may also
be the only way because debuggers are not always available or applicable. This is
usually the case for multithreaded applications and distributed applications at large.
Experience indicates that logging was an important component of the development
cycle. It offers several advantages. It provides precise context about a run of the
application. Once inserted into the code, the generation of logging output requires
no human intervention. Moreover, log output can be saved in persistent medium to
be studied at a later time. In addition to its use in the development cycle, a
sufficiently rich logging package can also be viewed as an auditing tool.

37. 31
Features of Log4J frameworks are:
1. It is thread-safe
2. It is optimized for speed
3. It is based on a named logger hierarchy
4. It supports internationalization
5. It supports multiple outputs of appenders per logger
6. It is not restricted to a predefined set of facilities
7. The format of the log output can be easily altered by extending the Layout
class
8. It is designed to manage Java exceptions from the start
9. Behavior of logging can be set at runtime using a configuration file
3.1.3 JUNIT 5
In computer programming, unit testing is a software testing method by which
individual units of source code, sets of one or more computer program modules
together with associated control data, usage procedures, and operating
procedures, are tested to determine whether they are fit for use.
Unit tests are typically automated tests written and run by software developers
to ensure that a section of an application (known as the "unit") meets its design
and behaves as intended. In procedural programming, a unit could be an entire
module, but it is more commonly an individual function or procedure. In object-
oriented programming, a unit is often an entire interface, such as a class, or an
individual method. By writing tests first for the smallest testable units, then the
compound behaviors between those, one can build up comprehensive tests for
complex applications.
During the upgrade, the software developer may test the terms, or results, known
as positive, to ensure that the unit is accurate. During the application of the test

38. 32
case, the log testing bodies fail any conditions and report it briefly. Because of
this, the most widely used method is test - function - the expected value.
Writing and storing unit tests can be done quickly using parameterized tests.
This allows for multiple tests to be performed on multiple input sets, thus
minimizing duplication of test code. Unlike traditional unit tests, which are
usually closed methods and unusual test scenarios, tests performed with
parameters take into account any set of parameters. The parameter test is
supported by TestNG, JUnit and its .Net counterpart, XUnit. Appropriate unit
test parameters can be provided manually or in some cases automatically
generated by the test framework. In recent years support has been added by
writing more robust tests (units), using theoretical concept, test cases performing
the same steps, but using test data performed during operation, unlike
standardized tests using the same action steps with pre-defined input sets.
For the unit testing, I have used JUnit framework.
3.7 JUnit 5
JUnit is one of the most popular unit-testing frameworks in the Java ecosystem.
The JUnit 5 version contains a number of exciting innovations, with the goal to
support new features in Java 8 and above, as well as enabling many different
styles of testing. JUnit 5 is a opensource and next generation of JUnit for Java 8
and greater versions.[8]
JUnit 5 is composed of several different modules from three different sub-
projects:

39. 33
JUnit Platform: The platform is responsible for launching testing frameworks
on the JVM. It defines a stable and powerful interface between JUnit and its
client such as build tools.
The final objective is how its clients get integrated easily with JUnit in
discovering and executing the tests.
JUnit Jupiter: This module includes new programming and extension models
for writing tests in JUnit 5.
JUnit Vintage: Supports running JUnit 3 and JUnit 4 based tests on the JUnit 5
platform.
3.1.4 JaCoCo
JaCoCo is a free code coverage library for Java, which has been created by the
EclEmma team based on the lessons learned from using and integration existing
libraries for many years.[10]
3.8 JaCoCo
Code coverage is a software metric used to measure how many lines of our code
are executed during automated tests.
Jacoco is an open source project, which can be used to check production code
for test code coverage. It creates reports and integrates well with IDEs like the
Eclipse IDE. Integration is also available for other IDEs and continuous
integration environments. So there are also Gradle, SonarQube and Jenkins

40. 34
plugins to make these code coverage checks outside the IDE and therefore
globally available to the development team.
4. CONCLUSION
The internship at Securonix have been a great learning journey. It helped me a
lot not only in improving my technical skills but also improved my industrial
exposure and cooperate mindset. This internship is teaching me a lot of new
technologies and opportunity to work on a multifaceted project. During the
internship I was mentored by very capable and talented engineers who made me
explore many new technologies and ways of doing things which helped me in
not only writing better code but also maintainable and clear code with proper
code style which is easier to read and understand.
The internship taught me to importance of work discipline and commitment to
my work and completing the work within the deadline and under pressure.
Going forward, I will continue with this internship and work along side with
other engineers at the company and learn and contribute to the product line of
Securonix and learn lots of new things along the way.

41. 35
5. REFERENCES
[1] https://documentation.securonix.com/onlinedoc/Content/Cloud/Content/ SNYP
R/Guide/ Spotter_Intro.html
[2] https://www.securonix.com/company/about/
[3] Parr, Terence (January 15, 2013), The Definitive ANTLR 4 Reference (1st ed.),
Pragmatic Bookshelf, p. 328
[4] Parr, T.J.; Quong, R.W. (July 1995). "ANTLR: A Predicated-LL(k) Parser
Generator". Software: Practice and Experience. 25 (7): 789–810.
[5] Wikipedia.com
[6] antlr.org
[7] http://www.slf4j.org/manual.html
[8] https://junit.org/junit5/
[9] https://logging.apache.org/log4j
[10] https://www.jacoco.org/jacoco