This document presents a deep learning approach using a bidirectional GRU and CNN-CRF model for named entity recognition of cyber security entities. The model is evaluated on a cyber security text corpus and achieves state-of-the-art performance with a precision of 90.8%, recall of 96.2%, and F1-score of 93.4%. This outperforms other deep learning models like LSTM-CRF and CNN-CRF. The proposed approach does not require feature engineering and could be extended to relation extraction tasks in the future.

1. Deep Learning Approach for Intelligent Named Entity
Recognition of Cyber Security
Simran K1, Sriram S1, Vinayakumar R2, 1, and Soman KP1
1Center for Computational Engineering and Networking, Amrita School Of Engineering,
Amrita Vishwa Vidyapeetham, Coimbatore, India.
2Division of Biomedical Informatics, Cincinnati Children’s Hospital Centre, Cincinnati, OH,
United States.
December 24, 2019
1 / 10

2. Overview
1 Motivation
2 Background
3 Description of the data set
4 Proposed Architecture
5 Experimentations
6 Results
7 Conclusion and Future work
8 References
2 / 10

3. Motivation
Technology has evolved tremendously in the previous couple of
decades making diverse online web sources (news, forums, blogs,
online databases, etc.) available for common use.
Newfound information regularly seems first on these kinds of online
web sources in unstructured text format.
Mostly, it requires a ton of time, sometimes years, for this information
to properly classify into a structured format.
However, cases related to Cyber Security does not have that kind of
time, so timely extraction of this information plays an imperative job
in numerous applications.
3 / 10

4. Background
Named Entity Recognition (NER) task is recognition of named
entities of a domain, such as products, versions, etc.
The best accuracy given by the current NER devices in this area of
Cyber Security, depend on feature engineering. [3], [4].
[1] compared the performance of LSTM-CRF with CRFSuite.
4 / 10

5. Description of the data set
Cyber Security domain text corpus provided by Bridges et al. [2] is
used in this work.
It comprises of around 40 entity types.
Each line in the data set consists of a word as well as its entity type.
The data set was divided into three subsets that is training, validation,
and testing consisting of 70%, 10%, and 20% data respectively.
5 / 10

6. Proposed Architecture
Figure: Proposed Architecture.
6 / 10

7. Experimentations
TensorFlow with Keras framework were used to implement all the
deep learning architectures.
Various deep learning architectures were implemented and their
performance was compared with Bidirectional GRU+CNN-CRF.
LSTM, RNN, GRU, Bidirectional GRU contains 128 hidden units.
Embedding dimension 128, Stochastic Gradient Descent (SGD)
learning method, and learning rate of 0.005 are used in all the
experiments.
Precision, Recall, F1-Score are utilized in order to evaluate the
performance.
7 / 10

8. Results
Table: Average weighted performance metrics for all entity types.
Model Precision Recall F1-score
LSTM-CRF [1] 85.3 94.1 89.5
CRF [1] 82.4 83.3 82.8
CNN-CRF 83.1 93.9 88.2
RNN-CRF 83.5 85.6 84.5
GRU-CRF 86.5 95.7 90.9
Bidirectional GRU-CRF 88.7 95.4 91.9
Bidirectional GRU+CNN-CRF 90.8 96.2 93.4
8 / 10

9. Conclusion and Future work
This work evaluates the performance of various deep learning
frameworks along with CRF model for NER Cyber Security text data.
Bidirectional GRU+CNN-CRF model performed better than other
deep learning architecture.
The best part about the proposed architecture is that it does not
require any feature engineering.
This work can be extended for Relation Extraction.
9 / 10

10. References
Gasmi, Houssem, Abdelaziz Bouras, and Jannik Laval. ”LSTM Recurrent Neural
Networks for Cybersecurity Named Entity Recognition.” ICSEA 2018 (2018): 11.
R. A. Bridges, C. L. Jones, M. D. Iannacone, K. M. Testa, and J. R. Goodall,
”Automatic labeling for entity extraction in cyber security,” arXiv preprint
arXiv:1308.4941, 2013.
V. Mulwad, W. Li, A. Joshi, T. Finin, and K. Viswanathan, “Extracting information
about security vulnerabilities from web text,” in Proceedings of the 2011
IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent
Agent Technology - Volume 03, ser. WI-IAT ’11. Washington, DC, USA: IEEE
Computer Society, 2011, pp. 257–260. [Online]. Available:
http://dx.doi.org/10.1109/WI-IAT.2011.26
J. R. Finkel, T. Grenager, and C. Manning, “Incorporating non-local information
into information extraction systems by Gibbs sampling,” in Proceedings of the 43rd
Annual Meeting on Association for Computational Linguistics,ser. ACL ’05.
Stroudsburg, PA,USA: Association for Computational Linguistics,2005, pp.
363–370. [Online]. Available: http://dx.doi.org/10.3115/1219840.1219885 s
10 / 10