Real Time Analysis of Cyber Attack Hotspots and Trends across the globe

1. Cyber Attacks
Analysis
Shwetha Narayanan
Insight Data Engineering Fellow
New York – Summer 2017
Real Time Analysis of Cyber Attack Hotspots

2. Motivation

3. Streaming Data Source
• Anti-virus software companies
• Augmented data to scale
• 4000 - 6000 events per minute
• Scaled up to 100,000 events per minute

4. Streaming Data Source
• Content
–Attack Type
•Malware
•DDOS
•Backdoor
–Location
Information of
victim and attacker

5. Metric for Hotspot
Analysis
Getis - Ord

6. Getis - Ord
• Used when you have geospatial data
• Calculates statistical significant clusters based
on a feature
• Estimate a Gi Score for every space in the
region
– Higher Gi score => Significant Hotspot
• Compares the feature score of current cell and
it’s neighbors with sum of all feature values

7. Getis Ord – Gi Score
• Steps to Calculate
– Divide the space into
cells
– Accumulate attack
counts in each cell
– Calculate Gi Score
• Blue vs Green
– Blue is surrounded
by cells of higher
attack count
5
3 2
4
1
5
10
14
9
9
10
1
2

8. Interactive Query
• Find events within a
radius of 10 miles
– Calculate Bounding
box
Bounding box
• min(x), max(x), min(y), max(y)
• Based on earth’s spherical
radius at that point

9. Data Pipeline
Cyber Attacks
Streaming Data
Source

10. Demo

11. Kafka Streams Technical Challenges
• Streams application should provide Serializers
and Deserializers to materialize the data
– Read input from stream / Write to stream
• Built in serializers are: String, Integer, Long,
Double

12. How to work with other data
formats?

13. Deserializer for other data formats

14. Creating a Serde - SerializerDeserializer

15. Kafka Streams Technical Challenges
• Kafka Streams Errors
– Internal Topic Error - Cannot create internal
topics
• User permissions to create topics - Stack Overflow
• Set Group ID and Application ID
• Used for co-ordinating between instances

16. About Me - Shwetha Narayanan
• Recently graduated with
Masters in Computer Science
• Worked for 2 years as a
Software Engineer
• Co-authored a paper on
“Enabling Real time crime
intelligence using mobile GIS and
prediction methods”, EISIC, 2013

17. Screenshots - Hotspots

18. Screenshots - Cyber Attack Trends

19. Getis Score - Calculation

20. Bounding Box calculation
acos(sin(input_lat) * sin(Lat) + cos(input_lat) *
cos(Lat) * cos(Lon - (input_lon))) * 6371 <=
1000;