This document discusses a study analyzing social media and stylometric features to identify spearphishing emails. The researchers extracted features from emails and linkedin profiles of recipients to classify emails as spearphishing, spam, or benign. Stylometric features from emails like attachment size and subject richness achieved over 96% accuracy distinguishing spearphishing from spam/benign emails. While social media features did not significantly improve classification, the researchers note attackers may use additional social networks to craft more effective spearphishing emails.

1. Analyzing Social and Stylometric Features to
Identify Spearphishing Emails
Prateek Dewan, Anand Kashyap,
Ponnurangam Kumaraguru
Indraprastha Institute of Information Technology – Delhi (IIITD), India
Unifying the
Global Response
to Cybercrime

2. Unifying the
Global Response
to Cybercrime
Overview
• What is spearphishing?
• Spearphishing and Online Social Media
• Challenges and dataset
• Feature extraction
• Classification results
• Discussion
1

3. What is spearphishing?
Unifying the
Global Response
to Cybercrime
• Targeted phishing attack
• Contains contextual content instead of random
messages
• Harder to detect, since spearphishing emails look
more genuine
• Victims are asked to
• Download malicious attachments
• Reply with sensitive information
• Click on URLs
• …
2

4. Why study spearphishing?
• Victims are 4.5 times more likely to fall for spear
phishing, than normal phishing [1].
• One of the main entry points for Advanced
Persistent Threats.
• Causes losses worth millions.
[1] M. Jakobsson. Modeling and preventing phishing attacks. In Financial Cryptography, volume 5. Citeseer, 2005.
Unifying the
Global Response
to Cybercrime
3

5. Spearphishing and social media
• Social media profiles can be a good source for
the “context” part of spear phishing emails
• FBI warning on July 04, 20131
• “…emails typically contain accurate information about
victims obtained from data posted on social networking
sites…”
1 http://www.computerweekly.com/news/2240187487/FBI-warns-of-increased-spear-phishing-attacks
Unifying the
Global Response
to Cybercrime
4

6. Unifying the
Global Response
to Cybercrime
Data
• Emails
• Spear phishing emails (Symantec)
• Spam / phishing emails (Symantec)
• Benign emails (Enron)
• LinkedIn profiles
• Recipients of emails in the three datasets mentioned
above
• LinkedIn People Search API
5

7. Challenges (social features)
• Limited information about victim to identify her on
social media
• Only first name, last name, organization available from
Unifying the
Global Response
to Cybercrime
victim’s email ID
• Hard to find victim on Facebook, Twitter, Google+
• Too many profiles with same first name, last name
• Work field not searchable.
6

8. Challenges (social features) contd.
• LinkedIn – Only network which provides searching
using work field
• People search API access restricted.
• We requested for access under their Vetted API access
Unifying the
Global Response
to Cybercrime
scheme.
• Rate limited
• Only 100 requests per day per app
7

9. Unifying the
Global Response
to Cybercrime
Dataset
• Emails sent to employees of 14 international
organizations
• SPEAR (Targeted spear phishing emails from Symantec)
• 4,742 emails à 2,434 victims / LinkedIn profiles
• SPAM (Spam / phishing emails from Symantec)
• 9,353 emails à 5,912 victims / LinkedIn profiles
• BENIGN (Sample from Enron email corpus)
• 6,601 emails à 1,240 victims / LinkedIn profiles
8

10. Feature set creation
Final
feature
vector
Unifying the
Global Response
to Cybercrime
SPAM
SPEAR
BENIGN
Stylometric
features
from
emails
1. firstName
2. lastName
3. organization
http://api.linkedin.com/v1/people-search:
LinkedIn
Profile(s)
Social
features
from
LinkedIn
Recipient
email
address
9

11. Stylometric Features
Unifying the
Global Response
to Cybercrime
• Subject based (7)
• Num. words, Num. characters, Richness
• Has words: “bank”, “verify”
• isReply, isForwarded
• Attachment based (2)
• Length of attachment name
• Attachment size
• Body based (9)
• Num. words, Num. characters, Num. unique words
• Has words: “attach”, “suspension”, “verify your account”
• Num. newlines, Richness, function words
10

12. Unifying the
Global Response
to Cybercrime
Social Features
• Location
• Connections
• Summary based (5)
• Num. words, Num. Characters, Num. unique words
• Length, Richness
• Profession based (2)
• Job Level (0-7)
• Job Type (0-9)
11

13. Results (SPEAR v/s SPAM)
Unifying the
Global Response
to Cybercrime
Feature Set (num.
features)
Classifier Random Forest J48 Decision
Tree
Naïve Bayes
Subject (7) Accuracy (%) 83.91 83.10 58.87
FP Rate 0.208 0.227 0.371
Attachment (2) Accuracy (%) 97.86 96.69 69.15
FP Rate 0.035 0.046 0.218
All email (9) Accuracy (%) 98.28 97.32 68.69
FP Rate 0.024 0.035 0.221
Social (9) Accuracy (%) 81.73 76.63 65.85
FP Rate 0.229 0.356 0.445
Email + Social (18) Accuracy (%) 96.47 95.90 69.35
FP Rate 0.052 0.054 0.232
12

14. Results (SPEAR v/s SPAM) contd.
Unifying the
Global Response
to Cybercrime
• Most informative features
• Attachment size
• Length of attachment name
• Subject Richness
• No. of characters in subject
• Location (from LinkedIn profile)
• No. of words in subject
• LinkedIn connections
• …
13

15. Results (SPEAR v/s SPAM) contd.
Unifying the
Global Response
to Cybercrime
14

16. SPEAR v/s SPAM subjects
ß Spam / phishing
Spear phishing à
15

17. Results (SPEAR v/s BENIGN)
Unifying the
Global Response
to Cybercrime
Feature Set (num.
features)
Classifier Random Forest J48 Decision
Tree
Naïve Bayes
Subject (7) Accuracy (%) 81.19 81.11 61.75
FP Rate 0.210 0.217 0.489
Body(9) Accuracy (%) 97.17 95.62 53.81
FP Rate 0.031 0.048 0.338
All email (16) Accuracy (%) 97.39 95.84 54.14
FP Rate 0.029 0.044 0.334
Social (9) Accuracy (%) 94.48 91.79 69.76
FP Rate 0.067 0.103 0.278
Email + Social (25) Accuracy (%) 97.04 95.28 57.27
FP Rate 0.032 0.052 0.316
16

18. Results (SPEAR v/s BENIGN) contd.
Unifying the
Global Response
to Cybercrime
• Most informative features
• Body richness
• No. of characters in body
• No. of words in body
• No. of unique words in body
• Location (from LinkedIn)
• No. of newlines in body
• Subject richness
• …
17

19. Results (SPEAR v/s SPAM + BENIGN)
Unifying the
Global Response
to Cybercrime
Feature Set (num.
features)
Classifier Random Forest J48 Decision
Tree
Naïve Bayes
Subject (7) Accuracy (%) 86.48 86.35 77.99
FP Rate 0.333 0.352 0.681
Social (9) Accuracy (%) 88.04 84.69 74.46
FP Rate 0.241 0.371 0.454
Email + Social (16) Accuracy (%) 89.86 88.38 73.97
FP Rate 0.202 0.248 0.381
18

20. Results (SPEAR v/s SPAM + BENIGN) contd.
Unifying the
Global Response
to Cybercrime
• Most informative features
• Subject richness
• No. of characters in subject
• Location (from LinkedIn)
• LinkedIn connections
• No. of words in subject
• Email forwarded? (True / false)
• Email is a reply? (True / false)
• …
19

21. Unifying the
Global Response
to Cybercrime
Discussion
• Social features features (from LinkedIn) did not help in
distinguishing spear phishing emails from non spear
phishing emails.
• Stylometric features from emails suffice to do so.
• Real world scenarios may be much different
• Attackers may use information from other sources / social
networks, viz. Facebook, Twitter, etc.
• Dataset limitation
• It is possible that no spear phishing mails in our dataset were
crafted using LinkedIn features
• We cannot conclude that such behavior would not
be found outside our dataset, or in future.
20

22. Unifying the
Global Response
to Cybercrime
Thanks!
Prateek Dewan
E: prateekd@iiitd.ac.in
W: http://precog.iiitd.edu.in/people/prateek
21

23. Unifying the
Global Response
to Cybercrime
Backup slides…

24. Results (SPEAR v/s SPAM) contd.
Unifying the
Global Response
to Cybercrime

25. Unifying the
Global Response
to Cybercrime
Attachment names

26. Results (SPEAR v/s BENIGN) contd.
ß Benign emails
Spear phishing à

27. Unifying the
Global Response
to Cybercrime
Attachment types

28. Details of organizations
Unifying the
Global Response
to Cybercrime