Office 365 Incident Response 2019 B-Sides Orlando

Office 365 Incident Response
B-Sides Orlando 2019
Alex Parsons @ParsonsProject
1

@ParsonsProject
Intro/Disclaimer
+ Alex Parsons
− Senior Consultant in Incident Response & Digital
Forensics for Aon’s Cyber Solutions (Formerly called
Stroz Friedberg)
− Lives in Seattle; from Pennsylvania
− Knows a lot about Microsoft technologies and Office 365
− Wrote one of the first papers on Windows 10 Forensics
− Doesn’t know everything about Office 365
− Used to own a Windows Phone 
− Opinions are my own and not Aon’s
@ParsonsProject

@ParsonsProject
Goals
+ Go over:
− O365 Basics
− Compromise Basics
− Collection Details
− Proactive Steps
− New Tricks
− Learn from my pain
− We use a basic compromise
example, but applicable for
other cases.
Assumption is you don’t have a SIEM connection in place.

@ParsonsProject
TL;DR
+ Place holds on your compromised Mailboxes
+ Check your Azure Sign in Logs
+ Export your Audit Logs correctly
+ Use HAWK:
− https://www.powershellgallery.com/packages/HAWK/1.0.0
+ Use Azure AD Conditional Access for prevention
+ Check out the new “MailItemsAccessed” operation
+ Enable Multi-Factor Authentication (MFA)*

@ParsonsProject
What is Office 365?
+ Simple Idea from 2010
− Bring Microsoft’s on-premise servers to the cloud
− Mail Servers
− SharePoint Servers
− Microsoft Lync/Skype for Business
− Add Office Web Apps (like Google Docs)
− Oh, and offer regular Office 2010 too
5

@ParsonsProject
Wait, but what IS SharePoint?
+ Whatever you want it to be! (And it’s normally terribly designed)
+ Custom Websites
+ Custom Forms
+ Team Sites
+ OneDrive for Business

@ParsonsProject
Does O365 do anything interesting though?
+ Since 2010 Microsoft has done a LOT
− More services are becoming O365 only
− OneDrive
− Microsoft Teams
− Yammer
− Planner
− Sway
− Flow
− Stream
− Much, much more

@ParsonsProject
Attacker’s End Goal
Subject: Re: March 2019 Invoice
To: Charles@client.com
From: Kent@acme.com
OR (Ransomware)

@ParsonsProject
Attackers are getting more Advanced

@ParsonsProject
Compromise Lifecycle
Attacker Sends Phish
• User Clicks on link,
gives away credentials.
Attacker Sends more
phishing e-mails from
trusted accounts, adds
Inbox Rules
• Additional users click on
phishing links
• Users don’t see e-mails
because the inbox rules
Attacker Sends Wire
Transfer request from
compromised user. Adds
Mailbox Rules
• Receiver of Wire
Transfer request trusts
the e-mail, sends the
money
Attacker uses all
Compromised accounts
to spread phishing
Campaign
• Customers/Clients click
on phishing links and
the cycle continues
New-InboxRule -StopProcessingRules:$True -
AlwaysDeleteOutlookRulesBlob:$False -Force:$False -Name ...
MarkAsRead:$True -DeleteMessage:$True -
SubjectOrBodyContainsWords "delivery failure"; "don't
open";"you have been
hacked";error;spam;hacked;docusign;10/08/2017; wire
Day 1 Day 5

@ParsonsProject
Scenario – Batman is Compromised
Batman@batman.com
Subject: Batman has shared a
document with you!
Ironman
From: Ironman@avengers.com
To: Accounting@starkindustries.com
Subject: Urgent Wire Request
Hey, we need $10b for another avengers movie ASAP, can you
wire this over by EOD? I promise we’ll do better this time.
New Bank Wire info is attached
Spiderman
Thor
Subject: Spiderman
has shared a
document with you!
Subject: THOR HAMMER
RAFFLE ANNOUNCEMENT
SMTP Forwarding enabled and sent
to TotallyNotJoker@Jok3r.com

@ParsonsProject
Questions Lawyers & CISOs have
+ USER LEVEL
− What did the attacker access?
− How long did the attacker have access?
− Is there potential for PII and/or PHI Exposure?
+ TENANT LEVEL
− How many other accounts are affected?
− Has the attacker been kicked out of the environment?
− Do we know the motive of the attacker?

@ParsonsProject
Scenario – How do you Prioritize?
What should be Collected first?
Time To Live for logs in default environments
− Deleted Mail 14 days (Unless you place a hold on the mailbox)
− Azure Active Directory Sign-ins: 7-30 days (Depends on what you pay for)
− Audit Logs: 90-180 days (Depends on what you pay for)
− Message Trace Logs: 90 days
− Exchange Audit Logs: 90 days if enabled

@ParsonsProject
O365 Incident Collection Checklist –Take a Picture
User Level
☐ Place a Preservation Hold & Collect a Mailbox
☐ Azure AD Sign Ins & Reports (7-30* days)
☐ Current Inbox Rules
☐ Current Forwarding Rules
☐ Unified Audit Logs (90-180* days)
☐ Mailbox Audit Logs (90 days)
☐ Message Trace Logs (90 days)
Tenant Level
☐ Last Password Change Report
☐ Unified Audit Logs Filtered for *-InboxRules operations
☐ Unified Audit Logs filtered for Set-Mailbox Events (Occurs when
SMTP forwarding is enabled)
☐ OAuth Application Report
☐ All CURRENT Inbox Rules (and hidden rules)
☐ All CURRENT Forwarding rules
☐ Unified Audit Logs matching known Malicious IPs

@ParsonsProject
Tying Questions to Artifacts
USER - What did the attacker access?
• Unified Audit Logs
• Mailbox Audit Logs with MailItemsAccessed Operation (+Trace Logs & PSTs to match IDs to emails)
• Azure AD Sign Ins
• Message Trace Logs (If Mail Forwarding occurred)
USER - How long did the attacker have access?
• Audit Logs (Within 90 days)
• PSTs (If compromise occurred Outside of Audit log retention period)
• User Report (Contains most recent password change)
• Azure AD Sign Ins
• Message Trace Logs (If Mail Forwarding occurred)
USER - Is there potential for PII and/or PHI Exposure
• PST PII Analysis (Filtered to the appropriate timeframe or e-mails accessed listed in Mailbox Audit Logs)
TENANT - How many other accounts are affected? / Are the attackers kicked out?
•Inbox Rules + Forwarding Settings (CURRENT & Historical Audit Logs)
•Trace Logs of Compromised Users
•Tenant-wide Audit Logs (Inbox Rules, Set-Mailbox, Logs with known IP Subnets; If Requested, ALL Logins or ALL Events)
•PSTs (If outside of 90 days)
•Azure AD Sign In Logs
•Oauth Application Report
TENANT - Do we know the motive of the attacker?
•All of the above

@ParsonsProject
Fun Seattle Fact #1

@ParsonsProject
Placing a hold on the Mailbox
+ TechNet Link
+ If you download you must use Microsoft Edge/IE

@ParsonsProject
Azure Active Directory Sign-Ins
+ Very quick win if data is within your time frame. (See TTL)
+ Every O365 environnent has Azure Active Directory
+ Look for Foreign logons and/or cloud providers
+ Acquire AD Sign-in logs @ portal.azure.com
+ Want to look at your own Sign-ins as a non-admin? You can!
+ There are no Sign-Out events

@ParsonsProject
Checking for Persistence Mechanisms
+ Check All Current Inbox/Mailbox rules
+ Check to see if any Current Inbox Rules are forwarding to an attacker
(Script)
+ Collect Last Password Change Info (Script)
+ Check if any mailboxes are currently being forwarded (Link)
+ Check OAuth Report in Azure AD
+ Beware of the Skeletons in your closet

@ParsonsProject
Hidden Inbox Rules 
+ Hidden Inbox Rules are now here!
+ Complex method but still possible (MFCMapi)
+ Forwarded emails still tracked in Message Trace Logs
+ Detect using MFCMapi or HAWK
+ Remediate with:
Source: https://blog.compass-security.com/2018/09/hidden-inbox-rules-in-microsoft-exchange/

@ParsonsProject
Unified Audit Logs
Guess which of these three are enabled by default?

@ParsonsProject
Unified Audit Logs
+ Unified Audit Logs detail user activity across the entire O365 environment
+ Office 365 Audit Logs are very useful but very frustrating
+ Audit Logs are not enabled by default*
+ Exchange/Mail related logs are not enabled by default*
+ JSON with nested JSON

@ParsonsProject
Mailbox/Exchange Audit Logs
Action Description Admin Delegate Owner
Copy An item is copied to another folder. Yes No No
Create An item is created in the Calendar,
Contacts, Notes, or Tasks folder in
the mailbox; for example, a new
meeting request is created. Note
that message or folder creation isn't
audited.
Yes* Yes* Yes
FolderBind A mailbox folder is accessed. Yes* Yes** No
HardDelete An item is deleted permanently from
the Recoverable Items folder.
Yes* Yes* Yes
MailboxLogin The user signed in to their mailbox. No No Yes***
MessageBind An item is accessed in the reading
pane or opened.
Yes No No
Move An item is moved to another folder. Yes* Yes Yes
MoveToDeletedItems An item is moved to the Deleted
Items folder.
Yes* Yes Yes
SendAs A message is sent using Send As
permissions.
Yes* Yes* No
SendOnBehalf A message is sent using Send on
Behalf permissions.
Yes* Yes No
SoftDelete An item is deleted from the Deleted
Items folder.
Yes* Yes* Yes
Update An item's properties are updated. Yes* Yes* Yes
Source: https://technet.microsoft.com/en-
us/library/ff461937(v=exchg.160).aspx
• MS has begun enabling Mailbox Audit Logs by default (Is taking months to roll out)

@ParsonsProject
Enabling Mailbox Audit Logs
Get-Mailbox -ResultSize Unlimited -Filter
{RecipientTypeDetails -eq "UserMailbox"} | Set-
Mailbox -AuditEnabled $true –AuditOwner “Create,
Update, HardDelete, MailboxLogin, Move,
MoveToDeletedItems, SoftDelete, MailItemsAccessed”
OR (In Theory)
Get-Mailbox -ResultSize Unlimited -Filter
{RecipientTypeDetails -eq "UserMailbox"} | Set-Mailbox -
DefaultAuditSet Admin,Delegate,Owner

@ParsonsProject
What if my logging wasn’t enabled in time?
+ If you go to your tenant and find that Unified Audit Logging and/or
mailbox logging is disabled at the time of an incident you MIGHT be
able to get some events still
+ The secret is within “Search-AdminAuditLog”
+ URL: https://docs.microsoft.com/en-
us/powershell/module/exchange/policy-and-compliance-audit/search-
adminauditlog?view=exchange-ps

@ParsonsProject
Unified Audit Logs Continued
{"CreationTime":"2018-03-12T21:02:46","Id":"b0f7472d-4830-4b7a-8fc8-
08d5425c9b00","Operation":"MailboxLogin","OrganizationId":"88af9a01-
997d-4990-8895-
25d100f62ba5","RecordType":2,"ResultStatus":"Succeeded","UserKey":"10
543BFFD9B5F8EDF","UserType":0,"Version":1,"Workload":"Exchange","User
Id":"aparsons@contoso.com","ClientIPAddress":“187.36.51.3","ClientInf
oString":"Client=/owa/SuiteServiceProxy.aspx; Mozilla/5.0 (Windows
NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/58.0.3029.110 Safari/537.36
Edge/16.16299","ExternalAccess":false,"InternalLogonType":0,"LogonTy
pe":0,"LogonUserSid":"S-1-5-21-4210148372-1463556831-2082377497-
6089575","MailboxGuid":"64288e9b-0bfd-42cc-b08f-
0007f8630d51","MailboxOwnerSid":"S-1-5-21-4010148372-1463556831-
2083377497-
6089575","MailboxOwnerUPN":"aparsons@contoso.com","OrganizationName":
"stroz.contoso.com","OriginatingServer":"DM5PR17MB1322"}

@ParsonsProject
Audit Logs Continued
{"CreationTime":"2018-03-12T21:02:41","Id":"701ae50c-7da5-49fd-ccf2-
08d5885c9879","Operation":"FilePreviewed","OrganizationId":"88af9a01-997d-
4990-8895-
25d100f62ba5","RecordType":6,"UserKey":"i:0h.f|membership|1003bffd9b5f8edf
@live.com","UserType":0,"Version":1,"Workload":"OneDrive","ClientIP":"
187.36.51.3","ObjectId":"https://contoso-
my.sharepoint.com/personal/aparsons_contoso_onmicrosoft_com/Documents/
Sensitive
data.docx","UserId":"aparsons@contoso.onmicrosoft.com","CorrelationId":"1a
708197-8123-43ec-b593-
1bae34e6432a","EventSource":"SharePoint","ItemType":"File","ListId":"8dd3b
323-d4e3-444d-9b33-adf13a56a411","ListItemUniqueId":"015cb92a-ea29-4bd8-
8650-8d965406047f","Site":"7a952c9d-8c29-471d-8d3a-
9b698639db45","UserAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110
Safari/537.36 Edge/16.16299","WebId":"577deac0-7c7e-4c60-9525-
942ac37d08ce","SourceFileExtension":"docx","SiteUrl":"https://contoso-
my.sharepoint.com/personal/aparsons_contoso_onmicrosoft_com/","SourceFi
leName":"Sensitive data.docx","SourceRelativeUrl":"Documents"}

@ParsonsProject
Pivoting with Audit Log Analysis
+ Take your Audit logs and do some IP lookups
− Identify suspicious countries
− Audit Logs (Protection.Office.com)
− Azure AD Sign In Logs (Portal.Azure.com)
− Identify suspicious Ips
− Proxy Providers
− Cloud Providers
− Identify common User Agents
","ClientIPAddress":“187.36.51.3
","ClientInfoString":"Client=/o
wa/SuiteServiceProxy.aspx;
Mozilla/5.0 (Windows NT 10.0;
Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko)
Chrome/58.0.3029.110
Safari/537.36 Edge/16.16299"

@ParsonsProject
Fun Fact #2 –Precipitation by City

@ParsonsProject
Acquiring Unified Audit Logs (Without a SIEM)
1. Never trust the Audit log GUI
2. Never trust the Audit log GUI
3. Never ever trust the Audit Log GUI
4. ALWAYS Acquire Audit logs via PowerShell
Audit Log GUI Issues
− It will only export up to 50,000 lines per request and will not warn you
− It sometimes won’t get all of the audit logs and won’t tell you
− It sometimes will lie to you on how far back it can acquire audit logs
Search-UnifiedAuditLog -StartDate 9/1/2017 -EndDate 10/1/2017 -UserIds
aparsons@contoso.com -ResultSize 5000 | Export-Csv “aparsons.csv”
Note: This command and others like it require you to connect to the Exchange Online shell via Powershell first (Tutorial)

@ParsonsProject
Acquiring Audit Logs

@ParsonsProject
Data Learned from Experience/Pain
+ Via PowerShell, you can’t acquire more than 5,000 records at a time, but you
can do it sequentially and it will show you if you don’t acquire them all more
clearly.
+ Microsoft has a Powershell script to get you started
https://blogs.msdn.microsoft.com/tehnoonr/2018/01/26/retrieving-office-365-
audit-data-using-powershell/
+ If you request too many logs in a short period of time Microsoft will lock you
out for a few minutes. You need to self-throttle. Check out Start-
RobustCloudCommand.ps1
+ If you use the GUI, you are limited to 50,000 events and no verification that
you have all of the logs
+ Overall, very frustrating process without a SIEM connection

@ParsonsProject
Useful Audit Log searches
+ IP Address Search in the Audit Logs (not 100% effective though):
Search-UnifiedAuditLog -ResultSize 5000 -StartDate $startDate -EndDate
$endDate -IPAddresses 45.77.147.170, 187.36.51.*| Export-Csv
"MaliciousIP.csv"
+ Inbox Rule event search for Malicious Inbox rules activity (Only if
Exchange logging has been enabled by the client)
Search-UnifiedAuditLog -StartDate $startDate -EndDate $endDate -Operations
*-InboxRule –Resultsize 5000 | export-csv "AuditLogs_FullInboxRules.csv"
+ Some “Set-Mailbox” Operation events show SMTP forwarding changes too
Search-UnifiedAuditLog -StartDate $startDate -EndDate $endDate -Operations
Set-Mailbox –Resultsize 5000 | export-csv "AuditLogs_Set-Mailbox.csv"

@ParsonsProject
Useful Audit Log searches – TIMailData, TIUrlClickData
+ Search-UnifiedAuditLog -StartDate $startDate -EndDate $endDate -Operations
TIMailData –Resultsize 5000 | export-csv "AuditLogs_ThreatIntel.csv"
+ {"CreationTime":"2019-01-10T14:31:20","Id":"7kd80k28-a294-25db- c024-
4dbpe3a2492d","Operation":"TIMailData","OrganizationId":"20do92a0-2p2m-21pd-a305-
205a05b33dd2","RecordType":28,"UserKey":"ThreatIntel","UserType":4,"Version":1,"Workload":"ThreatInt
elligence","ObjectId":"1s0f229s-1234-0b29-e391-
49k294c0f92a294020687296739248312","UserId":"Lex.Luther@contoso.com","AttachmentData":[{"FileN
ame":"signature.png","FileType":"Png","FileVerdict":0,"MalwareFamily":"","SHA256":"ECE1FCD7806
DE9E0186C7264781480F5515CAE494BE42C805B2655CC5F4FA5DF "}],"DetectionMethod":"Office
365 URL
reputation","DetectionType":"Inline","InternetMessageId":"<ASODI19AP396039PD91LKS51GD9381L19
4PE9BD@AHONQ94ND1380.namprd14.prod.outlook.com>","MessageTime":"2019-01-
10T14:20:22","NetworkMessageId":"0n3d229a-4729-2b32-a302-89b872d8b94b
","P1Sender":"Lex.Luther@contoso.com","P2Sender":"Lex.Luther@contoso.com","Recipients":["Clark.Ke
nt@contoso.com"],"SenderIp":"198.55.111.55","Subject":"Lois Lane Invites
YOU!","Verdict":"Phish"}

@ParsonsProject
Message Trace Logs
+ Why do we care about it?
− Lists e-mails received or sent within the past 90 days EVEN if deleted
− Lists IP Address of all E-mails Sent
− Easy to manage CSV file
− References Inbox Rule used to forward messages (if applicable)
− TL;DR: It’s like a timeline of E-mail’s sent & received and has IP
Addresses

@ParsonsProject
Clutter/Activities Logs – MailItemsAccess Prologue
+ Story Time

@ParsonsProject
Mailbox Audit Logs - MailItemsAccessed
+ New Operation in only Mailbox Audit Logs called “MailItemsAcessed”
+ Info below is from the Community (Not Documented by MS yet)
+ Records 2 Minutes of activity when triggered
+ MailItemsAccesesed events are triggered when
− New Client IP Address
− New User agent String
− New Username performing the read/access
− New Parent Mailbox Folder
− New Logon Type
− New Mailbox Session ID
+ Logs across multiple platforms
+ References Messages by Internet Message ID
Source: https://www.slideshare.net/DouglasBienstock/shmoocon-2019-becs-and-beyond-investigating-and-defending-office-365

@ParsonsProject
Surely we could Automate?

@ParsonsProject
HAWK
+ PowerShell Module released in December 2017
+ Made by Microsoft Support Engineers
+ HAWK will:
− Parse successful logins and resolve the locations
− Export Exchange related Audit Logs
− Export Current Inbox Rules and forwards (including hidden inbox rules)
− Export Historical Inbox Rules
− Export Permissions
− Oauth Permissions
− Azure AD Logs (If you have Azure AD P1 or P2)
+ HAWK will NOT:
− Collect all of your audit logs, trace logs, or PSTs for you
− Do your analysis for you

@ParsonsProject
HAWK
+ Process (Take a picture of this)
1. Install-Module –Name HAWK
2. Import-Module HAWK
3. Connect to Exchange Via PowerShell
4. Start-HawkTenantInvestigation
5. Start-HawkUserInvestigation
User Investigation Export Subset
Tenant Investigation Export Subset

@ParsonsProject
O365 Incident Collection Checklist – HAWK
User Level
Tenant Level

@ParsonsProject
Quick Recap: What do we know?
+ With the data collected so far we should know the following:
− Users that were compromised (If the attacker uses obvious foreign IP
addresses or Proxy/VPN solutions)
− Whether the attacker is currently in the environment or has malicious
Inbox Rules or forwarding enabled
− What mailbox rules (if any) the attacker may have created (If the client
had mailbox logging enabled)
− This can also help generate a list of users that were targeted.
+ Unanswered Questions
− How many e-mails were sent by the attacker while the user was
compromised?
− How was the user originally compromised?

@ParsonsProject
Finding Compromising Phishing E-mail
+ Unique the IP Addresses in the Trace Logs and perform Geo-IP
Lookups
+ Unique the IP Addresses from the PST file and perform Geo-IP
Lookups
+ Look for e-mails 5 days prior to the first malicious login
+ Often something like “John Smith has Shared a Document With you”
+ Attackers often delete and purge e-mails; Default TTL is 14 days
+ If you need to search for more e-mails across the entire company, you
can do that in the Search pane of the eDiscovery case (Tutorial)
Content Searches will also work exactly the same.

@ParsonsProject
Finding Propagated phishing e-mails
+ Unique the IP Addresses in the Trace Logs and perform Geo-IP
Lookups
+ Unique the IP Addresses from the PST file and perform Geo-IP
Lookups
+ Look for emails with large BCC recipients list.
+ Process for finding malicious IPs in a PST file
− Process the PST in X-ways
− Copy/export the processed EML files into a folder
− Run an automated script to lookup IP addresses
− Search for suspicious IPs in the report
− Use X-ways/Grep to then search for the identified IPs within the PST

@ParsonsProject
Proactive Techniques
+ Enable MFA
+ Look into Azure AD Conditional Access
− Can automatically block suspicious logins (if configured)
− Can blacklist IP subnets and locations
− Catch: Requires Azure Active Directory Premium

@ParsonsProject
Proactive Techniques Continued
+ Ingest your logs to a SIEM
+ Turn on a Report Phish Button
+ Disable Forwarding to External E-mails (or all emails)
+ Create Alerts for new Inbox rule creation events (in O365 with E5 or
your SIEM)
+ Double check to make sure both Unified Audit Logging and Mailbox
Audit Logging is enabled
+ Enable MFA

@ParsonsProject
Enabling MFA isn’t enough
+ MFA can be bypassed easily unless Legacy Authentication is disabled
+ Disabling Legacy Auth is quite involved.
+ Resources: https://docs.microsoft.com/en-us/azure/active-
directory/conditional-access/block-legacy-authentication
Modern AuthLegacy Auth

@ParsonsProject
Modlishka - AKA You will never be safe
Source: https://github.com/drk1wi/Modlishka

@ParsonsProject
Questions Lawyers & CISOs have -Recap
+ USER LEVEL
− What did the attacker access?
− How long did the attacker have access?
− Is there potential for PII and/or PHI Exposure?
+ TENANT LEVEL
− How many other accounts are affected?
− Has the attacker been kicked out of the environment?
− Do we know the motive of the attacker?

@ParsonsProject
O365 Incident Collection Checklist – Recap
User Level
Tenant Level

@ParsonsProject
Conclusion
+ Questions?
+ Slides will be shared on my Twitter
− @ParsonsProject

Office 365 Incident Response 2019 B-Sides Orlando

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Office 365 Incident Response 2019 B-Sides Orlando

Similar to Office 365 Incident Response 2019 B-Sides Orlando (20)

Recently uploaded

Recently uploaded (20)

Office 365 Incident Response 2019 B-Sides Orlando

Editor's Notes