Talk from Kim Thomson, Python at the Point, July 19, 2018, talking about the mobile forensics world, data extraction software and Python's role in all of it.

1. Why Can’t all Data be
the Same?
Python in the Mobile-Forensics World
Kim Thomson
H-11 Digital Forensics
kim@h11dfs.com

2. who am I?
• Retired SIGINT Soldier
• Nerd
• I love all things wireless
• My passion is extraction/recovery and decoding of phone/device data
• I teach courses in mobile forensics, chipoff for mobile forensics, JTAG-ISP for
mobile forensics, Python for mobile forensics, smartphone analysis, and wireless
tracking, mapping, and analysis
• I settled into mobile forensics because of the variety of areas within the field
• kim@h11dfs.com

3. mobile forensics =
The discovery, recovery, examination, analysis, and reporting of data from mobile
devices, generally to aid in some sort of investigation.
Mobile forensics involves:
• Extracting
• Parsing
• Organizing
• Correlating
• Reporting

4. what’s the point?
One of the persistent challenges of
mobile forensics or mobile data
recovery is getting the data in a
reportable format.
As no two phones are the same,
decoding the data into something
usable can be a problem. Nearly every
phone will have some sort of unique
data on it.
Simply obtaining the data usually isn’t
good enough. It has to be made
presentable.

5. mobile fragmentation

6. mobile forensics challenges
• Finding the data
• Phone, mobile network, SIM, SD card, cloud?
• Extracting the data
• Security locks (PINs, passwords, patterns, etc.), port difficulties, USB
debugging?
• Decoding the data
• Character encodings, file formats, database types, unknown/new apps?
• Analyzing the data
• What does it mean?

7. mobile forensics challenges - 2
• Many paths into digital forensics
• Most are not technical
• Python is scary

8. finding the data
• Cloud
• Many types of data simply aren’t found on the phone
• Depending on the case, may or may not be trivial to obtain
• SIM
• Not used for much data these days apart from last Location Area Code
(LAC) and account info
• Can contain old, deleted data from previous phones
• SD Card
• Apart from the phone itself, probably the most important piece
• Full of media and app data, backups, etc.
• May contain data from previously-used phones

9. finding the data
• Service Provider’s Network
• Tower dumps, subscriber data, call-detail/data records (CDR), SMS,
MMS, data usage, web sites accessed, etc.
• CDRs continue to be a prime source of location and activity information;
must be obtained with proper legal authority
• Synced devices
• Chrome, iCloud, Firefox, OneDrive, Dropbox, e-mail accounts, etc.
• Can contain web histories, connected WiFi networks, calls, contacts, e-
mail, synced files, etc.

10. finding the data
• Phone
• Calls, contacts, messaging, e-mails, media, location data, account info
• Databases, logs, event histories, connection timelines
• Connected cells, WiFi nets, application usage history, network usage statistics,
synced Bluetooth devices…
• Basically the user’s entire life may be found on the phone
The importance of the phone data cannot be overestimated. In 2018, I can
personally guarantee that we will never see ALL the info on a device. There’s
simply too much of it.

11. phone extraction types
Logical
File System
Physical
Some data is
not always
recoverable
Extraction Copy using Mobile Forensics
is not necessarily a
Bit Stream Clone/Image as with Computer Forensics
What you see on the mobile
device screen is what you get –
Sometimes ?
Operating System
Files
Unallocated Area
Available Area for Digital Storage

12. logical extraction
• Were originally based on AT commands, talking to the internal modem
• Simple, what you see is what you get
• Relatively Fast (unless they have loads of media on the phone)
• Will recover no truly deleted data
• May recover “deleted” database entries (calls, chats, contacts, etc.)
• Excellent choice for a “quick look”
• Can be stymied by different versions of an OS or blocked/disabled USB ports
• For Android and others, usually requires the installation of an extraction client (APK)
• Usually not possible on a locked phone (passcode/pattern/PIN)

13. file-system extraction
• Usually “good enough” depending on the phone (Android)
• Depending on the phone, may or may not be possible
• Jailbroken iPhone, rooted Android
• Other OS… maybe, maybe not
• Analogous to copy-paste all files in the file system; no truly deleted items
• Can be blocked by security protocols in Android and iOS
• There are “lesser” file-system extractions
• Android Debug Bridge (ADB) Backup
• iTunes Backup
• Partial file system based on MTP vulnerabilities in Android

14. physical extraction
• This has always been the real goal: get ALL the data on the phone… all the 1s and 0s
• Analogous to a physical image (dd) of a hard drive
• Requires root permissions in Android; most of the time just a temp root
• After iPhone 4 is impossible (improbable?) in iOS devices
• Gives the examiner the possibly of recovering truly deleted data
• Media
• Deleted Files, not just DB entries
• Getting a full physical extraction of a device has sometimes been rather difficult

15. automated-tool vendors
• Cellebrite
• UFED
• Physical Analyzer
• MicroSystemation
• XRY
• Oxygen
• Magnet Forensics
• Axiom Process
• Axiom Examine

16. benefits of automated tools
• Widely used
• Well-funded and researched
• Faster
• Easier
• Require less technical expertise
• In most cases, you only need to follow the instructions

17. benefits of automated tools
• Besides providing the extraction of data from the mobile device,
these tools also do the decoding, or parsing of the data
automagically
• Analytical tools are also included
• Reporting tools are also included
• Customer support
• Clicky buttons

18. problems with automated tools
• When they don’t work, they don’t work
• If a device isn’t supported, then you may not be able to find support at all
for it
• It may be possible, but sometimes it’s difficult to figure out which other method may work
• They are, depending on your background and point of view, insanely
expensive
• Automated tools produce… and there is no nice way to put this… the
script kiddies of the mobile-forensics world
• They don’t exactly produce a “technically-advanced user” necessarily
• They don’t (and can’t) decode ALL the data on a phone

19. biggest problem with automated tools
“If we don’t find it, it’s not there…”

20. moving beyond automated tools
• Carving for deleted records in whatever format they happen to be in
• Parsing previously-unknown or unsupported apps for data (usually SQLite)
• How many apps exist for iOS and Android???
• Log Parsing
• Parsing binary files in burner phones (usually proprietary formats)
• Writing scripts to find data in the binary dump of a phone whose file
system can’t be reconstructed 
• Correlating phone data with external sources (cell towers, wifi networks,
other geolocation data)

21. questions to ask
• Do I have all the data/evidence I need?
• What else do I need?
• In what format might it be stored?
• How can I convert it?
• Where can I put it?
• In the automated tool’s project tree
• CSV file?
• Database?
• External report?
• IS IT WORTH THE TIME???

22. • Timestamp formats
• GPS Epoch in BREW phones (feature phones)
• Unix, seconds, milliseconds, microseconds,
decimal, hex LE and BE…
• NSDates
• Mediatek
• Straight hexadecimal
• Flash memory storage characteristics
• ENCRYPTION!!
• File system
• Individual app files
• Different file types
• SQLite databases
• XML/JSON
• Proprietary DBs
• Straight Hex/Binary files
• Plists/proprietary logs
• Different Encodings
• 7-bit GSM alphabet data encoding (cheap
phones)
• ASCII vs. UTF-16 vs. UTF-8 vs. LE vs. BE
• Base64, Base32
• Reversed Nibbles
• Straight Hex Integer
phone data problems

23. phone data problems
There’s nothing quite like an IP
address represented in hex, changed
to Little-Endian, converted to a signed
decimal integer and stored as ASCII in
a SQLite DB along with other
connection attributes with a non-
descript entry name to make you lose
faith in humanity.
Seriously…?
0xCA21F10A -903745270  0A F1 21 CA  10:241:33:202

24. user data
• Calls
• Contacts
• Messaging
• Location data
• E-mails
• Paired devices
• WiFi networks
• Cookies, web history, videos, music, recordings, images, cell towers, account data, visited
web pages, bookmarks, notes, notifications, open apps, usage history, powering events,
network statistics, a compromising picture you took then deleted, etc….

25. how do we see the data?
• AccessData FTK Imager (free)
• http://marketing.accessdata.com/ftkimager4.2.0
• 7-zip (for some dd images and phone dumps)
• Autopsy and Sleuthkit (FOSS)
• Medusa Pro or Octoplus Pro Software (about 160 bucks)
• Cellebrite Physical Analyzer (paid)
• Oxygen Detective (paid)
• Magnet Axiom (paid)

26. Looking at the data – Cellebrite
Physical Analyzer

27. Looking at the data – Autopsy

28. Looking at the data – FTK Imager

29. Looking at the data – Octoplus/Medusa

30. What about Python?
• Case management, logs, moving files, blah, blah, blah
• Extracting data from SQLite databases for unsupported applications
• Parsing all the files of importance in “burner” phones
• They tend to not be well supported since they are VERY proprietary
• Phonebook, calls, SMS, device information, etc.
• Carving through unallocated space to find deleted remnants of things
• JSON, XML, logs, DB fragments...
• Carving through the binary dump of a phone that doesn’t parse at all (KMN)
• Standalone utilities for manual data decoding/searches/conversions/etc.

31. SQLite
• Easy
• Majority of apps in Android and iOS use SQLite
• Cellebrite and Magnet (Axiom) both have their own wrapper on sqlite3 to only
allow reads and no writes or “queries”. Every line of a table is read as a
dictionary. Dictionary keys are the column names.
• Whether I write something for Cellebrite or external to Cellebrite depends on
two things:
• Does Cellebrite have a category for the type of data?
• Who and what am I writing it for?

32. SQLite in Cellebrite
DB: macvendors.db
Table: macvendors
Columns: ‘mac’ and ‘vendor’
• Cellebrite’s SQLiteParser wants to
read every line by default.
• For most types of apps we want
every line.
• Calls
• Contacts
• Chats
• Messages
• Cell Towers

33. SQLite Strengths
• Standardized, used everywhere
• Analysis is relatively simple
• Tools abound to read and recover
data from SQLite DBs
• Once you’ve written one parser for a
messaging app, you can write another
with small modifications
• SQLite in Python is easy

34. BREW phone (every crappy flip phone)
• Proprietary file systems, app structures,
databases, etc.
• Not well supported by most mobile-
forensic tools
• Every model is different
• Very common to get a full physical
extraction but have no parsed data
• You have to either do everything manually
or write some stuff to parse it into a
presentable format

35. Carving
• It’s really just a regex/grep
• Certain data types have certain headers/signatures/footers on the data
• xFF xD8 xFF xE0 is one type of JPEG
• Can be done over a whole binary image or just the unallocated areas (possibly
deleted items)
• Depending on what you’re looking for you want, you may use re.search(),
re.findall(), or re.finditer()
• Data can then be exported to whatever file format you want or added to a
mobile-forensic tool’s data tree

36. Carving

37. Standalone Utilities
Mostly using QT, because
everything else is horrible

38. Standalone Utilities
Wireless analysis

39. modules/libraries I use
• os, sys, json, struct, binascii, base64, hashlib, argparse, time, datetime, re, etc…
• geopy – geolocation/geocoding, mapping, openstreetmaps, etc.
• simplekml – create KML files
• pygle – work with wigle.net
• requests, urllib2, tweepy – for grabbing thingies off the interwebz
• subprocess – running other programs, opening Google Earth instances, etc.
• sqlite3, csv
• pandas, matplotlib – chewing on data and plotting

40. online APIs I use
• wigle – wireless mapping, wardrving
• macvendors – OUI vs vendor/manufacturer
• Google Geolocation and Geocoding
• OpenStreetMaps, Nominatim from geopy
• opencellid.org (db download available)
• Mozilla Location Services (db download
available)

41. Summary
• There’s lots of data on phones in our futuristic world
• No product will be able to decode/parse all of it
• Python is cool and several mobile-forensics tools include a way to
extend functionality with Python
• We can use Python to decode/parse custom data types and
unsupported apps
• @ArdJect on Twitter
• kim@h11dfs.com for official things