• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Demystifying the Microsoft Extended FAT File System (exFAT)
 

Demystifying the Microsoft Extended FAT File System (exFAT)

on

  • 5,779 views

This was a presentation on the exFAT file system given back in September 2010 at the HTCIA conference in Atlanta Ga. This presentation is effectively superseded by a new presentation deck that was ...

This was a presentation on the exFAT file system given back in September 2010 at the HTCIA conference in Atlanta Ga. This presentation is effectively superseded by a new presentation deck that was uploaded to slideshare on June 6, 2014.

Statistics

Views

Total Views
5,779
Views on SlideShare
5,775
Embed Views
4

Actions

Likes
0
Downloads
51
Comments
0

1 Embed 4

http://canon6d.bluray9.com 4

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • exFAT is specifically designed for Removable media, but can be used for fixed media as well. NTFS is not recommended for removable media, especially because of the lazy write problem. Faster I/O through less file system overhead
  • You need to be able to locate the evidence, just in general You also need to know the hiding places where it can be hidden You need to validate what you found is correct, in order, and complete.
  • If the OS can’t recognize the file system, then it thinks the media is not formatted.
  • Little to nothing available in these areas Exception: Tuxera is the first independent software vendor to sign an exFAT development agreement with Microsoft. Linux and Open Source is used a lot Commercial tools are lacking Encase 6.14.3 in Dec 2009 started logical support, some issues reported FTK 3.2 – Maybe? Little documentation or publications on exFAT internals.
  • Microsoft published a patent that included the exFAT 1.00 specification. This presentation and the paper attempt to stick to the terminology used in the patent/specification as close as possible. Links to the patent and my paper will be on a later slide, and references to the paper will also be on my blog.
  • In some cases you might see ZB or ZIB, technically they are really different, but are close.
  • You never really see another sector size other than 512 bytes, but everyone just assumes that it is only 512 The 4096 size is special to support a device that is used for paging and supports 4K pages. But with the standard format, you can’t adjust sector size Clusters (or blocks) are 32K max in FAT32 Potential capacity, but the FAT can’t support 64Zib in its current configuration The volume label and file names are all 16 bit unicode Filenames to a maximum of 255 characters
  • Microsoft in the KB for Windows XP support indicated a capacity to 64ZiB and a file size maximum to 64ZiB. In reality, the file system can only support up to 128PiB, and the file size up to 16EiB. The volume size is limited by a 32-bit FAT and a 25-bit cluster size giving a 57-bit addressable volume size The file size is limited by the 8-byte (64-bit) number that holds the filesize.
  • With TexFAT there will be 2 FATS and 2 BITMAPS, with exFAT 1.0 – which does not have TexFAT (Transactional FAT) support, there is ony 1 FAT and 1 BITMAP, where previous FAT versions had 2 FATs.
  • Any FS is limited, even FAT32 and NTFS. This is Windows only, we are not talking GUID Partition Table (GPT) Although a MBR uses a 4 byte sector count, remember that the FS can be larger is you make the sectors larger (512 vs. 4096) and this causes a lot of confusion on how big a FS fits.
  • Windows would not format FAT32 beyond 32GB, it required using a FAT32 format on a different OS Some Windows utilities did not work properly with volume spaces GT 32GB, but you can mount a device that was GT 32GB Limitations of FAT32 File System: http://support.microsoft.com/kb/184006 SDXC predecessor (SDHC) had a max spec of 32GB. SDXC picks up from 32GB. SD 4.0 Specification – 300MB/s I/O speeds http://www.flashmemorysummit.com/English/Collaterals/Proceedings/2009/20090813_S204_Lin_Yee.pdf Starting at 104 mega bytes per second, and later to 300 mega bytes per second http://www.letsgodigital.org/en/20985/sdxc-cards/
  • The SDXC media will not be backward compatible Cameras and other devices have been announced, but I haven’t actually seen any devices yet, so it sounds like media is being announced and shipped with nothing that can read them.
  • New Devices may accept SDHC, but older devices will not.
  • With Sony adopting the XC memory stick to exFAT, plus the SD market, is almost 90% of the market today.
  • There are discussions of creation of exFAT on a Vista or Windows 7 machine that can’t be seen on Vista. This is usually a case of creating the media on a machine with exFAT support and then trying to read the media on a different machine without exFAT support. The common mistake is creation of the file system on removable media with a Vista SP1 (or higher machine) and trying to read it on a machine with Vista RTM.
  • exFAT uses 16 bit Unicode strings
  • It is important to note that Pentium processers use the little-endian format, so numbers stored in the file system are stored in little-endian. This can be significant because you need to change the order of the bytes in order to read the values from a hex dump.
  • Currently use exFAT 1.00, but if a later version of exFAT is in use, it will check the version # and not mount the FS unless it can suppoort it Checksums protect against corruption and viruses If there is a problem with critical directory entries, the FS should not mount.
  • FAT32 required a minimum of 65,525 clusters. exFAT does not have this restriction.
  • 4 Regions defined on the volume The FAT tables reside outside the cluster heap
  • Details follow in the next slides
  • If there was no restriction, then the size of a cluster could be 4 255
  • If the sector size is > 512 bytes, all space on the first sector of the VBR )Main Boot Sector) is not used.
  • Unlike the first sector, the other 8 bot sectors can use the entire sector and the signature marker is moved to the last 8 bytes of the sector
  • Repeats over and over again, 4 bytes = 32 bit checksum Can be used to determine if the VBR was modified 3 bytes in the VBR are not calculated in the checksum This sector does not have a signture
  • The BITMAP is used to track cluster allocation, and the FAT is only required for re-assembling the original file. If the original file is contiguous, then the FAT isn’t needed for THAT file. We will see later that a flag in the directory record is used to tell the FS whether the FAT should be used or ignored.
  • Because there is no floppy support, there is only one possible media descriptor value Cluster 0 and 1 are not defined, so 0 & 1 are not significant(Same as legacy FAT) Since the FAT is no longer used for cluster allocation, 0 (zero) is no longer significant (used to be unused)
  • The 3 main critical records: Allocation Bitmap, UP-Case Table, and Root Directory will use FAT chains. The Root Directory can grow and since it is dynamic in its growth, most likely will fragment. The UP-CASE Table and Allocation bitmap should be static and not grow or change, although theoretically they could probably be relocated and moved somewhere else on the volume. The locations (cluster addresses) of the 3 special metadata files may change, this is based on one formatting and in reality these files could eventually end up in any cluster.
  • If there are 2 FATs in a TexFAT Transactional Safe exFAT environment, then each FAT is paired with a allocation bitmap The allocation BITMAP is pointed to by a 0x81 entry.
  • This is an eye chart, but the idea is to show how to get to the bitmap. You start at the VBR (BPB), go to the root directory, look up the 0x81 entry to get the cluster address, and then go into the BITMAP table.
  • We will see details of the directory entry construction later, including what we mean by an entry type.
  • The first byte of every directory entry is the “entry type” and describes the directory entry.
  • When a file set is not in use, it is usually (but not always) a deleted file When a volume label is not in use, it means no volume label Only files have secondary entries so far Missing Benign entries usually won’t prevent the file system from being mounted. 0x80 is not defined.
  • Primary and Critical
  • Since we use 16 bit unicode without string termination, we need the length of the volume label – in unicode characters.
  • Primary and Critical. If the FS can’t find the BITMAP table, it can’t mount the FS
  • This was a small volume. 63 bytes can support maximum of 63x8 = 504 clusters.
  • Filenames are stored case insensitive, so when a search is done, the filenames are converted to upper case (folded). The UP-CASE table is used to convert the filename to all uppercase.
  • The UP-Case table is less than 6K – imagine if it was in a 32K cluster, now imagine if it was in a 32MB cluster, the amount of available slack space.
  • File Entry Set would have a File, Stream Extensions, and up to 17 File Name Extension for a total of 19. Later, when a new exFAT version comes out, the ACL will be another secondary entry bringing this up to 20. As more file secondary entries are added, let’s say one for encryption, this increases to a max of 255 secondaries.
  • Attributes and Timestamps in later slides Checksum is across the Primary and all secondaries in the set.
  • Modified, Access, and Create. Timestamps are NOT stored in this order, but MAC is a common acronym in the literature. Timestamps are not one single field like NTFS which uses a 64 bit value. exFAT combines pieces to make a UTC value. TZ offset is absent in Vista SP1, and does not appear in the exFAT 1.00 spec.
  • The standard DOS Date/Time, also used in the previous FAT versions, does not count to the second, but double seconds. To get seconds, a 33 bit number would have been needed.
  • FAT and exFAT timestamp behavior varies, but is just not reliable as far as last accessed.
  • These are pretty much the same as previous FAT versions. Since we have a separate volume label entry, there is no attribute for it, and since we don’t have 8.3 support, there is no LFN (Long File Name) attribute either.
  • The update behavior on the 10ms Modified is also not predictable, sometimes it is just set to zero. Note that the create time is really 3B866244 (reversed because of little-endian)
  • In order to validate the analysis in reverse engineering the FS, I had to write a C program to format the directory entries. This is an example of the output. All the timestamps are even because of the double seconds. But since the create is 168, this means that the create time was really 12:18:09.68 Secondary count is 4, meaning that this file set is 5 entries, 1 File, 1 Stream, and 3 filename.
  • There is 2 file lengths, one is supposed to be te file length and the other the amount of data actually written into the file so far. Length of name is needed because there is no string termination, but the file name (max 255) may require multiple directory entries (we will see later). This is where the FS indicates whether the FAT is used, if the FAT Invalid flag is set, then the FAT is ignored.
  • Since these values can vary based on the format parameters, for reference this is what the samples in this presentation is using.
  • Another output from the C program. Allocation possible indicates that the directory entry specifies a cluster address field FAT invalid indicates that this file does not use the FAT This file is 18MB and required 143 clusters to store the file. As we said before, there are 3 filename entries (each holds 15 characters of the filename), and as we see above, the filename is 40 characters in length.
  • Allocation not possible indicates that there is no cluster address in the entry. FAT Invalid has no meaning
  • Filename is 40 characters (80 bytes) and takes 3 entries to store it.
  • When the entries are not in use, some may be overwritten, and some ma not. This means that a complete set may not exist.
  • I need followers
  • My paper on exFAT and the Microsoft Patent that exposes the specification

Demystifying the Microsoft Extended FAT File System (exFAT) Demystifying the Microsoft Extended FAT File System (exFAT) Presentation Transcript

  • HTCIA International Conference September 20-22, 2010 Atlanta, GA
    • Demystifying the Microsoft Extended File System (exFAT)
    September 20th, 2010 Robert Shullich CPP, CISSP, CISM, CISA, CGEIT, CRISC, GSEC, GCFA
  • Agenda
    • About Me
    • Why a new file system
    • Forensics Relevance
    • Features
    • Advantages
    • Timelines
    • Support
    • Limits
    • Internals
    September 20th, 2010
  • About Me
    • I have been in the IT field for 35+ Years, and in InfoSec for over 15 Years
    • I carry many IT and InfoSec certifications
    • This research was part of a term project for a forensics class for my masters in Forensic Computing
    • I then expanded the term paper into a practical paper for my SANS GCFA certification
    • A link to the SANS paper and my blog is at the end of this presentation
    September 20th, 2010
  • Why do we need a new file system?
    • Current Limits Exhausted
    • Larger volumes (>2TB)
    • Larger files sizes (>4GB)
    • Faster I/O
      • (UHS-1: 104 MB/2 - UHS-2: 300MB/s)
    • Removable Media
    • Flexibility
    • Extensibility
    • NTFS Features without the overhead
    September 20th, 2010
  • Relevance to Forensics Study
    • Digital Evidence Extraction
      • Finding the evidence
      • Including the hiding places
      • Validation
    • Daubert Expert Testimony
      • Need to know and understand file org
    • New Media (SD Cards) will drive exFAT adoption, and the potential for CP investigations.
    September 20th, 2010
  • What happens when you have exFAT formatted media and no exFAT support? September 20th, 2010
  • Forensics Challenges
    • Linux OS Support
      • Tuxera drivers may help
    • Mac OS Support
    • Open Source Tools
    • Commercial Tools
      • Encase
      • FTK
    • Documentation
    September 20th, 2010
  • Disclaimer
    • The released specification and implementation is Release 1.00 of exFAT
    • The specification mentions additional features that were not implemented yet, but may at a future time/ Some of these are Windows CE holdovers
    • Both may be presented today
    • Some directory entries will be skipped
    September 20th, 2010
  • Exponents
    • 10 2 = 10 times 10 = 100
    • 10 3 = 10 times 10 times 10 = 1000 (1K)
    • 2 2 = 2 times 2 = 4
    • 2 9 = 2*2*2*2*2*2*2*2*2 = 512
    • 2 10 = 2*2*2*2*2*2*2*2*2*2 = 1024 (1K)
    • 2 12 = 2*2*2*2*2*2*2*2*2*2*2*2 = 4096
    September 20th, 2010
  • International System of Units (SI) Table
    • File System in powers of 2
    • Device characteristics in power of 10
    September 20th, 2010 Shorthand Longhand Nth Bytes KiB Kibibyte 2 10 1024 MiB Mebibyte 2 20 1024 KiB GiB Gibibyte 2 30 1024 MiB TiB Tebibyte 2 40 1024 GiB PiB Pebibyte 2 50 1024 TiB EiB Exbibyte 2 60 1024 PiB ZiB Zebibyte 2 70 1024 EiB YiB Yobibyte 2 80 1024 ZiB
  • Features of exFAT 1.00
    • Sector sizes from 512 to 4096 bytes
    • Clusters sizes to 32MiB
    • Root Directory Unlimited
    • Subdirectories to 256MiB
    • Built for speed, less overhead than NTFS but has some of the NTFS features
    • UTC Timestamp Support
      • Vista/Server 2008 SP2+, XP with KB
    September 20th, 2010
  • Features of exFAT 1.00 (cont’d)
    • OEM Parameters Sector for device dependent parameters
    • 12 sector VBR, support of larger boot program
    • Potential capacity to 64ZiB
      • Current support ≈ 128 PiB
    • Up to 2,796,202 files per subdirectory
    • File Names max to 255 Characters
    • Unicode File Names and Volume Labels
    September 20th, 2010
  • Future Features of exFAT
    • TexFAT (To be released later)
      • Exists in Windows CE
      • Transaction Safe exFAT
    • ACL (To be released later)
      • Exists in Windows CE
    • Encryption Support?
      • Not announced, but mentioned how easy to add
    September 20th, 2010
  • MBR Partition Limitations
    • Microsoft File Systems are limited when stored in a MBR partition
    • A partition is defined by a Master Boot Record
    • A MBR uses a 4 byte value for number of sectors
    • To get the maximum volume size, exFAT cannot be created within a partition
    September 20th, 2010
  • Advantages of exFAT
    • Handle growing capacities in media, increasing capacity to >32 GB.
    • > 1000 files in a single directory.
    • Speeds up storage allocation processes.
    • Breaks file size 4 GB barrier.
    • Supports interoperability with future desktop OSs.
    • Provides an extensible format.
    • Large cluster sizes
    September 20th, 2010
  • Disadvantages of exFAT
    • Not all Windows CE features implemented
    • No direct conversion to or from other FS
    • Cannot use CONVERT command to NTFS
    • No Floppy Support
    • Mostly a Microsoft Desktop and Server World
      • No Support for Older MS systems
      • No Support for Non-MS systems
      • No XBOX, PS3 or other special devices
    September 20th, 2010
  • Key Dates for exFAT
    • September 2006 – Windows CE 6.0
    • March 2008 – Windows Vista Service Pack 1
    • January 2009 – Announcement at CES of SDXC specification
    • January 2009 – Windows XP Drivers Available
    • May 2009 – Windows Vista Service Pack 2
    • August 2009 – Tuxera Signs File System IP Agreement with Microsoft
    • March 2009 – Pretec Releases first SDXC Cards
    • December 2009 – Microsoft (re)announces exFAT license program for third-parties
    • December 2009 – SDXC laptops due soon
    • December 2009 – Diskinternals releases exFAT recovery utility
    • December 2009 – Encase support
    September 20th, 2010
  • More Key Dates for exFAT
    • December 2009 Sony, Canon & Sanyo License
    • January 2010 Funai License (LCD TV)
    • February 2010 Panasonic License
    • February 2010 Panasonic 64/48GB SDXC
    • February 2010 Sony Memory Stick XC
    • February 2010 Sandisk Ultra XC 64GB Card 3.0 Spec $350
    September 20th, 2010
  • More Key Dates
    • June 1 st 2010 Tuxera Releases Linux & Android exFAT drivers
    • June 3 rd 2010 Kingston Releases Class 10 SDXC 64GB Card 60 MB/s read, 35 MB/s write.
    September 20th, 2010
  • SD Card Association
    • New Memory Card
    • Consumer Appliances
    • Follows SDHC
    • Specification for 2TB Capacity
    September 20th, 2010
  • September 20th, 2010
  • SDXC Storage Capabilities
    • From 32GB to 2TB on a card
    • Exclusively exFAT File System
    • 300 MB/s I/O Transfer
    • Storage
      • 4,000 RAW images
      • 100 HD movies
      • or 60 hours of HD recording
      • 17,000 fine-grade photos
      • in a single directory
    September 20th, 2010
  • Support for exFAT
    • Windows XP & Server 2003
      • KB955704 (requires SP2 or SP3)
    • Vista & Server 2008 SP1
    • Vista & Server 2008 SP2
      • (Adds UTC timestamp support)
    • Windows 7
    September 20th, 2010
  • Reference Standards
    • Bits are numbered right to left
      • 76543210
    • Decimal Offsets (zero based)
    • Little-Endian numbers
    • Unsigned numbers
    • Sectors vs. Clusters
    • Strings are 16 bit Unicode
    • Strings not Terminated
    September 20th, 2010
  • Endian
    • Numbering order may vary based on processor type, is determined by the order the data bytes are read from the register.
    • A 32 bit number is read as 4 8 bit bytes
    • If I have the number 0x01 02 03 04
    • Big-Endian will store it as:
      • 0x 01 02 03 04
    • Little-Endian will store it as:
      • 0x 04 03 02 01
    September 20th, 2010
  • File System Integrity
    • Version Verified
    • 3 Checksums
      • VBR
      • UP-Case Table
      • File Set
    • Critical Directory Entries
    • Other Checks and Balances
    • File System should NOT mount if failures
    September 20th, 2010
  • exFAT Limits
    • Volume size 128PiB
      • MS said 64ZiB
      • MS now says 256TiB
    • File Size 16 EiB (64 bit number)
      • Bigger than volume size
    • Subdirectory 256MiB
    • Sector 512-4096 bytes (2 9 -2 12 )
    • Cluster 32MiB (2 25 )
    • No floppy support
    • No FAT32 minimum cluster (65,525) restriction
    • No 8.3 file name support
    September 20th, 2010
  • Data Hide Alert!
    • FAT32 max cluster 32KiB
    • exFAT max cluster 32MiB
      • This is an increase of 1024 fold
    • Potential for massive slack space
    September 20th, 2010
  • Volume Space Layout
    • The Main Boot Region
      • Contains main VBR
    • The Backup Boot Region
      • Contains backup VBR
    • The FAT Region
      • Contains FAT Table(s)
    • The Data Region (Cluster Heap)
      • This is where data resides
    September 20th, 2010
  • September 20th, 2010
  • VBR – Volume Boot Record
    • Contains 12 sectors
      • 1 sector main boot sector
        • Jump Code (3 bytes)
        • BPB (BIOS Parameter Block)
        • Boot Strap Code
      • 8 sectors main extended boot sectors
      • 1 sector OEM parms
      • 1 sector reserved
      • 1 sector VBR Checksum
    September 20th, 2010
  • Boot Parameter Block (BPB)
    • OEM Label “EXFAT ”
    • Volume Length (64-bit) [sector]
    • FAT Location & Size [sector]
    • Heap Location & Size [sector, cluster]
    • Volume Serial Number
    • Location of Root Directory [cluster]
    • Volume Flags
    • Sector and Cluster Sizes [2-shift]
    • Percent in use
    • File System Revision (0x0010=1.00)
    September 20th, 2010
  • Sectors & Clusters
    • A 2-Shift is a power of 2
      • Another name for exponent
    • Sector size and sectors per cluster
      • Each stored in 1 byte
      • Theoretical maximum is 2 255
      • Sector Size Maximum 2 12
      • Sectors per cluster is derived
      • Cluster Size Maximum is 2 25
    September 20th, 2010
  • Executable Boot Code
    • First 3 bytes of Main Boot Sector
      • Jump Code
      • 0xEB7690
    • Offset 120 size 390
      • Remainder of boot code
    • Offset 510
      • End signature marker
      • 0xAA55 = “55AA”
    • Offset 512
      • Unused if defined
    September 20th, 2010
  • More Bootable Code
    • Up to 8 Main Extended Boot Sectors
      • FAT32 had 3 sector VBR with 1 MEBS
      • Entire sector can be used for boot code
      • Last 8 bytes of sector is marker
      • 0xAA550000 = “000055AA”
    • Larger capacity for boot virus!
    September 20th, 2010
  • VBR Checksum Sector
    • The 12 th sector of the VBR
    • Repeating 4 byte checksum
    • Checksum of previous 11 sectors
    • Flags and Percent excluded
      • These are volatile and change often
    • Boot Sector Virus & Checksum
    September 20th, 2010
  • VBR Checksum Sector September 20th, 2010
    • Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F
    • 00000000 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
    • 00000010 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
    • 00000020 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
    • 00000030 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
    • 00000040 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
          • Lines 00000050 through 01BF repeated
    • 000001C0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
    • 000001D0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
    • 000001E0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
    • 000001F0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
  • FAT – File Allocation Table
    • When it is used, same as legacy FAT
    • Not used when file contiguous
    • Never used for cluster allocation
    • FAT 32 has 32 bit cells, uses 28 bits
    • exFAT has 32 bit cells, uses 32 bits
      • There is no 64 bit FAT
    • Maximum clusters is 2 32 -11
    • With TexFAT – 2 FAT Tables (2 Bitmaps)
    • Addressed by pointer in VBR
    • Size stored in VBR
    September 20th, 2010
  • Cell Values in FAT Table
    • 0x00000000 – No significant meaning
    • 0x00000001 – Not a valid cell value
    • 0xFFFFFFF6 – Largest Value
    • 0xFFFFFFF7 – Bad Block
    • 0xFFFFFFF8 – Media Descriptor
      • Fixed Disk
    • 0xFFFFFFF9-0xFFFFFFFE – Not Defined
    • 0xFFFFFFFF – End of File (EOF)
    September 20th, 2010
  • September 20th, 2010
  • FAT Table Example September 20th, 2010 Offset 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0000 F8 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 0010 FF FF FF FF 00 00 00 00 00 00 00 00 00 00 00 00 0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Media Reserved UP-Case Table Allocation Bit Map Root Directory
  • Allocation Bitmap
    • Keeps track of cluster allocation status
      • Zero – Free Cluster
      • One – Allocated Cluster
    • 1 Byte = Tracking of 8 Clusters
    • Bit Zero – Byte Zero = Cluster 2
      • Cluster 0 & Cluster 1 are not defined
    • Addressed by Directory Entry
    • With TexFAT – 2 of these (FAT Pairing)
    September 20th, 2010
  • Data Hide Alert!
    • The Allocation Bitmap and the UP-Case Table are stored as files, and provide hiding space in the metadata
    • These files are static, typically won’t move, and have slack space.
    • Nothing prevents someone from moving these files elsewhere in the cluster heap, and actually making them larger
    September 20th, 2010
  • September 20th, 2010
  • Directories in exFAT
    • Root (VBR Pointer)
      • Contains certain critical entries
      • Almost unlimited in size
    • Subdirectory (by File Entry)
      • Contains file sets
      • 256MiB Max size
      • No physical “.” or “..” entries
    • Uses 16 Bit Unicode for strings
    • Every Entry 32 bytes in size
    • Entry 0x00 is end of directory
    • Has capabilities for user entries
    September 20th, 2010
  • Data Hide Alert!
    • Manipulation of the Allocation Bitmap, and creation of user directory entries provides the capability of hiding a file system within the file system
    • It may also be possible to hide data within the directory metadata itself
    September 20th, 2010
  • Entry Type September 20th, 2010 Type Field Offset (Bits) Size (Bits) In Use 7 1 Category 6 1 Importance 5 1 Code 0 5
  • Entry Type
    • In Use:
      • 0 – Not in Use, 1- In Use
    • Category:
      • 0 – Primary, 1 – Secondary
    • Importance:
      • 0 – Critical, 1 – Benign
    • Code: Identifies the entry
    September 20th, 2010
  • Volume Label Directory Entry
    • 0x83 or 0x03 Entry
    • Primary Entry
    • Only resident in Root Directory
    • Contains the Volume Label
    • 16 bit Unicode
    • 0x03 means no volume label
    September 20th, 2010
  • Volume Label Directory Entry September 20th, 2010 Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F 00000000 83 0A 65 00 78 00 46 00 41 00 54 00 2D 00 31 00 ƒ.e.x.F.A.T.-.1. 00000010 32 00 38 00 4B 00 00 00 00 00 00 00 00 00 00 00 2.8.K........... Type Volume Name Length (10) Volume Label (exFAT-128K)
  • Allocation Bitmap Directory Entry
    • 0x81 Entry
    • Primary Entry
    • Only resident in Root Directory
    • Points to the Allocation Bitmap
      • If TexFAT, then 2 of these
      • Flag bits says which FAT/Bitmap
    • Cluster Address of Bitmap
    • Size of Bitmap
    September 20th, 2010
  • Allocation Bitmap Directory Entry September 20th, 2010 Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F 0000 81 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0010 00 00 00 00 02 00 00 00 3F 00 00 00 00 00 00 00 Type Cluster Address (Cluster 2) Size (63 bytes)
  • UP-Case Table Directory Entry
    • 0x82 Entry
    • Primary Entry
    • Only resident in Root Directory
    • File names are case insensitive
    • Used to fold file name
    • Table has a checksum (32 bits)
    September 20th, 2010
  • UP-Case Table Directory Entry September 20th, 2010 Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F 0000 82 00 00 00 0D D3 19 E6 00 00 00 00 00 00 00 00 0010 00 00 00 00 03 00 00 00 CC 16 00 00 00 00 00 00 Type Cluster Address (3) Length (0x16CC = 5,836) Table Checksum
  • File Directory Entry Set
    • Used to define a file
    • May have 3 to 19 entries, or more
    • 1 Primary, many Secondary
    • Is considered an array
      • Must be in order
      • Must be contiguous (no gaps)
    • Entire Set has Checksum
    September 20th, 2010
  • File Directory Entry
    • 0x85 or 0x05 Entry
    • Primary Entry
    • Set Checksum (16 bits)
      • Not modified on file delete
    • Secondary Count
      • # Secondary entries that follow
    • File Attributes
    • Timestamps
    September 20th, 2010
  • Timestamps & Time Zones
    • 3 Timestamps (MAC)
    • 32 bit DOS Date/Time
      • Local Machine Time
    • 10ms Offset (MC)
    • TZ Offset (MAC)
      • 15 minute increments
      • 7 bit signed number
      • ±16 hours
      • Present with UTC support
    September 20th, 2010
  • Timestamp Accuracy
    • FAT32 – Last Access – Date only
    • exFAT – Last Access – Date/Time
    • All DOS DATE/TIME Double Seconds
    • 10ms adds 0-1990 ms to time
    • 10ms only for Create/Modify
    September 20th, 2010
  • Timestamp Reliability
    • Timestamps appear to be updated when the file is created or modified.
    • Last Accessed Timestamp appear to be updated when file is created or modified.
    • Last Accessed Timestamp appear NOT modified on file read.
    • Forensics Implication on MAC time analysis
    September 20th, 2010
  • File Attributes September 20th, 2010 Attribute Offset Size Mask Reserved2 6 10 Archive 5 1 0x20 Directory 4 1 0x10 Reserved1 3 1 System 2 1 0x04 Hidden 1 1 0x02 Read-Only 0 1 0x01
  • File Directory Entry September 20th, 2010 Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F 0000 85 04 D4 92 20 00 00 00 44 62 86 3B F1 62 BA 3A 0010 44 62 86 3B A8 00 EC EC EC 00 00 00 00 00 00 00 Type # Secondary Entries Set Checksum (0x92D4) Attributes (0x0020 = Archive) Create Modified TZ Offset CMA EC = GMT-5 Accessed Create 10ms Modified 10ms
  • Formatted File Directory Entry September 20th, 2010 Root Entry Type Read is: 85 Directory Entry Record Checksum: 92D4 Calculated Checksum is: 92D4 Size Directory Set (bytes): 160 Secondary Count 004 File Attributes: 0020 Archive Create Timestamp: 3B866244 12/06/2009 12:18:08 Last Modified Timestamp: 3ABA62F1 05/26/2009 12:23:34 Last Accessed Timestamp: 3B866244 12/06/2009 12:18:08 10 ms Offset Create A8 168 10 ms Offset Modified 00 0 Time Zone Create EC 236 Value of tz is: GMT -05:00 Time Zone Modified EC 236 Value of tz is: GMT -05:00 Time Zone Last Accessed EC 236 Value of tz is: GMT -05:00
  • Stream Extension Directory Entry
    • 0xC0 or 0x40 Entry
    • Secondary Entry
    • Length of Name
    • Length of File (2 of them)
    • Cluster address of first data block
    • Name Search Hash value
    • Secondary Flag
      • FAT Invalid
      • Allocation Possible
    September 20th, 2010
  • Stream Extension Directory Entry September 20th, 2010 Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F 0000 C0 03 00 28 AD 3C 00 00 1F 46 1D 01 00 00 00 00 0010 00 00 00 00 05 00 00 00 1F 46 1D 01 00 00 00 00 Entry Flags (Alloc Possible/Fat Invalid) Length of File Name (0x28= 40) Name Hash (0x3CAD) Cluster (5) Data Length 0x011d461f = 18,695,711
  • Parameters for Samples September 20th, 2010 Bytes Per Sector: 2 to the 09 power is: 512 Sectors Per Cluster: 2 to the 08 power is: 256 Bytes per Cluster: 131072 (128K)
  • Formatted Stream Extension September 20th, 2010 Root Entry Type Read is: C0 Directory Entry Record, Stream Extension Secondary Flags: 03 Flag Bit 0: Allocation Possible Flag Bit 1: FAT Chain Invalid Length of UniCode Filename is: 40 Name Hash Value is: AD3C Stream Extension First Cluster 5 Cluster 5 is Allocated Stream Extension Data Length 18695711 Bytes Slack: 83487 Clusters Used: 143 Stream Extension Valid Data Length 18695711 Bytes Slack: 83487 Clusters Used: 143
  • File Name Extension Directory Entry
    • 0xC1 or 0x41 Entry
    • Secondary Entry
    • Secondary Flags
      • Allocation not possible
      • FAT Invalid
    • 15 Characters (30 bytes) of Name
    • Name in 16 Bit Unicode
    • In order (FAT32 LFN was reversed)
    • Up to 17 max, total 255 character
    September 20th, 2010
  • File Name Extension Directory Entry September 20th, 2010 Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F 0000 C1 00 62 00 75 00 73 00 69 00 6E 00 65 00 73 00 Á.b.u.s.i.n.e.s. 0010 73 00 5F 00 6F 00 66 00 5F 00 73 00 65 00 63 00 s._.o.f._.s.e.c . 0000 C1 00 75 00 72 00 69 00 74 00 79 00 5F 00 5F 00 Á.u.r.i.t.y._._. 0010 62 00 75 00 73 00 2D 00 31 00 30 00 35 00 2D 00 b.u.s.-.1.0.5.-. 0000 C1 00 33 00 32 00 6B 00 62 00 70 00 73 00 2E 00 Á.3.2.k.b.p.s... 0010 6D 00 70 00 33 00 00 00 00 00 00 00 00 00 00 00 m.p.3........... File Name = business_of_security__bus-105-32kbps.mp3
  • Significance of “not in use” flag
    • 0x05, 0x40 & 0x41 Entries
      • “ Not in use” may mean deleted files
      • May also be reallocated rename
    • Set Checksum not changed when entries marked “not in use”
    September 20th, 2010
  • Summary
    • exFAT is a new generation of the FAT family of Microsoft File Systems
    • The need for forensics tools will heat up in 2010
    • We don’t have the right tools yet
    • Documentation and support for exFAT is scarce
    September 20th, 2010
  • Q&A September 20th, 2010
  • Contact Information
    • E-mail: [email_address]
    • Blog: rshullic.wordpress.com
    • Blog: shullich.blogspot.com
    September 20th, 2010
  • References
    • Sans Reading Room:
    • http://www.sans.org/reading_room/whitepapers/forensics/rss/reverse_engineering_the_microsoft_exfat_file_system_33274
    • Microsoft Patent:
    • Microsoft Patent 0164440 (June 25, 2009). Quick Filename Lookup Using Name Hash.
    • Pub No. US 2009/0164440 A1 Retrieved December 10, 2009 from
    • http://www.pat2pdf.org/patents/pat20090164440.pdf
    September 20th, 2010