Blog: SANS Digital Forensics and Incident Response Blog

Blog: SANS Digital Forensics and Incident Response Blog

Dealing with Split Raw Images in Digital Forensics

Hal Pomeranz, Deer Run Associates


Lately I've been working with images from a client whose policy is to create their dd type images as a series of 2GB chunks-- the so-called split raw format. While commercial forensic tools will typically handle this format easily, split raw images can present challenges for examiners using Open Source utilities and Linux command-line tools. With image sizes constantly increasing, recombining the individual chunks of a split raw image into a single, monolithic image file is not really practical either in terms of analyst time or disk space. Happily, there are some Open Source utilities that can make dealing with split raw images considerably easier.

The Sleuth Kit


The Sleuth Kit utilities have actually supported split raw format since v2. The trick is to use the "-i split" option and to make sure you use wildcards to include all of the image file chunks on the command line.

As an example, I've taken one of the small 2GB images that we use for exercises in the SANS Forensics curriculum and split it into 10 200MB chunks:

# ls
test_img.000 test_img.003 test_img.006 test_img.009
test_img.001 test_img.004 test_img.007 test_img.010
test_img.002 test_img.005 test_img.008
# mmls -i split -t dos test_img.0*
DOS Partition Table
Offset Sector: 0
Units are in 512-byte sectors

Slot Start End Length Description
00: Meta 0000000000 0000000000 0000000001 Primary Table (#0)
01: ----- 0000000000 0000000062 0000000063 Unallocated
02: 00:00 0000000063 0000112454 0000112392 Linux (0x83)
03: Meta 0000112455 0004192964 0004080510 DOS Extended (0x05)
04: Meta 0000112455 0000112455 0000000001 Extended Table (#1)
05: ----- 0000112455 0000112517 0000000063 Unallocated
06: 01:00 0000112518 0000642599 0000530082 Linux Swap / Solaris x86 (0x82)
07: Meta 0000642600 0004192964 0003550365 DOS Extended (0x05)
08: Meta 0000642600 0000642600 0000000001 Extended Table (#2)
09: ----- 0000642600 0000642662 0000000063 Unallocated
10: 02:00 0000642663 0004192964 0003550302 Linux (0x83)
11: ----- 0004192965 0004194303 0000001339 Unallocated


You can use all of the normal Sleuth Kit options with these split images. For example, we can address individual partitions in the image using byte offsets:
# fsstat -i split -o 642663 test_img.0*
FILE SYSTEM INFORMATION
--------------------------------------------
File System Type: Ext2
Volume Name:
Volume ID: 5584cf973ce42e8dd8112c4c82c7201b
[...]

And, yes, the Sleuthkit commands that accept unit numbers after the image file name still work fine in split raw mode:
# istat -i split -o 642663 test_img.0* 2
inode: 2
Allocated
Group: 0
[...]

Direct Blocks:
500
# blkcat -h -i split -o 642663 test_img.0* 500
0 02000000 0c000102 2e000000 02000000 .... .... .... ....
16 0c000202 2e2e0000 0b000000 14000a02 .... .... .... ....
32 6c6f7374 2b666f75 6e640000 017c0000 lost +fou nd.. .|..
[...]


So if all you need is the Sleuth Kit, then you're already good to go.

AFFLIB to the Rescue


Unfortunately, there are many cases where the tools or procedures you want to use do not have built-in support for split raw files like the Sleuth Kit does. However, Simson Garfinkle's AFFLIB suite includes the "affuse" utility which can make your split raw files appear as a single image file via the Linux FUSE (File system in USEr space) drivers.

The affuse utility could not be simpler to use:

# affuse test_img.000 /mnt/combine
# ls -lh /mnt/combine
total 0
-r--r--r-- 1 root root 2.0G 1969-12-31 16:00 test_img.000.raw

The arguments to the affuse command are the name of the first image file in your split raw image and an empty directory where you want affuse to create the virtual, combined image file. As you can see in the output above, the name of the virtual image file is the name of the first split raw image file you specified with the ".raw" extension added. You can also see the size of the virtual image under /mnt/combined is reported as being 2GB-- the size of the original image before splitting.

One tricky bit about the affuse utility is that it expects the names of the split raw image files to end with a dot followed by three digits: ".000", ".001", and so on. If your image files do not follow this naming convention, then affuse will only mount the first piece of your split raw image, and the resulting virtual image will not be usable.

Once set up properly via affuse, the virtual image file functions just as if you had created it by manually concatenating the pieces of your split raw image into a single, monolithic file. You can run Sleuth Kit tools against it:

# mmls -t dos /mnt/combine/test_img.000.raw 
DOS Partition Table
Offset Sector: 0
Units are in 512-byte sectors

Slot Start End Length Description
00: Meta 0000000000 0000000000 0000000001 Primary Table (#0)
01: ----- 0000000000 0000000062 0000000063 Unallocated
02: 00:00 0000000063 0000112454 0000112392 Linux (0x83)
[...]


But more interestingly you can now do things like mount partitions from the virtual image:
# mount -o ro,loop,offset=329043456 /mnt/combine/test_img.000.raw /mnt/test
# ls /mnt/test
bin dev home lost+found opt root tmp var
boot etc lib mnt proc sbin usr

Obviously, there has to be some overhead for accessing the data in the split raw image via the FUSE drivers. My extremely limited benchmarking, puts the performance degradation at around 10% as compared to accessing the same image as a single monolithic dd image without using FUSE. You'll have to weigh this performance penalty against the cost of creating the monolithic dd image from a large number of split raw pieces. If you're dealing with a 1TB disk image and can combine the raw pieces at a rate of 2GB/min, it may take you 6 hours to create the combined image. The FUSE overhead seems insignificant if it means you can start processing your image right away.

Note that affuse is is just a wrapper program that hides much of the complexity of the underlying FUSE driver interface. But when you no longer need your virtual image file, you will use the FUSE package's "fusermount" command to unmount the virtual image file:

# umount /mnt/test
# fusermount -u /mnt/combine
# ls -lh /mnt/combine
total 0

Note that we first use the regular umount command to unmount any partition images we may have mounted out of our virtual raw disk image. Then we use "fusermount -u" to tear down the virtual image file we created with affuse.

Conclusion


AFFLIB really does make processing split raw images a breeze. I recommend using it on a multi-core analysis workstation, where you'll have plenty of extra cycles for the FUSE driver that's supporting the virtual image file. This will minimize any overhead due to the FUSE drivers as much as possible.

Hal Pomeranz is an Independent IT/Security Consultant, a SANS Institute Faculty Fellow, and a GCFA. In his spare time, he enjoys breaking monolithic dd images into split raw format just to mess with other analysts.

8 Comments

Posted September 15, 2010 at 2:35 PM | Permalink | Reply

Rob Lee

Quick not on AFFLIB: AFFLIB will also mount EWF (E01) images and split (E01, E02) as well. This is found already in the SIFT Workstation as well.

Great post Hal!

Posted September 16, 2010 at 7:18 AM | Permalink | Reply

Rick El-D.

I can't say thank you enough -- I can slash my HDD costs in half now!

Posted October 01, 2010 at 5:02 PM | Permalink | Reply

Daniel Melo

My single image file looks fine, but cant mount: it:

#sudo mmls -t dos /mnt/combine/CVM.001.raw
DOS Partition Table
Offset Sector: 0
Units are in 512-byte sectors

Slot Start End Length Description
00: Meta 0000000000 0000000000 0000000001 Primary Table (#0)
01: ----- 0000000000 0000000062 0000000063 Unallocated
02: 00:00 0000000063 0156248189 0156248127 NTFS (0x07)

#mount -t ntfs-3g -o ro,loop,show_sys_files,offset=32256 /mnt/combine/CVM.001.raw /mnt/ntfs
[sudo] password for analista:
Failed to read last sector (156248126): Invalid argument
HINTS: Either the volume is a RAID/LDM but it wasn't setup yet,
or it was not setup correctly (e.g. by not using mdadm --build ...),
or a wrong device is tried to be mounted,
or the partition table is corrupt (partition is smaller than NTFS),
or the NTFS boot sector is corrupt (NTFS size is not valid).
Failed to mount '/dev/loop0': Invalid argument
The device '/dev/loop0' doesn't seem to have a valid NTFS.
Maybe the wrong device is used? Or the whole disk instead of a
partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?


If i don't split or copy the entire disk, it mounts without problem.

Any suggestion?

Posted October 09, 2010 at 12:55 AM | Permalink | Reply

Hal Pomeranz

Daniel, sorry it's taken so long to respond to your comment. I am seeing the same problem that you are and have spent a bunch of time this week trying to track down the issue. We're currently discussing the problem on the AFFLIB mailing list:

http://groups.google.com/group/aff-discuss/browse_thread/thread/e915600df2f4dc33?hl=en

Posted October 26, 2010 at 1:52 PM | Permalink | Reply

Hal Pomeranz

FYI, the issue with afflib ignoring the last segment of split raw images is now fixed in v3.6.3.

Related to Rob's initial comment regarding E01 format, Simson has removed support for E01 from afflib (see http://afflib.org/archives/75 for the rationale). If you want to mount E01 images, you need to get libewf from Sourceforge-- http://sourceforge.net/projects/libewf/

Posted October 30, 2010 at 2:17 AM | Permalink | Reply

joe

What size of chunks would you recommend splitting large images into?

Posted October 31, 2010 at 3:40 AM | Permalink | Reply

Hal Pomeranz

I wouldn't recommend splitting the image at all if you don't have to. Unfortunately, some organization's procedures as well as some tools out there default to using 2GB chunks, which is an unfortunate legacy of file systems that can't handle larger file sizes. Assuming you're writing the image to a modern file system (e.g., NTFS, EXT, UFS, etc) you shouldn't have a problem writing the image as a single large file.

Posted July 19, 2011 at 8:30 AM | Permalink | Reply

Nick Klein

I've also published some instructions on our blog for fellow Mac forensicators who want to mount split images under OSX. Check out:
kleinco.com.au/thoughts-events/item/mounting-split-disk-images-under-osx

Post a Comment






* Indicates a required field.