Blog: SANS Digital Forensics and Incident Response Blog

Blog: SANS Digital Forensics and Incident Response Blog

Sanitizing Media (The Linux Method)

Hal Pomeranz, Deer Run Associates


I've been wiping a lot of media lately. Mostly these are USB devices that we've used to share evidence and other data during an investigation. I want to be sure that I don't accidentally disclose any data from my cases, and I also want to know when I reach into my bag for a USB stick that it's not going to be polluted with other data. And when I get new media (from a vendor, trade show, or whatever) I always have a strict policy of wiping the drive completely from my Linux box (which is specifically configured not to automount new media) before it gets near any Windows machines that might have autoruns enabled.

Happily, Linux makes this whole process quite straightforward with just a few simple command-line tools. First we need to figure out what drive letter the media has been assigned. I find that fdisk is the easiest way to get this information:

$ sudo fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0004342b

Device Boot Start End Blocks Id System
/dev/sda1 * 1 60770 488134993+ 83 Linux
/dev/sda2 60771 60801 249007+ 5 Extended
/dev/sda5 60771 60801 248976 83 Linux

Disk /dev/sdd: 16.0 GB, 16049504256 bytes
5 heads, 32 sectors/track, 195916 cylinders
Units = cylinders of 160 * 512 = 81920 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sdd1 * 51 195917 15669312 c W95 FAT32 (LBA)


Here you can see my primary internal drive at /dev/sda and a 16GB thumb drive at /dev/sdd. The /dev/sdd device is the one I want to clean.

To sanitize the drive, I'm just going to completely overwrite it with zeroes using dd:

$ sudo dd if=/dev/zero of=/dev/sdd bs=1M
dd: writing `/dev/sdd': No space left on device
15307+0 records in
15306+0 records out
16049504256 bytes (16 GB) copied, 2728.03 s, 5.9 MB/s

Here I'm just reading nulls from /dev/zero and writing them over the entire physical device. Of course this clobbers the partition table at the front of the drive:
$ sudo parted /dev/sdd print
Error: /dev/sdd: unrecognised disk label

GNU parted is a useful command for interacting with disk labels at the command line. The general format of the command is "parted <device> <command> [options]". In this case I'm using the "print" command to try and print the partition label, but it's been eradicated by our dd command so we get an error.

We can also use parted to create a new disk label and even make a partition on the drive:

$ sudo parted /dev/sdd mklabel msdos
Information: You may need to update /etc/fstab.

$ sudo parted /dev/sdd mkpart primary NTFS 8s 17G
Information: You may need to update /etc/fstab.

$ sudo parted /dev/sdd print
Model: Kingston DataTraveler 2.0 (scsi)
Disk /dev/sdd: 16.0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 4096B 16.0GB 16.0GB primary


First we use "mklabel" to create a new DOS-style partition table in the first sector of the drive (parted also makes GPT labels and even Mac, BSD, and Sun disk labels).

Then we use "mkpart" to create a new "primary" partition labeled as NTFS starting at sector 8. That will leave 7 unallocated sectors after the disk label, but start us out on a nice clean 4K boundary. Notice that I specify the partition size as 17G-- slightly larger than the actual drive-- just to make sure I consume all of the space on the device. parted will take care of sizing the partition to the physical media. We can confirm our settings with the "print" command.

Notice in the output of the "print" command that the "File system" column is empty. That's because we haven't actually formatted a file system into the partition. I want to make an NTFS partition for sharing data between my Linux box and my clients' Windows systems. On Linux you use the mkntfs command for this:

$ sudo mkntfs -Q /dev/sdd1
Cluster size has been automatically set to 4096 bytes.
Creating NTFS volume structures.
mkntfs completed successfully. Have a nice day.

Note that I'm calling mkntfs on the partition we created with parted-- /dev/sdd1-- and not on the entire drive. I'm also using the -Q ("quick") option to tell mkntfs not to waste time overwriting the drive with zeroes since we've already done that. And it's also pleasant that the command wishes us a nice day as it's exiting.

If you're not excited about NTFS, you could just as easily make a FAT or EXT partition on the drive. For these file systems, it's easiest just to use the mkfs command. Here's an example of creating an EXT2 file system on our drive:

$ sudo mkfs -t ext2 /dev/sdd1
mke2fs 1.41.9 (22-Aug-2009)
[...]

I've never seen much point in bothering with EXT3 on a thumb drive, but it's just a matter of using "ext3" (or "ext4") instead of "ext2" in the command line shown above.

Making a FAT file system on the drive is only slightly more complicated:

$ sudo mkfs -t vfat -F 32 /dev/sdd1
mkfs.vfat 3.0.3 (18 May 2009)

Notice for the file system type we specify "vfat" and not just "fat". The -F option lets you specify the FAT type-- here I'm making a FAT32 file system.

Of course the label on the partition is "NTFS", which is what we chose when we made the partition with parted. Unfortunately, parted doesn't seem to be able to re-label a partition without deleting it and re-creating. Happily, fdsk will allow us to change partition types on the fly:

$ sudo fdisk /dev/sdd
[...]

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 83
Changed system type of partition 1 to 83 (Linux)

Command (m for help): p

Disk /dev/sdd: 16.0 GB, 16049504256 bytes
255 heads, 63 sectors/track, 1951 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000705e4

Device Boot Start End Blocks Id System
/dev/sdd1 1 1952 15673340 83 Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.


The "t" option can be used to interactively change the partition type. "p" prints the current partition table and "w" writes the modified table to the device and exits fdisk.

So there you have it. Almost certainly more than you ever wanted to know about cleaning and formatting drives under Linux. While we were working with a relatively small USB device in this case, all of the commands that you see here apply equally well to all kinds of storage devices of any size you might encounter.

Hal Pomeranz is an Independent IT/Security Consultant, a SANS Institute Faculty Fellow, and a GCFA. He is considering getting a new set of business cards printed that read "Disk Janitor".

14 Comments

Posted June 30, 2010 at 1:13 PM | Permalink | Reply

thanasisk

I have a small issue with this post.
While the information contained therein is technically sound and correct, it is not "sanitizing" per se, as recovery is still possible by an adversary. At the very least, dd if=/dev/random should be used, as opposed to /dev/zero. There has been a lot of publicized research on that subject, proving that overwriting with zeroes is not going to stop "the bad guys".

Posted June 30, 2010 at 3:11 PM | Permalink | Reply

Hal Pomeranz

Craig Wright's research (that was <a href="http://blogs.sans.org/computer-forensics/2009/01/15/overwriting-hard-drive-data/">previously published</a> on this blog) suggests otherwise.

Posted June 30, 2010 at 3:29 PM | Permalink | Reply

Mark

The reason the data is overwritten with all zeros instead of with random characters is so the wipe can be verified. Although we know using all random characters has the same result, it can not be proven in court to be wiped. Perhaps over writing with random then a second pass with all zeros for verification...

I have been using the method in this post for three years to wipe media without any problems.

Posted June 30, 2010 at 5:50 PM | Permalink | Reply

thanasisk

Thanks for the link, looks like my point was not valid for the case when you try to recover data from today's drive using the technology referenced in Guttmann's paper.

Posted June 30, 2010 at 7:59 PM | Permalink | Reply

Al

Err... Craig Wright's research was in regards to magnetic storage, not solid state. I completely agree with the first poster (although maybe for a different reason) - urandom is a much better choice. You have absolutely no idea what the storage controller on a flash drive will do with an instruction to write all zeros. Maybe "to save a write cycle on my solid state memory, I'll just mark this whole section as being zeros". Much better to use urandom, since there is so much entropy, the controller will have no choice but to actually commit the data. There are still other issues related to wear levelling that mean you probably aren't really sanitising it.

Posted July 01, 2010 at 4:08 PM | Permalink | Reply

Hal Pomeranz

Al, I was struck by your assertion that a flash drive might choose not to write zeroes if instructed to do so. So I did a little testing with the flash drives I have on hand: an old Lexar 1GB Jump Drive, an older Kingston 4GB Data Traveler, and a new 16GB Data Traveler that I just purchased. In all cases, my method was:

1. Use "dd if=/dev/zero ..." to overwrite the drive with nulls
2. Disconnect the drive from the system
3. "echo 3 &gt;/proc/sys/vm/drop-caches"
4. Reconnect drive to system
5. "dd if=/dev/sdd | xxd | grep -v '0000 0000 0000 0000 0000 0000 0000 0000'"

In all cases, the command in #5 returned no output, indicating the system was reading the drive as being completely nulled. Have you actually had real-world experiences where a flash drive did not behave this way under similar conditions?

Posted July 01, 2010 at 12:04 AM | Permalink | Reply

Mark

The reason for wiping with all zeros is so the wipe can be verified. If all random characters are used there us no way to prove in court the media was sterilized. Maybe using all random characters followed by a pass with all zeros...

I have been using the posted method for three years now and have not had any issue.

Posted July 01, 2010 at 2:42 AM | Permalink | Reply

JG

"[...] as recovery is still possible by an adversary."
If you referer to the now-dating "Secure Deletion of Data from Magnetic and Solid-State Memory" by Peter Gutmann, you may want to read the "Epilogue" and "Further Epilogue" section of the updated paper. It is state that the "[...] paper is now more than fifteen years old, and discusses disk storage technology that was current 15-20 years ago.", that "with the ever-increasing data density on disk platters and a corresponding reduction in feature size and use of exotic techniques to record data on the medium, it's unlikely that anything can be recovered from any recent drive" and finally that "with ultra-high densities and use of perpendicular recording I don't see how MFM would even get a usable image". Like Gutman said, unless you deal with mid-90s technologies or older, "even if you've got 10KB of sensitive data on a drive and can't erase it with 100% certainty, the chances of an adversary being able to find the erased traces of that 10KB in 80GB of other erased traces are close to zero."
So unless you need to erease "top secret for national security" kind of data, I don't see the benifit of doing more than one passes. The technology needed to maybe recover some data that maybe useful are quite expensive and way to much time consuming.

Now, between /dev/random VS /dev/zero, set apart the sense of more security that may provide the use of /dev/random, both give the same level of security (for 1 passe at least).

My 2 cents.

Posted July 02, 2010 at 8:52 PM | Permalink | Reply

Tony

My understanding of this technique is that it works great for standard hard drives but for flash drives it might or might not work properly because write levelling techniques distribute the write operations to non-contiguous blocks. How do you know you've hit the entire surface of the drive unless you fill up the entire capacity of the drive? I've just been deleting my junk, using Sdelete to clean the free space and then using Recuva to verify theres nothing there. Possibly more steps than I really need, but it seems to be working out ok. If someone has verified the DD method works for flash drives as well I'd love to start using that as thats how I handle standard hard drives now and I'd love to standardize my process.

Posted July 05, 2010 at 2:44 PM | Permalink | Reply

Hal Pomeranz

Tony, my technique does fill the entire capacity of the drive. It works for me (and others in this thread).

Posted July 04, 2010 at 4:54 PM | Permalink | Reply

Al

Two quick comments:

1) It's easy to verify the write:

dd if=/dev/urandom | tee /dev/sdb | md5sum ; md5sum /dev/sdb
But I agree - you are probably better off with a final all zero pass.

2) Hal - That is exactly what I'd expect. What I mean was without using enough entropy, how do you know the controller doesn't just record a bitmap of zeroed blocks or similar? When you ask what's on the device, it can still say "why, it's full of zeros!" and pass your test. It then saves a write over the whole physical device. I have absolutely no idea if any controller does this, but if I was trying to sanitise media, I wouldn't assume they didn't.... While some like JG might be prepared to assert categorically that no device on the market does it, I wouldn't be so bold :)

Al

Posted July 05, 2010 at 2:43 PM | Permalink | Reply

Hal Pomeranz

Al, I did a bit of Google-ing, but I can't find anything that suggests that any manufacturer implements the sort of controller caching that you're suggesting. Can you point me at some of the relevant technical docs?

Posted September 17, 2010 at 6:24 PM | Permalink | Reply

Samuel

Does anyone have an opinion on using the hdparm -security-erase commands to wipe hard drives?

Posted February 22, 2011 at 9:38 AM | Permalink | Reply

Al

@Hal, @JG

Not sure if you saw this:
http://www.usenix.org/events/fast11/tech/full_papers/Wei.pdf

From the article:
"Since some SSDs compress
data before storing, they will write fewer bits to the flash
if the data is highly compressible. This suggests that
for maximum effectiveness, SSD overwrite procedures
should use random data."

Even if there weren't specific examples, I still think random data is better, since the above becomes a non issue.

Cheers,
Alan

Post a Comment






Captcha

* Indicates a required field.