Thursday, November 22, 2007

Important Things I Forgot: #1

Thought I would start keeping a running list of stupid things that I find myself forgetting.

The Problem

In my last post I mentioned that I took an ImageX image of Dad's old system's hard drive.  I still have yet to set it up in a virtual machine as I planned, but I didn't mention the hours it took to capture it.

I had my Win PE 2.0 boot disk with ImageX loaded.

I was using my 3G 40GB iPod connected via USB (1.0) to the system.

So far, so good...right?

The total drive capacity on his old system was about 20GB.  Of that he probably had about 10-13GB in use with files.  I have over 30GB free on the iPod.

So the first time I did the ImageX capture of his C: it failed after about 3-4 hours at 76% saying the drive was full.

I checked and there was tons of space on the iPod.


So I deleted some odd files off the C: that were not needed and tried again.

After about 3-4 hours (blame it on USB 1.0) it failed at about 85%, again reporting the disk was full.

I checked and there was still tons of space on the iPod.


So I cleaned more files off.  This time (overnight) it got to 96% completed before failing, again reporting the disk was full.

Now I was getting irked.  I've gotten quite good at work capturing images from multiple systems for deployments and never ran into this issue.  What gives?

I had lost all the time I had budgeted to do the virtual image setup now.  All I had left was time for one more capture attempt before Alvis and I had to return home from Tyler.

So this time I got really aggressive with the system cleaning and file cleanup.  I uninstalled quite a number of unneeded applications from the old system, deleted downloaded installation files, cleaned out all the user's temp folders located in their profiles, deleted all the profile's cache files for IE and Firefox, and then a few more things.  I managed to seriously remove some extra-fileage from the system.

This time I finally got a 100% capture of the image with no errors.

Strange.  Still had tons of space on the iPod when done.  The image file came in at about 3.6 GB.

The Solution

It came to me Monday morning in the shower as I was getting ready for work (go figure, I have no idea why it usually works out that way).

It was actually pretty simple.

See, I had forgotten my iPod is formatted FAT32.

That means that the maximum single-file size is 4 GiB minus 1 Byte (232−1 bytes).

The image I was capturing kept hitting that limit, causing the ImageX file capture to error out saying the disk was out of space.


Had I been capturing the image to an NTFS formatted drive, it never would have happened.  By my successive attempts at cleaning Dad's drive of extra files, I ended up reducing the image size enough to drop below the 4GB limit.

Could I have reformatted the iPod drive to NTFS?

Yes, but it would no longer work as an iPod: Apple Help: iPod no longer plays music after formatting or partitioning the hard disk

iPod only works (as a music player) when its hard disk is formatted for the appropriate platform using the Mac OS Extended (HFS Plus) or FAT32 formats. Do not reformat the disk using Mac OS Standard (HFS), New Technology File System (NTFS), or UNIX File System (UFS) formats.

Looks like I need to invest in a nice portable USB hard-drive and format the drive as NTFS just for imaging captures.

Bonus...ImageX (Re)-Deployment

So how do I plan to get the image onto Dad's Virtual PC virtual hard drive?

I've been playing with various scenarios, but I think I will go easy.  I plan on making a fresh Win PE2.0 boot disk, and besides copying ImageX onto it, I will also add-on the .wim file image I captured.  Then I will create the ISO file with it all bundled together.

That way I just have to boot the virtual drive/system with the boot-ISO file I made, and simply apply the image contained in the ISO file to the drive.

I've done this at work for some of our system deployment images quite successfully.  It does require use of a DVD instead of CD-R due to the .wim image size, but otherwise works quite nicely.  Since I will be doing it all virtually (in Dad's case), the ISO file is all I need and I won't need to actually burn it to a physical DVD.

At work, I still prefer to use and image/deploy using WinPE 2.0 boot disk and a portable USB 2.0 drive for its speed and flexibility (over CD/DVD media), but despite the slower transfer rates from optical-media (versus USB 2.0 hard-drive), it presents a nice and flexible solution and allows team-members to make multiple copies of the image-deployment disk.

Happy Turkey-Day!



The Guru said...

This leads me to an interesting question or two. First is it better to use NTFS over FAT-32? Secondly, what is the difference in that the iPod would not function if the drive was formated as NTFS?

Anonymous said...

Another excellent post, and very timely for those of us expecting holiday family tech support. I have a similar project on the horizon with my own father.

Great reminder about the limitations of FAT-32. It can still come in handy when cross-compatibility is needed with a Linux dual boot machine, but the performance and security are certainly not as good.

I also wanted to share a couple of portable drive alternatives to the iPod that we use where I work. The first type is represented by the Smartdisk FireLite, recently acquired by Verbatim. It's a 2.5" unit that's bus powered so you don't need to carry a power supply, just a USB cable. They get even better if you switch to a Zip-Linq cable. I don't think of it as a truly pocket-portable solution like the smaller and slower 1.8" drive-based products, but if you are already carrying a laptop bag or similar then it takes up very little extra space.

The other is one we build with a large SATA drive and a drive enclosure like the Antec MX-1 or similar that supports USB and eSATA. Users that have a recurring need for external storage get an eSATA card for their pc and can transfer at the higher rate. Six times faster is a noticeable improvement.

Anyway, I love swinging by your blog for lots of great ideas and I hope you don't mind me trying to repay you with a few thoughts of my own. Thanks!


Claus said...

Hi Guys!

@Guru--I now prefer NTFS formatting over FAT-32. I'm not a disk-head and probably not the best person to compare the two formats. What I do know (besides the file size limits) is that NTFS volumes seem to perform better overall (speed/data-reliability/fragmentation) than FAT-32. In the past the biggest complaint was that I couldn't use a Win98/DOS boot disk to read/write to an NTFS drive if I was trying to recover files. Now with Linux Live CD's and WinPE/BartPE disk, that's no longer a concern as they both can read/write NTFS with ease. I used to format my multi-partition XP system drives with the primary being NTFS and the others FAT-32, but now I am working to convert them all to NTFS. For XP/Vista I don't think I would now use anything other than NTFS. At work it is NTFS all the way for our W2K/XP systems. Regarding the iPods problems if formatted with NTFS; I can't say for sure what the problems would be. I read A LOT of posts (forms/blogs) where everyone said just don't do it...stick with FAT-32. At the minimum I can say that it appears if you go with NTFS, then if you plug the iPod into a Mac system, it will not be able to recognize the NTFS formatting to write to it. So in a cross-OS environment that would be a deal-killer. If you are just using it in a Windows-only location, maybe it would work. See this post: Cross-platform iPod fun.

If Dad's 2nd partition was NTFS (it wasn't) I could have done the ImageX capture to it first, then used the imagex /split command to split the .WIM file into smaller bits. Unfortunately it doesn't support splitting during .WIM image capture (like Ghost can do, slicing it up into 600MB'ish-sized bites). Then I could have copied the split versions over via the iPod.

Did you know you can do a split of .vhd files as well if you run into this issue? Splitting a Virtual PC VHD

@Steve -- Thanks for the very kind words and tips. I appreciate them greatly. I have an similar PATA drive enclosure I use at home. It usually houses a DVD ROM player for my Shuttle XPS system, but I often have swapped it out with my system's primary PATA drive (or others) when I have needed to hook it up to one of our laptops to do file recovery in some of my system hard-drive disasters of late. SATA is the way to go now like you noted. The Smartdisk FireLite does look nice. The WesternDigital that I linked to is very similar to what we use for a portable USB 2.0 drive at work. They are dead-useful and though not quite as pocket-portable as a flash-stick, are probably a mandatory piece of gear for all desktop/sysadmin techs nowadays. I hadn't seen the Zip-Linq cables. That is a good reference to know about!

Thanks for the tips!