Sunday, March 09, 2014

All Kinds of USB Cloning Challenges…

Part one of this story…

One of the duties I have in the shop is take an image of a bootable USB master stick we get and clone that image to our team’s USB sticks for a project.

We then use that USB to image our systems.

In the past I have requested purchase of a dedicated USB duplication device such as this Aleratec 1:10 USB 3.0 Copy Cruiser Mini Duplicator. It’s a very cool tool and has some good supporting software and can handle a lot of concurrent image restorations efficiently.

That never passed though the approval process so I’ve been left with using a poor-man’s solution.

First, find a good multi-port USB hub. They are pretty inexpensive so you could get a bunch. I particularly like these as the capacity is high with 10 USB ports.

You can find some with up to 25 USB ports if you search deeply enough.  What I liked about these were that the ports seem pretty widely spaced to allow for oversized USB stick cases. That isn’t always the case.

Tip: If you pick up ones with the ports too closely spaced, then you are left either not using all the ports due to clearance issues or having to pick up a bunch of 1.5ft USB 2.0 A Male to A Female Extension cables to get them all plugged in with the clearance issue worked around.

OK. Now that the hardware is set up, the imaging software.

There are a lot of free USB imaging tools out there.

For one-off imaging work I prefer to use Alex’s USB Image Tool. It can capture an image, it can restore an image, you can take a full device image or a partition image.  What I really like is that it also provides additional details about the device such as the serial number and other coded information. That’s helpful if you want to log each device for tracking.  However I don’t find it quite as well suited for restoring the same image to multiple sticks concurrently.

Instead I like to use the free PassMark Software ImageUSB utility.

It captures a BIN file format of the entire USB device (rather than the partition).

And it can restore the image concurrently to as many USB sticks as you want with nary a fuss or hiccup.

It doesn’t give you the same amount of device details (serial numbers/etc.) that the USB Image Tool does, but that’s why having a few different tools in your toolbox is helpful.

So here’s the recipe for the poor man’s multi-USB duplicator:

Use ImageUSB to capture your USB stick to an image file.

Connect up your multi-port USB hub(s) as needed to your system.

Plug in your target USB drives to the hub ports.

Use ImageUSB to write your USB image to the USB sticks.


Important Note: The capacity of the USB sticks you are putting the image onto need either be equal to, or in excess of the capacity size of the original “master” USB stick. If not, you will either end up truncating your image and risking data loss, or (depending on the software) it won’t write the image at all.

This is important as not all USB sticks that say they have “X” capacity the same, actually have the same amount of accessible space on them. That leads us to…

Part 2 of this story.

See, the above process has been rolling on quite well for some months now.

Only we needed more USB sticks for each team-member to carry with the same image build to hike up multi-tasking efficiencies.

And the purchaser ended up selecting and buying USB sticks that were a different mode/make.  Even though they both were USB 3.0 sticks, and even though they were both “64 GB” capacity sticks, when I went to put the image taken from the original stick model on the new one, the accessible capacity of the second USB stick was significantly lower enough to not allow me to safely or confidently put the image on it without risking truncating the image/data in the process.

That’s not good.

So this is what I ended up doing on my x64 bit Win 7 system…

(Method 1)

I first captured an ImageX (file-based) image of one of my larger cloned USB sticks.

I used the x64 version imagex binary as found in the latest Windows Assessment and Deployment Kit (Windows ADK) (GSD blog link).

Depending on your drive letterings, the command-line may vary but the basic structure is thus:

imagex /capture E: c:\temp\USB_Image.wim “USB Image base”

where “E:” is the USB drive letter and “C:” is my local system hard drive.

This captured all the files on the (larger 64 GB) USB drive into a WIM format image.

Then I removed my larger master USB stick.

I then connected the other (smaller 64 GB) USB drive to my system.

And did this from the command line window.



note: this was to confirm which disk number the USB drive was showing at…in this case the USB stick was disk 1 as my system disk was disk 0. Be sure you get this part identified correctly or bad things can happen! Your system will almost certainly vary!







>format E: /fs:ntfs /q /y

note: at this point I have a freshly formatted (smaller) USB stick that is empty. Next I need to make it “bootable” so the USB stick will work as designed for system booting/imaging after the files are restored. I used the bootsect.exe tool to do this. You should be able to find it under the Windows ADK that you probably installed to get to this point to first have snagged imagex.

> bootsect /NT60 E:

note: now it is a “bootable” USB stick we need to put the files back on it again with the imagex utility.

imagex /apply c:\temp\USB_Image.wim 1 E:


At this point I now have the original USB file sets from the larger bootable USB stick ported over to the smaller (also now bootable) USB stick.

I then used ImageUSB to capture a fresh (and smaller) device-based image file of that USB stick.

I then wrote that image to all of the smaller USB sticks.

Done and they all worked just like their bigger brothers.

Now I have two image files to use depending on the target drive’s capacity.

Had the original image been from the smaller drive then I could have put it on the larger drive with no worries or concerns as I would not have faced data truncation in that instance. If the tiny bit of capacity difference mattered, I could have used something like gparted to then expand the partition to take in the remaining unused capacity.

But if you don’t want to do all this stuff, there was another path I could have taken (I’m not sure it was any less work though).

(Method 2)

In this path, I would have first taken a volume-based image of the larger USB stick first using the USB Image Tool.

This doesn’t get me any of the magic code that makes the device bootable, just captures all the partition files instead of using ImageX.

Then I would have removed the master stick, placed in the smaller target stick, still done the whole DISKPART & format & bootsect steps in method 1.

Then I would have used the USB Image Tool to put the volume image back on the USB stick.

Then I would have captured the device image using the ImageUSB utility, and used it to put that image on the remaining (smaller) USB sticks I needed to clone.

Lesson to be learned?

Well, not all 64 GB USB sticks are the same. And if you have a big project that requires cloning a lot of USB drives, it might be wise to stick with the same exact USB make/model/capacity for the project to avoid having to create multiple images to handle the different stick capacities.

Bonus Tip:

Once you have the image file captured, if you don’t want to plug in your USB stick but want to reference the files/structure in it, you can use any of a number of tools to mount that image for review:

The WIM file can be addressed all number of manners with Windows tools and utilities. Most are fun and geeky. But for a fast, no-fuss solution, PeaZip can be used to open/extract WIM files no problem.

If you have a BIN or IMG image file, then you have some other options as well.

OSFMount - from PassMark Software maker of ImageUSB we have been discussing has a free tool that handles all kinds of image files for mounting and some manipulations.

It is based on the ImDisk Virtual Disk Driver coded by Olof Lagerkvist.

There is also the amazing Gizmo Drive software that is free and handles mounting of a crazy-wide number of image file formats.

Then there is my other long-time favorite virtual drive mounting tool; SlySoft Virtual CloneDrive.

Happy USB cloning!

Claus Valca

03/11/14 post update -- Correction made to references. PassMark Software is maker of ImageUSB utility. Previously had noted it was OSForensics, which is their URL and also the name of their free/$$ multi-feature computer forensics application. Credit to Steve Si for catching my error and alerting me.

Also, check out Steve Si’s timely post Bulk duplication of USB drive images over at his RMPrepUSB, Easy2Boot and USB Booting… blog. Steve goes into great detail about the differences in storage capacity that can be found, even between the same USB devices from the same maker.  He also includes a tutorial on using his RMPrepUSB tool to handle the image capture portion rather than relying on some potential gotchas that lurk with using the ImageUSB tool. Check it out and his comment to this post below.


Steve Si said...

Even USB flash drives of the same make, model and version can vary in capacity. It depends on the number of 'bad' blocks found when they are factory formatted.
The best thing to do, if you want to be able to copy the whole image, is to format the drive to a slightly smaller size. Then the image will fit on them all (e.g. 63.6GB instead of 64GB.
P.S. ImageUSB is by Passmark.

The best thing to do is:
1. Create a slightly smaller partition (as big as the smallest USB drive you have minus say 300MB) - tip: RMPrepUSB improves FAT32 write speed on USB flash drives (but no difference if using NTFS).
2. Add files etc and get it all working
3. Capture an image of just the partition(s) using RMPrepUSB - Disk->File and use PALL as the size. This just captures the partitions and not the whole drive. Save as xxx.bin
4. Use ImageUSB to write the .bin file to all your other USB drives.

Claus said...

@ Steve - Thank you very much for your tips and comments!

I've corrected the reference to PassMark Software! Also I found I transposed the utility name a few times as well...all cleaned up now I think.

I liked your suggestions, to either just make all the drives only as big as the "smallest" true capacity drive you have, or clip the end of one already made using a partitioning tool. Brilliant! I'm not sure how much luck I will have convincing our building team to do that work, but I certainly can trim one when it gets to me...should shave some time off the process.

I saw (and updated my post with) your own blog post Bulk duplication of USB drive images.

Question: Am I correct in assuming that your RMPrepUSB tool capture method grabs all the Windows boot sector code on the USB drive as part of the process? That was the only part that wasn't quite clear to me. As you saw, I had to both format and add the bootsect code to the new-build USB drive as part of the process.

That's probably a silly question as I'm not as familiar with RMPrepUSB features as I should be!


--Claus V.