Tracking down a pagefile.sys mystery

A Prologue

About every four months, I build updated base images for our technicians.  We have about ten different hardware platforms current deployed for our end-user desktop and laptop systems.

It’s a touch time-consuming but the following procedure works well and gives us flexibility and redundancy.

I take the “official” company image and apply it to a cleaned and formatted system drive.  I then apply a round of Microsoft updates, plug-in updates, and a few others to the system, add in any system configuration or policy tweaks that we have discovered are needed or desired since the last image-build. I run CCleaner to dump temp files and stuff. I finally sysprep it and shut it down.

I then capture the image using both ImageX and Clonezilla.  ImageX (and the WinPE disk) seem to like 512 MB RAM or better to function. Clonezilla can tolerate 512 or less.  I know it’s kinda silly to have the same image in two different formats but the guys seem to enjoy the flexibility and with large USB hard-drives cheap, space isn’t an issue.

I could (and have) make a multi-image ImageX single WIM that contained all of the different systems in one WIM but it wasn’t very popular.  So I just stick with what works.

So this go-round I had prepped and captured a system WIM and was shocked to find that the size had jumped from about a 2.6 GB WIM file image to over 4 GB.  While there were over 34 Microsoft patches to deploy in October alone, I didn’t expect to see that big of an incremental jump.

Something seemed very, very off here.  Why the big jump?

I mounted up the WIM file on my main system and then tossed SpaceSniffer at the mount folder.

Immediately I found the issue; a 1.6 GB file called “pagingfile.sys”.

But the standard Windows pagefile.sys should be excluded by ImageX by default capture parameters.  What was going on?

That was the mystery to be solved.

Chapter One: “Fraternal Twins”

At first I thought I must have made some really bone-headed mistake during my system image preparation.

So I cleaned the last system I was building, rolled back to the original image, and went through the process again.

Before I ran sysprep I checked the root and looked for any hidden system page files.

Curiously, I found not only “pagefile.sys” with an early time/date stamp but also the “pagingfile.sys” file as well with a later time stamp.  Both were approximately the same 1.6 GB size for a combined total of just over 3 GB of drive-space taken up.

Hmmm.

Chapter Two: “A Windows Page File by Any Other Name…”

While I have from time to time “tweaked” a Windows system to adjust the page file I’ve never attempted to change the name of it and didn’t have a clue how that was done.

Turns out it is not controlled from a GUI but rather from the SYSTEM Registry hive.

 How to Move or Rename Your Paging File in Your Runtime – MSDN

Regarding the paging file, an uncommon question in the past has been "How can I rename the page file from 'pagefile.sys' to something like 'andy.sys'?"

A more common question on the pagefile has been "How can I relocate pagefile.sys to another partition that is not protected by EWF?"

The information for both of these questions can be found and changed at the following registry key:

HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management
Name: PagingFiles
Type: REG_MULT_SZ
Data: C:\pagefile.sys 150 500

Within that Data field you can adjust the NAME of the pagefile, its LOCATION and its MINIMUM & MAXIMUM values.

I off-line booted the system with my custom Win PE 3.0 bootable USB stick, loaded up the SYSTEM registry hive (how-to link #1 and how-to link #2) and inspected the key value.  Sure enough, it was set to “C:\pagingfile.sys 0 0”

However when I went back and compared the key value in the original “base” image I was building from, the key value was the default “C:\pagefile.sys 0 0”

Somehow between the original base image and the end of the building process, the page file name was being changed.  Was it a virus? A trojan? Some bone-headed system building step I was doing or missing?

Chapter Three: “The Inspector orders broiled Red Herring” 

Sysprep perhaps?

My first thought was that somehow the page file was getting changed in the pre/post Sysprep process.

I inspected both my sysprep.ini and unattended.txt file looking for any clues by didn’t see anything to cause the behavior.

• Description of the KeepPageFile= setting for Windows XP and Windows Server 2003 – Microsoft Help and Support Article ID: 813138
• Unattended_u – Microsoft TechNet
• SysPrep in Depth – Part 5 – Customizing Sysprep.inf – David Remy’s "Ping" blog
• Search Results sysprep – David Remy’s "Ping" blog

While you can control some pagefile settings in sysprep.ini, nothing I found dealt with renaming it.

This was getting crazy.

Chapter Four: “Hide and Evade”

I even ran BareGrep against the whole mounted WIM image file looking for “pagingfile.sys” with no luck.

So I got my system image all ready and sysprepped, shut it down, and off-line booted to manually reset the SYSTEM Registry hive already noted above to “pagefile.sys” and manually deleted both the pagefile.sys and pagingfile.sys files from the system root.

I then let the system power up and went through the semi-automated setup process.

When it was all done after a few reboots I checked the system.

Sure enough, both the pagefile.sys and pagingfile.sys were present again and a check of the Registry hive value showed it had been reset to “pagingfile.sys”.

Grrrr.

Chapter Five: “Fresh Evidence”

I was pretty frustrated at this point.

I needed to capture data that would tell me just what was causing the registry key to be changed and when.  Only one tool came to mind; Process Monitor.

In addition to logging all file, disk, process, and registry actions, Process Monitor also has an option for boot-time logging.  I’ve used it before but had no idea if it would survive through the sysprep process.

So I reprepped the system one more time, copied Process Monitor to the root, fired it up and enabled boot-logging, sysprepped the system and shut it down.  Then I (again) off-line Win PE booted the system, mounted the SYSTEM hive off-line and reset the key back (again) to “pagefile.sys” and deleted the paging files.

I then brought the system back up and walked through the setup process again.

For some reason it went through the initial setup process just fine but when it got to the custom company post-configuration wizard, the wizard locked up.  (This was because I was rushing a bit out of frustration and didn’t bother uninstalling an application the wizard automatically deploys/installs locally as part of the post-setup process.) 

Because the system seemed “hung” I wasn’t able to get at my Process Monitor logging data.  Bummer.  However both page files were again present as well as the modified registry key.

So I went back again and this time, did not copy over the post-configuration wizard exe files knowing it would just toss a “file not found” error when Sysprep first-boot hit that section.  I rearmed Process Monitor, sysprepped the system, off-line fixed the registry key, deleted both of the page files, and rebooted.

This time when the dust settled, the page file was still just “pagefile.sys” and the registry key was preserved.  Process Monitor logs didn’t find any evidence of the change to the registry key.

Very curious.

Chapter Six: “A Suspect is Identified.”

The only thing I did differently was to not run the post-configuration wizard exe.

I figured this needed some investigation as though I knew what we “did” interacting with it, I clearly might not be seeing what was going on beneath the surface.

Right after the post-boot Sysprep process completes, a line in the sysprep.ini files calls to a custom folder we deploy.  In the folder are a number of MSI and EXE install files for some base applications.

The “master” program that calls those is the executable noted in the sysprep.ini file.  When ran it requests the technician input some system naming information, a hard-drive label name, what kind (if any) of third-party network client to load, and what domain it should be attached to. Pretty simple stuff and nothing to do with page files.

It would be nice if I could take a look at the executable’s code and inspect it.  However I’m not a programmer and definitely not experienced in reverse engineering executable files.

But for kicks I tossed the file at Universal Extractor on the off chance it might just be a fancy pants packed batch file of sorts.

That didn’t work too well but the error code suggested it was an AutoIt package.  Now that was interesting.

AutoIt allows creation of automated and advanced scripts which can be packed into an EXE file for deployment.  I’ve seen some great utilities in the past that use AutoIt to create their functionality.

How can I decompile it to get at the underlying script?

Chapter Seven: “The Inspector Gets Help from Didier”

Didier Stevens has two great (but older) posts exploring AutoIt executables and extraction of the au3 script file I was looking for.

• RSR - Didier Stevens
• AutoIt Malware Revisited - Didier Stevens

I downloaded AutoIt and quickly found the Exe2Aut de-compiling utility included with it. However it would not handle the exe file I was working with.

Reading through the posts as well as extensively through the comments, I learned I might have a chance if I could find a hard-to-locate custom AutoIT package that could reverse decompile versions that Exe2Aut wouldn’t process.

The Google hunt began.

And I eventually hit pay-dirt:

• Yet Another Update for AutoIT cracking - obijuan: [techno]plebe blog

This showed me I needed to add UPX: the Ultimate Packer for eXecutables to my toolkit so I could uncompress my AutoIt exe file. Quoting from obijuan’s post:

…first use the upx.exe provided by autoit (\autoit-v3.2.0.1\Aut2Exe\upx.exe) to decompress the .exe:

• Open a command prompt and use upx.exe with the “-d” switch:
• EXAMPLE: upx -d mytestfile.exe
• More about UPX can be found here: http://en.wikipedia.org/wiki/UPX

This will decompress “mytestfile.exe” – it almost doubles in size.

But the links provid by obijuan to the AutoIt cracker were not working.  I did, however, now have a potential utility name “myAut2Exe” to aid in my search.

That led me to myAut2Exe, The Open Source AutoIT Script Decompiler 2.7 B.112 over at BootLand forums where there was a working RapidShare download link to the program package.  I also found and earlier version (2.4) /AutoIt Decompiler – Brad’s Projects – Trac.  And then at this location MyAut2Exe - Collaborative RCE Tool Library over at woodmann.com.  (I’m providing several in case any download links go dead.)

I unpacked the download, ran the main “myAut2Exe” AutoIt executable, dragged my (now upx –d ‘ed) suspect AutoIt exe file onto it and let it rip.

It managed to bypass the password protection, and unpack the file and offer a (semi) readable au3 file.  I say semi-readable as it didn’t seem to be a “pure” AutoIt script, but it had enough of the function calls I could inspect it in detail.

I opened it up with Notepad++ and began hunting.  It was very long and the content was very dense.  So I thought I might get lucky and ran a search for “pagingfile.sys”.

Sure enough, the suspect’s DNA was now hiding in plain sight under one of the function calls:

RegWrite("HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management", "PagingFiles", "REG_SZ", "C:\pagingfile.sys 0 0")

Chapter Eight: “The Trap is Sprung”

Every good Agatha Christie mystery usually ends up with the detective assembling all the participants and re-creating the crime to prove the solution.

So, for the final time, I reprepped the system, set Process Monitor to both enable boot event logging (as well as to not use the page file for logging but rather a custom file on the root so I didn’t lose data when the page-file got renamed, ensured the system was fully prepped and armed, including the post-config wizard folder/files, sysprepped the system and shut it down.  Then I off-line Win PE booted the system, mounted the SYSTEM hive off-line and reset the key back to “pagefile.sys” and deleted the paging files.

I then brought the system back up and walked through the setup process again, all the way through.

Finally, when it was completely done, including the post-boot, wizard-config process stage, I fired up Process Monitor, stopped the logging, and loaded up the captured log file.

I set the filter for the process name of the wizard, and then only for Registry Set values.

Bingo. Confirmed.  There was the wizard’s process changing the registry key to “pagingfile.sys”.

Turns out there actually was quite a bit more registry tweaking going on behind the scenes with the wizard and not just the visible environment/system/network entries the technicians were inputting.  Pretty interesting to find that out.

So, when a system is imaged, Sysprep does its usual pre-configuration system setups, the system reboots, and then sysprep hands the wrap-up off to the wizard, which does its things as well as changing the name of the Windows page file to a non-standard file name.

Mystery solved.

Epilogue

Some things still remain currently unresolved.

I’m not responsible for creating the wizard.  I’ve got a request for information from the group that might be responsible for it inquiring as to the purpose and/or reason for renaming the page file from the Windows default of “pagefile.sys”.  Not sure if it is a typo or some security feature.  Doesn’t make much sense from my perspective at the field level and the size is being left for the system to set automatically still.  However it is so unexpected a configuration “tweak” either it was accidentally copied into (retained) in the wizard if our custom AutoIt developer copied someone else’s original script elements into theirs while building it and didn’t realize (or notice) the impact, or it is for some really clever benefit.

Also, while the wizard changes the page file name, it doesn’t do anything to clean up or delete the old “pagefile.sys” one.  This results as noted in a now unused pagefile taking up over 1 GB of space on the drive.

The wizard does have great benefit and needs to remain in the system setup process.  However this particular behavior adds some headaches at the moment for my imaging building work.

I have to reset the Registry key (well maybe not but am doing so anyway) as well as off-line delete the “pagingfile.sys” file before doing my image captures to keep the image file down by that 1.6 GB size.

ImageX does skip certain files during the image capture process by default, pagefile.sys being one of them.

\$ntfs.log

\hiberfil.sys

\pagefile.sys

"\System Volume Information"

\RECYCLER

\Windows\CSC

I could take advantage of the ImageX Command-Line Options and create an ImageX Configuration File which will skip the “pagingfile.sys” one as well.

And this was cool to learn:

If you rename your Configuration_list.ini file to Wimscript.ini and store it in your ImageX directory (where the ImageX.exe file is located), it will automatically run when you run the /capture option, without requiring you to use the /config option.

Sweet.

However, while this helps me with the ImageX imaging, it doesn’t help with Clonezilla which would continue to capture that file.

So, until I can get clarification from the folks who wrote the custom AutoIt wizard on why it’s present (and maybe get it removed if not critical), looks like I will be manually removing the custom page file off-line before image capture unless I can figure out a way to include automatically cleaning/deleting it during the pre-shutdown sysprep process.  I’m less concerned with the registry correction as it doesn’t seem to hurt anything.

And I’m going to have to decide if I need to confuse our technicians even more by requesting them to delete the (now-unused) pagefile.sys file as part of their post-system image deployment setup steps (to gain back the 1.6 GB lost) or if it is more trouble than it is worth and just let the sleeping dog lay.

Now that was a puzzle worth investigating!

--Claus V.

Post Script:

While researching this issue I uncovered a tip on an annoying sysprep behavior I have noticed.  On some systems when I run sysprep, the process takes a few minutes or less to complete and shut down the system.  Other times sysprep will appear to run for over thirty-minutes before shutdown occurs.  I’ve not been able to find any rhyme or reason to this experience…nor apparently have many others.

Some say it has to do with the mass-storage controller section of the sysprep.ini file. Others say it has to do with rebuilding the driver stores on some systems.

While not really logical, at least one person suggested it is a known (but unacknowledged) bug in sysprep that can be prevented by moving the sysprep dialog boxes a bit while it is running:

Deployment Server Sysprep hanging – Egghead Cafe

Unfortunately, Windows XP has very little to no syprep logging.  I know of one bug only with sysprep that requires you to move, the sysprep dialog box and the hang will clear.

Amazingly, I’ve had success with this technique.  Again, it clearly doesn’t seem to be logical, and maybe I’m doing a rain-dance on a cloudy day, but hey, thought I would share…

--cv