Saturday, July 31, 2010

Network Monitoring Madness: Poor Man’s Resource Linkfest

image

CC image attribution: Network by Claus Rebler – flickr

I had so hoped that the recent GSD Network Linkfest had covered all the necessary bases.

Alas, that was not to be the case.

Despite parking my rear-end on a beautifully constructed office furniture set (cardboard box empties) for the past week and a half on a network infrastructure installation project while the facility remains empty of occupants and furniture, I had to remain productive in my primary duties as well; one of which seems to be enhancing network monitoring and performing utilization analysis.  Sweet.

The Setup

To recap that post I’ve been struggling to get stable long-term captures using Wireshark, but was getting awesome performance and stability under the nmcap CLI version of Network Monitor.  (I suspect it is a hardware resource issue rather than an application issue.).

From there I typically import the resulting capture file into NetworkMiner, export the data I need with SysExporter, then do analysis in Excel.  No problem right?

Well, as I noted, Network Monitor doesn’t save in a .pcap-format.  And NetworkMiner doesn’t process .cap file format.

So I needed to figure out a way (if possible) do handle the conversion, CLI preferred.  And I did.  Sort of.

Network Monitor .CAP to .PCAP format conversion

I could launch Wireshark and import the .cap file, but for some reason, a significant amount of frame information was not being displayed as expected. Not cool or useful.

However, I found the trick in this D-Fens.net LogParser page, about half-way down:

NETMON as an input type

One note about NETMON: I assume nowadays the majority of people use Wireshark (or still Ethereal as it was once called) for capturing network traffic. LogParsers input filter cannot read these captures files. You can then either tell Wireshark to save the capture files as NETMON v2.x file format or better: convert the native pcap files to NETMON later on with "editcap.exe" which comes along with the wireshark package.

editcap.exe -F netmon2 wireshark-input-capture.pcap netmon-output-capture.cap

I applied it to some of my .cap files and it worked like a charm, quickly converting 450 MB size .cap file to .pcap in less than a minute or two. Seriously.

In my case the editcap CLI looked like this.

editcap.exe –F libpcap wireshark-input-capture.cap netmon-output-capture.pcap

I was so excited it worked!

However I did find a potential “gotcha”.  While my results worked great on the capture machine I was working on, when I tried the same thing with a .cap file on my own local system, it editcap complained miserably and would not do the conversion.

Turns out that on the remote system I had installed the previous “current” version of Wireshark, 1.2.9 and the conversion worked great.

However on my local system I was running the Release Candidate version of Wireshark, 1.4.0rc2 and the conversion failed.  Something appears to be different in the editcap.exe shipping and the way they read the .cap file format of Network Monitor 3.4.

See Also’s:

I also found that InGuardians has a great tool that might also support conversion of Network Monitor 3.x .cap file outputs to .pcap: <--InGuardians --> Defensive Intelligence <Tools>.

Look closely under that page for the nm2lp (NetMon to LibPcap) tool.  From that page

While the NetMon UI has powerful features for analyzing packet captures, few attack tools include the ability to natively read from the NetMon stored capture file format. In order to leverage tools such as Aircrack-ng, coWPAtty and Cain for wireless analysis, the capture file format needs to be libpcap- compatible. Some tools such as Wireshark support reading and converting NetMon Ethernet captures, but do not correctly interpret NetMon wireless captures.

Fortunately, the NetMon API allows developers to write custom applications and interpret data from NetMon stored captures. Combined with the ability to create a libpcap capture file, it is possible to convert the NetMon file to a libpcap file. nm2lp converts NetMon wireless captures to libpcap format, making them useful in these other tools.

Also be sure to follow the link to read more about nm2lp in Josh Wright's "Vista Wireless Power Tools for the Penetration Tester" (PDF link) paper.

Be aware of the following bugs noted in the utility's “readme” file:

BUGS

+ Timestamps are messed up in the output libpcap file.  This needs resolution.
+ NetMon captures can include multiple link types in a single capture file, while libpcap
  requires the link type to be consistent for a single capture file.  nm2lp assumes the
  input NetMon capture file is all of WiFi link type.

It is a good tool, though I am squirreling away a few copies (exe installer and portable version) of the Wireshark 1.2.9 version until I can be sure that future versions of editcap will support the cap to pcap conversions I depend on.  If anyone can enlighten me on this observation between the editcap versions with some more background knowledge (I’m a noobie in this area!) I would appreciate it.

Network monitoring post interlude

I want to pause for a moment and digress.

Though I have been depending on NetworkMiner for my packet reassembly work, I’ve only been able to use version 0.88 on my XP SP3 systems.  That’s because as I posted in this cry for help to hjelmvik, for all later versions, when I load a .pcap file it loads fine in the application about 1/3 to 1/2 of the way then suddenly the program quits: no lockup, no crash, nothing. Just gone. Process Monitor traces find no smoking guns. I’m waiting for a response.

I suppose I could go to the more sophisticated app NetWitness Investigator which I do have installed and use for capture file analysis, but I just feel bonded with NetworkMiner.

I also found that hjelmvik has another project SplitCap - an open source pcap file splitter which looks interesting.

And while working out the .cap to .pcap method, I somehow stumbled upon the cool network application “Satori” which fingerprints network host OS’s based on network data.   Chatter on the Wire: OS Fingerprinting and Satori.  From the Software page description where you can download the file:

(July, 2010) - Uses WinPCap (almost all testing has been done with 4.1 beta 5 recently). This program listens on the wire for all traffic and does OS Identification based on what it sees. Main things it works to identify are: Windows Machines, HP devices (that use HP Switch Protocol), Cisco devices (that do CDP packets), IP Phones (that send out Skinny packets), and a lot of DHCP related stuff recently, plus some other things. Still early on, will make many changes and will add whatever features are requested, so just send them with packet captures if possible!

Download the zipped file, unextract, run the update.exe and grab the latest files.

It also is cool as Eric Kollmann has included a built-in component updater to ensure you are then downloading just what you need.

Check it out!

Finally, I’ve bookmarked Packet Life since I’m no network analyst though I’m not being paid as such to be one lately!  Gotta get up to speed somewhere!

RE: Port Spanning/Mirroring

I’ll leave out the juicy details on how we are getting our captures, but it is no real secret that it involves a system tied to a spanned port on our network Cisco switches.

As such here are some related materials on that subject for future refrence when needed.

…But first, read and review this brief TaoSecurity post on SPANs versus Taps: TaoSecurity: Expert Commentary on SPAN and RSPAN Weaknesses

It links to two MOST Excellent articles on the issues of using spanned switch ports for collecting your network capture data, both form Tim O’Neill:

OK, now the linkage on SPAN’ing

And my oldies but goodies favorites:

CDP - What Switch Am I Connected To? and Monitoring Traffic with Span Ports – SynJunkie.  Two really great posts out of series of ones touching on network monitoring, and Cisco switch/router configuration techniques.  I’m singling these out in particular as they are of interest to sysadmin troubleshooting on the network as well as traffic captures.

More Network Monitor Capture Tips and Tricks

I’ve really been having a fun time writing out CLI examples of nmcap. To recap (copy/paste)  from the recent GSD post on this subject:

I found out from the NetMon 3.4 blog post that high performance captures, Microsoft Network Monitor 3.4 has a CLI component that can be used for GUI-less captures.  And it is also very sophisticated.

PaulErLong, author of the above tutorial has some other great helpful videos on Network Monitor 3.x usage as well:  YouTube – PaulErLong’s Channel

As an added bonus, you can type “nmcap /examples” and get a list of pre-provided examples featuring more advanced CLI usage of the tool.

Another NMCap trick: The Quick and Easy on Using NMCap to Create Circular Network Traces Based on File Size - Microsoft Enterprise Networking Team

Anyway, if you do decide to save your nmcap file output in “chunk” forms, you might need to also know how to reassemble it from the CLI as well.

That second post is very useful as it includes a roll-it-yourself batch file to automate the process of assembling chained .cap files.

Nice.

More Network Odd’s ‘n End’s

Two extra bits of info I found this week:

Additional (Free) Network Monitoring Tools

While monitoring router charts and looking for indications of traffic utilization issues, then firing off a packet capture session, and then later reassembling it and analyzing it does have it’s fun, this clearly is more of a “reactive” approach.

What is the poor-man (or woman!) who is a network analyst/responder to do when the enterprise budget doesn’t support a true enterprise-class solution for real-time network monitoring?

I’m in that boat right now and think I came up with some options that are surprisingly robust and useful…even to us non-professional network guys and gals.

Here are some I have found and like a lot (note: read and respect the EULA’s here gang!).

WildPackets OmniPeek Personal

This one is an oldie, having been yanked and replaced by WildPackets commercial grade applications years ago.  However, based on the reviews I read it looked awesome powereful for a freeware (for personal use!) tool.  Amazingly so.

Unfortunately, as I've noted, WildPacket's seems to have removed the package from it’s website and most download links return back to WildPackets.  However, if you do a Google Search, one of the top three or so results might net you a working download link for now.

ntop/ntop-XTRA

I had come across ntop before as a network monitoring solution with Open Source support.

It looked really cool and robust.

Only one problem…for me…was that it was provided for Windows as a binary app with limited capture support (1000 packet limit).  To get the full version you can either register for a copy or re-compile the ntop source code yourself.  Something that some wiser folks have found fraught with headaches (recompiling that is, not registering…)

However, it turns out there was an OpenXTRA project hosting an NTop-XTRA build.  Perfect for Windows out of the box.

FREE: NTop-XTRA – Shows network usage - 4sysops

Only it has been killed.

Only again, with some Google searching and tips from 4sysops readers, there is (currently) an active link where NTop_XTRA_3_18_0.exe is still “hosted”.

Snag it while you can.  Right now!

Back? Good. Read/View on..

It also seemed to be providing awesome stats, though with not quite the same level of eye-candy and intuitiveness that OmniPeek Personal generates.

One potential “gotcha” I ran into was trying to log into the settings.  It wanted me to provide credentials.  admin/admin seemed to work fine though I’m not sure if that is what it was or that is what I was setting it to be!

PRTG Network Solutions

Another network traffic monitoring name I kept seeing was PRTG from Paessler.

Turns out that while they do provide enterprise-class network monitoring solutions with their PRTG Network Monitor application, they also provide some fantastic free network tools, frequently updated, free for personal and commercial use!  Now that is citizenship!

Other free tools of note from Paessler are

  • Card Packet Counter – collect stats on network packets passed by a local network card.
  • WMI Tester – measure accessibility to WMI counters on systems.
  • Site Inspector – web browser that combines IE/Mozilla browser engines into a single platform for site analysis
  • URL Recorder – find the URL/POSTDATA strings passed by user when the browse a series of URLS. Might be good for malware/web forensics.

Even MORE (Free) Network Monitoring Tools

This is turning into a post Linkfest overload but it has taken me weeks to track all these down!

Why let them go to waste?

Additional Network Monitoring Resources

Here are additional links that have a ton of resources for software applications as well as methodologies.  It’s a great place to get lost in.

In closing (if you’ve bothered to read this far, thank you!) I do want to say this.

One of the challenges many of us in the IT shop have is convincing the board-members and holders of the budget strings just how critical software like this is.  Often we are tasked with solving a complex IT problem and have little or no resources (at the onset) to support us.

That includes not just day-to-day operations, but particularly network management and incident response.

Having companies and individuals who produce tools like these, and provide them for free (or even demo/trial versions) can really make life better for us all.

By being able to deploy them in real “live-fire” situations after some testing/vetting in a lab really allows us to then go back after a while and make the case with real data and real results why investment into a commercial/enterprise professional application might be a win-win for both the organization and the poor IT shop charged with the task.

So to all the folks named and un-named above who have or do provide these tools and services, for free/trial basis, I say, “Thank you.”

And to you in the IT shops who might come across this post…I hope you find something inspirational and useful.

Cheers!

--Claus V.

Sunday, July 25, 2010

Network Linkfest

I decided these links merited a post of their own.

I really like the content on wirewatcher as Alec Waters does a fantastic job showing the capabilities of network traffic analysis along side with system analysis response.  I’m just a kindergartener in network analysis but the concepts and methodologies used by the professors are top-notch introductions to key concepts.  Added to my RSS feed list.

In other news,  our ongoing peak traffic capture work is netting some interesting results.

One of the observations is that our dedicated capture systems may not be robust enough to handle the volume of traffic the spanned port is throwing at them.  We have been using the latest stable Windows version of Wireshark but even though we set captures to run in “chunks” for limited periods of times, there have been multiple occasions when we return to the systems to retrieve the .pcap sets for analysis, it turns out the Wireshark capture crashed mid-way through the run.  I’m almost certain it is a resource issue.

So it was with interest that I read this post:

Recent Network Monitor builds ship with several capture filters, one of which is a “high performance capture” filter.  So I installed NetMon 3.4 on a dedicated capture system, got the latest parser sets, and then configured a test session to run at a peak time (around lunchtime at the remote site), and let it rip.

I came back an hour later and it had captured a tremendous level of frames, with no drops found….and it was still chugging away until I ended the capture session.

Nice.  I was very impressed with the results.

Only Network Monitor saves the captures in the “.cap” format, something NetworkMiner doesn’t handle.

Wireshark does, so I imported the massive .cap file into Wireshark, intending to then convert it into ".pcap” format, which NetworkMiner does recognize.  Unfortunately, I got the oft-seen Wireshark crash do to insufficient memory resources error.  Bummer.  I’ve gotten that before assembling chunks as well and in that case had to use the command line Wireshark tool mergecap to do so without memory errors.

So firstly, I’m wondering if maybe using Wireshark’s dumpcap to do non-GUI captures might be more stable for longer capture runs.  Figure I can make up some batch files for different scenarios and fire at will. And these would be in the .pcap format.

Also, secondly, I could possibly use the command-line tool tshark or editcap to do the .cap to .pcap conversions with fewer overhead resources were I to stick with NMcap as my capture engine?  Looks like I got some experimenting to do.

I’m also going to deploy and try the Wireshark Development Release version as well to see if maybe these builds help with the memory resource crashes I’m running into in processing the .cap/.pcap files in the GUI environment.

Turning back to Network Monitor…

Likewise, I also soon found out from the NetMon 3.4 blog post that for even higher performance captures, it also has a CLI component that can be used for GUI-less captures.  And it is also very sophisticated.

PaulErLong, author of the above tutorial has some other great helpful videos on Network Monitor 3.x usage as well:  YouTube – PaulErLong’s Channel

Definitely worth bookmarking and reviewing if you are new to Network Monitor usage.

As an added bonus, you can type “nmcap /examples” and get a list of pre-provided examples featuring more advanced CLI usage of the tool.

Another NMCap trick: The Quick and Easy on Using NMCap to Create Circular Network Traces Based on File Size - Microsoft Enterprise Networking Team

And the Network Monitor parsers on CodePlex are even more updated than those shipping in the NM 3.4 download package…

Then before you leave, snag the following “Experts” that Network Monitor 3.x supports

  • NMTopUsers - Release: Top Users 2.1 – Look carefully as they have two sets, “Top Users by Conversation” and “Top Users by Endpoint”.  Again, both are available in x32 and x64 depending on which Network Monitor build you are running.

Goodness my head is spinning now!

So much work/learning to do….

--Claus V.

Saturday, July 24, 2010

Super-Fast Linkfest Throw-down: Pt II

More for the masses!

Stuff

Microsoft Security Essentials Beta: The Next Gen Edition (2.0)

I logged into Microsoft Connect and downloaded both the x32 and x64 flavors for various systems I work/play on.  Installations went great and no ill effects seen so far.

Utilities Galore

  • SearchMyFiles – freeware – Nir Sofer’s tool has some more options now.  My favorite power file-search tool!  See also the freeware tool SMF – Search my Files over at funk.eu also a rockin’ tool.
  • Updates: TCPView v3.0, Autoruns v10.02, ProcDump v1.81, Disk2vhd v1.61 – Sysinternals
  • FREE: imagepatcher – Update WIM images – 4sysops links to a tool (script) that will put patches on all/selected images in a WIM/VHD file.  Pretty cool and based on Microsoft Powershell.  imagepatcher home.
  • WhatChanged 1.07 – freeware – VTask Studio – this version update for file/registry diff’ing got a speed/performance enhancement.  Quite a handy tool for when Windows System State Analyzer is just too much brawn for your needs.
  • Double Driver – freeware – This utility for backing up/restoring your Windows system drivers (Microsoft/OEM/third-party) is an awesome update.  I used it recently to back up and collect all my XP AT&T Sierra 881 Wireless card drivers.  Then we took those to an identical platform that wouldn’t recognize those same drivers from the same installer and restored them to that system.  Flawless.  I liked the options in the older 2.0 builds but this 4.0 version GUI is super-slick and mature.  Stop and get it right now!  You never know when your driver backup may come in handy!  See also DriverBackup! tool at SourceForge.net and DriverMax for alternative driver backup tools and Nir Sofer’s DriverView as well for cataloging them.
  • Microsoft PowerToy Image Resizer – (XP only) – is a wonderful tool I add to supported user’s systems.  They can then right-click/resize photos from digital cameras in a no-brainer method so our email doesn’t get clogged with super-sized image attachments.  However it isn’t an option for Vista/Windows7 users.  So along comes…
  • Image Resizer Powertoy Clone for Windows – via CodePlex and Brice Lambson – it mimics XP’s Image Resizer features perfectly on Vista/Windows7.  (Note: as for my own on-the-fly image resizing, I use these, but for batch-image resizing work rely on FastStone Photo Resizer and FastStone Image Viewer.  (spotted via this freewaregenius review)

Sandboxing

I’ve been surprised to see the following super-flood of interest again in sandboxing applications and processes!  I guess that’s a good thing.

And since Adobe PDF files seem to be a very common and growing vector for malware attack, it seems good measure to see the following focus:

Then there is system sandboxing in general.

And some tools/software to help make things easy for the end user

Oh My! Firewall Overload

And an updated round-up of Free Windows Firewalls

    1. Comodo Internet Security 
    2. PC Tools Firewall Plus Free Edition
    3. ZoneAlarm Free Firewall
    4. Ashampoo FireWall Free
    5. Online Armor Free
    6. Agnitum Outpost Firewall Free 
    7. Filseclab Personal Firewall Professional Edition

Also see these additional ones:

Please do your research very carefully if you are really looking specifically for "out-bound” leak protection as many malware/root-kits have tricks up their sleves to evade and bypass less-than-robust outbound filtering by firewall products.

MatouSec is still probably one of the leading groups doing firewall leak testing/validation.  See the following site for a roundup

Of course, if all you want is solid inbound firewall protection  Windows Firewall - Windows 7 is seriously robust, you just have to remember to enable it!

Vertical Tabs in Chrome

I was reading a recent TechBlog daily link-post and followed a link to Chrome 6: What made the cut--and what missed it.  While that article was interesting, what caught my eye was reference to "side tabs" were pushed back to Chrome's 7.  Was this possibly the elusive Work under way to add sidebars to Google Chrome I and other Chrome/ium fans are longing for so we can have bookmarks managed from the side like in Firefox, IE, Opera and (kinda-sorta) in Safari?

Nope, but it is cool still, and you can sort-of get them in Chrome anyway…

…though the effect isn’t terribly pleasing in my eyes.

--Claus V.

Fun Tiny Find: Mouse Jiggler

I’ve been doing some video conversions and noticed that when my screen-saver kicks on, it sometimes impacts the recoding process.

I guess I could have turned off the screen saver, but that is such a hassle.

I could have purchased the WiebeTech - Mouse Jiggler™ product, but that seems a waste of cash.

I could have rigged up a Poor Man’s Mouse Jiggler (video) but couldn’t figure out how to strap the mouse to my wrist over my watch.

Then I found the freeware Mouse Jiggler utility.

No installation required. Windows compatible (does need .NET 2.0 or higher).

Just 49 KB in size

Did the trick perfectly.

Jiggle away!

--Claus V

Super-Fast Linkfest Throw-down: Pt I

07/31/10: More updates posted at the bottom of this page regarding the .LNK vulnerability.

I had planned for a sleepy weekend.  You know, the kind to recharge your batteries after a crazy-insane work-week?

Yeah right.  Like that would come to pass….work responses required weekend dedication.  Bummer

So I offer only several rapid-fire posts to get the linkage unblocked.

Part I here continues with additional information I’ve saved on the Windows LNK exploit; building upon the my previous post: Windows zero-day exploit?: USB storage + .lnk file...

Unless something radical drops, this will probably be the last on this theme.  I’ve found it an interesting look into incident response, knowledge-sharing/growth, and a few more cool tools.

07/31/10: More updates

Out of band Microsoft update to fix the .LNK vulnerability exploit coming August 2, 2010.  Just be aware, no soup for XP-SP2 and W2K systems!

Saddle up!

--Claus V.

Saturday, July 17, 2010

Security and Forensics Linkfest: Weekend Edition

Keeping with the GSD Theme this weekend…

Hang on, I’ve tried to group these a bit but it is still a ride all over the place.

PlainSight – Forensic LiveCD that incorporates counts RegRipper amongst the power-tools in the features. Can be run from a bootable USB device or CD and can be pointed to disk images or local disks.

The Windows Forensic Environment blog has some WinFE Teaser Screenshots showing some of some of Colin Ramsden’s custom WinFE build Work.  Very sexy stuff!  I can’t wait!  And for the CLI-challenged crowd, working with DiskPart in WinFE might take a moment to get used to. Luckily Brett Shavers has some easy tips in his DiskPart article as well as a cross link provided to his exhaustive The (Nearly) Perfect Boot CD (PDF) document.

For the past two weeks I’ve been working on a special project recovering lost files from a 1 TB drive (well, recovering them from a cloned copy of the original drive).  Although I already had my tools and methodology down before beginning, I did some some research in the process to make sure there weren’t any tips and tricks that could enhance my response work.   Here are some links that I found resourceful to save.  I may or may not be able to share details on this project sometime in the future.

REMnux: A Linux Distribution for Reverse-Engineering Malware is another pretty cool LiveCD ISO and/or VMWare image.

Finally, the off-line bootable Offline NT Password & Registry Editor for Windows systems has been updated to 100627 at the end of June 2010 by Petter Nordahl-Hagen.  Supports NT/2k/XP/Vista/Win7.

Computer Forensics - Windows Search forensics – by Joachim Metz is a great review on “Analyzing the Windows (Desktop) Search Extensible Storage Engine database”

Quickpost: Preventing the /Launch Action “cmd.exe” Bypass and Quickpost: No Escape From PDF by Didier Stevens get into PDF security issues.

Harlan Carvey Windows Incident Response: Links post touches on Didier’s work and then goes deeper on infection vector sources.  Also spotted in that post is that ShadowExplorer is at version 0.7.  This Manual page should give you an overview.

Lots of goodies in this Digital Forensics Case Leads: Spies, Social Networking Experiments, Live CDs & More SANS Forensic Blog post including the Orion Live CD (read the paper PDF first), and an assortment of free tools (link lists) provided by Forensic Control.

We are now doing regular network traffic monitoring, so all things Wireshark and forensic packet analysis are top on my read/watch list.  Such as this video of Hansang Bae’s presentation with Wireshark regarding his responsibility for Packet Capture Infrastructure at Citi.

The H Security has an incident summary by Frank Boldewin CSI:Internet - The image of death based on deconstruction of an email containing a suspicious PowerPoint presenation.  Great detail and chock-full of tips for incident responders to take note of.  See also his previous incident story Alarm at the pizza service.

Frank’s website, reconstructer.org, contains even more incident responder goodies.

Nir Sofer’s been busy with a post on Recover L2TP,PPTP,PPPOE,DNS password from the router Web interface as well as updates to SniffPass Password Sniffer and SmartSniff.

Autoruns and Dead Computer Forensics at the SANS Forensic and Incident Response blog details added features (and limitations) of the latest vrsion of Microsoft Sysinternals Autoruns tool; the ability to load and analyze the autorun items from an offline system.  While not perfect it is a large step forward.  In addition, I found that it now loads saved autorun session files (*.ARN) much more cleanly than before.

Help Net Security had a tip to Casper Secure Drive Backup 2.0, a commercial ($) solution for complete system backup of PGP-WDE drives while still in their encrypted state.  This is one of the headaches of WDE.  I work with PGP-WDE systems almost daily, and performing incident response and system support to them is a headache. Sure you can make a backup of your system/data, but unless you capture a sector-based disk image of the whole system for restoration, or off-load the system backup in some “unencrypted” manner, and then re-encrypt that back up data, your back-up version might circumvent the whole point of having WDE in the first place.  Anyway, this is a pretty cool featured solution and the only one specifically for PGP-WDE that I have come across. A free 30-day trial version is provided by the company.  So from both a drive imaging/cloning and PGP-WDE perspective, I’m intrigued.

cqure.net contains quite a collection of script/tools by Patrik Karlsson, primarily vulnerability and penetration testing.  I came across this while looking for a tool to extract my home TightVNC 2.0.2 passwords (which I stupidly set without writing down!).  Alas even Nir’s VNCPassView wasn’t able to handle the new version.  So I just uninstalled TVNC from my couple of systems, then reinstalled and set new passwords, writing them down this time.

WinTaylor 2.0 is out! via the CAINE forensic LiveCD folks.  WinTaylor is a Windows based auto-launching interface for a collection of utilites for incident response, system auditing, and forensics work.  There are a few distros like this, boot the system with the LiveCD side, or auto-run the CD in a running Windows system and use these tools. 

The Sleuth Kit (TSK) & Autopsy: Open Source Digital Investigation Tools – yes, probably anyone and everyone who cares knows that TSK is now updated to 3.1.3 at the start of July 2010.

JL’s stuff: MovingHow To Respond To An Unexpected Security Event Forward was a tiny post but packed quite a punch by including these links:

As you may or may-not know, Google has been offering https:// based encrypted search support now via https://www.google.com.  However, that did cause some issues for users doing regular Google site searches when network administrators blocked the https:// based google location.  So now it has bee moved to https://encrypted.google.com.  I mention this mostly as traffic monitoring might turn up some indication of these sessions but without the details normally expected to be found from google.com bases search traffic. This Official Google Enterprise Blog: An update on encrypted web search in schools post gets into some of the details as do these posts:

For forensic packet capture reassembly, NetworkMiner Network Forensic Analysis Tool (NFAT) and Packet Sniffer is a favorite tool and I see it was updated to version 0.92 back at the end of May 2010.  I’ve had stability issues on my XP/Win7 systems for some reason with it, the packet capture assembly bombs out when loading PCAP files.  I’ve not tried 0.92 yet and had to go all the way back to v 0.88 or 0.89 for a stable build for some reason.  Don’t forget about NetWitness Investigator Software (also free) and Xplico - Internet Traffic Decoder updated to version 0.5.8 in late June 2010.  You can find it now on many LiveCd’s but they also now offer a VirtualBox image for it as well. Cool!

Finally, this Digital Forensics Case Leads: Ann’s Aurora Edition at the SANS Forensic and Incident Response blog ended being a real time-sink for me!

Somehow from that post I tripped over all the following links chasing the white rabbit.

Looks like it is one of many tools that uses Python.  I’m seeing it pretty often so I wonder if it would be worthy my while to get some foundational knowledge and experience in working with it on Windows systems?

Wozers!

--Claus V.

Windows zero-day exploit?: USB storage + .lnk files + file explorer = FAIL

Update 07-17-2010 – 6:45 PM (CST) I’ve added a couple more late-breaking details added to the bottom of the post

The best summary brief on the issue I’ve seen comes from F-Secure Weblog: Espionage Attack Uses LNK Shortcut Files. Quoting…

There's a possible new zero day in the wild which is being used in targeted espionage attacks. Belorussian antivirus company, VirusBlokAda, recently published news about two new rootkit samples, and quite interestingly, the infection vector is a USB storage device and Windows shortcut [.LNK] files.

The rootkit uses a LNK file that infects the operating system when viewed by an icon rendering file explorer such as Windows Explorer or Total Commander.

According to Krebs on Security, the method is capable of infecting a fully patched Windows 7 computer.

From Krebs: Jerry Bryant, of Microsoft, stated that "Microsoft is investigating new public claims of malware propagating via USB storage devices. When we have completed our investigations we will take appropriate action to protect users and the Internet ecosystem."

Our initial analysis of the samples appears to indicate that the shortcuts somehow take advantage of the way in which Windows handles Control Panel shortcut files.

Got the gist?

According to Microsoft, it appears all versions of Windows from XP through Windows 7 are vulnerable to this attack method.  Oh bother!

OK, more reading now:

Finally initial deep analysis for the hard-core set:

The current thinking is that an expired but still valid Realtek Semiconductor Corp driver signing certificate is being used to load/install the root-kit malware files “mrxnet.sys” and “mrxcls.sys”. 

Microsoft also reported they have worked with VeriSign and Realtek and now have had the particular driver-signing certificate used in this initial attack revoked.

From the Microsoft Malware Protection Center Post:

Threat details

What is unique about Stuxnet is that it utilizes a new method of propagation. Specifically, it takes advantage of specially-crafted shortcut files (also known as .lnk files) placed on USB drives to automatically execute malware as soon as the .lnk file is read by the operating system. In other words, simply browsing to the removable media drive using an application that displays shortcut icons (like Windows Explorer) runs the malware without any additional user interaction. We anticipate other malware authors taking advantage of this technique. Stuxnet will infect any usb drive that is attached to the system, and for this reason we’ve classified the malware as a worm.  This classification for the malware should not be confused with another vector used by this worm, the newly disclosed vulnerability (CVE-2010-2568) covered in today’s advisory.  The vulnerability itself is not wormable.

Stuxnet uses the aforementioned .lnk technique to install additional malware components.  It first injects a backdoor (Worm:Win32/Stuxnet.A) onto the compromised system, and then drops two drivers:

  • Trojan:WinNT/Stuxnet.A - hides the presence of the .lnk files
  • Trojan:WinNT/Stuxnet.B - injects (formerly) encrypted data blobs (.tmp files) into memory, each of which appear to serve different purposes as the Stuxnet deployment system infrastructure (drivers, .lnk files, propagation, etc.).

These drivers are signed with a digital certificate belonging to a well-known hardware manufacturer called Realtek Semiconductor Corp., which is unusual because it would imply that the malware authors somehow had access to Realtek’s private key.  Microsoft MMPC has been working with Verisign to revoke this certificate, and did so at 08:05:42 PM UTC with the agreement and support of Realtek.

Also the fact that currently some evidence exists that Siemens WinCC SCADA systems seem to be the target could be based on the fact they seem to require use of a standard id/password set to correctly operate.  However that is sure to change as other attackers build upon the now disclosed vulnerability.

A workaround (temporary fix) offered by Microsoft in the Microsoft Security Advisory (2286198) is to disable the display of icons for shortcuts as follows:

1. Click Start, click Run, type Regedit in the Open box, and then click OK

2. Locate and then click the following registry key:
HKEY_CLASSES_ROOT\lnkfile\shellex\IconHandler

3. Click the File menu and select Export

4.In the Export Registry File dialog box, enter LNK_Icon_Backup.reg and click Save
Note This will create a backup of this registry key in the My Documents folder by default

5.Select the value (Default) on the right hand window in the Registy Editor. Press Enter to edit the value of the key. Remove the value, so that the value is blank, and press Enter.

6. Restart explorer.exe or restart the computer.

Impact of workaround.Disabling icons from being displayed for shortcuts prevents the issue from being exploited on affected systems. When this workaround is implemented, shortcut files and Internet Explorer shortcuts will no longer have an icon displayed.

It seems that most all GUI-based Windows file explorer tools, including those not from Microsoft, could trip the vulnerability when used to view an infected USB storage device. One such text-based Windows file explorer tool that does not is reported to be Far Manager so you probably would do well to keep a version of this one on your systems when doing examinations of USB devices.

Of course, it might be even better to use a lab-system, and one running a LiveCD distro of a Linux-based OS to do your suspect USB storage device examinations….just saying…

Finally, incident responders might also be well to know that Windows Incident Response bloggest Harlan Carvey has also weighed in on this, as well as the issues it illustrates for responders:

Whew!

This is still a breaking event so I’m sure more information will be coming in the days ahead as everyone devotes more resources to research and analysis (and hopefully Microsoft with a vulnerability patch…unless it turns to be a “feature”).

Update 07-17-2010 – 6:45 PM (CST)

The H Security team in their Trojan spreads via new Windows hole post adds this observation:

Microsoft has been informed about the vulnerability, but appears to have problems with reproducing it. Andreas Marx of AV-Test says that every .lnk file is linked to the ID of the newly infected USB Flash drive. This means that the sample trojans found so far can't simply be started on an arbitrary Windows system – the malware will only start in the OllyDbg debugger after some modifications to the code.

And the SANS-ISC Storm Center Handler’s Diary has this post Vulnerability in Windows "LNK" files? with findings from the handler’s work on malcode they got their hands on.  Quoting from Bojan’s update to the article notice,

I've tested the exploit and can confirm that it works in Windows XP, Vista and Windows 7. The exploit uses a specially crafted LNK file. This file allows the attacker to execute an arbitrary file by carefully specifying its location – the LNK file in itself does not exploit any vulnerability such as buffer overflows, for example, so it is a legitimate LNK file. The LNK file used in targeted attacks was manually crafted as some fields that are normally present, such as CreationTime, AccessTime or WriteTime are all set to 0.

I will not be posting details about how the exploit works, but here are some things that you should be aware of:

  • If autorun is disabled, when a USB device with malicious LNK files is inserted, the exploit will not be triggered automatically.
  • The exploit is triggered every time a folder containing a malicious LNK files is opened (for example, with Windows Explorer). It does not matter where this folder is – it does not have to be on a USB device, but in order to execute to malicious binary, the attacker has to specify its location correctly.

What makes this vulnerability extremely serious is the fact that it can be opened from any place, including remote shares, for example. The victim just has to browse to the remote share in order to trigger the vulnerability. So double check permissions on any remote shares you use in your companies (you shouldn't allow users to write in root folders, for example).

Seems like it’s a security-focused posting weekend here at GSD!

Cheers!

--Claus V.

Threat Vector: Xerox WorkCentre Pro scanned to email documents?

I was checking my security feeds this weekend and found a nice little nugget via Donna’s SecurityFlash

Seems the Tech Herald’s offices got hit with an email scam claiming to have a file sent by a local Xerox WorkCentre Pro multifunction device.

More and more organizations are deploying these devices than can function as a fax, copier, network printer, and scanner resource.  By combining multiple features in a single networked device, I’m sure businesses are hoping to leverage cost savings and production efficiencies to their employees.

Overall, while they can be an IT management headache to manage, configure, monitor, update, and support…that’s why the company pays the vendors and IT department all those big bucks we see monthly, right?

Anyway, a quick examination of the email set of warning signs…the scanned document was presented as a “ZIP” compressed file, rather than as a more common PDF file (or TIFF or XPS format as mentioned).  Plus the message body didn’t quite match.

When the attachment embedded in the ZIP file was unpacked and scanned with malware tools, it was flagged immediately as a trojan in Microsoft Security Essentials though, as Steve points out, many other vendor scans via Virus Total at the time didn’t identify it as such.

Using emails as a malicious infection-vector isn’t new by any means.  However, with the increase in these multi-function devices to workplace environments, coupled with many employees receiving little to no training, the risks seem higher.

Image the following scenario.

Users at the mythical industrial leader CorporationX (I just made that up) recently have a similar multi-function device installed across their organization.  Besides network printing, they do experience great buy-in upon learning of the “scan-to-email” feature.  Previously only certain executives and their administrative-support pool had access to document scanners.  Now they can all digitize hard-copy material simply by placing it on the machine, selecting the scan-to-email feature, and putting in their email address.

Automagically when they go back to their desk, there is a standard email with their PDF document waiting!

Who wouldn’t like the idea of being freed from hard-copy handling and moving finally to the digital world?

Only in this case, multiple administrative and executive users at CorporationX got an email from a Xerox system that they themselves didn’t initiate/scan to themselves.  No matter.  It looks legit and because the default setting allows the email to go out with a generic “From” Xerox sender, a few of them figure that maybe one of their peers was copying them in (you can enter other email address names besides your own when sending the scanned document).

Must be some important corporate info!

Better open it up and take a peek immediately!

Strangely, Adobe Reader opened the document, flashed briefly then closed.  Then reopened with a document that had nothing to do with CorporationX.

Oh well, think all the users, someone must have scanned in the wrong document…

Unfortunately, the email (like that received by The Tech Herald) was in fact, not sent from within CorporationX but was a spoofed/forged email.

Embedded within this PDF was specially crafted exploit code that ended up dropping a root-kit/trojan on the system.  Now CorporationX was serving its secrets right out the back door.

It wasn’t until an IT team-member also received the email, questioned the authenticity and first checked the message header code that they discovered the email had been spoofed and came from an external source, and not from a CorporationX Xerox device. 

Additional investigation found the PDF was in fact embedded with malicious code, and off-line scans of some sample corporate field systems did find evidence of the root-kit/trojan.

So a formal incident-response kicked off and the migraines began for CorporationX as they now tried to determine what corporate info had leaked and what the damage might be and starting trying to find infected systems across the thousands they manage.

Bummer.

Still don’t believe an unsolicited/spoofed PDF attachment is a potential threat vector?

And I would be remiss to mention all of Didier Stevens’ extremely detailed work on PDF exploit research in the same breath.

To be very clear, I’m not at all positing that Xerox WorkCentre systems are bad or a threat (they are in fact just one manufacturer/model of many such option-capable devices in this crowded office-machine category). No I am not picking on Xerox in particular, all such scan-to-email devices can lead to the same complacency and attack vector via email spoofing.

Pretty useful things, they are.  However, their ubiquitous nature (it’s just a fancy copy machine) and the fact that the default configuration sends messages that are so cookie-cutter standardized, really sets up users for some social-engineering FAIL.  How can one expect the average user to first authenticate that the message is valid and legitimate if they are sending them daily to themselves safely, and others can include them as well?  And no one is adding their personal “From” identification into it?

Couple that with the potential threats from malware-hacked PDF file exploits (even more so if the Adobe Reader versions installed haven’t been updated/patched in a very long time) and it could be a nightmare.

I’m still not sure about solutions…disabling scan-to-email and using the more administratively managed “scan-to-mailbox” feature might be one method, or putting in place policy that requires users who do scan such documents to manually put in a valid identifying word or phrase in the subject line, or requiring them to put in their own corporate email address rather than using the default machine one might be a start.  Perhaps a more detailed system deployment that changes the default Scan to Email configuration so that a custom “WorkCentre Email address” name is used that better legitimizes the email notices by checking the “From” field on emails supposedly sent from the corporate device(s)?  Or even the “signature” line?  Check out this Scan to Email (PDF…I know…) quick configuration guide for some of the customization options available on many Xerox WorkCentre systems.  Please do something, anything, to make your internal scanned emails special and identifiable to your employees as being more legitimate.  That will help set the spoofed ones apart much more clearly from your users.

That’s not to say that end-user education, a strong A/V software solution installed on the user system desktops, security software that scans attachments at the email server level, and an IT policy that ensures Adobe Reader is kept current and patched also would be good practices.

I personally get a few “unsolicited” PDF’s scanned from Xerox systems in my email a week.  And I promise you, I check them all very carefully before actually opening the attachment.

So far I’ve not seen any such malware personally, but I think an ounce or two of caution is a Good Thing in this case.

Constant Vigilance!

--Claus V.

Tracking down a BSOD Crash: AESTAud.sys

So a very unusual thing happened on my work system this past week.

It experienced a BSOD.

While that really isn’t a globally unusual thing for Windows users, for me, on my systems, that is pretty rare.

Not only was it rare, it was extraordinarily rare, as it was the second time it happened, out of nowhere, with the same root cause being reported..

Here’s how I dissected the bugger, now that it had captured my full attention.

The first time I experienced the BSOD was back on 06-24-10.  Stuff happens so I just rebooted and went on relatively unconcerned.

The next BSOD event was on 07-14-10.  This go-round I had much more time.

The system is a Dell Latitude E6400 laptop system, freshly issued and sporting a fresh image of XP Pro, SP3.

Once the system rebooted, I fired up Nir Sofer's BlueScreenView to get some quick details on the crash.

Not surprisingly, both of the crash events had been logged and BSV was able to report their details.  Surprisingly, both crashes involved the following element:

AESTAud.sys by the Andrea Electronics Corporation.  The file version was 2.0.0.3 / 32-bit flavor.  This is the Andrea Audio Driver

A round of Google work on that one indicated that it was a legit system driver.

I then fired up Autoruns for Windows and quickly found both the auto-loader for this driver under the HKLM\System\CurrentControlSet\Services

aestaud.sys, 111 K, Andrea Audio Driver, Time: 04/21/2009 10:13 PM, version 2.0.0.3 system32\drivers\AESTAud.sys

…as well as just one other Andrea-releated executable in the auto-run groups under the HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run

aestfltr.exe, 456 K, AEFltrs MFC Application, Time:05/20/2008 10:21 PM, version 4.5.10.0 %SystemRoot%\system32\AESTFltr.exe /NoDlg

More Google work on this executable also indicated it was legit.  And that curious “/NoDlg” argument appended to the executable seems to be a “no dialog / silent” manner of executing it, probably suppressing a windows launching dialog box.

So I unchecked them both in Autoruns to prevent them from loading.

I also consulted with Process Explorer to check what the (still running for now) AESTFltr.exe process was calling:

Process: AESTFltr.exe Pid: 5308

Name    Description    Company Name    Version
ADVAPI32.dll    Advanced Windows 32 Base API    Microsoft Corporation    5.1.2600.5755
AESTFltr.exe    AEFltrs MFC Application    Andrea Electronics Corporation    4.5.10.0
comctl32.dll    User Experience Controls Library    Microsoft Corporation    6.0.2900.5512
comctl32.dll    Common Controls Library    Microsoft Corporation    5.82.2900.5512
ctype.nls           
GDI32.dll    GDI Client DLL    Microsoft Corporation    5.1.2600.5698
IMM32.DLL    Windows XP IMM32 API Client DLL    Microsoft Corporation    5.1.2600.5512
kernel32.dll    Windows NT BASE API Client DLL    Microsoft Corporation    5.1.2600.5781
locale.nls           
MFC42.DLL    MFCDLL Shared Library - Retail Version    Microsoft Corporation    6.2.4131.0
MSCTF.dll    MSCTF Server DLL    Microsoft Corporation    5.1.2600.5512
msctfime.ime    Microsoft Text Frame Work Service IME    Microsoft Corporation    5.1.2600.5512
msvcrt.dll    Windows NT CRT DLL    Microsoft Corporation    7.0.2600.5512
ntdll.dll    NT Layer DLL    Microsoft Corporation    5.1.2600.5755
ole32.dll    Microsoft OLE for Windows    Microsoft Corporation    5.1.2600.5512
RPCRT4.dll    Remote Procedure Call Runtime    Microsoft Corporation    5.1.2600.5795
Secur32.dll    Security Support Provider Interface    Microsoft Corporation    5.1.2600.5834
SHELL32.dll    Windows Shell Common Dll    Microsoft Corporation    6.0.2900.5622
SHLWAPI.dll    Shell Light-weight Utility Library    Microsoft Corporation    6.0.2900.5912
sortkey.nls           
sorttbls.nls           
unicode.nls           
USER32.dll    Windows XP USER API Client DLL    Microsoft Corporation    5.1.2600.5512
uxtheme.dll    Microsoft UxTheme Library    Microsoft Corporation    6.0.2900.5512
VERSION.dll    Version Checking and File Installation Libraries    Microsoft Corporation    5.1.2600.5512
WINMM.dll    MCI API DLL    Microsoft Corporation    5.1.2600.5512

Then I rebooted and now got an error dialog box related to stacsv.exe. I was able to cancel that and all was well…except I didn’t have any audio now and all my control-panel options for the Audio were grayed out.

So I did some more searching for that file in Autoruns and located it under HKLM\System\CurrentControlSet\Services

stacsv.exe, 224 K, Manages audio jack, IDT, Inc. Time: 03/09/2010 11:56 PM, Version 1.0.6274.0 c:\program files\idt\wdm\stacsv.exe

Man, these things were all hooked together quite tightly!

I was hopeful that maybe the Andrea Electronics items were part of an “custom software” package to allow for enhanced sound control management on the system.  Unfortunately a deep search through the Add/Remove Programs (via Nir’s MyUninstaller utility) didn’t find any references to one.

I did find an InstallShield reference to IDT, Inc however.

And in searching on “stacsv.exe” on my system with Nir’s SearchMyFiles tool, I found it in the following locations:

C:\dell\drivers\R267815\WDM
C:\Program Files\IDT\WDM

Dropping to the IDT folder I found the setup.exe file, ran it (to see if a reload helped or maybe it would kick off an uninstall/repair option), the installer balked that the setup was not the right image for the system….and it promptly removed everything in there.

I guess that was progress.

Because I hadn’t logged the files in the IDT\WDM folder before running the setup file which removed them, I next mounted a WIM file I have of the stock system image and looked in the same location.  Lots of stuff in there this time (42 files).  I’ll save you the list, but there were unpacked driver sys files, CPL files, exe files, dll files for all kinds of both x32 and x64 supported systems, and….

…both the AESTAud.sys and AESTFltr.exe files were present and the commonality in the date-stamps seemed to be May-2008 for the most part.

So now I had two more bits of critical info; Andrea Electronics which appears to be supplying the audio driver controls for the IDT provided audio hardware, and that Dell clearly provides a driver package for this stuff known under the moniker “R227815”, and our images seemed to ship with hardware drivers back from 2008.

I next went into C:\dell\drivers\R267815\WDM location and re-ran the setup from that set. Again it complained that the setup was not the right image for the system.  Checking the C:\Program Files\IDT\WDM location again, found 44 files now in that location and that they were all from the March 2009 period.

Hmmm.

Still getting bad driver install errors…not sure why…better to to Dell to pull down a clean set.

I quickly found the R267815.exe –Dell Drivers and Downloads page, confirmed it was compatible with the E6400 Latitude system, and this one had a release date of 05/04/2010, version 5.10.0.6274,A11.  It is for the IDT 92HDxxx HD Audio hardware and “Fixes issue where line-in was selected as default recording device instead of microphone.”  Previous versions can be found at this Dell 92HDxxx HD Audio Support page, where both 2008 and a March 2009 release versions could be seen.

I downloaded the most current 05/2010 version and it did not complain this time when the setup installer was run.

Rebooted the system for good measure and the calls to the previously disabled items in AutoRuns were present and activated (note the disabled ones were still present, so I removed those duplicates), but now were reporting as follows:

aestfltr.exe, 720 K, AEFltrs MFC Application, Time:07/07/2009 2:06 AM, version 5.0.0.5 %SystemRoot%\system32\AESTFltr.exe /NoDlg

stacsv.exe and aestaud.sys remained unchanged.

I’m not sure why the previous IDT setup packages I found already present on the system failed due to an image compatibility problem.  However the last I downloaded directly from Dell did work and I can see evidence that some files related to the original BSOD party have now been updated to newer release versions.

To date, I’ve not had any additional BSOD issues, and will be hopeful whatever triggered both crashes has now been resolved with this last update.

Nor is it clear to me (I haven’t tried to do a debugging session on the original crash data yet) why audio-drivers were causing a system crash.

In the meantime, I will be watching closely and plan to clean up these older/cranky audio driver packages from our base image next time I build a refreshed system image.

Cheers!

--Claus V.