Network Monitoring Madness: Poor Man’s Resource Linkfest

CC image attribution: Network by Claus Rebler – flickr

I had so hoped that the recent GSD Network Linkfest had covered all the necessary bases.

Alas, that was not to be the case.

Despite parking my rear-end on a beautifully constructed office furniture set (cardboard box empties) for the past week and a half on a network infrastructure installation project while the facility remains empty of occupants and furniture, I had to remain productive in my primary duties as well; one of which seems to be enhancing network monitoring and performing utilization analysis.  Sweet.

The Setup

To recap that post I’ve been struggling to get stable long-term captures using Wireshark, but was getting awesome performance and stability under the nmcap CLI version of Network Monitor.  (I suspect it is a hardware resource issue rather than an application issue.).

From there I typically import the resulting capture file into NetworkMiner, export the data I need with SysExporter, then do analysis in Excel.  No problem right?

Well, as I noted, Network Monitor doesn’t save in a .pcap-format.  And NetworkMiner doesn’t process .cap file format.

So I needed to figure out a way (if possible) do handle the conversion, CLI preferred.  And I did.  Sort of.

Network Monitor .CAP to .PCAP format conversion

I could launch Wireshark and import the .cap file, but for some reason, a significant amount of frame information was not being displayed as expected. Not cool or useful.

However, I found the trick in this D-Fens.net LogParser page, about half-way down:

NETMON as an input type

One note about NETMON: I assume nowadays the majority of people use Wireshark (or still Ethereal as it was once called) for capturing network traffic. LogParsers input filter cannot read these captures files. You can then either tell Wireshark to save the capture files as NETMON v2.x file format or better: convert the native pcap files to NETMON later on with "editcap.exe" which comes along with the wireshark package.

editcap.exe -F netmon2 wireshark-input-capture.pcap netmon-output-capture.cap

I applied it to some of my .cap files and it worked like a charm, quickly converting 450 MB size .cap file to .pcap in less than a minute or two. Seriously.

In my case the editcap CLI looked like this.

editcap.exe –F libpcap wireshark-input-capture.cap netmon-output-capture.pcap

I was so excited it worked!

However I did find a potential “gotcha”.  While my results worked great on the capture machine I was working on, when I tried the same thing with a .cap file on my own local system, it editcap complained miserably and would not do the conversion.

Turns out that on the remote system I had installed the previous “current” version of Wireshark, 1.2.9 and the conversion worked great.

However on my local system I was running the Release Candidate version of Wireshark, 1.4.0rc2 and the conversion failed.  Something appears to be different in the editcap.exe shipping and the way they read the .cap file format of Network Monitor 3.4.

See Also’s:

• Editcap Guide: 11 Examples To Handle Network Packet Dumps Effectively – The Geek Stuff blog
• Network Packet Dump – The Geek Stuff blog

I also found that InGuardians has a great tool that might also support conversion of Network Monitor 3.x .cap file outputs to .pcap: <--InGuardians --> Defensive Intelligence <Tools>.

Look closely under that page for the nm2lp (NetMon to LibPcap) tool.  From that page

While the NetMon UI has powerful features for analyzing packet captures, few attack tools include the ability to natively read from the NetMon stored capture file format. In order to leverage tools such as Aircrack-ng, coWPAtty and Cain for wireless analysis, the capture file format needs to be libpcap- compatible. Some tools such as Wireshark support reading and converting NetMon Ethernet captures, but do not correctly interpret NetMon wireless captures.

Fortunately, the NetMon API allows developers to write custom applications and interpret data from NetMon stored captures. Combined with the ability to create a libpcap capture file, it is possible to convert the NetMon file to a libpcap file. nm2lp converts NetMon wireless captures to libpcap format, making them useful in these other tools.

Also be sure to follow the link to read more about nm2lp in Josh Wright's "Vista Wireless Power Tools for the Penetration Tester" (PDF link) paper.

Be aware of the following bugs noted in the utility's “readme” file:

BUGS

+ Timestamps are messed up in the output libpcap file.  This needs resolution.
+ NetMon captures can include multiple link types in a single capture file, while libpcap
  requires the link type to be consistent for a single capture file.  nm2lp assumes the
  input NetMon capture file is all of WiFi link type.

It is a good tool, though I am squirreling away a few copies (exe installer and portable version) of the Wireshark 1.2.9 version until I can be sure that future versions of editcap will support the cap to pcap conversions I depend on.  If anyone can enlighten me on this observation between the editcap versions with some more background knowledge (I’m a noobie in this area!) I would appreciate it.

Network monitoring post interlude

I want to pause for a moment and digress.

Though I have been depending on NetworkMiner for my packet reassembly work, I’ve only been able to use version 0.88 on my XP SP3 systems.  That’s because as I posted in this cry for help to hjelmvik, for all later versions, when I load a .pcap file it loads fine in the application about 1/3 to 1/2 of the way then suddenly the program quits: no lockup, no crash, nothing. Just gone. Process Monitor traces find no smoking guns. I’m waiting for a response.

I suppose I could go to the more sophisticated app NetWitness Investigator which I do have installed and use for capture file analysis, but I just feel bonded with NetworkMiner.

I also found that hjelmvik has another project SplitCap - an open source pcap file splitter which looks interesting.

And while working out the .cap to .pcap method, I somehow stumbled upon the cool network application “Satori” which fingerprints network host OS’s based on network data.   Chatter on the Wire: OS Fingerprinting and Satori.  From the Software page description where you can download the file:

(July, 2010) - Uses WinPCap (almost all testing has been done with 4.1 beta 5 recently). This program listens on the wire for all traffic and does OS Identification based on what it sees. Main things it works to identify are: Windows Machines, HP devices (that use HP Switch Protocol), Cisco devices (that do CDP packets), IP Phones (that send out Skinny packets), and a lot of DHCP related stuff recently, plus some other things. Still early on, will make many changes and will add whatever features are requested, so just send them with packet captures if possible!

Download the zipped file, unextract, run the update.exe and grab the latest files.

It also is cool as Eric Kollmann has included a built-in component updater to ensure you are then downloading just what you need.

Check it out!

Finally, I’ve bookmarked Packet Life since I’m no network analyst though I’m not being paid as such to be one lately!  Gotta get up to speed somewhere!

RE: Port Spanning/Mirroring

I’ll leave out the juicy details on how we are getting our captures, but it is no real secret that it involves a system tied to a spanned port on our network Cisco switches.

As such here are some related materials on that subject for future refrence when needed.

…But first, read and review this brief TaoSecurity post on SPANs versus Taps: TaoSecurity: Expert Commentary on SPAN and RSPAN Weaknesses

It links to two MOST Excellent articles on the issues of using spanned switch ports for collecting your network capture data, both form Tim O’Neill:

• SPAN Port or TAP? CSO Beware (by Tim O’Neill)
• RSPAN … Friend or Foe? (by Tim O’Neill)

OK, now the linkage on SPAN’ing

• Catalyst Switched Port Analyzer (SPAN) Configuration Example - Cisco Systems. A definitive resource.
• Port Mirroring on a Cisco 3550 Switch -danielmiessler.com
• Security Wizardry - Switch Port Mirroring
• How to Configure Local SPAN Port on Cisco Catalyst Switch - ItsyourIP.com

And my oldies but goodies favorites:

CDP - What Switch Am I Connected To? and Monitoring Traffic with Span Ports – SynJunkie.  Two really great posts out of series of ones touching on network monitoring, and Cisco switch/router configuration techniques.  I’m singling these out in particular as they are of interest to sysadmin troubleshooting on the network as well as traffic captures.

More Network Monitor Capture Tips and Tricks

I’ve really been having a fun time writing out CLI examples of nmcap. To recap (copy/paste)  from the recent GSD post on this subject:

I found out from the NetMon 3.4 blog post that high performance captures, Microsoft Network Monitor 3.4 has a CLI component that can be used for GUI-less captures.  And it is also very sophisticated.

• NM3 Command Line Capturing with NMCap – YouTube. Awesome introduction video to the options and power of NMCap CLI. Wow.

PaulErLong, author of the above tutorial has some other great helpful videos on Network Monitor 3.x usage as well:  YouTube – PaulErLong’s Channel

As an added bonus, you can type “nmcap /examples” and get a list of pre-provided examples featuring more advanced CLI usage of the tool.

Another NMCap trick: The Quick and Easy on Using NMCap to Create Circular Network Traces Based on File Size - Microsoft Enterprise Networking Team

Anyway, if you do decide to save your nmcap file output in “chunk” forms, you might need to also know how to reassemble it from the CLI as well.

• NMCap: the easy way to Automate Capturing - Network Monitor Blogs
• Chained Captures and Stitching Them Back Together - Network Monitor Blogs

That second post is very useful as it includes a roll-it-yourself batch file to automate the process of assembling chained .cap files.

Nice.

• Using Color Rules to Show Direction - Network Monitor Blogs.  Ooohhh. Pretty colors!

More Network Odd’s ‘n End’s

Two extra bits of info I found this week:

• TaoSecurity: Time Issues in Libpcap Traces – Mind your timestamps and how your tools interpret them!
• Web Traffic Analysis with httpry – SANS-ISC Handler’s Diary

Additional (Free) Network Monitoring Tools

While monitoring router charts and looking for indications of traffic utilization issues, then firing off a packet capture session, and then later reassembling it and analyzing it does have it’s fun, this clearly is more of a “reactive” approach.

What is the poor-man (or woman!) who is a network analyst/responder to do when the enterprise budget doesn’t support a true enterprise-class solution for real-time network monitoring?

I’m in that boat right now and think I came up with some options that are surprisingly robust and useful…even to us non-professional network guys and gals.

Here are some I have found and like a lot (note: read and respect the EULA’s here gang!).

WildPackets OmniPeek Personal

This one is an oldie, having been yanked and replaced by WildPackets commercial grade applications years ago.  However, based on the reviews I read it looked awesome powereful for a freeware (for personal use!) tool.  Amazingly so.

• OmniPeek Personal – free network monitor, better than Wireshark - 4sysops
• OmniPeek Personal takes on Wireshark : Introduction – Tom’s Guide Review by Davey Winder

Unfortunately, as I've noted, WildPacket's seems to have removed the package from it’s website and most download links return back to WildPackets.  However, if you do a Google Search, one of the top three or so results might net you a working download link for now.

ntop/ntop-XTRA

I had come across ntop before as a network monitoring solution with Open Source support.

It looked really cool and robust.

Only one problem…for me…was that it was provided for Windows as a binary app with limited capture support (1000 packet limit).  To get the full version you can either register for a copy or re-compile the ntop source code yourself.  Something that some wiser folks have found fraught with headaches (recompiling that is, not registering…)

However, it turns out there was an OpenXTRA project hosting an NTop-XTRA build.  Perfect for Windows out of the box.

FREE: NTop-XTRA – Shows network usage - 4sysops

Only it has been killed.

Only again, with some Google searching and tips from 4sysops readers, there is (currently) an active link where NTop_XTRA_3_18_0.exe is still “hosted”.

Snag it while you can.  Right now!

Back? Good. Read/View on..

• How to Install NTop On Windows - eHow.com
• How to configure NTOP for Windows – YouTube video by GigaFinVideo

It also seemed to be providing awesome stats, though with not quite the same level of eye-candy and intuitiveness that OmniPeek Personal generates.

One potential “gotcha” I ran into was trying to log into the settings.  It wanted me to provide credentials.  admin/admin seemed to work fine though I’m not sure if that is what it was or that is what I was setting it to be!

PRTG Network Solutions

Another network traffic monitoring name I kept seeing was PRTG from Paessler.

Turns out that while they do provide enterprise-class network monitoring solutions with their PRTG Network Monitor application, they also provide some fantastic free network tools, frequently updated, free for personal and commercial use!  Now that is citizenship!

• PRTG Network Monitor – See this more info and download page for the freeware version (limited to 10 sensors) download link and PDF manual documentation.
• PRTG Traffic Grapher – Again, see this more info and download page for the freeware version (limited to 10 sensors) download link and PDF manual documentation.

Other free tools of note from Paessler are

• Card Packet Counter – collect stats on network packets passed by a local network card.
• WMI Tester – measure accessibility to WMI counters on systems.
• Site Inspector – web browser that combines IE/Mozilla browser engines into a single platform for site analysis
• URL Recorder – find the URL/POSTDATA strings passed by user when the browse a series of URLS. Might be good for malware/web forensics.

Even MORE (Free) Network Monitoring Tools

This is turning into a post Linkfest overload but it has taken me weeks to track all these down!

Why let them go to waste?

• Show Traffic – By Demonsten at SourceForge.net
• ARGUS- Auditing Network Activity – and ARGUS- Auditing Network Activity - Getting Started (check out the video in action).  Argus is a really detailed tool to monitor and process packet captures and then report it back in digestable methods.  Many sites use Argus to perform network activity audits.  Or as the main page says, “The audit data that Argus generates is great for network forensics, non-repudiation, network asset and service inventory, behavioral baselining of server and client relationships, detecting very slow scans, and supporting Zero day events.”
• ArgusEye - A GUI for Argus
• Periscope – Network monitoring application.
• SecViz - Security Visualization – Site to discuss and view ways reams of data can be better visualized for analysis and understanding.
• Isis: An Infovis System for Investigating Intrusions – Concept project information.  “Isis is an infovis system for investigating intrusions. It allows network security administrators to visualize traffic using timelines and event plots in order to reconstruct the sequence of events that make up an intrusion.” Seems interesting.
• Zenoss Open Source Server and Network Monitoring – Neat and deep reaching platform.
• OSSIM - The OSS Correlation and Security Suite – Hosted by AlienVault this is another fan-favorite and is very sophisticated.
• SolarWinds is one of the enterprise-class industry giants in this field. However they do offer some free free tools beyond the trial versions.  Specifically worth looking into is their SolarWinds Free Real-time NetFlow Analyzer
• Nagios – Hunt around and then find their Nagios Core OSS edition.  There is also a Nagios Core 32bit Windows Installer to check out.

Additional Network Monitoring Resources

Here are additional links that have a ton of resources for software applications as well as methodologies.  It’s a great place to get lost in.

• ArcaneTek from LowTek: Network Analyzers and Top Talkers for Windows – My  original starting point for much of the follow-on tools listed in this post.
• Top 6 Traffic Monitoring Tools – SecTools.org
• Top 100 Network Security Tools – SecTools.org
• Network Monitoring Tools – Clearly updated with love and OCD by Les Cottrell.  I promise your head will spin with the info on this page!

In closing (if you’ve bothered to read this far, thank you!) I do want to say this.

One of the challenges many of us in the IT shop have is convincing the board-members and holders of the budget strings just how critical software like this is.  Often we are tasked with solving a complex IT problem and have little or no resources (at the onset) to support us.

That includes not just day-to-day operations, but particularly network management and incident response.

Having companies and individuals who produce tools like these, and provide them for free (or even demo/trial versions) can really make life better for us all.

By being able to deploy them in real “live-fire” situations after some testing/vetting in a lab really allows us to then go back after a while and make the case with real data and real results why investment into a commercial/enterprise professional application might be a win-win for both the organization and the poor IT shop charged with the task.

So to all the folks named and un-named above who have or do provide these tools and services, for free/trial basis, I say, “Thank you.”

And to you in the IT shops who might come across this post…I hope you find something inspirational and useful.

Cheers!

--Claus V.