A while back we deployed some special purpose server systems. They are pretty simple in terms of function but do provide some critical infrastructure support.
So when they go off-line for whatever reason (power failure, unplugged network cable, etc.) we need to respond to get them back online.
Proactive monitoring is pretty thin and currently we have more of a reactive monitoring solution. Someone needs one of these systems, finds it is down, and call us to fix it fast.
Nice.
So one solution developed was as simple batch file that ran a ping against the server IP’s. If a ping failed, then notice would be auto-emailed to selected staff to check it out.
That seems ok, but what happens when the NIC is up and responding to ping, but the core applications/OS has actually hung and it really isn’t “operationally” on-line, though the NIC is? Kinda gives the impression you don’t have a problem that you really do.
These systems are very simple and we can’t run any additional “client" software on it to “phone home” for service health and availability…something like Paessler's PRTG Network Monitor.
I did identify a few critical network services running on the systems and found that they communicated out on specific ports.
If we could run port-scans against those ports and found them open/listening, then that might provide a more accurate assessment of the servers’ health rather than the basic ping reply/no-reply feedback.
So here are a few of the tools and utilities I considered in that approach.
- Advanced Port Scanner 1.3 - free - Radmin. I liked this tool the best. Not only is it small and light, you can specify a set list of IP’s to scan against, as well as specific ports. The IP’s of our servers were loaded as well as the ports in question and I quickly can find out which systems are up and have the targeted services running.
- Microsoft PortQry Command Line Port Scanner Version 2.0 & PortQryUI - free - This is a command-line tool to query ports and get output. This is a great tool for a wide-number of network administration tasks, but really shines in that it is at the core a single exe file and can be used in batch files. The GUI tool version makes things a bit more visual if you need the functionality but without the CLI. Some additional resource information on the PortQry tool are provided below:
- New features and functionality in PortQry version 2.0 - Microsoft Support
- How to Make Portqry.exe Only Report Listening Ports - Microsoft Support
- How To: Mastering PortQry.exe (Part 1) - WindowsSecurity.com
- How To: Mastering PortQry.exe (Part 2) - WindowsSecurity.com
- Thread Batch file for checking port status of multiple IP Address - BoardReader forum thread.
- Simple Admin Tricks: Quick & Dirty Monitoring - Vircom Blog
- Angry IP Scanner - free - It had been a long time since I checked in on this IP/Port scanning tool. It has been in my utility toolbox for a long time, but the newer version supports running on Linux, Mac OSX and Windows. The Windows version in both a 32 and 64-bit version. It does require Java.
- SoftPerfect Network Scanner - free - This is my go-to favorite network scanner. It always provides very fast scan results and you can also specify port scanning feedback.
- Blue's Port Scanner - free - Nice and simple port scanner in both GUI and CLI versions.
- YAPS - free - another nice CLI/GUI support port scanner.
- PortScanner - free - a simple port scanner by Richard Moss
- SLCheck 2.0 - free - This is an alternative CLI port checking tool somewhat similar to PortQry but with a lot more bells and whistles.
- Nmap - Free Security Scanner - free - this tool is like serious overkill for our simple task but if you need to do extensive port/network monitoring, it might be the trick.
- Service Availability Tool - free - Fluke Networks. This is a pretty cool network monitoring application tool although it does have it's limits. Set up IP/Servers you are monitoring, define services on those IP’s by port numbers or services names and let it run. Worth checking out if you need a more GUI/reporting-based tool. Download available from link with registration or from CNET Download.com
If you have any additional utilities or tricks for remotely monitoring server/service availability please drop a tip into the comment jar!
Cheers,
--Claus V.
2 comments:
What you need is to setup a Nagios (Open Source monitoring solution) to monitor the state of your applications, not just the state of the servers they run on. We run Nagios on a spare Linux VM at work and use it to monitor all kinds of infrastructure services (e.g., LDAP, OCSP, DNS, SSH, HTTP, etc, etc...). It's very flexible in the types of alerts it can generate in response to an event as well: email, pager, SMS, etc...
http://www.nagios.org/
@ Anonymous - That would be a simple and nice solution. Unfortunately, as I understand it deployment of that solution would require installation of an "active" client component on the target box.
As I said, we cannot do that in this model, so we are restricted to ONLY passive remote monitoring of the system. I know it isn't logical but that decision is above our heads. :p
So though far from perfect, this is a somewhat better that simply relying on a ping routine to see if the box is up or not.
Thanks for the solution you posted and it might be a perfect option for others who have more flexibility.
Cheers!
--Claus V.
Post a Comment