Sunday, July 20, 2014

For Lavie…Good Luck!

To build on the recent reference post I left for Lavie:

Lavie Struggles with Dreamweaver CS 5.5 - grandstreamdreams blog

Here are some more references you might want to bookmark…

I had actually shared my old AP Style guide and Chicago Manual of Style hard-copies with Lavie. I still had them from when I was a journalism major at University for a while! However styles do seem to change a bit!

And for those side-job publishing tasks…when sometimes MS isn’t always the best tool provider to rely on…

Best of luck!

--Claus V.

Rough IT notes for those who are left to clean up…

This is not a rant or a complaint.  Just some observations based on several frustrating weeks recently.

A few weeks ago the staff member responsible who maintained the sysadmin functions around the church house alongside of his primary duties left for a new ministry position elsewhere.

Things like that do occur and best wishes were shared all around.

The week after the staff member left, through a cascading coincidence of events, the church network lost all network connectivity, the church website domain was taken off line, and the in-house Exchange mail-service/server appears to have gone belly up.

We actually do (it appears) have a computer and network services committee. Who knew?! And I’ve been working with the staff and committee leadership to try to make some recommendations and help get a handle on things. It seems that’s what I do.

So in the space of a few minutes…I shot what can best be described as a stream-of-consciousness email with IT/Sysadmin recommendations to that leadership last week as they were preparing to meet with a potential IT consultant to help get a handle on things.

Note the order was just how things came out as I composed and based on site specific issues and the (non-technical) audience. It definitely wouldn’t apply this same wall to all organizations. Likewise, you will need to prioritize the items based on your particular situation.

Here is the main body of that email.

If nothing else, it might help with putting together a technology "roadmap" for the church operations to prioritize and plan out how to cover these items at some point.

In my line of work...if it isn't documented...then it doesn't exist...and cannot be effectively, efficiently, and securely supported. And in IT operations...unknowns are terrifying because the potential impact of an event cannot be measured or mitigated.

  1. Start with a network mapping survey.
    1. come up with a mapping of all your physical connectivity runs and points.
    2. start with the incoming cable, what it connects to, 
    3. the switches and what they connect to and split off, and
    4. where each of the switch ports is connected (physical room jacks/etc.).
  2. Continue with a connected device survey. Move on to document all the, 
    1. network printers, 
    2. PC's, laptops, smart-devices,
    3. WiFi access points, 
    4. the server(s),
    5. physical backup storage units, 
    6. any network appliances (perimeter firewalls, etc.),
    7. routers,
    8. switches, etc.
    9. Note: the documentation for all items would include:
      1. their make/model/serial # information,
      2. warranty information, 
      3. OS versions running, etc.
  3. Move on to audit/document the account/user levels
    1. what accounts are active on the domain and which ones are not. 
    2. which accounts have what permissions? Who are administrators and who are standard users? 
    3. any accounts need to be disabled? 
    4. are all accounts appropriate? (do the security/access levels fit the job descriptions/functions?)
  4. Get a handle on the domain environment.
    1. what is the domain structure? 
    2. does the domain organization/structure make sense? 
    3. what group policy rules are in effect? Do they make sense? 
    4. is file/folder access appropriate for the users/guests? 
    5. how is the domain environment actually administrated?
  5. Audit and understand access and administration of the switches and other network appliances.
    1. are the switches "managed" or not? If so, who does & how? If not, why not? 
    2. are unused network connections/ports disabled if not in use? 
    3. what account(s) exist for the switches/router/network perimeter appliances? 
    4. VPN access and accounts? 
    5. Wi-Fi accounts?
      1. for staff only?
      2. member guest accounts?
      3. password rotation/aging?
      4. what parts of the network could be accessed or seen by a guest on the WiFi network (devices/storage/etc.)?
  6. Backups and storage; what is being backed up & at what frequency?
    1. are backups being done automatically? to where? taken/kept off site? Validation you actually got a good backup!
    2. does the backup storage remain attached to the network at all times? If so what is the risk if a virus/Cryptolocker malware strikes? Could it attack and lock up the backup files as well?
    3. are the backup routines and recovery methods documented so anyone can perform a safe and controlled restoration in a disaster recovery?
  7. Applications
    1. are all software applications maintained in a "library" setting? 
      1. copies of media kept centralized & logged for who has what installed/when? 
      2. license/registration keys physically printed and stored digitally in a central place? 
    2. are operating systems kept patched/updated? How? 
    3. are critical applications kept patched frequently?
    4. does the current critical application licensing model fit the needs and operational requirements of the church staff and workers?
  8. Security
    1. what physical controls are in place to restrict access to critical infrastructure? locks on doors? access logs to rooms? etc. 
    2. are systems all running/current on Anti-virus/Anti-Malware protection software and data files? 
    3. are server/system log files periodically reviewed? 
    4. are periodic scans done of the entire network to look for unauthorized/rouge devices?  (IP scans/port scans/NMAP/random traffic capture and analysis/etc.)
    5. how is PII/HIPPA/financial/etc. information kept restricted, and secured from hackers or unauthorized users? 
    6. is file/whole-disk encryption (Bitlocker) used on the server, laptops, desktops? If not, what would be the loss/risk if a staff member laptop was lost/stolen or if the church office was broken into and desktop(s) stolen? what information would the thief potentially have access to? 
    7. are all Windows desktop/laptop systems inoculated against CryptoLocker type threats? if not, why not?
    8. an assessment must be performed to balance risk of threats to consequences of damage if security compromised....not just about inconvenience, but impact of loss of financial or personal data information of members, staff, etc.  It would look really bad to have data leak (hack/breach) of member information out in public. 
    9. Remote access into the network?
      1. can staff remotely log into the network? workstations? server?
      2. how is that remote access managed/audited?
      3. risk vs. convenience evaluation?
  9. Auditing
    1. who is responsible for auditing laptops/desktops to ensure compliance standards are met? 
    2. how are the audits done? What specific checkpoints must be reviewed? 
    3. is the server(s) audited to ensure group policy or folder rights access have not been changed, compromised, and they meet security expectations? 
    4. software license audits?  Is software installed & licensed appropriately?
  10. Church website; aside from the site itself being kept up to date and accessible...
    1. is it secure?
    2. is is audited to make sure it isn't hosting malware or off-site links to compromised sites?
    3. is it accessible and viewable on mobile devices?
    4. does it contain items (schedules/forms/downloadable materials) that some members of the church might consider private or personal information?
    5. does "meta-data" information exist in downloadable files/forms/photos/etc. that could present security or privacy issues? Are all such items reviewed and "scrubbed" before being posted/uploaded?

Thoughts? Additional recommendations?

Please remember that while this isn’t exactly an “enterprise” operation, if it is large enough to have multiple switches, some servers, WiFi access points, etc. then there should be some sysadmin organization to the operation. And I am well aware that church networks should offer no safe-haven or sanctuary to all the threats in the “secular” IT space.

Cheers!

--Claus Valca

PS: While I still love Windows Live Writer for a blogging platform, the way it handles only the most basic “outlining” formatting leaves me super-frustrated.

What I did to generate this particular outlined list is to reformat it first in KompoZer Portable then copy and paste the source-code for the content back into WLW.

Saturday, June 21, 2014

Debugging a BSOD

A few posts back I mentioned the ongoing battle with periodic BSOD’s on our Win 7 x64 system at the church house.

So I was finally able to find the time to pull the MEMORY.DMP file and the minidump files for closer and more thoughtful review.

First I loaded up the minidump files in BlueScreenView from NirSoft.

3v4ubhbg.vtr

Turns out there were a whole lot more “MEMORY_MANAGEMENT” crashes than I realized!

Having watched enough recent Channel 9 and TechEd presentations lately…more than a few with BSOD/WinDbg troubleshooting, my confidence was up enough to toss the MEMORY.DMP file at Windbg to let it analyze the output to see if that gave any clues.

So I had to get it updated/loaded on my home system.  That took a bit of work in itself.

I went to download the latest version with WDK 8.1 - Windows 8.1: Download kits and tools

However every single time I tried to install it, it failed.

After about a half-hour I gave up and hit the Google.

And found this: Why does the SDK 7.1 installation fail with an "Installation Failed" message on my Windows system? - MATLAB Answers - MATLAB Central

I was using SDK 8.1 but the result was the same…as was the solution: from a comment in that post by the MathWorks Support Team:

This is an issue with Microsoft Windows SDK 7.1. It may occur under two scenarios:

1. If you have Microsoft Visual C++ 2010 SP1 (Express or Professional) installed.

2. If you have Microsoft Visual C++ 2010 redistributable packages (x64 or x86) installed.

The details on the issue from Microsoft are below:

http://support.microsoft.com/kb/2717426

http://support.microsoft.com/kb/2519277

To avoid this issue:

1. Uninstall the Microsoft Visual C++ 2010 redistributable packages (both x86 as well as x64) from “Control Panel” > “Programs and Features”. If you have trouble uninstalling them, see related solution 1-NBI41W at the bottom.

2. Install the Windows SDK 7.1. During installation, under the "Installation Options" menu, UNCHECK the "Visual C++ Compilers" and "Microsoft Visual C++ 2010" components.

3. Apply the SDK 7.1 patch from below:

http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=4422

4. Reinstall the Microsoft Visual C++ 2010 redistributable packages.

x64:

http://www.microsoft.com/en-us/download/details.aspx?id=14632

x86:

http://www.microsoft.com/en-us/download/details.aspx?id=5555

OK. Got it on! Uninstalling the previous Visual C++ packages was the trick.

Next, when trying to run the WinDbg, it kept loading up symbol errors, despite my thinking I had them configured properly.

I vaguely remember covering this ground before…but I was rusty. All the guides said to use this path:

SRV*c:\WINDOWS\symbols*http://msdl.microsoft.com/download/symbols

But it didn’t like it event though it looked perfect.

Eventually, I found a “space” tacked on to the end of the string (user select/copy error I suppose) and got it cleaned up. Then OK.

The default Bugcheck Analysis came back:

Probably caused by : memory_corruption

Followup: memory_corruption

Next I used !analyze -v to get detailed debugging information which netted me this.

MODULE_NAME: memory_corruption

IMAGE_NAME:  memory_corruption

FOLLOWUP_NAME:  memory_corruption

DEBUG_FLR_IMAGE_TIMESTAMP:  0

MEMORY_CORRUPTOR:  LARGE

STACK_COMMAND:  .cxr 0xfffff88005105ee0 ; kb

FAILURE_BUCKET_ID:  X64_MEMORY_CORRUPTION_LARGE

BUCKET_ID:  X64_MEMORY_CORRUPTION_LARGE

Followup: memory_corruption

And pretty much hit the limit of my current mad-crazy debugging skill…but!

I had one other clue still to process.

Although rounds of Memtest86+ and MemTest86 came back clean I did recently note several instances when I booted the system and the BIOS reported the amount of memory in the system shifting between several different sizes.

Because of my DIMM sets, that did give me a clue.  I had two OEM smaller size DIMMs and two larger DIMM sticks. The two newer/larger sticks = the lower RAM and the two OES sticks = the missing RAM.

So I opened up the case after shutting it down, and reseated all the DIMMS.

Rebooted…still lower value.

Shut down again and popped them all out, then reseated them all again, firmly seating them in the slots and making sure they clicked in.

Rebooted…now RAM fully back up.

So far after several weeks, the BSOD’s have stopped.  I suspect (at this time) that at least one of the OEM DIMMS had a flaky seating in the slot and when the system got hot, it broke a contact point, causing the BSOD and memory management error. Time will tell.

Here are some more tools and tips:

Cheers.

--Claus V.

Lavie’s iPhone loss Mystery - Resolved

This past Monday, Lavie called me at work terribly upset after her first visit to our new doctor. (Another rant story for another day.)

Turns out the visit had gone very well and our new doctor meets Lavie’s approval. Yea!

Turns out that somewhere between leaving the practice and getting home, she discovered her iPhone was lost. Noo!

I immediately logged into the Apple “Find my Phone” app on my own iPhone at work, entered her information in, and saw her phone…kind of.

The phone showed up but it couldn’t be located on a map. Turned off perhaps? In a dead zone?

We both had tried calling it to no avail.

I used the option to send a message to the phone and have a finder call us when it was discovered.

Alas, at the end of the day no call and the phone still wasn’t showing up.

Lavie was convinced it was on its way across one of our borders, I wasn’t sure and figured it was at the bottom of the elevator shaft or kicked under one of those heavy examination table/cabinet combos.

The doctor’s office staff said they looked and didn’t see it.

The practice security desk was contacted and didn’t report it being turned in but made a note in their log just in case.

Lavie was still deeply upset with the lost. (She has never lost any mobile phone she has ever owned.)

I was so calm about the loss that contributed to Lavie’s freaking out worse.

So the following day we worked on damage control.

I called our cellular carrier who disabled the SIMM card/# for the phone to prevent any unauthorized phone calls on our account. The rep was very kind and helpful.  No the phone hadn’t been used since her last call that morning to me. No new data usage or activity was showing up.

The phone had both a passlock code set on it as well as the “Find My Phone” feature with iCloud enabled.

I logged back into the iCloud and set it to “auto-wipe”.  I was pretty confident we wouldn’t have any data leakage/breach from it (hence my calmness) but was still curious why it was “dead” in the iCloud.

Luckily, we still had Alvis’s old iPhone 4 as well. After her marriage, her husband had bought her a new iPhone 5s on their own account. Her old phone was on our account, and a few months away from the 2-year contract end, so I elected to keep in on rather than paying an ETF to remove it.

So Lavie just carried Alvis’s phone for the day until I got back off work and we could drop by the local AT&T storefront.

The AT&T rep was very helpful. He cut a new SIMM card (for free!) with Lavie’s cell # on it, then swapped out the SIMM from Alvis’s phone with the new one, also releasing the hold on Lavie’s cell #. Almost good to go. I hung on to Alvis’s SIMM card as it was still good.

Back home, I backed up Alvis’s phone in iTunes, copied off all the photos from it (she said he already had them but I wanted to be sure), then wiped the phone.

I then restored the last backup we had in iTunes for Lavie’s old phone to this one. It was from late December 2013 but it had most everything.

I did have to spend some time re-adding a few apps but not that big a deal. Two hours later Alvis’s old iPhone was now fully migrated to being Lavie’s phone.

Lastly, I checked Lavie’s iCloud account again, and now there were two “Lavie’s iPhone” objects listed. The new one I just finished setting up (with GPS locator hovering over our residence on the map active) and the old one…still not located and “dead” with wipe pending.

So…we were out one iPhone 4..with one to two months left on our contract…and that pretty much it.

Only guess what?

Thursday night the security desk at the practice called.

Lavie’s phone had been found…where they couldn’t say…but she was welcome to come pick it up at our convenience.

So Friday Lavie picked up her phone.

It was almost drained but very much still powered on.  It did say “No Service” as the SIMM had been disabled by our carrier but it connected to our Wi-Fi with nary an issue like a grinning tomcat dragging in after a long night of adventure.

And the phone didn’t wipe.

Curious.

So today I figured out why the phone didn’t show up in iCloud, nor wipe itself as told.

First, Lavie recovered a few missing phone numbers out of her contacts that had been added since the original backup.

Then I got digging.

Going in the Settings and iCloud area, I could clearly see “Find my Phone” was switched on with a nice green indicator showing. What up iPhone?!!

Only there was a hazy semi-opaque haze to the page.

Lavie’s information was all present, but it appears she (we I) didn’t actually log back into iCloud on it after the last iOS 7 upgrade.

Once I did that, Bammo!  The phone wiped.

So, lessons learned from the experience:

  • Make sure your iPhone/iPad is pass-coded. A longer passcode option can be selected over the standard four digit one.
  • Set up Find my iPhone/iPad on your device. Correctly. iCloud: Set up Find My iPhone
  • Test iCloud - Find My iPhone, iPad to make sure it really is seeing and tracking your device!
  • If you carry a lot of passwords on your iPad/iPhone, be sure to keep them in a password manager app, not in Notes. MiniKeePass.
  • Back up your iPad or iPhone device in iTunes (or via iCloud if that is  your thing) regularly. Like every week or so to capture Contacts changes and stuff.
  • If you do loose your device, set the call-back message if found in iCloud.
  • Call your mobile carrier and suspend your number just to be safe you don’t end up with any unauthorized calls.
  • If in deep doubt you will find it again, set it to wipe.

More handy linkages:

Cheers.

--Claus V.

Windows Live Writer - Movement towards Open Source?

I almost missed this tidbit from overseas in my RSS feeds:

Wird der Windows Live Writer Open Source? - Caschy’s Blog (German)

Here is the Google Translate page: Google Translate

In in, Scott Hanselman was observed tweeting involvement in talks to possibly open-source the code for Windows Live Writer.

I love WLW, and it does still get incremental updates from Microsoft from time to time if you can catch them, but there are a lot of little nuisances that need cleaning up (IMHO) and it is exciting to think what some crafty developers could do with the code.

Fingers crossed!

--Claus V.

Dad’s iPad Air Mystery - Resolved

Dad had purchased a very nice 4th generation iPad Retina for his wife some time ago.

She loved it and would use it primarily to catch up on emails, let the little nieces play games on it, maybe watch a saved movie or TV show while traveling. It was the Wi-Fi only model.

Sadly, while bustling about the kitchen, Dad knocked it off the island and it fell onto their tile floor.

It landed on a corner, denting the aluminum case inward by several millimeters, and cracking the glass in the black masking area in that corner.

Other than the cosmetic damage and cracked glass corner, it worked fine and the touch sensor and Retina display were no worse for wear.  Some well placed tape protected fingers and kept the glass shards from falling out.

Dad felt bad although his wife took it in stride and after some teasing, just continued to use it.

Fast forward to about a month ago when Dad finally decided to replace it for his wife’s birthday.

After some consultation, he upgraded to what she originally “really” wanted. This time it was an iPad Air with Wi-Fi and a cellular data plan. Now she wasn’t tied to using it just at home and the school’s Wi-Fi but could fully use it on the road to check her mail, etc. (Yes, I suppose we could have tethered the original one to her iPhone to piggy-back on that cellular data but she really wanted an “all-in-one” device connection.

Dad is a good sport.

We backed up her original device and then restored most of her items to the new one without fuss. She was thrilled.

The original cracked Wi-Fi-only iPad found a second home with me (sweet) after a device wipe. For now I picked out the remaining shards of glass from the dinged corner, then put a layer of super-clear packing tape over that damaged corner, colored it with a black Sharpie, then put another layer of super-clear packing tape over it again. For the untrained eye it isn’t noticeable at all. Maybe I will try to replace the glass myself later…or put a thin layer of putty in the “floor” of the now-glass-free area, then fill it in with a layer of clear epoxy for a more permanent fix. Minor details.

So now the stage has been set…

Dad called me yesterday at work. He had got his bill from AT&T and the data usage charge for the billing period showing on the device was just over 650 MB.  That’s pretty small potatoes to me. I carry a 3 GB data plan on each of our iPhones.

Bad for Dad however, as he is a bit more frugal and has just a 250 MB data plan on the iPad.

That resulted in an unexpected 400 MB of data usage; with overage charges applied accordingly.

So we discussed why it could have been suddenly so high.

First thing that came to mind was that the Wi-Fi connection wasn’t set up correctly and the device was using cellular data rather than their home Wi-Fi connection. I walked him though the Settings, but we were able to confirm it was active on their Wi-Fi network just fine.

“Running” apps can leak data, and I always close out any active apps that I am not using on my iPhone and iPad just out of habit…and to save a few bits of battery as well.

Dad apparently wasn’t aware of this so when I showed him how to check (iOS 7) double-tap the home button then swipe up to close them, he was amazed at all the apps that were loaded and running.

Looking at the apps seen, there were a few that “might” be data-leak culprits (email attachments and/or push notifications), but not really any that I would expect to pull down 400 MB + of data in a month.  That seemed to me more like an iOS software upgrade package.

After talking through it more, Dad decided he would follow up with the AT&T store reps to see if they could drill into his bill with him. He doesn’t use the AT&T online site or an AT&T iOS app. If he did, he could have gone into the data usage status for that device which will show how much cellular data is used on a daily basis for each device. If you haven’t looked, it is pretty cool and helpful to understand your usage patterns. On-line account management via the web or apps still seem a bit dodgy for him…so we just roll with it old-school for now.

Dad called me back that night with his findings.

The AT&T store rep took a look at his account and then the device.  When they went into the Safari app, they found it was closed out, but when re-launched, about a bazillion page “tabs” open. Dad and his wife didn’t seem to realize with the Safari for iOS gui you may not catch that you have a bazillion tabs open. (With the Chrome browser for iOS app it’s a bit more apparent.)  More than a few were for their church’s web-page, which -- wait for it -- hosts streaming and playable video content.  Drilling into the iPad’s detail setting page for app data usage confirmed it was the Safari web browser that was the offending eater of almost all of that 650 MB data. I had forgotten about that detail info view while talking to him earlier.  More. Apparently even though Safari wasn’t “running” it was still periodically communicating and pulling down media files…which would account for the excessive MB usage rates seen.

So Dad and his wife got another lesson about closing out tab/pages in Safari as well.

All is well and Dad was a good sport to chalk it up to a learning experience.

So…mystery cellular data usage solved…mind those Safari page tabs and monitor your data/app data usage periodically for good measure to look for developing trends.

Cheers!

--Claus V.

Wednesday, June 04, 2014

TechEd in Houston Texas; and other troubleshooting bits

Microsoft TechEd North America 2014 rolled though Houston, Texas last month.

I didn’t have the opportunity to attend, but thankfully, Microsoft’s Channel 9 had the event well-covered.

Almost every presentation or session has an online video and/or slide-deck material for your review.

I’ve picked out a handful of ones that I found particularly interesting considering my IT focuses and am listing them here for future reference and playback.

Enjoy!

  • TechEd North America 2014 - Channel 9 main-page coverage of the Houston Texas event.
  • TechEd North America 2014 - Listing of all available presentations and sessions - Channel 9 - five very-long web-pages of items to pick through!
  • Defrag Tools: Live - TechEd 2014 - Mark Russinovich (~25 min) - Defrag Tools | Channel 9 - Mark spend some time highlighting updates to a selection of the Sysinternals tools.
  • TWC: Sysinternals Primer: TechEd 2014 Edition (~1 hr) - Channel 9 - Aaron Margosis presents tutorials on advanced usage of some of the core Sysinternals tools.
  • Case of the Unexplained: Troubleshooting with Mark Russinovich (~1 hr 20 min) - Channel 9 - Mark does his standard outstanding presentation on how to deep-dive into troubleshooting unusual Windows issues.
  • TWC: Bulletproofing Your Network Security (~1 hr 20 min) - Channel 9 - “This session demonstrates the best tools and techniques to harden your devices—from your laptop, to your cell phone, to your servers and your services.”
  • The State of Windows 8.1 Security: Malware Resistance (~1 hr 15 min) - Channel 9 - “Windows 8.1 offers an enormous leap forward when it comes to security, and when it comes to malware resistance that couldn’t be more true. … In this session we drill into the details of the malware threats that you’re facing and then show you how you can help your organization and users enjoy a malware free experience on Windows.”
  • Windows 8.1: Black Belt Troubleshooting (~1 hr 15 min) - Channel 9 - New tips and tricks in troubleshooting Windows 8.1
  • Windows 8 Security Internals (~1 hr 15 min) - Channel 9 - “Windows 8 extends the security and isolation capabilities of Windows to help build a far more trusted application experience. In this session, get a review of the Windows features that have been evolving since Windows Vista, and culminating in a whole new level of application isolation in Windows 8 Applications.”
  • TWC: Social Engineering: Manipulations, Targeted Attacks, and IT Security (~1 hr 15 min) - Channel 9 - “…explore how social engineering has grown over time and examine lessons learned from the field on how to best mitigate those traps.”
  • TWC: Pass-the-Hash: How Attackers Spread and How to Stop Them (~1 hr 15 min) - Channel 9 - “…deconstruct the PtH threat, show how the attack is performed, and how it can be addressed using new features and functionality recently introduced in Windows.”
  • JitJea: A Windows PowerShell Toolkit to Secure a Post-Snowden World (~1 hr 15 min) - Channel 9 - “It is a Windows PowerShell toolkit that you can use to “man up and defend yourselves” by allowing admins to perform functions without giving them admin privileges.”
  • Windows Performance Deep Dive Troubleshooting (~1 hr 10 min) - Channel 9 - “Join us for a deep dive on the free Windows Performance Toolkit (WPT), Windows Assessment Services (WAS) part of the Assessment and Deployment Toolkit (ADK), developed to help you troubleshoot and resolve these issues. Download the toolkit, and get ready to tackle performance issues that can impact organizations of all sizes running Windows Vista, Windows 7, and Windows 8.”

Not appearing at TechEd Houston, but very good presentations in line with the above topics.

--Claus Valca