It has been a long time since my Case of the Unexplained Donut of Death post where I dip into the deeper waters of Windows Performance troubleshooting.
I might use…
- Windows Task manager - taskmgr
- Process Explorer - Sysinternals
- Process Monitor - Sysinternals
- MyEventViewer - NirSoft
- Performance Monitor - Perfmon.exe
- Performance Monitor (Standalone mode) - Perfmon /sys
- Resource Monitor - perfmon /res or resmon.exe
- Windows Reliability Monitor - perfmon /rel
- Xperf - Microsoft WPT
- Command-Line Options for Faster Access to Performance Data - TechNet Magazine
- How to use the Resource Monitor in Windows 7 & Windows 8 - Digital Citizen
- How to Work with the Performance Monitor in Windows 7 & Windows 8 - Digital Citizen
- The Basics About Working with the Event Viewer in Windows - Digital Citizen
But eventually I hit upon the Windows Performance Analysis Toolkit in Windows 8 (SDK 8) and newer (Win 8.1 & 10). See this GSD post for linkages to more tool tips for the above items.
Later I would find an additional tool PerfView that helps with trace file collection and analysis.
- Improving Your App's Performance with PerfView - .NET Blog
- PerfView - Microsoft Download Center
- PerfView Tutorial - Channel 9
Now Windows 10 is out and while I really need to invest in a deep dive of a resource like the Windows Performance Analysis Field Guide (amazon link) by Clint Huffman, there are still a lot of tools and resources still to be discovered.
I’m still trying to understand the new feature set and capability of Windows Performance Toolkit for Windows 10.
- Windows Performance Toolkit - Windows 10 hardware dev
- Windows Assessment and Deployment Kit - Microsoft direct download link
- What's New in the Windows Performance Toolkit - Windows 10 hardware dev
- WPR Quick Start - Windows 10 hardware dev
- WPT Getting Started - Windows 10 hardware dev
- Windows Performance Toolkit Technical Reference - Windows 10 hardware dev
- Windows Performance Analyzer - Windows 10 hardware dev
- Windows Performance Recorder - Windows 10 hardware dev
The latest tool that has inspired me is Bruce Dawson’s “UIforETW” tool.
- UIforETW – Windows Performance Made Easier - Random ASCII
This is a tool that records ETW traces, works around ETW performance bugs, allows configuration of trace recording options, works as a trace management UI, and more.
Basically you download it, run it, allow it to install the required WPT packages, and then pretty much start tracing. It records to a circular buffer so if you experience something interesting, save the trace and rename/note it, and the trace keeps on going while you go exploring the captured trace file. Read Bruce’s post for a full walk-through.
- ETW Central - Random ASCII - round-up post collecting all of the linkage and key bits for ETW usage.
- Random ASCII | Forecast for randomascii: programming, tech topics, with a chance of unicycling - Bruce’s blog home page.
- UIforETW - GitHub - download page to get the latest releases.
Bruce also has some good posts on performance troubleshooting.
- Xperf Basics: Recording a Trace (the ultimate easy way) - Random ASCII
- New Xperf and new WPA in the new WPT - Random ASCII
- Windows Slowdown, Investigated and Identified - Random ASCII
- Windows Slowdown, Investigated, Identified, and Now Fixed - Random ASCII
This is an awesomely helpful tool to keep on your USB drive!
I had also just discovered Clint Huffman’s PAL (Performance Analysis of Logs) Tool
- Performance Troubleshooting using the PAL tool - Mike Lagase
- Performance Analysis of Logs (PAL) Tool - Home
- Get a Handle on Windows Performance Analysis | Systems Management content from Windows IT Pro
Both the ETW tool and the PerfView tool in particular should make it easy for me to leave a shortcut on a user’s desk so they can run the tool if I need them to capture a recurring performance issue, then let me collect the (large) trace file at a later time for deeper analysis.
However it would be nice to make a scripted “click-to-run” or “circular capture at startup” routine for ongoing troubleshooting; assuming resources (disk space and system performance) are up to the task.
I’m looking at these posts for inspiration and refinement.
- How to create a “black box” performance counter data collector - Clint Huffman's Windows Troubleshooting in the Field Blog
- Leveraging Windows Native Functionality to Capture Network Traces Remotely - Windows Wide Open - Victor Zapata
Of course, just being able to collect ETL trace files and look at them with the Windows Performance Analyzer tool isn’t remotely close to fixing the issue. Generally there aren’t any singing angels present pointing the way to the problem process or issue.
You have to deep dive into the trace file to isolate the issue and drill down into the root cause; then come up with a solution.
So here are a BUNCH of video resources (and a few blog posts) to help with the learning process and to illustrate the tools mentioned and techniques used in Windows performance troubleshooting. I’m collecting them here so I can find them quickly when I need some inspiration or deeper learning.
- PerfGuide: User Mode Versus Privileged Mode Processor Usage - TechNet Articles
- Check problematic events with the Windows Reliability Monitor - gHacks Tech News
Defrag Tools - Windows Performance Toolkit (WPT) Videos
- Defrag Tools: #39 - Windows Performance Toolkit | Defrag Tools | Channel 9
- Defrag Tools: #40 - WPT - WPR & WPA
- Defrag Tools: #41 - WPT - Command Line
- Defrag Tools: #42 - WPT - CPU Analysis
- Defrag Tools: #43 - WPT - Wait Analysis
- Defrag Tools: #44 - WPT - DiskIO Analysis
- Defrag Tools: #45 - WPT - File & Registry Analysis
- Defrag Tools: #46 - WPT - Driver Analysis
- Defrag Tools: #47 - WPT - MiniFilter Analysis
- Defrag Tools: #48 - WPT - Memory Analysis - Pool
- Defrag Tools: #49 - WPT - Memory Analysis - VirtualAlloc
- Defrag Tools: #50 - WPT - Memory Analysis - Heap
- Defrag Tools: #77 - WPT Example - CPU | Defrag Tools | Channel 9
- Defrag Tools: #78 - WPT Example - Disk
Defrag Tools - PerfView Videos
- Defrag Tools #113 - PerfView Part 1
- Defrag Tools #114 - PerfView Part 2
- Defrag Tools #115 - PerfView Part 3
- Defrag Tools #116 - PerfView Part 4
- Defrag Tools #117 - PerfView Part 5
- Defrag Tools #118 - PerfView Part 6
- Defrag Tools #119 - PerfView Part 7
- Defrag Tools #125 - PerfView - Part 8
Defrag Tools - General Troubleshooting Videos
- Defrag Tools: #12 - TaskMgr and ResMon
- Defrag Tools: #51 - Support Diagnostics
- Defrag Tools: #53 - Crashes, Hangs and Slow Performance
- Defrag Tools: #56 - Explorer Hang
- The Defrag Show - Channel 9
- Defrag Tools - Channel 9
Advanced, Deep, and Challenging Performance Troubleshooting Videos
- Maximizing Windows 7 Performance: Troubleshooting Tips - TechEd North America 2012 | Channel 9
- Windows Performance Deep Dive Troubleshooting - TechEd North America 2014 | Channel 9
- BlackBelt Troubleshooting Windows Performance Issues - Microsoft Ignite 2015 | Channel 9
- Using Perfmon to Diagnose Application Issues - 02 - .NET Debugging for the Production Environment | Channel 9
- 2014 Edition: How Many Coffees Can You Drink While Your PC Starts? - TechEd North America 2014 | Channel 9
- Windows 8.1: Black Belt Troubleshooting - TechEd North America 2014 | Channel 9
- Windows 8 - Black Belt Troubleshooting - TechEd Australia 2013 | Channel 9
- TWC: Malware Hunting with Mark Russinovich and the Sysinternals Tools - TechEd North America 2014 | Channel 9
- Case of the Unexplained: Troubleshooting with Mark Russinovich - TechEd North America 2014 | Channel 9
- TWC: Sysinternals Primer: TechEd 2014 Edition - TechEd North America 2014 | Channel 9
- Improving performance with the Windows Performance Toolkit - YouTube - Michael Milirud //build presentation.
- Troubleshooting Windows 8 - Windows Virtual Academy
That should keep us busy for a while!