Tài liệu Memory Dump Analysis Anthology- P2 doc

Hangs Explained 31
HANGS EXPLAINED
Another category of problems happens very often where we also need a dump
for analysis: hangs. There is some confusion exists in understanding the difference be-
tween these two categories: crash and hang. Although sometimes a hang is a direct
consequence of a crash most of the time hangs happen independently. They also manif-
est themselves differently. Let’s look at application (process) crashes and hangs first.
When a crash happens an application (process) often disappears. When a hang hap-
pens an application (process) is still in memory: we can see it in Task Manager, for
example, but it doesn’t respond to user commands or to any other requests like ping-
ing a TCP/IP port. If we have a crash in OS then the most visible manifestation is blue
screen and/or reboot. If we have a hang then everything freezes.
Application or system hang happens because from the high level of view the
interaction between applications and OS components (modules) is done via messages.
One component sends a message to another and waits for a response. Some
components are critical, for example, a registry. The following hand-made
picture depicts very common system hang situations when the register component
stops responding. Then every running application (process) stops responding if its
execution path depends on registry access.
Registry
Internet Explorer
ServiceDriver

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
32 PART 1: Crash Dumps for Beginners
The very common reason for hang is the so called deadlock when two running
applications, their execution paths or threads are waiting for each other. Here is an anal-
ogy with a blocked road:
Car 1 blocked the road,
waiting for service Car 2
car

car
Service Car 2 – waiting for
Car 1 to unblock the road

In order to see what’s inside the process or OS which caused a hang we need a
memory dump. Usually this dump is called a crash dump too because in order to get it
the usual method is to make some sort of a trap which causes an application or OS to
crash and to save the dump. I personally prefer to call these crash dumps just memory
dumps to avoid confusion.
Some FAQ:
Q. How can we get a memory dump if our application or service hangs?
A. It is possible to do using various methods:
by using NTSD command line options (remember that NTSD is always
present on pre-Vista system)
by using userdump.exe
by attaching and saving the dump interactively via NTSD
(
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Hangs Explained 33
by attaching and saving the dump interactively via WinDbg
(
by using ADPlus in a hang mode
(
Q. How can we get a memory dump if our system hangs?
A. Two common methods are:
manually via keyboard (
by using Citrix SystemDump tool remotely or via GUI if some session is still
alive (
For most system hangs choosing Kernel memory dump option in Control
Panel\System\Advanced\Startup and Recovery applet is sufficient. Kernel memory

dumps are smaller and less susceptible to corruption or truncation due to small page file
size. If you discover that you need to peer inside running user applications then you can
always ask for another complete memory dump when the problem happens again.

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
34 PART 1: Crash Dumps for Beginners
SYMBOL FILES EXPLAINED
Symbol files are usually called PDB files because they have .PDB extension al-
though the older ones can have .DBG extension. PDB files are needed to read dump files
properly. Without PDB files the dump file data is just a collection of numbers, the con-
tents of memory, without any meaning. PDB files help tools like WinDbg to interpret the
data and present it in a human-readable format. Roughly speaking, PDB
files contain associations between numbers and their meanings expressed in short text
strings:
Dump data (memory contents)
...
...
...
773f8ea4 0012f9f4 6be82f08
00000000 7e4188da 00000000
0012fa80 7fffffff 000003e8
00406258 00000000 00000001
00da00ab 00aa00f3 00dc0000
...
...
...
comctl32.pdb
...
773f8ea4 Button_WndProc
...

ProductA.pdb
...
0012f9f4 ProcessPayment
...
imgutil.pdb
...
6be82f08 DrawImage
...

Because these associations are changed when we have a fix or a service pack on
a computer and we have a crash dump from it we need newer PDB files that correspond
to updated components such as DLLs or drivers.
Long time ago we had to download symbol files manually from Microsoft or get
them from CDs. Now Microsoft has its dedicated internet symbol server and WinDbg
can download PDB files automatically. However we need to specify Microsoft symbol
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Symbol Files Explained 35
server location in File\Symbol File Path… dialog and check Reload. The location is usually
(check ):
SRV*c:\websymbols*
If we don’t remember the location when we run WinDbg for the first time or on a
new computer we can enter .symfix command to set Microsoft symbol server path
automatically and specify the location where to download symbol files. We can check
our current symbol search path by using .sympath command and then reload symbols
by entering .reload command:
0:000> .symfix
No downstream store given, using C:\Program Files\Debugging Tools for
Windows\sym
0:000> .sympath
Symbol search path is: SRV**

0:000> .symfix c:\websymbols
0:000> .sympath
Symbol search path is:
SRV*c:\websymbols*
0:000> .reload

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
36 PART 1: Crash Dumps for Beginners
CRASHES AND HANGS DIFFERENTIATED
In the articles Crashes Explained (page 28) and Hangs Explained (page 31)
I highlighted the difference between crashes and hangs. In this part I will elaborate on
this terminology a bit further. First of all, we have to unify them as manifestations of a
functional failure. Considering a computer as a system of components having certain
functions we shall subdivide failures into system and component failures. Of course,
systems themselves may be components in some larger hierarchy, like in the case of
virtualization. Application and service process failures fall under component failures
category. Blue screen and server freezes fall under system failures category. Now it is
obvious why most computer users confuse crashes and hangs. They are just failures and
often the distinction between them is blurred from the user perspective.
Software developers tend to make sharp distinction between crashes and
hangs because they consider a situation when a computer accesses wrong memory or
gets and executes an invalid instruction as a crash. However, after such situation a com-
puter system may or may not terminate that application or service.
Therefore, I propose to consider crashes as situations when a system or a compo-
nent is not observed anymore. For example, a running application or service disappears
from Task Manager, computer system shows blue screen or reboots. In hang situations
we can observe that existence of a failed component in Task Manager or a computer
system doesn’t reboot automatically and shows some screen image different from BSOD
or panic message. The so called sluggish behavior or long response time can also be
considered as hang situations.

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Crashes and Hangs Differentiated 37
Here is a simple rough diagram I devised to illuminate the proposed terminologi-
cal difference:
Functional failure
System failure Component failure
Visibility
Crash Hang
T
F

Based on the clarification above the task of collecting memory or crash dumps is
much simpler and clearer.
In the case of a system crash or hang we need to setup correct crash dump op-
tions in Advanced System Settings in Control Panel and check page file size in case of the
complete memory dump option. A system crash will save the dump automatically. For
system hangs we need to actively trigger crash dump saving procedure using either
standard keyboard method, SystemDump tool or live system debugging.
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
38 PART 1: Crash Dumps for Beginners
In the case of an application crash we need to set up a postmortem debugger,
get WER report or attach a debugger to a component and wait for a failure to happen. In
the case of a hang we save a memory dump manually either by using process dumpers
like userdump.exe or attaching a debugger.
Links to some dump collection techniques can be found in previous Crashes Ex-
plained and Hangs Explained articles. Forthcoming Windows® Crash Dump Analysis book
(ISBN-13: 978-0-9558328-2-6) will discuss all memory dump collection methods tho-
roughly and in detail.

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Proactive Crash Dumps 39
PROACTIVE CRASH DUMPS
In Crashes and Hangs Differentiated article (page 36) I introduced clear separa-
tion between crashes and hangs and outlined memory dump capturing methods for
each category. However, looking from user point of view we need to tell them what is
the best way to capture a dump based on observations they have and their failure level,
system or component. The latter failure type usually happens with user applications and
services.
For user applications the best way is to get a memory dump proactively or put in
another words, manually, and do not rely on a postmortem debugger that may not be
set up correctly on a problem server in one hundred server farm. If any error message
box appears with a message that an application stopped working or that it has encoun-
tered an application error then we can use process dumpers like userdump.exe.
Suppose we have the following error message when TestDefaultDebugger
application crashes on Vista x64 (the same technique is applicable to earlier OS too):


Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
40 PART 1: Crash Dumps for Beginners
Then we can dump the process while it displays the problem message if we know
its process ID:

In Vista this can be done even more easily by dumping the process from Task
Manager directly:


Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Proactive Crash Dumps 41
If we choose Create Dump File we see this message box:

The process dump is saved to a user location for temporary files:

Although the application above is the native Windows application the same me-
thod applies for .NET applications. For example, TestDefaultDebugger.NET application
shows the unhandled exception message when we click on Crash Me button and we can
dump the process manually while it displays that message:


Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
42 PART 1: Crash Dumps for Beginners
Although both applications will disappear from Task Manager if we choose Close
or Quit on their error message boxes and therefore will be considered as crashes under
new refined terminology, at the time when they show their stop messages they are
considered as application hangs and this is why we use manual process dumpers.

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.