Tài liệu Memory Dump Analysis Anthology- P20 pdf

Disassembler 571
The second GDB command is x/[N]i address where N is the number of instruc-
tions to disassemble:
(gdb) x/i 0x4012f0
0x4012f0 <main>: push ebp
(gdb) x/2i 0x4012f0
0x4012f0 <main>: push ebp
0x4012f1 <main+1>: mov ebp,esp
(gdb) x/3i 0x4012f0
0x4012f0 <main>: push ebp
0x4012f1 <main+1>: mov ebp,esp
0x4012f3 <main+3>: sub esp,0x8
(gdb) x/4i $pc
0x4012f6 <main+6>: and esp,0xfffffff0
0x4012f9 <main+9>: mov eax,0x0
0x4012fe <main+14>: add eax,0xf
0x401301 <main+17>: add eax,0xf
(gdb)
It seems to be no way to disassemble just N instructions in WinDbg. However in
WinDbg we can disassemble backwards (ub). This is useful, for example, if we have a
return address and we want to see the CALL instruction:
0:000> k
ChildEBP RetAddr
0012ff7c 0040117a test!main [test.cpp @ 3]
0012ffc0 7d4e992a test!__tmainCRTStartup+0×10f
[f:\sp\vctools\crt_bld\self_x86\crt\src\crtexe.c @ 597]
0012fff0 00000000 kernel32!BaseProcessStart+0×28
0:000> ub 7d4e992a
kernel32!BaseProcessStart+0×10:
7d4e9912 call kernel32!BasepReport32bitAppLaunching (7d4e9949)
7d4e9917 push 4

7d4e9919 lea eax,[ebp+8]
7d4e991c push eax
7d4e991d push 9
7d4e991f push 0FFFFFFFEh
7d4e9921 call dword ptr [kernel32!_imp__NtSetInformationThread
(7d4d032c)]
7d4e9927 call dword ptr [ebp+8]

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
572 PART 7: WinDbg For GDB Users and Vice Versa
Our next version of the map contains these new commands:
Action | GDB | WinDbg

Start the process | run | g
Exit | (q)uit | q
Disassemble (forward) | (disas)semble | uf, u
Disassemble N instructions | x/i | -
Disassemble (backward) | - | ub


Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Stack Trace (Backtrace) 573
STACK TRACE (BACKTRACE)
Displaying thread stack trace is the most used action in crash or core dump analy-
sis and debugging. To show various available GDB commands I created the next version
of the test program with the following source code:
#include <stdio.h>
void func_1(int param_1, char param_2, int *param_3, char *param_4);
void func_2(int param_1, char param_2, int *param_3, char *param_4);
void func_3(int param_1, char param_2, int *param_3, char *param_4);

void func_4();
int val_1;
char val_2;
int *pval_1 = &val_1;
char *pval_2 = &val_2;
int main()
{
val_1 = 1;
val_2 = '1';
func_1(val_1, val_2, (int *)pval_1, (char *)pval_2);
return 0;
}
void func_1(int param_1, char param_2, int *param_3, char *param_4)
{
val_1 = 2;
val_2 = '2';
func_2(param_1, param_2, param_3, param_4);
}
void func_2(int param_1, char param_2, int *param_3, char *param_4)
{
val_1 = 3;
val_2 = '3';
func_3(param_1, param_2, param_3, param_4);
}
void func_3(int param_1, char param_2, int *param_3, char *param_4)
{
*pval_1 += param_1;
*pval_2 += param_2;
func_4();
}

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
574 PART 7: WinDbg For GDB Users and Vice Versa
void func_4()
{
puts("Hello World!");
}
We need to compile it with -g gcc compiler option to generate symbolic informa-
tion. It is needed for GDB to display function arguments and local variables.
C:\MinGW\examples> \bin\gcc -g -o test.exe test.c
If you have a crash in func_4 then we can examine stack trace (backtrace) once
we open a core dump. Because we don’t have a core dump of our test program we will
simulate the stack trace by putting a breakpoint on func_4. In GDB this can be done by
break command:
C:\MinGW\examples> \bin\gdb test.exe



(gdb) break func_4
Breakpoint 1 at 0x40141d
(gdb) run
Starting program: C:\MinGW\examples/test.exeBreakpoint 1, 0x0040141d in
func_4 ()
(gdb)
In WinDbg the breakpoint command is bp:
CommandLine: C:\dmitri\test\release\test.exe
Symbol search path is:
SRV*c:\websymbols*
Executable search path is:
ModLoad: 00400000 0040f000 test.exe

ModLoad: 7d4c0000 7d5f0000 NOT_AN_IMAGE
ModLoad: 7d600000 7d6f0000 C:\W2K3\SysWOW64\ntdll32.dll
ModLoad: 7d4c0000 7d5f0000 C:\W2K3\syswow64\kernel32.dll
(103c.17d8): Break instruction exception - code 80000003 (first chance)
eax=7d600000 ebx=7efde000 ecx=00000005 edx=00000020 esi=7d6a01f4
edi=00221f38
eip=7d61002d esp=0012fb4c ebp=0012fcac iopl=0 nv up ei pl nz na po nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
ntdll32!DbgBreakPoint:
7d61002d cc int 3
0:000> bp func_4
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Stack Trace (Backtrace) 575
0:000> g
ModLoad: 71c20000 71c32000 C:\W2K3\SysWOW64\tsappcmp.dll
ModLoad: 77ba0000 77bfa000 C:\W2K3\syswow64\msvcrt.dll
ModLoad: 77f50000 77fec000 C:\W2K3\syswow64\ADVAPI32.dll
ModLoad: 7da20000 7db00000 C:\W2K3\syswow64\RPCRT4.dll
Breakpoint 0 hit
eax=0040c9d0 ebx=7d4d8dc9 ecx=0040c9d0 edx=00000064 esi=00000002
edi=00000ece
eip=00408be0 esp=0012ff24 ebp=0012ff28 iopl=0 nv up ei pl nz na po nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
test!func_4:
00408be0 55 push ebp
We have to disable optimization in the project properties otherwise Visual C++
compiler optimizes away all function calls and produces the following short code:
0:000> uf main
00401000 push offset test!`string' (004020f4)
00401005 mov dword ptr [test!val_1 (0040337c)],4

0040100f mov byte ptr [test!val_2 (00403378)],64h
00401016 call dword ptr [test!_imp__puts (004020a0)]
0040101c add esp,4
0040101f xor eax,eax
00401021 ret
Now we are going to concentrate on commands that examine a call
stack. backtrace or bt command shows stack trace. backtrace <N> or bt <N> shows only
the innermost N stack frames. backtrace -<N> or bt -<N> shows only the outermost N
stack frames. backtrace full or bt full additionally shows local variables. There are also
variants backtrace full <N> or bt full <N> and backtrace full -<N> or bt full -<N>:
(gdb) backtrace
#0 func_4 () at test.c:48
#1 0x00401414 in func_3 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:43
#2 0x004013da in func_2 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:35
#3 0x0040139a in func_1 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:27
#4 0x00401355 in main () at test.c:18
(gdb) bt
#0 func_4 () at test.c:48
#1 0x00401414 in func_3 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:43
#2 0x004013da in func_2 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:35
#3 0x0040139a in func_1 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:27
#4 0x00401355 in main () at test.c:18
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
576 PART 7: WinDbg For GDB Users and Vice Versa

(gdb) bt 2
#0 func_4 () at test.c:48
#1 0x00401414 in func_3 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:43
(More stack frames follow )
(gdb) bt -2
#3 0x0040139a in func_1 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:27
#4 0x00401355 in main () at test.c:18
(gdb) bt full
#0 func_4 () at test.c:48
No locals.
#1 0x00401414 in func_3 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:43
param_2 = 49 '1'
#2 0x004013da in func_2 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:35
param_2 = 49 '1'
#3 0x0040139a in func_1 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:27
param_2 = 49 '1'
#4 0x00401355 in main () at test.c:18
No locals.
(gdb) bt full 2
#0 func_4 () at test.c:48
No locals.
#1 0x00401414 in func_3 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:43
param_2 = 49 '1'
(More stack frames follow )

(gdb) bt full -2
#3 0x0040139a in func_1 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:27
param_2 = 49 '1'
#4 0x00401355 in main () at test.c:18
No locals.
(gdb)

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Stack Trace (Backtrace) 577
In WinDbg there is only one k command but it has many parameters, for exam-
ple:
Default stack trace with source code lines:
0:000> k
ChildEBP RetAddr
0012ff20 00408c30 test!func_4 [c:\dmitri\test\test\test.cpp @ 47]
0012ff28 00408c69 test!func_3+0x30 [c:\dmitri\test\test\test.cpp @ 44]
0012ff40 00408c99 test!func_2+0x29 [c:\dmitri\test\test\test.cpp @ 35]
0012ff58 00408cd3 test!func_1+0x29 [c:\dmitri\test\test\test.cpp @ 27]
0012ff70 00401368 test!main+0x33 [c:\dmitri\test\test\test.cpp @ 18]
0012ffc0 7d4e992a test!__tmainCRTStartup+0x15f
[f:\sp\vctools\crt_bld\self_x86\crt\src\crt0.c @ 327]
0012fff0 00000000 kernel32!BaseProcessStart+0x28
Stack trace without source code lines:
0:000> kL
ChildEBP RetAddr
0012ff20 00408c30 test!func_4
0012ff28 00408c69 test!func_3+0x30
0012ff40 00408c99 test!func_2+0x29
0012ff58 00408cd3 test!func_1+0x29

0012ff70 00401368 test!main+0x33
0012ffc0 7d4e992a test!__tmainCRTStartup+0x15f
0012fff0 00000000 kernel32!BaseProcessStart+0x28
Full stack trace without source code lines showing 3 stack arguments for every
stack frame, calling convention and optimization information:
0:000> kvL
ChildEBP RetAddr Args to Child
0012ff20 00408c30 0012ff40 00408c69 00000001 test!func_4 (CONV: cdecl)
0012ff28 00408c69 00000001 00000031 0040c9d4 test!func_3+0x30 (CONV:
cdecl)
0012ff40 00408c99 00000001 00000031 0040c9d4 test!func_2+0x29 (CONV:
cdecl)
0012ff58 00408cd3 00000001 00000031 0040c9d4 test!func_1+0x29 (CONV:
cdecl)
0012ff70 00401368 00000001 004230e0 00423120 test!main+0x33 (CONV: cdecl)
0012ffc0 7d4e992a 00000000 00000000 7efde000 test!__tmainCRTStartup+0x15f
(FPO: [Non-Fpo]) (CONV: cdecl)
0012fff0 00000000 004013bf 00000000 00000000
kernel32!BaseProcessStart+0x28 (FPO: [Non-Fpo])

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
578 PART 7: WinDbg For GDB Users and Vice Versa
Stack trace without source code lines showing all function parameters:
0:000> kPL
ChildEBP RetAddr
0012ff20 00408c30 test!func_4(void)
0012ff28 00408c69 test!func_3(
int param_1 = 1,
char param_2 = 49 '1',
int * param_3 = 0x0040c9d4,

char * param_4 = 0x0040c9d0 "d")+0x30
0012ff40 00408c99 test!func_2(
int param_1 = 1,
char param_2 = 49 '1',
int * param_3 = 0x0040c9d4,
char * param_4 = 0x0040c9d0 "d")+0x29
0012ff58 00408cd3 test!func_1(
int param_1 = 1,
char param_2 = 49 '1',
int * param_3 = 0x0040c9d4,
char * param_4 = 0x0040c9d0 "d")+0x29
0012ff70 00401368 test!main(void)+0x33
0012ffc0 7d4e992a test!__tmainCRTStartup(void)+0x15f
0012fff0 00000000 kernel32!BaseProcessStart+0x28
Stack trace without source code lines showing stack frame numbers:
0:000> knL
# ChildEBP RetAddr
00 0012ff20 00408c30 test!func_4
01 0012ff28 00408c69 test!func_3+0x30
02 0012ff40 00408c99 test!func_2+0x29
03 0012ff58 00408cd3 test!func_1+0x29
04 0012ff70 00401368 test!main+0x33
05 0012ffc0 7d4e992a test!__tmainCRTStartup+0x15f
06 0012fff0 00000000 kernel32!BaseProcessStart+0x28
Stack trace without source code lines showing the distance between stack frames
in bytes:
0:000> knfL
# Memory ChildEBP RetAddr
00 0012ff20 00408c30 test!func_4
01 8 0012ff28 00408c69 test!func_3+0x30

02 18 0012ff40 00408c99 test!func_2+0x29
03 18 0012ff58 00408cd3 test!func_1+0x29
04 18 0012ff70 00401368 test!main+0x33
05 50 0012ffc0 7d4e992a test!__tmainCRTStartup+0x15f
06 30 0012fff0 00000000 kernel32!BaseProcessStart+0x28

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Stack Trace (Backtrace) 579
Stack trace without source code lines showing the innermost 2 frames:
0:000> kL 2
ChildEBP RetAddr
0012ff20 00408c30 test!func_4
0012ff28 00408c69 test!func_3+0x30
If we want to see stack traces from all threads in a process we can use the follow-
ing command:
(gdb) thread apply all bt
Thread 1 (thread 728.0xc0c):
#0 func_4 () at test.c:48
#1 0x00401414 in func_3 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:43
#2 0x004013da in func_2 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:35
#3 0x0040139a in func_1 (param_1=1, param_2=49 '1', param_3=0x404080,
param_4=0x404070 "d") at test.c:27
#4 0x00401355 in main () at test.c:18
(gdb)
In WinDbg it is ~*k. Any parameter shown above can be used, for example:
0:000> ~*kL
. 0 Id: 103c.17d8 Suspend: 1 Teb: 7efdd000 Unfrozen
ChildEBP RetAddr

0012ff20 00408c30 test!func_4
0012ff28 00408c69 test!func_3+0x30
0012ff40 00408c99 test!func_2+0x29
0012ff58 00408cd3 test!func_1+0x29
0012ff70 00401368 test!main+0x33
0012ffc0 7d4e992a test!__tmainCRTStartup+0x15f
0012fff0 00000000 kernel32!BaseProcessStart+0x28

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
580 PART 7: WinDbg For GDB Users and Vice Versa
Therefore, the next version of the map contains these new commands:
Action | GDB | WinDbg

Start the process | run | g
Exit | (q)uit | q
Disassemble (forward) | (disas)semble | uf, u
Disassemble N instructions | x/<N>i | -
Disassemble (backward) | - | ub
Stack trace | backtrace (bt) | k
Full stack trace | bt full | kv
Partial trace (innermost) | bt <N> | k <N>
Partial trace (outermost) | bt -<N> | -
Stack trace for all threads | thread apply all bt | ~*k
Breakpoint | break | bp

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Local Variables 581
LOCAL VARIABLES
Once we get backtrace in GDB or stack trace in WinDbg we are interested in con-
crete stack frames, their arguments and local variables. I slightly modified the program

used in the previous part to include some local variables:
#include <stdio.h>
void func_1(int param_1, char param_2, int *param_3, char *param_4);
void func_2(int param_1, char param_2, int *param_3, char *param_4);
void func_3(int param_1, char param_2, int *param_3, char *param_4);
void func_4();
int g_val_1;
char g_val_2;
int *g_pval_1 = &g_val_1;
char *g_pval_2 = &g_val_2;
int main()
{
int local_0 = 0;
char *hello = "Hello World!";
g_val_1 = 1;
g_val_2 = '1';
func_1(g_val_1, g_val_2, (int *)g_pval_1, (char *)g_pval_2);
return 0;
}
void func_1(int param_1, char param_2, int *param_3, char *param_4)
{
int local_1 = 1;
g_val_1 = 2;
g_val_2 = '2';
param_3 = &local_1;
func_2(g_val_1, g_val_2, param_3, param_4);
}

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
582 PART 7: WinDbg For GDB Users and Vice Versa

void func_2(int param_1, char param_2, int *param_3, char *param_4)
{
int local_2 = 2;
g_val_1 = 3;
g_val_2 = '3';

param_3 = &local_2;
func_3(g_val_1, g_val_2, param_3, param_4);
}
void func_3(int param_1, char param_2, int *param_3, char *param_4)
{
int local_3 = 3;
*g_pval_1 += param_1;
*g_pval_2 += param_2;
func_4();
}
void func_4()
{
puts("Hello World!");
}
In GDB the frame command is used to set the current stack frame. Then info args
command can be used to list function arguments and info locals command can be used
to list local variables:
(gdb) break func_4
Breakpoint 1 at 0x401455: file test.c, line 61.
(gdb) run
Starting program: C:\MinGW\examples/test.exe
Breakpoint 1, func_4 () at test.c:61
61 puts("Hello World!");
(gdb) bt

#0 func_4 () at test.c:61
#1 0x0040144d in func_3 (param_1=3, param_2=51 '3', param_3=0x22ff10,
param_4=0x404070 "f") at test.c:56
#2 0x0040140c in func_2 (param_1=2, param_2=50 '2', param_3=0x22ff10,
param_4=0x404070 "f") at test.c:46

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Local Variables 583
#3 0x004013ba in func_1 (param_1=1, param_2=49 '1', param_3=0x22ff30,
param_4=0x404070 "f") at test.c:34
#4 0x00401363 in main () at test.c:21
(gdb) frame
#0 func_4 () at test.c:61
61 puts("Hello World!");
(gdb) frame 0
#0 func_4 () at test.c:61
61 puts("Hello World!");
(gdb) info args
No arguments.
(gdb) info locals
No locals.
(gdb) frame 1
#1 0x0040144d in func_3 (param_1=3, param_2=51 '3', param_3=0x22ff10,
param_4=0x404070 "f") at test.c:56
56 func_4();
(gdb) info args
param_1 = 3
param_2 = 51 '3'
param_3 = (int *) 0x22ff10
param_4 = 0x404070 "f"

(gdb) info locals
local_3 = 3
param_2 = 51 '3'
(gdb) frame 2
#2 0x0040140c in func_2 (param_1=2, param_2=50 '2', param_3=0x22ff10,
param_4=0x404070 "f") at test.c:46
46 func_3(g_val_1, g_val_2, param_3, param_4);
(gdb) info args
param_1 = 2
param_2 = 50 '2'
param_3 = (int *) 0x22ff10
param_4 = 0x404070 "f"
(gdb) info locals
local_2 = 2
param_2 = 50 '2'

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
584 PART 7: WinDbg For GDB Users and Vice Versa
(gdb) frame 3
#3 0x004013ba in func_1 (param_1=1, param_2=49 '1', param_3=0x22ff30,
param_4=0x404070 "f") at test.c:34
34 func_2(g_val_1, g_val_2, param_3, param_4);
(gdb) info args
param_1 = 1
param_2 = 49 '1'
param_3 = (int *) 0x22ff30
param_4 = 0x404070 "f"
(gdb) info locals
local_1 = 1
param_2 = 49 '1'

(gdb) frame 4
#4 0x00401363 in main () at test.c:21
21 func_1(g_val_1, g_val_2, (int *)g_pval_1, (char *)g_pval_2);
(gdb) info args
No arguments.
(gdb) info locals
local_0 = 0
hello = 0x403000 "Hello World!"
(gdb)
In WinDbg kn command shows stack trace with frame numbers, knL command
additionally omits source code references, .frame command switches to particular stack
frame, dv command shows parameters and local variables together, dv /i command
classifies them into categories, parameters or locals, dv /V command shows their ad-
dresses and offsets for the relevant base frame register, usually EBP, dv /t command
shows type information:
Microsoft (R) Windows Debugger Version 6.7.0005.1
Copyright (c) Microsoft Corporation. All rights reserved.
CommandLine: C:\dmitri\test\release\test.exe
Symbol search path is:
SRV*c:\websymbols*
Executable search path is:
ModLoad: 00400000 0040f000 test.exe
ModLoad: 7d4c0000 7d5f0000 NOT_AN_IMAGE
ModLoad: 7d600000 7d6f0000 C:\W2K3\SysWOW64\ntdll32.dll
ModLoad: 7d4c0000 7d5f0000 C:\W2K3\syswow64\kernel32.dll

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Local Variables 585
(e38.ac0): Break instruction exception - code 80000003 (first chance)
eax=7d600000 ebx=7efde000 ecx=00000005 edx=00000020 esi=7d6a01f4

edi=00221f38
eip=7d61002d esp=0012fb4c ebp=0012fcac iopl=0 nv up ei pl nz na po nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
ntdll32!DbgBreakPoint:
7d61002d cc int 3
0:000> bp func_4
0:000> g
ModLoad: 71c20000 71c32000 C:\W2K3\SysWOW64\tsappcmp.dll
ModLoad: 77ba0000 77bfa000 C:\W2K3\syswow64\msvcrt.dll
ModLoad: 00410000 004ab000 C:\W2K3\syswow64\ADVAPI32.dll
ModLoad: 7da20000 7db00000 C:\W2K3\syswow64\RPCRT4.dll
ModLoad: 7d8d0000 7d920000 C:\W2K3\syswow64\Secur32.dll
Breakpoint 0 hit
eax=0040c9d4 ebx=7d4d8df9 ecx=0040c9d4 edx=00000066 esi=00000002
edi=00000ece
eip=00408be0 esp=0012ff10 ebp=0012ff18 iopl=0 nv up ei pl nz na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206
test!func_4:
00408be0 55 push ebp
0:000> knL
# ChildEBP RetAddr
00 0012ff0c 00408c38 test!func_4
01 0012ff18 00408c7c test!func_3+0x38
02 0012ff34 00408ccc test!func_2+0x3c
03 0012ff50 00408d24 test!func_1+0x3c
04 0012ff70 00401368 test!main+0x44
05 0012ffc0 7d4e7d2a test!__tmainCRTStartup+0x15f
06 0012fff0 00000000 kernel32!BaseProcessStart+0x28
0:000> .frame
00 0012ff0c 00408c38 test!func_4 [c:\dmitri\test\test\test.cpp @ 60]

0:000> .frame 0
00 0012ff0c 00408c38 test!func_4 [c:\dmitri\test\test\test.cpp @ 60]
0:000> dv
0:000> .frame 1
01 0012ff18 00408c7c test!func_3+0x38 [c:\dmitri\test\test\test.cpp @ 57]
0:000> dv
param_1 = 3
param_2 = 51 '3'
param_3 = 0x0012ff30
param_4 = 0x0040c9d4 "f"
local_3 = 3
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
586 PART 7: WinDbg For GDB Users and Vice Versa
0:000> dv /i
prv param param_1 = 3
prv param param_2 = 51 '3'
prv param param_3 = 0x0012ff30
prv param param_4 = 0x0040c9d4 "f"
prv local local_3 = 3
0:000> dv /i /V
prv param 0012ff20 @ebp+0x08 param_1 = 3
prv param 0012ff24 @ebp+0x0c param_2 = 51 '3'
prv param 0012ff28 @ebp+0x10 param_3 = 0x0012ff30
prv param 0012ff2c @ebp+0x14 param_4 = 0x0040c9d4 "f"
prv local 0012ff14 @ebp-0x04 local_3 = 3
0:000> .frame 4
04 0012ff70 00401368 test!main+0x44 [c:\dmitri\test\test\test.cpp @ 21]
0:000> dv
local_0 = 0
hello = 0x0040a274 "Hello World!"

0:000> dv /i
prv local local_0 = 0
prv local hello = 0x0040a274 "Hello World!"
0:000> dv /i /V
prv local 0012ff68 @ebp-0x08 local_0 = 0
prv local 0012ff6c @ebp-0x04 hello = 0x0040a274 "Hello World!"
0:000> dv /t
int local_0 = 0
char * hello = 0x0040a274 "Hello World!"

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Local Variables 587
Our comparison table has grown a bit:

Action | GDB | WinDbg

Start the process | run | g
Exit | (q)uit | q
Disassemble (forward) | (disas)semble | uf, u
Disassemble N instructions | x/<N>i | -
Disassemble (backward) | - | ub
Stack trace | backtrace (bt) | k
Full stack trace | bt full | kv
Stack trace with parameters | bt full | kP
Partial trace (innermost) | bt <N> | k <N>
Partial trace (outermost) | bt -<N> | -
Stack trace for all threads | thread apply all bt | ~*k
Breakpoint | break | bp
Frame numbers | any bt command | kn
Select frame | frame | .frame

Display parameters | info args | dv /t /i /V
Display locals | info locals | dv /t /i /V


Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
588 PART 7: WinDbg For GDB Users and Vice Versa

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Four Pillars 589
PART 8: SOFTWARE TROUBLESHOOTING
FOUR PILLARS
They are (sorted alphabetically):
1. Crash Dump Analysis (also called Memory Dump Analysis or Core Dump
Analysis)
2. Problem Reproduction
3. Trace and Log Analysis
4. Virtual Assistance (also called Remote Assistance)





Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
590 PART 8: Software Troubleshooting
FIVE GOLDEN RULES
It is difficult to analyze a problem when we have crash dumps and/or
traces from various tracing tools and supporting information we have is incomplete or
missing. I came up with this easy to remember 4WS questions to ask when sending or
requesting traces and memory dumps:
What - What had happened or had been observed? Crash or hang, for example?

When - When did the problem happen if traces were recorded for hours?
Where - What server or workstation had been used for tracing or where memory
dumps came from? For example, one trace is from a primary server and two others are
from backup servers or one trace is from a client workstation and the other is from a
server.
Why - Why did a customer or a support engineer request a dump file or a trace?
This could shed the light on various assumptions including presuppositions hidden in
problem description.
Supporting information - needed to find a needle in a hay: process id, thread id,
etc. Also, the answer to the following question is important: how memory dumps and
traces were created?
Every trace or memory dump shall be accompanied by 4WS answers.
4WS rule can be applied to any troubleshooting because even the problem
description itself is some kind of a trace.

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Critical Thinking 591
CRITICAL THINKING
Faulty thinking happens all the time in technical support environments partly due
to hectic and demanding business realities.
There is an interesting website that taxonomically organizes fallacies:

Take, for example, False Cause fallacy. Technical examples might include false
causes inferred from trace analysis, customer problem description that includes steps to
reproduce the problem, and so on. This also applies to debugging and importance of
critical thinking skills has been emphasized in the following book:
Debugging by Thinking: A Multidisciplinary Approach
Surface-level of basic crash dump analysis is less influenced by false cause falla-
cies because it doesn’t have explicitly recorded sequence of events although some cau-
tion should be exercised during detailed analysis of thread waiting times and other

historical information.
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
592 PART 8: Software Troubleshooting
TROUBLESHOOTING AS DDEBUGGING
This post is motivated by TRAFFIC steps introduced by Andreas Zeller in his book
“Why Programs Fail?”. This book is wonderful and it gives practical debugging skills co-
herent and solid systematical foundation.
However these steps are for fixing defects in code, the traditional view of the
software debugging process. Based on an analogy with systems theories where we
have different levels of abstraction like psychology, biology, chemistry and physics, I
would say that debugging starts when we have a failure at the system level.
If we compare systems to applications, troubleshooting to source code debug-
ging, the question we ask at the higher level is “Who caused the product to fail?” which
also has a business and political flavor. Therefore I propose a different acronym:
VERSION. If you always try to fix system problems at the code level you will get a huge
“traffic” in all sense but if you troubleshoot them first you get a different system /
subsystem / component version and get your problem solved faster. This is why we have
technical support departments in organizations.
There are some parallels between TRAFFIC and VERSION steps:
Track View the problem
Reproduce Environment/repro steps
Automate (and simplify) Relevant description
Find origins Subsystem/component
identification
Focus Identify the origin
(subsystem/component)
Isolate (defect in code) Obtain the solution
(replace/eliminate
subsystem/component)
Correct (defect in code) New case study

(document,
postmortem analysis)
Troubleshooting doesn’t eliminate the need to look at source code. In many
cases a support engineer has to be proficient in code reading skill to be able to map
from traces to source code. This will help in component identification, especially if the
product has extensive tracing facility.

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Pooltags 593
PART 9: CITRIX
POOLTAGS
Citrix drivers have their own pooltags. Please refer to the following article:

When we see the following or similar output from !poolused WinDbg command
we can update pooltag.txt file located in Debugging Tools for Windows installation
triage folder:
WD UNKNOWN pooltag 'WD ', please update pooltag.txt
Note: ‘Ica’ pooltag doesn’t belong to Citrix drivers although it sounds like “ICA
protocol”. It comes from Microsoft termdd.sys driver.

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
594 PART 9: Citrix
THE LIST OF CITRIX SERVICES
In kernel or complete memory dumps coming from Windows servers running
Citrix Presentation Server 4.x we might see the following processes running in session 0,
for example:
2: kd> !process 0 0
PROCESS 895c7380 SessionId: 0 Cid: 03f0 Peb: 7ffdf000 ParentCid:
01a8
DirBase: 0a43d220 ObjectTable: 895c7628 HandleCount: 684.

Image: CpSvc.exe
PROCESS 892e3320 SessionId: 0 Cid: 060c Peb: 7ffdf000 ParentCid:
01a8
DirBase: 0a43d440 ObjectTable: 892e76c8 HandleCount: 93.
Image: cdmsvc.exe
PROCESS 892ed4a0 SessionId: 0 Cid: 05f8 Peb: 7ffdf000 ParentCid:
01a8
DirBase: 0a43d420 ObjectTable: 892f1268 HandleCount: 107.
Image: CdfSvc.exe
PROCESS 89297020 SessionId: 0 Cid: 06ac Peb: 7ffdf000 ParentCid:
01a8
DirBase: 0a43d520 ObjectTable: 892991c8 HandleCount: 62.
Image: encsvc.exe
PROCESS 892a4020 SessionId: 0 Cid: 06d4 Peb: 7ffdf000 ParentCid:
01a8
DirBase: 0a43d540 ObjectTable: 892b9a48 HandleCount: 1088.
Image: ImaSrv.exe
PROCESS 892a5020 SessionId: 0 Cid: 070c Peb: 7ffdf000 ParentCid:
01a8
DirBase: 0a43d560 ObjectTable: 8927b568 HandleCount: 188.
Image: mfcom.exe
PROCESS 890e8620 SessionId: 0 Cid: 0cc4 Peb: 7ffdf000 ParentCid:
01a8
DirBase: 0a43d6e0 ObjectTable: 890e8948 HandleCount: 691.
Image: SmaService.exe
PROCESS 8901bd60 SessionId: 0 Cid: 0d80 Peb: 7ffdf000 ParentCid:
01a8
DirBase: 0a43d880 ObjectTable: 89021e88 HandleCount: 148.
Image: XTE.exe
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.

The List of Citrix Services 595
PROCESS 88fce020 SessionId: 0 Cid: 1204 Peb: 7ffdf000 ParentCid:
01a8
DirBase: 0a43d900 ObjectTable: 88fcfac8 HandleCount: 186.
Image: ctxwmisvc.exe
These are Citrix services and the following Citrix article describes them briefly:
Citrix Presentation Server Services Overview


Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.