So yesterday I was troubleshooting some window creation issues, and had to fool around in the kernel side of window creation, down in win32k.sys. Specifically I was looking at window class registration, which happens when you call RegisterClassEx from your Windows app. Down in the kernel, some magic happens with creating Atoms as part of the window class registration. I traced through a bunch of win32k.sys routines to figure out where in memory they were storing this, and then I wanted to dump the table. After dumping about 4 of the entries manually, I got bored and wrote this little gem:

r $t0=poi(poi(win32k!UserAtomTableHandle)+c)
.for ( r $t1=0; @$t1 < @$t0; r $t1 = @$t1 + 1 ) { du poi(poi(win32k!UserAtomTableHandle)+10+( @$t1 * 4))+c }

Basically, this uses the symbol win32k!UserAtomTableHandle to find the length of the table, and then uses a for loop to go through, calculating the offset of each item, and them dumping its string value. On my Windows 7 system it produced something like this:

8c2a3d1c  "Native"
878b0c9c  "ObjectLink"
87e1e18c  "AeroWizardInternalFrameButtonCli"
87e1e1cc  "cked"
878cb314  "Static"
878cb104  "DDEMLUnicodeClient"
9620faec  "DataObject"
8c2affa4  "ACTIVATESHELLWINDOW"
8c2afe34  "FlashWState"
9620fa84  "SysCH"
8c2b2ce4  "PBrush"
8c3b8f24  "MSUIM.Msg.RpcSendReceive"
878bb7b4  "SysIC"
878cb1ec  "DDEMLEvent"
878bb784  "SHELLHOOK"
8c2b2e0c  "Custom Link Source"
9159dc84  "AltTab_KeyHookWnd"
91529084  "Search Box"
878bb6f4  "SysDT"
8c2b2dd4  "Link Source"
9620fb8c  "FileName"
87e35b0c  "GDI+ Accessibility"
878bb664  "SysWNDO"
878bb854  "DDEMLAnsiServer"
87e0c0bc  "SysLink"
9620fb24  "NetworkName"
8c2cde3c  "USER32"
8c2b2d14  "OleDraw"
9620fb5c  "FileNameW"
8c2b2bec  "MoreOlePrivateData"
8c282434  "Edit"
9620fbbc  "Binary"
878cb374  "OleClipboardPersistOnFlush"
8c2a3d4c  "OwnerLink"
878cb2e4  "ListBox"
8c2b2e54  "Embed Source"
878bb634  "SysIMEL"
878cb224  "ComboLBox"

I had to debug an annoying little problem today that I thought might be worth writing about. I was interested in walking through some code that was failing, but the same code was getting called in a recursive loop, so there were literally hundreds of successful runs that I was not interested in prior to the single failure I did care about.

Now a normal usermode developer might just add some special code at the point of failure to detect the failure and recall the failing function. Nice and easy. But that’s really not any fun, and when you’re doing kernel debugging, writing some new code and getting it running on the machine is not quite as simple (it’s not hard, just more time consuming).

Enter this neato debugging trick…

bp address "j (dwo(status)!=0) 'r @rip=fffff880`02b5bd1f'; 'gc'"

Basically this executes a conditional test (the “j” command) each time the breakpoint is hit. If the DWORD value represented by the variable named ‘status’ is non-zero, then I know I’ve hit the failure condition. In that case, I just adjust the instruction pointer back up to before the failing function call, leaving me right where I am ready to trace into the function and see the failure. Otherwise, the breakpoint essentially just hits ‘Go’ to continue on to the next hit.

The syntax here is a bit rough, and would have to be modified if your program isn’t always at the same code location (since I hard-coded the rip register). It could be replaced with an offset from the current location to be a bit more elegant. But since I was working on a driver, it was always in memory and at the same place, so I was lazy. (A habit that always pays off immediately.)

Quite some time back I wrote a blog entry on how to make conditional breakpoints. In particular, I was looking for a way to find when a certain pool tag allocation occurred. Well it turns out there is a MUCH better way of doing it that what I posted in that blog entry.

The latest issue of “The NT Insider” from OSR just came out, and has an article on debugging techniques. Apparently there is a global value that you can set to a given pool tag that you would like to break on allocation for. So if I were looking for the tag ‘Test’ I could set the following value in the debugger:

kd> ed nt!PoolHitTag 'tseT'

Note that this technique will only work if pool tagging is enabled on the system. But nevertheless, this technique is a lot more efficient and faster than the method I showed previously. (Although the technique could still be good for other scenarios.)

Sometimes you have an ASSERT in your code and for some reason it starts being hit all the time. It’s good to know about it the first time, but if it’s happening hundreds of times a second, it can make debugging (or just replacing the code with something that doesn’t ASSERT) very difficult. Enter windbg and the power to kill an ASSERT.

When the debugger breaks in on an ASSERT, look at your disassembly. You will probably see something similar to the following. Yours may look a little different, depending on what you’re ASSERTING on, but it will be similar.

f6bc9923 7414 jz driver!function+0x1d9 (f6bc9939)
f6bc9925 6819050000 push 0x519
f6bc992a 68c096bcf6 push 0xf6bc96c0
f6bc992f e874010300 call driver!DbgPrint (f6bf9aa8)
f6bc9934 83c408 add esp,0x8
f6bc9937 cd01 int 01

The key element here is the first line which is actually doing the test on the ASSERTion. If the ASSERT comparison evaluates to zero (i.e., the ASSERT succeeds), then it jumps over the call to output a debugger string and the int 01 which breaks into the debugger.

So what we want it to do is always skip that section. If we enter windbg into the assembler mode using the “a” command (of course telling it what address to assemble into), and then replace the jz instruction with a jmp instruction, that’s all we need.

kd> a f6bc9923
f6bc9923 jmp driver!function+0x1d9
jmp driver!function+0x1d9
f6bc9925

kd> g

Note that the only affect on the binary code is to change the first byte from a 74 to an EB. You could accomplish the same thing that way instead of using the assembler, which is as simple as:

kd> eb f6bc9923 eb

Today I ran into a need to set a breakpoint that would only stop when a certain string was encountered. In the past I have just modified the code to test for the string, and then update my driver, reboot, etc. A very time consuming process. So today I decided that I wanted to figure out how to do it right in the debugger. I knew it was possible from comments, but didn’t know how to implement it.

First of all, since runs a non-trivial set of commands each time the breakpoint is hit, so I placed the commands in a secondary file. There may be a way to get this all on a single line breakpoint command, but I don’t see it. So the breakpoint we create is just going to run the commands from the secondary file. The command to create the breakpoint is something like this:

bp driver!functionName “$$< C:\debugCommands.txt”

Then comes the important part – the actual commands that get executed. We need to evaluate a string against a pattern, which the masm expression evaluator can handle using the “$spat” command. The hard part about that is that at first glance it only appears to work with string literals. So $spat( “Big string”, “*str*” ) will work, while $spat( poi(variableName), “*str*” ) just laughs mockingly at you.

The key here is to assign the string to an alias which will then allow it to be evaluated by the $spat command. So using our example comparison, the commands in the secondary file look like:

as /mu ${/v:MyAlias} poi(variableName);
.if ( $spat( “${MyAlias}”, “*str*” ) != 0 ) { g }

The commands evaluate the string. If a match is not found, the g[o] command is executed, otherwise execution will stop at the point when the pattern is found. Note that there are much more complicated pattern matching expressions available as well.

I found a very strange bug with a kernel driver I do some work on this last week. It only seems to appear when we have GoogleDesktop or the MSN Desktop Search installed. After some period of time the graphics display just starts doing some whacky stuff: fonts don’t display, repaints don’t work quite right, just general whackiness.

So since this is only happening after some period of time on a machine, we started looking at resources. Using the pooltag tool from sysinternals we found one particular tag that looked out of control. The tag was FSrN, which according to the lists I could find is “File-system runtime”. Helpful, no?

So that brings me to the meat of the story. I wanted to find out who was allocating that and what the call stack looked like at the time. In order to do this I needed to put a breakpoint on ExAllocatePoolWithTag, but that gets called ALL the time. I only wanted to break when we hit the right tag value. So I came up with the following command to set the breakpoint in windbg:

bp nt!ExAllocatePoolWithTag “j (Poi(ss:esp+c) = ‘NrSF’) ‘kb; db ss:esp+c’; ‘gc’ “

You can check out the windbg help for more information on conditional breakpoints. It will explain what the above command means. I wanted to post it since I couldn’t find any good example of how to do this with a non-integer value.