Jeremy Hurren

Blog

Open Source Project for Testing Native API System Calls

Jeremy Hurren

At work I have recently been working on some system software that integrates closely with the registry. This means I require some fairly in-depth knowledge about the Registry APIs, and I also need a way to test their functionality (both with and without my software involved).

I have done this in the past by writing custom code, which then gets lost or out-of-date. This time around I decided to make a more interactive and reusable tool to help me accomplish this, and my company – FSLogix, Inc. – decided to release the tool as open source on CodePlex. I also wrote a blog article with more in-depth information about the tool over at the FSLogix Blog.

Assembly Language is Still Relevant

Jeremy Hurren

If you have read my blog at all, you already know that I enjoy poking around at the assembly-code-level in a debugger. But I believe it’s really NOT just for the geek interested in the arcane. Understanding what your code looks like at the assembly level can help you figure out what’s really going on, and in some cases it can be a whole lot easier than the same task at a higher level language, such as C. (All you C# and Java guys are laughing at the idea that C is a high level language, but this idea can be applied to understanding the byte code generated from those languages as well.)

As a first example, a good defensive programming technique is to use asserts to make sure that your assumptions about the code are true. In a case where your assumption turns out to be wrong, these asserts can make a debugging session a nightmare. In a complicated product, such as assert may make it impossible to step through code running on a different thread. Fortunately it is super-easy to disable such an assert right in the debugger by just changing an opcode. No re-compiling, re-deploying, getting the scenario set up to reproduce again. In the assembly code, you can find the test for your assertion, which will be followed by a jump instruction of some type. If you are testing the return value of a function for success, you may see something like:

test eax, eax
jne <address>

If you look at the byte code for the jne (jump not equal) instruction it will likely begin with a 0x75 (assuming an x86 instruction set and a short jump). Disabling that assert is a simple as changing that 0x75 to an 0x74, which changes the jne instruction to a je (jump equal). Good bye assertion.

Another very closely related example I ran into the other day. I forgot to put a logical negation in a statement. The if statement was testing for the exact opposite of what I intended. Well setting up the scenario to test it was about a 5 minute process. But using the same trick as with the assert, I was able to switch my code to do the right thing and continue debugging the remainder of the scenario.

One more example that will be more applicable to you C# guys out there. In the latest version of the C# language and .NET Framework, Microsoft added a couple of new keywords: async and await. Well the descriptions of the functionality and the explanations I read didn’t quite add up. Being quite familiar with multi-threaded programming, I just didn’t feel like I was getting a full picture of what was really happening. (I actually asked about it at a developers conference, and the speaker said “I don’t know how they do it, it’s just magic.”) So I wrote a quick sample, fired up a debugger and looked at the generated byte code, to get the complete picture of what was happening. It works about as I expected it would, but without understanding assembly level code, I never would have been able to figure that out.

So the next time you’re tempted to think that assembly language programming is a dead art, think again.

Basics of Pending I/O in Windows Kernel

Jeremy Hurren

So I was troubleshooting a bug with a co-worker the other day and what we were seeing was a little confusing, and it turned out to be a difference in when I/O gets pended in the system. The basic scenario was that we were calling ZwQueryDirectoryFile to do some testing of a filter driver that was changing the directory query results. In testing without the driver installed, the return value from ZwQueryDirectoryFile was always STATUS_PENDING. So the assumption was made that this would continue to always be true.

Well as soon as our filter was installed and we changed the results of the query, the api started returning immediately. Our code was assuming that then I/O would be pending and was waiting on the handle and checking the IoStatusBlock for the results, and it just wasn’t looking correct: the IoStatusBlock was showing STATUS_SUCCESS when we thought our filter had returned STATUS_NO_SUCH_FILE. After figuring out what was going on, I thought I would write a bit about how pending I/O works in the kernel.

When an API call is made, there are times when it can be immediately determined what result should be returned. For example, if you pass a bad parameter, the system doesn’t need to pend the I/O because it can return STATUS_INVALID_PARAMETER immediately. In this case, ZwQueryDirectoryFile would just return STATUS_INVALID_PARAMETER as its result, and the IoStatusBlock should be ignored.

If the API call returns STATUS_PENDING, only then should the caller wait on the handle to be signaled and then check the value of the IoStatusBlock parameter to determine the outcome of the I/O operation.

In our specific case, our filter driver was populating the results and returning STATUS_NO_SUCH_FILE. This was changing the behavior of the test call to ZwQueryDirectoryFile from sending the request to disk and pending the I/O (the behavior without our driver) to returning the status code immediately.

So it seems that a correct way to call one of these functions should look something like the following:

IO_STATUS_BLOCK IoStatusBlock;
NTSTATUS status = ZwQueryDirectoryFile( ... );
if ( STATUS_PENDING == status )
{
   ZwWaitForSingleObject( ... );
   status = IoStatusBlock.Status;
}
// Do something with the result of the query directory call

Building products consumers actually want (Repost)

Jeremy Hurren

This is a repost of an great article by Bob Gower that I read this morning. Since becoming one of the founders of FSLogix, I have become vastly more focused on how to build a product that people are happy to pay for. Working for a large company you sometimes lose sight of this oh-so-important idea, but in a small company it is paramount. This article talks about how Agile development ideas help a development team get to a product that customers want to buy.

Building Products Consumers Actually Want

Directory Notification Revisited

Jeremy Hurren

Today I had to implement a service that used directory notification to watch for changes in a directory and then handle each change by migrating it to another location. I went back to my previous post about directory notification and used some of the code there to implement what I needed. Unfortunately, the code in that post for processing the returned data is not quite right. It skips that last entry in the returned buffer, which in practice means it skips most of the entries, since it is most common for there to be a single entry in the buffer.

Following is a code segment that processes the entries correctly.
Continue reading “Directory Notification Revisited”

New blog

Jeremy Hurren

I recently decided to move my blog, reformat my web site, and just basically redefine my web presence. This is my first official post on the new blog, and I will be moving my older blog entries over from the wordpress.com hosted site as time permits. My personal and political posts will be going away, as the new blog format will focus entirely on professional endeavors. Read and enjoy!

ZwQueryInformationFile with FileStreamInformation equals an empty buffer

Jeremy Hurren

Debugged a fun crash today. I had some driver code that was enumerating all the child streams of a file. Because of a bug it ended up calling the code for a directory rather than a file. What I would have expected to see is that a call to ZwQueryInformationFile for the FileStreamInformation class would have returned at least one FILE_STREAM_INFORMATION structure, or an error code such as a not found. Instead it seems that the call returns STATUS_SUCCESS, and the IoStatusBlock.Information contains zero bytes returned. From what I can tell this only happens on a directory.

IoGetCurrentIrpStackLocation in the debugger

Jeremy Hurren

Today I had occasion to debug a problem with some IRP handling code in my driver. In the particular debugger session I found myself in, I wanted to examine some of the Irp parameters, found in the current stack location. Unfortunately I had only a pointer to the Irp in this code, and therefore needed to figure out how to find the stack location pointer using the debugger. Fun!

The function that does this in code is IoGetCurrentIrpStackLocation, which I decided to disassemble. The first part of the function basically just checks the StackCount and CurrentLocation members of the Irp to make sure that everything is ok. (This fires an assert if it doesn’t check out.) Then near the bottom of the function, we find

drv!IoGetCurrentIrpStackLocation+0x42 [source language="removed"][/source]:
23318 9dc43ea2 8b4d08          mov     ecx,dword ptr [ebp+8]
23318 9dc43ea5 8b4160          mov     eax,dword ptr [ecx+60h]
23319 9dc43ea8 8be5            mov     esp,ebp
23319 9dc43eaa 5d              pop     ebp
23319 9dc43eab c20400          ret     4

So first this moves the Irp pointer into ecx, and then goes to offset 60 within that structure (which happens to be outside the range of the documented structure), and puts the pointer there into eax for return to the caller. So I try that in my debugger and compare with the output of the !irp command.

kd> !irp 9f104f68
Irp is active with 1 stacks 1 is current (= 0x9f104fd8)
 No Mdl: System buffer=9f08cbf0: Thread 88e05558:  Irp stack trace.  
     cmd  flg cl Device   File     Completion-Context
>[  e, 0]   5  1 88dcbd18 88e05ab8 00000000-00000000    pending
	       FileSystemFSLX
			Args: 0000040c 00000000 94000004 00000000

kd> dd 9f104f68+60 L1
9f104fc8  9f104fd8

kd> db 9f104fd8
9f104fd8  0e 00 05 01 0c 04 00 00-00 00 00 00 04 00 00 94  ................
9f104fe8  00 00 00 00 18 bd dc 88-b8 5a e0 88 00 00 00 00  .........Z......
9f104ff8  00 00 00 00 15 15 15 15-?? ?? ?? ?? ?? ?? ?? ??  ........????????

At this point, I now realized that this structure that I needed to look at really doesn’t provide anything that’s not already provided in the !irp output. The Args output of that command correspond to the members of the IO_STACK_LOCATION.Parameters union. In this case, I am looking at a device control Irp, so these parameters are OutputBufferLength (40c), InputBufferLength (0), IoControlCode (94000004), and Type3InputBuffer (0).

So I guess the bottom line of this post is that !irp is cool and does just what you need it to, I just had to poke around a little bit before I realized it.