Every once in a while, when I am debugging something at work, I run into the problem of missing symbols. All of my products build symbols should be copied up to a symbol server, which makes everything “just work.” When it doesn’t just work, it always takes a manually process of figuring out how the hell symbols are supposed to work… again.

The basic scenario is that a debugger needs to use information it has available (e.g., in a crash dump file) to find the pdb file that matches the code you are debugging. Often, not a lot of information is available to the debugger, but where we are going to start is the header for the image. Let us first examine the module information…

0:017> lmvm mysvc
Browse full module list
start end module name
00007ff6`1a070000 00007ff6`1a2c2000 mysvc T (no symbols)
   Loaded symbol image file: mysvc.exe
   Image path: mysvc.exe
   Image name: mysvc.exe
   Browse all global symbols functions data
   Timestamp: Sun May 15 23:47:13 2022 (6281E561)
   CheckSum: 00254124
   ImageSize: 00252000
   File version: 2.9.8171.14983
   Product version: 2.9.8171.14983
   File flags: 0 (Mask 3F)
   File OS: 40004 NT Win32
   File type: 1.0 App
   File date: 00000000.00000000
   Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
   Information from resource tables:

Notice the checksum and image size here. These two pieces of information will be used by the debugger to build a hash that it uses to locate the indexed image file on the symbol server. To see this in action, we can configure the debugger to point to an empty symbol server path and turn on noisy symbol output…

0:017> .sympath SRV*D:\dust
DBGHELP: Symbol Search Path: srv*d:\dust
Symbol search path is: SRV*D:\dust
Expanded Symbol search path is: srv*d:\dust
************* Path validation summary **************
Response Time (ms) Location
Deferred SRV*D:\dust

0:017> !sym noisy

0:017> .reload /f mysvc.exe
SYMSRV: BYINDEX: 0x7
   d:\dust
   mysvc.exe
   6281E561252000
SYMSRV: UNC: d:\dust\mysvc.exe\6281E561252000\mysvc.exe - path not found
SYMSRV: UNC: d:\dust\mysvc.exe\6281E561252000\mysvc.ex_ - path not found
SYMSRV: UNC: d:\dust\mysvc.exe\6281E561252000\file.ptr - path not found
SYMSRV: RESULT: 0x80070003
DBGHELP: DBGENG: mysvc.exe - Image mapping disallowed by non-local path.
Unable to load image mysvc.exe, Win32 error 0n2
DBGENG: mysvc.exe - Partial symbol image load missing image info
DBGHELP: No header for mysvc.exe. Searching for dbg file
DBGHELP: .\mysvc.dbg - file not found
DBGHELP: .\exe\mysvc.dbg - path not found
DBGHELP: .\symbols\exe\mysvc.dbg - path not found
DBGHELP: mysvc.exe missing debug info. Searching for pdb anyway
DBGHELP: Can't use symbol server for mysvc.pdb - no header information available
DBGHELP: mysvc.pdb - file not found
*** WARNING: Unable to verify timestamp for mysvc.exe
DBGHELP: mysvc- no symbols loaded

We can see that the debugger tries to find the indexed binary file based on a concatenated image timestamp and image size. If it can’t find this (i.e., if the binary itself is not indexed), then the debugger is not going to be able to figure out the hash to lookup your pdb. So next, let us manually add the binary to our symbol store and see what the next step looks like.

D:\dev> & symstore.exe add /f mysvc.exe /s D:\dust /t ProductName
Finding ID... 0000000001

SYMSTORE: Number of files stored = 1
SYMSTORE: Number of errors = 0
SYMSTORE: Number of files ignored = 0

0:017> .reload /f mysvc.exe
SYMSRV: BYINDEX: 0x8
   d:\dust
   mysvc.exe
   6281E561252000
SYMSRV: PATH: d:\dust\mysvc.exe\6281E561252000\mysvc.exe
SYMSRV: RESULT: 0x00000000
DBGHELP: d:\dust\mysvc.exe\6281E561252000\mysvc.exe - OK
DBGENG: d:\dust\mysvc.exe\6281E561252000\mysvc.exe - Mapped image memory
SYMSRV: BYINDEX: 0x9
   d:\dust
   mysvc.pdb
   B8FA9DF507C542C097244F72C28E50EE1
SYMSRV: UNC: d:\dust\mysvc.pdb\B8FA9DF507C542C097244F72C28E50EE1\mysvc.pdb - path not found
SYMSRV: UNC: d:\dust\mysvc.pdb\B8FA9DF507C542C097244F72C28E50EE1\mysvc.pd_ - path not found
SYMSRV: UNC: d:\dust\mysvc.pdb\B8FA9DF507C542C097244F72C28E50EE1\file.ptr - path not found
SYMSRV: RESULT: 0x80070003
DBGHELP: mysvc.pdb - file not found
DBGHELP: D:\a\_work\1\s\bld\x64\Release\mysvc.pdb - file not found
DBGHELP: mysvc- no symbols loaded

Ok, so the debugger was able to find the indexed binary, and somehow got this magical hash, B8FA9DF507C542C097244F72C28E50EE1, that it is trying to use to find the pdb file. To see where this comes from, we can examine the headers of the binary file using dumpbin.exe. For brevity, I will filter the output to just find the section I am interested in…

dumpbin /headers mysvc.exe | Select-String "Format: RSDS"

6281E561 cv            42 001FAA10   1F9610    Format: RSDS, {B8FA9DF5-07C5-42C0-9724-4F72C28E50EE}, 1, D:\a\_work\1\s\bld\x64\Release\mysvc.pdb

Here the exe file has embedded a link to the debug information. The guid here is taken as a pure string of hex digits (strip out the dashes and braces) and used as part of the file path to find the pdb. Again, to confirm, we will manually add the pdb file to our symbol store and retry the reload command.

D:\dev> & symstore.exe add /f mysvc.pdb /s D:\dust /t ProductName
Finding ID... 0000000002

SYMSTORE: Number of files stored = 1
SYMSTORE: Number of errors = 0
SYMSTORE: Number of files ignored = 0

0:017> .reload /f mysvc.exe
SYMSRV: BYINDEX: 0xA
   d:\dust
   mysvc.exe
   6281E561252000
SYMSRV: PATH: d:\dust\mysvc.exe\6281E561252000\mysvc.exe
SYMSRV: RESULT: 0x00000000
DBGHELP: d:\dust\mysvc.exe\6281E561252000\mysvc.exe - OK
DBGENG: d:\dust\mysvc.exe\6281E561252000\mysvc.exe - Mapped image memory
SYMSRV: BYINDEX: 0xB
   d:\dust
   mysvc.pdb
   B8FA9DF507C542C097244F72C28E50EE1
SYMSRV: PATH: d:\dust\mysvc.pdb\B8FA9DF507C542C097244F72C28E50EE1\mysvc.pdb
SYMSRV: RESULT: 0x00000000
DBGHELP: mysvc- private symbols & lines
d:\dust\mysvc.pdb\B8FA9DF507C542C097244F72C28E50EE1\mysvc.pdb

Now remember in elementary school how they used to teach you all the basic ways to do things that mostly worked, but then after all that they would just show you the “standard algorithm” that just always works? Yeah, well apparently, I do the same thing, so here is a nifty little debugger command that just kind does everything above…

0:017> !lmi mysvc
Loaded Module Info: [mysvc] 
         Module: mysvc
   Base Address: 00007ff61a070000
     Image Name: mysvc.exe
   Machine Type: 34404 (X64)
     Time Stamp: 6281e561 Sun May 15 23:47:13 2022
           Size: 252000
       CheckSum: 254124
Characteristics: 22  
Debug Data Dirs: Type  Size     VA  Pointer
             CODEVIEW    42, 1faa10,  1f9610 RSDS - GUID: {B8FA9DF5-07C5-42C0-9724-4F72C28E50EE}
               Age: 1, Pdb: D:\a\_work\1\s\bld\x64\Release\mysvc.pdb
           VC_FEATURE    14, 1faa54,  1f9654 [Data not mapped]
                 POGO   414, 1faa68,  1f9668 [Data not mapped]
    Symbol Type: DEFERRED - No error - symbol load deferred
    Load Report: no symbols loaded

Well, that about does it for the basics of how symbol resolution works. In a future article I will address some more advanced topics, such as indexing pointers to files and compressed files and such advanced topics.