Jeff Sanders Technical Blog

I am a Microsoft employee that has worked on all aspects of the Web Stack for a long time. I hope these blogs are useful to you! Use this information at your own risk.


<< Go Back

Create A Script For Debugdiag To Analyze Managed Webexceptions

- 19 Dec 2008

DebugDiag was originally designed to help IIS engineers find issues in IIS.  It does not have a very rich experience however when analyzing Managed Exceptions.  I will show you how to modify some existing scripts and use then to analyze issues with the HttpWebRequest class.

First copy the existing script CrashHangAnalysis.asp to a file in the same directory and call it HttpWebRequestCrash.asp.  Next let’s make some modifications to call the new CLRAnalysis script we will develop.

Change the description to this: <%@ Description = Crash and hang analysis for .NET WebExceptions %>
And then change this line: <!– #include File = “inc/AnalyzeCrash.asp” –>  to point to the file we will create: <!– #include File = “inc/CLRAnalyzeCrash.asp” –>

Next Copy AnalyzeCrash.asp to our new file CLRAnalyzeCrash.asp.

Now the work begins.  We will modify CLRAnalyzeCrash.asp to do some enhanced CLR analysis of WebExceptions.

This file already has a section to analyze CLR exceptions, but we want to modify it to go deep into WebExceptions.  Find the function Sub AnalyzeClr(ByVal Exception, ByVal ExceptionThread, ByVal suppressSummary)

We will need some new variables so add these at the beginning of the function:

 Dim CmdOutput
 Dim Lines
 Dim iStatus
 Dim pos
 Dim nextPE
 Dim theResponseAddr

Next we will modify the arguments passed to the function GetCLRExceptionType and modify the function itself.  Find this function where it is called and change it to this:
GetCLRExceptionType(GetCLRExceptionPtr(ExceptionThread), False, iStatus, theResponseAddr)

The iStatus and theResponseAddr variables will be added to return us more information about the exception.  Next lets modify that function.

First change the signature: Function GetCLRExceptionType(ByVal ExceptionObjHexAddr, ByVal bInnerException, ByRef iStatus, ByRef theResponseAddr)

Then add this variable to the top of the function: Dim pos

Now we want to output a header if there is an inner exception, so change this code to write out ‘Inner’

     If InStr(Lines(i), “_innerException”) <> 0 Then
      GetCLRExceptionType = GetCLRExceptionType(“0x” & Mid(Lines(i), 60, 8), False)
      Manager.Write “
<h1>inner</h1></br>”
      Exit For

Now change the outer exception code to get the status and response members (if they exist):

   ElseIf Len(Lines(i)) >= 7 Then
    pos = InStr(Lines(i), “m_Status”)
    if pos >= 1 Then
        iStatus = Mid(Lines(i), pos-9,8 )
       
    End If
    pos = InStr(Lines(i), “m_Response”)
    if pos >= 1 Then
        theResponseAddr = Mid(Lines(i), pos-9,8 )
    End If

    If InStr(Lines(i), “Name: “) =   1 Then
     GetCLRExceptionType = Mid(Lines(i), 7)
     
    End If
   End If

That should cover getting the variables, Status and Response, from the exception object, now back to the other function to do something with those variables.

The rest of the code in the previous function will make some decisions if this is a WebException and print out some possible resolutions.  Finally it will dump the managed stack:

 If ClrExceptionType <> “” Then
     Message = Message & “</b> has caused a CLR Exception of type (<font color=’red’>” & ClrExceptionType & “</font>)
    
     If ClrExceptionType = “System.Net.WebException” Then
            ‘remove the vendor statement
            pos = InStr( Recommendation,”Please follow up with vendor”)
            if (pos<>0) then
                Recommendation = Mid(Recommendation, 1, pos-1)
            end if
           
           
            ‘public enum WebExceptionStatus
            ‘{
            ‘    Success, 0
            ‘    NameResolutionFailure, 1
            ‘    ConnectFailure, 2
            ‘    ReceiveFailure, 3
            ‘    SendFailure, 4
            ‘    PipelineFailure, 5
            ‘    RequestCanceled, 6
            ‘    ProtocolError, 7
            ‘    ConnectionClosed, 8
            ‘    TrustFailure, 9
            ‘    SecureChannelFailure, 10
            ‘    ServerProtocolViolation, 11
            ‘    KeepAliveFailure, 12
            ‘    Pending, 13
            ‘    Timeout, 14
            ‘    ProxyNameResolutionFailure, 15
            ‘    UnknownError, 16
            ‘    MessageLengthLimitExceeded, 17
            ‘    CacheEntryNotFound, 18
            ‘    RequestProhibitedByCachePolicy, 19
            ‘    RequestProhibitedByProxy 20
            ‘}
 
            Select Case CInt(iStatus)
                Case 1
                    Recommendation = Recommendation &”

This is a DNS error.  For some reason the server name cannot be resolved.  Do you have permissions to the HOST file?  Is this a service such as ASP.NET?  See this article for help: <a target=’_blank’ href=’http://support.microsoft.com/kb/318140′>The underlying connection was closed: The remote name could not be resolved.</a>” 
                Case 2
                    ‘ “Unable to connect to the remote server”
                    Recommendation = Recommendation &”

Lookup the error ‘Unable to connect to the remote server’ here for a resolution: <a target=’_blank’ href=’http://support.microsoft.com/kb/915599′>http://support.microsoft.com/kb/915599</a>”
                Case 3
                    ‘ “The underlying connection was closed: An unexpected error occurred on a receive.”                                    
                    Recommendation = Recommendation &”

Lookup the error ‘An unexpected error occurred on a receive’ here for a resolution: <a target=’_blank’ href=’http://support.microsoft.com/kb/915599′>http://support.microsoft.com/kb/915599</a>”
                Case 4
                    ‘ The underlying connection was closed: An unexpected error occurred on a send                    
                    Recommendation = Recommendation &”

Lookup the error ‘An unexpected error occurred on a send’ here for a resolution: <a target=’_blank’ href=’http://support.microsoft.com/kb/915599′>http://support.microsoft.com/kb/915599</a>”
                   
                   
                Case 9
                    ‘TrustFailure A server certificate could not be validated.
                    ‘The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel
                    Recommendation = Recommendation &”

Lookup the error ‘Unable to connect to the remote server’ here for a resolution: <a target=’_blank’ href=’http://support.microsoft.com/kb/915599′>http://support.microsoft.com/kb/915599</a>
to override the error you can do ignore the certificate error until you determine why the certificate is not trusted (see KB article for this information).”
                   
         &nb
sp;      Case 14
                    ‘The operation has timed out’
                    Recommendation = Recommendation &”

Lookup the error ‘The operation has timed out’ here for a resolution: <a target=’_blank’ href=’http://support.microsoft.com/kb/915599′>http://support.microsoft.com/kb/915599</a>
the request is taking too long to come back.  Also see <a target=’_blank’ href= ‘http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.timeout.aspx’>HttpWebRequest Timeout Property</a>”
                Case 7                      
                    ‘Protocol Error
                    ‘ check the response status code.  Could be something like:   System.Net.HttpStatusCode.Unauthorized
                    ‘ !do theResponseAddr jps
                    CmdOutput = g_Debugger.Execute(“!DumpObj ” & theResponseAddr)
                    Lines = split(CmdOutput, chr(10))
                 For I = 0 to UBound(Lines)
                     pos = InStr(Lines(i), “m_StatusCode”)
                     If pos <> 0 Then
                            ‘ if it is there then get the m_StatusCode
                            ‘ dump that out.
                            ‘ todo… work on status code…
                            CmdOutput = Mid(Lines(i), pos-9,8 )
                           
                            Select Case CInt(CmdOutput)
                                Case 401
                                    Recommendation = Recommendation &”Unauthorized Http Status 401 means your credentials are denied access.  See if you are passing credentials in your code.  Example: <a target=’_blank’ href=’http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.credentials.aspx’> HttpWebRequest Credentials property</a>.  Also see ASP.NET impersonation to pass the logged on user credentials: <a target=’_blank’ href=’http://msdn.microsoft.com/en-us/library/xh507fc5.aspx’>ASP.NET Impersonation</a> and this Wiki <a target=’_blank’ href=’http://wiki.asp.net/page.aspx/44/impersonation-and-delegation/’>Impersonation and Delegation</a>”
                                Case 407
                                    ‘hot fix? http://support.microsoft.com/kb/928563
                                    Recommendation = Recommendation &”Http Status 407 means your proxy requires credentials.  See if you are passing proxy credentials in your code.  Example: <a target=’_blank’ href=’http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.proxy.aspx’> HttpWebRequest Proxy property</a> and <a target=’_blank’ href=’http://support.microsoft.com:80/kb/318140′>Proxy servers that require NTLM authentication</a>.  It is possible you may need a hotfix.  See <a target=’_blank’ href=’http://support.microsoft.com/kb/928563′>FIX: The System.Net.HttpWebRequest class may not maintain a persistent connection to a proxy in the .NET Framework 2.0</a>”
                                Case Else

                            End Select
                           
                            Recommendation = Recommendation & “
See HttpStatusCode ” & CmdOutput & ” in this article: <a target=’_blank’ href=’http://msdn.microsoft.com/en-us/library/system.net.httpstatuscode.aspx’>HttpStatusCode Enumeration</a>”
                           
                         Exit For   
                     End If
                    Next                    
                Case Else
                     Recommendation = Recommendation & “

Status Code = ” & iStatus & ”
this error is an enumeration (starts at 0) you can look up here <a target=’_blank’ href=’http://msdn.microsoft.com/en-us/library/system.net.webexceptionstatus.aspx’>WebExceptionStatus Enumeration</a>

         End Select
        

     End If
 Else
     Message = Message & “</b> has caused a <font color=’red’>CLR Exception</font>)</b>”
 End If
 
 Message = Message & ” on thread ” &_
  GetThreadIDWithLink(ExceptionThread.ThreadID) & “

    
 Manager.Write “<h4>Enhanced CLR analysis</h4>”
 
 CmdOutput = g_Debugger.Execute(“!CLRStack”)
 
 Manager.Write “Managed Stack

 
 Lines = split(CmdOutput, chr(10))
 
 For I = 0 to UBound(Lines)
     Manager.Write Lines(I) & “

 Next
 
 CmdOutput = g_Debugger.Execute(“!pe”)
 Lines = split(CmdOutput, chr(10))
 
 
 
 Message = Message & “
Managed Exception:

 For I = 0 to UBound(Lines)
     pos = InStr(Lines(i), “!PrintException”)
     If pos <> 0 Then
         ‘todo expand this as deep as we can
     End If
     Message = Message & Lines(I) & “

 Next
 
  

 If DirectCaller > 0 Then
  Message = Message & “
This exception originated from ” & g_Debugger.GetSymbolFromAddress(DirectCaller) & “. “
 End If
 
 Manager.Write Message
 If Not suppressSummary Then  Manager.ReportError Message, Recommendation, 1000
 
 If Not Module is Nothing And g_Debugger.IsCrashDump Then
  ModuleInfo Module
 End If   
End Sub ‘AnalyzeClr<p mce_keep="true">

Completed function for AnalyzeClr:</p> <p mce_keep="true">Sub AnalyzeClr(ByVal Exception, ByVal ExceptionThread, ByVal suppressSummary)
 Dim Module, StackFrame, StackFrames
 Dim FunctionName, FuncName
 Dim Message, Recommendation
 Dim ExceptionCode, CallingFunction
 Dim DirectCaller, TargetFunctions
 Dim CLRExceptionType
 Dim CmdOutput
 Dim Lines
 Dim iStatus
 Dim pos
 Dim nextPE
 Dim theResponseAddr
 
 
 ExceptionCode = g_Debugger.GetAs32BitHexString(Exception.ExceptionCode)
 FunctionName = g_Debugger.GetSymbolFromAddress(Exception.ExceptionAddress)
 Set Module = g_Debugger.GetModuleByAddress(Exception.ExceptionAddress)
 
 
 
 Message = “In ” & g_ShortDumpFileName & ” the assembly instruction at ” & FunctionName
 
 If Module is Nothing Then
  Message = Message & “
which does not correspond to any known native module in the process”
  Recommendation = “Please contact Microsoft Corporation for troubleshooting steps on stack corruption

 Else
  Message = Message & “</b> in ” & Module.ImageName
  
  If Module.VSCompanyName <> “” Then
   Message = Message & “
from ” & Module.VSCompanyName
  End If
  
  If InStr(1, UCase(FunctionName), “KERNEL32!RAISEEXCEPTION”) <> 0 Then  
   DirectCaller = GetDirectCaller(ExceptionThread.StackFrames, “KERNEL32!RAISEEXCEPTION”, 0)
  
   If DirectCaller > 0 Then
    Recommendation = “Review the faulting call stack for thread ” & GetThreadIDWithLink(ExceptionThread.ThreadID) & ” to determine ” & _
     “root cause for the exception.
” & GetVendorMessage(DirectCaller)
   Else
    Recommendation = “
An exception thrown by <Font Color=’Red’>Kernel32!RaiseException</Font> usually indicates a problem with another module. ” &_
        “Please review the stack for the faulting thread (” & GetThreadIDWithLink(ExceptionThread.ThreadID) & “) ” &_
        “further to determine which module actually threw the exception raised by Kernel32.dll.

”   
   End If  
  ElseIf InStr(1, UCase(FunctionName), “NTDLL!KIRAISEUSEREXCEPTIONDISPATCHER”) <> 0 Then
   TargetFunctions = “ADVAPI32!REGCLOSEKEY;KERNEL32!CLOSEHANDLE”
   
   Set StackFrames = ExceptionThread.StackFrames
   For I = 0 to StackFrames.Count – 1
    Set StackFrame = StackFrames(I)
    FuncName = Split(UCase(g_Debugger.GetSymbolFromAddress(StackFrame.InstructionAddress)), “+”, -1)
    
    If InStr(1, TargetFunctions, UCase(FuncName(0))) <> 0 Then
     Set CallingFunction = g_Debugger.GetModuleByAddress(StackFrame.ReturnAddress)
     DirectCaller = StackFrame.ReturnAddress
     
     Recommendation = “This exception occured as a result of an invalid handle passed to ” & FuncName(0) & ” by the following function:

” &_
     g_Debugger.GetSymbolFromAddress(StackFrame.ReturnAddress) & “


Please follow up with the vendor of this module”
    
     If CallingFunction.VSCompanyName <> “” Then
      Recommendation = Recommendation & “, ” &  CallingFunction.VSCompanyName & “,”
     End If
     
     Recommendation = Recommendation & ” for further assistance with this issue.”
     Set Module = g_Debugger.GetModuleByAddress(DirectCaller)
    End If
   Next
  Else
   Recommendation = GetVendorMessage(Exception.ExceptionAddress)
  End If  
 End If</p> <p mce_keep="true"> CLRExceptionType = GetCLRExceptionType(GetCLRExceptionPtr(ExceptionThread), False, iStatus, theResponseAddr)
 If ClrExceptionType <> “” Then
     Message = Message & “
has caused a CLR Exception of type (<font color=’red’>” & ClrExceptionType & “</font>)
    
     If ClrExceptionType = “System.Net.WebException” Then
            ‘remove the vendor statement
            pos = InStr( Recommendation,”Please follow up with vendor”)
            if (pos<>0) then
                Recommendation = Mid(Recommendation, 1, pos-1)
            end if
           
           
            ‘public enum WebExceptionStatus
            ‘{
            ‘    Success, 0
            ‘    NameResolutionFailure, 1
            ‘    ConnectFailure, 2
            ‘    ReceiveFailure, 3
            ‘    SendFailure, 4
            ‘    PipelineFailure, 5
            ‘    RequestCanceled, 6
            ‘    ProtocolError, 7
            ‘    ConnectionClosed, 8
            ‘    TrustFailure, 9
            ‘    SecureChannelFailure, 10
            ‘    ServerProtocolViolation, 11
            ‘    KeepAliveFailure, 12
            ‘    Pending, 13
            ‘    Timeout, 14
            ‘    ProxyNameResolutionFailure, 15
            ‘    UnknownError, 16
            ‘    MessageLengthLimitExceeded, 17
            ‘    CacheEntryNotFound, 18
            ‘    RequestProhibitedByCachePolicy, 19
            ‘    RequestProhibitedByProxy 20
            ‘}
 
            Select Case CInt(iStatus)
                Case 1
                    Recommendation = Recommendation &”

This is a DNS error.  For some reason the server name cannot be resolved.  Do you have permissions to the HOST file?  Is this a service such as ASP.NET?  See this article for help: <a target=’blank’ href=’http://support.microsoft.com/kb/318140′>The underlying connection was closed: The remote name could not be resolved.</a>” 
                Case 2
                    ‘ “Unable to connect to the remote server”
                    Recommendation = Recommendation &”

Lookup the error ‘Unable to connect to the remote server’ here for a resolution: <a target=’_blank’ href=’http://support.microsoft.com/kb/915599′>http://support.microsoft.com/kb/915599</a>”
                Case 3
                    ‘ “The underlying connection was closed: An unexpected error occurred on a receive.”                                    
                    Recommendation = Recommendation &”

Lookup the error ‘An unexpected error occurred on a receive’ here for a resolution: <a target=’_blank’ href=’http://support.microsoft.com/kb/915599′>http://support.microsoft.com/kb/915599</a>”
                Case 4
                    ‘ The underlying connection was closed: An unexpected error occurred on a send                    
                    Recommendation = Recommendation &”

Lookup the error ‘An unexpected error occurred on a send’ here for a resolution: <a target=’_blank’ href=’http://support.microsoft.com/kb/915599′>http://support.microsoft.com/kb/915599</a>”
                   
                   
                Case 9
                    ‘TrustFailure A server certificate could not be validated.
                    ‘The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel
                    Recommendation = Recommendation &”

You could add this delegate to your code and dump the error to fix it: <a target=’_blank’ href=’http://msdn.microsoft.com/en-us/library/system.net.security.remotecertificatevalidationcallback.aspx’>RemoteCertificateValidationCallback Delegate</a>.
Try a console app with the same code, if it succeeds but an asp.net app fails, this means the Local Machine store may not have the Trusted Roo
t and Intermediate Authority certificates in the store.
Lookup the error ‘Unable to connect to the remote server’ here for a resolution: <a target=’_blank’ href=’http://support.microsoft.com/kb/915599′>http://support.microsoft.com/kb/915599</a>
to override the error you can do ignore the certificate error until you determine why the certificate is not trusted (see KB article for this information).”
                   
                Case 14
                    ‘The operation has timed out’
                    Recommendation = Recommendation &”

Lookup the error ‘The operation has timed out’ here for a resolution: <a target=’_blank’ href=’http://support.microsoft.com/kb/915599′>http://support.microsoft.com/kb/915599</a>
the request is taking too long to come back.  Also see <a target=’_blank’ href= ‘http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.timeout.aspx’>HttpWebRequest Timeout Property</a>”
                Case 7                      
                    ‘Protocol Error
                    ‘ check the response status code.  Could be something like:   System.Net.HttpStatusCode.Unauthorized
                    ‘ !do theResponseAddr jps
                    CmdOutput = g_Debugger.Execute(“!DumpObj ” & theResponseAddr)
                    Lines = split(CmdOutput, chr(10))
                 For I = 0 to UBound(Lines)
                     pos = InStr(Lines(i), “m_StatusCode”)
                     If pos <> 0 Then
                            ‘ if it is there then get the m_StatusCode
                            ‘ dump that out.
                            ‘ todo… work on status code…
                            CmdOutput = Mid(Lines(i), pos-9,8 )
                           
                            Select Case CInt(CmdOutput)
                                Case 401
                                    Recommendation = Recommendation &”Unauthorized Http Status 401 means your credentials are denied access.  See if you are passing credentials in your code.  Example: <a target=’_blank’ href=’http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.credentials.aspx’> HttpWebRequest Credentials property</a>.  Also see ASP.NET impersonation to pass the logged on user credentials: <a target=’_blank’ href=’http://msdn.microsoft.com/en-us/library/xh507fc5.aspx’>ASP.NET Impersonation</a> and this Wiki <a target=’_blank’ href=’http://wiki.asp.net/page.aspx/44/impersonation-and-delegation/’>Impersonation and Delegation</a>”
                                Case 407
                                    ‘hot fix? http://support.microsoft.com/kb/928563
                                    Recommendation = Recommendation &”Http Status 407 means your proxy requires credentials.  See if you are passing proxy credentials in your code.  Example: <a target=’_blank’ href=’http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.proxy.aspx’> HttpWebRequest Proxy property</a> and <a target=’_blank’ href=’http://support.microsoft.com:80/kb/318140′>Proxy servers that require NTLM authentication</a>.  It is possible you may need a hotfix.  See <a target=’_blank’ href=’http://support.microsoft.com/kb/928563′>FIX: The System.Net.HttpWebRequest class may not maintain a persistent connection to a proxy in the .NET Framework 2.0</a>”
                                Case Else</p> <p mce_keep="true">                            End Select
                           
                            Recommendation = Recommendation & “
See HttpStatusCode ” & CmdOutput & ” in this article: <a target=’_blank’ href=’http://msdn.microsoft.com/en-us/library/system.net.httpstatuscode.aspx’>HttpStatusCode Enumeration</a>”
                           
                         Exit For   
                     End If
                    Next                    
                Case Else
                     Recommendation = Recommendation & “

Status Code = ” & iStatus & ”
this error is an enumeration (starts at 0) you can look up here <a target=’_blank’ href=’http://msdn.microsoft.com/en-us/library/system.net.webexceptionstatus.aspx’>WebExceptionStatus Enumeration</a>

         End Select
        
        Else
         If ClrExceptionType = “System.Security.Authentication.AuthenticationException” Then
                ‘remove the vendor statement
                pos = InStr( Recommendation,”Please follow up with vendor”)
                if (pos<>0) then
                    Recommendation = Mid(Recommendation, 1, pos-1)
                end if
                Recommendation = Recommendation &”

You could add this delegate to your code and dump the error to fix it: <a target=’_blank’ href=’http://msdn.microsoft.com/en-us/library/system.net.security.remotecertificatevalidationcallback.aspx’>RemoteCertificateValidationCallback Delegate</a>.
Try a console app with the same code, if it succeeds but an asp.net app fails, this means the Local Machine store may not have the Trusted Root and Intermediate Authority certificates in the store.
Lookup the error ‘Unable to connect to the remote server’ here for a resolution: <a target=’_blank’ href=’http://support.microsoft.com/kb/915599′>http://support.microsoft.com/kb/915599</a>
to override the error you can do ignore the certificate error until you determine why the certificate is not trusted (see KB article for this information).”</p> <p mce_keep="true">         End If
    
     End If
 Else
     Message = Message & “</b> has caused a <font color=’red’>CLR Exception</font>)</b>”
 End If
 
 Message = Message & ” on thread ” &

  GetThreadIDWithLink(ExceptionThread.ThreadID) & “
”</p> <p mce_keep="true">    
 Manager.Write “<h4>Enhanced CLR analysis</h4>”
 
 CmdOutput = g_Debugger.Execute(“!CLRStack”)
 
 Manager.Write “Managed Stack

 
 Lines = split(CmdOutput, chr(10))
 
 For I = 0 to UBound(Lines)
     Manager.Write Lines(I) & “

 Next
 
 CmdOutput = g_Debugger.Execute(“!pe”)
 Lines = split(CmdOutput, chr(10))
 
 
 
 Message = Message & “
Managed Exception:

 For I = 0 to UBound(Lines)
     pos = InStr(Lines(i), “!PrintException”)
     If pos <> 0 Then
         ‘todo expand this as deep as we can
     End If
     Message = Message & Lines(I) & “

 Next
 
  </p> <p mce_keep="true"> If DirectCaller > 0 Then
  Message = Message & “
This exception originated from ” & g_Debugger.GetSymbolFromAddress(DirectCaller) & “. “
 End If
 
 Manager.Write Message
 If Not suppressSummary Then  Manager.ReportError Message, Recommendation, 1000
 
 If Not Module is Nothing And g_Debugger.IsCrashDump Then
  ModuleInfo Module
 End If   
End Sub ‘AnalyzeClr</p> <p mce_keep="true">Please let me know if this is useful to you and if you run into any problems!</p>

<< Go Back