Print out MBOX/INTR state info on DMA request hang

We have been stuck on a hang that occurs during memdiags on
our multi-node p9 systems. It appears that Hostboot is never
receiving the response to the request to reclaim DMA buffers
from the FSP. From debugging we know the FSP thinks it has sent
the message over the FSI mbox but hostboot isnt seeing it. Next
time this happens if this is in the code we should be able to
get a better idea of what is happening.

Change-Id: I6b702e4094da3576ba454b5cdf0660841961baff
Reviewed-on: http://ralgit01.raleigh.ibm.com/gerrit1/60977
Reviewed-by: Richard Ward <rward15@us.ibm.com>
Tested-by: Jenkins Server <pfd-jenkins+hostboot@us.ibm.com>
Tested-by: Jenkins OP Build CI <op-jenkins+hostboot@us.ibm.com>
Tested-by: Jenkins OP HW <op-hw-jenkins+hostboot@us.ibm.com>
Reviewed-by: Roland Veloz <rveloz@us.ibm.com>
Tested-by: FSP CI Jenkins <fsp-CI-jenkins+hostboot@us.ibm.com>
Reviewed-by: William G. Hoffa <wghoffa@us.ibm.com>

2 files changed, 29 insertions, 0 deletions

diff --git a/src/include/usr/intr/interrupt.H b/src/include/usr/intr/interrupt.H
index b03f08f82..5456eefbe 100644
--- a/src/include/usr/intr/interrupt.H
+++ b/src/include/usr/intr/interrupt.H
@@ -115,6 +115,7 @@ namespace INTR
         MSG_INTR_DRAIN_QUEUE,       //!< Allow intrp to drain Q of EOI
         MSG_INTR_COALESCE,          //!< Pending interrupt to be handled
         MSG_INTR_ENABLE_PSI_INTR,   //!< Enable PSIHB Interrupts
+        MSG_INTR_DUMP,              //!< Print out interrupt info to slow buffer
     };
 
 
@@ -144,6 +145,14 @@ namespace INTR
         uint64_t phbsecure; //PSI Host Bridge Secure Control reg  - 0x90
     };
 
+    enum esbStates
+    {
+        ESB_STATE_RESET   = 0,
+        ESB_STATE_OFF     = 1,
+        ESB_STATE_PENDING = 2,
+        ESB_STATE_QUEUED  = 3,
+    };
+
 
     /**
      * Register a message queue for an interrupt type
@@ -192,6 +201,18 @@ namespace INTR
     void sendEOI(msg_q_t i_q, msg_t* i_msg);
 
     /**
+    * Sends a message to the intrrp Q to tell it to print out current state
+    * of interrupts from hostboot perspective to the slow trace buffer
+    *
+    * This will call the private intrrp functions:
+    *  printEsbStates
+    *  printPSIHBInfo
+    *  printLSIInfo
+    * @return errlHndl_t on error.
+    */
+    errlHndl_t printInterruptInfo();
+
+    /**
      * Un register a message queue from the interrupt handler
      * @param[in] i_type the type of interrupt (ISN value)
      * @return The message queue that was unregistered with i_type
@@ -241,6 +262,13 @@ namespace INTR
      * drain     */
     void drainQueue(void);
 
+    /**
+    * Convert the esb state to a human readable string
+    * @param[in] i_esbState  ESB state as hex
+    * @return human readable string, "INVALID" if cannot translate
+    */
+    void esbStateToString(uint64_t i_esbState, const char** o_esbStateString);
+
 };
 
 #endif
diff --git a/src/include/usr/intr/intr_reasoncodes.H b/src/include/usr/intr/intr_reasoncodes.H
index 4dc3cc496..511008cd9 100644
--- a/src/include/usr/intr/intr_reasoncodes.H
+++ b/src/include/usr/intr/intr_reasoncodes.H
@@ -53,6 +53,7 @@ namespace INTR
         MOD_INTRRP_RESETINTUNIT      = 0x11, /**< intrrp.C : IntrRp::resetIntUnit */
         MOD_INTRRP_XIVE_SENDEOI      = 0x12,
         MOD_INTRRP_IPC               = 0x13,
+        MOD_INTR_DUMP                = 0x14, /**< intrrp.C : INTR::printInterruptInfo */
     };
 
     enum IntrReasonCode