Save HRMOR in mbox scratch reg for IPC messaging

In order to know where the IPC message of a given node is,
we save off the HRMOR of every node in a hw register.
Originally, we were saving this information in the core
scratch register. Since, the core scratch registers are
wiped off when the cores go into the winkle state, therefore,
we were writing to the register after we come out of winkle.
But, at that point, we ran into race conditions because
other nodes could be ahead and try to access the register
on a node that is not exactly out of winkle yet.

This fixes the problems by using the mbox scratch register
rather than the core scratch register because they are saved
off even when the cores go into winkle state. Because the
registers are preseved, we can set the value prior to
cores coming out of winkle, so, we don't run into the race
condition where one node is trying to read a value prior to
the other one writing the value.

Change-Id: I822bfc8defe09cbb418edc5f36a99b7cd41eec88
CQ:SW435271
Reviewed-on: http://ralgit01.raleigh.ibm.com/gerrit1/61093
Tested-by: Jenkins Server <pfd-jenkins+hostboot@us.ibm.com>
Tested-by: Jenkins OP Build CI <op-jenkins+hostboot@us.ibm.com>
Tested-by: Jenkins OP HW <op-hw-jenkins+hostboot@us.ibm.com>
Tested-by: FSP CI Jenkins <fsp-CI-jenkins+hostboot@us.ibm.com>
Reviewed-by: Daniel M. Crowell <dcrowell@us.ibm.com>

1 files changed, 5 insertions, 11 deletions

diff --git a/src/usr/initservice/istepdispatcher/istepdispatcher.C b/src/usr/initservice/istepdispatcher/istepdispatcher.C
index 47d0ebc76..e4037e860 100644
--- a/src/usr/initservice/istepdispatcher/istepdispatcher.C
+++ b/src/usr/initservice/istepdispatcher/istepdispatcher.C
@@ -2139,6 +2139,11 @@ void IStepDispatcher::handleProcFabIovalidMsg(msg_t * & io_pMsg)
             msg_respond(iv_msgQ, io_pMsg);
             io_pMsg = NULL;
 
+            //Setup for IPC messages to continue after we come out of winkle
+            //This stores the location of IPC messages of current drawer in the mbox scratch
+            //register.
+            IPC::IpcSp::distributeLocalNodeAddr();
+
             // call to suspend the MBOX so that all messages are flushed
             err = MBOX::suspend();
             if (err)
@@ -2182,17 +2187,6 @@ void IStepDispatcher::handleProcFabIovalidMsg(msg_t * & io_pMsg)
                    "Returned from cpu_all_winkle." );
             }
 
-            //Temporary hack to sleep for 15 seconds to avoid race condition
-            //where other nodes are trying to read the core scratch registers
-            //while this node might not be out of winkle. To make it worse,
-            //if we fail in this case, we don't TI gracefully.
-            nanosleep(15,0);
-
-            // identify IPC msg address to remote node(s)
-            // this must be after the winkle call because the data is stored
-            // in a core scom which will get lost
-            IPC::IpcSp::distributeLocalNodeAddr();
-
             err = MBOX::resume();
             if (err)
             {