Add msgsync to doorbell wakeup logic to avoid weak consistency bug

POWER9 added a new sync mode called 'msgsync' that is required
to avoid weak consistency issues when you are using doorbell
(msgsnd) functions.

See POWER ISA Section 5.9.2 for details, excerpt here:
The ordering done by sync (and ptesync) provides
the appearance of "causality" across a sequence of
msgsnd instructions, as in the following example.
"msgsnd->T1" means "msgsnd instruction target-
ting thread T1". "<DHDI 0>" means "occurrence of
Directed Hypervisor Doorbell interrupt caused by
msgsnd executed on T0". On T0, register r1 is
assumed to contain the value 1.
  T0           T1           T2
  std r1,X     <DHDI 0>     <DHDI 1>
  sync         msgsnd->T2   msgsync
  msgsnd->T1                ld r1,X
In this example, T2's load from X must return 1.

The change here adds the msgsync call to the code that executes
any time we handle a doorbell interrupt.  In addition there is a
POWER9 DD2 errata that indicates we also require a lwsync to
ensure consistency.

Change-Id: Ib0f3571926d71efcbffa205093278e2a1d58df85
CQ: SW454611
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/70648
Reviewed-by: Nicholas E. Bofferding <bofferdn@us.ibm.com>
Reviewed-by: Dean Sanner <dsanner@us.ibm.com>
Tested-by: Jenkins Server <pfd-jenkins+hostboot@us.ibm.com>
Tested-by: Jenkins OP Build CI <op-jenkins+hostboot@us.ibm.com>
Tested-by: FSP CI Jenkins <fsp-CI-jenkins+hostboot@us.ibm.com>
Tested-by: Jenkins OP HW <op-hw-jenkins+hostboot@us.ibm.com>
Reviewed-by: William G. Hoffa <wghoffa@us.ibm.com>

1 files changed, 15 insertions, 1 deletions

diff --git a/src/include/arch/ppc.H b/src/include/arch/ppc.H
index 4b07fd107..2cfd95e9b 100644
--- a/src/include/arch/ppc.H
+++ b/src/include/arch/ppc.H
@@ -5,7 +5,7 @@
 /*                                                                        */
 /* OpenPOWER HostBoot Project                                             */
 /*                                                                        */
-/* Contributors Listed Below - COPYRIGHT 2011,2018                        */
+/* Contributors Listed Below - COPYRIGHT 2011,2019                        */
 /* [+] Google Inc.                                                        */
 /* [+] International Business Machines Corp.                              */
 /*                                                                        */
@@ -226,6 +226,20 @@ inline void eieio()
 }
 
 ALWAYS_INLINE
+inline void msgsync()
+{
+    // See POWER ISA 5.9.2 for details
+    //asm volatile("msgsync" ::: "memory");
+    asm volatile(".long 0x7C0006EC"); // Our GCC doesn't support 'msgsync' yet
+    //  [011111 ///// ///// ///// 11011 10110/]
+
+    // There is a P9 DD2 workaround that a lwsync is also required
+    //  after a msgsync
+    asm volatile("lwsync" ::: "memory");
+}
+
+
+ALWAYS_INLINE
 inline uint64_t getHMER()
 {
     register uint64_t hmer = 0;