[NFC] Darwin llgs support from Week of Code

This code represents the Week of Code work I did on bringing up
lldb-server LLGS support for Darwin.  It does not include the
Xcode project changes needed, as we don't want to throw that switch
until more support is implemented (i.e. this change is inert, no
build systems use it yet.  I've verified on Ubuntu 16.04, macOS
Xcode and macOS cmake builds).

This change does some minimal refactoring of code that is shared
with the Linux LLGS portion, moving it from NativeProcessLinux into
NativeProcessProtocol.  That code is also used by NativeProcessDarwin.

Current state on Darwin:
* Process launching is implemented.  (Attach is not).
  Launching on devices has not yet been tested (FBS/BKS might
  need a bit of work).
* Inferior waitpid monitoring and communication of exit status
  via MainLoop callback is implemented.
* Memory read/write, breakpoints, thread register context, etc.
  are not yet implemented.  This impacts process stop/resume, as
  the initial launch suspended immediately starts the process
  up and running because it doesn't know it is supposed to remain
  stopped.
* I implemented the equivalent of MachThreadList as
  NativeThreadListDarwin, in anticipation that we might want to
  factor out common parts into NativeThreadList{Protocol} and share
  some code here.  After writing it, though, the fallout from merging
  Mach Task/Process into a single concept plus some other minor
  changes makes the whole NativeThreadListDarwin concept nothing more
  than dead weight.  I am likely going to get rid of this class and
  just manage it directly in NativeProcessDarwin, much like I did
  for NativeProcessLinux.
* There is a stub-out call for starting a STDIO thread.  That will
  go away and adopt the MainLoop pselect-based IOObject reading.

I am developing the fully-integrated changes in the following repo,
which contains the necessary Xcode bits and the glue that enables
lldb-debugserver on a macOS system:

  https://github.com/tfiala/lldb/tree/llgs-darwin

This change also breaks out a few of the lldb-server tests into
their own directory, and adds some $qHostInfo tests (not sure why
I didn't write those tests back when I initially implemented that
on the Linux side).

llvm-svn: 280604

7 files changed, 300 insertions, 96 deletions

diff --git a/lldb/packages/Python/lldbsuite/test/tools/lldb-server/TestGdbRemoteProcessInfo.py b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/TestGdbRemoteProcessInfo.py
index f26b6204361..fd89ddfa28f 100644
--- a/lldb/packages/Python/lldbsuite/test/tools/lldb-server/TestGdbRemoteProcessInfo.py
+++ b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/TestGdbRemoteProcessInfo.py
@@ -155,6 +155,13 @@ class TestGdbRemoteProcessInfo(gdbremote_testcase.GdbRemoteTestCaseBase):
         self.build()
         self.qProcessInfo_contains_keys(set(['cputype', 'cpusubtype']))
 
+    @skipUnlessDarwin
+    @llgs_test
+    def test_qProcessInfo_contains_cputype_cpusubtype_llgs_darwin(self):
+        self.init_llgs_test()
+        self.build()
+        self.qProcessInfo_contains_keys(set(['cputype', 'cpusubtype']))
+
     @skipUnlessPlatform(["linux"])
     @llgs_test
     def test_qProcessInfo_contains_triple_llgs_linux(self):
@@ -172,6 +179,16 @@ class TestGdbRemoteProcessInfo(gdbremote_testcase.GdbRemoteTestCaseBase):
         # for the remote Host and Process.
         self.qProcessInfo_does_not_contain_keys(set(['triple']))
 
+    @skipUnlessDarwin
+    @llgs_test
+    def test_qProcessInfo_does_not_contain_triple_llgs_darwin(self):
+        self.init_llgs_test()
+        self.build()
+        # We don't expect to see triple on darwin.  If we do, we'll prefer triple
+        # to cputype/cpusubtype and skip some darwin-based ProcessGDBRemote ArchSpec setup
+        # for the remote Host and Process.
+        self.qProcessInfo_does_not_contain_keys(set(['triple']))
+
     @skipUnlessPlatform(["linux"])
     @llgs_test
     def test_qProcessInfo_does_not_contain_cputype_cpusubtype_llgs_linux(self):
diff --git a/lldb/packages/Python/lldbsuite/test/tools/lldb-server/TestLldbGdbServer.py b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/TestLldbGdbServer.py
index b60d08dc136..9cc20c11f74 100644
--- a/lldb/packages/Python/lldbsuite/test/tools/lldb-server/TestLldbGdbServer.py
+++ b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/TestLldbGdbServer.py
@@ -97,101 +97,6 @@ class LldbGdbServerTestCase(gdbremote_testcase.GdbRemoteTestCaseBase):
         self.init_llgs_test()
         self.list_threads_in_stop_reply_supported()
 
-    def install_and_create_launch_args(self):
-        exe_path = os.path.abspath('a.out')
-        if not lldb.remote_platform:
-            return [exe_path]
-        remote_path = lldbutil.append_to_process_working_directory(os.path.basename(exe_path))
-        remote_file_spec = lldb.SBFileSpec(remote_path, False)
-        err = lldb.remote_platform.Install(lldb.SBFileSpec(exe_path, True), remote_file_spec)
-        if err.Fail():
-            raise Exception("remote_platform.Install('%s', '%s') failed: %s" % (exe_path, remote_path, err))
-        return [remote_path]
-
-    def start_inferior(self):
-        launch_args = self.install_and_create_launch_args()
-
-        server = self.connect_to_debug_monitor()
-        self.assertIsNotNone(server)
-
-        self.add_no_ack_remote_stream()
-        self.test_sequence.add_log_lines(
-            ["read packet: %s" % lldbgdbserverutils.build_gdbremote_A_packet(launch_args),
-             "send packet: $OK#9a"],
-            True)
-        self.expect_gdbremote_sequence()
-
-    @debugserver_test
-    def test_start_inferior_debugserver(self):
-        self.init_debugserver_test()
-        self.build()
-        self.start_inferior()
-
-    @llgs_test
-    def test_start_inferior_llgs(self):
-        self.init_llgs_test()
-        self.build()
-        self.start_inferior()
-
-    def inferior_exit_0(self):
-        launch_args = self.install_and_create_launch_args()
-
-        server = self.connect_to_debug_monitor()
-        self.assertIsNotNone(server)
-
-        self.add_no_ack_remote_stream()
-        self.add_verified_launch_packets(launch_args)
-        self.test_sequence.add_log_lines(
-            ["read packet: $vCont;c#a8",
-             "send packet: $W00#00"],
-            True)
-
-        self.expect_gdbremote_sequence()
-
-    @debugserver_test
-    def test_inferior_exit_0_debugserver(self):
-        self.init_debugserver_test()
-        self.build()
-        self.inferior_exit_0()
-
-    @llgs_test
-    def test_inferior_exit_0_llgs(self):
-        self.init_llgs_test()
-        self.build()
-        self.inferior_exit_0()
-
-    def inferior_exit_42(self):
-        launch_args = self.install_and_create_launch_args()
-
-        server = self.connect_to_debug_monitor()
-        self.assertIsNotNone(server)
-
-        RETVAL = 42
-
-        # build launch args
-        launch_args += ["retval:%d" % RETVAL]
-
-        self.add_no_ack_remote_stream()
-        self.add_verified_launch_packets(launch_args)
-        self.test_sequence.add_log_lines(
-            ["read packet: $vCont;c#a8",
-             "send packet: $W{0:02x}#00".format(RETVAL)],
-            True)
-
-        self.expect_gdbremote_sequence()
-
-    @debugserver_test
-    def test_inferior_exit_42_debugserver(self):
-        self.init_debugserver_test()
-        self.build()
-        self.inferior_exit_42()
-
-    @llgs_test
-    def test_inferior_exit_42_llgs(self):
-        self.init_llgs_test()
-        self.build()
-        self.inferior_exit_42()
-
     def c_packet_works(self):
         launch_args = self.install_and_create_launch_args()
 
diff --git a/lldb/packages/Python/lldbsuite/test/tools/lldb-server/exit-code/Makefile b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/exit-code/Makefile
new file mode 100644
index 00000000000..1370b53b5a6
--- /dev/null
+++ b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/exit-code/Makefile
@@ -0,0 +1,10 @@
+LEVEL = ../../../make
+
+VPATH = ..
+
+override CFLAGS_EXTRAS += -D__STDC_LIMIT_MACROS -D__STDC_FORMAT_MACROS
+ENABLE_THREADS := YES
+CXX_SOURCES := main.cpp
+MAKE_DSYM :=NO
+
+include $(LEVEL)/Makefile.rules
diff --git a/lldb/packages/Python/lldbsuite/test/tools/lldb-server/exit-code/TestGdbRemoteExitCode.py b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/exit-code/TestGdbRemoteExitCode.py
new file mode 100644
index 00000000000..e77f2b7acec
--- /dev/null
+++ b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/exit-code/TestGdbRemoteExitCode.py
@@ -0,0 +1,124 @@
+from __future__ import print_function
+
+# lldb test suite imports
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import TestBase
+
+# gdb-remote-specific imports
+import lldbgdbserverutils
+from gdbremote_testcase import GdbRemoteTestCaseBase
+
+
+class TestGdbRemoteExitCode(GdbRemoteTestCaseBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    FAILED_LAUNCH_CODE = "E08"
+
+    def get_launch_fail_reason(self):
+        self.reset_test_sequence()
+        self.test_sequence.add_log_lines(
+            ["read packet: $qLaunchSuccess#00"],
+            True)
+        self.test_sequence.add_log_lines(
+            [{"direction": "send", "regex": r"^\$(.+)#[0-9a-fA-F]{2}$",
+              "capture": {1: "launch_result"}}],
+            True)
+        context = self.expect_gdbremote_sequence()
+        self.assertIsNotNone(context)
+        return context.get("launch_result")[1:]
+
+    def start_inferior(self):
+        launch_args = self.install_and_create_launch_args()
+
+        server = self.connect_to_debug_monitor()
+        self.assertIsNotNone(server)
+
+        self.add_no_ack_remote_stream()
+        self.test_sequence.add_log_lines(
+            ["read packet: %s" % lldbgdbserverutils.build_gdbremote_A_packet(
+                launch_args)],
+            True)
+        self.test_sequence.add_log_lines(
+            [{"direction": "send", "regex": r"^\$(.+)#[0-9a-fA-F]{2}$",
+              "capture": {1: "A_result"}}],
+            True)
+        context = self.expect_gdbremote_sequence()
+        self.assertIsNotNone(context)
+
+        launch_result = context.get("A_result")
+        self.assertIsNotNone(launch_result)
+        if launch_result == self.FAILED_LAUNCH_CODE:
+            fail_reason = self.get_launch_fail_reason()
+            self.fail("failed to launch inferior: " + fail_reason)
+
+    @debugserver_test
+    def test_start_inferior_debugserver(self):
+        self.init_debugserver_test()
+        self.build()
+        self.start_inferior()
+
+    @llgs_test
+    def test_start_inferior_llgs(self):
+        self.init_llgs_test()
+        self.build()
+        self.start_inferior()
+
+    def inferior_exit_0(self):
+        launch_args = self.install_and_create_launch_args()
+
+        server = self.connect_to_debug_monitor()
+        self.assertIsNotNone(server)
+
+        self.add_no_ack_remote_stream()
+        self.add_verified_launch_packets(launch_args)
+        self.test_sequence.add_log_lines(
+            ["read packet: $vCont;c#a8",
+             "send packet: $W00#00"],
+            True)
+
+        self.expect_gdbremote_sequence()
+
+    @debugserver_test
+    def test_inferior_exit_0_debugserver(self):
+        self.init_debugserver_test()
+        self.build()
+        self.inferior_exit_0()
+
+    @llgs_test
+    def test_inferior_exit_0_llgs(self):
+        self.init_llgs_test()
+        self.build()
+        self.inferior_exit_0()
+
+    def inferior_exit_42(self):
+        launch_args = self.install_and_create_launch_args()
+
+        server = self.connect_to_debug_monitor()
+        self.assertIsNotNone(server)
+
+        RETVAL = 42
+
+        # build launch args
+        launch_args += ["retval:%d" % RETVAL]
+
+        self.add_no_ack_remote_stream()
+        self.add_verified_launch_packets(launch_args)
+        self.test_sequence.add_log_lines(
+            ["read packet: $vCont;c#a8",
+             "send packet: $W{0:02x}#00".format(RETVAL)],
+            True)
+
+        self.expect_gdbremote_sequence()
+
+    @debugserver_test
+    def test_inferior_exit_42_debugserver(self):
+        self.init_debugserver_test()
+        self.build()
+        self.inferior_exit_42()
+
+    @llgs_test
+    def test_inferior_exit_42_llgs(self):
+        self.init_llgs_test()
+        self.build()
+        self.inferior_exit_42()
diff --git a/lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py
index 4db630d50d1..12c1033cba1 100644
--- a/lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py
+++ b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py
@@ -133,7 +133,7 @@ class GdbRemoteTestCaseBase(TestBase):
             self.debug_monitor_extra_args.append("--log-file=" + log_file)
             self.debug_monitor_extra_args.append("--log-channels={}".format(":".join(lldbtest_config.channels)))
         else:
-            self.debug_monitor_extra_args = ["--log-file=" + self.log_file, "--log-flags=0x800000"]
+            self.debug_monitor_extra_args = ["--log-file=" + log_file, "--log-flags=0x800000"]
 
     def get_next_port(self):
         return 12000 + random.randint(0,3999)
@@ -1370,3 +1370,16 @@ class GdbRemoteTestCaseBase(TestBase):
     def maybe_strict_output_regex(self, regex):
         return '.*'+regex+'.*' if lldbplatformutil.hasChattyStderr(self) else '^'+regex+'$'
 
+    def install_and_create_launch_args(self):
+        exe_path = os.path.abspath('a.out')
+        if not lldb.remote_platform:
+            return [exe_path]
+        remote_path = lldbutil.append_to_process_working_directory(
+            os.path.basename(exe_path))
+        remote_file_spec = lldb.SBFileSpec(remote_path, False)
+        err = lldb.remote_platform.Install(lldb.SBFileSpec(exe_path, True),
+                                           remote_file_spec)
+        if err.Fail():
+            raise Exception("remote_platform.Install('%s', '%s') failed: %s" %
+                            (exe_path, remote_path, err))
+        return [remote_path]
diff --git a/lldb/packages/Python/lldbsuite/test/tools/lldb-server/host-info/Makefile b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/host-info/Makefile
new file mode 100644
index 00000000000..1370b53b5a6
--- /dev/null
+++ b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/host-info/Makefile
@@ -0,0 +1,10 @@
+LEVEL = ../../../make
+
+VPATH = ..
+
+override CFLAGS_EXTRAS += -D__STDC_LIMIT_MACROS -D__STDC_FORMAT_MACROS
+ENABLE_THREADS := YES
+CXX_SOURCES := main.cpp
+MAKE_DSYM :=NO
+
+include $(LEVEL)/Makefile.rules
diff --git a/lldb/packages/Python/lldbsuite/test/tools/lldb-server/host-info/TestGdbRemoteHostInfo.py b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/host-info/TestGdbRemoteHostInfo.py
new file mode 100644
index 00000000000..6502f03336a
--- /dev/null
+++ b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/host-info/TestGdbRemoteHostInfo.py
@@ -0,0 +1,125 @@
+from __future__ import print_function
+
+# lldb test suite imports
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import TestBase
+
+# gdb-remote-specific imports
+import lldbgdbserverutils
+from gdbremote_testcase import GdbRemoteTestCaseBase
+
+
+class TestGdbRemoteHostInfo(GdbRemoteTestCaseBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    KNOWN_HOST_INFO_KEYS = set([
+        "cputype",
+        "cpusubtype",
+        "distribution_id",
+        "endian",
+        "hostname",
+        "ostype",
+        "os_build",
+        "os_kernel",
+        "os_version",
+        "ptrsize",
+        "triple",
+        "vendor",
+        "watchpoint_exceptions_received"
+    ])
+
+    DARWIN_REQUIRED_HOST_INFO_KEYS = set([
+        "cputype",
+        "cpusubtype",
+        "endian",
+        "ostype",
+        "ptrsize",
+        "vendor",
+        "watchpoint_exceptions_received"
+    ])
+
+    def add_host_info_collection_packets(self):
+        self.test_sequence.add_log_lines(
+            ["read packet: $qHostInfo#9b",
+             {"direction": "send", "regex": r"^\$(.+)#[0-9a-fA-F]{2}$",
+              "capture": {1: "host_info_raw"}}],
+            True)
+
+    def parse_host_info_response(self, context):
+        # Ensure we have a host info response.
+        self.assertIsNotNone(context)
+        host_info_raw = context.get("host_info_raw")
+        self.assertIsNotNone(host_info_raw)
+
+        # Pull out key:value; pairs.
+        host_info_dict = {match.group(1): match.group(2)
+                          for match in re.finditer(r"([^:]+):([^;]+);",
+                                                   host_info_raw)}
+
+        import pprint
+        print("\nqHostInfo response:")
+        pprint.pprint(host_info_dict)
+
+        # Validate keys are known.
+        for (key, val) in list(host_info_dict.items()):
+            self.assertTrue(key in self.KNOWN_HOST_INFO_KEYS,
+                            "unknown qHostInfo key: " + key)
+            self.assertIsNotNone(val)
+
+        # Return the key:val pairs.
+        return host_info_dict
+
+    def get_qHostInfo_response(self):
+        # Launch the debug monitor stub, attaching to the inferior.
+        server = self.connect_to_debug_monitor()
+        self.assertIsNotNone(server)
+        self.add_no_ack_remote_stream()
+
+        # Request qHostInfo and get response
+        self.add_host_info_collection_packets()
+        context = self.expect_gdbremote_sequence()
+        self.assertIsNotNone(context)
+
+        # Parse qHostInfo response.
+        host_info = self.parse_host_info_response(context)
+        self.assertIsNotNone(host_info)
+        self.assertGreater(len(host_info), 0, "qHostInfo should have returned "
+                           "at least one key:val pair.")
+        return host_info
+
+    def validate_darwin_minimum_host_info_keys(self, host_info_dict):
+        self.assertIsNotNone(host_info_dict)
+        missing_keys = [key for key in self.DARWIN_REQUIRED_HOST_INFO_KEYS
+                        if key not in host_info_dict]
+        self.assertEquals(0, len(missing_keys),
+                          "qHostInfo is missing the following required "
+                          "keys: " + str(missing_keys))
+
+    @debugserver_test
+    def test_qHostInfo_returns_at_least_one_key_val_pair_debugserver(self):
+        self.init_debugserver_test()
+        self.build()
+        self.get_qHostInfo_response()
+
+    @llgs_test
+    def test_qHostInfo_returns_at_least_one_key_val_pair_llgs(self):
+        self.init_llgs_test()
+        self.build()
+        self.get_qHostInfo_response()
+
+    @skipUnlessDarwin
+    @debugserver_test
+    def test_qHostInfo_contains_darwin_required_keys_debugserver(self):
+        self.init_debugserver_test()
+        self.build()
+        host_info_dict = self.get_qHostInfo_response()
+        self.validate_darwin_minimum_host_info_keys(host_info_dict)
+
+    @skipUnlessDarwin
+    @llgs_test
+    def test_qHostInfo_contains_darwin_required_keys_llgs(self):
+        self.init_llgs_test()
+        self.build()
+        host_info_dict = self.get_qHostInfo_response()
+        self.validate_darwin_minimum_host_info_keys(host_info_dict)