Initial merge of some of the iOS 8 / Mac OS X Yosemite specific

lldb support.  I'll be doing more testing & cleanup but I wanted to
get the initial checkin done.

This adds a new SBExpressionOptions::SetLanguage API for selecting a
language of an expression.

I added adds a new SBThread::GetInfoItemByPathString for retriving
information about a thread from that thread's StructuredData.

I added a new StructuredData class for representing
key-value/array/dictionary information (e.g. JSON formatted data).
Helper functions to read JSON and create a StructuredData object,
and to print a StructuredData object in JSON format are included.

A few Cocoa / Cocoa Touch data formatters were updated by Enrico
to track changes in iOS 8 / Yosemite.

Before we query a thread's extended information, the system runtime may 
provide hints to the remote debug stub that it will use to retrieve values
out of runtime structures.  I added a new SystemRuntime method 
AddThreadExtendedInfoPacketHints which allows the SystemRuntime to add 
key-value type data to the initial request that we send to the remote stub.

The thread-format formatter string can now retrieve values out of a thread's
extended info structured data.  The default thread-format string picks up
two of these - thread.info.activity.name and thread.info.trace_messages.

I added a new "jThreadExtendedInfo" packet in debugserver; I will
add documentation to the lldb-gdb-remote.txt doc soon.  It accepts
JSON formatted arguments (most importantly, "thread":threadnum) and
it returns a variety of information regarding the thread to lldb
in JSON format.  This JSON return is scanned into a StructuredData
object that is associated with the thread; UI layers can query the
thread's StructuredData to see if key-values are present, and if
so, show them to the user.  These key-values are likely to be
specific to different targets with some commonality among many
targets.  For instance, many targets will be able to advertise the
pthread_t value for a thread.

I added an initial rough cut of "thread info" command which will print
the information about a thread from the jThreadExtendedInfo result.
I need to do more work to make this format reasonably.

Han Ming added calls into the pmenergy and pmsample libraries if
debugserver is run on Mac OS X Yosemite to get information about the
inferior's power use.

I added support to debugserver for gathering the Genealogy information
about threads, if it exists, and returning it in the jThreadExtendedInfo
JSON result.

llvm-svn: 210874

3 files changed, 531 insertions, 0 deletions

diff --git a/lldb/source/Core/CMakeLists.txt b/lldb/source/Core/CMakeLists.txt
index 16730ac46b2..709433f96ff 100644
--- a/lldb/source/Core/CMakeLists.txt
+++ b/lldb/source/Core/CMakeLists.txt
@@ -52,6 +52,7 @@ add_lldb_library(lldbCore
   StreamGDBRemote.cpp
   StreamString.cpp
   StringList.cpp
+  StructuredData.cpp
   Timer.cpp
   UserID.cpp
   UserSettingsController.cpp
diff --git a/lldb/source/Core/Debugger.cpp b/lldb/source/Core/Debugger.cpp
index 4a159145c6e..bb982f3e4db 100644
--- a/lldb/source/Core/Debugger.cpp
+++ b/lldb/source/Core/Debugger.cpp
@@ -15,6 +15,7 @@
 
 #include "clang/AST/DeclCXX.h"
 #include "clang/AST/Type.h"
+#include "llvm/ADT/StringRef.h"
 
 #include "lldb/lldb-private.h"
 #include "lldb/Core/ConnectionFileDescriptor.h"
@@ -26,6 +27,7 @@
 #include "lldb/Core/StreamCallback.h"
 #include "lldb/Core/StreamFile.h"
 #include "lldb/Core/StreamString.h"
+#include "lldb/Core/StructuredData.h"
 #include "lldb/Core/Timer.h"
 #include "lldb/Core/ValueObject.h"
 #include "lldb/Core/ValueObjectVariable.h"
@@ -105,6 +107,8 @@ g_language_enumerators[] =
     FILE_AND_LINE\
     "{, name = '${thread.name}'}"\
     "{, queue = '${thread.queue}'}"\
+    "{, activity = '${thread.info.activity.name}'}" \
+    "{, ${thread.info.trace_messages} messages}" \
     "{, stop reason = ${thread.stop-reason}}"\
     "{\\nReturn value: ${thread.return-value}}"\
     "\\n"
@@ -1430,6 +1434,96 @@ IsTokenWithFormat(const char *var_name_begin, const char *var, std::string &form
     return false;
 }
 
+// Find information for the "thread.info.*" specifiers in a format string
+static bool
+FormatThreadExtendedInfoRecurse
+(
+    const char *var_name_begin,
+    StructuredData::ObjectSP thread_info_dictionary,
+    const SymbolContext *sc,
+    const ExecutionContext *exe_ctx,
+    Stream &s
+)
+{
+    bool var_success = false;
+    std::string token_format;
+
+    llvm::StringRef var_name(var_name_begin);
+    size_t percent_idx = var_name.find('%');
+    size_t close_curly_idx = var_name.find('}');
+    llvm::StringRef path = var_name;
+    llvm::StringRef formatter = var_name;
+
+    // 'path' will be the dot separated list of objects to transverse up until we hit
+    // a close curly brace, a percent sign, or an end of string.
+    if (percent_idx != llvm::StringRef::npos || close_curly_idx != llvm::StringRef::npos)
+    {
+        if (percent_idx != llvm::StringRef::npos && close_curly_idx != llvm::StringRef::npos)
+        {
+            if (percent_idx < close_curly_idx)
+            {
+                path = var_name.slice(0, percent_idx);
+                formatter = var_name.substr (percent_idx);
+            }
+            else
+            {
+                path = var_name.slice(0, close_curly_idx);
+                formatter = var_name.substr (close_curly_idx);
+            }
+        }
+        else if (percent_idx != llvm::StringRef::npos)
+        {
+            path = var_name.slice(0, percent_idx);
+            formatter = var_name.substr (percent_idx);
+        }
+        else if (close_curly_idx != llvm::StringRef::npos)
+        {
+            path = var_name.slice(0, close_curly_idx);
+            formatter = var_name.substr (close_curly_idx);
+        }
+    }
+
+    StructuredData::ObjectSP value = thread_info_dictionary->GetObjectForDotSeparatedPath (path);
+
+    if (value.get())
+    {
+        if (value->GetType() == StructuredData::Type::eTypeInteger)
+        {
+            if (IsTokenWithFormat (formatter.str().c_str(), "", token_format, "0x%4.4" PRIx64, exe_ctx, sc))
+            {
+                s.Printf(token_format.c_str(), value->GetAsInteger()->GetValue());
+                var_success = true;
+            }
+        }
+        else if (value->GetType() == StructuredData::Type::eTypeFloat)
+        {
+            s.Printf ("%f", value->GetAsFloat()->GetValue());
+            var_success = true;
+        }
+        else if (value->GetType() == StructuredData::Type::eTypeString)
+        {
+            s.Printf("%s", value->GetAsString()->GetValue().c_str());
+            var_success = true;
+        }
+        else if (value->GetType() == StructuredData::Type::eTypeArray)
+        {
+            if (value->GetAsArray()->GetSize() > 0)
+            {
+                s.Printf ("%zu", value->GetAsArray()->GetSize());
+                var_success = true;
+            }
+        }
+        else if (value->GetType() == StructuredData::Type::eTypeDictionary)
+        {
+            s.Printf ("%zu", value->GetAsDictionary()->GetKeys()->GetAsArray()->GetSize());
+            var_success = true;
+        }
+    }
+
+    return var_success;
+}
+
+
 static bool
 FormatPromptRecurse
 (
@@ -1969,6 +2063,15 @@ FormatPromptRecurse
                                             if (RunScriptFormatKeyword (s, script_interpreter, thread, script_name))
                                                 var_success = true;
                                         }
+                                        else if (IsToken (var_name_begin, "info."))
+                                        {
+                                            var_name_begin += ::strlen("info.");
+                                            StructuredData::ObjectSP object_sp = thread->GetExtendedInfo();
+                                            if (object_sp && object_sp->GetType() == StructuredData::Type::eTypeDictionary)
+                                            {
+                                                var_success = FormatThreadExtendedInfoRecurse (var_name_begin, object_sp, sc, exe_ctx, s);
+                                            }
+                                        }
                                     }
                                 }
                             }
diff --git a/lldb/source/Core/StructuredData.cpp b/lldb/source/Core/StructuredData.cpp
new file mode 100644
index 00000000000..8bbc8292b02
--- /dev/null
+++ b/lldb/source/Core/StructuredData.cpp
@@ -0,0 +1,427 @@
+//===---------------------StructuredData.cpp ---------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "lldb/Core/StructuredData.h"
+
+#include <errno.h>
+#include <stdlib.h>
+#include <inttypes.h>
+
+using namespace lldb_private;
+
+
+static StructuredData::ObjectSP read_json_object (const char **ch);
+static StructuredData::ObjectSP read_json_array (const char **ch);
+
+static StructuredData::ObjectSP
+read_json_number (const char **ch)
+{
+    StructuredData::ObjectSP object_sp;
+    while (isspace (**ch))
+        (*ch)++;
+    const char *start_of_number = *ch;
+    bool is_integer = true;
+    bool is_float = false;
+    while (isdigit(**ch) || **ch == '-' || **ch == '.' || **ch == '+' || **ch == 'e' || **ch == 'E')
+    {
+        if (isdigit(**ch) == false && **ch != '-')
+        {
+            is_integer = false;
+            is_float = true;
+        }
+        (*ch)++;
+    }
+    while (isspace (**ch))
+        (*ch)++;
+    if (**ch == ',' || **ch == ']' || **ch == '}')
+    {
+        if (is_integer)
+        {
+            errno = 0;
+            uint64_t val = strtoul (start_of_number, NULL, 10);
+            if (errno == 0)
+            {
+                object_sp.reset(new StructuredData::Integer());
+                object_sp->GetAsInteger()->SetValue (val);
+            }
+        }
+        if (is_float)
+        {
+            char *end_of_number = NULL;
+            errno = 0;
+            double val = strtod (start_of_number, &end_of_number);
+            if (errno == 0 && end_of_number != start_of_number && end_of_number != NULL)
+            {
+                object_sp.reset(new StructuredData::Float());
+                object_sp->GetAsFloat()->SetValue (val);
+            }
+        }
+    }
+    return object_sp;
+}
+
+static std::string
+read_json_string (const char **ch)
+{
+    std::string string;
+    if (**ch == '"')
+    {
+        (*ch)++;
+        while (**ch != '\0')
+        {
+            if (**ch == '"')
+            {
+                (*ch)++;
+                while (isspace (**ch))
+                    (*ch)++;
+                break;
+            }
+            else if (**ch == '\\')
+            {
+                switch (**ch)
+                {
+                    case '"':
+                        string.push_back('"');
+                        *ch += 2;
+                        break;
+                    case '\\':
+                        string.push_back('\\');
+                        *ch += 2;
+                        break;
+                    case '/':
+                        string.push_back('/');
+                        *ch += 2;
+                        break;
+                    case 'b':
+                        string.push_back('\b');
+                        *ch += 2;
+                        break;
+                    case 'f':
+                        string.push_back('\f');
+                        *ch += 2;
+                        break;
+                    case 'n':
+                        string.push_back('\n');
+                        *ch += 2;
+                        break;
+                    case 'r':
+                        string.push_back('\r');
+                        *ch += 2;
+                        break;
+                    case 't':
+                        string.push_back('\t');
+                        *ch += 2;
+                        break;
+                    case 'u':
+                        // FIXME handle four-hex-digits 
+                        *ch += 10;
+                        break;
+                    default:
+                        *ch += 1;
+                }
+            }
+            else
+            {
+                string.push_back (**ch);
+            }
+            (*ch)++;
+        }
+    }
+    return string;
+}
+
+static StructuredData::ObjectSP
+read_json_value (const char **ch)
+{
+    StructuredData::ObjectSP object_sp;
+    while (isspace (**ch))
+        (*ch)++;
+
+    if (**ch == '{')
+    {
+        object_sp = read_json_object (ch);
+    }
+    else if (**ch == '[')
+    {
+        object_sp = read_json_array (ch);
+    }
+    else if (**ch == '"')
+    {
+        std::string string = read_json_string (ch);
+        object_sp.reset(new StructuredData::String());
+        object_sp->GetAsString()->SetValue(string);
+    }
+    else
+    {
+        if (strncmp (*ch, "true", 4) == 0)
+        {
+            object_sp.reset(new StructuredData::Boolean());
+            object_sp->GetAsBoolean()->SetValue(true);
+            *ch += 4;
+        }
+        else if (strncmp (*ch, "false", 5) == 0)
+        {
+            object_sp.reset(new StructuredData::Boolean());
+            object_sp->GetAsBoolean()->SetValue(false);
+            *ch += 5;
+        }
+        else if (strncmp (*ch, "null", 4) == 0)
+        {
+            object_sp.reset(new StructuredData::Null());
+            *ch += 4;
+        }
+        else
+        {
+            object_sp = read_json_number (ch);
+        }
+    }
+    return object_sp;
+}
+
+static StructuredData::ObjectSP
+read_json_array (const char **ch)
+{
+    StructuredData::ObjectSP object_sp;
+    if (**ch == '[')
+    {
+        (*ch)++;
+        while (isspace (**ch))
+            (*ch)++;
+
+        bool first_value = true;
+        while (**ch != '\0' && (first_value || **ch == ','))
+        {
+            if (**ch == ',')
+                (*ch)++;
+            first_value = false;
+            while (isspace (**ch))
+                (*ch)++;
+            lldb_private::StructuredData::ObjectSP value_sp = read_json_value (ch);
+            if (value_sp)
+            {
+                if (object_sp.get() == NULL)
+                {
+                    object_sp.reset(new StructuredData::Array());
+                }
+                object_sp->GetAsArray()->Push (value_sp);
+            }
+            while (isspace (**ch))
+                (*ch)++;
+        }
+        if (**ch == ']')
+        {
+            // FIXME should throw an error if we don't see a } to close out the JSON object
+            (*ch)++;
+            while (isspace (**ch))
+                (*ch)++;
+        }
+    }
+    return object_sp;
+}
+
+static StructuredData::ObjectSP
+read_json_object (const char **ch)
+{
+    StructuredData::ObjectSP object_sp;
+    if (**ch == '{')
+    {
+        (*ch)++;
+        while (isspace (**ch))
+            (*ch)++;
+        bool first_pair = true;
+        while (**ch != '\0' && (first_pair || **ch == ','))
+        {
+            first_pair = false;
+            if (**ch == ',')
+                (*ch)++;
+            while (isspace (**ch))
+                (*ch)++;
+            if (**ch != '"')
+                break;
+            std::string key_string = read_json_string (ch);
+            while (isspace (**ch))
+                (*ch)++;
+            if (key_string.size() > 0 && **ch == ':')
+            {
+                (*ch)++;
+                while (isspace (**ch))
+                    (*ch)++;
+                lldb_private::StructuredData::ObjectSP value_sp = read_json_value (ch);
+                if (value_sp.get())
+                {
+                    if (object_sp.get() == NULL)
+                    {
+                        object_sp.reset(new StructuredData::Dictionary());
+                    }
+                    object_sp->GetAsDictionary()->AddItem (key_string.c_str(), value_sp);
+                }
+            }
+            while (isspace (**ch))
+                (*ch)++;
+        }
+        if (**ch == '}')
+        {
+            // FIXME should throw an error if we don't see a } to close out the JSON object
+            (*ch)++;
+            while (isspace (**ch))
+                (*ch)++;
+        }
+    }
+    return object_sp;
+}
+
+
+StructuredData::ObjectSP
+StructuredData::ParseJSON (std::string json_text)
+{
+    StructuredData::ObjectSP object_sp;
+    const size_t json_text_size = json_text.size();
+    if (json_text_size > 0)
+    {
+        const char *start_of_json_text = json_text.c_str();
+        const char *c = json_text.c_str();
+        while (*c != '\0' && c - start_of_json_text <= json_text_size)
+        {
+            while (isspace (*c) && c - start_of_json_text < json_text_size)
+                c++;
+            if (*c == '{')
+            {
+                object_sp = read_json_object (&c);
+            }
+            else
+            {
+                // We have bad characters here, this is likely an illegal JSON string.
+                return object_sp;
+            }
+        }
+    }
+    return object_sp;
+}
+
+StructuredData::ObjectSP
+StructuredData::Object::GetObjectForDotSeparatedPath (llvm::StringRef path)
+{
+    if (this->GetType() == Type::eTypeDictionary)
+    {
+        std::pair<llvm::StringRef, llvm::StringRef> match = path.split('.');
+        std::string key = match.first.str();
+        ObjectSP value = this->GetAsDictionary()->GetValueForKey (key.c_str());
+        if (value.get())
+        {
+            // Do we have additional words to descend?  If not, return the
+            // value we're at right now.
+            if (match.second.empty())
+            {
+                return value;
+            }
+            else
+            {
+                return value->GetObjectForDotSeparatedPath (match.second);
+            }
+        }
+        return ObjectSP();
+    }
+
+    if (this->GetType() == Type::eTypeArray)
+    {
+        std::pair<llvm::StringRef, llvm::StringRef> match = path.split('[');
+        if (match.second.size() == 0)
+        {
+            return this->shared_from_this();
+        }
+        errno = 0;
+        uint64_t val = strtoul (match.second.str().c_str(), NULL, 10);
+        if (errno == 0)
+        {
+            return this->GetAsArray()->GetItemAtIndex(val);
+        }
+        return ObjectSP();
+    }
+
+    return this->shared_from_this();
+}
+
+void
+StructuredData::Array::Dump (Stream &s) const
+{
+    s << "[";
+    const size_t arrsize = m_items.size();
+    for (size_t i = 0; i < arrsize; ++i)
+    {
+        m_items[i]->Dump(s);
+        if (i + 1 < arrsize)
+            s << ",";
+    }
+    s << "]";
+}
+
+void
+StructuredData::Integer::Dump (Stream &s) const
+{
+    s.Printf ("%" PRIu64, m_value);
+}
+
+
+void
+StructuredData::Float::Dump (Stream &s) const
+{
+    s.Printf ("%lf", m_value);
+}
+
+void
+StructuredData::Boolean::Dump (Stream &s) const
+{
+    if (m_value == true)
+        s.PutCString ("true");
+    else
+        s.PutCString ("false");
+}
+
+
+void
+StructuredData::String::Dump (Stream &s) const
+{
+    std::string quoted;
+    const size_t strsize = m_value.size();
+    for (size_t i = 0; i < strsize ; ++i)
+    {
+        char ch = m_value[i];
+        if (ch == '"')
+            quoted.push_back ('\\');
+        quoted.push_back (ch);
+    }
+    s.Printf ("\"%s\"", quoted.c_str());
+}
+
+void
+StructuredData::Dictionary::Dump (Stream &s) const
+{
+    bool have_printed_one_elem = false;
+    s << "{";
+    for (collection::const_iterator iter = m_dict.begin(); iter != m_dict.end(); ++iter)
+    {
+        if (have_printed_one_elem == false)
+        {
+            have_printed_one_elem = true;
+        }
+        else
+        {
+            s << ",";
+        }
+        s << "\"" << iter->first.AsCString() << "\":";
+        iter->second->Dump(s);
+    }
+    s << "}";
+}
+
+void
+StructuredData::Null::Dump (Stream &s) const
+{
+    s << "null";
+}