Use __builtin_isnan/isinf/isfinite in complex

The libc-provided isnan/isinf/isfinite macro implementations are specifically
designed to function correctly, even in the presence of -ffast-math (or, more
specifically, -ffinite-math-only). As such, on most implementation, these
either always turn into external function calls (e.g. glibc) or are
specifically function calls when FINITE_MATH_ONLY is defined (e.g. Darwin).

Our implementation of complex arithmetic makes heavy use of isnan/isinf/isfinite
to deal with corner cases involving non-finite quantities. This was problematic
in two respects:

  1. On systems where these are always function calls (e.g. Linux/glibc), there was a
     performance penalty
  2. When compiling with -ffast-math, there was a significant performance
     penalty (in fact, on Darwin and systems with similar implementations, the code
     may in fact be slower than not using -ffast-math, because the inline
     definitions provided by libc become unavailable to prevent the checks from
     being optimized out).

Eliding these inf/nan checks in -ffast-math mode is consistent with what
happens with libstdc++, and in my experience, what users expect. This is
critical to getting high-performance code when using complex<T>. This change
replaces uses of those functions on basic floating-point types with calls to
__builtin_isnan/isinf/isfinite, which Clang will always expand inline. When
using -ffast-math (or -ffinite-math-only), the optimizer will remove the checks
as expected.

Differential Revision: https://reviews.llvm.org/D18639

llvm-svn: 283051

1 files changed, 60 insertions, 0 deletions

diff --git a/libcxx/include/cmath b/libcxx/include/cmath
index a1d4ee01d73..ec28e0e456e 100644
--- a/libcxx/include/cmath
+++ b/libcxx/include/cmath
@@ -558,6 +558,66 @@ hypot(_A1 __lcpp_x, _A2 __lcpp_y, _A3 __lcpp_z) _NOEXCEPT
 }
 #endif
 
+template <class _A1>
+_LIBCPP_ALWAYS_INLINE
+typename enable_if<is_floating_point<_A1>::value, bool>::type
+__libcpp_isnan(_A1 __lcpp_x) _NOEXCEPT
+{
+#if __has_builtin(__builtin_isnan)
+    return __builtin_isnan(__lcpp_x);
+#else
+    return isnan(__lcpp_x);
+#endif
+}
+
+template <class _A1>
+_LIBCPP_ALWAYS_INLINE
+typename enable_if<!is_floating_point<_A1>::value, bool>::type
+__libcpp_isnan(_A1 __lcpp_x) _NOEXCEPT
+{
+    return isnan(__lcpp_x);
+}
+
+template <class _A1>
+_LIBCPP_ALWAYS_INLINE
+typename enable_if<is_floating_point<_A1>::value, bool>::type
+__libcpp_isinf(_A1 __lcpp_x) _NOEXCEPT
+{
+#if __has_builtin(__builtin_isinf)
+    return __builtin_isinf(__lcpp_x);
+#else
+    return isinf(__lcpp_x);
+#endif
+}
+
+template <class _A1>
+_LIBCPP_ALWAYS_INLINE
+typename enable_if<!is_floating_point<_A1>::value, bool>::type
+__libcpp_isinf(_A1 __lcpp_x) _NOEXCEPT
+{
+    return isinf(__lcpp_x);
+}
+
+template <class _A1>
+_LIBCPP_ALWAYS_INLINE
+typename enable_if<is_floating_point<_A1>::value, bool>::type
+__libcpp_isfinite(_A1 __lcpp_x) _NOEXCEPT
+{
+#if __has_builtin(__builtin_isfinite)
+    return __builtin_isfinite(__lcpp_x);
+#else
+    return isfinite(__lcpp_x);
+#endif
+}
+
+template <class _A1>
+_LIBCPP_ALWAYS_INLINE
+typename enable_if<!is_floating_point<_A1>::value, bool>::type
+__libcpp_isfinite(_A1 __lcpp_x) _NOEXCEPT
+{
+    return isfinite(__lcpp_x);
+}
+
 _LIBCPP_END_NAMESPACE_STD
 
 #endif  // _LIBCPP_CMATH