summaryrefslogtreecommitdiffstats
path: root/clang/lib/CodeGen/CGAtomic.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* Revert "NFC: unify clang / LLVM atomic ordering"JF Bastien2016-04-171-73/+93
| | | | | | | | This reverts commit b0495df9eae2824bee830cc4c94f5441f0d4cbc9. Same as for the corresponding LLVM revert, an assert seems to fire. llvm-svn: 266575
* NFC: unify clang / LLVM atomic orderingJF Bastien2016-04-171-93/+73
| | | | | | | | | | | | | | | Summary: Depends on http://reviews.llvm.org/D18875 This makes the C11 / C++11 *ABI* atomic ordering accessible from LLVM, as discussed in http://reviews.llvm.org/D18200#inline-151433 Reviewers: jyknight, reames Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D18876 llvm-svn: 266574
* NFC: use AtomicOrdering isStrongerThanJF Bastien2016-04-061-4/+6
| | | | | | | | | | Summary: As discussed in D18775. Reviewers: jyknight Differential Revision: http://reviews.llvm.org/D18840 llvm-svn: 265617
* NFC: make AtomicOrdering an enum classJF Bastien2016-04-061-39/+46
| | | | | | | | | | | | Summary: See LLVM change D18775 for details, this change depends on it. Reviewers: jyknight, reames Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D18776 llvm-svn: 265569
* Preserve ExtParameterInfos into CGFunctionInfo.John McCall2016-03-111-2/+1
| | | | | | | | | As part of this, make the function-arrangement interfaces a little simpler and more semantic. NFC. llvm-svn: 263191
* [MSVC Compat] Don't provide /volatile:ms semantics to types > pointerDavid Majnemer2016-01-221-3/+16
| | | | | | | | | | | | Volatile loads of type wider than a pointer get split by MSVC because the base x86 ISA doesn't provide loads which are wider than pointer width. LLVM assumes that it can emit an cmpxchg8b but this is problematic if the memory is in a CONST memory segment. Instead, provide behavior compatible with MSVC: split loads wider than a pointer. llvm-svn: 258506
* Update for LLVM function name change.Rui Ueyama2016-01-141-1/+1
| | | | llvm-svn: 257802
* Correct atomic libcall support for __atomic_*_fetch builtins.James Y Knight2015-11-121-39/+39
| | | | | | | | | | | | | | In r244063, I had caused these builtins to call the same-named library functions, __atomic_*_fetch_SIZE. However, this was incorrect: while those functions are in fact supported by GCC's libatomic, they're not documented by the spec (and gcc doesn't ever call them). Instead, you're /supposed/ to call the __atomic_fetch_* builtins and then redo the operation inline to return the final value. Differential Revision: http://reviews.llvm.org/D14385 llvm-svn: 252920
* Atomics: support __c11_* calls on _Atomic struct types.Tim Northover2015-11-091-36/+64
| | | | | | | | | | | | | When a struct's size is not a power of 2, the corresponding _Atomic() type is promoted to the nearest. We already correctly handled normal C++ expressions of this form, but direct calls to the __c11_atomic_whatever builtins ended up performing dodgy operations on the smaller non-atomic types (e.g. memcpy too much). Later optimisations removed this as undefined behaviour. This patch converts EmitAtomicExpr to allocate its temporaries at the full atomic width, sidestepping the issue. llvm-svn: 252507
* Decorating vptr load & stores with !invariant.groupPiotr Padlewski2015-09-151-2/+2
| | | | | | | | | | Adding !invariant.group to vptr load/stores for devirtualization purposes. For more goto: http://lists.llvm.org/pipermail/cfe-dev/2015-July/044227.html http://reviews.llvm.org/D12026 llvm-svn: 247725
* Compute and preserve alignment more faithfully in IR-generation.John McCall2015-09-081-262/+251
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
* Add missing atomic libcall support.James Y Knight2015-08-051-3/+71
| | | | | | | | | | Support for emitting libcalls for __atomic_fetch_nand and __atomic_{add,sub,and,or,xor,nand}_fetch was missing; add it, and some test cases. Differential Revision: http://reviews.llvm.org/D10847 llvm-svn: 244063
* Respect alignment of nested bitfieldsUlrich Weigand2015-07-101-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | tools/clang/test/CodeGen/packed-nest-unpacked.c contains this test: struct XBitfield { unsigned b1 : 10; unsigned b2 : 12; unsigned b3 : 10; }; struct YBitfield { char x; struct XBitfield y; } __attribute((packed)); struct YBitfield gbitfield; unsigned test7() { // CHECK: @test7 // CHECK: load i32, i32* getelementptr inbounds (%struct.YBitfield, %struct.YBitfield* @gbitfield, i32 0, i32 1, i32 0), align 4 return gbitfield.y.b2; } The "align 4" is actually wrong. Accessing all of "gbitfield.y" as a single i32 is of course possible, but that still doesn't make it 4-byte aligned as it remains packed at offset 1 in the surrounding gbitfield object. This alignment was changed by commit r169489, which also introduced changes to bitfield access code in CGExpr.cpp. Code before that change used to take into account *both* the alignment of the field to be accessed within the current struct, *and* the alignment of that outer struct itself; this logic was removed by the above commit. Neglecting to consider both values can cause incorrect code to be generated (I've seen an unaligned access crash on SystemZ due to this bug). In order to always use the best known alignment value, this patch removes the CGBitFieldInfo::StorageAlignment member and replaces it with a StorageOffset member specifying the offset from the start of the surrounding struct to the bitfield's underlying storage. This offset can then be combined with the best-known alignment for a bitfield access lvalue to determine the alignment to use when accessing the bitfield's storage. Differential Revision: http://reviews.llvm.org/D11034 llvm-svn: 241916
* Revert r240270 ("Fixed/added namespace ending comments using clang-tidy").Alexander Kornienko2015-06-221-1/+1
| | | | llvm-svn: 240353
* Fixed/added namespace ending comments using clang-tidy. NFCAlexander Kornienko2015-06-221-1/+1
| | | | | | | | | | | | The patch is generated using this command: $ tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \ -checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \ work/llvm/tools/clang To reduce churn, not touching namespaces spanning less than 10 lines. llvm-svn: 240270
* CGAtomic.cpp: Fix bogus \brief(s). Did you mean "\param"? [-Wdocumentation]NAKAMURA Takumi2015-05-151-3/+3
| | | | llvm-svn: 237447
* [OPENMP] Fixed bug in atomic update/capture/write constructs.Alexey Bataev2015-05-151-130/+279
| | | | | | Fixed a bug with codegen for destination atomic l-value with padding and junk in this padding bytes. llvm-svn: 237422
* [OPENMP] Fix crash on reductions codegen for short circuit reduction operations.Alexey Bataev2015-04-291-2/+2
| | | | llvm-svn: 236084
* [opaque pointer type] More GEP API migrationsDavid Blaikie2015-04-051-5/+6
| | | | | | | Looks like the VTable code in particular will need some work to pass around the pointee type explicitly. llvm-svn: 234128
* [OPENMP] Codegen for 'atomic update' construct.Alexey Bataev2015-03-301-41/+103
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adds atomic update codegen for the following forms of expressions: x binop= expr; x++; ++x; x--; --x; x = x binop expr; x = expr binop x; If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted: bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: ... Differential Revision: http://reviews.llvm.org/D8536 llvm-svn: 233513
* [OPENMP] Fixed bug in codegen of 'atomic write'.Alexey Bataev2015-03-191-2/+2
| | | | | | Fixed codegen for exit/continue order after success/failed atomic cmpxchg instruction for 'atomic write' construct. llvm-svn: 232712
* Silence an MSVC warning about not all control paths returning a value; NFC.Aaron Ballman2015-02-271-0/+1
| | | | llvm-svn: 230754
* [OPENMP] Codegen for "#pragma omp atomic write"Alexey Bataev2015-02-271-183/+378
| | | | | | | | | | For global reg lvalue - use regular store through global register. For simple lvalue - use simple atomic store. For bitfields, vector element, extended vector elements - the original value of the whole storage (for vector elements) or of some aligned value (for bitfields) is atomically read, the part of this value for the given lvalue is modified and then use atomic compare-and-exchange operation to try to atomically write modified value (if it was not modified). Also, changes in this patch fix the bug for '#pragma omp atomic read' applied to extended vector elements. Differential Revision: http://reviews.llvm.org/D7369 llvm-svn: 230736
* Try to unbreak the Hexagon botDavid Majnemer2015-02-141-1/+1
| | | | llvm-svn: 229219
* Revert "Revert r229082 for a bit, it caused PR22577."David Majnemer2015-02-141-5/+56
| | | | | | | This reverts commit r229123. It was a red herring, the bug was present without r229082. llvm-svn: 229205
* Revert r229082 for a bit, it caused PR22577.Nico Weber2015-02-131-56/+5
| | | | llvm-svn: 229123
* MS ABI: Implement /volatile:msDavid Majnemer2015-02-131-5/+56
| | | | | | | | | | | | The /volatile:ms semantics turn volatile loads and stores into atomic acquire and release operations. This distinction is important because volatile memory operations do not form a happens-before relationship with non-atomic memory. This means that a volatile store is not sufficient for implementing a mutex unlock routine. Differential Revision: http://reviews.llvm.org/D7580 llvm-svn: 229082
* [OPENMP] CodeGen for "omp atomic read [seq_cst]" directive.Alexey Bataev2015-01-221-72/+151
| | | | | | | | | | | | | "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
* Revert commit revision 226786Alexey Bataev2015-01-221-150/+72
| | | | | | Need to add initialization of AtomicInfo::EvaluationKind field llvm-svn: 226787
* [OPENMP] CodeGen for "omp atomic read [seq_cst]" directive.Alexey Bataev2015-01-221-72/+150
| | | | | | | | | | | | | "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226786
* Revert commit r226784.Alexey Bataev2015-01-221-150/+72
| | | | | | Accidentally modified file SemaType.cpp must be restored to its original state. llvm-svn: 226785
* [OPENMP] CodeGen for "omp atomic read [seq_cst]" directive.Alexey Bataev2015-01-221-72/+150
| | | | | | | | | | | | | "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v". Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226784
* Bugfix for Codegen of atomic load/store/other ops.Alexey Bataev2014-12-151-72/+145
| | | | | | | Currently clang fires assertions on x86-64 on any atomic operations for long double operands. Patch fixes codegen for such operations. Differential Revision: http://reviews.llvm.org/D6499 llvm-svn: 224230
* CodeGen: Loads/stores to allocas for atomic ops shouldn't be volatileDavid Majnemer2014-12-121-7/+10
| | | | | | | | | Don't inherit the volatile-ness of the input pointer to the volatile operation for memory allocated on the side. This fixes PR17306. llvm-svn: 224110
* CodeGen: Fix emission of __atomic_compare_exchangeDavid Majnemer2014-11-251-22/+22
| | | | | | We (wrongly) discarded the return value of the call. llvm-svn: 222798
* CodeGen: Make atomic operations play nice with address spacesDavid Majnemer2014-11-221-6/+8
| | | | | | | | We were being a little sloppy with our pointer/address space casts. This fixes PR21643. llvm-svn: 222615
* CodeGen: Don't completely mess-up optimized atomic libcallsDavid Majnemer2014-08-291-13/+31
| | | | | | | | | | | | | | | | | | | | | | | Summary: We did a great job getting this wrong: - We messed up which LLVM IR types to use for arguments and return values. The optimized libcalls use integer types for values. Clang attempted to use the IR type which corresponds to the value passed in instead of using an appropriately sized integer type. This would result in violations of the ABI for, as an example, floating point types. - We didn't bother recording the result of the atomic libcall in the destination memory. Instead, call the functions with arguments matching the type of the libcall prototype's parameters. This fixes PR20780. Differential Revision: http://reviews.llvm.org/D5098 llvm-svn: 216714
* MS ABI: Consider alignment attributes on typedefs for layoutDavid Majnemer2014-07-301-7/+11
| | | | | | | | | | | | | | | | | | | The MS ABI has a notion of 'required alignment' for fields; this alignment supercedes pragma pack directives. MSVC takes into account alignment attributes on typedefs when determining whether or not a field has a certain required alignment. Do the same in clang by tracking whether or not we saw such an attribute when calculating the type's bitwidth and alignment. This fixes PR20418. Reviewers: rnk Differential Revision: http://reviews.llvm.org/D4714 llvm-svn: 214274
* Atomics: emit "cmpxchg weak" where possibleTim Northover2014-06-131-26/+60
| | | | | | | | | | | | | | | | | | Most builtins date from before the "cmpxchg weak" was a gleam in the C++ committee's eye, so fortunately not much needs to change. But a few of them *do* acknowledge that failure is possible. For these, we'll emit the usual cartesian product of cmpxchg operations if we can't statically determine weakness. CodeGen can sort it out later if the function gets inlined. The only other non-trivial aspect of this is (I think) that we emit the scalar expression for "IsWeak" once, at the beginning, and propagate its value through the successive blocks. There's not much in it, but it's slightly more consistent with the existing handling of FailureOrder. llvm-svn: 210932
* IR-change: cmpxchg operations now return { iN, i1 }.Tim Northover2014-06-131-3/+4
| | | | | | | | This is a minimal fix for clang. I'll soon add support for generating weak variants when requested, but that's not really necessary for the LLVM change in isolation. llvm-svn: 210907
* [C++11] Use 'nullptr'. CodeGen edition.Craig Topper2014-05-211-11/+13
| | | | llvm-svn: 209272
* Fix atomic libcall.Logan Chien2014-03-261-2/+4
| | | | | | | | This commit fixes a cast instruction assertion failure due to the incompatible type cast. This will only happen when the target requires atomic libcalls. llvm-svn: 204834
* CodeGen: use symbolic enumerators for memory orderTim Northover2014-03-131-5/+10
| | | | | | | It makes the code clearer and less error-prone to use our (already known) special values for the various memory order cases. llvm-svn: 203838
* CodeGen: make use of weaker failure orders on cmpxchg.Tim Northover2014-03-131-67/+151
| | | | | | | | | | | | This makes Clang take advantage of the recent IR addition of a "failure" memory ordering requirement. As with the "success" ordering, we try to emit just a single version if the expression is constant, but fall back to runtime detection (to allow optimisation across function-call boundaries). rdar://problem/15996804 llvm-svn: 203837
* Sema: produce error when invalid ordering is passed to atomic builtinTim Northover2014-03-111-20/+10
| | | | | | | | | This is a conservative check, because it's valid for the expression to be non-constant, and in cases like that we just don't know whether it's valid. rdar://problem/16242991 llvm-svn: 203561
* Update for IR: add a second AtomicOrdering to cmpxchg insts.Tim Northover2014-03-111-3/+4
| | | | | | rdar://problem/15996804 llvm-svn: 203560
* IRGen: __c11/__atomic compare-and-exchange should respect the standardDavid Majnemer2014-03-101-10/+36
| | | | | | | | | | | | | | | Summary: 'Expected' should only be modified if the operation fails. This fixes PR18899. Reviewers: chandlerc, rsmith, rjmccall CC: cfe-commits Differential Revision: http://llvm-reviews.chandlerc.com/D2922 llvm-svn: 203493
* [C++11] Replace llvm::tie with std::tie.Benjamin Kramer2014-03-021-2/+2
| | | | llvm-svn: 202639
* Add CodeGenABITypes.h for use in LLDB.Mark Lacey2013-10-301-0/+1
| | | | | | | | | | | | | | | | | | CodeGenABITypes is a wrapper built on top of CodeGenModule that exposes some of the functionality of CodeGenTypes (held by CodeGenModule), specifically methods that determine the LLVM types appropriate for function argument and return values. I addition to CodeGenABITypes.h, CGFunctionInfo.h is introduced, and the definitions of ABIArgInfo, RequiredArgs, and CGFunctionInfo are moved into this new header from the private headers ABIInfo.h and CGCall.h. Exposing this functionality is one part of making it possible for LLDB to determine the actual ABI locations of function arguments and return values, making it possible for it to determine this for any supported target without hard-coding ABI knowledge in the LLDB code. llvm-svn: 193717
* Thread a SourceLocation into the EmitCheck for "load_invalid_value". This occursNick Lewycky2013-10-021-21/+32
| | | | | | when scalars are loaded / undergo lvalue-to-rvalue conversion. llvm-svn: 191808
OpenPOWER on IntegriCloud