summaryrefslogtreecommitdiffstats
path: root/openmp/runtime/src/kmp_tasking.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [OpenMP] NFC: Fix trivial typos in commentsKazuaki Ishizaki2020-01-071-3/+4
| | | | | | | | | | | | Reviewers: jdoerfert, Jim Reviewed By: Jim Subscribers: Jim, mgorny, guansong, jfb, openmp-commits Tags: #openmp Differential Revision: https://reviews.llvm.org/D72285
* [OpenMP] NFC: Fix trivial typos in commentsKelvin Li2020-01-031-7/+7
| | | | | | Submitted by: kiszk Differential Revision: https://reviews.llvm.org/D72171
* [OpenMP] Remove OMP spec versioningJonathan Peyton2019-07-121-149/+10
| | | | | | | | | | Remove all older OMP spec versioning from the runtime and build system. Patch by Terry Wilmarth Differential Revision: https://reviews.llvm.org/D64534 llvm-svn: 365963
* Create a runtime option to disable task throttling.Andrey Churbanov2019-07-021-2/+4
| | | | | | | | Patch by viroulep (Philippe Virouleau) Differential Revision: https://reviews.llvm.org/D63196 llvm-svn: 364934
* New implementation of OpenMP 5.0 detached tasks.Andrey Churbanov2019-06-191-22/+122
| | | | | | | | Patch by Alex Duran Differential Revision: https://reviews.llvm.org/D62485 llvm-svn: 363799
* [OpenMP] Add task alloc functionGheorghe-Teodor Bercea2019-06-141-0/+10
| | | | | | | | | | | | | | | | Summary: Add the target task allocation function to the interface. Reviewers: ABataev, AlexEichenberger, caomhin, jlpeyton, AndreyChurbanov, RaviNarayanaswamy, hbae Reviewed By: AlexEichenberger, hbae Subscribers: hbae, RaviNarayanaswamy, cfe-commits, Hahnfeld, guansong, jdoerfert, openmp-commits Tags: #openmp, #clang Differential Revision: https://reviews.llvm.org/D63010 llvm-svn: 363449
* [OpenMP] Implement task modifier for reduction clauseJonathan Peyton2019-05-011-55/+324
| | | | | | | | | | | | | | | | | Implemented task modifier in two versions - one without taking into account omp_orig variable (the omp_orig still can be processed by compiler without help of the library, but each reduction object will need separate initializer with global access to omp_orig), another with omp_orig variable included into interface (single initializer can be used for multiple reduction objects of the same type). Second version can be used when the omp_orig is not globally accessible, or to optimize code in case of multiple reduction objects of the same type. Patch by Andrey Churbanov Differential Revision: https://reviews.llvm.org/D60976 llvm-svn: 359710
* [OpenMP] Make use of sched_yield optional in runtimeJonathan Peyton2019-02-281-7/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch cleans up the yielding code and makes it optional. An environment variable, KMP_USE_YIELD, was added. Yielding is still on by default (KMP_USE_YIELD=1), but can be turned off completely (KMP_USE_YIELD=0), or turned on only when oversubscription is detected (KMP_USE_YIELD=2). Note that oversubscription cannot always be detected by the runtime (for example, when the runtime is initialized and the process forks, oversubscription cannot be detected currently over multiple instances of the runtime). Because yielding can be controlled by user now, the library mode settings (from KMP_LIBRARY) for throughput and turnaround have been adjusted by altering blocktime, unless that was also explicitly set. In the original code, there were a number of places where a double yield might have been done under oversubscription. This version checks oversubscription and if that's not going to yield, then it does the spin check. Patch by Terry Wilmarth Differential Revision: https://reviews.llvm.org/D58148 llvm-svn: 355120
* [OMPT] Make sure that OMPT is enabled when accessing internals of the runtimeJoachim Protze2019-02-041-0/+1
| | | | | | | | | | | | | | | | | The three switch fallthrough generate a warning with -Wimplicit-fallthrough. Two are documented as fallthrough, one is not, but I think the intention is to also fallthrough in kmp_tasking.cpp. Not sure whether kmp.h is the best place to define the macro. Reviewers: jlpeyton, AndreyChurbanov, Hahnfeld Reviewed By: jlpeyton Tags: #openmp Differential Revision: https://reviews.llvm.org/D56397 llvm-svn: 353052
* Update more file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-191-4/+3
| | | | | | | | | | | | | | | | | | to reflect the new license. These used slightly different spellings that defeated my regular expressions. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351648
* [OpenMP] Add omp_pause_resource* APIJonathan Peyton2019-01-161-1/+1
| | | | | | | | | | | | Add omp_pause_resource and omp_pause_resource_all API and enum, plus stub for internal implementation. Implemented callable helper function to do local pause, and added basic functionality for hard and soft pause. Patch by Terry Wilmarth Differential Revision: https://reviews.llvm.org/D55078 llvm-svn: 351372
* Doc: fixed description of a parameter of the __kmpc_taskloopAndrey Churbanov2019-01-091-1/+1
| | | | | | | | Patch by sergi.mateo.bellido@gmail.com Differential Revision: https://reviews.llvm.org/D56432 llvm-svn: 350713
* [OMPT] First chunk of final OMPT 5.0 interface updatesJoachim Protze2018-12-181-22/+27
| | | | | | | | | | | | | This patch updates the implementation of the ompt_frame_t, ompt_wait_id_t and ompt_state_t. The final version of the OpenMP 5.0 spec added the "t" for these types. Furthermore the structure for ompt_frame_t changed and allows to specify that the reenter frame belongs to the runtime. Patch partially prepared by Simon Convent Reviewers: hbae llvm-svn: 349458
* [OpenMP] Fixes for LIBOMP_OMP_VERSION=45/40Roman Lebedev2018-12-151-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: I have discovered this because i wanted to experiment with building static libomp (with openmp-4.0 support only) for debugging purposes. There are three kinds of problems here: 1. `__kmp_compare_and_store_acq()` simply does not exist. It was added in D47903 by @jlpeyton. I'm guessing `__kmp_atomic_compare_store_acq()` was meant. 2. In `__kmp_is_ticket_lock_initialized()`, `lck->lk.initialized` is `std::atomic<bool>`, while `lck` is `kmp_ticket_lock_t *`. Naturally, they can't be equality-compared. Either, it should return the value read from `lck->lk.initialized`, or do what `__kmp_is_queuing_lock_initialized()` does, compare the passed pointer with the field in the struct pointed by the pointer. I think the latter is correct-er choice here. 3. Tests were not versioned. They assume that `LIBOMP_OMP_VERSION` is at the latest version. This does not touch LIBOMP_OMP_VERSION=30. That is still broken. Reviewers: jlpeyton, Hahnfeld, AndreyChurbanov Reviewed By: AndreyChurbanov Subscribers: guansong, jfb, openmp-commits, jlpeyton Tags: #openmp Differential Revision: https://reviews.llvm.org/D55496 llvm-svn: 349260
* [OpenMP] Add stubs for Task affinity APIJonathan Peyton2018-11-291-0/+22
| | | | | | | | | | | | | This patch adds __kmpc_omp_reg_task_with_affinity to register affinity information for tasks. For now, the affinity information is not used, and the function always succeeds. This also adds the kmp_task_affinity_info_t structure to store the task affinity information. Patch by Terry Wilmarth Differential Revision: https://reviews.llvm.org/D55026 llvm-svn: 347907
* Implementation of OpenMP 5.0 mutexinoutset task dependency type.Andrey Churbanov2018-11-071-132/+184
| | | | | | Differential Revision: https://reviews.llvm.org/D53380 llvm-svn: 346307
* [OpenMP] Add missing __kmpc_critical_with_hint to dllexportsJonathan Peyton2018-09-261-6/+3
| | | | | | | | | This patch puts the __kmpc_critical_with_hint function in dllexports and also replaces some OMP_45_ENABLED to OMP_50_ENABLED Differential Revision: https://reviews.llvm.org/D52380 llvm-svn: 343143
* [OpenMP] Fix performance issue from 376.kdtreeJonathan Peyton2018-09-261-7/+7
| | | | | | | | | | | | | | This change improves the performance of 376.kdtree by giving the compiler an opportunity to do inlining and other optimizations for the call path, __kmpc_omp_task_complete_if0()->__kmp_task_finish(), which is one of the hot paths in the program; some functions in kmp_taskdeps.cpp were moved to the new header file, kmp_taskdeps.h to achieve this. Patch by Hansang Bae Differential Revision: https://reviews.llvm.org/D51889 llvm-svn: 343138
* [OMPT] Update types according to TR7Joachim Protze2018-09-101-4/+4
| | | | | | | | | | | | | | | | | | Some types and callback signatures have changed from TR6 to TR7. Major changes (only adding signatures and stubs): (-remove idle callback) done by D48362 -add reduction and dispatch callback -add get_task_memory and finalize_tool runtime entry points -ompt_invoker_t becomes ompt_parallel_flag_t -more types of sync_regions Patch provided by Simon Convent Reviewers: hbae, protze.joachim Differential Revision: https://reviews.llvm.org/D50774 llvm-svn: 341834
* [OpenMP] Fix tasking bug for decreasing hot team nthreadsJonathan Peyton2018-08-241-1/+1
| | | | | | | | | | | | | | | | | | The __kmp_execute_tasks_template() function reads the task_team and current_task from the thread structure. There appears to be a pathological timing where the number of threads in the hot team decreases and so a thread is put in the pool via __kmp_free_thread(). It could be the case that: 1) A thread reads th_task_team into task_team local variables and is then interrupted by the OS 2) Master frees the thread and sets current task and task team to NULL 3) The thread reads current_task as NULL When this happens, current_task is dereferenced and a segfault occurs. This patch just checks for current_task to not be NULL as well. Differential Revision: https://reviews.llvm.org/D50651 llvm-svn: 340632
* [OpenMP] Fix incorrect barrier imbalance reporting in ITTNOTIFYJonathan Peyton2018-08-241-21/+25
| | | | | | | | | Exclude nested explicit tasks from timing, only outer level explicit task counted and its time added to barrier arrive time for the thread. Differential Revision: https://reviews.llvm.org/D50584 llvm-svn: 340628
* [OpenMP] Cleanup codeJonathan Peyton2018-08-091-5/+6
| | | | | | | | | | | | | | | | This patch cleans up unused functions, variables, sign compare issues, and addresses some -Warning flags which are now enabled including -Wcast-qual. Not all the warning flags in LibompHandleFlags.cmake are enabled, but some are with this patch. Some __kmp_gtid_from_* macros in kmp.h are switched to static inline functions which allows us to remove the awkward definition of KMP_DEBUG_ASSERT() and KMP_ASSERT() macros which used the comma operator. This had to be done for the innumerable -Wunused-value warnings related to KMP_DEBUG_ASSERT() Differential Revision: https://reviews.llvm.org/D49105 llvm-svn: 339393
* [OpenMP] Fix tasking + parallel bugJonathan Peyton2018-07-301-0/+3
| | | | | | | | | | | | From the bug report, the runtime needs to initialize the nproc variables (inside middle init) for each root when the task is encountered, otherwise, a segfault can occur. Bugzilla: https://bugs.llvm.org/show_bug.cgi?id=36720 Differential Revision: https://reviews.llvm.org/D49996 llvm-svn: 338313
* [OpenMP] Fix new task creationGheorghe-Teodor Bercea2018-07-301-1/+1
| | | | | | | | | | | | | | | | Summary: When OMPT is not supported the __kmp_omp_task() function is passed the parameters in the wrong order. This is a fix related to patch D47709. Reviewers: Hahnfeld, sconvent, caomhin, jlpeyton Reviewed By: Hahnfeld Subscribers: guansong, openmp-commits Differential Revision: https://reviews.llvm.org/D50001 llvm-svn: 338295
* [OMPT] Fix OMPT callbacks for the taskloop construct and add testcaseJoachim Protze2018-07-271-61/+170
| | | | | | | | | | | | | | | | | Fix the order of callbacks related to the taskloop construct. Add the iteration_count to work callbacks (according to the spec). Use kmpc_omp_task() instead of kmp_omp_task() to include OMPT callbacks. Add a testcase. Patch by Simon Convent Reviewed by: protze.joachim, hbae Subscribers: openmp-commits Differential Revision: https://reviews.llvm.org/D47709 llvm-svn: 338146
* [OMPT] Adapt OMPT callbacks for tasks to handle untied tasks correctlyJoachim Protze2018-07-271-50/+62
| | | | | | | | | | | | | | | | | | | | | | The ompt/tasks/task_types.c testcase did not test untied tasks properly. Now, frame addresses are tested and two scheduling points are added at which the task can switch to another thread. Due to scheduling effects, the frame address could be NULL. This needed a restructure of the way OMPT callbacks are called. __ompt_task_finish() now as an extra parameter, whether a task is completed. Its invocation has been moved into __kmp_task_finish(). Thus, the order of the writes to the frame addresses is not subject to scheduling effects anymore. Patch by Simon Convent Reviewed by: protze.joachim, hbae Subscribers: openmp-commits Differential Revision: https://reviews.llvm.org/D49181 llvm-svn: 338145
* [OpenMP] Use C++11 Atomics - barrier, tasking, and lock codeJonathan Peyton2018-07-091-53/+54
| | | | | | | | | | | | | | | | | These are preliminary changes that attempt to use C++11 Atomics in the runtime. We are expecting better portability with this change across architectures/OSes. Here is the summary of the changes. Most variables that need synchronization operation were converted to generic atomic variables (std::atomic<T>). Variables that are updated with combined CAS are packed into a single atomic variable, and partial read/write is done through unpacking/packing Patch by Hansang Bae Differential Revision: https://reviews.llvm.org/D47903 llvm-svn: 336563
* [OMPT] Rename ompt_frame_t to omp_frame_tJoachim Protze2018-05-281-1/+1
| | | | | | | | Rename ompt_frame_t to omp_frame_t, as defined in the spec. Differential Revision: https://reviews.llvm.org/D43568 llvm-svn: 333367
* Introduce GOMP_taskloop APIJonathan Peyton2018-04-181-18/+138
| | | | | | | | | | | | | | | This patch introduces GOMP_taskloop to our API. It adds GOMP_4.5 to our version symbols. Being a wrapper around __kmpc_taskloop, the function creates a task with the loop bounds properly nested in the shareds so that the GOMP task thunk will work properly. Also, the firstprivate copy constructors are properly handled using the __kmp_gomp_task_dup() auxiliary function. Currently, only linear spawning of tasks is supported for the GOMP_taskloop interface. Differential Revision: https://reviews.llvm.org/D45327 llvm-svn: 330282
* Minor code cleanupJonathan Peyton2018-01-101-2/+3
| | | | | | | | Patch by Terry Wilmarth Differential Revision: https://reviews.llvm.org/D41831 llvm-svn: 322203
* Improve stability of the runtime in parent/child processesJonathan Peyton2018-01-101-1/+1
| | | | | | | | | | | | | | | This change improves stability of the runtime when the application forks child processes. Acquiring/releasing __kmp_initz_lock and __kmp_forkjoin_lock in the atfork handlers insures that the actual fork does not occur while those two locks are held, and __kmp_itt_reset() reverts the itt's global state to the initial state which also initializes the mutex stored in the global state. Some missing initialization code was also inserted in the child's atfork handler. Patch by Hansang Bae Differential Revision: https://reviews.llvm.org/D41462 llvm-svn: 322202
* Exclude untied tasks from checking of task scheduling constraint (TSC).Andrey Churbanov2017-11-161-94/+156
| | | | | | | | This can improve performance of tests with untied tasks. Differential Revision: https://reviews.llvm.org/D39613 llvm-svn: 318388
* Rename fields of ompt_frame_tJoachim Protze2017-11-051-16/+14
| | | | | | | | | | This is part of the renaming of data types from OpenMP TR4 to TR6 Patch by Simon Convent Differential Revision: https://reviews.llvm.org/D39326 llvm-svn: 317435
* Revert "Rename fields of ompt_frame_t"Jonas Hahnfeld2017-11-031-14/+16
| | | | | | This reverts commit r317338 which discarded some recent commits. llvm-svn: 317347
* Revert "Updating implementation of OMPT as specified in OpenMP 5.0 Preview 2 ↵Jonas Hahnfeld2017-11-031-29/+24
| | | | | | | | (TR6)" This reverts commit r317339 which discarded some recent commits. llvm-svn: 317346
* Updating implementation of OMPT as specified in OpenMP 5.0 Preview 2 (TR6)Joachim Protze2017-11-031-24/+29
| | | | | | | | | | | The TR6 document is expected to be publically released around November 15. This patch does not implement OMPT for libomptarget. Patch by Simon Convent and Joachim Protze Differential Revision: https://reviews.llvm.org/D39182 llvm-svn: 317339
* Rename fields of ompt_frame_tJoachim Protze2017-11-031-16/+14
| | | | | | | | | | This is part of the renaming of data types from OpenMP TR4 to TR6 Patch by Simon Convent Differential Revision: https://reviews.llvm.org/D39326 llvm-svn: 317338
* Update implementation of OMPT to the specification OpenMP 5.0 Preview 1 (TR4).Joachim Protze2017-11-011-130/+395
| | | | | | | | | | | | | | The code is tested to work with latest clang, GNU and Intel compiler. The implementation is optimized for low overhead when no tool is attached shifting the cost to execution with tool attached. This patch does not implement OMPT for libomptarget. Patch by Simon Convent and Joachim Protze Differential Revision: https://reviews.llvm.org/D38185 llvm-svn: 317085
* Apply formatting changesJonathan Peyton2017-10-201-2/+0
| | | | | | | | | | .clang-format's comments are removed and a (hopefully) final set of formatting changes are applied. Differential Revision: https://reviews.llvm.org/D38837 Differential Revision: https://reviews.llvm.org/D38920 llvm-svn: 316227
* Remove unnecessary semicolonsJonathan Peyton2017-09-271-2/+2
| | | | | | | | Removes semicolons after if {} blocks, function definitions, etc. I was able to apply the large OMPT patch cleanly on top of this one with no conflicts. llvm-svn: 314340
* Move lock acquire/release functions in task deque cleanup codeJonathan Peyton2017-08-021-3/+2
| | | | | | | | | | | | | The original locations can be reached without initializing the lock variable (td_deque_lock), so it is potentially unsafe. It is guaranteed that the lock is initialized if the deque (td_deque) is not NULL, and lock functions can be safely called. Patch by Hansang Bae Differential Revision: https://reviews.llvm.org/D36017 llvm-svn: 309875
* Cleanup: __kmp_env_* variablesJonathan Peyton2017-07-251-23/+27
| | | | | | | | | | Removed unused __kmp_env_* variables. Also clangified other people's code. Patch by Terry Wilmarth Differential Revision: https://reviews.llvm.org/D35808 llvm-svn: 309000
* Add recursive task scheduling strategy to taskloop implementationJonathan Peyton2017-07-181-93/+287
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Taskloop implementation is extended by using recursive task scheduling. Envirable KMP_TASKLOOP_MIN_TASKS added as a manual threshold for the user to switch from recursive to linear tasks scheduling. Details: * The calculations for the loop parameters are moved from __kmp_taskloop_linear upper level * Initial calculation is done in the __kmpc_taskloop, further range splitting is done in the __kmp_taskloop_recur. * Added threshold to switch from recursive to linear tasks scheduling; * One half of split range is scheduled as an internal task which just moves sub-range parameters to the stealing thread that continues recursive scheduling (if number of tasks still enough), the other half is processed recursively; * Internal task duplication routine fixed to assign parent task, that was not needed when all tasks were scheduled by same thread, but is needed now. Patch by Andrey Churbanov Differential Revision: https://reviews.llvm.org/D35273 llvm-svn: 308338
* Fix sporadic segfaults in tasking tests.Andrey Churbanov2017-07-181-1/+1
| | | | | | | | Patch by Terry Wilmarth Differential Revision: https://reviews.llvm.org/D35535 llvm-svn: 308298
* OpenMP RTL cleanup: eliminated warnings with -Wcast-qual, patch 2.Andrey Churbanov2017-07-171-23/+13
| | | | | | | | | Changes are: got all atomics to accept volatile pointers that allowed to simplify many type conversions. Windows specific code fixed correspondingly. Differential Revision: https://reviews.llvm.org/D35417 llvm-svn: 308164
* OpenMP RTL cleanup: eliminated warnings with -Wcast-qual.Andrey Churbanov2017-07-031-37/+45
| | | | | | | | | | | Changes are: replaced C-style casts with cons_cast and reinterpret_cast; type of several counters changed to signed; type of parameters of 32-bit and 64-bit AND and OR intrinsics changes to unsigned; changed files formatted using clang-format version 3.8.1. Differential Revision: https://reviews.llvm.org/D34759 llvm-svn: 307020
* Fix minor formatting issuesJonathan Peyton2017-06-011-2/+2
| | | | | | | | | | | Some code was restructured to move it under KMP_DEBUG. The rest is formatting changes to fix some things broken by clang-format Patch by Terry Wilmarth Differential Revision: https://reviews.llvm.org/D33744 llvm-svn: 304438
* Remove some outdated commentsJonathan Peyton2017-05-151-11/+0
| | | | llvm-svn: 303086
* Clang-format and whitespace cleanup of source codeJonathan Peyton2017-05-121-2582/+2668
| | | | | | | | | | | | | This patch contains the clang-format and cleanup of the entire code base. Some of clang-formats changes made the code look worse in places. A best effort was made to resolve the bulk of these problems, but many remain. Most of the problems were mangling line-breaks and tabbing of comments. Patch by Terry Wilmarth Differential Revision: https://reviews.llvm.org/D32659 llvm-svn: 302929
* Run-time library part of OpenMP 5.0 task reduction implementation.Andrey Churbanov2017-02-161-0/+208
| | | | | | | | | Added test kmp_task_reduction_nest.cpp which has an example of possible compiler codegen. Differential Revision: https://reviews.llvm.org/D29600 llvm-svn: 295343
OpenPOWER on IntegriCloud