summaryrefslogtreecommitdiffstats
path: root/drivers/gpu/drm/nouveau/nouveau_svm.c
Commit message (Collapse)AuthorAgeFilesLines
* nouveau: use mmu_interval_notifier instead of hmm_mirrorJason Gunthorpe2019-11-231-80/+99
| | | | | | | | | | | | | | | | | | Remove the hmm_mirror object and use the mmu_interval_notifier API instead for the range, and use the normal mmu_notifier API for the general invalidation callback. While here re-organize the pagefault path so the locking pattern is clear. nouveau is the only driver that uses a temporary range object and instead forwards nearly every invalidation range directly to the HW. While this is not how the mmu_interval_notifier was intended to be used, the overheads on the pagefaulting path are similar to the existing hmm_mirror version. Particularly since the interval tree will be small. Link: https://lore.kernel.org/r/20191112202231.3856-10-jgg@ziepe.ca Tested-by: Ralph Campbell <rcampbell@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* nouveau: use mmu_notifier directly for invalidate_range_startJason Gunthorpe2019-11-231-32/+63
| | | | | | | | | There is no reason to get the invalidate_range_start() callback via an indirection through hmm_mirror, just register a normal notifier directly. Link: https://lore.kernel.org/r/20191112202231.3856-9-jgg@ziepe.ca Tested-by: Ralph Campbell <rcampbell@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* mm/hmm: remove the page_shift member from struct hmm_rangeChristoph Hellwig2019-08-071-1/+0
| | | | | | | | | | | | | All users pass PAGE_SIZE here, and if we wanted to support single entries for huge pages we should really just add a HMM_FAULT_HUGEPAGE flag instead that uses the huge page size instead of having the caller calculate that size once, just for the hmm code to verify it. Link: https://lore.kernel.org/r/20190806160554.14046-8-hch@lst.de Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* mm/hmm: remove superfluous arguments from hmm_range_registerChristoph Hellwig2019-08-071-3/+2
| | | | | | | | | | | The start, end and page_shift values are all saved in the range structure, so we might as well use that for argument passing. Link: https://lore.kernel.org/r/20190806160554.14046-7-hch@lst.de Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* nouveau: pass struct nouveau_svmm to nouveau_range_faultChristoph Hellwig2019-08-071-6/+6
| | | | | | | | | | | | We'll need the nouveau_svmm structure to improve the function soon. For now this allows using the svmm->mm reference to unlock the mmap_sem, and thus the same dereference chain that the caller uses to lock and unlock it. Link: https://lore.kernel.org/r/20190806160554.14046-4-hch@lst.de Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* mm/hmm: remove hmm_range vmaRalph Campbell2019-07-261-4/+3
| | | | | | | | | | | | Since hmm_range_fault() doesn't use the struct hmm_range vma field, remove it. Link: https://lore.kernel.org/r/20190726005650.2566-8-rcampbell@nvidia.com Suggested-by: Jason Gunthorpe <jgg@mellanox.com> Signed-off-by: Ralph Campbell <rcampbell@nvidia.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* mm/hmm: replace the block argument to hmm_range_fault with a flags valueChristoph Hellwig2019-07-261-1/+1
| | | | | | | | | | | This allows easier expansion to other flags, and also makes the callers a little easier to read. Link: https://lore.kernel.org/r/20190726005650.2566-4-rcampbell@nvidia.com Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Ralph Campbell <rcampbell@nvidia.com> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* mm/hmm: replace hmm_update with mmu_notifier_rangeRalph Campbell2019-07-261-2/+2
| | | | | | | | | | | | | | The hmm_mirror_ops callback function sync_cpu_device_pagetables() passes a struct hmm_update which is a simplified version of struct mmu_notifier_range. This is unnecessary so replace hmm_update with mmu_notifier_range directly. Link: https://lore.kernel.org/r/20190726005650.2566-2-rcampbell@nvidia.com Signed-off-by: Ralph Campbell <rcampbell@nvidia.com> Reviewed: Christoph Hellwig <hch@lst.de> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> [jgg: white space tuning] Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* nouveau: return -EBUSY when hmm_range_wait_until_valid failsChristoph Hellwig2019-07-251-1/+1
| | | | | | | | | | | | -EAGAIN has a magic meaning for non-blocking faults, so don't overload it. Given that the caller doesn't check for specific error codes this change is purely cosmetic. Link: https://lore.kernel.org/r/20190724065258.16603-6-hch@lst.de Tested-by: Ralph Campbell <rcampbell@nvidia.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* nouveau: unlock mmap_sem on all errors from nouveau_range_faultChristoph Hellwig2019-07-251-5/+6
| | | | | | | | | | | | | | Currently nouveau_svm_fault expects nouveau_range_fault to never unlock mmap_sem, but the latter unlocks it for a random selection of error codes. Fix this up by always unlocking mmap_sem for non-zero return values in nouveau_range_fault, and only unlocking it in the caller for successful returns. Link: https://lore.kernel.org/r/20190724065258.16603-5-hch@lst.de Tested-by: Ralph Campbell <rcampbell@nvidia.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* nouveau: remove the block parameter to nouveau_range_faultChristoph Hellwig2019-07-251-6/+4
| | | | | | | | | | | The parameter is always false, so remove it as well as the -EAGAIN handling that can only happen for the non-blocking case. Link: https://lore.kernel.org/r/20190724065258.16603-4-hch@lst.de Tested-by: Ralph Campbell <rcampbell@nvidia.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* mm/hmm: move hmm_vma_range_done and hmm_vma_fault to nouveauChristoph Hellwig2019-07-251-2/+44
| | | | | | | | | | | | | | These two functions are marked as a legacy APIs to get rid of, but seem to suit the current nouveau flow. Move it to the only user in preparation for fixing a locking bug involving caller and callee. All comments referring to the old API have been removed as this now is a driver private helper. Link: https://lore.kernel.org/r/20190724065258.16603-3-hch@lst.de Tested-by: Ralph Campbell <rcampbell@nvidia.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
* mm/hmm: Use hmm_mirror not mm as an argument for hmm_range_registerJason Gunthorpe2019-06-101-1/+1
| | | | | | | | | | | | | | | | | | Ralph observes that hmm_range_register() can only be called by a driver while a mirror is registered. Make this clear in the API by passing in the mirror structure as a parameter. This also simplifies understanding the lifetime model for struct hmm, as the hmm pointer must be valid as part of a registered mirror so all we need in hmm_register_range() is a simple kref_get. Suggested-by: Ralph Campbell <rcampbell@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com> Reviewed-by: John Hubbard <jhubbard@nvidia.com> Reviewed-by: Ralph Campbell <rcampbell@nvidia.com> Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Tested-by: Philip Yang <Philip.Yang@amd.com>
* drm/nouveau/svm: new ioctl to migrate process memory to GPU memoryJérôme Glisse2019-02-201-0/+96
| | | | | | | | | | | This add an ioctl to migrate a range of process address space to the device memory. On platform without cache coherent bus (x86, ARM, ...) this means that CPU can not access that range directly, instead CPU will fault which will migrate the memory back to system memory. This is behind a staging flag so that we can evolve the API. Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
* drm/nouveau/dmem: device memory helpers for SVMJérôme Glisse2019-02-201-0/+2
| | | | | | | | | Device memory can be use in SVM, in which case we do not have any of the existing buffer object. This commit add infrastructure to allow use of device memory without nouveau_bo. Again this is a temporary solution until a rework of GPU memory management. Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
* drm/nouveau/svm: initial support for shared virtual memoryBen Skeggs2019-02-201-0/+737
This uses HMM to mirror a process' CPU page tables into a channel's page tables, and keep them synchronised so that both the CPU and GPU are able to access the same memory at the same virtual address. While this code also supports Volta/Turing, it's only enabled for Pascal GPUs currently due to channel recovery being unreliable right now on the later GPUs. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
OpenPOWER on IntegriCloud