summaryrefslogtreecommitdiffstats
path: root/drivers/iommu/intel-iommu.c
Commit message (Collapse)AuthorAgeFilesLines
...
| * iommu/vt-d: Update to use PCI DMA aliasesAlex Williamson2014-07-041-122/+106
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | VT-d code currently makes use of pci_find_upstream_pcie_bridge() in order to find the topology based alias of a device. This function has a few problems. First, it doesn't check the entire alias path of the device to the root bus, therefore if a PCIe device is masked upstream, the wrong result is produced. Also, it's known to get confused and give up when it crosses a bridge from a conventional PCI bus to a PCIe bus that lacks a PCIe capability. The PCI-core provided DMA alias support solves both of these problems and additionally adds support for DMA function quirks allowing VT-d to work with devices like Marvell and Ricoh with known broken requester IDs. Signed-off-by: Alex Williamson <alex.williamson@redhat.com> Cc: David Woodhouse <dwmw2@infradead.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
| * iommu/vt-d: Use iommu_group_get_for_dev()Alex Williamson2014-07-041-75/+4
| | | | | | | | | | | | | | | | | | | | The IOMMU code now provides a common interface for finding or creating an IOMMU group for a device on PCI buses. Make use of it and remove piles of code. Signed-off-by: Alex Williamson <alex.williamson@redhat.com> Cc: David Woodhouse <dwmw2@infradead.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
* | iommu/vt-d: Suppress compiler warningsYijing Wang2014-07-041-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | suppress compiler warnings: drivers/iommu/intel-iommu.c: In function ‘device_to_iommu’: drivers/iommu/intel-iommu.c:673: warning: ‘segment’ may be used uninitialized in this function drivers/iommu/intel-iommu.c: In function ‘get_domain_for_dev.clone.3’: drivers/iommu/intel-iommu.c:2217: warning: ‘bridge_bus’ may be used uninitialized in this function drivers/iommu/intel-iommu.c:2217: warning: ‘bridge_devfn’ may be used uninitialized in this function Signed-off-by: Yijing Wang <wangyijing@huawei.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
* | iommu/vt-d: Remove the useless dma_pte_addrYijing Wang2014-07-041-4/+2
| | | | | | | | | | Signed-off-by: Yijing Wang <wangyijing@huawei.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
* | iommu/vt-d: Don't use magic number in dma_pte_superpageJoerg Roedel2014-07-041-1/+1
| | | | | | | | | | | | | | Use the already defined DMA_PTE_LARGE_PAGE for testing instead of hardcoding the value again. Signed-off-by: Joerg Roedel <jroedel@suse.de>
* | iommu/vt-d: Fix reference count in iommu_prepare_isaYijing Wang2014-07-041-0/+1
| | | | | | | | | | | | | | Decrease the device reference count avoid memory leak. Signed-off-by: Yijing Wang <wangyijing@huawei.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
* | iommu/vt-d: Use inline function dma_pte_superpage instead of macrosYijing Wang2014-07-041-1/+1
| | | | | | | | | | | | | | | | Use inline function dma_pte_superpage() instead of macro for better readability. Signed-off-by: Yijing Wang <wangyijing@huawei.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
* | iommu/vt-d: Clear the redundant assignment for domain->nidYijing Wang2014-07-041-1/+0
| | | | | | | | | | | | | | | | | | Alloc_domain() will initialize domain->nid to -1. So the initialization for domain->nid in md_domain_init() is redundant, clear it. Signed-off-by: Yijing Wang <wangyijing@huawei.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
* | iommu/vt-d: Use list_for_each_safe() to simplify codeYijing Wang2014-07-041-4/+2
|/ | | | | | | | Use list_for_each_entry_safe() instead of list_entry() to simplify code. Signed-off-by: Yijing Wang <wangyijing@huawei.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
* iommu/vt-d: fix bug in handling multiple RMRRs for the same PCI deviceJiang Liu2014-06-201-6/+3
| | | | | | | | | | | | | | | Function dmar_iommu_notify_scope_dev() makes a wrong assumption that there's one RMRR for each PCI device at most, which causes DMA failure on some HP platforms. So enhance dmar_iommu_notify_scope_dev() to handle multiple RMRRs for the same PCI device. Fixbug: https://bugzilla.novell.com/show_bug.cgi?id=879482 Cc: <stable@vger.kernel.org> # 3.15 Reported-by: Tom Mingarelli <thomas.mingarelli@hp.com> Tested-by: Linda Knippers <linda.knippers@hp.com> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
* intel-iommu: integrate DMA CMAAkinobu Mita2014-06-041-8/+25
| | | | | | | | | | | | | | | | | | | | | | | This adds support for the DMA Contiguous Memory Allocator for intel-iommu. This change enables dma_alloc_coherent() to allocate big contiguous memory. It is achieved in the same way as nommu_dma_ops currently does, i.e. trying to allocate memory by dma_alloc_from_contiguous() and alloc_pages() is used as a fallback. Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com> Cc: Marek Szyprowski <m.szyprowski@samsung.com> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Cc: David Woodhouse <dwmw2@infradead.org> Cc: Don Dutile <ddutile@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: Yinghai Lu <yinghai@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* iommu/vt-d: Fix get_domain_for_dev() handling of upstream PCIe bridgesDavid Woodhouse2014-04-141-1/+3
| | | | | | | | | | | | | | | | Commit 146922ec79 ("iommu/vt-d: Make get_domain_for_dev() take struct device") introduced new variables bridge_bus and bridge_devfn to identify the upstream PCIe to PCI bridge responsible for the given target device. Leaving the original bus/devfn variables to identify the target device itself, now that it is no longer assumed to be PCI and we can no longer trivially find that information. However, the patch failed to correctly use the new variables in all cases; instead using the as-yet-uninitialised 'bus' and 'devfn' variables. Reported-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: fix memory leakage caused by commit ea8ea46Jiang Liu2014-04-131-2/+4
| | | | | | | | | | | | | | | | | Commit ea8ea46 "iommu/vt-d: Clean up and fix page table clear/free behaviour" introduces possible leakage of DMA page tables due to: for (pte = page_address(pg); !first_pte_in_page(pte); pte++) { if (dma_pte_present(pte) && !dma_pte_superpage(pte)) freelist = dma_pte_list_pagetables(domain, level - 1, pte, freelist); } For the first pte in a page, first_pte_in_page(pte) will always be true, thus dma_pte_list_pagetables() will never be called and leak DMA page tables if level is bigger than 1. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: returning free pointer in get_domain_for_dev()Dan Carpenter2014-03-281-0/+1
| | | | | | | | If we hit this error condition then we want to return a NULL pointer and not a freed variable. Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Include ACPI devices in iommu=ptDavid Woodhouse2014-03-241-13/+48
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Finally enable translation for non-PCI devicesDavid Woodhouse2014-03-241-3/+0
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Remove to_pci_dev() in intel_map_page()David Woodhouse2014-03-241-1/+1
| | | | | | It might not be... Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Remove pdev from intel_iommu_attach_device()David Woodhouse2014-03-241-3/+2
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Remove pdev from iommu_no_mapping()David Woodhouse2014-03-241-7/+5
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Make domain_add_dev_info() take struct deviceDavid Woodhouse2014-03-241-9/+8
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Make domain_remove_one_dev_info() take struct deviceDavid Woodhouse2014-03-241-14/+11
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Rename 'hwdev' variables to 'dev' now that that's the normDavid Woodhouse2014-03-241-22/+22
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Remove some pointless to_pci_dev() callsDavid Woodhouse2014-03-241-8/+4
| | | | | | | Mostly made redundant by using dev_name() instead of pci_name(), and one instance of using *dev->dma_mask instead of pdev->dma_mask. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Make get_valid_domain_for_dev() take struct deviceDavid Woodhouse2014-03-241-14/+12
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Make iommu_should_identity_map() take struct deviceDavid Woodhouse2014-03-241-50/+57
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Handle RMRRs for non-PCI devicesDavid Woodhouse2014-03-241-16/+14
| | | | | | | Should hopefully never happen (RMRRs are an abomination) but while we're busy eliminating all the PCI assumptions, we might as well do it. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Make get_domain_for_dev() take struct deviceDavid Woodhouse2014-03-241-39/+36
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Make domain_context_mapp{ed,ing}() take struct deviceDavid Woodhouse2014-03-241-14/+17
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Make device_to_iommu() cope with non-PCI devicesDavid Woodhouse2014-03-241-32/+46
| | | | | | | Pass the struct device to it, and also make it return the bus/devfn to use, since that is also stored in the DMAR table. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Make identity_mapping() take struct device not struct pci_devDavid Woodhouse2014-03-241-3/+3
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Remove segment from struct device_domain_info()David Woodhouse2014-03-241-10/+7
| | | | | | It's accessible via info->iommu->segment so this is redundant. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Remove device_to_iommu() call from domain_remove_dev_info()David Woodhouse2014-03-241-5/+3
| | | | | | | | | This was problematic because it works by domain/bus/devfn and we want to make device_to_iommu() use only a struct device * (for handling non-PCI devices). Now that the iommu pointer is reliably stored in the device_domain_info, we don't need to look it up. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Simplify iommu check in domain_remove_one_dev_info()David Woodhouse2014-03-241-2/+1
| | | | | | | Now we store the iommu in the device_domain_info, we don't need to do a lookup. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Always store iommu in device_domain_infoDavid Woodhouse2014-03-241-16/+31
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Use domain_remove_one_dev_info() in domain_add_dev_info() error pathDavid Woodhouse2014-03-241-7/+1
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: use dmar_insert_dev_info() from dma_add_dev_info()David Woodhouse2014-03-241-15/+7
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Stop dmar_insert_dev_info() freeing domains on losing raceDavid Woodhouse2014-03-241-24/+21
| | | | | | | | By moving this into get_domain_for_dev() we can make dmar_insert_dev_info() suitable for use with "special" domains such as the si_domain, which currently use domain_add_dev_info(). Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Pass iommu to domain_context_mapping_one() and ↵David Woodhouse2014-03-241-18/+17
| | | | | | iommu_support_dev_iotlb() Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Use struct device in device_domain_info, not struct pci_devDavid Woodhouse2014-03-241-22/+32
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Make dmar_insert_dev_info() take struct device instead of struct ↵David Woodhouse2014-03-241-11/+10
| | | | | | pci_dev Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Make iommu_dummy() take struct device instead of struct pci_devDavid Woodhouse2014-03-241-5/+5
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Change scope lists to struct device, bus, devfnDavid Woodhouse2014-03-241-18/+25
| | | | | | | | | | | | It's not only for PCI devices any more, and the scope information for an ACPI device provides the bus and devfn so that has to be stored here too. It is the device pointer itself which needs to be protected with RCU, so the __rcu annotation follows it into the definition of struct dmar_dev_scope, since we're no longer just passing arrays of device pointers around. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Be less pessimistic about domain coherency where possibleDavid Woodhouse2014-03-191-4/+17
| | | | | | | | | | | | | In commit 2e12bc29 ("intel-iommu: Default to non-coherent for domains unattached to iommus") we decided to err on the side of caution and always assume that it's possible that a device will be attached which is behind a non-coherent IOMMU. In some cases, however, that just *cannot* happen. If there *are* no IOMMUs in the system which are non-coherent, then we don't need to do it. And flushing the dcache is a *significant* performance hit. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Honour intel_iommu=sp_off for non-VMM domainsDavid Woodhouse2014-03-191-1/+5
| | | | Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Clean up and fix page table clear/free behaviourDavid Woodhouse2014-03-191-45/+187
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a race condition between the existing clear/free code and the hardware. The IOMMU is actually permitted to cache the intermediate levels of the page tables, and doesn't need to walk the table from the very top of the PGD each time. So the existing back-to-back calls to dma_pte_clear_range() and dma_pte_free_pagetable() can lead to a use-after-free where the IOMMU reads from a freed page table. When freeing page tables we actually need to do the IOTLB flush, with the 'invalidation hint' bit clear to indicate that it's not just a leaf-node flush, after unlinking each page table page from the next level up but before actually freeing it. So in the rewritten domain_unmap() we just return a list of pages (using pg->freelist to make a list of them), and then the caller is expected to do the appropriate IOTLB flush (or tear down the domain completely, whatever), before finally calling dma_free_pagelist() to free the pages. As an added bonus, we no longer need to flush the CPU's data cache for pages which are about to be *removed* from the page table hierarchy anyway, in the non-cache-coherent case. This drastically improves the performance of large unmaps. As a side-effect of all these changes, this also fixes the fact that intel_iommu_unmap() was neglecting to free the page tables for the range in question after clearing them. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Clean up size handling for intel_iommu_unmap()David Woodhouse2014-03-191-12/+25
| | | | | | | | | | | | | | | We have this horrid API where iommu_unmap() can unmap more than it's asked to, if the IOVA in question happens to be mapped with a large page. Instead of propagating this nonsense to the point where we end up returning the page order from dma_pte_clear_range(), let's just do it once and adjust the 'size' parameter accordingly. Augment pfn_to_dma_pte() to return the level at which the PTE was found, which will also be useful later if we end up changing the API for iommu_iova_to_phys() to behave the same way as is being discussed upstream. Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
* iommu/vt-d: Update IOMMU state when memory hotplug happensJiang Liu2014-03-041-1/+70
| | | | | | | | | If static identity domain is created, IOMMU driver needs to update si_domain page table when memory hotplug event happens. Otherwise PCI device DMA operations can't access the hot-added memory regions. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: Joerg Roedel <joro@8bytes.org>
* iommu/vt-d: Unify the way to process DMAR device scope arrayJiang Liu2014-03-041-53/+18
| | | | | | | | | | Now we have a PCI bus notification based mechanism to update DMAR device scope array, we could extend the mechanism to support boot time initialization too, which will help to unify and simplify the implementation. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: Joerg Roedel <joro@8bytes.org>
* iommu/vt-d: Update DRHD/RMRR/ATSR device scope caches when PCI hotplug happensJiang Liu2014-03-041-0/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current Intel DMAR/IOMMU driver assumes that all PCI devices associated with DMAR/RMRR/ATSR device scope arrays are created at boot time and won't change at runtime, so it caches pointers of associated PCI device object. That assumption may be wrong now due to: 1) introduction of PCI host bridge hotplug 2) PCI device hotplug through sysfs interfaces. Wang Yijing has tried to solve this issue by caching <bus, dev, func> tupple instead of the PCI device object pointer, but that's still unreliable because PCI bus number may change in case of hotplug. Please refer to http://lkml.org/lkml/2013/11/5/64 Message from Yingjing's mail: after remove and rescan a pci device [ 611.857095] dmar: DRHD: handling fault status reg 2 [ 611.857109] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff7000 [ 611.857109] DMAR:[fault reason 02] Present bit in context entry is clear [ 611.857524] dmar: DRHD: handling fault status reg 102 [ 611.857534] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff6000 [ 611.857534] DMAR:[fault reason 02] Present bit in context entry is clear [ 611.857936] dmar: DRHD: handling fault status reg 202 [ 611.857947] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff5000 [ 611.857947] DMAR:[fault reason 02] Present bit in context entry is clear [ 611.858351] dmar: DRHD: handling fault status reg 302 [ 611.858362] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff4000 [ 611.858362] DMAR:[fault reason 02] Present bit in context entry is clear [ 611.860819] IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready [ 611.860983] dmar: DRHD: handling fault status reg 402 [ 611.860995] dmar: INTR-REMAP: Request device [[86:00.3] fault index a4 [ 611.860995] INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear This patch introduces a new mechanism to update the DRHD/RMRR/ATSR device scope caches by hooking PCI bus notification. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: Joerg Roedel <joro@8bytes.org>
* iommu/vt-d: Use RCU to protect global resources in interrupt contextJiang Liu2014-03-041-4/+16
| | | | | | | | | Global DMA and interrupt remapping resources may be accessed in interrupt context, so use RCU instead of rwsem to protect them in such cases. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Signed-off-by: Joerg Roedel <joro@8bytes.org>
OpenPOWER on IntegriCloud