summaryrefslogtreecommitdiffstats
path: root/arch/mips/mm
Commit message (Collapse)AuthorAgeFilesLines
...
* | MIPS: Retrieve ASID masks using function accepting struct cpuinfo_mipsPaul Burton2016-05-133-12/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In preparation for supporting variable ASID masks, retrieve ASID masks using functions in asm/cpu-info.h which accept struct cpuinfo_mips. This will allow those functions to determine the ASID mask based upon the CPU in a later patch. This also allows for the r3k & r8k cases to be handled in Kconfig, which is arguably cleaner than the previous #ifdefs. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/13210/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: remove aliasing alignment if HW has antialising supportLeonid Yegoshin2016-05-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MIPS hardware may have an antialising support and it works even page size is small. Setup a shared memory aliasing mask to page size if hardware has an antialising support. Big shared memory mask forces a disruption in page address assignment and that corrupts Android library memory handling. Signed-off-by: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Cc: cernekee@gmail.com Cc: paul.gortmaker@windriver.com Cc: kumba@gentoo.org Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/11516/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: Loongson-3: Introduce CONFIG_LOONGSON3_ENHANCEMENTHuacai Chen2016-05-132-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | New Loongson 3 CPU (since Loongson-3A R2, as opposed to Loongson-3A R1, Loongson-3B R1 and Loongson-3B R2) has many enhancements, such as FTLB, L1-VCache, EI/DI/Wait/Prefetch instruction, DSP/DSPv2 ASE, User Local register, Read-Inhibit/Execute-Inhibit, SFB (Store Fill Buffer), Fast TLB refill support, etc. This patch introduce a config option, CONFIG_LOONGSON3_ENHANCEMENT, to enable those enhancements which are not probed at run time. If you want a generic kernel to run on all Loongson 3 machines, please say 'N' here. If you want a high-performance kernel to run on new Loongson 3 machines only, please say 'Y' here. Some additional explanations: 1) SFB locates between core and L1 cache, it causes memory access out of order, so writel/outl (and other similar functions) need a I/O reorder barrier. 2) Loongson 3 has a bug that di instruction can not save the irqflag, so arch_local_irq_save() is modified. Since CPU_MIPSR2 is selected by CONFIG_LOONGSON3_ENHANCEMENT, generic kernel doesn't use ei/di at all. 3) CPU_HAS_PREFETCH is selected by CONFIG_LOONGSON3_ENHANCEMENT, so MIPS_CPU_PREFETCH (used by uasm) probing is also put in this patch. Signed-off-by: Huacai Chen <chenhc@lemote.com> Cc: Aurelien Jarno <aurelien@aurel32.net> Cc: Steven J . Hill <sjhill@realitydiluted.com> Cc: Fuxin Zhang <zhangfx@lemote.com> Cc: Zhangjin Wu <wuzhangjin@gmail.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/12755/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: Loongson-3: Fast TLB refill handlerHuacai Chen2016-05-133-3/+126
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Loongson-3A R2 has pwbase/pwfield/pwsize/pwctl registers in CP0 (this is very similar to HTW) and lwdir/lwpte/lddir/ldpte instructions which can be used for fast TLB refill. [ralf@linux-mips.org: Resolve conflict.] Signed-off-by: Huacai Chen <chenhc@lemote.com> Cc: Aurelien Jarno <aurelien@aurel32.net> Cc: Steven J . Hill <sjhill@realitydiluted.com> Cc: Fuxin Zhang <zhangfx@lemote.com> Cc: Zhangjin Wu <wuzhangjin@gmail.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/12754/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: Loongson: Invalidate special TLBs when neededHuacai Chen2016-05-131-12/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Loongson-2 has a 4 entry itlb which is a subset of jtlb, Loongson-3 has a 4 entry itlb and a 4 entry dtlb which are subsets of jtlb. We should write diag register to invalidate itlb/dtlb when flushing jtlb because itlb/dtlb are not totally transparent to software. For Loongson-3A R2 (and newer), we should invalidate ITLB, DTLB, VTLB and FTLB before we enable/disable FTLB. Signed-off-by: Huacai Chen <chenhc@lemote.com> Cc: Aurelien Jarno <aurelien@aurel32.net> Cc: Steven J . Hill <sjhill@realitydiluted.com> Cc: Fuxin Zhang <zhangfx@lemote.com> Cc: Zhangjin Wu <wuzhangjin@gmail.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/12753/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: Loongson-3: Set cache flush handlers to cache_noopHuacai Chen2016-05-131-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Loongson-3 maintains cache coherency by hardware, this means: 1) It's icache is coherent with dcache. 2) It's dcaches don't alias (maybe depend on PAGE_SIZE). 3) It maintains cache coherency across cores (and for DMA). So we can skip most cache flush operations by setting relevant handlers to `cache_noop' in `r4k_cache_init'. Signed-off-by: Huacai Chen <chenhc@lemote.com> Cc: Aurelien Jarno <aurelien@aurel32.net> Cc: Steven J . Hill <sjhill@realitydiluted.com> Cc: Fuxin Zhang <zhangfx@lemote.com> Cc: Zhangjin Wu <wuzhangjin@gmail.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/12752/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: Loongson: Add Loongson-3A R2 basic supportHuacai Chen2016-05-131-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Loongson-3 CPU family: Code-name Brand-name PRId Loongson-3A R1 Loongson-3A1000 0x6305 Loongson-3A R2 Loongson-3A2000 0x6308 Loongson-3B R1 Loongson-3B1000 0x6306 Loongson-3B R2 Loongson-3B1500 0x6307 Features of R2 revision of Loongson-3A: - Primary cache includes I-Cache, D-Cache and V-Cache (Victim Cache). - I-Cache, D-Cache and V-Cache are 16-way set-associative, linesize is 64 bytes. - 64 entries of VTLB (classic TLB), 1024 entries of FTLB (8-way set-associative). - Supports DSP/DSPv2 instructions, UserLocal register and Read-Inhibit/ Execute-Inhibit. [ralf@linux-mips.org: Resolved merge conflicts.] Signed-off-by: Huacai Chen <chenhc@lemote.com> Cc: Aurelien Jarno <aurelien@aurel32.net> Cc: Steven J . Hill <sjhill@realitydiluted.com> Cc: Fuxin Zhang <zhangfx@lemote.com> Cc: Zhangjin Wu <wuzhangjin@gmail.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/12751/ Patchwork: https://patchwork.linux-mips.org/patch/13136/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: BMIPS: local_r4k___flush_cache_all needs to blast S-cacheFlorian Fainelli2016-05-131-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | local_r4k___flush_cache_all() is missing a special check for BMIPS5000 processors, we need to blast the S-cache, just like other MTI processors since we have an inclusive cache. We also need an additional __sync() to make sure this is completed. Fixes: d74b0172e4e2c ("MIPS: BMIPS: Add special cache handling in c-r4k.c") Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/13012/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: BMIPS: Clear MIPS_CACHE_ALIASES earlierFlorian Fainelli2016-05-131-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | BMIPS5000 and BMIPS5200 processor have no D cache aliases, and this is properly handled by the per-CPU override added at the end of r4k_cache_init(), the problem is that the output of probe_pcache() disagrees with that, since this is too late: Primary instruction cache 32kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, VIPT, cache aliases, linesize 32 bytes With the change moved earlier, we now have a consistent output with the settings we are intending to have: Primary instruction cache 32kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, VIPT, no aliases, linesize 32 bytes Fixes: d74b0172e4e2c ("MIPS: BMIPS: Add special cache handling in c-r4k.c") Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/13011/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: BMIPS: BMIPS5000 has I cache filing from D cacheFlorian Fainelli2016-05-131-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | BMIPS5000 and BMIPS52000 processors have their I-cache filling from the D-cache. Since BMIPS_GENERIC does not provide (yet) a cpu-feature-overrides.h file, this was not set anywhere, so make sure the R4K cache detection takes care of that. Fixes: d74b0172e4e2c ("MIPS: BMIPS: Add special cache handling in c-r4k.c") Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/13010/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: Sync icache & dcache in set_pte_atPaul Burton2016-05-131-16/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's possible for pages to become visible prior to update_mmu_cache running if a thread within the same address space preempts the current thread or runs simultaneously on another CPU. That is, the following scenario is possible: CPU0 CPU1 write to page flush_dcache_page flush_icache_page set_pte_at map page update_mmu_cache If CPU1 maps the page in between CPU0's set_pte_at, which marks it valid & visible, and update_mmu_cache where the dcache flush occurs then CPU1s icache will fill from stale data (unless it fills from the dcache, in which case all is good, but most MIPS CPUs don't have this property). Commit 4d46a67a3eb8 ("MIPS: Fix race condition in lazy cache flushing.") attempted to fix that by performing the dcache flush in flush_icache_page such that it occurs before the set_pte_at call makes the page visible. However it has the problem that not all code that writes to pages exposed to userland call flush_icache_page. There are many callers of set_pte_at under mm/ and only 2 of them do call flush_icache_page. Thus the race window between a page becoming visible & being coherent between the icache & dcache remains open in some cases. To illustrate some of the cases, a WARN was added to __update_cache with this patch applied that triggered in cases where a page about to be flushed from the dcache was not the last page provided to flush_icache_page. That is, backtraces were obtained for cases in which the race window is left open without this patch. The 2 standout examples follow. When forking a process: [ 15.271842] [<80417630>] __update_cache+0xcc/0x188 [ 15.277274] [<80530394>] copy_page_range+0x56c/0x6ac [ 15.282861] [<8042936c>] copy_process.part.54+0xd40/0x17ac [ 15.289028] [<80429f80>] do_fork+0xe4/0x420 [ 15.293747] [<80413808>] handle_sys+0x128/0x14c When exec'ing an ELF binary: [ 14.445964] [<80417630>] __update_cache+0xcc/0x188 [ 14.451369] [<80538d88>] move_page_tables+0x414/0x498 [ 14.457075] [<8055d848>] setup_arg_pages+0x220/0x318 [ 14.462685] [<805b0f38>] load_elf_binary+0x530/0x12a0 [ 14.468374] [<8055ec3c>] search_binary_handler+0xbc/0x214 [ 14.474444] [<8055f6c0>] do_execveat_common+0x43c/0x67c [ 14.480324] [<8055f938>] do_execve+0x38/0x44 [ 14.485137] [<80413808>] handle_sys+0x128/0x14c These code paths write into a page, call flush_dcache_page then call set_pte_at without flush_icache_page inbetween. The end result is that the icache can become corrupted & userland processes may execute unexpected or invalid code, typically resulting in a reserved instruction exception, a trap or a segfault. Fix this race condition fully by performing any cache maintenance required to keep the icache & dcache in sync in set_pte_at, before the page is made valid. This has the added bonus of ensuring the cache maintenance always happens in one location, rather than being duplicated in flush_icache_page & update_mmu_cache. It also matches the way other architectures solve the same problem (see arm, ia64 & powerpc). Signed-off-by: Paul Burton <paul.burton@imgtec.com> Reported-by: Ionela Voinescu <ionela.voinescu@imgtec.com> Cc: Lars Persson <lars.persson@axis.com> Fixes: 4d46a67a3eb8 ("MIPS: Fix race condition in lazy cache flushing.") Cc: Steven J. Hill <sjhill@realitydiluted.com> Cc: David Daney <david.daney@cavium.com> Cc: Huacai Chen <chenhc@lemote.com> Cc: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Jerome Marchand <jmarchan@redhat.com> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Cc: stable <stable@vger.kernel.org> # v4.1+ Patchwork: https://patchwork.linux-mips.org/patch/12722/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: Handle highmem pages in __update_cachePaul Burton2016-05-131-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The following patch will expose __update_cache to highmem pages. Handle them by mapping them in for the duration of the cache maintenance, just like in __flush_dcache_page. The code for that isn't shared because we need the page address in __update_cache so sharing became messy. Given that the entirity is an extra 5 lines, just duplicate it. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Lars Persson <lars.persson@axis.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Jerome Marchand <jmarchan@redhat.com> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Cc: stable <stable@vger.kernel.org> # v4.1+ Patchwork: https://patchwork.linux-mips.org/patch/12721/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: Flush highmem pages in __flush_dcache_pagePaul Burton2016-05-131-3/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When flush_dcache_page is called on an executable page, that page is about to be provided to userland & we can presume that the icache contains no valid entries for its address range. However if the icache does not fill from the dcache then we cannot presume that the pages content has been written back as far as the memories that the dcache will fill from (ie. L2 or further out). This was being done for lowmem pages, but not for highmem which can lead to icache corruption. Fix this by mapping highmem pages & flushing their content from the dcache in __flush_dcache_page before providing the page to userland, just as is done for lowmem pages. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Lars Persson <lars.persson@axis.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/12720/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: Add M6250 cases to CPU switch statementsPaul Burton2016-05-131-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add casses supporting the M6250 CPU to various switch statements in the core MIPS kernel code that define behaviour dependent upon the CPU. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Joshua Kinard <kumba@gentoo.org> Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Cc: Paul Gortmaker <paul.gortmaker@windriver.com> Cc: Maciej W. Rozycki <macro@codesourcery.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/12374/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: Add P6600 cases to CPU switch statementsPaul Burton2016-05-132-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add cases supporting the P6600 CPU to various switch statements in core MIPS kernel code that define behaviour dependent upon the CPU. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Maciej W. Rozycki <macro@imgtec.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Joshua Kinard <kumba@gentoo.org> Cc: Andrzej Hajda <a.hajda@samsung.com> Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Cc: Paul Gortmaker <paul.gortmaker@windriver.com> Cc: James Hogan <james.hogan@imgtec.com> Cc: Arnaldo Carvalho de Melo <acme@kernel.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Petri Gynther <pgynther@google.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/12343/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: dma-default: Defend against NULL dev in massage_gfp_flagsMatt Redfearn2016-05-091-3/+4
| | | | | | | | | | | | | | | | | | | | | | This patch ensures that the dev parameter is checked for NULL before it is dereferenced in massage_gfp_flags. If dev is NULL, then fall back setting the GFP flag requested and available. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11919/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: I6400: Icache fills from dcacheJames Hogan2016-05-091-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Coherence Manager 3 (CM3) as present in I6400 can fill icache lines effectively from dirty dcaches, so there is no need to flush dirty lines from dcaches through to L2 prior to icache invalidation. Set the MIPS_CACHE_IC_F_DC flag such that cpu_has_ic_fills_f_dc evaluates to true, which avoids those dcache flushes. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com> Cc: Manuel Lauss <manuel.lauss@gmail.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/12180/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | MIPS: c-r4k: Sync icache when it fills from dcacheJames Hogan2016-05-092-3/+10
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is still necessary to handle icache coherency in flush_cache_range() and copy_to_user_page() when the icache fills from the dcache, even though the dcache does not need to be written back. However when this handling was added in commit 2eaa7ec286db ("[MIPS] Handle I-cache coherency in flush_cache_range()"), it did not do any icache flushing when it fills from dcache. Therefore fix r4k_flush_cache_range() to run local_r4k_flush_cache_range() without taking into account whether icache fills from dcache, so that the icache coherency gets handled. Checks are also added in local_r4k_flush_cache_range() so that the dcache blast doesn't take place when icache fills from dcache. A test to mmap a page PROT_READ|PROT_WRITE, modify code in it, and mprotect it to VM_READ|VM_EXEC (similar to case described in above commit) can hit this case quite easily to verify the fix. A similar check was added in commit f8829caee311 ("[MIPS] Fix aliasing bug in copy_to_user_page / copy_from_user_page"), so also fix copy_to_user_page() similarly, to call flush_cache_page() without taking into account whether icache fills from dcache, since flush_cache_page() already takes that into account to avoid performing a dcache flush. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com> Cc: Manuel Lauss <manuel.lauss@gmail.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/12179/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: Fix misspellings in comments.Adam Buchbinder2016-04-032-2/+2
| | | | | | | | Signed-off-by: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: linux-mips@linux-mips.org Cc: trivial@kernel.org Patchwork: https://patchwork.linux-mips.org/patch/12617/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: tlb-r4k: panic if the MMU doesn't support PAGE_SIZEPaul Burton2016-04-031-0/+5
| | | | | | | | | | | | | | | | | | | After writing the appropriate mask to the cop0 PageMask register, read the register back & check it matches what we want. If it doesn't then the MMU does not support the page size the kernel is configured for and we're better off bailing than continuing to do odd things with TLB exceptions. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Steven J. Hill <Steven.Hill@imgtec.com> Cc: Joshua Kinard <kumba@gentoo.org> Cc: Rafał Miłecki <zajec5@gmail.com> Cc: James Hogan <james.hogan@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/10691/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* Merge branch 'mm-pkeys-for-linus' of ↵Linus Torvalds2016-03-201-2/+1
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip Pull x86 protection key support from Ingo Molnar: "This tree adds support for a new memory protection hardware feature that is available in upcoming Intel CPUs: 'protection keys' (pkeys). There's a background article at LWN.net: https://lwn.net/Articles/643797/ The gist is that protection keys allow the encoding of user-controllable permission masks in the pte. So instead of having a fixed protection mask in the pte (which needs a system call to change and works on a per page basis), the user can map a (handful of) protection mask variants and can change the masks runtime relatively cheaply, without having to change every single page in the affected virtual memory range. This allows the dynamic switching of the protection bits of large amounts of virtual memory, via user-space instructions. It also allows more precise control of MMU permission bits: for example the executable bit is separate from the read bit (see more about that below). This tree adds the MM infrastructure and low level x86 glue needed for that, plus it adds a high level API to make use of protection keys - if a user-space application calls: mmap(..., PROT_EXEC); or mprotect(ptr, sz, PROT_EXEC); (note PROT_EXEC-only, without PROT_READ/WRITE), the kernel will notice this special case, and will set a special protection key on this memory range. It also sets the appropriate bits in the Protection Keys User Rights (PKRU) register so that the memory becomes unreadable and unwritable. So using protection keys the kernel is able to implement 'true' PROT_EXEC on x86 CPUs: without protection keys PROT_EXEC implies PROT_READ as well. Unreadable executable mappings have security advantages: they cannot be read via information leaks to figure out ASLR details, nor can they be scanned for ROP gadgets - and they cannot be used by exploits for data purposes either. We know about no user-space code that relies on pure PROT_EXEC mappings today, but binary loaders could start making use of this new feature to map binaries and libraries in a more secure fashion. There is other pending pkeys work that offers more high level system call APIs to manage protection keys - but those are not part of this pull request. Right now there's a Kconfig that controls this feature (CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) that is default enabled (like most x86 CPU feature enablement code that has no runtime overhead), but it's not user-configurable at the moment. If there's any serious problem with this then we can make it configurable and/or flip the default" * 'mm-pkeys-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (38 commits) x86/mm/pkeys: Fix mismerge of protection keys CPUID bits mm/pkeys: Fix siginfo ABI breakage caused by new u64 field x86/mm/pkeys: Fix access_error() denial of writes to write-only VMA mm/core, x86/mm/pkeys: Add execute-only protection keys support x86/mm/pkeys: Create an x86 arch_calc_vm_prot_bits() for VMA flags x86/mm/pkeys: Allow kernel to modify user pkey rights register x86/fpu: Allow setting of XSAVE state x86/mm: Factor out LDT init from context init mm/core, x86/mm/pkeys: Add arch_validate_pkey() mm/core, arch, powerpc: Pass a protection key in to calc_vm_flag_bits() x86/mm/pkeys: Actually enable Memory Protection Keys in the CPU x86/mm/pkeys: Add Kconfig prompt to existing config option x86/mm/pkeys: Dump pkey from VMA in /proc/pid/smaps x86/mm/pkeys: Dump PKRU with other kernel registers mm/core, x86/mm/pkeys: Differentiate instruction fetches x86/mm/pkeys: Optimize fault handling in access_error() mm/core: Do not enforce PKEY permissions on remote mm access um, pkeys: Add UML arch_*_access_permitted() methods mm/gup, x86/mm/pkeys: Check VMAs and PTEs for protection keys x86/mm/gup: Simplify get_user_pages() PTE bit handling ...
| * mm/gup: Switch all callers of get_user_pages() to not pass tsk/mmDave Hansen2016-02-161-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We will soon modify the vanilla get_user_pages() so it can no longer be used on mm/tasks other than 'current/current->mm', which is by far the most common way it is called. For now, we allow the old-style calls, but warn when they are used. (implemented in previous patch) This patch switches all callers of: get_user_pages() get_user_pages_unlocked() get_user_pages_locked() to stop passing tsk/mm so they will no longer see the warnings. Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave@sr71.net> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: jack@suse.cz Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/20160212210156.113E9407@viggo.jf.intel.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
* | mm: introduce page reference manipulation functionsJoonsoo Kim2016-03-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The success of CMA allocation largely depends on the success of migration and key factor of it is page reference count. Until now, page reference is manipulated by direct calling atomic functions so we cannot follow up who and where manipulate it. Then, it is hard to find actual reason of CMA allocation failure. CMA allocation should be guaranteed to succeed so finding offending place is really important. In this patch, call sites where page reference is manipulated are converted to introduced wrapper function. This is preparation step to add tracepoint to each page reference manipulation function. With this facility, we can easily find reason of CMA allocation failure. There is no functional change in this patch. In addition, this patch also converts reference read sites. It will help a second step that renames page._count to something else and prevents later attempt to direct access to it (Suggested by Andrew). Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com> Acked-by: Michal Nazarewicz <mina86@mina86.com> Acked-by: Vlastimil Babka <vbabka@suse.cz> Cc: Minchan Kim <minchan@kernel.org> Cc: Mel Gorman <mgorman@techsingularity.net> Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Cc: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> Cc: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* | MIPS: scache: Fix scache init with invalid line size.Govindraj Raja2016-02-291-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In current scache init cache line_size is determined from cpu config register, however if there there no scache then mips_sc_probe_cm3 function populates a invalid line_size of 2. The invalid line_size can cause a NULL pointer deference during r4k_dma_cache_inv as r4k_blast_scache is populated based on line_size. Scache line_size of 2 is invalid option in r4k_blast_scache_setup. This issue was faced during a MIPS I6400 based virtual platform bring up where scache was not available in virtual platform model. Signed-off-by: Govindraj Raja <Govindraj.Raja@imgtec.com> Fixes: 7d53e9c4cd21("MIPS: CM3: Add support for CM3 L2 cache.") Cc: Paul Burton <paul.burton@imgtec.com> Cc: James Hogan <james.hogan@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: James Hartley <James.Hartley@imgtec.com> Cc: linux-mips@linux-mips.org Cc: stable@vger.kernel.org # v4.2+ Patchwork: https://patchwork.linux-mips.org/patch/12710/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | mm: ASLR: use get_random_long()Daniel Cashman2016-02-271-2/+2
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace calls to get_random_int() followed by a cast to (unsigned long) with calls to get_random_long(). Also address shifting bug which, in case of x86 removed entropy mask for mmap_rnd_bits values > 31 bits. Signed-off-by: Daniel Cashman <dcashman@android.com> Acked-by: Kees Cook <keescook@chromium.org> Cc: "Theodore Ts'o" <tytso@mit.edu> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: David S. Miller <davem@davemloft.net> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Nick Kralevich <nnk@google.com> Cc: Jeff Vander Stoep <jeffv@google.com> Cc: Mark Salyzyn <salyzyn@android.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* MIPS: Fix early CM probingPaul Burton2016-02-091-10/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit c014d164f21d ("MIPS: Add platform callback before initializing the L2 cache") added a platform_early_l2_init function in order to allow platforms to probe for the CM before L2 initialisation is performed, so that CM GCRs are available to mips_sc_probe. That commit actually fails to do anything useful, since it checks mips_cm_revision to determine whether it should call mips_cm_probe but the result of mips_cm_revision will always be 0 until mips_cm_probe has been called. Thus the "early" mips_cm_probe call never occurs. Fix this & drop the useless weak platform_early_l2_init function by simply calling mips_cm_probe from setup_arch. For platforms that don't select CONFIG_MIPS_CM this will be a no-op, and for those that do it removes the requirement for them to call mips_cm_probe manually (although doing so isn't harmful for now). Signed-off-by: Paul Burton <paul.burton@imgtec.com> Reviewed-by: Alexander Sverdlin <alexander.sverdlin@nokia.com> Cc: Andrzej Hajda <a.hajda@samsung.com> Cc: Aaro Koskinen <aaro.koskinen@nokia.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Rob Herring <robh@kernel.org> Cc: Peter Hurley <peter@hurleysoftware.com> Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Cc: Jaedon Shin <jaedon.shin@gmail.com> Cc: James Hogan <james.hogan@imgtec.com> Cc: Jonas Gorski <jogo@openwrt.org> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/12475/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* Merge branch 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linusLinus Torvalds2016-01-241-1/+1
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Pull MIPS updates from Ralf Baechle: "This is the main pull request for MIPS for 4.5 plus some 4.4 fixes. The executive summary: - ATH79 platform improvments, use DT bindings for the ATH79 USB PHY. - Avoid useless rebuilds for zboot. - jz4780: Add NEMC, BCH and NAND device tree nodes - Initial support for the MicroChip's DT platform. As all the device drivers are missing this is still of limited use. - Some Loongson3 cleanups. - The unavoidable whitespace polishing. - Reduce clock skew when synchronizing the CPU cycle counters on CPU startup. - Add MIPS R6 fixes. - Lots of cleanups across arch/mips as fallout from KVM. - Lots of minor fixes and changes for IEEE 754-2008 support to the FPU emulator / fp-assist software. - Minor Ralink, BCM47xx and bcm963xx platform support improvments. - Support SMP on BCM63168" * 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus: (84 commits) MIPS: zboot: Add support for serial debug using the PROM MIPS: zboot: Avoid useless rebuilds MIPS: BMIPS: Enable ARCH_WANT_OPTIONAL_GPIOLIB MIPS: bcm63xx: nvram: Remove unused bcm63xx_nvram_get_psi_size() function MIPS: bcm963xx: Update bcm_tag field image_sequence MIPS: bcm963xx: Move extended flash address to bcm_tag header file MIPS: bcm963xx: Move Broadcom BCM963xx image tag data structure MIPS: bcm63xx: nvram: Use nvram structure definition from header file MIPS: bcm963xx: Add Broadcom BCM963xx board nvram data structure MAINTAINERS: Add KVM for MIPS entry MIPS: KVM: Add missing newline to kvm_err() MIPS: Move KVM specific opcodes into asm/inst.h MIPS: KVM: Use cacheops.h definitions MIPS: Break down cacheops.h definitions MIPS: Use EXCCODE_ constants with set_except_vector() MIPS: Update trap codes MIPS: Move Cause.ExcCode trap codes to mipsregs.h MIPS: KVM: Make kvm_mips_{init,exit}() static MIPS: KVM: Refactor added offsetof()s MIPS: KVM: Convert EXPORT_SYMBOL to _GPL ...
| * MIPS: Fix some missing CONFIG_CPU_MIPSR6 #ifdefsHuacai Chen2016-01-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit be0c37c985eddc4 (MIPS: Rearrange PTE bits into fixed positions.) defines fixed PTE bits for MIPS R2. Then, commit d7b631419b3d230a4d383 (MIPS: pgtable-bits: Fix XPA damage to R6 definitions.) adds the MIPS R6 definitions in the same way as MIPS R2. But some R6 #ifdefs in the later commit are missing, so in this patch I fix that. Signed-off-by: Huacai Chen <chenhc@lemote.com> Cc: Aurelien Jarno <aurelien@aurel32.net> Cc: Steven J. Hill <Steven.Hill@imgtec.com> Cc: Fuxin Zhang <zhangfx@lemote.com> Cc: Zhangjin Wu <wuzhangjin@gmail.com> Cc: linux-mips@linux-mips.org Cc: stable@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/12164/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | mm: differentiate page_mapped() from page_mapcount() for compound pagesKirill A. Shutemov2016-01-153-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Let's define page_mapped() to be true for compound pages if any sub-pages of the compound page is mapped (with PMD or PTE). On other hand page_mapcount() return mapcount for this particular small page. This will make cases like page_get_anon_vma() behave correctly once we allow huge pages to be mapped with PTE. Most users outside core-mm should use page_mapcount() instead of page_mapped(). Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Tested-by: Sasha Levin <sasha.levin@oracle.com> Tested-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> Acked-by: Jerome Marchand <jmarchan@redhat.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Hugh Dickins <hughd@google.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Rik van Riel <riel@redhat.com> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Steve Capper <steve.capper@linaro.org> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@suse.cz> Cc: Christoph Lameter <cl@linux.com> Cc: David Rientjes <rientjes@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* | mips, thp: remove infrastructure for handling splitting PMDsKirill A. Shutemov2016-01-153-27/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With new refcounting we don't need to mark PMDs splitting. Let's drop code to handle this. pmdp_splitting_flush() is not needed too: on splitting PMD we will do pmdp_clear_flush() + set_pte_at(). pmdp_clear_flush() will do IPI as needed for fast_gup. Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> Cc: Jerome Marchand <jmarchan@redhat.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Hugh Dickins <hughd@google.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Rik van Riel <riel@redhat.com> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Steve Capper <steve.capper@linaro.org> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@suse.cz> Cc: Christoph Lameter <cl@linux.com> Cc: David Rientjes <rientjes@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* | mm: drop tail page refcountingKirill A. Shutemov2016-01-151-4/+0
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Tail page refcounting is utterly complicated and painful to support. It uses ->_mapcount on tail pages to store how many times this page is pinned. get_page() bumps ->_mapcount on tail page in addition to ->_count on head. This information is required by split_huge_page() to be able to distribute pins from head of compound page to tails during the split. We will need ->_mapcount to account PTE mappings of subpages of the compound page. We eliminate need in current meaning of ->_mapcount in tail pages by forbidding split entirely if the page is pinned. The only user of tail page refcounting is THP which is marked BROKEN for now. Let's drop all this mess. It makes get_page() and put_page() much simpler. Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Tested-by: Sasha Levin <sasha.levin@oracle.com> Tested-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> Acked-by: Vlastimil Babka <vbabka@suse.cz> Acked-by: Jerome Marchand <jmarchan@redhat.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Hugh Dickins <hughd@google.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Rik van Riel <riel@redhat.com> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Steve Capper <steve.capper@linaro.org> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@suse.cz> Cc: Christoph Lameter <cl@linux.com> Cc: David Rientjes <rientjes@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* MIPS: fix DMA contiguous allocationQais Yousef2015-12-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Recent changes to how GFP_ATOMIC is defined seems to have broken the condition to use mips_alloc_from_contiguous() in mips_dma_alloc_coherent(). I couldn't bottom out the exact change but I think it's this commit d0164adc89f6 ("mm, page_alloc: distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd"). GFP_ATOMIC has multiple bits set and the check for !(gfp & GFP_ATOMIC) isn't enough. The reason behind this condition is to check whether we can potentially do a sleeping memory allocation. Use gfpflags_allow_blocking() instead which should be more robust. Signed-off-by: Qais Yousef <qais.yousef@imgtec.com> Acked-by: Mel Gorman <mgorman@techsingularity.net> Cc: Ralf Baechle <ralf@linux-mips.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* Merge branch 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linusLinus Torvalds2015-11-154-38/+206
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Pull MIPS updates from Ralf Baechle: "These are the highlists of the main MIPS pull request for 4.4: - Add latencytop support - Support appended DTBs - VDSO support and initially use it for gettimeofday. - Drop the .MIPS.abiflags and ELF NOTE sections from vmlinux - Support for the 5KE, an internal test core. - Switch all MIPS platfroms to libata drivers. - Improved support, cleanups for ralink and Lantiq platforms. - Support for the new xilfpga platform. - A number of DTB improvments for BMIPS. - Improved support for CM and CPS. - Minor JZ4740 and BCM47xx enhancements" * 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus: (120 commits) MIPS: idle: add case for CPU_5KE MIPS: Octeon: Support APPENDED_DTB MIPS: vmlinux: create a section for appended DTB MIPS: Clean up compat_siginfo_t MIPS: Fix PAGE_MASK definition MIPS: BMIPS: Enable GZIP ramdisk and timed printks MIPS: Add xilfpga defconfig MIPS: xilfpga: Add mipsfpga platform code MIPS: xilfpga: Add xilfpga device tree files. dt-bindings: MIPS: Document xilfpga bindings and boot style MIPS: Make MIPS_CMDLINE_DTB default MIPS: Make the kernel arguments from dtb available MIPS: Use USE_OF as the guard for appended dtb MIPS: BCM63XX: Use pr_* instead of printk MIPS: Loongson: Cleanup CONFIG_LOONGSON_SUSPEND. MIPS: lantiq: Disable xbar fpi burst mode MIPS: lantiq: Force the crossbar to big endian MIPS: lantiq: Initialize the USB core on boot MIPS: lantiq: Return correct value for fpi clock on ar9 MIPS: ralink: Add missing clock on rt305x ...
| * MIPS: Extend hardware table walking support to MIPS64Paul Burton2015-11-111-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Extend the existing support for Hardware Table Walking (HTW) to MIPS64 systems by supporting PMDs & setting the pointer size bit in PWSize, then ceasing to blacklist HTW on MIPS64 systems. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Cc: Steven J. Hill <Steven.Hill@imgtec.com> Cc: Joshua Kinard <kumba@gentoo.org> Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Cc: Maciej W. Rozycki <macro@linux-mips.org> Cc: Paul Gortmaker <paul.gortmaker@windriver.com> Cc: linux-kernel@vger.kernel.org Cc: James Hogan <james.hogan@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Patchwork: https://patchwork.linux-mips.org/patch/11224/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
| * MIPS: tlbex: Avoid placing software PTE bits in Entry* PFN fieldsPaul Burton2015-11-111-2/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 748e787eb6de ("MIPS: Optimize TLB refill for RI/XI configurations.") stopped explicitly clearing the bits used by software in PTEs by making use of a rotate instruction that rotates them into the fill bits of the Entry{Lo,Hi} register. This can only work if there are actually enough fill bits in the register to cover the software maintained bits, otherwise we end up writing those bits into the upper bits of the PFN or PFNX field of the Entry{Lo,Hi} register. Fix this by detecting the number of fill bits present in the Entry{Lo,Hi} registers & explicitly clearing the software bits where necessary. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Cc: Steven J. Hill <Steven.Hill@imgtec.com> Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Cc: Paul Gortmaker <paul.gortmaker@windriver.com> Cc: linux-kernel@vger.kernel.org Cc: James Hogan <james.hogan@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Patchwork: https://patchwork.linux-mips.org/patch/11218/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
| * MIPS: tlbex: Share MIPS32 32 bit phys & MIPS64 64 bit phys codePaul Burton2015-11-111-11/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code in build_update_entries for 64 bit physical addresses on a MIPS64 CPU and 32 bit physical addresses on a MIPS32 CPU is now identical, with the exception of r4k bug workaround in the latter which would simply not apply to the former. Remove the duplication and some Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Cc: Steven J. Hill <Steven.Hill@imgtec.com> Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Cc: linux-kernel@vger.kernel.org Cc: James Hogan <james.hogan@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Patchwork: https://patchwork.linux-mips.org/patch/11216/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
| * MIPS: tlbex: Remove some RIXI redundancyPaul Burton2015-11-111-24/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The cpu_has_rixi cases in build_update_entries are now identical to the non-RIXI cases with the one exception of the r45k_bvahwbug case which is hardcoded as never happening anyway & presumably was either missed from the RIXI path or would never happen on a CPU with RIXI support. Remove the redundant checks & duplication. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Cc: Steven J. Hill <Steven.Hill@imgtec.com> Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Cc: Paul Gortmaker <paul.gortmaker@windriver.com> Cc: linux-kernel@vger.kernel.org Cc: James Hogan <james.hogan@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Patchwork: https://patchwork.linux-mips.org/patch/11215/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
| * MIPS: tlbex: Stop open-coding build_convert_pte_to_entryloPaul Burton2015-11-111-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make use of build_convert_pte_to_entrylo in the RIXI cases within build_update_entries rather than open-coding it 4 times. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Cc: Steven J. Hill <Steven.Hill@imgtec.com> Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Cc: linux-kernel@vger.kernel.org Cc: James Hogan <james.hogan@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Patchwork: https://patchwork.linux-mips.org/patch/11214/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
| * MIPS: Allow L2 prefetch to be configured via debugfsPaul Burton2015-10-262-0/+82
| | | | | | | | | | | | | | | | | | | | | | | | | | When debugging or examining the performance of a system it can be useful to examine the effect of L2 prefetching. Provide an optional debugfs entry to allow a user to enable or disable L2 prefetching. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Maciej W. Rozycki <macro@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/11182/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
| * MIPS: Enable L2 prefetching for CM >= 2.5Paul Burton2015-10-261-1/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On systems with CM 2.5 & beyond there may be L2 prefetch units present which are not enabled by default. Detect them, configuring & enabling prefetching when available. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Cc: linux-kernel@vger.kernel.org Cc: James Hogan <james.hogan@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Patchwork: https://patchwork.linux-mips.org/patch/11180/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
| * MIPS: Remove invalid checkAndrzej Hajda2015-10-261-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Unsigned values cannot be lesser than zero. The problem has been detected using proposed semantic patch scripts/coccinelle/tests/unsigned_lesser_than_zero.cocci [1]. [ralf@linux-mips.org: Chris Dearman's original commit 9318c51acd9689505850152cc98277a6d6f2d752 ([MIPS] MIPS32/MIPS64 secondary cache management) introduced these less than zero checks in 2.6.18.] [1]: http://permalink.gmane.org/gmane.linux.kernel/2038576 Signed-off-by: Andrzej Hajda <a.hajda@samsung.com> Cc: linux-kernel@vger.kernel.org Cc: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com> Cc: Marek Szyprowski <m.szyprowski@samsung.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: Chris Dearman <chris.dearman@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11165/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* | kmap_atomic_to_page() has no users, remove itNicolas Pitre2015-11-091-13/+0
|/ | | | | | | | | Removal started in commit 5bbeed12bdc3 ("sparc32: drop unused kmap_atomic_to_page"). Let's do it across the whole tree. Signed-off-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* MIPS: dma-default: Fix 32-bit fall back to GFP_DMAJames Hogan2015-09-301-1/+1
| | | | | | | | | | | | | | | | | | | If there is a DMA zone (usually 24bit = 16MB I believe), but no DMA32 zone, as is the case for some 32-bit kernels, then massage_gfp_flags() will cause DMA memory allocated for devices with a 32..63-bit coherent_dma_mask to fall back to using __GFP_DMA, even though there may only be 32-bits of physical address available anyway. Correct that case to compare against a mask the size of phys_addr_t instead of always using a 64-bit mask. Signed-off-by: James Hogan <james.hogan@imgtec.com> Fixes: a2e715a86c6d ("MIPS: DMA: Fix computation of DMA flags from device's coherent_dma_mask.") Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: <stable@vger.kernel.org> # 2.6.36+ Patchwork: https://patchwork.linux-mips.org/patch/9610/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: Initialise MAARs on secondary CPUsPaul Burton2015-09-271-3/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MAARs should be initialised on each CPU (or rather, core) in the system in order to achieve consistent behaviour & performance. Previously they have only been initialised on the boot CPU which leads to performance problems if tasks are later scheduled on a secondary CPU, particularly if those tasks make use of unaligned vector accesses where some CPUs don't handle any cases in hardware for non-speculative memory regions. Fix this by recording the MAAR configuration from the boot CPU and applying it to secondary CPUs as part of their bringup. Reported-by: Doug Gilmore <doug.gilmore@imgtec.com> Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Cc: Rusty Russell <rusty@rustcorp.com.au> Cc: Steven J. Hill <Steven.Hill@imgtec.com> Cc: Andrew Bresticker <abrestic@chromium.org> Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: David Hildenbrand <dahi@linux.vnet.ibm.com> Cc: linux-kernel@vger.kernel.org Cc: Aaro Koskinen <aaro.koskinen@iki.fi> Cc: James Hogan <james.hogan@imgtec.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: Hemmo Nieminen <hemmo.nieminen@iki.fi> Cc: Alex Smith <alex.smith@imgtec.com> Cc: Peter Zijlstra (Intel) <peterz@infradead.org> Patchwork: https://patchwork.linux-mips.org/patch/11239/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: print MAAR configuration during bootPaul Burton2015-09-271-0/+29
| | | | | | | | | | | | | | | Verifying that the MAAR configuration is as expected is useful when debugging the performance of a system. Print out the memory regions configured via MAAR along with their attributes. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Cc: Steven J. Hill <Steven.Hill@imgtec.com> Cc: David Hildenbrand <dahi@linux.vnet.ibm.com> Cc: linux-kernel@vger.kernel.org Cc: Peter Zijlstra (Intel) <peterz@infradead.org> Patchwork: https://patchwork.linux-mips.org/patch/11238/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* MIPS: mm: compile maar_init unconditionallyPaul Burton2015-09-271-63/+63
| | | | | | | | | | | | | | | | | | maar_init was previously only compiled when CONFIG_NEED_MULTIPLE_NODES was not set, which has been fine since it is only called from the standard implementation of mem_init which has the same condition. In preparation for calling it from the SMP startup code on secondary CPUs, move maar_init outside of the #ifndef such that it is always compiled. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: Steven J. Hill <Steven.Hill@imgtec.com> Cc: David Hildenbrand <dahi@linux.vnet.ibm.com> Cc: linux-kernel@vger.kernel.org Cc: Ingo Molnar <mingo@kernel.org> Patchwork: https://patchwork.linux-mips.org/patch/11237/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* dma-mapping: consolidate dma_{alloc,free}_noncoherentChristoph Hellwig2015-09-101-5/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Most architectures do not support non-coherent allocations and either define dma_{alloc,free}_noncoherent to their coherent versions or stub them out. Openrisc uses dma_{alloc,free}_attrs to implement them, and only Mips implements them directly. This patch moves the Openrisc version to common code, and handles the DMA_ATTR_NON_CONSISTENT case in the mips dma_map_ops instance. Note that actual non-coherent allocations require a dma_cache_sync implementation, so if non-coherent allocations didn't work on an architecture before this patch they still won't work after it. [jcmvbkbc@gmail.com: fix xtensa] Signed-off-by: Christoph Hellwig <hch@lst.de> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Russell King <linux@arm.linux.org.uk> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Cc: Michal Simek <monstr@monstr.eu> Cc: Jonas Bonn <jonas@southpole.se> Cc: Chris Metcalf <cmetcalf@ezchip.com> Cc: Guan Xuetao <gxt@mprc.pku.edu.cn> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Andy Shevchenko <andy.shevchenko@gmail.com> Signed-off-by: Max Filippov <jcmvbkbc@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* dma-mapping: consolidate dma_{alloc,free}_{attrs,coherent}Christoph Hellwig2015-09-101-7/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since 2009 we have a nice asm-generic header implementing lots of DMA API functions for architectures using struct dma_map_ops, but unfortunately it's still missing a lot of APIs that all architectures still have to duplicate. This series consolidates the remaining functions, although we still need arch opt outs for two of them as a few architectures have very non-standard implementations. This patch (of 5): The coherent DMA allocator works the same over all architectures supporting dma_map operations. This patch consolidates them and converges the minor differences: - the debug_dma helpers are now called from all architectures, including those that were previously missing them - dma_alloc_from_coherent and dma_release_from_coherent are now always called from the generic alloc/free routines instead of the ops dma-mapping-common.h always includes dma-coherent.h to get the defintions for them, or the stubs if the architecture doesn't support this feature - checks for ->alloc / ->free presence are removed. There is only one magic instead of dma_map_ops without them (mic_dma_ops) and that one is x86 only anyway. Besides that only x86 needs special treatment to replace a default devices if none is passed and tweak the gfp_flags. An optional arch hook is provided for that. [linux@roeck-us.net: fix build] [jcmvbkbc@gmail.com: fix xtensa] Signed-off-by: Christoph Hellwig <hch@lst.de> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Russell King <linux@arm.linux.org.uk> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Cc: Michal Simek <monstr@monstr.eu> Cc: Jonas Bonn <jonas@southpole.se> Cc: Chris Metcalf <cmetcalf@ezchip.com> Cc: Guan Xuetao <gxt@mprc.pku.edu.cn> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Andy Shevchenko <andy.shevchenko@gmail.com> Signed-off-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Max Filippov <jcmvbkbc@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* MIPS: Set trap_no field in thread_struct on exception.Ralf Baechle2015-09-031-4/+4
| | | | | This reverts commit 7281cd22973008a782860e48ed8d85d00204168c and adds actual functionality to use the field.
* MIPS: Add implementation of dma_map_ops.mmap()Alex Smith2015-09-031-0/+35
| | | | | | | | | | | | | | | | | | | | | | The generic implementation of dma_map_ops.mmap(), dma_common_mmap(), is not correct for non-coherent devices. It expects to be passed a virtual address previously returned by dma_alloc_coherent(), which for a non-coherent device will return a KSEG1 address. It then attempts to convert that virtual address to a physical address using virt_to_page() which will yield an incorrect address. Also, dma_common_mmap() does not handle the DMA_ATTR_WRITE_COMBINE attribute, and therefore dma_mmap_writecombine() will not actually set the appropriate pgprot_t flags for write combining. This patch adds an implementation of dma_map_ops.mmap() that correctly handles KSEG1 addresses, and enables write combining when requested. Signed-off-by: Alex Smith <alex.smith@imgtec.com> Cc: Sadegh Abbasi <Sadegh.Abbasi@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/10808/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
OpenPOWER on IntegriCloud