summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* x86: split large page mapping for AMD TSEGAndi Kleen2008-04-172-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On AMD SMM protected memory is part of the address map, but handled internally like an MTRR. That leads to large pages getting split internally which has some performance implications. Check for the AMD TSEG MSR and split the large page mapping on that area explicitely if it is part of the direct mapping. There is also SMM ASEG, but it is in the first 1MB and already covered by the earlier split first page patch. Idea for this came from an earlier patch by Andreas Herrmann On a RevF dual Socket Opteron system kernbench shows a clear improvement from this: (together with the earlier patches in this series, especially the split first 2MB patch) [lower is better] no split stddev split stddev delta Elapsed Time 87.146 (0.727516) 84.296 (1.09098) -3.2% User Time 274.537 (4.05226) 273.692 (3.34344) -0.3% System Time 34.907 (0.42492) 34.508 (0.26832) -1.1% Percent CPU 322.5 (38.3007) 326.5 (44.5128) +1.2% => About 3.2% improvement in elapsed time for kernbench. With GB pages on AMD Fam1h the impact of splitting is much higher of course, since it would split two full GB pages (together with the first 1MB split patch) instead of two 2MB pages. I could not benchmark a clear difference in kernbench on gbpages, so I kept it disabled for that case That was only limited benchmarking of course, so if someone was interested in running more tests for the gbpages case that could be revisited (contributions welcome) I didn't bother implementing this for 32bit because it is very unlikely the 32bit lowmem mapping overlaps into the TSEG near 4GB and the 2MB low split is already handled for both. [ mingo@elte.hu: do it on gbpages kernels too, there's no clear reason why it shouldnt help there. ] Signed-off-by: Andi Kleen <ak@suse.de> Acked-by: andreas.herrmann3@amd.com Cc: mingo@elte.hu Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: re-add rdmsrl_safeAndi Kleen2008-04-172-0/+15
| | | | | | | | | | | | | RDMSR for 64bit values with exception handling. Makes it easier to deal with 64bit valued MSRs. The old 64bit code base had that too as checking_rdmsrl(), but it got dropped somehow. Signed-off-by: Andi Kleen <andi@firstfloor.org> Cc: andreas.herrmann3@amd.com Cc: mingo@elte.hu Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: don't use large pages to map the first 2/4MB of memoryAndi Kleen2008-04-172-1/+18
| | | | | | | | | | | | | | | | | | | | | | | Intel recommends to not use large pages for the first 1MB of the physical memory because there are fixed size MTRRs there which cause splitups in the TLBs. On AMD doing so is also a good idea. The implementation is a little different between 32bit and 64bit. On 32bit I just taught the initial page table set up about this because it was very simple to do. This also has the advantage that the risk of a prefetch ever seeing the page even if it only exists for a short time is minimized. On 64bit that is not quite possible, so use set_memory_4k() a little later (in check_bugs) instead. Signed-off-by: Andi Kleen <ak@suse.de> Acked-by: andreas.herrmann3@amd.com Cc: mingo@elte.hu Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: add set_memory_4k to pageattr.cAndi Kleen2008-04-172-4/+17
| | | | | | | | | | | | | | | | | | | Add a new function to force split large pages into 4k pages. This is needed for some followup optimizations. I had to add a new field to cpa_data to pass down the information that try_preserve_large_page should not run. Right now no set_page_4k() because I didn't need it and all the specialized users I have in mind would be more comfortable with pure addresses. I also didn't export it because it's unlikely external code needs it. Signed-off-by: Andi Kleen <ak@suse.de> Cc: andreas.herrmann3@amd.com Cc: mingo@elte.hu Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: account overlapped mappings in max_pfn_mappedAndi Kleen2008-04-174-14/+27
| | | | | | | | | | | When end_pfn is not aligned to 2MB (or 1GB) then the kernel might map more memory than end_pfn. Account this in max_pfn_mapped. Signed-off-by: Andi Kleen <ak@suse.de> Cc: andreas.herrmann3@amd.com Cc: mingo@elte.hu Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: replace the now useless max_pfn_mapped defineThomas Gleixner2008-04-176-23/+21
| | | | | Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: implement true end_pfn_mapped for 32bitAndi Kleen2008-04-173-2/+7
| | | | | | | | | | | | | | Even on 32bit 2MB pages can map more memory than is in the true max_low_pfn if end_pfn is not highmem and not aligned to 2MB. Add a end_pfn_map similar to x86-64 that accounts for this fact. This is important for code that really needs to know about all mapping aliases. Signed-off-by: Andi Kleen <ak@suse.de> Cc: andreas.herrmann3@amd.com Cc: mingo@elte.hu Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: move early exception handlers into init.textAndi Kleen2008-04-171-0/+2
| | | | | | | | | | | | Currently they are in .text.head because the rest of head_64.S. .text.head is not removed as init data, but the early exception handlers should be because they are not needed after early boot of the BP. So move them over. Signed-off-by: Andi Kleen <ak@suse.de> Cc: mingo@elte.hu Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: replace early exception setup macro recursion with loopAndi Kleen2008-04-171-10/+6
| | | | | | | | | | | | | | | The early exception handlers are currently set up using a macro recursion. There is only one user left. Replace the macro with a standard loop in place. Noop patch, just a cleanup. [ tglx@linutronix.de: simplified ] Signed-off-by: Andi Kleen <ak@suse.de> Cc: mingo@elte.hu Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: don't set up early exception handlers for external interruptsAndi Kleen2008-04-173-6/+5
| | | | | | | | | | | | All of early setup runs with interrupts disabled, so there is no need to set up early exception handlers for vectors >= 32 This saves some minor text size. Signed-off-by: Andi Kleen <ak@suse.de> Cc: mingo@elte.hu Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: relocate_kernel - use predefined macroses for page attributesgorcunov@gmail.com2008-04-172-3/+5
| | | | | Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: relocate_kernel - use predefined macroses for processor stategorcunov@gmail.com2008-04-172-29/+20
| | | | | Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: relocate_kernel - use PAGE_SIZE instead of numeric constantgorcunov@gmail.com2008-04-172-2/+2
| | | | | Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: relocate_kernel_32.S - clear register in more elegant waygorcunov@gmail.com2008-04-171-2/+1
| | | | | Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: fix test_poke for vmalloced pagesMathieu Desnoyers2008-04-171-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Ingo Molnar (mingo@elte.hu) wrote: > > * Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> wrote: > > > The shadow vmap for DEBUG_RODATA kernel text modification uses > > virt_to_page to get the pages from the pointer address. > > > > However, I think vmalloc_to_page would be required in case the page is > > used for modules. > > > > Since only the core kernel text is marked read-only, use > > kernel_text_address() to make sure we only shadow map the core kernel > > text, not modules. > > actually, i think we should mark module text readonly too. > Yes, but in the meantime, the x86 tree would need this patch to make kprobes work correctly on modules. I suspect that without this fix, with the enhanced hotplug and kprobes patch, kprobes will use text_poke to insert breakpoints in modules (vmalloced pages used), which will map the wrong pages and corrupt random kernel locations instead of updating the correct page. Work that would write protect the module pages should clearly be done, but it can come in a later time. We have to make sure we interact correctly with the page allocation debugging, as an example. Here is the patch against x86.git 2.6.25-rc5 : The shadow vmap for DEBUG_RODATA kernel text modification uses virt_to_page to get the pages from the pointer address. However, I think vmalloc_to_page would be required in case the page is used for modules. Since only the core kernel text is marked read-only, use kernel_text_address() to make sure we only shadow map the core kernel text, not modules. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> CC: akpm@linux-foundation.org Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: clean up vSMP detectionRavikiran G Thirumalai2008-04-171-9/+19
| | | | | | | | | vSMP detection: access pci config space early in boot to detect if the system is a vSMPowered box, and cache the result in a flag, so that is_vsmp_box() retrieves the value of the flag always. Signed-off-by: Ravikiran Thirumalai <kiran@scalex86.org> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: pgtable, document pde bitsJiri Slaby2008-04-171-7/+7
| | | | | | | | Some of pde bits weren't documented, add the short description to them. Signed-off-by: Jiri Slaby <jirislaby@gmail.com> Cc: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: spinlock ops are always-inlinedIngo Molnar2008-04-171-6/+6
| | | | Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: only enable interrupts when kernel state has been set upJeremy Fitzhardinge2008-04-171-9/+10
| | | | | | | | | | | | | | | The sysenter path tries to enable interrupts immediately. Unfortunately this doesn't work in a paravirt environment, because not enough kernel state has been set up at that point (namely, pointing %fs to the kernel percpu data segment). To fix this, defer ENABLE_INTERRUPTS until after the kernel state has been set up. Unfortunately this means that we're running with interrupts disabled for a while without calling the IRQ tracing code, but that can't be called without setting up %fs either. Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/xor_64.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-145/+149
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/xor_32.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-246/+248
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/voyager.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-20/+31
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/vmi.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-44/+44
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/vm86.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-3/+6
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/vga.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-2/+2
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/vdso.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-6/+10
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/user_64.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-7/+9
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/user32.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-3/+4
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/user_32.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-3/+3
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/unistd_64.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-1/+1
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/unistd_32.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-1/+1
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/unaligned.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-1/+1
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/uaccess_64.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-149/+227
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/uaccess_32.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-129/+187
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/tsc.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-1/+1
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/topology.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-1/+1
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/tlbflush.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-3/+2
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/thread_info_64.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-37/+41
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/thread_info_32.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-42/+46
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/tce.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-1/+1
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/system.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-53/+51
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/sync_bitops.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-28/+28
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/swiotlb.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-14/+14
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/suspend_64.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-3/+2
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/suspend_32.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-6/+6
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/string_64.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-33/+33
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/string_32.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-1/+1
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/srat.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-1/+1
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/spinlock.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-55/+50
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* include/asm-x86/smp_64.h: checkpatch cleanups - formatting onlyJoe Perches2008-04-171-3/+3
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
OpenPOWER on IntegriCloud