summaryrefslogtreecommitdiffstats
path: root/arch/powerpc/kernel/exceptions-64s.S
Commit message (Collapse)AuthorAgeFilesLines
* powerpc: Provide initial documentation for PAPR hcallsVaibhav Jain2020-01-291-16/+3
| | | | | | | | | | | | | | | | | | This doc patch provides an initial description of the hcall op-codes that are used by Linux kernel running as a guest (LPAR) on top of PowerVM or any other sPAPR compliant hyper-visor (e.g qemu). Apart from documenting the hcalls the doc-patch also provides a rudimentary overview of how hcall ABI, how they are issued with the Linux kernel and how information/control flows between the guest and hypervisor. Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Reviewed-by: Laurent Dufour <ldufour@linux.ibm.com> Acked-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Add SPDX tag, add it to index.rst] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190828082729.16695-1-vaibhav@linux.ibm.com
* powerpc/64s: Reimplement power4_idle code in CNicholas Piggin2020-01-161-2/+11
| | | | | | | | | | | | | | | | | This implements the tricky tracing and soft irq handling bits in C, leaving the low level bit to asm. A functional difference is that this redirects the interrupt exit to a return stub to execute blr, rather than the lr address itself. This is probably barely measurable on real hardware, but it keeps the link stack balanced. Tested with QEMU. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Move power4_fixup_nap back into exceptions-64s.S] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190711022404.18132-1-npiggin@gmail.com
* powerpc/64s/exception: Fix kaup -> kuap typoAndrew Donnellan2019-11-131-3/+3
| | | | | | | | It's KUAP, not KAUP. Fix typo in INT_COMMON macro. Signed-off-by: Andrew Donnellan <ajd@linux.ibm.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20191022060603.24101-1-ajd@linux.ibm.com
* powerpc/64s/exception: reduce page fault unnecessary loadsNicholas Piggin2019-08-301-21/+17
| | | | | | | | | | This avoids 3 loads in the radix page fault case, 1 load in the hash fault case, and 2 loads in the hash miss page fault case. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-37-npiggin@gmail.com
* powerpc/64s/exception: Remove pointless KVM handler name bifurcationNicholas Piggin2019-08-301-58/+44
| | | | | | | Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-36-npiggin@gmail.com
* powerpc/64s/exception: program check handler do not branch into a macroNicholas Piggin2019-08-301-7/+8
| | | | | | | | | | | | | | | | It is clever, but the small code saving is not worth the spaghetti of jumping to a label in an expanded macro, particularly when the label is just a number rather than a descriptive name. So expand the INT_COMMON macro twice, once for the stack and no stack cases, and branch to those. The slight code size increase is worth the improved clarity of branches for this non-performance critical code. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-35-npiggin@gmail.com
* powerpc/64s/exception: move interrupt entry code above the common handlerNicholas Piggin2019-08-301-225/+225
| | | | | | | | | | This better reflects the order in which the code is executed. No generated code change except BUG line number constants. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-34-npiggin@gmail.com
* powerpc/64s/exception: INT_COMMON add DAR, DSISR, reconcile optionsNicholas Piggin2019-08-301-60/+51
| | | | | | | | | | Move DAR and DSISR saving to pt_regs into INT_COMMON. Also add an option to expand RECONCILE_IRQ_STATE. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-33-npiggin@gmail.com
* powerpc/64s/exception: Expand EXCEPTION_PROLOG_COMMON_1 and 2 into callerNicholas Piggin2019-08-301-45/+40
| | | | | | | | | No generated code change except BUG line number constants. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-32-npiggin@gmail.com
* powerpc/64s/exception: Expand EXCEPTION_COMMON macro into callerNicholas Piggin2019-08-301-27/+27
| | | | | | | | | No generated code change except BUG line number constants. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-31-npiggin@gmail.com
* powerpc/64s/exception: Add INT_COMMON gas macro to generate common exception ↵Nicholas Piggin2019-08-301-20/+32
| | | | | | | | | | | code No generated code change except BUG line number constants. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-30-npiggin@gmail.com
* powerpc/64s/exception: Merge EXCEPTION_PROLOG_COMMON_2/3Nicholas Piggin2019-08-301-12/+6
| | | | | | | | | | | Merge EXCEPTION_PROLOG_COMMON_3 into EXCEPTION_PROLOG_COMMON_2. No generated code change except BUG line number constants. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-29-npiggin@gmail.com
* powerpc/64s/exception: KVM_HANDLER reorder arguments to match other macrosNicholas Piggin2019-08-301-8/+8
| | | | | | | | | | | Also change argument name (n -> vec) to match others. No generated code change. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-28-npiggin@gmail.com
* powerpc/64s/exception: Add INT_KVM_HANDLER gas macroNicholas Piggin2019-08-301-59/+40
| | | | | | | | | | | Replace the 4 variants of cpp macros with one gas macro. No generated code change except BUG line number constants. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-27-npiggin@gmail.com
* powerpc/64s/exception: INT_HANDLER support HDAR/HDSISR and use it in HDSINicholas Piggin2019-08-301-6/+10
| | | | | | Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-26-npiggin@gmail.com
* powerpc/64s/exception: Add the virt variant of the denorm interrupt handlerNicholas Piggin2019-08-301-1/+5
| | | | | | | | | | | All other virt handlers have the prolog code in the virt vector rather than branch to the real vector. Follow this pattern in the denorm virt handler. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-25-npiggin@gmail.com
* powerpc/64s/exception: remove EXCEPTION_PROLOG_0/1, rename _2Nicholas Piggin2019-08-301-78/+73
| | | | | | | | | | | | | | | EXCEPTION_PROLOG_0 and _1 have only a single caller, so expand them into it. Rename EXCEPTION_PROLOG_2_REAL to INT_SAVE_SRR_AND_JUMP and EXCEPTION_PROLOG_2_VIRT to INT_VIRT_SAVE_SRR_AND_JUMP, which are more descriptive. No generated code change except BUG line number constants. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-24-npiggin@gmail.com
* powerpc/64s/exceptions: Use keyword params to shorten arg listsMichael Ellerman2019-08-301-60/+60
| | | | | | | | The argument lists for the INT_HANDLER macro are getting a bit unwieldy. Use keyword parameters with default values to shorten them. Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190830011426.16810-1-mpe@ellerman.id.au
* powerpc/64s/exception: Replace PROLOG macros and EXC helpers with a gas macroNicholas Piggin2019-08-301-296/+206
| | | | | | | | | | | | | | | | | | This creates a single macro that generates the exception prolog code, with variants specified by arguments, rather than assorted nested macros for different variants. The increasing length of macro argument list is not nice to read or modify, but this is a temporary condition that will be improved in later changes. No generated code change except BUG line number constants and label names. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-23-npiggin@gmail.com
* powerpc/64s/exception: remove 0xb00 handlerNicholas Piggin2019-08-301-4/+2
| | | | | | | | | | | | This vector is not used by any supported processor, and has been implemented as an unknown exception going back to 2.6. There is nothing special about 0xb00, so remove it like other unused vectors. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-22-npiggin@gmail.com
* powerpc/64s/exception: Fix performance monitor virt handlerNicholas Piggin2019-08-301-1/+1
| | | | | | | | | | | The perf virt handler uses EXCEPTION_PROLOG_2_REAL rather than _VIRT. In practice this is okay because the _REAL variant is usable by virt mode interrupts, but should be fixed (and is a performance win). Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-21-npiggin@gmail.com
* powerpc/64s/exception: Add EXC_HV_OR_STD, which selects HSRR if HVMODENicholas Piggin2019-08-301-23/+79
| | | | | | | | | | | | Add EXC_HV_OR_STD and use it to consolidate the 0x500 external interrupt. Executed code is unchanged. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-20-npiggin@gmail.com
* powerpc/64s/exception: move head-64.h exception code to exception-64s.SNicholas Piggin2019-08-301-0/+41
| | | | | | | | | | | | The head-64.h code should deal only with the head code sections and offset calculations. No generated code change except BUG line number constants. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-19-npiggin@gmail.com
* powerpc/64s/exception: Fix DAR load for handle_page_fault error caseNicholas Piggin2019-08-301-1/+1
| | | | | | | | | | | This buglet goes back to before the 64/32 arch merge, but it does not seem to have had practical consequences because bad_page_fault does not use the 2nd argument, but rather regs->dar/nip. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-18-npiggin@gmail.com
* powerpc/64s/exception: machine check improve labels and commentsNicholas Piggin2019-08-301-4/+10
| | | | | | | | | | | | | Short forward and backward branches can be given number labels, but larger significant divergences in code path a more readable if they're given descriptive names. Also adjusts a comment to account for guest delivery. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-17-npiggin@gmail.com
* powerpc/64s/exception: untangle early machine check handler branchNicholas Piggin2019-08-301-67/+62
| | | | | | | | | | | | machine_check_early_common now branches to machine_check_handle_early which is its only caller. Move interleaving code out of the way, and remove the branch. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-16-npiggin@gmail.com
* powerpc/64s/exception: machine check move unrecoverable handling out of lineNicholas Piggin2019-08-301-43/+44
| | | | | | | | | | | | | | | Similarly to the previous change, all callers of the unrecoverable handler run relocated so can reach it with a direct branch. This makes it easy to move out of line, which makes the "normal" path less cluttered and easier to follow. MSR[ME] manipulation still requires the rfi, so that is moved out of line to its own function. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-15-npiggin@gmail.com
* powerpc/64s/exception: simplify machine check early pathNicholas Piggin2019-08-301-9/+22
| | | | | | | | | | | | | | machine_check_handle_early_common can reach machine_check_handle_early directly now that it runs at the relocated address, so just branch directly. The rfi sequence is required to enable MSR[ME] but that step is moved into a helper function, making the code easier to follow. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-14-npiggin@gmail.com
* powerpc/64s/exception: machine check move tramp codeNicholas Piggin2019-08-301-11/+11
| | | | | | | | | | Following convention, move the tramp code (unrelocated) above the common handlers (relocated). Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-13-npiggin@gmail.com
* powerpc/64s/exception: machine check restructure to reuse common macrosNicholas Piggin2019-08-301-35/+36
| | | | | | | | | | | | | Follow the pattern of sreset and HMI handlers more closely: use EXCEPTION_PROLOG_COMMON_1 rather than open-coding it, and run the handler at the relocated location. This helps later simplification and code sharing. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-12-npiggin@gmail.com
* powerpc/64s/exception: machine check pseries should skip the late handler ↵Nicholas Piggin2019-08-301-20/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | for kernel MCEs The powernv machine check handler copes with taking a MCE from one of three contexts, guest, kernel, and user. In each case the early handler runs first on a special stack, then: - The guest case branches to the KVM interrupt handler (via standard interrupt macros). - The user case will run the "late" handler which is like a normal interrupt that runs in virtual mode and uses the regular kernel stack. - The kernel case queues the event and schedules it for processing with irq work. The last case is important, it must not enable virtual memory because the MMU state may not be set up to deal with that (e.g., SLB might be clear), it must not use the regular kernel stack for similar reasons (e.g., might be in OPAL with OPAL stack in r1), and the kernel does not expect anything to touch its stack if interrupts are disabled. The pseries handler does not do this queueing, but instead it always runs the late handler for host MCEs, which has some of the same problems. Now that pseries is using machine_check_events, change it to do the same as powernv and queue events for kernel MCEs. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-11-npiggin@gmail.com
* powerpc/64s/exception: machine check use correct cfar for late handlerNicholas Piggin2019-08-301-0/+4
| | | | | | | | | | | | | | | | | Bare metal machine checks run an "early" handler in real mode before running the main handler which reports the event. The main handler runs exactly as a normal interrupt handler, after the "windup" which sets registers back as they were at interrupt entry. CFAR does not get restored by the windup code, so that will be wrong when the handler is run. Restore the CFAR to the saved value before running the late handler. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-8-npiggin@gmail.com
* powerpc/64s/exception: machine check remove machine_check_pSeries_0 branchNicholas Piggin2019-08-301-13/+10
| | | | | | | | | | | This label has only one caller, so unwind the branch and move it inline. The location of the comment is adjusted to match similar one in system reset. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-7-npiggin@gmail.com
* powerpc/64s/exception: machine check pseries should always run the early handlerNicholas Piggin2019-08-301-4/+0
| | | | | | | | | | | | | | Now that pseries with fwnmi registered runs the early machine check handler, there is no good reason to special case the non-fwnmi case and skip the early handler. Reducing the code and number of paths is a top priority for asm code, it's better to handle this in C where possible (and the pseries early handler is a no-op if fwnmi is not registered). Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-6-npiggin@gmail.com
* powerpc/64s/exception: machine check adjust RFI targetNicholas Piggin2019-08-301-1/+1
| | | | | | | | | | | | | | The host kernel delivery case for powernv does RFI_TO_USER_OR_KERNEL, but should just use RFI_TO_KERNEL which makes it clear this is not a user case. This is not a bug because RFI_TO_USER_OR_KERNEL deals with kernel returns just fine. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-5-npiggin@gmail.com
* powerpc/64s/exception: machine check fix KVM guest testNicholas Piggin2019-08-301-20/+13
| | | | | | | | | | | | | | | The machine_check_handle_early hypervisor guest test is skipped if !HVMODE or MSR[HV]=0, which is wrong for PR or nested hypervisors that could be running a guest in this state. Test HSTATE_IN_GUEST up front and use that to branch out to the KVM handler, then MSR[PR] alone can test for this kernel's userspace. This matches all other interrupt handling. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-4-npiggin@gmail.com
* powerpc/64s/exception: machine check remove bitrotted commentNicholas Piggin2019-08-301-4/+0
| | | | | | | Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-3-npiggin@gmail.com
* powerpc/64s/exception: machine check fwnmi remove HV caseNicholas Piggin2019-08-301-2/+1
| | | | | | | | | fwnmi does not trigger in HV mode, so remove always-true feature test. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190802105709.27696-2-npiggin@gmail.com
* powerpc/64: remove support for kernel-mode syscallsNicholas Piggin2019-08-281-2/+0
| | | | | | | | | | | There is support for the kernel to execute the 'sc 0' instruction and make a system call to itself. This is a relic that is unused in the tree, therefore untested. It's also highly questionable for modules to be doing this. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20190827033010.28090-3-npiggin@gmail.com
* powerpc/prom: convert PROM_BUG() to standard trapChristophe Leroy2019-08-281-1/+2
| | | | | | | | | | | | | | | | | Prior to commit 1bd98d7fbaf5 ("ppc64: Update BUG handling based on ppc32"), BUG() family was using BUG_ILLEGAL_INSTRUCTION which was an invalid instruction opcode to trap into program check exception. That commit converted them to using standard trap instructions, but prom/prom_init and their PROM_BUG() macro were left over. head_64.S and exception-64s.S were left aside as well. Convert them to using the standard BUG infrastructure. Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/cdaf4bbbb64c288a077845846f04b12683f8875a.1566817807.git.christophe.leroy@c-s.fr
* docs: powerpc: convert docs to ReST and rename to *.rstMauro Carvalho Chehab2019-07-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | Convert docs to ReST and add them to the arch-specific book. The conversion here was trivial, as almost every file there was already using an elegant format close to ReST standard. The changes were mostly to mark literal blocks and add a few missing section title identifiers. One note with regards to "--": on Sphinx, this can't be used to identify a list, as it will format it badly. This can be used, however, to identify a long hyphen - and "---" is an even longer one. At its new index.rst, let's add a :orphan: while this is not linked to the main index.rst file, in order to avoid build warnings. Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org> Acked-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> # cxl
* powerpc/64s/exception: simplify hmi control flowNicholas Piggin2019-07-031-16/+10
| | | | | | | | | | | | | Branch to the relocated 0xc000 address early (still in real mode), to simplify subsequent branches. Have the virt mode handler avoid just 'windup' and redo the exception from scratch, rather than branching back to the trampoline. Rearrange the stack setup instruction location to match the system reset handler (e.g., right before EXCEPTION_PROLOG_COMMON). Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
* powerpc/64s/exception: hmi remove special case macroNicholas Piggin2019-07-031-12/+4
| | | | | | | No code change. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
* powerpc/64s/exception: sreset move trampoline ahead of common codeNicholas Piggin2019-07-031-12/+12
| | | | | | | Follow convention and move tramp ahead of common. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
* powerpc/64s/exception: optimise system_reset for idle, clean up non-idle caseNicholas Piggin2019-07-031-31/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The idle wake up code in the system reset interrupt is not very optimal. There are two requirements: perform idle wake up quickly; and save everything including CFAR for non-idle interrupts, with no performance requirement. The problem with placing the idle test in the middle of the handler and using the normal handler code to save CFAR, is that it's quite costly (e.g., mfcfar is serialising, speculative workarounds get applied, SRR1 has to be reloaded, etc). It also prevents the standard interrupt handler boilerplate being used. This pain can be avoided by using a dedicated idle interrupt handler at the start of the interrupt handler, which restores all registers back to the way they were in case it was not an idle wake up. CFAR is preserved without saving it before the non-idle case by making that the fall-through, and idle is a taken branch. Performance seems to be in the noise, but possibly around 0.5% faster, the executed instructions certainly look better. The bigger benefit is being able to drop in standard interrupt handlers after the idle code, which helps with subsequent cleanup and consolidation. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Fixup BE by using DOTSYM for idle_return_gpr_loss call] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
* powerpc/64s/exception: remove bad stack branchNicholas Piggin2019-07-021-80/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The bad stack test in interrupt handlers has a few problems. For performance it is taken in the common case, which is a fetch bubble and a waste of i-cache. For code development and maintainence, it requires yet another stack frame setup routine, and that constrains all exception handlers to follow the same register save pattern which inhibits future optimisation. Remove the test/branch and replace it with a trap. Teach the program check handler to use the emergency stack for this case. This does not result in quite so nice a message, however the SRR0 and SRR1 of the crashed interrupt can be seen in r11 and r12, as is the original r1 (adjusted by INT_FRAME_SIZE). These are the most important parts to debugging the issue. The original r9-12 and cr0 is lost, which is the main downside. kernel BUG at linux/arch/powerpc/kernel/exceptions-64s.S:847! Oops: Exception in kernel mode, sig: 5 [#1] BE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 0 PID: 1 Comm: swapper/0 Not tainted NIP: c000000000009108 LR: c000000000cadbcc CTR: c0000000000090f0 REGS: c0000000fffcbd70 TRAP: 0700 Not tainted MSR: 9000000000021032 <SF,HV,ME,IR,DR,RI> CR: 28222448 XER: 20040000 CFAR: c000000000009100 IRQMASK: 0 GPR00: 000000000000003d fffffffffffffd00 c0000000018cfb00 c0000000f02b3166 GPR04: fffffffffffffffd 0000000000000007 fffffffffffffffb 0000000000000030 GPR08: 0000000000000037 0000000028222448 0000000000000000 c000000000ca8de0 GPR12: 9000000002009032 c000000001ae0000 c000000000010a00 0000000000000000 GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 GPR20: c0000000f00322c0 c000000000f85200 0000000000000004 ffffffffffffffff GPR24: fffffffffffffffe 0000000000000000 0000000000000000 000000000000000a GPR28: 0000000000000000 0000000000000000 c0000000f02b391c c0000000f02b3167 NIP [c000000000009108] decrementer_common+0x18/0x160 LR [c000000000cadbcc] .vsnprintf+0x3ec/0x4f0 Call Trace: Instruction dump: 996d098a 994d098b 38610070 480246ed 48005518 60000000 38200000 718a4000 7c2a0b78 3821fd00 41c20008 e82d0970 <0981fd00> f92101a0 f9610170 f9810178 Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
* powerpc/64s/exception: move SET_SCRATCH0 into EXCEPTION_PROLOG_0Nicholas Piggin2019-07-021-24/+1
| | | | | | | No generated code change. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
* powerpc/64s/exception: denorm handler use standard scratch save macroNicholas Piggin2019-07-021-1/+1
| | | | | | | | | | Although the 0x1500 interrupt only applies to bare metal, it is better to just use the standard macro for scratch save. Runtime code path remains unchanged (due to instruction patching). Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
* powerpc/64s/exception: machine check use standard macros to save dar/dsisrNicholas Piggin2019-07-021-5/+1
| | | | | Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
* powerpc/64s/exception: add dar and dsisr options to exception macroNicholas Piggin2019-07-021-56/+45
| | | | | | | | | | | | | | | | | | Some exception entry requires DAR and/or DSISR to be saved into the paca exception save area. Add options to the standard exception macros for these. Generated code changes slightly due to code structure. - 554: a6 02 72 7d mfdsisr r11 - 558: a8 00 4d f9 std r10,168(r13) - 55c: b0 00 6d 91 stw r11,176(r13) + 554: a8 00 4d f9 std r10,168(r13) + 558: a6 02 52 7d mfdsisr r10 + 55c: b0 00 4d 91 stw r10,176(r13) Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
OpenPOWER on IntegriCloud