summaryrefslogtreecommitdiffstats
path: root/arch/x86/crypto
Commit message (Collapse)AuthorAgeFilesLines
* crypto: glue_helper - remove the le128_gf128mul_x_ble functionOndrej Mosnáček2017-04-051-1/+2
| | | | | | | | | The le128_gf128mul_x_ble function in glue_helper.h is now obsolete and can be replaced with the gf128mul_x_ble function from gf128mul.h. Signed-off-by: Ondrej Mosnacek <omosnacek@gmail.com> Reviewd-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: gf128mul - switch gf128mul_x_ble to le128Ondrej Mosnáček2017-04-053-6/+6
| | | | | | | | | | | | | Currently, gf128mul_x_ble works with pointers to be128, even though it actually interprets the words as little-endian. Consequently, it uses cpu_to_le64/le64_to_cpu on fields of type __be64, which is incorrect. This patch fixes that by changing the function to accept pointers to le128 and updating all users accordingly. Signed-off-by: Ondrej Mosnacek <omosnacek@gmail.com> Reviewd-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto, x86: aesni - fix token pasting for clangMichael Davidson2017-03-241-5/+2
| | | | | | | | | | | | | | | | | aes_ctrby8_avx-x86_64.S uses the C preprocessor for token pasting of character sequences that are not valid preprocessor tokens. While this is allowed when preprocessing assembler files it exposes an incompatibilty between the clang and gcc preprocessors where clang does not strip leading white space from macro parameters, leading to the CONCAT(%xmm, i) macro expansion on line 96 resulting in a token with a space character embedded in it. While this could be resolved by deleting the offending space character, the assembler is perfectly capable of doing the token pasting correctly for itself so we can just get rid of the preprocessor macros. Signed-off-by: Michael Davidson <md@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha512-mb - Protect sha512 mb ctx mgr accessTim Chen2017-02-111-22/+42
| | | | | | | | | | The flusher and regular multi-buffer computation via mcryptd may race with another. Add here a lock and turn off interrupt to to access multi-buffer computation state cstate->mgr before a round of computation. This should prevent the flusher code jumping in. Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* Merge git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6Herbert Xu2017-02-031-5/+6
|\ | | | | | | Merge the crypto tree to pick up arm64 output IV patch.
| * crypto: aesni - Fix failure when pcbc module is absentHerbert Xu2017-02-031-4/+4
| | | | | | | | | | | | | | | | | | | | | | When aesni is built as a module together with pcbc, the pcbc module must be present for aesni to load. However, the pcbc module may not be present for reasons such as its absence on initramfs. This patch allows the aesni to function even if the pcbc module is enabled but not present. Reported-by: Arkadiusz Miśkiewicz <arekm@maven.pl> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * crypto: aesni - Fix failure when built-in with modular pcbcHerbert Xu2016-12-301-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | If aesni is built-in but pcbc is built as a module, then aesni will fail completely because when it tries to register the pcbc variant of aes the pcbc template is not available. This patch fixes this by modifying the pcbc presence test so that if aesni is built-in then pcbc must also be built-in for it to be used by aesni. Fixes: 85671860caac ("crypto: aesni - Convert to skcipher") Reported-by: Stephan Müller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* | crypto: x86 - make constants readonly, allow linker to merge themDenys Vlasenko2017-01-2333-74/+229
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A lot of asm-optimized routines in arch/x86/crypto/ keep its constants in .data. This is wrong, they should be on .rodata. Mnay of these constants are the same in different modules. For example, 128-bit shuffle mask 0x000102030405060708090A0B0C0D0E0F exists in at least half a dozen places. There is a way to let linker merge them and use just one copy. The rules are as follows: mergeable objects of different sizes should not share sections. You can't put them all in one .rodata section, they will lose "mergeability". GCC puts its mergeable constants in ".rodata.cstSIZE" sections, or ".rodata.cstSIZE.<object_name>" if -fdata-sections is used. This patch does the same: .section .rodata.cst16.SHUF_MASK, "aM", @progbits, 16 It is important that all data in such section consists of 16-byte elements, not larger ones, and there are no implicit use of one element from another. When this is not the case, use non-mergeable section: .section .rodata[.VAR_NAME], "a", @progbits This reduces .data by ~15 kbytes: text data bss dec hex filename 11097415 2705840 2630712 16433967 fac32f vmlinux-prev.o 11112095 2690672 2630712 16433479 fac147 vmlinux.o Merged objects are visible in System.map: ffffffff81a28810 r POLY ffffffff81a28810 r POLY ffffffff81a28820 r TWOONE ffffffff81a28820 r TWOONE ffffffff81a28830 r PSHUFFLE_BYTE_FLIP_MASK <- merged regardless of ffffffff81a28830 r SHUF_MASK <------------- the name difference ffffffff81a28830 r SHUF_MASK ffffffff81a28830 r SHUF_MASK .. ffffffff81a28d00 r K512 <- merged three identical 640-byte tables ffffffff81a28d00 r K512 ffffffff81a28d00 r K512 Use of object names in section name suffixes is not strictly necessary, but might help if someday link stage will use garbage collection to eliminate unused sections (ld --gc-sections). Signed-off-by: Denys Vlasenko <dvlasenk@redhat.com> CC: Herbert Xu <herbert@gondor.apana.org.au> CC: Josh Poimboeuf <jpoimboe@redhat.com> CC: Xiaodong Liu <xiaodong.liu@intel.com> CC: Megha Dey <megha.dey@intel.com> CC: linux-crypto@vger.kernel.org CC: x86@kernel.org CC: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* | crypto: x86/crc32c - fix %progbits -> @progbitsDenys Vlasenko2017-01-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | %progbits form is used on ARM (where @ is a comment char). x86 consistently uses @progbits everywhere else. Signed-off-by: Denys Vlasenko <dvlasenk@redhat.com> CC: Herbert Xu <herbert@gondor.apana.org.au> CC: Josh Poimboeuf <jpoimboe@redhat.com> CC: Xiaodong Liu <xiaodong.liu@intel.com> CC: Megha Dey <megha.dey@intel.com> CC: George Spelvin <linux@horizon.com> CC: linux-crypto@vger.kernel.org CC: x86@kernel.org CC: linux-kernel@vger.kernel.org Reviewed-by: Josh Poimboeuf <jpoimboe@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* | crypto: x86/chacha20 - Manually align stack bufferHerbert Xu2017-01-131-1/+4
| | | | | | | | | | | | | | | | | | | | The kernel on x86-64 cannot use gcc attribute align to align to a 16-byte boundary. This patch reverts to the old way of aligning it by hand. Fixes: 9ae433bc79f9 ("crypto: chacha20 - convert generic and...") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
* | crypto: aesni-intel - RFC4106 can zero copy when !PageHighMemIlya Lesokhin2016-12-271-4/+8
| | | | | | | | | | | | | | | | In the common case of !PageHighMem we can do zero copy crypto even if sg crosses a pages boundary. Signed-off-by: Ilya Lesokhin <ilyal@mellanox.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* | crypto: chacha20 - convert generic and x86 versions to skcipherArd Biesheuvel2016-12-271-38/+31
|/ | | | | | | | | | | | | | | This converts the ChaCha20 code from a blkcipher to a skcipher, which is now the preferred way to implement symmetric block and stream ciphers. This ports the generic and x86 versions at the same time because the latter reuses routines of the former. Note that the skcipher_walk() API guarantees that all presented blocks except the final one are a multiple of the chunk size, so we can simplify the encrypt() routine somewhat. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* Merge branch 'linus' of ↵Linus Torvalds2016-12-149-576/+422
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 Pull crypto updates from Herbert Xu: "Here is the crypto update for 4.10: API: - add skcipher walk interface - add asynchronous compression (acomp) interface - fix algif_aed AIO handling of zero buffer Algorithms: - fix unaligned access in poly1305 - fix DRBG output to large buffers Drivers: - add support for iMX6UL to caam - fix givenc descriptors (used by IPsec) in caam - accelerated SHA256/SHA512 for ARM64 from OpenSSL - add SSE CRCT10DIF and CRC32 to ARM/ARM64 - add AEAD support to Chelsio chcr - add Armada 8K support to omap-rng" * 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (148 commits) crypto: testmgr - fix overlap in chunked tests again crypto: arm/crc32 - accelerated support based on x86 SSE implementation crypto: arm64/crc32 - accelerated support based on x86 SSE implementation crypto: arm/crct10dif - port x86 SSE implementation to ARM crypto: arm64/crct10dif - port x86 SSE implementation to arm64 crypto: testmgr - add/enhance test cases for CRC-T10DIF crypto: testmgr - avoid overlap in chunked tests crypto: chcr - checking for IS_ERR() instead of NULL crypto: caam - check caam_emi_slow instead of re-lookup platform crypto: algif_aead - fix AIO handling of zero buffer crypto: aes-ce - Make aes_simd_algs static crypto: algif_skcipher - set error code when kcalloc fails crypto: caam - make aamalg_desc a proper module crypto: caam - pass key buffers with typesafe pointers crypto: arm64/aes-ce-ccm - Fix AEAD decryption length MAINTAINERS: add crypto headers to crypto entry crypt: doc - remove misleading mention of async API crypto: doc - fix header file name crypto: api - fix comment typo crypto: skcipher - Add separate walker for AEAD decryption ..
| * crypto: aesni - Convert to skcipherHerbert Xu2016-11-282-569/+343
| | | | | | | | | | | | | | | | | | This patch converts aesni (including fpu) over to the skcipher interface. The LRW implementation has been removed as the generic LRW code can now be used directly on top of the accelerated ECB implementation. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * crypto: glue_helper - Add skcipher xts helpersHerbert Xu2016-11-281-1/+73
| | | | | | | | | | | | | | This patch adds xts helpers that use the skcipher interface rather than blkcipher. This will be used by aesni_intel. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * crypto: sha-mb - Fix total_len for correct hash when larger than 512MBGreg Tucker2016-11-176-6/+6
| | | | | | | | | | | | | | | | | | | | Current multi-buffer hash implementations have a restriction on the total length of a hash job to 512MB. Hashing larger buffers will result in an incorrect hash. This extends the limit to 2^62 - 1. Signed-off-by: Greg Tucker <greg.b.tucker@intel.com> Acked-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* | Merge branch 'linus' into x86/fpu, to resolve conflictsIngo Molnar2016-11-231-2/+2
|\ \ | | | | | | | | | | | | | | | | | | Conflicts: arch/x86/kernel/fpu/core.c Signed-off-by: Ingo Molnar <mingo@kernel.org>
| * | crypto: aesni: shut up -Wmaybe-uninitialized warningArnd Bergmann2016-11-111-2/+2
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The rfc4106 encrypy/decrypt helper functions cause an annoying false-positive warning in allmodconfig if we turn on -Wmaybe-uninitialized warnings again: arch/x86/crypto/aesni-intel_glue.c: In function ‘helper_rfc4106_decrypt’: include/linux/scatterlist.h:67:31: warning: ‘dst_sg_walk.sg’ may be used uninitialized in this function [-Wmaybe-uninitialized] The problem seems to be that the compiler doesn't track the state of the 'one_entry_in_sg' variable across the kernel_fpu_begin/kernel_fpu_end section. This takes the easy way out by adding a bogus initialization, which should be harmless enough to get the patch into v4.9 so we can turn on this warning again by default without producing useless output. A follow-up patch for v4.10 rearranges the code to make the warning go away. Signed-off-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* | x86/fpu: Remove use_eager_fpu()Andy Lutomirski2016-10-071-13/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This removes all the obvious code paths that depend on lazy FPU mode. It shouldn't change the generated code at all. Signed-off-by: Andy Lutomirski <luto@kernel.org> Signed-off-by: Rik van Riel <riel@redhat.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: pbonzini@redhat.com Link: http://lkml.kernel.org/r/1475627678-20788-5-git-send-email-riel@redhat.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
* | x86/crypto, x86/fpu: Remove X86_FEATURE_EAGER_FPU #ifdef from the crc32c codeAndy Lutomirski2016-10-071-5/+0
|/ | | | | | | | | | | | | | | | | | | | | | | | | The crypto code was checking both use_eager_fpu() and defined(X86_FEATURE_EAGER_FPU). The latter was nonsensical, so remove it. This will avoid breakage when we remove X86_FEATURE_EAGER_FPU. Signed-off-by: Andy Lutomirski <luto@kernel.org> Signed-off-by: Rik van Riel <riel@redhat.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: pbonzini@redhat.com Link: http://lkml.kernel.org/r/1475627678-20788-2-git-send-email-riel@redhat.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
* crypto: sha512-mb - fix ctx pointerXiaodong Liu2016-08-161-2/+2
| | | | | | | | | | | 1. fix ctx pointer Use req_ctx which is the ctx for the next job that have been completed in the lanes instead of the first completed job rctx, whose completion could have been called and released. Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha256-mb - fix ctx pointer and digest copyXiaodong Liu2016-08-162-5/+6
| | | | | | | | | | | | | | 1. fix ctx pointer Use req_ctx which is the ctx for the next job that have been completed in the lanes instead of the first completed job rctx, whose completion could have been called and released. 2. fix digest copy Use XMM register to copy another 16 bytes sha256 digest instead of a regular register. Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha-mb - Cleanup code to use || instead of |Tim Chen2016-07-123-9/+15
| | | | | | | | | | | | | | | for condition comparison and cleanup multiline comment style In sha*_ctx_mgr_submit, we currently use the | operator instead of || ((ctx->partial_block_buffer_length) | (len < SHA1_BLOCK_SIZE)) Switching it to || and remove extraneous paranthesis to adhere to coding style. Also cleanup inconsistent multiline comment style. Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: aesni - Use crypto_cipher to derive rfc4106 subkeyHerbert Xu2016-07-011-65/+11
| | | | | | | | | | | | | Currently aesni uses an async ctr(aes) to derive the rfc4106 subkey, which was presumably copied over from the generic rfc4106 code. Over there it's done that way because we already have a ctr(aes) spawn. But it is simply overkill for aesni since we have to go get a ctr(aes) from scratch anyway. This patch simplifies the subkey derivation by using a straight aes cipher instead. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha512-mb - Crypto computation (x4 AVX2)Megha Dey2016-06-281-0/+529
| | | | | | | | | | | | This patch introduces the assembly routines to do SHA512 computation on buffers belonging to several jobs at once. The assembly routines are optimized with AVX2 instructions that have 4 data lanes and using AVX2 registers. Signed-off-by: Megha Dey <megha.dey@linux.intel.com> Reviewed-by: Fenghua Yu <fenghua.yu@intel.com> Reviewed-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha512-mb - Algorithm data structuresMegha Dey2016-06-283-0/+515
| | | | | | | | | | | | This patch introduces the data structures and prototypes of functions needed for computing SHA512 hash using multi-buffer. Included are the structures of the multi-buffer SHA512 job, job scheduler in C and x86 assembly. Signed-off-by: Megha Dey <megha.dey@linux.intel.com> Reviewed-by: Fenghua Yu <fenghua.yu@intel.com> Reviewed-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha512-mb - submit/flush routines for AVX2Megha Dey2016-06-283-0/+580
| | | | | | | | | | | This patch introduces the routines used to submit and flush buffers belonging to SHA512 crypto jobs to the SHA512 multibuffer algorithm. It is implemented mostly in assembly optimized with AVX2 instructions. Signed-off-by: Megha Dey <megha.dey@linux.intel.com> Reviewed-by: Fenghua Yu <fenghua.yu@intel.com> Reviewed-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha512-mb - SHA512 multibuffer job manager and glue codeMegha Dey2016-06-283-0/+1055
| | | | | | | | | | | | | | | | This patch introduces the multi-buffer job manager which is responsible for submitting scatter-gather buffers from several SHA512 jobs to the multi-buffer algorithm. It also contains the flush routine that's called by the crypto daemon to complete the job when no new jobs arrive before the deadline of maximum latency of a SHA512 crypto job. The SHA512 multi-buffer crypto algorithm is defined and initialized in this patch. Signed-off-by: Megha Dey <megha.dey@linux.intel.com> Reviewed-by: Fenghua Yu <fenghua.yu@intel.com> Reviewed-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha1-mb - rename sha-mb to sha1-mbMegha Dey2016-06-2710-4/+4
| | | | | | | | | | | | Until now, there was only support for the SHA1 multibuffer algorithm. Hence, there was just one sha-mb folder. Now, with the introduction of the SHA256 multi-buffer algorithm , it is logical to name the existing folder as sha1-mb. Signed-off-by: Megha Dey <megha.dey@linux.intel.com> Reviewed-by: Fenghua Yu <fenghua.yu@intel.com> Reviewed-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha256-mb - Crypto computation (x8 AVX2)Megha Dey2016-06-271-0/+593
| | | | | | | | | | | | This patch introduces the assembly routines to do SHA256 computation on buffers belonging to several jobs at once. The assembly routines are optimized with AVX2 instructions that have 8 data lanes and using AVX2 registers. Signed-off-by: Megha Dey <megha.dey@linux.intel.com> Reviewed-by: Fenghua Yu <fenghua.yu@intel.com> Reviewed-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha256-mb - Algorithm data structuresMegha Dey2016-06-273-0/+548
| | | | | | | | | | | | This patch introduces the data structures and prototypes of functions needed for computing SHA256 hash using multi-buffer. Included are the structures of the multi-buffer SHA256 job, job scheduler in C and x86 assembly. Signed-off-by: Megha Dey <megha.dey@linux.intel.com> Reviewed-by: Fenghua Yu <fenghua.yu@intel.com> Reviewed-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha256-mb - submit/flush routines for AVX2Megha Dey2016-06-273-0/+584
| | | | | | | | | | | This patch introduces the routines used to submit and flush buffers belonging to SHA256 crypto jobs to the SHA256 multibuffer algorithm. It is implemented mostly in assembly optimized with AVX2 instructions. Signed-off-by: Megha Dey <megha.dey@linux.intel.com> Reviewed-by: Fenghua Yu <fenghua.yu@intel.com> Reviewed-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha256-mb - SHA256 multibuffer job manager and glue codeMegha Dey2016-06-273-0/+1039
| | | | | | | | | | | | | | | | This patch introduces the multi-buffer job manager which is responsible for submitting scatter-gather buffers from several SHA256 jobs to the multi-buffer algorithm. It also contains the flush routine to that's called by the crypto daemon to complete the job when no new jobs arrive before the deadline of maximum latency of a SHA256 crypto job. The SHA256 multi-buffer crypto algorithm is defined and initialized in this patch. Signed-off-by: Megha Dey <megha.dey@linux.intel.com> Reviewed-by: Fenghua Yu <fenghua.yu@intel.com> Reviewed-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha1-mb - async implementation for sha1-mbMegha Dey2016-06-231-81/+101
| | | | | | | | | | | | Herbert wants the sha1-mb algorithm to have an async implementation: https://lkml.org/lkml/2016/4/5/286. Currently, sha1-mb uses an async interface for the outer algorithm and a sync interface for the inner algorithm. This patch introduces a async interface for even the inner algorithm. Signed-off-by: Megha Dey <megha.dey@linux.intel.com> Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: ghash-clmulni - Fix cryptd reorderingHerbert Xu2016-06-231-23/+17
| | | | | | | | | | | | | | This patch fixes an old bug where requests can be reordered because some are processed by cryptd while others are processed directly in softirq context. The fix is to always postpone to cryptd if there are currently requests outstanding from the same tfm. This patch also removes the redundant use of cryptd in the async init function as init never touches the FPU. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: aesni - Fix cryptd reordering problem on gcmHerbert Xu2016-06-231-6/+12
| | | | | | | | | | | This patch fixes an old bug where gcm requests can be reordered because some are processed by cryptd while others are processed directly in softirq context. The fix is to always postpone to cryptd if there are currently requests outstanding from the same tfm. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: chacha20-simd - Use generic code for small requestsHerbert Xu2016-06-231-1/+1
| | | | | | | | | On 16-byte requests the optimised version is actually slower than the generic code, so we should simply use that instead. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Cheers,
* crypto: sha1-mb - stylistic cleanupMegha Dey2016-06-021-36/+74
| | | | | | | | | | Currently there are several checkpatch warnings in the sha1_mb.c file: 'WARNING: line over 80 characters' in the sha1_mb.c file. Also, the syntax of some multi-line comments are not correct. This patch fixes these issues. Signed-off-by: Megha Dey <megha.dey@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* crypto: sha-ssse3 - add MODULE_ALIASStephan Mueller2016-05-313-0/+22
| | | | | | | | Add the MODULE_ALIAS for the cra_driver_name of the different ciphers to allow an automated loading if a driver name is used. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* Merge branch 'linus' of ↵Linus Torvalds2016-05-201-2/+11
|\ | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 Pull crypto fix from Herbert Xu: "Fix a regression that causes sha-mb to crash" * 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: crypto: sha1-mb - make sha1_x8_avx2() conform to C function ABI
| * crypto: sha1-mb - make sha1_x8_avx2() conform to C function ABIJosh Poimboeuf2016-05-171-2/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Megha Dey reported a kernel panic in crypto code. The problem is that sha1_x8_avx2() clobbers registers r12-r15 without saving and restoring them. Before commit aec4d0e301f1 ("x86/asm/crypto: Simplify stack usage in sha-mb functions"), those registers were saved and restored by the callers of the function. I removed them with that commit because I didn't realize sha1_x8_avx2() clobbered them. Fix the potential undefined behavior associated with clobbering the registers and make the behavior less surprising by changing the registers to be callee saved/restored to conform with the C function call ABI. Also, rdx (aka RSP_SAVE) doesn't need to be saved: I verified that none of the callers rely on it being saved, and it's not a callee-saved register in the C ABI. Fixes: aec4d0e301f1 ("x86/asm/crypto: Simplify stack usage in sha-mb functions") Cc: stable@vger.kernel.org # 4.6 Reported-by: Megha Dey <megha.dey@linux.intel.com> Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* | Merge branch 'x86-cleanups-for-linus' of ↵Linus Torvalds2016-05-161-2/+2
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip Pull x86 cleanup from Ingo Molnar: "Inline optimizations" * 'x86-cleanups-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: x86: Fix non-static inlines
| * | x86: Fix non-static inlinesDenys Vlasenko2016-04-161-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Four instances of incorrect usage of non-static "inline" crept up in arch/x86, all trivial; cleaning them up: EVT_TO_HPET_DEV() - made static, it is only used in kernel/hpet.c Debug version of check_iommu_entries() is an __init function. Non-debug dummy empty version of it is declared "inline" instead - which doesn't help to eliminate it (the caller is in a different unit, inlining doesn't happen). Switch to non-inlined __init function, which does eliminate it (by discarding it as part of __init section). crypto/sha-mb/sha1_mb.c: looks like they just forgot to add "static" to their two internal inlines, which emitted two unused functions into vmlinux. text data bss dec hex filename 95903394 20860288 35991552 152755234 91adc22 vmlinux_before 95903266 20860288 35991552 152755106 91adba2 vmlinux Signed-off-by: Denys Vlasenko <dvlasenk@redhat.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-kernel@vger.kernel.org Link: http://lkml.kernel.org/r/1460739626-12179-1-git-send-email-dvlasenk@redhat.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
* | | Merge tag 'v4.6-rc6' into x86/asm, to refresh the treeIngo Molnar2016-05-051-2/+2
|\ \ \ | | |/ | |/| | | | Signed-off-by: Ingo Molnar <mingo@kernel.org>
| * | crypto: sha1-mb - use corrcet pointer while completing jobsXiaodong Liu2016-04-151-2/+2
| |/ | | | | | | | | | | | | | | | | | | | | | | In sha_complete_job, incorrect mcryptd_hash_request_ctx pointer is used when check and complete other jobs. If the memory of first completed req is freed, while still completing other jobs in the func, kernel will crash since NULL pointer is assigned to RIP. Cc: <stable@vger.kernel.org> Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com> Acked-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* | x86/cpufeature: Replace cpu_has_avx with boot_cpu_has() usageBorislav Petkov2016-04-138-8/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Borislav Petkov <bp@suse.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-crypto@vger.kernel.org Link: http://lkml.kernel.org/r/1459801503-15600-4-git-send-email-bp@alien8.de Signed-off-by: Ingo Molnar <mingo@kernel.org>
* | x86/cpufeature: Replace cpu_has_aes with boot_cpu_has() usageBorislav Petkov2016-04-132-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Borislav Petkov <bp@suse.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-crypto@vger.kernel.org Link: http://lkml.kernel.org/r/1459801503-15600-3-git-send-email-bp@alien8.de Signed-off-by: Ingo Molnar <mingo@kernel.org>
* | x86/cpufeature: Replace cpu_has_avx2 with boot_cpu_has() usageBorislav Petkov2016-04-134-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Borislav Petkov <bp@suse.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-crypto@vger.kernel.org Link: http://lkml.kernel.org/r/1459801503-15600-2-git-send-email-bp@alien8.de Signed-off-by: Ingo Molnar <mingo@kernel.org>
* | x86/cpufeature: Remove cpu_has_xmm2Borislav Petkov2016-03-312-2/+2
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Borislav Petkov <bp@suse.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-crypto@vger.kernel.org Link: http://lkml.kernel.org/r/1459266123-21878-8-git-send-email-bp@alien8.de Signed-off-by: Ingo Molnar <mingo@kernel.org>
* | x86/cpufeature: Remove cpu_has_osxsaveBorislav Petkov2016-03-313-3/+4
|/ | | | | | | | | | | | Use boot_cpu_has() instead. Signed-off-by: Borislav Petkov <bp@suse.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-crypto@vger.kernel.org Link: http://lkml.kernel.org/r/1459266123-21878-4-git-send-email-bp@alien8.de Signed-off-by: Ingo Molnar <mingo@kernel.org>
OpenPOWER on IntegriCloud