[PW_SID:1161443] optimize pgtable_l4|l5_enabled - #2630
Open
linux-riscv-bot wants to merge 5 commits into
Open
Conversation
riscv always selects RISCV_ALTERNATIVE now, so we can remove this Kconfig option and enable RISCV_ALTERNATIVE code unconditionally. Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
This is the preparation of optimizing pgtable_l4|l5_enabled(). No functionality change. Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
…bled The pgtable_l4|[l5]_enabled check sits at hot code path, performance is impacted a lot. Since pgtable_l4|[l5]_enabled isn't changed after boot, we can use alternative mechanism to optimize them. So the question is whether we can add RISCV_ISA_EXT_SV48/SV5 and use riscv_has_extension_*() or not. Per [1] and [2], SV48 and SV57 are ISA exensions too. From another side, riscv_has_extension_[un]likely() and other related functions report whether the extension is supported and enabled on the platform. So SV48 and SV57 can be supported with current isa extension alternative mechanism. However, to use it to optimize pgtable_l4|l5_enabled, we have support the "early" characteristic, I.E besides risc_isa bitmap setting, we need to support appling alternative early before MMU on. After that, use it to optimize pgtable_l4|l5_enabled. For the typical access_ok(addr, 1); before the patch: ... auipc a5,0xb43 lbu a5,100(a5) # ffffffff80b51f68 <pgtable_l5_enabled> bnez a5,ffffffff8000ef46 <foo+0x56> auipc a5,0xb43 lbu a5,91(a5) # ffffffff80b51f69 <pgtable_l4_enabled> beqz a5,ffffffff8000ef5a <foo+0x6a> ... after the patch: These memory load and test branch instructions are replaced with only two j or nop instructions. Initial test lmbench's lat_syscall write on TH1520 platforms shows that the write syscall latency is reduced by about 2.38%. Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Link: https://github.com/riscv/riscv-isa-manual/blob/main/src/profiles/profiles.adoc [1] Link: https://riscv.atlassian.net/wiki/spaces/HOME/pages/16154732/Ratified+ISA+Extensions [2] Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
In last commit, we have optimized the pgtable_l4|l5_enabled with isa extension alternative mechanism, thus a typical access_ok(addr, 1); is compiled as: ffffffff8001dd30: 1141 addi sp,sp,-16 ffffffff8001dd32: e022 sd s0,0(sp) ffffffff8001dd34: e406 sd ra,8(sp) ffffffff8001dd36: 0800 addi s0,sp,16 ffffffff8001dd38: 00000013 nop ffffffff8001dd3c: 02a0006f j ffffffff8001dd66 <foo+0x3e> ffffffff8001dd40: 60a2 ld ra,8(sp) ffffffff8001dd42: 6402 ld s0,0(sp) ffffffff8001dd44: 57fd li a5,-1 ffffffff8001dd46: 83a1 srli a5,a5,0x8 ffffffff8001dd48: 00a7b533 sltu a0,a5,a0 ffffffff8001dd4c: 00154513 xori a0,a0,1 ffffffff8001dd50: 0141 addi sp,sp,16 ffffffff8001dd52: 8082 ret ... ffffffff8001dd64: bfe1 j ffffffff8001dd3c <foo+0x14> ffffffff8001dd66: 0180006 j ffffffff8001dd7e <foo+0x56> ffffffff8001dd6a: 60a2 ld ra,8(sp) ffffffff8001dd6c: 6402 ld s0,0(sp) ffffffff8001dd6e: 57fd li a5,-1 ffffffff8001dd70: 83c5 srli a5,a5,0x11 ffffffff8001dd72: 00a7b533 sltu a0,a5,a0 ffffffff8001dd76: 00154513 xori a0,a0,1 ffffffff8001dd7a: 0141 addi sp,sp,16 ffffffff8001dd7c: 8082 ret ffffffff8001dd7e: 60a2 ld ra,8(sp) ffffffff8001dd80: 6402 ld s0,0(sp) ffffffff8001dd82: 57fd li a5,-1 ffffffff8001dd84: 83e9 srli a5,a5,0x1a ffffffff8001dd86: 00a7b533 sltu a0,a5,a0 ffffffff8001dd8a: 00154513 xori a0,a0,1 ffffffff8001dd8e: 0141 addi sp,sp,16 ffffffff8001dd90: 8082 ret As can be seen, different branches for SV39/SV48/SV57 are still there, since access_ok() sits at hot code path, why not reduce the code size to optimize to make the instruction cache happy? Introduce RISCV_ISA_SV48 and RISCV_ISA_SV57, so that the embedded platforms can choose the best option themselves, while still keep the feature of one unified kernel Image for all SV39, SV48 and SV57. Before the patch, vmlinux built with pure RISCV64 defconfig text data bss dec hex filename 13439437 6790446 483221 20713104 13c0e90 /tmp/old/vmlinux After the patch, vmlinux built with pure RISCV64 defconfig: text data bss dec hex filename 13314445 6787934 466837 20569216 139dc80 /tmp/new/vmlinux .text section is reduced by 122KB! And after the patch, typical access_ok(addr, 1) is compiled as: ffffffff8001809a: 1141 addi sp,sp,-16 ffffffff8001809c: e022 sd s0,0(sp) ffffffff8001809e: e406 sd ra,8(sp) ffffffff800180a0: 0800 addi s0,sp,16 ffffffff800180a2: 00000013 nop ffffffff800180a6: 60a2 ld ra,8(sp) ffffffff800180a8: 6402 ld s0,0(sp) ffffffff800180aa: 4785 li a5,1 ffffffff800180ac: 179a slli a5,a5,0x26 ffffffff800180ae: 00f53533 sltu a0,a0,a5 ffffffff800180b2: 0141 addi sp,sp,16 ffffffff800180b4: 8082 ret only 12 instruction!! For refernece, before the patch, the access(addr, 1) needs 32 instructions. Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Now, these two vars are only used by kernel itself, especially the code before MMU on. No modules users any more, unexport them. Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR for series 1161443 applied to workflow__riscv__fixes
Name: optimize pgtable_l4|l5_enabled
URL: https://patchwork.kernel.org/series/1161443/
Version: 3