This patch series ended up much larger than expected, please bear with me! The goal here is to support vendor extensions, starting at probing the device tree and ending with reporting to userspace.
The main design objective was to allow vendors to operate independently of each other. This has been achieved by delegating vendor extensions to a their own files and then accumulating the extensions in arch/riscv/kernel/vendor_extensions.c.
Each vendor will have their own list of extensions they support.
There is a new hwprobe key RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0 that is used to request which thead vendor extensions are supported on the current platform. This allows future vendors to allocate hwprobe keys for their vendor.
On to the xtheadvector specific code. xtheadvector is a custom extension that is based upon riscv vector version 0.7.1 [1]. All of the vector routines have been modified to support this alternative vector version based upon whether xtheadvector was determined to be supported at boot. I have tested this with an Allwinner Nezha board. I ran into issues booting the board on 6.9-rc1 so I applied these patches to 6.8. There are a couple of minor merge conflicts that do arrise when doing that, so please let me know if you have been able to boot this board with a 6.9 kernel. I used SkiffOS [2] to manage building the image, but upgraded the U-Boot version to Samuel Holland's more up-to-date version [3] and changed out the device tree used by U-Boot with the device trees that are present in upstream linux and this series. Thank you Samuel for all of the work you did to make this task possible.
To test the integration, I used the riscv vector kselftests. I modified the test cases to be able to more easily extend them, and then added a xtheadvector target that works by calling hwprobe and swapping out the vector asm if needed.
[1] https://github.com/T-head-Semi/thead-extension-spec/blob/95358cb2cca9489361c... [2] https://github.com/skiffos/SkiffOS/tree/master/configs/allwinner/nezha [3] https://github.com/smaeul/u-boot/commit/2e89b706f5c956a70c989cd31665f1429e9a...
Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- Changes in v3: - Allow any hardware to support any vendor extension, rather than restricting the vendor extensions to the same vendor as the hardware - Introduce config options to enable/disable a vendor's extensions - Link to v2: https://lore.kernel.org/r/20240415-dev-charlie-support_thead_vector_6_9-v2-0...
Changes in v2: - Added commit hash to xtheadvector - Simplified riscv,isa vector removal fix to not mess with the DT riscv,vendorid - Moved riscv,vendorid parsing into a different patch and cache the value to be used by alternative patching - Reduce riscv,vendorid missing severity to "info" - Separate vendor extension list to vendor files - xtheadvector no longer puts v in the elf_hwcap - Only patch vendor extension if all harts are associated with the same vendor. This is the best chance the kernel has for working properly if there are multiple vendors. - Split hwprobe vendor keys out into vendor file - Add attribution for Heiko's patches - Link to v1: https://lore.kernel.org/r/20240411-dev-charlie-support_thead_vector_6_9-v1-0...
--- Charlie Jenkins (15): riscv: cpufeature: Fix thead vector hwcap removal dt-bindings: riscv: Add xtheadvector ISA extension description riscv: vector: Use vlenb from DT riscv: dts: allwinner: Add xtheadvector to the D1/D1s devicetree riscv: Fix extension subset checking riscv: Extend cpufeature.c to detect vendor extensions riscv: Introduce vendor variants of extension helpers riscv: drivers: Convert xandespmu to use the vendor extension framework riscv: csr: Add CSR encodings for VCSR_VXRM/VCSR_VXSAT riscv: Add xtheadvector instruction definitions riscv: vector: Support xtheadvector save/restore riscv: hwprobe: Add thead vendor extension probing riscv: hwprobe: Document thead vendor extensions and xtheadvector extension selftests: riscv: Fix vector tests selftests: riscv: Support xtheadvector in vector tests
Conor Dooley (1): dt-bindings: riscv: cpus: add a vlen register length property
Heiko Stuebner (1): RISC-V: define the elements of the VCSR vector CSR
Documentation/arch/riscv/hwprobe.rst | 10 + Documentation/devicetree/bindings/riscv/cpus.yaml | 6 + .../devicetree/bindings/riscv/extensions.yaml | 10 + arch/riscv/Kconfig | 2 + arch/riscv/Kconfig.vendor | 32 +++ arch/riscv/boot/dts/allwinner/sun20i-d1s.dtsi | 3 +- arch/riscv/errata/sifive/errata.c | 3 + arch/riscv/errata/thead/errata.c | 3 + arch/riscv/include/asm/cpufeature.h | 106 +++++--- arch/riscv/include/asm/csr.h | 13 + arch/riscv/include/asm/hwcap.h | 1 - arch/riscv/include/asm/hwprobe.h | 4 +- arch/riscv/include/asm/sbi.h | 2 + arch/riscv/include/asm/switch_to.h | 2 +- arch/riscv/include/asm/vector.h | 247 +++++++++++++---- arch/riscv/include/asm/vendor_extensions.h | 82 ++++++ arch/riscv/include/asm/vendor_extensions/andes.h | 19 ++ arch/riscv/include/asm/vendor_extensions/thead.h | 45 ++++ .../include/asm/vendor_extensions/thead_hwprobe.h | 11 + arch/riscv/include/uapi/asm/hwprobe.h | 3 +- arch/riscv/include/uapi/asm/vendor/thead.h | 3 + arch/riscv/kernel/Makefile | 2 + arch/riscv/kernel/cpu.c | 40 ++- arch/riscv/kernel/cpufeature.c | 138 +++++++--- arch/riscv/kernel/kernel_mode_vector.c | 8 +- arch/riscv/kernel/process.c | 4 +- arch/riscv/kernel/signal.c | 6 +- arch/riscv/kernel/sys_hwprobe.c | 9 + arch/riscv/kernel/vector.c | 26 +- arch/riscv/kernel/vendor_extensions.c | 69 +++++ arch/riscv/kernel/vendor_extensions/Makefile | 5 + arch/riscv/kernel/vendor_extensions/andes.c | 35 +++ arch/riscv/kernel/vendor_extensions/thead.c | 36 +++ .../riscv/kernel/vendor_extensions/thead_hwprobe.c | 42 +++ drivers/perf/riscv_pmu_sbi.c | 8 +- tools/testing/selftests/riscv/vector/.gitignore | 3 +- tools/testing/selftests/riscv/vector/Makefile | 17 +- .../selftests/riscv/vector/v_exec_initval_nolibc.c | 93 +++++++ tools/testing/selftests/riscv/vector/v_helpers.c | 67 +++++ tools/testing/selftests/riscv/vector/v_helpers.h | 7 + tools/testing/selftests/riscv/vector/v_initval.c | 22 ++ .../selftests/riscv/vector/v_initval_nolibc.c | 68 ----- .../selftests/riscv/vector/vstate_exec_nolibc.c | 20 +- .../testing/selftests/riscv/vector/vstate_prctl.c | 295 ++++++++++++--------- 44 files changed, 1277 insertions(+), 350 deletions(-) --- base-commit: 4cece764965020c22cff7665b18a012006359095 change-id: 20240411-dev-charlie-support_thead_vector_6_9-1591fc2a431d
The riscv_cpuinfo struct that contains mvendorid and marchid is not populated until all harts are booted which happens after the DT parsing. Use the vendorid/archid values from the DT if available or assume all harts have the same values as the boot hart as a fallback.
Fixes: d82f32202e0d ("RISC-V: Ignore V from the riscv,isa DT property on older T-Head CPUs") Signed-off-by: Charlie Jenkins charlie@rivosinc.com Reviewed-by: Conor Dooley conor.dooley@microchip.com --- arch/riscv/include/asm/sbi.h | 2 ++ arch/riscv/kernel/cpu.c | 40 ++++++++++++++++++++++++++++++++++++---- arch/riscv/kernel/cpufeature.c | 12 ++++++++++-- 3 files changed, 48 insertions(+), 6 deletions(-)
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h index 6e68f8dff76b..0fab508a65b3 100644 --- a/arch/riscv/include/asm/sbi.h +++ b/arch/riscv/include/asm/sbi.h @@ -370,6 +370,8 @@ static inline int sbi_remote_fence_i(const struct cpumask *cpu_mask) { return -1 static inline void sbi_init(void) {} #endif /* CONFIG_RISCV_SBI */
+unsigned long riscv_get_mvendorid(void); +unsigned long riscv_get_marchid(void); unsigned long riscv_cached_mvendorid(unsigned int cpu_id); unsigned long riscv_cached_marchid(unsigned int cpu_id); unsigned long riscv_cached_mimpid(unsigned int cpu_id); diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c index d11d6320fb0d..c1f3655238fd 100644 --- a/arch/riscv/kernel/cpu.c +++ b/arch/riscv/kernel/cpu.c @@ -139,6 +139,34 @@ int riscv_of_parent_hartid(struct device_node *node, unsigned long *hartid) return -1; }
+unsigned long __init riscv_get_marchid(void) +{ + struct riscv_cpuinfo *ci = this_cpu_ptr(&riscv_cpuinfo); + +#if IS_ENABLED(CONFIG_RISCV_SBI) + ci->marchid = sbi_spec_is_0_1() ? 0 : sbi_get_marchid(); +#elif IS_ENABLED(CONFIG_RISCV_M_MODE) + ci->marchid = csr_read(CSR_MARCHID); +#else + ci->marchid = 0; +#endif + return ci->marchid; +} + +unsigned long __init riscv_get_mvendorid(void) +{ + struct riscv_cpuinfo *ci = this_cpu_ptr(&riscv_cpuinfo); + +#if IS_ENABLED(CONFIG_RISCV_SBI) + ci->mvendorid = sbi_spec_is_0_1() ? 0 : sbi_get_mvendorid(); +#elif IS_ENABLED(CONFIG_RISCV_M_MODE) + ci->mvendorid = csr_read(CSR_MVENDORID); +#else + ci->mvendorid = 0; +#endif + return ci->mvendorid; +} + DEFINE_PER_CPU(struct riscv_cpuinfo, riscv_cpuinfo);
unsigned long riscv_cached_mvendorid(unsigned int cpu_id) @@ -170,12 +198,16 @@ static int riscv_cpuinfo_starting(unsigned int cpu) struct riscv_cpuinfo *ci = this_cpu_ptr(&riscv_cpuinfo);
#if IS_ENABLED(CONFIG_RISCV_SBI) - ci->mvendorid = sbi_spec_is_0_1() ? 0 : sbi_get_mvendorid(); - ci->marchid = sbi_spec_is_0_1() ? 0 : sbi_get_marchid(); + if (!ci->mvendorid) + ci->mvendorid = sbi_spec_is_0_1() ? 0 : sbi_get_mvendorid(); + if (!ci->marchid) + ci->marchid = sbi_spec_is_0_1() ? 0 : sbi_get_marchid(); ci->mimpid = sbi_spec_is_0_1() ? 0 : sbi_get_mimpid(); #elif IS_ENABLED(CONFIG_RISCV_M_MODE) - ci->mvendorid = csr_read(CSR_MVENDORID); - ci->marchid = csr_read(CSR_MARCHID); + if (!ci->mvendorid) + ci->mvendorid = csr_read(CSR_MVENDORID); + if (!ci->marchid) + ci->marchid = csr_read(CSR_MARCHID); ci->mimpid = csr_read(CSR_MIMPID); #else ci->mvendorid = 0; diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 3ed2359eae35..c6e27b45e192 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -490,6 +490,8 @@ static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwcap) struct acpi_table_header *rhct; acpi_status status; unsigned int cpu; + u64 boot_vendorid; + u64 boot_archid;
if (!acpi_disabled) { status = acpi_get_table(ACPI_SIG_RHCT, 0, &rhct); @@ -497,6 +499,13 @@ static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwcap) return; }
+ /* + * Naively assume that all harts have the same mvendorid/marchid as the + * boot hart. + */ + boot_vendorid = riscv_get_mvendorid(); + boot_archid = riscv_get_marchid(); + for_each_possible_cpu(cpu) { struct riscv_isainfo *isainfo = &hart_isa[cpu]; unsigned long this_hwcap = 0; @@ -544,8 +553,7 @@ static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwcap) * CPU cores with the ratified spec will contain non-zero * marchid. */ - if (acpi_disabled && riscv_cached_mvendorid(cpu) == THEAD_VENDOR_ID && - riscv_cached_marchid(cpu) == 0x0) { + if (acpi_disabled && boot_vendorid == THEAD_VENDOR_ID && boot_archid == 0x0) { this_hwcap &= ~isa2hwcap[RISCV_ISA_EXT_v]; clear_bit(RISCV_ISA_EXT_v, isainfo->isa); }
On Sun, Apr 21, 2024 at 9:04 AM Charlie Jenkins charlie@rivosinc.com wrote:
The riscv_cpuinfo struct that contains mvendorid and marchid is not populated until all harts are booted which happens after the DT parsing. Use the vendorid/archid values from the DT if available or assume all harts have the same values as the boot hart as a fallback.
Fixes: d82f32202e0d ("RISC-V: Ignore V from the riscv,isa DT property on older T-Head CPUs") Signed-off-by: Charlie Jenkins charlie@rivosinc.com Reviewed-by: Conor Dooley conor.dooley@microchip.com
arch/riscv/include/asm/sbi.h | 2 ++ arch/riscv/kernel/cpu.c | 40 ++++++++++++++++++++++++++++++++++++---- arch/riscv/kernel/cpufeature.c | 12 ++++++++++-- 3 files changed, 48 insertions(+), 6 deletions(-)
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h index 6e68f8dff76b..0fab508a65b3 100644 --- a/arch/riscv/include/asm/sbi.h +++ b/arch/riscv/include/asm/sbi.h @@ -370,6 +370,8 @@ static inline int sbi_remote_fence_i(const struct cpumask *cpu_mask) { return -1 static inline void sbi_init(void) {} #endif /* CONFIG_RISCV_SBI */
+unsigned long riscv_get_mvendorid(void); +unsigned long riscv_get_marchid(void); unsigned long riscv_cached_mvendorid(unsigned int cpu_id); unsigned long riscv_cached_marchid(unsigned int cpu_id); unsigned long riscv_cached_mimpid(unsigned int cpu_id); diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c index d11d6320fb0d..c1f3655238fd 100644 --- a/arch/riscv/kernel/cpu.c +++ b/arch/riscv/kernel/cpu.c @@ -139,6 +139,34 @@ int riscv_of_parent_hartid(struct device_node *node, unsigned long *hartid) return -1; }
+unsigned long __init riscv_get_marchid(void) +{
struct riscv_cpuinfo *ci = this_cpu_ptr(&riscv_cpuinfo);+#if IS_ENABLED(CONFIG_RISCV_SBI)
ci->marchid = sbi_spec_is_0_1() ? 0 : sbi_get_marchid();+#elif IS_ENABLED(CONFIG_RISCV_M_MODE)
ci->marchid = csr_read(CSR_MARCHID);+#else
ci->marchid = 0;+#endif
return ci->marchid;+}
+unsigned long __init riscv_get_mvendorid(void) +{
struct riscv_cpuinfo *ci = this_cpu_ptr(&riscv_cpuinfo);+#if IS_ENABLED(CONFIG_RISCV_SBI)
ci->mvendorid = sbi_spec_is_0_1() ? 0 : sbi_get_mvendorid();+#elif IS_ENABLED(CONFIG_RISCV_M_MODE)
ci->mvendorid = csr_read(CSR_MVENDORID);+#else
ci->mvendorid = 0;+#endif
return ci->mvendorid;+}
DEFINE_PER_CPU(struct riscv_cpuinfo, riscv_cpuinfo);
unsigned long riscv_cached_mvendorid(unsigned int cpu_id) @@ -170,12 +198,16 @@ static int riscv_cpuinfo_starting(unsigned int cpu) struct riscv_cpuinfo *ci = this_cpu_ptr(&riscv_cpuinfo);
#if IS_ENABLED(CONFIG_RISCV_SBI)
ci->mvendorid = sbi_spec_is_0_1() ? 0 : sbi_get_mvendorid();ci->marchid = sbi_spec_is_0_1() ? 0 : sbi_get_marchid();
if (!ci->mvendorid)ci->mvendorid = sbi_spec_is_0_1() ? 0 : sbi_get_mvendorid();if (!ci->marchid)ci->marchid = sbi_spec_is_0_1() ? 0 : sbi_get_marchid(); ci->mimpid = sbi_spec_is_0_1() ? 0 : sbi_get_mimpid();#elif IS_ENABLED(CONFIG_RISCV_M_MODE)
ci->mvendorid = csr_read(CSR_MVENDORID);ci->marchid = csr_read(CSR_MARCHID);
if (!ci->mvendorid)ci->mvendorid = csr_read(CSR_MVENDORID);if (!ci->marchid)ci->marchid = csr_read(CSR_MARCHID); ci->mimpid = csr_read(CSR_MIMPID);#else ci->mvendorid = 0; diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 3ed2359eae35..c6e27b45e192 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -490,6 +490,8 @@ static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwcap) struct acpi_table_header *rhct; acpi_status status; unsigned int cpu;
u64 boot_vendorid;u64 boot_archid; if (!acpi_disabled) { status = acpi_get_table(ACPI_SIG_RHCT, 0, &rhct);@@ -497,6 +499,13 @@ static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwcap) return; }
/** Naively assume that all harts have the same mvendorid/marchid as the* boot hart.*/boot_vendorid = riscv_get_mvendorid();boot_archid = riscv_get_marchid();for_each_possible_cpu(cpu) { struct riscv_isainfo *isainfo = &hart_isa[cpu]; unsigned long this_hwcap = 0;@@ -544,8 +553,7 @@ static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwcap) * CPU cores with the ratified spec will contain non-zero * marchid. */
if (acpi_disabled && riscv_cached_mvendorid(cpu) == THEAD_VENDOR_ID &&riscv_cached_marchid(cpu) == 0x0) {
if (acpi_disabled && boot_vendorid == THEAD_VENDOR_ID && boot_archid == 0x0) {
LGTM! Reviewed-by: Guo Ren guoren@kernel.org
this_hwcap &= ~isa2hwcap[RISCV_ISA_EXT_v]; clear_bit(RISCV_ISA_EXT_v, isainfo->isa); }-- 2.44.0
The xtheadvector ISA extension is described on the T-Head extension spec Github page [1] at commit 95358cb2cca9.
Link: https://github.com/T-head-Semi/thead-extension-spec/blob/95358cb2cca9489361c... [1]
Signed-off-by: Charlie Jenkins charlie@rivosinc.com Reviewed-by: Conor Dooley conor.dooley@microchip.com --- Documentation/devicetree/bindings/riscv/extensions.yaml | 10 ++++++++++ 1 file changed, 10 insertions(+)
diff --git a/Documentation/devicetree/bindings/riscv/extensions.yaml b/Documentation/devicetree/bindings/riscv/extensions.yaml index 468c646247aa..99d2a9e8c52d 100644 --- a/Documentation/devicetree/bindings/riscv/extensions.yaml +++ b/Documentation/devicetree/bindings/riscv/extensions.yaml @@ -477,6 +477,10 @@ properties: latency, as ratified in commit 56ed795 ("Update riscv-crypto-spec-vector.adoc") of riscv-crypto.
+ # vendor extensions, each extension sorted alphanumerically under the + # vendor they belong to. Vendors are sorted alphanumerically as well. + + # Andes - const: xandespmu description: The Andes Technology performance monitor extension for counter overflow @@ -484,5 +488,11 @@ properties: Registers in the AX45MP datasheet. https://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet....
+ # T-HEAD + - const: xtheadvector + description: + The T-HEAD specific 0.7.1 vector implementation as written in + https://github.com/T-head-Semi/thead-extension-spec/blob/95358cb2cca9489361c.... + additionalProperties: true ...
From: Conor Dooley conor.dooley@microchip.com
Add a property analogous to the vlenb CSR so that software can detect the vector length of each CPU prior to it being brought online. Currently software has to assume that the vector length read from the boot CPU applies to all possible CPUs. On T-Head CPUs implementing pre-ratification vector, reading the th.vlenb CSR may produce an illegal instruction trap, so this property is required on such systems.
Signed-off-by: Conor Dooley conor.dooley@microchip.com Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- Documentation/devicetree/bindings/riscv/cpus.yaml | 6 ++++++ 1 file changed, 6 insertions(+)
diff --git a/Documentation/devicetree/bindings/riscv/cpus.yaml b/Documentation/devicetree/bindings/riscv/cpus.yaml index d87dd50f1a4b..edcb6a7d9319 100644 --- a/Documentation/devicetree/bindings/riscv/cpus.yaml +++ b/Documentation/devicetree/bindings/riscv/cpus.yaml @@ -94,6 +94,12 @@ properties: description: The blocksize in bytes for the Zicboz cache operations.
+ riscv,vlenb: + $ref: /schemas/types.yaml#/definitions/uint32 + description: + VLEN/8, the vector register length in bytes. This property is required in + systems where the vector register length is not identical on all harts. + # RISC-V has multiple properties for cache op block sizes as the sizes # differ between individual CBO extensions cache-op-block-size: false
If vlenb is provided in the device tree, prefer that over reading the vlenb csr.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- arch/riscv/include/asm/cpufeature.h | 2 ++ arch/riscv/kernel/cpufeature.c | 26 ++++++++++++++++++++++++++ arch/riscv/kernel/vector.c | 13 +++++++++---- 3 files changed, 37 insertions(+), 4 deletions(-)
diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index 347805446151..809f61ffb667 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -31,6 +31,8 @@ DECLARE_PER_CPU(struct riscv_cpuinfo, riscv_cpuinfo); /* Per-cpu ISA extensions. */ extern struct riscv_isainfo hart_isa[NR_CPUS];
+extern u32 riscv_vlenb_dt[NR_CPUS]; + void riscv_user_isa_enable(void);
#if defined(CONFIG_RISCV_MISALIGNED) diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index c6e27b45e192..48874aac4871 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -35,6 +35,8 @@ static DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly; /* Per-cpu ISA extensions. */ struct riscv_isainfo hart_isa[NR_CPUS];
+u32 riscv_vlenb_dt[NR_CPUS]; + /** * riscv_isa_extension_base() - Get base extension word * @@ -656,6 +658,28 @@ static int __init riscv_isa_fallback_setup(char *__unused) early_param("riscv_isa_fallback", riscv_isa_fallback_setup); #endif
+static void riscv_set_vlenb_from_dt(void) +{ + int cpu; + + for_each_possible_cpu(cpu) { + struct device_node *cpu_node; + + cpu_node = of_cpu_device_node_get(cpu); + if (!cpu_node) { + pr_warn("Unable to find cpu node\n"); + continue; + } + + if (!of_property_read_u32(cpu_node, "riscv,vlenb", &riscv_vlenb_dt[cpu])) { + of_node_put(cpu_node); + continue; + } + + of_node_put(cpu_node); + } +} + void __init riscv_fill_hwcap(void) { char print_str[NUM_ALPHA_EXTS + 1]; @@ -675,6 +699,8 @@ void __init riscv_fill_hwcap(void) } else { int ret = riscv_fill_hwcap_from_ext_list(isa2hwcap);
+ riscv_set_vlenb_from_dt(); + if (ret && riscv_isa_fallback) { pr_info("Falling back to deprecated "riscv,isa"\n"); riscv_fill_hwcap_from_isa_string(isa2hwcap); diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c index 6727d1d3b8f2..fb7f3ca80d9e 100644 --- a/arch/riscv/kernel/vector.c +++ b/arch/riscv/kernel/vector.c @@ -32,11 +32,16 @@ EXPORT_SYMBOL_GPL(riscv_v_vsize); int riscv_v_setup_vsize(void) { unsigned long this_vsize; + int cpu = smp_processor_id();
- /* There are 32 vector registers with vlenb length. */ - riscv_v_enable(); - this_vsize = csr_read(CSR_VLENB) * 32; - riscv_v_disable(); + if (riscv_vlenb_dt[cpu]) { + this_vsize = riscv_vlenb_dt[cpu]; + } else { + /* There are 32 vector registers with vlenb length. */ + riscv_v_enable(); + this_vsize = csr_read(CSR_VLENB) * 32; + riscv_v_disable(); + }
if (!riscv_v_vsize) { riscv_v_vsize = this_vsize;
On Sat, Apr 20, 2024 at 06:04:36PM -0700, Charlie Jenkins wrote:
If vlenb is provided in the device tree, prefer that over reading the vlenb csr.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com
arch/riscv/include/asm/cpufeature.h | 2 ++ arch/riscv/kernel/cpufeature.c | 26 ++++++++++++++++++++++++++ arch/riscv/kernel/vector.c | 13 +++++++++---- 3 files changed, 37 insertions(+), 4 deletions(-)
diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index 347805446151..809f61ffb667 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -31,6 +31,8 @@ DECLARE_PER_CPU(struct riscv_cpuinfo, riscv_cpuinfo); /* Per-cpu ISA extensions. */ extern struct riscv_isainfo hart_isa[NR_CPUS]; +extern u32 riscv_vlenb_dt[NR_CPUS];
void riscv_user_isa_enable(void); #if defined(CONFIG_RISCV_MISALIGNED) diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index c6e27b45e192..48874aac4871 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -35,6 +35,8 @@ static DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly; /* Per-cpu ISA extensions. */ struct riscv_isainfo hart_isa[NR_CPUS]; +u32 riscv_vlenb_dt[NR_CPUS];
/**
- riscv_isa_extension_base() - Get base extension word
@@ -656,6 +658,28 @@ static int __init riscv_isa_fallback_setup(char *__unused) early_param("riscv_isa_fallback", riscv_isa_fallback_setup); #endif +static void riscv_set_vlenb_from_dt(void)
I'd expect to see a name here that had "of" in it, not "dt".
+{
- int cpu;
- for_each_possible_cpu(cpu) {
struct device_node *cpu_node;cpu_node = of_cpu_device_node_get(cpu);if (!cpu_node) {pr_warn("Unable to find cpu node\n");continue;}if (!of_property_read_u32(cpu_node, "riscv,vlenb", &riscv_vlenb_dt[cpu])) {of_node_put(cpu_node);continue;}of_node_put(cpu_node);- }
+}
void __init riscv_fill_hwcap(void) { char print_str[NUM_ALPHA_EXTS + 1]; @@ -675,6 +699,8 @@ void __init riscv_fill_hwcap(void) } else { int ret = riscv_fill_hwcap_from_ext_list(isa2hwcap);
riscv_set_vlenb_from_dt();
Hmm, I think we can go a step further here. We know all of the CPUs widths by the time we get to the first call to riscv_v_setup_vsize(), can we examine the whole list and decide not to enable vector if they do not match, rather than continuing and failing to online CPUs that having the mismatched size?
I guess that can go into the `if (elf_hwcap & COMPAT_HWCAP_ISA_V)` condition we already have, and would require clearing the bit from the mask we have at the moment.
Cheers, Conor.
- if (ret && riscv_isa_fallback) { pr_info("Falling back to deprecated "riscv,isa"\n"); riscv_fill_hwcap_from_isa_string(isa2hwcap);
diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c index 6727d1d3b8f2..fb7f3ca80d9e 100644 --- a/arch/riscv/kernel/vector.c +++ b/arch/riscv/kernel/vector.c @@ -32,11 +32,16 @@ EXPORT_SYMBOL_GPL(riscv_v_vsize); int riscv_v_setup_vsize(void) { unsigned long this_vsize;
- int cpu = smp_processor_id();
- /* There are 32 vector registers with vlenb length. */
- riscv_v_enable();
- this_vsize = csr_read(CSR_VLENB) * 32;
- riscv_v_disable();
- if (riscv_vlenb_dt[cpu]) {
this_vsize = riscv_vlenb_dt[cpu];
- } else {
/* There are 32 vector registers with vlenb length. */riscv_v_enable();this_vsize = csr_read(CSR_VLENB) * 32;riscv_v_disable();- }
if (!riscv_v_vsize) { riscv_v_vsize = this_vsize;
-- 2.44.0
On Fri, Apr 26, 2024 at 04:17:52PM +0100, Conor Dooley wrote:
On Sat, Apr 20, 2024 at 06:04:36PM -0700, Charlie Jenkins wrote:
If vlenb is provided in the device tree, prefer that over reading the vlenb csr.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com
arch/riscv/include/asm/cpufeature.h | 2 ++ arch/riscv/kernel/cpufeature.c | 26 ++++++++++++++++++++++++++ arch/riscv/kernel/vector.c | 13 +++++++++---- 3 files changed, 37 insertions(+), 4 deletions(-)
diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index 347805446151..809f61ffb667 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -31,6 +31,8 @@ DECLARE_PER_CPU(struct riscv_cpuinfo, riscv_cpuinfo); /* Per-cpu ISA extensions. */ extern struct riscv_isainfo hart_isa[NR_CPUS]; +extern u32 riscv_vlenb_dt[NR_CPUS];
void riscv_user_isa_enable(void); #if defined(CONFIG_RISCV_MISALIGNED) diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index c6e27b45e192..48874aac4871 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -35,6 +35,8 @@ static DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly; /* Per-cpu ISA extensions. */ struct riscv_isainfo hart_isa[NR_CPUS]; +u32 riscv_vlenb_dt[NR_CPUS];
/**
- riscv_isa_extension_base() - Get base extension word
@@ -656,6 +658,28 @@ static int __init riscv_isa_fallback_setup(char *__unused) early_param("riscv_isa_fallback", riscv_isa_fallback_setup); #endif +static void riscv_set_vlenb_from_dt(void)
I'd expect to see a name here that had "of" in it, not "dt".
Also, "set" - I think "get" is more suitable here given that this doesn't actually set the vlen, we only do any setting later on in riscv_v_set_vsize().
+{
- int cpu;
- for_each_possible_cpu(cpu) {
struct device_node *cpu_node;cpu_node = of_cpu_device_node_get(cpu);if (!cpu_node) {pr_warn("Unable to find cpu node\n");continue;}if (!of_property_read_u32(cpu_node, "riscv,vlenb", &riscv_vlenb_dt[cpu])) {of_node_put(cpu_node);continue;}of_node_put(cpu_node);- }
+}
void __init riscv_fill_hwcap(void) { char print_str[NUM_ALPHA_EXTS + 1]; @@ -675,6 +699,8 @@ void __init riscv_fill_hwcap(void) } else { int ret = riscv_fill_hwcap_from_ext_list(isa2hwcap);
riscv_set_vlenb_from_dt();Hmm, I think we can go a step further here. We know all of the CPUs widths by the time we get to the first call to riscv_v_setup_vsize(), can we examine the whole list and decide not to enable vector if they do not match, rather than continuing and failing to online CPUs that having the mismatched size?
I guess that can go into the `if (elf_hwcap & COMPAT_HWCAP_ISA_V)` condition we already have, and would require clearing the bit from the mask we have at the moment.
Cheers, Conor.
- if (ret && riscv_isa_fallback) { pr_info("Falling back to deprecated "riscv,isa"\n"); riscv_fill_hwcap_from_isa_string(isa2hwcap);
diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c index 6727d1d3b8f2..fb7f3ca80d9e 100644 --- a/arch/riscv/kernel/vector.c +++ b/arch/riscv/kernel/vector.c @@ -32,11 +32,16 @@ EXPORT_SYMBOL_GPL(riscv_v_vsize); int riscv_v_setup_vsize(void) { unsigned long this_vsize;
- int cpu = smp_processor_id();
- /* There are 32 vector registers with vlenb length. */
- riscv_v_enable();
- this_vsize = csr_read(CSR_VLENB) * 32;
- riscv_v_disable();
- if (riscv_vlenb_dt[cpu]) {
this_vsize = riscv_vlenb_dt[cpu];
- } else {
/* There are 32 vector registers with vlenb length. */riscv_v_enable();this_vsize = csr_read(CSR_VLENB) * 32;riscv_v_disable();- }
if (!riscv_v_vsize) { riscv_v_vsize = this_vsize;
-- 2.44.0
On Fri, Apr 26, 2024 at 05:21:16PM +0100, Conor Dooley wrote:
On Fri, Apr 26, 2024 at 04:17:52PM +0100, Conor Dooley wrote:
On Sat, Apr 20, 2024 at 06:04:36PM -0700, Charlie Jenkins wrote:
If vlenb is provided in the device tree, prefer that over reading the vlenb csr.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com
arch/riscv/include/asm/cpufeature.h | 2 ++ arch/riscv/kernel/cpufeature.c | 26 ++++++++++++++++++++++++++ arch/riscv/kernel/vector.c | 13 +++++++++---- 3 files changed, 37 insertions(+), 4 deletions(-)
diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index 347805446151..809f61ffb667 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -31,6 +31,8 @@ DECLARE_PER_CPU(struct riscv_cpuinfo, riscv_cpuinfo); /* Per-cpu ISA extensions. */ extern struct riscv_isainfo hart_isa[NR_CPUS]; +extern u32 riscv_vlenb_dt[NR_CPUS];
void riscv_user_isa_enable(void); #if defined(CONFIG_RISCV_MISALIGNED) diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index c6e27b45e192..48874aac4871 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -35,6 +35,8 @@ static DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly; /* Per-cpu ISA extensions. */ struct riscv_isainfo hart_isa[NR_CPUS]; +u32 riscv_vlenb_dt[NR_CPUS];
/**
- riscv_isa_extension_base() - Get base extension word
@@ -656,6 +658,28 @@ static int __init riscv_isa_fallback_setup(char *__unused) early_param("riscv_isa_fallback", riscv_isa_fallback_setup); #endif +static void riscv_set_vlenb_from_dt(void)
I'd expect to see a name here that had "of" in it, not "dt".
Also, "set" - I think "get" is more suitable here given that this doesn't actually set the vlen, we only do any setting later on in riscv_v_set_vsize().
+{
- int cpu;
- for_each_possible_cpu(cpu) {
struct device_node *cpu_node;cpu_node = of_cpu_device_node_get(cpu);if (!cpu_node) {pr_warn("Unable to find cpu node\n");continue;}if (!of_property_read_u32(cpu_node, "riscv,vlenb", &riscv_vlenb_dt[cpu])) {of_node_put(cpu_node);continue;}of_node_put(cpu_node);- }
+}
void __init riscv_fill_hwcap(void) { char print_str[NUM_ALPHA_EXTS + 1]; @@ -675,6 +699,8 @@ void __init riscv_fill_hwcap(void) } else { int ret = riscv_fill_hwcap_from_ext_list(isa2hwcap);
riscv_set_vlenb_from_dt();Hmm, I think we can go a step further here. We know all of the CPUs widths by the time we get to the first call to riscv_v_setup_vsize(), can we examine the whole list and decide not to enable vector if they do not match, rather than continuing and failing to online CPUs that having the mismatched size?
I guess that can go into the `if (elf_hwcap & COMPAT_HWCAP_ISA_V)` condition we already have, and would require clearing the bit from the mask we have at the moment.
Good point, thank you. Since this is not supported with ACPI, I will clear the COMPAT_HWCAP_ISA_V bit from elf_hwcap if riscv_set_vlenb_from_dt() (which will be renamed) determines that the the riscv,vlenb field changes between CPU entries.
- Charlie
Cheers, Conor.
- if (ret && riscv_isa_fallback) { pr_info("Falling back to deprecated "riscv,isa"\n"); riscv_fill_hwcap_from_isa_string(isa2hwcap);
diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c index 6727d1d3b8f2..fb7f3ca80d9e 100644 --- a/arch/riscv/kernel/vector.c +++ b/arch/riscv/kernel/vector.c @@ -32,11 +32,16 @@ EXPORT_SYMBOL_GPL(riscv_v_vsize); int riscv_v_setup_vsize(void) { unsigned long this_vsize;
- int cpu = smp_processor_id();
- /* There are 32 vector registers with vlenb length. */
- riscv_v_enable();
- this_vsize = csr_read(CSR_VLENB) * 32;
- riscv_v_disable();
- if (riscv_vlenb_dt[cpu]) {
this_vsize = riscv_vlenb_dt[cpu];
- } else {
/* There are 32 vector registers with vlenb length. */riscv_v_enable();this_vsize = csr_read(CSR_VLENB) * 32;riscv_v_disable();- }
if (!riscv_v_vsize) { riscv_v_vsize = this_vsize;
-- 2.44.0
The D1/D1s SoCs support xtheadvector so it can be included in the devicetree. Also include vlenb for the cpu.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- arch/riscv/boot/dts/allwinner/sun20i-d1s.dtsi | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/boot/dts/allwinner/sun20i-d1s.dtsi b/arch/riscv/boot/dts/allwinner/sun20i-d1s.dtsi index 64c3c2e6cbe0..50c9f4ec8a7f 100644 --- a/arch/riscv/boot/dts/allwinner/sun20i-d1s.dtsi +++ b/arch/riscv/boot/dts/allwinner/sun20i-d1s.dtsi @@ -27,7 +27,8 @@ cpu0: cpu@0 { riscv,isa = "rv64imafdc"; riscv,isa-base = "rv64i"; riscv,isa-extensions = "i", "m", "a", "f", "d", "c", "zicntr", "zicsr", - "zifencei", "zihpm"; + "zifencei", "zihpm", "xtheadvector"; + riscv,vlenb = <128>; #cooling-cells = <2>;
cpu0_intc: interrupt-controller {
This loop is supposed to check if ext->subset_ext_ids[j] is valid, rather than if ext->subset_ext_ids[i] is valid, before setting the extension id ext->subset_ext_ids[j] in isainfo->isa.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com Reviewed-by: Conor Dooley conor.dooley@microchip.com Fixes: 0d8295ed975b ("riscv: add ISA extension parsing for scalar crypto") --- arch/riscv/kernel/cpufeature.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 48874aac4871..b537731cadef 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -609,7 +609,7 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap)
if (ext->subset_ext_size) { for (int j = 0; j < ext->subset_ext_size; j++) { - if (riscv_isa_extension_check(ext->subset_ext_ids[i])) + if (riscv_isa_extension_check(ext->subset_ext_ids[j])) set_bit(ext->subset_ext_ids[j], isainfo->isa); } }
Hi Charlie,
On 21/04/2024 03:04, Charlie Jenkins wrote:
This loop is supposed to check if ext->subset_ext_ids[j] is valid, rather than if ext->subset_ext_ids[i] is valid, before setting the extension id ext->subset_ext_ids[j] in isainfo->isa.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com Reviewed-by: Conor Dooley conor.dooley@microchip.com Fixes: 0d8295ed975b ("riscv: add ISA extension parsing for scalar crypto")
arch/riscv/kernel/cpufeature.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 48874aac4871..b537731cadef 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -609,7 +609,7 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) if (ext->subset_ext_size) { for (int j = 0; j < ext->subset_ext_size; j++) {
if (riscv_isa_extension_check(ext->subset_ext_ids[i]))
if (riscv_isa_extension_check(ext->subset_ext_ids[j])) set_bit(ext->subset_ext_ids[j], isainfo->isa); } }
I think this should go into -fixes, let's check with Palmer if he wants to take this patch only or if you should send the patch on its own.
You can add:
Reviewed-by: Alexandre Ghiti alexghiti@rivosinc.com
Thanks,
Alex
On Wed, Apr 24, 2024 at 04:22:02PM +0200, Alexandre Ghiti wrote:
Hi Charlie,
On 21/04/2024 03:04, Charlie Jenkins wrote:
This loop is supposed to check if ext->subset_ext_ids[j] is valid, rather than if ext->subset_ext_ids[i] is valid, before setting the extension id ext->subset_ext_ids[j] in isainfo->isa.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com Reviewed-by: Conor Dooley conor.dooley@microchip.com Fixes: 0d8295ed975b ("riscv: add ISA extension parsing for scalar crypto")
arch/riscv/kernel/cpufeature.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 48874aac4871..b537731cadef 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -609,7 +609,7 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) if (ext->subset_ext_size) { for (int j = 0; j < ext->subset_ext_size; j++) {
if (riscv_isa_extension_check(ext->subset_ext_ids[i]))
if (riscv_isa_extension_check(ext->subset_ext_ids[j])) set_bit(ext->subset_ext_ids[j], isainfo->isa); } }I think this should go into -fixes, let's check with Palmer if he wants to take this patch only or if you should send the patch on its own.
I think splitting out this and patch 1 into a new series targeting fixes would probably make things clearer?
On Wed, Apr 24, 2024 at 03:51:54PM +0100, Conor Dooley wrote:
On Wed, Apr 24, 2024 at 04:22:02PM +0200, Alexandre Ghiti wrote:
Hi Charlie,
On 21/04/2024 03:04, Charlie Jenkins wrote:
This loop is supposed to check if ext->subset_ext_ids[j] is valid, rather than if ext->subset_ext_ids[i] is valid, before setting the extension id ext->subset_ext_ids[j] in isainfo->isa.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com Reviewed-by: Conor Dooley conor.dooley@microchip.com Fixes: 0d8295ed975b ("riscv: add ISA extension parsing for scalar crypto")
arch/riscv/kernel/cpufeature.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 48874aac4871..b537731cadef 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -609,7 +609,7 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) if (ext->subset_ext_size) { for (int j = 0; j < ext->subset_ext_size; j++) {
if (riscv_isa_extension_check(ext->subset_ext_ids[i]))
if (riscv_isa_extension_check(ext->subset_ext_ids[j])) set_bit(ext->subset_ext_ids[j], isainfo->isa); } }I think this should go into -fixes, let's check with Palmer if he wants to take this patch only or if you should send the patch on its own.
I think splitting out this and patch 1 into a new series targeting fixes would probably make things clearer?
Okay I can do that. I will give it a bit more time before I send this series split into two to allow time for the rest of the patches to gather comments so I avoid sending too many duplicate patches.
- Charlie
On Wed, Apr 24, 2024 at 11:13:40AM -0400, Charlie Jenkins wrote:
On Wed, Apr 24, 2024 at 03:51:54PM +0100, Conor Dooley wrote:
On Wed, Apr 24, 2024 at 04:22:02PM +0200, Alexandre Ghiti wrote:
Hi Charlie,
On 21/04/2024 03:04, Charlie Jenkins wrote:
This loop is supposed to check if ext->subset_ext_ids[j] is valid, rather than if ext->subset_ext_ids[i] is valid, before setting the extension id ext->subset_ext_ids[j] in isainfo->isa.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com Reviewed-by: Conor Dooley conor.dooley@microchip.com Fixes: 0d8295ed975b ("riscv: add ISA extension parsing for scalar crypto")
arch/riscv/kernel/cpufeature.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 48874aac4871..b537731cadef 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -609,7 +609,7 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) if (ext->subset_ext_size) { for (int j = 0; j < ext->subset_ext_size; j++) {
if (riscv_isa_extension_check(ext->subset_ext_ids[i]))
if (riscv_isa_extension_check(ext->subset_ext_ids[j])) set_bit(ext->subset_ext_ids[j], isainfo->isa); } }I think this should go into -fixes, let's check with Palmer if he wants to take this patch only or if you should send the patch on its own.
I think splitting out this and patch 1 into a new series targeting fixes would probably make things clearer?
Okay I can do that. I will give it a bit more time before I send this series split into two to allow time for the rest of the patches to gather comments so I avoid sending too many duplicate patches.
Ye, I do hope to get back to this series later in the week when I have time to actually read through all of the patches in detail.
However, you wouldn't have to resend both parts of the series - you can just split out the fixes portion and send that, leaving the rest of the series sitting on the list to gather comments.
On Wed, Apr 24, 2024 at 04:21:05PM +0100, Conor Dooley wrote:
On Wed, Apr 24, 2024 at 11:13:40AM -0400, Charlie Jenkins wrote:
On Wed, Apr 24, 2024 at 03:51:54PM +0100, Conor Dooley wrote:
On Wed, Apr 24, 2024 at 04:22:02PM +0200, Alexandre Ghiti wrote:
Hi Charlie,
On 21/04/2024 03:04, Charlie Jenkins wrote:
This loop is supposed to check if ext->subset_ext_ids[j] is valid, rather than if ext->subset_ext_ids[i] is valid, before setting the extension id ext->subset_ext_ids[j] in isainfo->isa.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com Reviewed-by: Conor Dooley conor.dooley@microchip.com Fixes: 0d8295ed975b ("riscv: add ISA extension parsing for scalar crypto")
arch/riscv/kernel/cpufeature.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 48874aac4871..b537731cadef 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -609,7 +609,7 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) if (ext->subset_ext_size) { for (int j = 0; j < ext->subset_ext_size; j++) {
if (riscv_isa_extension_check(ext->subset_ext_ids[i]))
if (riscv_isa_extension_check(ext->subset_ext_ids[j])) set_bit(ext->subset_ext_ids[j], isainfo->isa); } }I think this should go into -fixes, let's check with Palmer if he wants to take this patch only or if you should send the patch on its own.
I think splitting out this and patch 1 into a new series targeting fixes would probably make things clearer?
Okay I can do that. I will give it a bit more time before I send this series split into two to allow time for the rest of the patches to gather comments so I avoid sending too many duplicate patches.
Ye, I do hope to get back to this series later in the week when I have time to actually read through all of the patches in detail.
However, you wouldn't have to resend both parts of the series - you can just split out the fixes portion and send that, leaving the rest of the series sitting on the list to gather comments.
Oh cool, I will send those two patches out in their own series then.
- Charlie
Separate vendor extensions out into one struct per vendor instead of adding vendor extensions onto riscv_isa_ext.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- arch/riscv/Kconfig | 2 + arch/riscv/Kconfig.vendor | 9 +++ arch/riscv/include/asm/cpufeature.h | 18 ++++++ arch/riscv/include/asm/vendor_extensions.h | 26 ++++++++ arch/riscv/include/asm/vendor_extensions/thead.h | 19 ++++++ arch/riscv/kernel/Makefile | 2 + arch/riscv/kernel/cpufeature.c | 75 +++++++++++++++++------- arch/riscv/kernel/vendor_extensions.c | 18 ++++++ arch/riscv/kernel/vendor_extensions/Makefile | 3 + arch/riscv/kernel/vendor_extensions/thead.c | 36 ++++++++++++ 10 files changed, 188 insertions(+), 20 deletions(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index be09c8836d56..fec86fba3acd 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -759,6 +759,8 @@ config RISCV_EFFICIENT_UNALIGNED_ACCESS
endchoice
+source "arch/riscv/Kconfig.vendor" + endmenu # "Platform type"
menu "Kernel features" diff --git a/arch/riscv/Kconfig.vendor b/arch/riscv/Kconfig.vendor new file mode 100644 index 000000000000..d57254f28ea6 --- /dev/null +++ b/arch/riscv/Kconfig.vendor @@ -0,0 +1,9 @@ +config RISCV_ISA_VENDOR_EXT_THEAD + bool "T-Head vendor extension support" + default y + help + Say N here if you want to disable all T-Head vendor extension + support. This will cause any T-Head vendor extensions that are + requested by hardware probing to be ignored. + + If you don't know what to do here, say Y. diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index 809f61ffb667..db6a6d7d6b2e 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -35,6 +35,24 @@ extern u32 riscv_vlenb_dt[NR_CPUS];
void riscv_user_isa_enable(void);
+#define _RISCV_ISA_EXT_DATA(_name, _id, _subset_exts, _subset_exts_size) { \ + .name = #_name, \ + .property = #_name, \ + .id = _id, \ + .subset_ext_ids = _subset_exts, \ + .subset_ext_size = _subset_exts_size \ +} + +#define __RISCV_ISA_EXT_DATA(_name, _id) _RISCV_ISA_EXT_DATA(_name, _id, NULL, 0) + +/* Used to declare pure "lasso" extension (Zk for instance) */ +#define __RISCV_ISA_EXT_BUNDLE(_name, _bundled_exts) \ + _RISCV_ISA_EXT_DATA(_name, RISCV_ISA_EXT_INVALID, _bundled_exts, ARRAY_SIZE(_bundled_exts)) + +/* Used to declare extensions that are a superset of other extensions (Zvbb for instance) */ +#define __RISCV_ISA_EXT_SUPERSET(_name, _id, _sub_exts) \ + _RISCV_ISA_EXT_DATA(_name, _id, _sub_exts, ARRAY_SIZE(_sub_exts)) + #if defined(CONFIG_RISCV_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); void unaligned_emulation_finish(void); diff --git a/arch/riscv/include/asm/vendor_extensions.h b/arch/riscv/include/asm/vendor_extensions.h new file mode 100644 index 000000000000..0af1ddd0af70 --- /dev/null +++ b/arch/riscv/include/asm/vendor_extensions.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright 2024 Rivos, Inc + */ + +#ifndef _ASM_VENDOR_EXTENSIONS_H +#define _ASM_VENDOR_EXTENSIONS_H + +#include <asm/cpufeature.h> + +#include <linux/array_size.h> +#include <linux/types.h> + +struct riscv_isa_vendor_ext_data_list { + const struct riscv_isa_ext_data *ext_data; + struct riscv_isainfo *per_hart_vendor_bitmap; + unsigned long *vendor_bitmap; + const size_t ext_data_count; + const size_t bitmap_size; +}; + +extern const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[]; + +extern const size_t riscv_isa_vendor_ext_list_size; + +#endif /* _ASM_VENDOR_EXTENSIONS_H */ diff --git a/arch/riscv/include/asm/vendor_extensions/thead.h b/arch/riscv/include/asm/vendor_extensions/thead.h new file mode 100644 index 000000000000..92eec729888d --- /dev/null +++ b/arch/riscv/include/asm/vendor_extensions/thead.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_RISCV_VENDOR_EXTENSIONS_THEAD_H +#define _ASM_RISCV_VENDOR_EXTENSIONS_THEAD_H + +#include <asm/vendor_extensions.h> + +#include <linux/types.h> + +#define RISCV_ISA_VENDOR_EXT_XTHEADVECTOR 0 + +/* + * Extension keys should be strictly less than max. + * It is safe to increment this when necessary. + */ +#define RISCV_ISA_VENDOR_EXT_MAX_THEAD 32 + +extern const struct riscv_isa_vendor_ext_data_list riscv_isa_vendor_ext_list_thead; + +#endif diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 81d94a8ee10f..53361c50fb46 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -58,6 +58,8 @@ obj-y += riscv_ksyms.o obj-y += stacktrace.o obj-y += cacheinfo.o obj-y += patch.o +obj-y += vendor_extensions.o +obj-y += vendor_extensions/ obj-y += probes/ obj-y += tests/ obj-$(CONFIG_MMU) += vdso.o vdso/ diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index b537731cadef..c9f36822e337 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -24,6 +24,7 @@ #include <asm/processor.h> #include <asm/sbi.h> #include <asm/vector.h> +#include <asm/vendor_extensions.h>
#define NUM_ALPHA_EXTS ('z' - 'a' + 1)
@@ -102,24 +103,6 @@ static bool riscv_isa_extension_check(int id) return true; }
-#define _RISCV_ISA_EXT_DATA(_name, _id, _subset_exts, _subset_exts_size) { \ - .name = #_name, \ - .property = #_name, \ - .id = _id, \ - .subset_ext_ids = _subset_exts, \ - .subset_ext_size = _subset_exts_size \ -} - -#define __RISCV_ISA_EXT_DATA(_name, _id) _RISCV_ISA_EXT_DATA(_name, _id, NULL, 0) - -/* Used to declare pure "lasso" extension (Zk for instance) */ -#define __RISCV_ISA_EXT_BUNDLE(_name, _bundled_exts) \ - _RISCV_ISA_EXT_DATA(_name, RISCV_ISA_EXT_INVALID, _bundled_exts, ARRAY_SIZE(_bundled_exts)) - -/* Used to declare extensions that are a superset of other extensions (Zvbb for instance) */ -#define __RISCV_ISA_EXT_SUPERSET(_name, _id, _sub_exts) \ - _RISCV_ISA_EXT_DATA(_name, _id, _sub_exts, ARRAY_SIZE(_sub_exts)) - static const unsigned int riscv_zk_bundled_exts[] = { RISCV_ISA_EXT_ZBKB, RISCV_ISA_EXT_ZBKC, @@ -353,6 +336,14 @@ static void __init riscv_parse_isa_string(unsigned long *this_hwcap, struct risc bool ext_long = false, ext_err = false;
switch (*ext) { + case 'x': + case 'X': + pr_warn("Vendor extensions are ignored in riscv,isa. Use riscv,isa-extensions instead."); + /* + * In canonical order, the remaining extensions in the + * isa string will be vendor extensions so exit. + */ + break; case 's': /* * Workaround for invalid single-letter 's' & 'u' (QEMU). @@ -368,8 +359,6 @@ static void __init riscv_parse_isa_string(unsigned long *this_hwcap, struct risc } fallthrough; case 'S': - case 'x': - case 'X': case 'z': case 'Z': /* @@ -580,6 +569,48 @@ static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwcap) acpi_put_table((struct acpi_table_header *)rhct); }
+static void __init riscv_add_cpu_vendor_ext(struct device_node *cpu_node, int cpu) +{ + for (int i = 0; i < riscv_isa_vendor_ext_list_size; i++) { + const struct riscv_isa_vendor_ext_data_list *ext_list = riscv_isa_vendor_ext_list[i]; + + for (int j = 0; j < ext_list->ext_data_count; j++) { + const struct riscv_isa_ext_data ext = ext_list->ext_data[j]; + struct riscv_isainfo *isavendorinfo = &ext_list->per_hart_vendor_bitmap[cpu]; + + if (of_property_match_string(cpu_node, "riscv,isa-extensions", + ext.property) < 0) + continue; + + /* + * Assume that subset extensions are all members of the + * same vendor. + */ + if (ext.subset_ext_size) + for (int k = 0; k < ext.subset_ext_size; k++) + set_bit(ext.subset_ext_ids[k], isavendorinfo->isa); + + set_bit(ext.id, isavendorinfo->isa); + } + } +} + +static void __init set_riscv_isa_vendor(int cpu) +{ + for (int i = 0; i < riscv_isa_vendor_ext_list_size; i++) { + const struct riscv_isa_vendor_ext_data_list *ext_list = riscv_isa_vendor_ext_list[i]; + + if (bitmap_empty(ext_list->vendor_bitmap, ext_list->bitmap_size)) + bitmap_copy(ext_list->vendor_bitmap, + ext_list->per_hart_vendor_bitmap[cpu].isa, + ext_list->bitmap_size); + else + bitmap_and(ext_list->vendor_bitmap, ext_list->vendor_bitmap, + ext_list->per_hart_vendor_bitmap[cpu].isa, + ext_list->bitmap_size); + } +} + static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) { unsigned int cpu; @@ -623,6 +654,8 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) } }
+ riscv_add_cpu_vendor_ext(cpu_node, cpu); + of_node_put(cpu_node);
/* @@ -638,6 +671,8 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) bitmap_copy(riscv_isa, isainfo->isa, RISCV_ISA_EXT_MAX); else bitmap_and(riscv_isa, riscv_isa, isainfo->isa, RISCV_ISA_EXT_MAX); + + set_riscv_isa_vendor(cpu); }
if (bitmap_empty(riscv_isa, RISCV_ISA_EXT_MAX)) diff --git a/arch/riscv/kernel/vendor_extensions.c b/arch/riscv/kernel/vendor_extensions.c new file mode 100644 index 000000000000..f76cb3013c2d --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions.c @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright 2024 Rivos, Inc + */ + +#include <asm/vendor_extensions.h> +#include <asm/vendor_extensions/thead.h> + +#include <linux/array_size.h> +#include <linux/types.h> + +const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[] = { +#ifdef CONFIG_RISCV_ISA_VENDOR_EXT_THEAD + &riscv_isa_vendor_ext_list_thead, +#endif +}; + +const size_t riscv_isa_vendor_ext_list_size = ARRAY_SIZE(riscv_isa_vendor_ext_list); diff --git a/arch/riscv/kernel/vendor_extensions/Makefile b/arch/riscv/kernel/vendor_extensions/Makefile new file mode 100644 index 000000000000..3383066baaab --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0-only + +obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_THEAD) += thead.o diff --git a/arch/riscv/kernel/vendor_extensions/thead.c b/arch/riscv/kernel/vendor_extensions/thead.c new file mode 100644 index 000000000000..a0a47414ed22 --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions/thead.c @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include <asm/cpufeature.h> +#include <asm/vendor_extensions.h> +#include <asm/vendor_extensions/thead.h> + +#include <linux/array_size.h> +#include <linux/types.h> + +/* All T-Head vendor extensions supported in Linux */ +const struct riscv_isa_ext_data riscv_isa_vendor_ext_thead[] = { + _RISCV_ISA_EXT_DATA(xtheadvector, RISCV_ISA_VENDOR_EXT_XTHEADVECTOR, NULL, 0), +}; + +/* + * The first member of this struct must be a bitmap named isa so it can be + * compatible with riscv_isainfo even though the sizes of the bitmaps may be + * different. + */ +struct riscv_isavendorinfo_thead { + DECLARE_BITMAP(isa, RISCV_ISA_VENDOR_EXT_MAX_THEAD); +}; + +/* Hart specific T-Head vendor extension support */ +static struct riscv_isavendorinfo_thead hart_vendorinfo_thead[NR_CPUS]; + +/* Set of T-Head vendor extensions supported on all harts */ +DECLARE_BITMAP(vendorinfo_thead, RISCV_ISA_VENDOR_EXT_MAX_THEAD); + +const struct riscv_isa_vendor_ext_data_list riscv_isa_vendor_ext_list_thead = { + .ext_data = riscv_isa_vendor_ext_thead, + .per_hart_vendor_bitmap = (struct riscv_isainfo *)hart_vendorinfo_thead, + .vendor_bitmap = vendorinfo_thead, + .ext_data_count = ARRAY_SIZE(riscv_isa_vendor_ext_thead), + .bitmap_size = RISCV_ISA_VENDOR_EXT_MAX_THEAD +};
On Sat, Apr 20, 2024 at 06:04:39PM -0700, Charlie Jenkins wrote:
Separate vendor extensions out into one struct per vendor instead of adding vendor extensions onto riscv_isa_ext.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com
arch/riscv/Kconfig | 2 + arch/riscv/Kconfig.vendor | 9 +++ arch/riscv/include/asm/cpufeature.h | 18 ++++++ arch/riscv/include/asm/vendor_extensions.h | 26 ++++++++ arch/riscv/include/asm/vendor_extensions/thead.h | 19 ++++++ arch/riscv/kernel/Makefile | 2 + arch/riscv/kernel/cpufeature.c | 75 +++++++++++++++++------- arch/riscv/kernel/vendor_extensions.c | 18 ++++++ arch/riscv/kernel/vendor_extensions/Makefile | 3 + arch/riscv/kernel/vendor_extensions/thead.c | 36 ++++++++++++ 10 files changed, 188 insertions(+), 20 deletions(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index be09c8836d56..fec86fba3acd 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -759,6 +759,8 @@ config RISCV_EFFICIENT_UNALIGNED_ACCESS endchoice +source "arch/riscv/Kconfig.vendor"
endmenu # "Platform type" menu "Kernel features" diff --git a/arch/riscv/Kconfig.vendor b/arch/riscv/Kconfig.vendor new file mode 100644 index 000000000000..d57254f28ea6 --- /dev/null +++ b/arch/riscv/Kconfig.vendor @@ -0,0 +1,9 @@ +config RISCV_ISA_VENDOR_EXT_THEAD
- bool "T-Head vendor extension support"
- default y
- help
Say N here if you want to disable all T-Head vendor extensionsupport. This will cause any T-Head vendor extensions that arerequested by hardware probing to be ignored.If you don't know what to do here, say Y.diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index 809f61ffb667..db6a6d7d6b2e 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -35,6 +35,24 @@ extern u32 riscv_vlenb_dt[NR_CPUS]; void riscv_user_isa_enable(void); +#define _RISCV_ISA_EXT_DATA(_name, _id, _subset_exts, _subset_exts_size) { \
- .name = #_name, \
- .property = #_name, \
- .id = _id, \
- .subset_ext_ids = _subset_exts, \
- .subset_ext_size = _subset_exts_size \
+}
+#define __RISCV_ISA_EXT_DATA(_name, _id) _RISCV_ISA_EXT_DATA(_name, _id, NULL, 0)
+/* Used to declare pure "lasso" extension (Zk for instance) */ +#define __RISCV_ISA_EXT_BUNDLE(_name, _bundled_exts) \
- _RISCV_ISA_EXT_DATA(_name, RISCV_ISA_EXT_INVALID, _bundled_exts, ARRAY_SIZE(_bundled_exts))
+/* Used to declare extensions that are a superset of other extensions (Zvbb for instance) */ +#define __RISCV_ISA_EXT_SUPERSET(_name, _id, _sub_exts) \
- _RISCV_ISA_EXT_DATA(_name, _id, _sub_exts, ARRAY_SIZE(_sub_exts))
#if defined(CONFIG_RISCV_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); void unaligned_emulation_finish(void); diff --git a/arch/riscv/include/asm/vendor_extensions.h b/arch/riscv/include/asm/vendor_extensions.h new file mode 100644 index 000000000000..0af1ddd0af70 --- /dev/null +++ b/arch/riscv/include/asm/vendor_extensions.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/*
- Copyright 2024 Rivos, Inc
- */
+#ifndef _ASM_VENDOR_EXTENSIONS_H +#define _ASM_VENDOR_EXTENSIONS_H
+#include <asm/cpufeature.h>
+#include <linux/array_size.h> +#include <linux/types.h>
+struct riscv_isa_vendor_ext_data_list {
- const struct riscv_isa_ext_data *ext_data;
- struct riscv_isainfo *per_hart_vendor_bitmap;
- unsigned long *vendor_bitmap;
- const size_t ext_data_count;
- const size_t bitmap_size;
+};
+extern const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[];
+extern const size_t riscv_isa_vendor_ext_list_size;
+#endif /* _ASM_VENDOR_EXTENSIONS_H */ diff --git a/arch/riscv/include/asm/vendor_extensions/thead.h b/arch/riscv/include/asm/vendor_extensions/thead.h new file mode 100644 index 000000000000..92eec729888d --- /dev/null +++ b/arch/riscv/include/asm/vendor_extensions/thead.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_RISCV_VENDOR_EXTENSIONS_THEAD_H +#define _ASM_RISCV_VENDOR_EXTENSIONS_THEAD_H
+#include <asm/vendor_extensions.h>
+#include <linux/types.h>
+#define RISCV_ISA_VENDOR_EXT_XTHEADVECTOR 0
+/*
- Extension keys should be strictly less than max.
- It is safe to increment this when necessary.
- */
+#define RISCV_ISA_VENDOR_EXT_MAX_THEAD 32
+extern const struct riscv_isa_vendor_ext_data_list riscv_isa_vendor_ext_list_thead;
+#endif diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 81d94a8ee10f..53361c50fb46 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -58,6 +58,8 @@ obj-y += riscv_ksyms.o obj-y += stacktrace.o obj-y += cacheinfo.o obj-y += patch.o +obj-y += vendor_extensions.o +obj-y += vendor_extensions/ obj-y += probes/ obj-y += tests/ obj-$(CONFIG_MMU) += vdso.o vdso/ diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index b537731cadef..c9f36822e337 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -24,6 +24,7 @@ #include <asm/processor.h> #include <asm/sbi.h> #include <asm/vector.h> +#include <asm/vendor_extensions.h> #define NUM_ALPHA_EXTS ('z' - 'a' + 1) @@ -102,24 +103,6 @@ static bool riscv_isa_extension_check(int id) return true; } -#define _RISCV_ISA_EXT_DATA(_name, _id, _subset_exts, _subset_exts_size) { \
- .name = #_name, \
- .property = #_name, \
- .id = _id, \
- .subset_ext_ids = _subset_exts, \
- .subset_ext_size = _subset_exts_size \
-}
-#define __RISCV_ISA_EXT_DATA(_name, _id) _RISCV_ISA_EXT_DATA(_name, _id, NULL, 0)
-/* Used to declare pure "lasso" extension (Zk for instance) */ -#define __RISCV_ISA_EXT_BUNDLE(_name, _bundled_exts) \
- _RISCV_ISA_EXT_DATA(_name, RISCV_ISA_EXT_INVALID, _bundled_exts, ARRAY_SIZE(_bundled_exts))
-/* Used to declare extensions that are a superset of other extensions (Zvbb for instance) */ -#define __RISCV_ISA_EXT_SUPERSET(_name, _id, _sub_exts) \
- _RISCV_ISA_EXT_DATA(_name, _id, _sub_exts, ARRAY_SIZE(_sub_exts))
static const unsigned int riscv_zk_bundled_exts[] = { RISCV_ISA_EXT_ZBKB, RISCV_ISA_EXT_ZBKC, @@ -353,6 +336,14 @@ static void __init riscv_parse_isa_string(unsigned long *this_hwcap, struct risc bool ext_long = false, ext_err = false; switch (*ext) {
case 'x':case 'X':pr_warn("Vendor extensions are ignored in riscv,isa. Use riscv,isa-extensions instead.");
I think I pointed out before that this should only really warn once, not once per extension per cpu. pr_warn_once() please.
/** In canonical order, the remaining extensions in the* isa string will be vendor extensions so exit.*/
Yeah, but the binding doesn't require canonical order, so please don't exit & just move along to the next extension.
break;@@ -368,8 +359,6 @@ static void __init riscv_parse_isa_string(unsigned long *this_hwcap, struct risc } fallthrough; case 'S':
case 'x':case 'X':@@ -580,6 +569,48 @@ static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwcap) acpi_put_table((struct acpi_table_header *)rhct); } +static void __init riscv_add_cpu_vendor_ext(struct device_node *cpu_node, int cpu)
Can we have a consistent naming here? The other functions are using "fill".
+{
- for (int i = 0; i < riscv_isa_vendor_ext_list_size; i++) {
const struct riscv_isa_vendor_ext_data_list *ext_list = riscv_isa_vendor_ext_list[i];for (int j = 0; j < ext_list->ext_data_count; j++) {const struct riscv_isa_ext_data ext = ext_list->ext_data[j];struct riscv_isainfo *isavendorinfo = &ext_list->per_hart_vendor_bitmap[cpu];if (of_property_match_string(cpu_node, "riscv,isa-extensions",ext.property) < 0)continue;/** Assume that subset extensions are all members of the* same vendor.*/if (ext.subset_ext_size)for (int k = 0; k < ext.subset_ext_size; k++)set_bit(ext.subset_ext_ids[k], isavendorinfo->isa);set_bit(ext.id, isavendorinfo->isa);}- }
+}
+static void __init set_riscv_isa_vendor(int cpu)
Again, consistent naming here would have riscv_isa_set... And it's not setting a vendor, can you pick a name that matches the action that the function is actually taking please? riscv_isa_set_vendor_bitmask() or w/e.
Also, you've added Kconfig options for the different vendors but this code is unconditionally compiled. Do we need to have a hidden Kconfig option that all of the vendor options select that gets all of this new code culled when none of them are enabled? Could just be as simple as if (!IS_ENABLED(CONFIG_FOO)) return; in these functions in cpufeature.c?
+{
- for (int i = 0; i < riscv_isa_vendor_ext_list_size; i++) {
const struct riscv_isa_vendor_ext_data_list *ext_list = riscv_isa_vendor_ext_list[i];if (bitmap_empty(ext_list->vendor_bitmap, ext_list->bitmap_size))bitmap_copy(ext_list->vendor_bitmap,ext_list->per_hart_vendor_bitmap[cpu].isa,ext_list->bitmap_size);elsebitmap_and(ext_list->vendor_bitmap, ext_list->vendor_bitmap,ext_list->per_hart_vendor_bitmap[cpu].isa,ext_list->bitmap_size);- }
+}
static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) { unsigned int cpu; @@ -623,6 +654,8 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) } }
riscv_add_cpu_vendor_ext(cpu_node, cpu);- of_node_put(cpu_node);
/* @@ -638,6 +671,8 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) bitmap_copy(riscv_isa, isainfo->isa, RISCV_ISA_EXT_MAX); else bitmap_and(riscv_isa, riscv_isa, isainfo->isa, RISCV_ISA_EXT_MAX);
set_riscv_isa_vendor(cpu);if (bitmap_empty(riscv_isa, RISCV_ISA_EXT_MAX)) diff --git a/arch/riscv/kernel/vendor_extensions.c b/arch/riscv/kernel/vendor_extensions.c new file mode 100644 index 000000000000..f76cb3013c2d --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions.c @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: GPL-2.0-only +/*
- Copyright 2024 Rivos, Inc
- */
+#include <asm/vendor_extensions.h> +#include <asm/vendor_extensions/thead.h>
+#include <linux/array_size.h> +#include <linux/types.h>
+const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[] = { +#ifdef CONFIG_RISCV_ISA_VENDOR_EXT_THEAD
- &riscv_isa_vendor_ext_list_thead,
+#endif +};
+const size_t riscv_isa_vendor_ext_list_size = ARRAY_SIZE(riscv_isa_vendor_ext_list); diff --git a/arch/riscv/kernel/vendor_extensions/Makefile b/arch/riscv/kernel/vendor_extensions/Makefile new file mode 100644 index 000000000000..3383066baaab --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_THEAD) += thead.o diff --git a/arch/riscv/kernel/vendor_extensions/thead.c b/arch/riscv/kernel/vendor_extensions/thead.c new file mode 100644 index 000000000000..a0a47414ed22 --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions/thead.c @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0-only
+#include <asm/cpufeature.h> +#include <asm/vendor_extensions.h> +#include <asm/vendor_extensions/thead.h>
+#include <linux/array_size.h> +#include <linux/types.h>
+/* All T-Head vendor extensions supported in Linux */ +const struct riscv_isa_ext_data riscv_isa_vendor_ext_thead[] = {
- _RISCV_ISA_EXT_DATA(xtheadvector, RISCV_ISA_VENDOR_EXT_XTHEADVECTOR, NULL, 0),
+};
+/*
- The first member of this struct must be a bitmap named isa so it can be
- compatible with riscv_isainfo even though the sizes of the bitmaps may be
- different.
- */
+struct riscv_isavendorinfo_thead {
- DECLARE_BITMAP(isa, RISCV_ISA_VENDOR_EXT_MAX_THEAD);
+};
+/* Hart specific T-Head vendor extension support */ +static struct riscv_isavendorinfo_thead hart_vendorinfo_thead[NR_CPUS];
+/* Set of T-Head vendor extensions supported on all harts */ +DECLARE_BITMAP(vendorinfo_thead, RISCV_ISA_VENDOR_EXT_MAX_THEAD);
+const struct riscv_isa_vendor_ext_data_list riscv_isa_vendor_ext_list_thead = {
- .ext_data = riscv_isa_vendor_ext_thead,
- .per_hart_vendor_bitmap = (struct riscv_isainfo *)hart_vendorinfo_thead,
- .vendor_bitmap = vendorinfo_thead,
- .ext_data_count = ARRAY_SIZE(riscv_isa_vendor_ext_thead),
- .bitmap_size = RISCV_ISA_VENDOR_EXT_MAX_THEAD
+};
-- 2.44.0
On Fri, Apr 26, 2024 at 05:00:36PM +0100, Conor Dooley wrote:
On Sat, Apr 20, 2024 at 06:04:39PM -0700, Charlie Jenkins wrote:
Separate vendor extensions out into one struct per vendor instead of adding vendor extensions onto riscv_isa_ext.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com
arch/riscv/Kconfig | 2 + arch/riscv/Kconfig.vendor | 9 +++ arch/riscv/include/asm/cpufeature.h | 18 ++++++ arch/riscv/include/asm/vendor_extensions.h | 26 ++++++++ arch/riscv/include/asm/vendor_extensions/thead.h | 19 ++++++ arch/riscv/kernel/Makefile | 2 + arch/riscv/kernel/cpufeature.c | 75 +++++++++++++++++------- arch/riscv/kernel/vendor_extensions.c | 18 ++++++ arch/riscv/kernel/vendor_extensions/Makefile | 3 + arch/riscv/kernel/vendor_extensions/thead.c | 36 ++++++++++++ 10 files changed, 188 insertions(+), 20 deletions(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index be09c8836d56..fec86fba3acd 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -759,6 +759,8 @@ config RISCV_EFFICIENT_UNALIGNED_ACCESS endchoice +source "arch/riscv/Kconfig.vendor"
endmenu # "Platform type" menu "Kernel features" diff --git a/arch/riscv/Kconfig.vendor b/arch/riscv/Kconfig.vendor new file mode 100644 index 000000000000..d57254f28ea6 --- /dev/null +++ b/arch/riscv/Kconfig.vendor @@ -0,0 +1,9 @@ +config RISCV_ISA_VENDOR_EXT_THEAD
- bool "T-Head vendor extension support"
- default y
- help
Say N here if you want to disable all T-Head vendor extensionsupport. This will cause any T-Head vendor extensions that arerequested by hardware probing to be ignored.If you don't know what to do here, say Y.diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index 809f61ffb667..db6a6d7d6b2e 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -35,6 +35,24 @@ extern u32 riscv_vlenb_dt[NR_CPUS]; void riscv_user_isa_enable(void); +#define _RISCV_ISA_EXT_DATA(_name, _id, _subset_exts, _subset_exts_size) { \
- .name = #_name, \
- .property = #_name, \
- .id = _id, \
- .subset_ext_ids = _subset_exts, \
- .subset_ext_size = _subset_exts_size \
+}
+#define __RISCV_ISA_EXT_DATA(_name, _id) _RISCV_ISA_EXT_DATA(_name, _id, NULL, 0)
+/* Used to declare pure "lasso" extension (Zk for instance) */ +#define __RISCV_ISA_EXT_BUNDLE(_name, _bundled_exts) \
- _RISCV_ISA_EXT_DATA(_name, RISCV_ISA_EXT_INVALID, _bundled_exts, ARRAY_SIZE(_bundled_exts))
+/* Used to declare extensions that are a superset of other extensions (Zvbb for instance) */ +#define __RISCV_ISA_EXT_SUPERSET(_name, _id, _sub_exts) \
- _RISCV_ISA_EXT_DATA(_name, _id, _sub_exts, ARRAY_SIZE(_sub_exts))
#if defined(CONFIG_RISCV_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); void unaligned_emulation_finish(void); diff --git a/arch/riscv/include/asm/vendor_extensions.h b/arch/riscv/include/asm/vendor_extensions.h new file mode 100644 index 000000000000..0af1ddd0af70 --- /dev/null +++ b/arch/riscv/include/asm/vendor_extensions.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/*
- Copyright 2024 Rivos, Inc
- */
+#ifndef _ASM_VENDOR_EXTENSIONS_H +#define _ASM_VENDOR_EXTENSIONS_H
+#include <asm/cpufeature.h>
+#include <linux/array_size.h> +#include <linux/types.h>
+struct riscv_isa_vendor_ext_data_list {
- const struct riscv_isa_ext_data *ext_data;
- struct riscv_isainfo *per_hart_vendor_bitmap;
- unsigned long *vendor_bitmap;
- const size_t ext_data_count;
- const size_t bitmap_size;
+};
+extern const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[];
+extern const size_t riscv_isa_vendor_ext_list_size;
+#endif /* _ASM_VENDOR_EXTENSIONS_H */ diff --git a/arch/riscv/include/asm/vendor_extensions/thead.h b/arch/riscv/include/asm/vendor_extensions/thead.h new file mode 100644 index 000000000000..92eec729888d --- /dev/null +++ b/arch/riscv/include/asm/vendor_extensions/thead.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_RISCV_VENDOR_EXTENSIONS_THEAD_H +#define _ASM_RISCV_VENDOR_EXTENSIONS_THEAD_H
+#include <asm/vendor_extensions.h>
+#include <linux/types.h>
+#define RISCV_ISA_VENDOR_EXT_XTHEADVECTOR 0
+/*
- Extension keys should be strictly less than max.
- It is safe to increment this when necessary.
- */
+#define RISCV_ISA_VENDOR_EXT_MAX_THEAD 32
+extern const struct riscv_isa_vendor_ext_data_list riscv_isa_vendor_ext_list_thead;
+#endif diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 81d94a8ee10f..53361c50fb46 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -58,6 +58,8 @@ obj-y += riscv_ksyms.o obj-y += stacktrace.o obj-y += cacheinfo.o obj-y += patch.o +obj-y += vendor_extensions.o +obj-y += vendor_extensions/ obj-y += probes/ obj-y += tests/ obj-$(CONFIG_MMU) += vdso.o vdso/ diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index b537731cadef..c9f36822e337 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -24,6 +24,7 @@ #include <asm/processor.h> #include <asm/sbi.h> #include <asm/vector.h> +#include <asm/vendor_extensions.h> #define NUM_ALPHA_EXTS ('z' - 'a' + 1) @@ -102,24 +103,6 @@ static bool riscv_isa_extension_check(int id) return true; } -#define _RISCV_ISA_EXT_DATA(_name, _id, _subset_exts, _subset_exts_size) { \
- .name = #_name, \
- .property = #_name, \
- .id = _id, \
- .subset_ext_ids = _subset_exts, \
- .subset_ext_size = _subset_exts_size \
-}
-#define __RISCV_ISA_EXT_DATA(_name, _id) _RISCV_ISA_EXT_DATA(_name, _id, NULL, 0)
-/* Used to declare pure "lasso" extension (Zk for instance) */ -#define __RISCV_ISA_EXT_BUNDLE(_name, _bundled_exts) \
- _RISCV_ISA_EXT_DATA(_name, RISCV_ISA_EXT_INVALID, _bundled_exts, ARRAY_SIZE(_bundled_exts))
-/* Used to declare extensions that are a superset of other extensions (Zvbb for instance) */ -#define __RISCV_ISA_EXT_SUPERSET(_name, _id, _sub_exts) \
- _RISCV_ISA_EXT_DATA(_name, _id, _sub_exts, ARRAY_SIZE(_sub_exts))
static const unsigned int riscv_zk_bundled_exts[] = { RISCV_ISA_EXT_ZBKB, RISCV_ISA_EXT_ZBKC, @@ -353,6 +336,14 @@ static void __init riscv_parse_isa_string(unsigned long *this_hwcap, struct risc bool ext_long = false, ext_err = false; switch (*ext) {
case 'x':case 'X':pr_warn("Vendor extensions are ignored in riscv,isa. Use riscv,isa-extensions instead.");I think I pointed out before that this should only really warn once, not once per extension per cpu. pr_warn_once() please.
Yes that slipped my mind, thank you for reminding me.
/** In canonical order, the remaining extensions in the* isa string will be vendor extensions so exit.*/Yeah, but the binding doesn't require canonical order, so please don't exit & just move along to the next extension.
Oh! I will remove that then.
break;@@ -368,8 +359,6 @@ static void __init riscv_parse_isa_string(unsigned long *this_hwcap, struct risc } fallthrough; case 'S':
case 'x':case 'X':@@ -580,6 +569,48 @@ static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwcap) acpi_put_table((struct acpi_table_header *)rhct); } +static void __init riscv_add_cpu_vendor_ext(struct device_node *cpu_node, int cpu)
Can we have a consistent naming here? The other functions are using "fill".
I will rename it "riscv_fill_cpu_vendor_ext"
+{
- for (int i = 0; i < riscv_isa_vendor_ext_list_size; i++) {
const struct riscv_isa_vendor_ext_data_list *ext_list = riscv_isa_vendor_ext_list[i];for (int j = 0; j < ext_list->ext_data_count; j++) {const struct riscv_isa_ext_data ext = ext_list->ext_data[j];struct riscv_isainfo *isavendorinfo = &ext_list->per_hart_vendor_bitmap[cpu];if (of_property_match_string(cpu_node, "riscv,isa-extensions",ext.property) < 0)continue;/** Assume that subset extensions are all members of the* same vendor.*/if (ext.subset_ext_size)for (int k = 0; k < ext.subset_ext_size; k++)set_bit(ext.subset_ext_ids[k], isavendorinfo->isa);set_bit(ext.id, isavendorinfo->isa);}- }
+}
+static void __init set_riscv_isa_vendor(int cpu)
Again, consistent naming here would have riscv_isa_set... And it's not setting a vendor, can you pick a name that matches the action that the function is actually taking please? riscv_isa_set_vendor_bitmask() or w/e.
How about "riscv_fill_vendor_ext_list"
Also, you've added Kconfig options for the different vendors but this code is unconditionally compiled. Do we need to have a hidden Kconfig option that all of the vendor options select that gets all of this new code culled when none of them are enabled? Could just be as simple as if (!IS_ENABLED(CONFIG_FOO)) return; in these functions in cpufeature.c?
That is a good idea, I can add that.
- Charlie
+{
- for (int i = 0; i < riscv_isa_vendor_ext_list_size; i++) {
const struct riscv_isa_vendor_ext_data_list *ext_list = riscv_isa_vendor_ext_list[i];if (bitmap_empty(ext_list->vendor_bitmap, ext_list->bitmap_size))bitmap_copy(ext_list->vendor_bitmap,ext_list->per_hart_vendor_bitmap[cpu].isa,ext_list->bitmap_size);elsebitmap_and(ext_list->vendor_bitmap, ext_list->vendor_bitmap,ext_list->per_hart_vendor_bitmap[cpu].isa,ext_list->bitmap_size);- }
+}
static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) { unsigned int cpu; @@ -623,6 +654,8 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) } }
riscv_add_cpu_vendor_ext(cpu_node, cpu);- of_node_put(cpu_node);
/* @@ -638,6 +671,8 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) bitmap_copy(riscv_isa, isainfo->isa, RISCV_ISA_EXT_MAX); else bitmap_and(riscv_isa, riscv_isa, isainfo->isa, RISCV_ISA_EXT_MAX);
set_riscv_isa_vendor(cpu);if (bitmap_empty(riscv_isa, RISCV_ISA_EXT_MAX)) diff --git a/arch/riscv/kernel/vendor_extensions.c b/arch/riscv/kernel/vendor_extensions.c new file mode 100644 index 000000000000..f76cb3013c2d --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions.c @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: GPL-2.0-only +/*
- Copyright 2024 Rivos, Inc
- */
+#include <asm/vendor_extensions.h> +#include <asm/vendor_extensions/thead.h>
+#include <linux/array_size.h> +#include <linux/types.h>
+const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[] = { +#ifdef CONFIG_RISCV_ISA_VENDOR_EXT_THEAD
- &riscv_isa_vendor_ext_list_thead,
+#endif +};
+const size_t riscv_isa_vendor_ext_list_size = ARRAY_SIZE(riscv_isa_vendor_ext_list); diff --git a/arch/riscv/kernel/vendor_extensions/Makefile b/arch/riscv/kernel/vendor_extensions/Makefile new file mode 100644 index 000000000000..3383066baaab --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_THEAD) += thead.o diff --git a/arch/riscv/kernel/vendor_extensions/thead.c b/arch/riscv/kernel/vendor_extensions/thead.c new file mode 100644 index 000000000000..a0a47414ed22 --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions/thead.c @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0-only
+#include <asm/cpufeature.h> +#include <asm/vendor_extensions.h> +#include <asm/vendor_extensions/thead.h>
+#include <linux/array_size.h> +#include <linux/types.h>
+/* All T-Head vendor extensions supported in Linux */ +const struct riscv_isa_ext_data riscv_isa_vendor_ext_thead[] = {
- _RISCV_ISA_EXT_DATA(xtheadvector, RISCV_ISA_VENDOR_EXT_XTHEADVECTOR, NULL, 0),
+};
+/*
- The first member of this struct must be a bitmap named isa so it can be
- compatible with riscv_isainfo even though the sizes of the bitmaps may be
- different.
- */
+struct riscv_isavendorinfo_thead {
- DECLARE_BITMAP(isa, RISCV_ISA_VENDOR_EXT_MAX_THEAD);
+};
+/* Hart specific T-Head vendor extension support */ +static struct riscv_isavendorinfo_thead hart_vendorinfo_thead[NR_CPUS];
+/* Set of T-Head vendor extensions supported on all harts */ +DECLARE_BITMAP(vendorinfo_thead, RISCV_ISA_VENDOR_EXT_MAX_THEAD);
+const struct riscv_isa_vendor_ext_data_list riscv_isa_vendor_ext_list_thead = {
- .ext_data = riscv_isa_vendor_ext_thead,
- .per_hart_vendor_bitmap = (struct riscv_isainfo *)hart_vendorinfo_thead,
- .vendor_bitmap = vendorinfo_thead,
- .ext_data_count = ARRAY_SIZE(riscv_isa_vendor_ext_thead),
- .bitmap_size = RISCV_ISA_VENDOR_EXT_MAX_THEAD
+};
-- 2.44.0
Vendor extensions are maintained in per-vendor structs (separate from standard extensions which live in riscv_isa). Create vendor variants for the existing extension helpers to interface with the riscv_isa_vendor bitmaps. There is a good amount of overlap between these functions, so the alternative checking code can be factored out.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- arch/riscv/errata/sifive/errata.c | 3 ++ arch/riscv/errata/thead/errata.c | 3 ++ arch/riscv/include/asm/cpufeature.h | 86 +++++++++++++++++------------- arch/riscv/include/asm/vendor_extensions.h | 56 +++++++++++++++++++ arch/riscv/kernel/cpufeature.c | 20 ++++--- arch/riscv/kernel/vendor_extensions.c | 40 ++++++++++++++ 6 files changed, 164 insertions(+), 44 deletions(-)
diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c index 3d9a32d791f7..b29b6e405ff2 100644 --- a/arch/riscv/errata/sifive/errata.c +++ b/arch/riscv/errata/sifive/errata.c @@ -12,6 +12,7 @@ #include <asm/alternative.h> #include <asm/vendorid_list.h> #include <asm/errata_list.h> +#include <asm/vendor_extensions.h>
struct errata_info_t { char name[32]; @@ -99,6 +100,8 @@ void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end, for (alt = begin; alt < end; alt++) { if (alt->vendor_id != SIFIVE_VENDOR_ID) continue; + if (alt->patch_id >= RISCV_VENDOR_EXT_ALTERNATIVES_BASE) + continue; if (alt->patch_id >= ERRATA_SIFIVE_NUMBER) { WARN(1, "This errata id:%d is not in kernel errata list", alt->patch_id); continue; diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c index b1c410bbc1ae..d8e78cc687bc 100644 --- a/arch/riscv/errata/thead/errata.c +++ b/arch/riscv/errata/thead/errata.c @@ -18,6 +18,7 @@ #include <asm/io.h> #include <asm/patch.h> #include <asm/vendorid_list.h> +#include <asm/vendor_extensions.h>
static bool errata_probe_pbmt(unsigned int stage, unsigned long arch_id, unsigned long impid) @@ -163,6 +164,8 @@ void thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end, for (alt = begin; alt < end; alt++) { if (alt->vendor_id != THEAD_VENDOR_ID) continue; + if (alt->patch_id >= RISCV_VENDOR_EXT_ALTERNATIVES_BASE) + continue; if (alt->patch_id >= ERRATA_THEAD_NUMBER) continue;
diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index db6a6d7d6b2e..83e1143db9ad 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -103,22 +103,13 @@ bool __riscv_isa_extension_available(const unsigned long *isa_bitmap, unsigned i __riscv_isa_extension_available(isa_bitmap, RISCV_ISA_EXT_##ext)
static __always_inline bool -riscv_has_extension_likely(const unsigned long ext) +__riscv_has_extension_likely_alternatives(const unsigned long vendor, const unsigned long ext) { - compiletime_assert(ext < RISCV_ISA_EXT_MAX, - "ext must be < RISCV_ISA_EXT_MAX"); - - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) { - asm goto( - ALTERNATIVE("j %l[l_no]", "nop", 0, %[ext], 1) - : - : [ext] "i" (ext) - : - : l_no); - } else { - if (!__riscv_isa_extension_available(NULL, ext)) - goto l_no; - } + asm goto(ALTERNATIVE("j %l[l_no]", "nop", %[vendor], %[ext], 1) + : + : [vendor] "i" (vendor), [ext] "i" (ext) + : + : l_no);
return true; l_no: @@ -126,42 +117,65 @@ riscv_has_extension_likely(const unsigned long ext) }
static __always_inline bool -riscv_has_extension_unlikely(const unsigned long ext) +__riscv_has_extension_unlikely_alternatives(const unsigned long vendor, const unsigned long ext) { - compiletime_assert(ext < RISCV_ISA_EXT_MAX, - "ext must be < RISCV_ISA_EXT_MAX"); - - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) { - asm goto( - ALTERNATIVE("nop", "j %l[l_yes]", 0, %[ext], 1) - : - : [ext] "i" (ext) - : - : l_yes); - } else { - if (__riscv_isa_extension_available(NULL, ext)) - goto l_yes; - } + asm goto(ALTERNATIVE("nop", "j %l[l_yes]", %[vendor], %[ext], 1) + : + : [vendor] "i" (vendor), [ext] "i" (ext) + : + : l_yes);
return false; l_yes: return true; }
+static __always_inline bool +riscv_has_extension_likely(const unsigned long ext) +{ + compiletime_assert(ext < RISCV_ISA_EXT_MAX, + "ext must be < RISCV_ISA_EXT_MAX"); + + if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) + return __riscv_has_extension_likely_alternatives(0, ext); + else + return __riscv_isa_extension_available(NULL, ext); +} + +static __always_inline bool +riscv_has_extension_unlikely(const unsigned long ext) +{ + compiletime_assert(ext < RISCV_ISA_EXT_MAX, + "ext must be < RISCV_ISA_EXT_MAX"); + + if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) + return __riscv_has_extension_unlikely_alternatives(0, ext); + else + return __riscv_isa_extension_available(NULL, ext); +} + static __always_inline bool riscv_cpu_has_extension_likely(int cpu, const unsigned long ext) { - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && riscv_has_extension_likely(ext)) - return true; + compiletime_assert(ext < RISCV_ISA_EXT_MAX, + "ext must be < RISCV_ISA_EXT_MAX");
- return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); + if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && + __riscv_has_extension_likely_alternatives(0, ext)) + return true; + else + return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); }
static __always_inline bool riscv_cpu_has_extension_unlikely(int cpu, const unsigned long ext) { - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && riscv_has_extension_unlikely(ext)) - return true; + compiletime_assert(ext < RISCV_ISA_EXT_MAX, + "ext must be < RISCV_ISA_EXT_MAX");
- return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); + if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && + __riscv_has_extension_unlikely_alternatives(0, ext)) + return true; + else + return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); }
#endif diff --git a/arch/riscv/include/asm/vendor_extensions.h b/arch/riscv/include/asm/vendor_extensions.h index 0af1ddd0af70..3e676a96016e 100644 --- a/arch/riscv/include/asm/vendor_extensions.h +++ b/arch/riscv/include/asm/vendor_extensions.h @@ -23,4 +23,60 @@ extern const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[];
extern const size_t riscv_isa_vendor_ext_list_size;
+/* + * The alternatives need some way of distinguishing between vendor extensions + * and errata. Incrementing all of the vendor extension keys so they are at + * least 0x8000 accomplishes that. + */ +#define RISCV_VENDOR_EXT_ALTERNATIVES_BASE 0x8000 + +bool __riscv_isa_vendor_extension_available(int cpu, unsigned long vendor, unsigned int bit); +#define riscv_cpu_isa_vendor_extension_available(cpu, vendor, ext) \ + __riscv_isa_vendor_extension_available(cpu, vendor, RISCV_ISA_VENDOR_EXT_##ext) +#define riscv_isa_vendor_extension_available(vendor, ext) \ + __riscv_isa_vendor_extension_available(-1, vendor, RISCV_ISA_VENDOR_EXT_##ext) + +static __always_inline bool +riscv_has_vendor_extension_likely(const unsigned long vendor, const unsigned long ext) +{ + if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) + return __riscv_has_extension_likely_alternatives(vendor, + ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE); + else + return __riscv_isa_vendor_extension_available(-1, vendor, ext); +} + +static __always_inline bool +riscv_has_vendor_extension_unlikely(const unsigned long vendor, const unsigned long ext) +{ + if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) + return __riscv_has_extension_unlikely_alternatives(vendor, + ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE); + else + return __riscv_isa_vendor_extension_available(-1, vendor, ext); +} + +static __always_inline bool riscv_cpu_has_vendor_extension_likely(const unsigned long vendor, + int cpu, const unsigned long ext) +{ + if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && + __riscv_has_extension_likely_alternatives(vendor, + ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE)) + return true; + else + return __riscv_isa_vendor_extension_available(cpu, vendor, ext); +} + +static __always_inline bool riscv_cpu_has_vendor_extension_unlikely(const unsigned long vendor, + int cpu, + const unsigned long ext) +{ + if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && + __riscv_has_extension_unlikely_alternatives(vendor, + ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE)) + return true; + else + return __riscv_isa_vendor_extension_available(cpu, vendor, ext); +} + #endif /* _ASM_VENDOR_EXTENSIONS_H */ diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index c9f36822e337..17371887abcc 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -833,25 +833,29 @@ void __init_or_module riscv_cpufeature_patch_func(struct alt_entry *begin, { struct alt_entry *alt; void *oldptr, *altptr; - u16 id, value; + u16 id, value, vendor;
if (stage == RISCV_ALTERNATIVES_EARLY_BOOT) return;
for (alt = begin; alt < end; alt++) { - if (alt->vendor_id != 0) - continue; - id = PATCH_ID_CPUFEATURE_ID(alt->patch_id); + vendor = PATCH_ID_CPUFEATURE_ID(alt->vendor_id);
- if (id >= RISCV_ISA_EXT_MAX) { + if (id < RISCV_ISA_EXT_MAX) { + if (alt->vendor_id != 0) + continue; + + if (!__riscv_isa_extension_available(NULL, id)) + continue; + } else if (id >= RISCV_VENDOR_EXT_ALTERNATIVES_BASE) { + if (!__riscv_isa_vendor_extension_available(-1, vendor, id)) + continue; + } else { WARN(1, "This extension id:%d is not in ISA extension list", id); continue; }
- if (!__riscv_isa_extension_available(NULL, id)) - continue; - value = PATCH_ID_CPUFEATURE_VALUE(alt->patch_id); if (!riscv_cpufeature_patch_check(id, value)) continue; diff --git a/arch/riscv/kernel/vendor_extensions.c b/arch/riscv/kernel/vendor_extensions.c index f76cb3013c2d..eced93eec5a6 100644 --- a/arch/riscv/kernel/vendor_extensions.c +++ b/arch/riscv/kernel/vendor_extensions.c @@ -3,6 +3,7 @@ * Copyright 2024 Rivos, Inc */
+#include <asm/vendorid_list.h> #include <asm/vendor_extensions.h> #include <asm/vendor_extensions/thead.h>
@@ -16,3 +17,42 @@ const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[] = { };
const size_t riscv_isa_vendor_ext_list_size = ARRAY_SIZE(riscv_isa_vendor_ext_list); + +/** + * __riscv_isa_vendor_extension_available() - Check whether given vendor + * extension is available or not. + * + * @cpu: check if extension is available on this cpu + * @vendor: vendor that the extension is a member of + * @bit: bit position of the desired extension + * Return: true or false + * + * NOTE: When cpu is -1, will check if extension is available on all cpus + */ +bool __riscv_isa_vendor_extension_available(int cpu, unsigned long vendor, unsigned int bit) +{ + unsigned long *bmap; + struct riscv_isainfo *cpu_bmap; + size_t bmap_size; + + switch (vendor) { +#ifdef CONFIG_RISCV_ISA_VENDOR_EXT_THEAD + case THEAD_VENDOR_ID: + bmap = riscv_isa_vendor_ext_list_thead.vendor_bitmap; + cpu_bmap = riscv_isa_vendor_ext_list_thead.per_hart_vendor_bitmap; + bmap_size = riscv_isa_vendor_ext_list_thead.bitmap_size; + break; +#endif + default: + return false; + } + + if (cpu != -1) + bmap = cpu_bmap[cpu].isa; + + if (bit >= bmap_size) + return false; + + return test_bit(bit, bmap) ? true : false; +} +EXPORT_SYMBOL_GPL(__riscv_isa_vendor_extension_available);
On Sat, Apr 20, 2024 at 06:04:40PM -0700, Charlie Jenkins wrote:
Vendor extensions are maintained in per-vendor structs (separate from standard extensions which live in riscv_isa). Create vendor variants for the existing extension helpers to interface with the riscv_isa_vendor bitmaps.
There is a good amount of overlap between these functions, so the alternative checking code can be factored out.
Can you please split this out?
Signed-off-by: Charlie Jenkins charlie@rivosinc.com
arch/riscv/errata/sifive/errata.c | 3 ++ arch/riscv/errata/thead/errata.c | 3 ++ arch/riscv/include/asm/cpufeature.h | 86 +++++++++++++++++------------- arch/riscv/include/asm/vendor_extensions.h | 56 +++++++++++++++++++ arch/riscv/kernel/cpufeature.c | 20 ++++--- arch/riscv/kernel/vendor_extensions.c | 40 ++++++++++++++ 6 files changed, 164 insertions(+), 44 deletions(-)
diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c index 3d9a32d791f7..b29b6e405ff2 100644 --- a/arch/riscv/errata/sifive/errata.c +++ b/arch/riscv/errata/sifive/errata.c @@ -12,6 +12,7 @@ #include <asm/alternative.h> #include <asm/vendorid_list.h> #include <asm/errata_list.h> +#include <asm/vendor_extensions.h> struct errata_info_t { char name[32]; @@ -99,6 +100,8 @@ void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end, for (alt = begin; alt < end; alt++) { if (alt->vendor_id != SIFIVE_VENDOR_ID) continue;
if (alt->patch_id >= RISCV_VENDOR_EXT_ALTERNATIVES_BASE)continue;diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c index b1c410bbc1ae..d8e78cc687bc 100644 --- a/arch/riscv/errata/thead/errata.c +++ b/arch/riscv/errata/thead/errata.c @@ -18,6 +18,7 @@ #include <asm/io.h> #include <asm/patch.h> #include <asm/vendorid_list.h> +#include <asm/vendor_extensions.h> static bool errata_probe_pbmt(unsigned int stage, unsigned long arch_id, unsigned long impid) @@ -163,6 +164,8 @@ void thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end, for (alt = begin; alt < end; alt++) { if (alt->vendor_id != THEAD_VENDOR_ID) continue;
if (alt->patch_id >= RISCV_VENDOR_EXT_ALTERNATIVES_BASE)continue;
if (alt->patch_id >= ERRATA_THEAD_NUMBER)
This number is 2, how does the patching actually work for vendor stuff when the base is always greater than 2?
continue;diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index db6a6d7d6b2e..83e1143db9ad 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -103,22 +103,13 @@ bool __riscv_isa_extension_available(const unsigned long *isa_bitmap, unsigned i __riscv_isa_extension_available(isa_bitmap, RISCV_ISA_EXT_##ext) static __always_inline bool -riscv_has_extension_likely(const unsigned long ext) +__riscv_has_extension_likely_alternatives(const unsigned long vendor, const unsigned long ext) {
- compiletime_assert(ext < RISCV_ISA_EXT_MAX,
"ext must be < RISCV_ISA_EXT_MAX");- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) {
asm goto(ALTERNATIVE("j %l[l_no]", "nop", 0, %[ext], 1):: [ext] "i" (ext):: l_no);- } else {
if (!__riscv_isa_extension_available(NULL, ext))goto l_no;- }
- asm goto(ALTERNATIVE("j %l[l_no]", "nop", %[vendor], %[ext], 1)
- :
- : [vendor] "i" (vendor), [ext] "i" (ext)
- :
- : l_no);
return true; l_no: @@ -126,42 +117,65 @@ riscv_has_extension_likely(const unsigned long ext) } static __always_inline bool -riscv_has_extension_unlikely(const unsigned long ext) +__riscv_has_extension_unlikely_alternatives(const unsigned long vendor, const unsigned long ext)
ngl, I think you could drop the _alternatives from these - the likely/unlikely is only actually a thing because of the alternatives in the first place & just retain the __ as a differentiator. That'd help you with some of the long-line wrangling you've been doing below.
{
- compiletime_assert(ext < RISCV_ISA_EXT_MAX,
"ext must be < RISCV_ISA_EXT_MAX");- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) {
asm goto(ALTERNATIVE("nop", "j %l[l_yes]", 0, %[ext], 1):: [ext] "i" (ext):: l_yes);- } else {
if (__riscv_isa_extension_available(NULL, ext))goto l_yes;- }
- asm goto(ALTERNATIVE("nop", "j %l[l_yes]", %[vendor], %[ext], 1)
- :
- : [vendor] "i" (vendor), [ext] "i" (ext)
- :
- : l_yes);
return false; l_yes: return true; } +static __always_inline bool +riscv_has_extension_likely(const unsigned long ext)
Can you format this so that its on one line & wrap the arguments if needs be?
+{
- compiletime_assert(ext < RISCV_ISA_EXT_MAX,
"ext must be < RISCV_ISA_EXT_MAX");- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE))
return __riscv_has_extension_likely_alternatives(0, ext);- else
I'm almost certain I said this before, but none of the else branches are needed here, there's a return in the if branch, so the remainder of the function becomes unconditionally executed.
return __riscv_isa_extension_available(NULL, ext);+}
+static __always_inline bool +riscv_has_extension_unlikely(const unsigned long ext) +{
- compiletime_assert(ext < RISCV_ISA_EXT_MAX,
"ext must be < RISCV_ISA_EXT_MAX");- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE))
return __riscv_has_extension_unlikely_alternatives(0, ext);- else
return __riscv_isa_extension_available(NULL, ext);+}
static __always_inline bool riscv_cpu_has_extension_likely(int cpu, const unsigned long ext) {
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && riscv_has_extension_likely(ext))
return true;
- compiletime_assert(ext < RISCV_ISA_EXT_MAX,
"ext must be < RISCV_ISA_EXT_MAX");
- return __riscv_isa_extension_available(hart_isa[cpu].isa, ext);
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) &&
__riscv_has_extension_likely_alternatives(0, ext))
0 is meaningless, please make this more understandable using a define of some sort.
return true;- else
return __riscv_isa_extension_available(hart_isa[cpu].isa, ext);} static __always_inline bool riscv_cpu_has_extension_unlikely(int cpu, const unsigned long ext) {
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && riscv_has_extension_unlikely(ext))
return true;
- compiletime_assert(ext < RISCV_ISA_EXT_MAX,
"ext must be < RISCV_ISA_EXT_MAX");
- return __riscv_isa_extension_available(hart_isa[cpu].isa, ext);
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) &&
__riscv_has_extension_unlikely_alternatives(0, ext))return true;- else
return __riscv_isa_extension_available(hart_isa[cpu].isa, ext);} #endif diff --git a/arch/riscv/include/asm/vendor_extensions.h b/arch/riscv/include/asm/vendor_extensions.h index 0af1ddd0af70..3e676a96016e 100644 --- a/arch/riscv/include/asm/vendor_extensions.h +++ b/arch/riscv/include/asm/vendor_extensions.h @@ -23,4 +23,60 @@ extern const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[]; extern const size_t riscv_isa_vendor_ext_list_size; +/*
- The alternatives need some way of distinguishing between vendor extensions
- and errata. Incrementing all of the vendor extension keys so they are at
- least 0x8000 accomplishes that.
- */
+#define RISCV_VENDOR_EXT_ALTERNATIVES_BASE 0x8000
+bool __riscv_isa_vendor_extension_available(int cpu, unsigned long vendor, unsigned int bit); +#define riscv_cpu_isa_vendor_extension_available(cpu, vendor, ext) \
- __riscv_isa_vendor_extension_available(cpu, vendor, RISCV_ISA_VENDOR_EXT_##ext)
+#define riscv_isa_vendor_extension_available(vendor, ext) \
- __riscv_isa_vendor_extension_available(-1, vendor, RISCV_ISA_VENDOR_EXT_##ext)
+static __always_inline bool +riscv_has_vendor_extension_likely(const unsigned long vendor, const unsigned long ext) +{
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE))
return __riscv_has_extension_likely_alternatives(vendor,ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE);- else
return __riscv_isa_vendor_extension_available(-1, vendor, ext);+}
+static __always_inline bool +riscv_has_vendor_extension_unlikely(const unsigned long vendor, const unsigned long ext) +{
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE))
return __riscv_has_extension_unlikely_alternatives(vendor,ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE);- else
return __riscv_isa_vendor_extension_available(-1, vendor, ext);+}
+static __always_inline bool riscv_cpu_has_vendor_extension_likely(const unsigned long vendor,
int cpu, const unsigned long ext)+{
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) &&
__riscv_has_extension_likely_alternatives(vendor,ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE))return true;- else
return __riscv_isa_vendor_extension_available(cpu, vendor, ext);+}
+static __always_inline bool riscv_cpu_has_vendor_extension_unlikely(const unsigned long vendor,
int cpu,const unsigned long ext)+{
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) &&
__riscv_has_extension_unlikely_alternatives(vendor,ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE))return true;- else
return __riscv_isa_vendor_extension_available(cpu, vendor, ext);+}
#endif /* _ASM_VENDOR_EXTENSIONS_H */ diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index c9f36822e337..17371887abcc 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -833,25 +833,29 @@ void __init_or_module riscv_cpufeature_patch_func(struct alt_entry *begin, { struct alt_entry *alt; void *oldptr, *altptr;
- u16 id, value;
- u16 id, value, vendor;
if (stage == RISCV_ALTERNATIVES_EARLY_BOOT) return; for (alt = begin; alt < end; alt++) {
if (alt->vendor_id != 0)continue;- id = PATCH_ID_CPUFEATURE_ID(alt->patch_id);
vendor = PATCH_ID_CPUFEATURE_ID(alt->vendor_id);
if (id >= RISCV_ISA_EXT_MAX) {
if (id < RISCV_ISA_EXT_MAX) {
I think any reliance on the standard ext max requires a comment explaining what the interaction is between it and the vendor stuff.
if (alt->vendor_id != 0)continue;
If this happens, it's a bug, should we be continuing silently?
Cheers, Conor.
if (!__riscv_isa_extension_available(NULL, id))continue;} else if (id >= RISCV_VENDOR_EXT_ALTERNATIVES_BASE) {if (!__riscv_isa_vendor_extension_available(-1, vendor, id))continue;} else { WARN(1, "This extension id:%d is not in ISA extension list", id); continue;
if (!__riscv_isa_extension_available(NULL, id))continue;- value = PATCH_ID_CPUFEATURE_VALUE(alt->patch_id); if (!riscv_cpufeature_patch_check(id, value)) continue;
diff --git a/arch/riscv/kernel/vendor_extensions.c b/arch/riscv/kernel/vendor_extensions.c index f76cb3013c2d..eced93eec5a6 100644 --- a/arch/riscv/kernel/vendor_extensions.c +++ b/arch/riscv/kernel/vendor_extensions.c @@ -3,6 +3,7 @@
- Copyright 2024 Rivos, Inc
*/ +#include <asm/vendorid_list.h> #include <asm/vendor_extensions.h> #include <asm/vendor_extensions/thead.h> @@ -16,3 +17,42 @@ const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[] = { }; const size_t riscv_isa_vendor_ext_list_size = ARRAY_SIZE(riscv_isa_vendor_ext_list);
+/**
- __riscv_isa_vendor_extension_available() - Check whether given vendor
- extension is available or not.
- @cpu: check if extension is available on this cpu
- @vendor: vendor that the extension is a member of
- @bit: bit position of the desired extension
- Return: true or false
- NOTE: When cpu is -1, will check if extension is available on all cpus
- */
+bool __riscv_isa_vendor_extension_available(int cpu, unsigned long vendor, unsigned int bit) +{
- unsigned long *bmap;
- struct riscv_isainfo *cpu_bmap;
- size_t bmap_size;
- switch (vendor) {
+#ifdef CONFIG_RISCV_ISA_VENDOR_EXT_THEAD
- case THEAD_VENDOR_ID:
bmap = riscv_isa_vendor_ext_list_thead.vendor_bitmap;cpu_bmap = riscv_isa_vendor_ext_list_thead.per_hart_vendor_bitmap;bmap_size = riscv_isa_vendor_ext_list_thead.bitmap_size;break;+#endif
- default:
return false;- }
- if (cpu != -1)
bmap = cpu_bmap[cpu].isa;- if (bit >= bmap_size)
return false;- return test_bit(bit, bmap) ? true : false;
+} +EXPORT_SYMBOL_GPL(__riscv_isa_vendor_extension_available);
-- 2.44.0
On Fri, Apr 26, 2024 at 05:19:59PM +0100, Conor Dooley wrote:
On Sat, Apr 20, 2024 at 06:04:40PM -0700, Charlie Jenkins wrote:
Vendor extensions are maintained in per-vendor structs (separate from standard extensions which live in riscv_isa). Create vendor variants for the existing extension helpers to interface with the riscv_isa_vendor bitmaps.
There is a good amount of overlap between these functions, so the alternative checking code can be factored out.
Can you please split this out?
Sure.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com
arch/riscv/errata/sifive/errata.c | 3 ++ arch/riscv/errata/thead/errata.c | 3 ++ arch/riscv/include/asm/cpufeature.h | 86 +++++++++++++++++------------- arch/riscv/include/asm/vendor_extensions.h | 56 +++++++++++++++++++ arch/riscv/kernel/cpufeature.c | 20 ++++--- arch/riscv/kernel/vendor_extensions.c | 40 ++++++++++++++ 6 files changed, 164 insertions(+), 44 deletions(-)
diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c index 3d9a32d791f7..b29b6e405ff2 100644 --- a/arch/riscv/errata/sifive/errata.c +++ b/arch/riscv/errata/sifive/errata.c @@ -12,6 +12,7 @@ #include <asm/alternative.h> #include <asm/vendorid_list.h> #include <asm/errata_list.h> +#include <asm/vendor_extensions.h> struct errata_info_t { char name[32]; @@ -99,6 +100,8 @@ void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end, for (alt = begin; alt < end; alt++) { if (alt->vendor_id != SIFIVE_VENDOR_ID) continue;
if (alt->patch_id >= RISCV_VENDOR_EXT_ALTERNATIVES_BASE)continue;diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c index b1c410bbc1ae..d8e78cc687bc 100644 --- a/arch/riscv/errata/thead/errata.c +++ b/arch/riscv/errata/thead/errata.c @@ -18,6 +18,7 @@ #include <asm/io.h> #include <asm/patch.h> #include <asm/vendorid_list.h> +#include <asm/vendor_extensions.h> static bool errata_probe_pbmt(unsigned int stage, unsigned long arch_id, unsigned long impid) @@ -163,6 +164,8 @@ void thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end, for (alt = begin; alt < end; alt++) { if (alt->vendor_id != THEAD_VENDOR_ID) continue;
if (alt->patch_id >= RISCV_VENDOR_EXT_ALTERNATIVES_BASE)continue;if (alt->patch_id >= ERRATA_THEAD_NUMBER)This number is 2, how does the patching actually work for vendor stuff when the base is always greater than 2?
Patching is handled through the patcher at the end of cpufeature.c. The vendor_id field is set in the alternatives for errata and now also for vendor extensions. The vendor extension patching is all handled generically.
This is distinguished by the patch_id being greater than RISCV_VENDOR_EXT_ALTERNATIVES_BASE, which should leave way more than enough room for errata ids. Since the code already checks if the patch_id is greater than the errata number, I can drop the "if (alt->patch_id >= RISCV_VENDOR_EXT_ALTERNATIVES_BASE)" check.
continue;diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index db6a6d7d6b2e..83e1143db9ad 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -103,22 +103,13 @@ bool __riscv_isa_extension_available(const unsigned long *isa_bitmap, unsigned i __riscv_isa_extension_available(isa_bitmap, RISCV_ISA_EXT_##ext) static __always_inline bool -riscv_has_extension_likely(const unsigned long ext) +__riscv_has_extension_likely_alternatives(const unsigned long vendor, const unsigned long ext) {
- compiletime_assert(ext < RISCV_ISA_EXT_MAX,
"ext must be < RISCV_ISA_EXT_MAX");- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) {
asm goto(ALTERNATIVE("j %l[l_no]", "nop", 0, %[ext], 1):: [ext] "i" (ext):: l_no);- } else {
if (!__riscv_isa_extension_available(NULL, ext))goto l_no;- }
- asm goto(ALTERNATIVE("j %l[l_no]", "nop", %[vendor], %[ext], 1)
- :
- : [vendor] "i" (vendor), [ext] "i" (ext)
- :
- : l_no);
return true; l_no: @@ -126,42 +117,65 @@ riscv_has_extension_likely(const unsigned long ext) } static __always_inline bool -riscv_has_extension_unlikely(const unsigned long ext) +__riscv_has_extension_unlikely_alternatives(const unsigned long vendor, const unsigned long ext)
ngl, I think you could drop the _alternatives from these - the likely/unlikely is only actually a thing because of the alternatives in the first place & just retain the __ as a differentiator. That'd help you with some of the long-line wrangling you've been doing below.
Sounds good.
{
- compiletime_assert(ext < RISCV_ISA_EXT_MAX,
"ext must be < RISCV_ISA_EXT_MAX");- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) {
asm goto(ALTERNATIVE("nop", "j %l[l_yes]", 0, %[ext], 1):: [ext] "i" (ext):: l_yes);- } else {
if (__riscv_isa_extension_available(NULL, ext))goto l_yes;- }
- asm goto(ALTERNATIVE("nop", "j %l[l_yes]", %[vendor], %[ext], 1)
- :
- : [vendor] "i" (vendor), [ext] "i" (ext)
- :
- : l_yes);
return false; l_yes: return true; } +static __always_inline bool +riscv_has_extension_likely(const unsigned long ext)
Can you format this so that its on one line & wrap the arguments if needs be?
+{
- compiletime_assert(ext < RISCV_ISA_EXT_MAX,
"ext must be < RISCV_ISA_EXT_MAX");- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE))
return __riscv_has_extension_likely_alternatives(0, ext);- else
I'm almost certain I said this before, but none of the else branches are needed here, there's a return in the if branch, so the remainder of the function becomes unconditionally executed.
Will fix.
return __riscv_isa_extension_available(NULL, ext);+}
+static __always_inline bool +riscv_has_extension_unlikely(const unsigned long ext) +{
- compiletime_assert(ext < RISCV_ISA_EXT_MAX,
"ext must be < RISCV_ISA_EXT_MAX");- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE))
return __riscv_has_extension_unlikely_alternatives(0, ext);- else
return __riscv_isa_extension_available(NULL, ext);+}
static __always_inline bool riscv_cpu_has_extension_likely(int cpu, const unsigned long ext) {
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && riscv_has_extension_likely(ext))
return true;
- compiletime_assert(ext < RISCV_ISA_EXT_MAX,
"ext must be < RISCV_ISA_EXT_MAX");
- return __riscv_isa_extension_available(hart_isa[cpu].isa, ext);
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) &&
__riscv_has_extension_likely_alternatives(0, ext))0 is meaningless, please make this more understandable using a define of some sort.
I'll add a define.
return true;- else
return __riscv_isa_extension_available(hart_isa[cpu].isa, ext);} static __always_inline bool riscv_cpu_has_extension_unlikely(int cpu, const unsigned long ext) {
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && riscv_has_extension_unlikely(ext))
return true;
- compiletime_assert(ext < RISCV_ISA_EXT_MAX,
"ext must be < RISCV_ISA_EXT_MAX");
- return __riscv_isa_extension_available(hart_isa[cpu].isa, ext);
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) &&
__riscv_has_extension_unlikely_alternatives(0, ext))return true;- else
return __riscv_isa_extension_available(hart_isa[cpu].isa, ext);} #endif diff --git a/arch/riscv/include/asm/vendor_extensions.h b/arch/riscv/include/asm/vendor_extensions.h index 0af1ddd0af70..3e676a96016e 100644 --- a/arch/riscv/include/asm/vendor_extensions.h +++ b/arch/riscv/include/asm/vendor_extensions.h @@ -23,4 +23,60 @@ extern const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[]; extern const size_t riscv_isa_vendor_ext_list_size; +/*
- The alternatives need some way of distinguishing between vendor extensions
- and errata. Incrementing all of the vendor extension keys so they are at
- least 0x8000 accomplishes that.
- */
+#define RISCV_VENDOR_EXT_ALTERNATIVES_BASE 0x8000
+bool __riscv_isa_vendor_extension_available(int cpu, unsigned long vendor, unsigned int bit); +#define riscv_cpu_isa_vendor_extension_available(cpu, vendor, ext) \
- __riscv_isa_vendor_extension_available(cpu, vendor, RISCV_ISA_VENDOR_EXT_##ext)
+#define riscv_isa_vendor_extension_available(vendor, ext) \
- __riscv_isa_vendor_extension_available(-1, vendor, RISCV_ISA_VENDOR_EXT_##ext)
+static __always_inline bool +riscv_has_vendor_extension_likely(const unsigned long vendor, const unsigned long ext) +{
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE))
return __riscv_has_extension_likely_alternatives(vendor,ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE);- else
return __riscv_isa_vendor_extension_available(-1, vendor, ext);+}
+static __always_inline bool +riscv_has_vendor_extension_unlikely(const unsigned long vendor, const unsigned long ext) +{
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE))
return __riscv_has_extension_unlikely_alternatives(vendor,ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE);- else
return __riscv_isa_vendor_extension_available(-1, vendor, ext);+}
+static __always_inline bool riscv_cpu_has_vendor_extension_likely(const unsigned long vendor,
int cpu, const unsigned long ext)+{
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) &&
__riscv_has_extension_likely_alternatives(vendor,ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE))return true;- else
return __riscv_isa_vendor_extension_available(cpu, vendor, ext);+}
+static __always_inline bool riscv_cpu_has_vendor_extension_unlikely(const unsigned long vendor,
int cpu,const unsigned long ext)+{
- if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) &&
__riscv_has_extension_unlikely_alternatives(vendor,ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE))return true;- else
return __riscv_isa_vendor_extension_available(cpu, vendor, ext);+}
#endif /* _ASM_VENDOR_EXTENSIONS_H */ diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index c9f36822e337..17371887abcc 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -833,25 +833,29 @@ void __init_or_module riscv_cpufeature_patch_func(struct alt_entry *begin, { struct alt_entry *alt; void *oldptr, *altptr;
- u16 id, value;
- u16 id, value, vendor;
if (stage == RISCV_ALTERNATIVES_EARLY_BOOT) return; for (alt = begin; alt < end; alt++) {
if (alt->vendor_id != 0)continue;- id = PATCH_ID_CPUFEATURE_ID(alt->patch_id);
vendor = PATCH_ID_CPUFEATURE_ID(alt->vendor_id);
if (id >= RISCV_ISA_EXT_MAX) {
if (id < RISCV_ISA_EXT_MAX) {I think any reliance on the standard ext max requires a comment explaining what the interaction is between it and the vendor stuff.
I'll add a comment describing that anything with an id below RISCV_ISA_EXT_MAX is interpreted as a standard extension and anything with an id above RISCV_VENDOR_EXT_ALTERNATIVES_BASE will be interpreted as a vendor extension.
if (alt->vendor_id != 0)continue;If this happens, it's a bug, should we be continuing silently?
This indicates that the patch should be treated as errata and that's why it silently continues. I can add a comment that says that.
- Charlie
Cheers, Conor.
if (!__riscv_isa_extension_available(NULL, id))continue;} else if (id >= RISCV_VENDOR_EXT_ALTERNATIVES_BASE) {if (!__riscv_isa_vendor_extension_available(-1, vendor, id))continue;} else { WARN(1, "This extension id:%d is not in ISA extension list", id); continue;
if (!__riscv_isa_extension_available(NULL, id))continue;- value = PATCH_ID_CPUFEATURE_VALUE(alt->patch_id); if (!riscv_cpufeature_patch_check(id, value)) continue;
diff --git a/arch/riscv/kernel/vendor_extensions.c b/arch/riscv/kernel/vendor_extensions.c index f76cb3013c2d..eced93eec5a6 100644 --- a/arch/riscv/kernel/vendor_extensions.c +++ b/arch/riscv/kernel/vendor_extensions.c @@ -3,6 +3,7 @@
- Copyright 2024 Rivos, Inc
*/ +#include <asm/vendorid_list.h> #include <asm/vendor_extensions.h> #include <asm/vendor_extensions/thead.h> @@ -16,3 +17,42 @@ const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[] = { }; const size_t riscv_isa_vendor_ext_list_size = ARRAY_SIZE(riscv_isa_vendor_ext_list);
+/**
- __riscv_isa_vendor_extension_available() - Check whether given vendor
- extension is available or not.
- @cpu: check if extension is available on this cpu
- @vendor: vendor that the extension is a member of
- @bit: bit position of the desired extension
- Return: true or false
- NOTE: When cpu is -1, will check if extension is available on all cpus
- */
+bool __riscv_isa_vendor_extension_available(int cpu, unsigned long vendor, unsigned int bit) +{
- unsigned long *bmap;
- struct riscv_isainfo *cpu_bmap;
- size_t bmap_size;
- switch (vendor) {
+#ifdef CONFIG_RISCV_ISA_VENDOR_EXT_THEAD
- case THEAD_VENDOR_ID:
bmap = riscv_isa_vendor_ext_list_thead.vendor_bitmap;cpu_bmap = riscv_isa_vendor_ext_list_thead.per_hart_vendor_bitmap;bmap_size = riscv_isa_vendor_ext_list_thead.bitmap_size;break;+#endif
- default:
return false;- }
- if (cpu != -1)
bmap = cpu_bmap[cpu].isa;- if (bit >= bmap_size)
return false;- return test_bit(bit, bmap) ? true : false;
+} +EXPORT_SYMBOL_GPL(__riscv_isa_vendor_extension_available);
-- 2.44.0
On Fri, Apr 26, 2024 at 01:01:10PM -0700, Charlie Jenkins wrote:
On Fri, Apr 26, 2024 at 05:19:59PM +0100, Conor Dooley wrote:
On Sat, Apr 20, 2024 at 06:04:40PM -0700, Charlie Jenkins wrote:
@@ -163,6 +164,8 @@ void thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end, for (alt = begin; alt < end; alt++) { if (alt->vendor_id != THEAD_VENDOR_ID) continue;
if (alt->patch_id >= RISCV_VENDOR_EXT_ALTERNATIVES_BASE)continue;if (alt->patch_id >= ERRATA_THEAD_NUMBER)This number is 2, how does the patching actually work for vendor stuff when the base is always greater than 2?
Patching is handled through the patcher at the end of cpufeature.c. The vendor_id field is set in the alternatives for errata and now also for vendor extensions. The vendor extension patching is all handled generically.
This is distinguished by the patch_id being greater than RISCV_VENDOR_EXT_ALTERNATIVES_BASE, which should leave way more than enough room for errata ids. Since the code already checks if the patch_id is greater than the errata number, I can drop the "if (alt->patch_id >= RISCV_VENDOR_EXT_ALTERNATIVES_BASE)" check.
Ah, ye, I was being dumb - I was somehow thinking that you were using the errata patch function to patch this, which obviously wouldn't be a good design. If that had been the case, you'd never patch anything, but you patch the extensions using the cpufeature patch function so it all works out.
If anything, I guess there could be an assert here somewhere that enforces ERRATA_VENDOR_NUMBER < VENDOR_EXT_BASE.
Migrate xandespmu out of riscv_isa_ext and into a new Andes-specific vendor namespace.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- arch/riscv/Kconfig.vendor | 10 +++++++ arch/riscv/include/asm/hwcap.h | 1 - arch/riscv/include/asm/vendor_extensions/andes.h | 19 +++++++++++++ arch/riscv/kernel/cpufeature.c | 1 - arch/riscv/kernel/vendor_extensions.c | 11 ++++++++ arch/riscv/kernel/vendor_extensions/Makefile | 1 + arch/riscv/kernel/vendor_extensions/andes.c | 35 ++++++++++++++++++++++++ drivers/perf/riscv_pmu_sbi.c | 8 ++++-- 8 files changed, 81 insertions(+), 5 deletions(-)
diff --git a/arch/riscv/Kconfig.vendor b/arch/riscv/Kconfig.vendor index d57254f28ea6..d5555ed696fb 100644 --- a/arch/riscv/Kconfig.vendor +++ b/arch/riscv/Kconfig.vendor @@ -7,3 +7,13 @@ config RISCV_ISA_VENDOR_EXT_THEAD requested by hardware probing to be ignored.
If you don't know what to do here, say Y. + +config RISCV_ISA_VENDOR_EXT_ANDES + bool "Andes vendor extension support" + default y + help + Say N here if you want to disable all Andes vendor extension + support. This will cause any Andes vendor extensions that are + requested by hardware probing to be ignored. + + If you don't know what to do here, say Y. diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index e17d0078a651..1f2d2599c655 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -80,7 +80,6 @@ #define RISCV_ISA_EXT_ZFA 71 #define RISCV_ISA_EXT_ZTSO 72 #define RISCV_ISA_EXT_ZACAS 73 -#define RISCV_ISA_EXT_XANDESPMU 74
#define RISCV_ISA_EXT_XLINUXENVCFG 127
diff --git a/arch/riscv/include/asm/vendor_extensions/andes.h b/arch/riscv/include/asm/vendor_extensions/andes.h new file mode 100644 index 000000000000..54b0ad1a409d --- /dev/null +++ b/arch/riscv/include/asm/vendor_extensions/andes.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_RISCV_VENDOR_EXTENSIONS_ANDES_H +#define _ASM_RISCV_VENDOR_EXTENSIONS_ANDES_H + +#include <asm/vendor_extensions.h> + +#include <linux/types.h> + +#define RISCV_ISA_VENDOR_EXT_XANDESPMU 0 + +/* + * Extension keys should be strictly less than max. + * It is safe to increment this when necessary. + */ +#define RISCV_ISA_VENDOR_EXT_MAX_ANDES 32 + +extern const struct riscv_isa_vendor_ext_data_list riscv_isa_vendor_ext_list_andes; + +#endif diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 17371887abcc..0cef08a3d891 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -289,7 +289,6 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = { __RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL), __RISCV_ISA_EXT_DATA(svnapot, RISCV_ISA_EXT_SVNAPOT), __RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT), - __RISCV_ISA_EXT_DATA(xandespmu, RISCV_ISA_EXT_XANDESPMU), };
const size_t riscv_isa_ext_count = ARRAY_SIZE(riscv_isa_ext); diff --git a/arch/riscv/kernel/vendor_extensions.c b/arch/riscv/kernel/vendor_extensions.c index eced93eec5a6..e31f16b8cec9 100644 --- a/arch/riscv/kernel/vendor_extensions.c +++ b/arch/riscv/kernel/vendor_extensions.c @@ -5,6 +5,7 @@
#include <asm/vendorid_list.h> #include <asm/vendor_extensions.h> +#include <asm/vendor_extensions/andes.h> #include <asm/vendor_extensions/thead.h>
#include <linux/array_size.h> @@ -14,6 +15,9 @@ const struct riscv_isa_vendor_ext_data_list *riscv_isa_vendor_ext_list[] = { #ifdef CONFIG_RISCV_ISA_VENDOR_EXT_THEAD &riscv_isa_vendor_ext_list_thead, #endif +#ifdef CONFIG_RISCV_ISA_VENDOR_EXT_ANDES + &riscv_isa_vendor_ext_list_andes, +#endif };
const size_t riscv_isa_vendor_ext_list_size = ARRAY_SIZE(riscv_isa_vendor_ext_list); @@ -42,6 +46,13 @@ bool __riscv_isa_vendor_extension_available(int cpu, unsigned long vendor, unsig cpu_bmap = riscv_isa_vendor_ext_list_thead.per_hart_vendor_bitmap; bmap_size = riscv_isa_vendor_ext_list_thead.bitmap_size; break; +#endif +#ifdef CONFIG_RISCV_VENDOR_EXT_ANDES + case ANDES_VENDOR_ID: + bmap = riscv_isa_vendor_ext_list_andes.vendor_bitmap; + cpu_bmap = riscv_isa_vendor_ext_list_andes.per_hart_vendor_bitmap; + bmap_size = riscv_isa_vendor_ext_list_andes.bitmap_size; + break; #endif default: return false; diff --git a/arch/riscv/kernel/vendor_extensions/Makefile b/arch/riscv/kernel/vendor_extensions/Makefile index 3383066baaab..8f1c5a4dc38f 100644 --- a/arch/riscv/kernel/vendor_extensions/Makefile +++ b/arch/riscv/kernel/vendor_extensions/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only
obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_THEAD) += thead.o +obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_ANDES) += andes.o diff --git a/arch/riscv/kernel/vendor_extensions/andes.c b/arch/riscv/kernel/vendor_extensions/andes.c new file mode 100644 index 000000000000..8451106d2e07 --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions/andes.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include <asm/cpufeature.h> +#include <asm/vendor_extensions.h> +#include <asm/vendor_extensions/andes.h> + +#include <linux/array_size.h> +#include <linux/types.h> + +const struct riscv_isa_ext_data riscv_isa_vendor_ext_andes[] = { + __RISCV_ISA_EXT_DATA(xandespmu, RISCV_ISA_VENDOR_EXT_XANDESPMU), +}; + +/* + * The first member of this struct must be a bitmap named isa so it can be + * compatible with riscv_isainfo even though the sizes of the bitmaps may be + * different. + */ +struct riscv_isavendorinfo_andes { + DECLARE_BITMAP(isa, RISCV_ISA_VENDOR_EXT_MAX_ANDES); +}; + +/* Hart specific T-Head vendor extension support */ +static struct riscv_isavendorinfo_andes hart_vendorinfo_andes[NR_CPUS]; + +/* Set of T-Head vendor extensions supported on all harts */ +DECLARE_BITMAP(vendorinfo_andes, RISCV_ISA_VENDOR_EXT_MAX_ANDES); + +const struct riscv_isa_vendor_ext_data_list riscv_isa_vendor_ext_list_andes = { + .ext_data = riscv_isa_vendor_ext_andes, + .per_hart_vendor_bitmap = (struct riscv_isainfo *)hart_vendorinfo_andes, + .vendor_bitmap = vendorinfo_andes, + .ext_data_count = ARRAY_SIZE(riscv_isa_vendor_ext_andes), + .bitmap_size = RISCV_ISA_VENDOR_EXT_MAX_ANDES +}; diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c index 8cbe6e5f9c39..84760ce61e03 100644 --- a/drivers/perf/riscv_pmu_sbi.c +++ b/drivers/perf/riscv_pmu_sbi.c @@ -24,6 +24,8 @@ #include <asm/errata_list.h> #include <asm/sbi.h> #include <asm/cpufeature.h> +#include <asm/vendorid_list.h> +#include <asm/vendor_extensions/andes.h>
#define ALT_SBI_PMU_OVERFLOW(__ovl) \ asm volatile(ALTERNATIVE_2( \ @@ -32,7 +34,7 @@ asm volatile(ALTERNATIVE_2( \ THEAD_VENDOR_ID, ERRATA_THEAD_PMU, \ CONFIG_ERRATA_THEAD_PMU, \ "csrr %0, " __stringify(ANDES_CSR_SCOUNTEROF), \ - 0, RISCV_ISA_EXT_XANDESPMU, \ + ANDES_VENDOR_ID, RISCV_ISA_VENDOR_EXT_XANDESPMU, \ CONFIG_ANDES_CUSTOM_PMU) \ : "=r" (__ovl) : \ : "memory") @@ -41,7 +43,7 @@ asm volatile(ALTERNATIVE_2( \ asm volatile(ALTERNATIVE( \ "csrc " __stringify(CSR_IP) ", %0\n\t", \ "csrc " __stringify(ANDES_CSR_SLIP) ", %0\n\t", \ - 0, RISCV_ISA_EXT_XANDESPMU, \ + ANDES_VENDOR_ID, RISCV_ISA_VENDOR_EXT_XANDESPMU, \ CONFIG_ANDES_CUSTOM_PMU) \ : : "r"(__irq_mask) \ : "memory") @@ -837,7 +839,7 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pde riscv_cached_mimpid(0) == 0) { riscv_pmu_irq_num = THEAD_C9XX_RV_IRQ_PMU; riscv_pmu_use_irq = true; - } else if (riscv_isa_extension_available(NULL, XANDESPMU) && + } else if (riscv_isa_vendor_extension_available(-1, XANDESPMU) && IS_ENABLED(CONFIG_ANDES_CUSTOM_PMU)) { riscv_pmu_irq_num = ANDES_SLI_CAUSE_BASE + ANDES_RV_IRQ_PMOVI; riscv_pmu_use_irq = true;
On Sat, Apr 20, 2024 at 06:04:41PM -0700, Charlie Jenkins wrote:
Migrate xandespmu out of riscv_isa_ext and into a new Andes-specific vendor namespace. diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c index 8cbe6e5f9c39..84760ce61e03 100644 --- a/drivers/perf/riscv_pmu_sbi.c +++ b/drivers/perf/riscv_pmu_sbi.c @@ -24,6 +24,8 @@ #include <asm/errata_list.h> #include <asm/sbi.h> #include <asm/cpufeature.h> +#include <asm/vendorid_list.h> +#include <asm/vendor_extensions/andes.h> #define ALT_SBI_PMU_OVERFLOW(__ovl) \ asm volatile(ALTERNATIVE_2( \ @@ -32,7 +34,7 @@ asm volatile(ALTERNATIVE_2( \ THEAD_VENDOR_ID, ERRATA_THEAD_PMU, \ CONFIG_ERRATA_THEAD_PMU, \ "csrr %0, " __stringify(ANDES_CSR_SCOUNTEROF), \
0, RISCV_ISA_EXT_XANDESPMU, \
ANDES_VENDOR_ID, RISCV_ISA_VENDOR_EXT_XANDESPMU, \@@ -41,7 +43,7 @@ asm volatile(ALTERNATIVE_2( \ asm volatile(ALTERNATIVE( \ "csrc " __stringify(CSR_IP) ", %0\n\t", \ "csrc " __stringify(ANDES_CSR_SLIP) ", %0\n\t", \
0, RISCV_ISA_EXT_XANDESPMU, \
ANDES_VENDOR_ID, RISCV_ISA_VENDOR_EXT_XANDESPMU, \@@ -837,7 +839,7 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pde riscv_cached_mimpid(0) == 0) { riscv_pmu_irq_num = THEAD_C9XX_RV_IRQ_PMU; riscv_pmu_use_irq = true;
- } else if (riscv_isa_extension_available(NULL, XANDESPMU) &&
- } else if (riscv_isa_vendor_extension_available(-1, XANDESPMU) &&
What's the rationale for this not using riscv_has_extension_unlikely()? Happens once in probe so don't bother? I forget if we discussed it when the code was added, but it would save us from the NULL/-1 syntax, neither of which I think is a good interface.
Also, I'd prob drop the "drivers" from $subject.
I'll come back and look at the rest of this Monday, it's a sunny Friday here and I've still got my devicetree patch queue to clear..
Cheers, Conor.
IS_ENABLED(CONFIG_ANDES_CUSTOM_PMU)) { riscv_pmu_irq_num = ANDES_SLI_CAUSE_BASE + ANDES_RV_IRQ_PMOVI; riscv_pmu_use_irq = true;-- 2.44.0
On Fri, Apr 26, 2024 at 05:25:20PM +0100, Conor Dooley wrote:
On Sat, Apr 20, 2024 at 06:04:41PM -0700, Charlie Jenkins wrote:
Migrate xandespmu out of riscv_isa_ext and into a new Andes-specific vendor namespace. diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c index 8cbe6e5f9c39..84760ce61e03 100644 --- a/drivers/perf/riscv_pmu_sbi.c +++ b/drivers/perf/riscv_pmu_sbi.c @@ -24,6 +24,8 @@ #include <asm/errata_list.h> #include <asm/sbi.h> #include <asm/cpufeature.h> +#include <asm/vendorid_list.h> +#include <asm/vendor_extensions/andes.h> #define ALT_SBI_PMU_OVERFLOW(__ovl) \ asm volatile(ALTERNATIVE_2( \ @@ -32,7 +34,7 @@ asm volatile(ALTERNATIVE_2( \ THEAD_VENDOR_ID, ERRATA_THEAD_PMU, \ CONFIG_ERRATA_THEAD_PMU, \ "csrr %0, " __stringify(ANDES_CSR_SCOUNTEROF), \
0, RISCV_ISA_EXT_XANDESPMU, \
ANDES_VENDOR_ID, RISCV_ISA_VENDOR_EXT_XANDESPMU, \@@ -41,7 +43,7 @@ asm volatile(ALTERNATIVE_2( \ asm volatile(ALTERNATIVE( \ "csrc " __stringify(CSR_IP) ", %0\n\t", \ "csrc " __stringify(ANDES_CSR_SLIP) ", %0\n\t", \
0, RISCV_ISA_EXT_XANDESPMU, \
ANDES_VENDOR_ID, RISCV_ISA_VENDOR_EXT_XANDESPMU, \@@ -837,7 +839,7 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pde riscv_cached_mimpid(0) == 0) { riscv_pmu_irq_num = THEAD_C9XX_RV_IRQ_PMU; riscv_pmu_use_irq = true;
- } else if (riscv_isa_extension_available(NULL, XANDESPMU) &&
- } else if (riscv_isa_vendor_extension_available(-1, XANDESPMU) &&
What's the rationale for this not using riscv_has_extension_unlikely()? Happens once in probe so don't bother? I forget if we discussed it when the code was added, but it would save us from the NULL/-1 syntax, neither of which I think is a good interface.
Doesn't look like something that was ever commented on in the series, but I may have missed it. I can change this to use the alternatives.
This also wasn't supposed to be -1, it's supposed to be the id of the vendor.
Also, I'd prob drop the "drivers" from $subject.
I'll come back and look at the rest of this Monday, it's a sunny Friday here and I've still got my devicetree patch queue to clear..
- Charlie
Cheers, Conor.
IS_ENABLED(CONFIG_ANDES_CUSTOM_PMU)) { riscv_pmu_irq_num = ANDES_SLI_CAUSE_BASE + ANDES_RV_IRQ_PMOVI; riscv_pmu_use_irq = true;-- 2.44.0
On Fri, Apr 26, 2024 at 01:34:19PM -0700, Charlie Jenkins wrote:
On Fri, Apr 26, 2024 at 05:25:20PM +0100, Conor Dooley wrote:
On Sat, Apr 20, 2024 at 06:04:41PM -0700, Charlie Jenkins wrote:
Migrate xandespmu out of riscv_isa_ext and into a new Andes-specific vendor namespace. diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c index 8cbe6e5f9c39..84760ce61e03 100644 --- a/drivers/perf/riscv_pmu_sbi.c +++ b/drivers/perf/riscv_pmu_sbi.c @@ -24,6 +24,8 @@ #include <asm/errata_list.h> #include <asm/sbi.h> #include <asm/cpufeature.h> +#include <asm/vendorid_list.h> +#include <asm/vendor_extensions/andes.h> #define ALT_SBI_PMU_OVERFLOW(__ovl) \ asm volatile(ALTERNATIVE_2( \ @@ -32,7 +34,7 @@ asm volatile(ALTERNATIVE_2( \ THEAD_VENDOR_ID, ERRATA_THEAD_PMU, \ CONFIG_ERRATA_THEAD_PMU, \ "csrr %0, " __stringify(ANDES_CSR_SCOUNTEROF), \
0, RISCV_ISA_EXT_XANDESPMU, \
ANDES_VENDOR_ID, RISCV_ISA_VENDOR_EXT_XANDESPMU, \@@ -41,7 +43,7 @@ asm volatile(ALTERNATIVE_2( \ asm volatile(ALTERNATIVE( \ "csrc " __stringify(CSR_IP) ", %0\n\t", \ "csrc " __stringify(ANDES_CSR_SLIP) ", %0\n\t", \
0, RISCV_ISA_EXT_XANDESPMU, \
ANDES_VENDOR_ID, RISCV_ISA_VENDOR_EXT_XANDESPMU, \@@ -837,7 +839,7 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pde riscv_cached_mimpid(0) == 0) { riscv_pmu_irq_num = THEAD_C9XX_RV_IRQ_PMU; riscv_pmu_use_irq = true;
- } else if (riscv_isa_extension_available(NULL, XANDESPMU) &&
- } else if (riscv_isa_vendor_extension_available(-1, XANDESPMU) &&
What's the rationale for this not using riscv_has_extension_unlikely()? Happens once in probe so don't bother? I forget if we discussed it when the code was added, but it would save us from the NULL/-1 syntax, neither of which I think is a good interface.
Doesn't look like something that was ever commented on in the series, but I may have missed it. I can change this to use the alternatives.
Yeha, not really a question for you but thinking aloud and wondering if someone would remind me. I really don't like riscv_isa_extension_available() because it doesn't respect config options etc, but ultimately I think the series that Clement is currently working on for Zc* is could be the saviour there, as the callbacks his most recent version has I think could make it much easier to hook in and turn off extensions. Should be helpful for the sort of confusing shit that Eric was complaining about last week on Andy's vector series.
This also wasn't supposed to be -1, it's supposed to be the id of the vendor.
Also, I'd prob drop the "drivers" from $subject.
I'll come back and look at the rest of this Monday, it's a sunny Friday here and I've still got my devicetree patch queue to clear..
- Charlie
Cheers, Conor.
IS_ENABLED(CONFIG_ANDES_CUSTOM_PMU)) { riscv_pmu_irq_num = ANDES_SLI_CAUSE_BASE + ANDES_RV_IRQ_PMOVI; riscv_pmu_use_irq = true;-- 2.44.0
From: Heiko Stuebner heiko@sntech.de
The VCSR CSR contains two elements VXRM[2:1] and VXSAT[0].
Define constants for those to access the elements in a readable way.
Acked-by: Guo Ren guoren@kernel.org Reviewed-by: Conor Dooley conor.dooley@microchip.com Signed-off-by: Heiko Stuebner heiko.stuebner@vrull.eu Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- arch/riscv/include/asm/csr.h | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h index 2468c55933cd..13bc99c995d1 100644 --- a/arch/riscv/include/asm/csr.h +++ b/arch/riscv/include/asm/csr.h @@ -215,6 +215,11 @@ #define SMSTATEEN0_SSTATEEN0_SHIFT 63 #define SMSTATEEN0_SSTATEEN0 (_ULL(1) << SMSTATEEN0_SSTATEEN0_SHIFT)
+/* VCSR flags */ +#define VCSR_VXRM_MASK 3 +#define VCSR_VXRM_SHIFT 1 +#define VCSR_VXSAT_MASK 1 + /* symbolic CSR names: */ #define CSR_CYCLE 0xc00 #define CSR_TIME 0xc01
The VXRM vector csr for xtheadvector has an encoding of 0xa and VXSAT has an encoding of 0x9.
Co-developed-by: Heiko Stuebner heiko@sntech.de Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- arch/riscv/include/asm/csr.h | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h index 13bc99c995d1..e5a35efd56e0 100644 --- a/arch/riscv/include/asm/csr.h +++ b/arch/riscv/include/asm/csr.h @@ -219,6 +219,8 @@ #define VCSR_VXRM_MASK 3 #define VCSR_VXRM_SHIFT 1 #define VCSR_VXSAT_MASK 1 +#define VCSR_VXSAT 0x9 +#define VCSR_VXRM 0xa
/* symbolic CSR names: */ #define CSR_CYCLE 0xc00
xtheadvector uses different encodings than standard vector for vsetvli and vector loads/stores. Write the instruction formats to be used in assembly code.
Co-developed-by: Heiko Stuebner heiko@sntech.de Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- arch/riscv/include/asm/vendor_extensions/thead.h | 26 ++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
diff --git a/arch/riscv/include/asm/vendor_extensions/thead.h b/arch/riscv/include/asm/vendor_extensions/thead.h index 92eec729888d..6e3c85351f82 100644 --- a/arch/riscv/include/asm/vendor_extensions/thead.h +++ b/arch/riscv/include/asm/vendor_extensions/thead.h @@ -16,4 +16,30 @@
extern const struct riscv_isa_vendor_ext_data_list riscv_isa_vendor_ext_list_thead;
+/* Extension specific helpers */ + +/* + * Vector 0.7.1 as used for example on T-Head Xuantie cores, uses an older + * encoding for vsetvli (ta, ma vs. d1), so provide an instruction for + * vsetvli t4, x0, e8, m8, d1 + */ +#define THEAD_VSETVLI_T4X0E8M8D1 ".long 0x00307ed7\n\t" +#define THEAD_VSETVLI_X0X0E8M8D1 ".long 0x00307057\n\t" + +/* + * While in theory, the vector-0.7.1 vsb.v and vlb.v result in the same + * encoding as the standard vse8.v and vle8.v, compilers seem to optimize + * the call resulting in a different encoding and then using a value for + * the "mop" field that is not part of vector-0.7.1 + * So encode specific variants for vstate_save and _restore. + */ +#define THEAD_VSB_V_V0T0 ".long 0x02028027\n\t" +#define THEAD_VSB_V_V8T0 ".long 0x02028427\n\t" +#define THEAD_VSB_V_V16T0 ".long 0x02028827\n\t" +#define THEAD_VSB_V_V24T0 ".long 0x02028c27\n\t" +#define THEAD_VLB_V_V0T0 ".long 0x012028007\n\t" +#define THEAD_VLB_V_V8T0 ".long 0x012028407\n\t" +#define THEAD_VLB_V_V16T0 ".long 0x012028807\n\t" +#define THEAD_VLB_V_V24T0 ".long 0x012028c07\n\t" + #endif
Use alternatives to add support for xtheadvector vector save/restore routines.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- arch/riscv/Kconfig.vendor | 13 ++ arch/riscv/include/asm/csr.h | 6 + arch/riscv/include/asm/switch_to.h | 2 +- arch/riscv/include/asm/vector.h | 247 ++++++++++++++++++++++++++------- arch/riscv/kernel/cpufeature.c | 2 +- arch/riscv/kernel/kernel_mode_vector.c | 8 +- arch/riscv/kernel/process.c | 4 +- arch/riscv/kernel/signal.c | 6 +- arch/riscv/kernel/vector.c | 13 +- 9 files changed, 233 insertions(+), 68 deletions(-)
diff --git a/arch/riscv/Kconfig.vendor b/arch/riscv/Kconfig.vendor index d5555ed696fb..c89a81969de3 100644 --- a/arch/riscv/Kconfig.vendor +++ b/arch/riscv/Kconfig.vendor @@ -8,6 +8,19 @@ config RISCV_ISA_VENDOR_EXT_THEAD
If you don't know what to do here, say Y.
+config RISCV_ISA_XTHEADVECTOR + bool "xtheadvector extension support" + depends on RISCV_ISA_VENDOR_EXT_THEAD + depends on RISCV_ISA_V + depends on FPU + default y + help + Say N here if you want to disable all xtheadvector related procedure + in the kernel. This will disable vector for any T-Head board that + contains xtheadvector rather than the standard vector. + + If you don't know what to do here, say Y. + config RISCV_ISA_VENDOR_EXT_ANDES bool "Andes vendor extension support" default y diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h index e5a35efd56e0..13657d096e7d 100644 --- a/arch/riscv/include/asm/csr.h +++ b/arch/riscv/include/asm/csr.h @@ -30,6 +30,12 @@ #define SR_VS_CLEAN _AC(0x00000400, UL) #define SR_VS_DIRTY _AC(0x00000600, UL)
+#define SR_VS_THEAD _AC(0x01800000, UL) /* xtheadvector Status */ +#define SR_VS_OFF_THEAD _AC(0x00000000, UL) +#define SR_VS_INITIAL_THEAD _AC(0x00800000, UL) +#define SR_VS_CLEAN_THEAD _AC(0x01000000, UL) +#define SR_VS_DIRTY_THEAD _AC(0x01800000, UL) + #define SR_XS _AC(0x00018000, UL) /* Extension Status */ #define SR_XS_OFF _AC(0x00000000, UL) #define SR_XS_INITIAL _AC(0x00008000, UL) diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h index 7efdb0584d47..ada6b5cf2d94 100644 --- a/arch/riscv/include/asm/switch_to.h +++ b/arch/riscv/include/asm/switch_to.h @@ -78,7 +78,7 @@ do { \ struct task_struct *__next = (next); \ if (has_fpu()) \ __switch_to_fpu(__prev, __next); \ - if (has_vector()) \ + if (has_vector() || has_xtheadvector()) \ __switch_to_vector(__prev, __next); \ ((last) = __switch_to(__prev, __next)); \ } while (0) diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index 731dcd0ed4de..db851dc81870 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -18,6 +18,27 @@ #include <asm/cpufeature.h> #include <asm/csr.h> #include <asm/asm.h> +#include <asm/vendorid_list.h> +#include <asm/vendor_extensions.h> +#include <asm/vendor_extensions/thead.h> + +#define __riscv_v_vstate_or(_val, TYPE) ({ \ + typeof(_val) _res = _val; \ + if (has_xtheadvector()) \ + _res = (_res & ~SR_VS_THEAD) | SR_VS_##TYPE##_THEAD; \ + else \ + _res = (_res & ~SR_VS) | SR_VS_##TYPE; \ + _res; \ +}) + +#define __riscv_v_vstate_check(_val, TYPE) ({ \ + bool _res; \ + if (has_xtheadvector()) \ + _res = ((_val) & SR_VS_THEAD) == SR_VS_##TYPE##_THEAD; \ + else \ + _res = ((_val) & SR_VS) == SR_VS_##TYPE; \ + _res; \ +})
extern unsigned long riscv_v_vsize; int riscv_v_setup_vsize(void); @@ -40,39 +61,62 @@ static __always_inline bool has_vector(void) return riscv_has_extension_unlikely(RISCV_ISA_EXT_v); }
+static __always_inline bool has_xtheadvector_no_alternatives(void) +{ + if (IS_ENABLED(CONFIG_RISCV_ISA_XTHEADVECTOR)) + return riscv_isa_vendor_extension_available(THEAD_VENDOR_ID, XTHEADVECTOR); + else + return false; +} + +static __always_inline bool has_xtheadvector(void) +{ + if (IS_ENABLED(CONFIG_RISCV_ISA_XTHEADVECTOR)) + return riscv_has_vendor_extension_unlikely(THEAD_VENDOR_ID, + RISCV_ISA_VENDOR_EXT_XTHEADVECTOR); + else + return false; +} + static inline void __riscv_v_vstate_clean(struct pt_regs *regs) { - regs->status = (regs->status & ~SR_VS) | SR_VS_CLEAN; + regs->status = __riscv_v_vstate_or(regs->status, CLEAN); }
static inline void __riscv_v_vstate_dirty(struct pt_regs *regs) { - regs->status = (regs->status & ~SR_VS) | SR_VS_DIRTY; + regs->status = __riscv_v_vstate_or(regs->status, DIRTY); }
static inline void riscv_v_vstate_off(struct pt_regs *regs) { - regs->status = (regs->status & ~SR_VS) | SR_VS_OFF; + regs->status = __riscv_v_vstate_or(regs->status, OFF); }
static inline void riscv_v_vstate_on(struct pt_regs *regs) { - regs->status = (regs->status & ~SR_VS) | SR_VS_INITIAL; + regs->status = __riscv_v_vstate_or(regs->status, INITIAL); }
static inline bool riscv_v_vstate_query(struct pt_regs *regs) { - return (regs->status & SR_VS) != 0; + return !__riscv_v_vstate_check(regs->status, OFF); }
static __always_inline void riscv_v_enable(void) { - csr_set(CSR_SSTATUS, SR_VS); + if (has_xtheadvector()) + csr_set(CSR_SSTATUS, SR_VS_THEAD); + else + csr_set(CSR_SSTATUS, SR_VS); }
static __always_inline void riscv_v_disable(void) { - csr_clear(CSR_SSTATUS, SR_VS); + if (has_xtheadvector()) + csr_clear(CSR_SSTATUS, SR_VS_THEAD); + else + csr_clear(CSR_SSTATUS, SR_VS); }
static __always_inline void __vstate_csr_save(struct __riscv_v_ext_state *dest) @@ -81,10 +125,47 @@ static __always_inline void __vstate_csr_save(struct __riscv_v_ext_state *dest) "csrr %0, " __stringify(CSR_VSTART) "\n\t" "csrr %1, " __stringify(CSR_VTYPE) "\n\t" "csrr %2, " __stringify(CSR_VL) "\n\t" - "csrr %3, " __stringify(CSR_VCSR) "\n\t" - "csrr %4, " __stringify(CSR_VLENB) "\n\t" : "=r" (dest->vstart), "=r" (dest->vtype), "=r" (dest->vl), - "=r" (dest->vcsr), "=r" (dest->vlenb) : :); + "=r" (dest->vcsr) : :); + + if (has_xtheadvector()) { + u32 tmp_vcsr; + bool restore_fpu = false; + unsigned long status = csr_read(CSR_SSTATUS); + + /* + * CSR_VCSR is defined as + * [2:1] - vxrm[1:0] + * [0] - vxsat + * The earlier vector spec implemented by T-Head uses separate + * registers for the same bit-elements, so just combine those + * into the existing output field. + * + * Additionally T-Head cores need FS to be enabled when accessing + * the VXRM and VXSAT CSRs, otherwise ending in illegal instructions. + * Though the cores do not implement the VXRM and VXSAT fields in the + * FCSR CSR that vector-0.7.1 specifies. + */ + if ((status & SR_FS) == SR_FS_OFF) { + csr_set(CSR_SSTATUS, (status & ~SR_FS) | SR_FS_CLEAN); + restore_fpu = true; + } + + asm volatile ( + "csrr %[tmp_vcsr], " __stringify(VCSR_VXRM) "\n\t" + "slliw %[vcsr], %[tmp_vcsr], " __stringify(VCSR_VXRM_SHIFT) "\n\t" + "csrr %[tmp_vcsr], " __stringify(VCSR_VXSAT) "\n\t" + "or %[vcsr], %[vcsr], %[tmp_vcsr]\n\t" + : [vcsr] "=r" (dest->vcsr), [tmp_vcsr] "=&r" (tmp_vcsr)); + + if (restore_fpu) + csr_set(CSR_SSTATUS, status); + } else { + asm volatile ( + "csrr %[vcsr], " __stringify(CSR_VCSR) "\n\t" + "csrr %[vlenb], " __stringify(CSR_VLENB) "\n\t" + : [vcsr] "=r" (dest->vcsr), [vlenb] "=r" (dest->vlenb)); + } }
static __always_inline void __vstate_csr_restore(struct __riscv_v_ext_state *src) @@ -95,9 +176,37 @@ static __always_inline void __vstate_csr_restore(struct __riscv_v_ext_state *src "vsetvl x0, %2, %1\n\t" ".option pop\n\t" "csrw " __stringify(CSR_VSTART) ", %0\n\t" - "csrw " __stringify(CSR_VCSR) ", %3\n\t" - : : "r" (src->vstart), "r" (src->vtype), "r" (src->vl), - "r" (src->vcsr) :); + : : "r" (src->vstart), "r" (src->vtype), "r" (src->vl)); + + if (has_xtheadvector()) { + u32 tmp_vcsr; + bool restore_fpu = false; + unsigned long status = csr_read(CSR_SSTATUS); + + /* + * Similar to __vstate_csr_save above, restore values for the + * separate VXRM and VXSAT CSRs from the vcsr variable. + */ + if ((status & SR_FS) == SR_FS_OFF) { + csr_set(CSR_SSTATUS, (status & ~SR_FS) | SR_FS_CLEAN); + restore_fpu = true; + } + + asm volatile ( + "srliw %[tmp_vcsr], %[vcsr], " __stringify(VCSR_VXRM_SHIFT) "\n\t" + "andi %[tmp_vcsr], %[tmp_vcsr], " __stringify(VCSR_VXRM_MASK) "\n\t" + "csrw " __stringify(VCSR_VXRM) ", %[tmp_vcsr]\n\t" + "andi %[tmp_vcsr], %[vcsr], " __stringify(VCSR_VXSAT_MASK) "\n\t" + "csrw " __stringify(VCSR_VXSAT) ", %[tmp_vcsr]\n\t" + : [tmp_vcsr] "=&r" (tmp_vcsr) : [vcsr] "r" (src->vcsr)); + + if (restore_fpu) + csr_set(CSR_SSTATUS, status); + } else { + asm volatile ( + "csrw " __stringify(CSR_VCSR) ", %[vcsr]\n\t" + : : [vcsr] "r" (src->vcsr)); + } }
static inline void __riscv_v_vstate_save(struct __riscv_v_ext_state *save_to, @@ -107,19 +216,33 @@ static inline void __riscv_v_vstate_save(struct __riscv_v_ext_state *save_to,
riscv_v_enable(); __vstate_csr_save(save_to); - asm volatile ( - ".option push\n\t" - ".option arch, +v\n\t" - "vsetvli %0, x0, e8, m8, ta, ma\n\t" - "vse8.v v0, (%1)\n\t" - "add %1, %1, %0\n\t" - "vse8.v v8, (%1)\n\t" - "add %1, %1, %0\n\t" - "vse8.v v16, (%1)\n\t" - "add %1, %1, %0\n\t" - "vse8.v v24, (%1)\n\t" - ".option pop\n\t" - : "=&r" (vl) : "r" (datap) : "memory"); + if (has_xtheadvector()) { + asm volatile ( + "mv t0, %0\n\t" + THEAD_VSETVLI_T4X0E8M8D1 + THEAD_VSB_V_V0T0 + "add t0, t0, t4\n\t" + THEAD_VSB_V_V0T0 + "add t0, t0, t4\n\t" + THEAD_VSB_V_V0T0 + "add t0, t0, t4\n\t" + THEAD_VSB_V_V0T0 + : : "r" (datap) : "memory", "t0", "t4"); + } else { + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvli %0, x0, e8, m8, ta, ma\n\t" + "vse8.v v0, (%1)\n\t" + "add %1, %1, %0\n\t" + "vse8.v v8, (%1)\n\t" + "add %1, %1, %0\n\t" + "vse8.v v16, (%1)\n\t" + "add %1, %1, %0\n\t" + "vse8.v v24, (%1)\n\t" + ".option pop\n\t" + : "=&r" (vl) : "r" (datap) : "memory"); + } riscv_v_disable(); }
@@ -129,55 +252,77 @@ static inline void __riscv_v_vstate_restore(struct __riscv_v_ext_state *restore_ unsigned long vl;
riscv_v_enable(); - asm volatile ( - ".option push\n\t" - ".option arch, +v\n\t" - "vsetvli %0, x0, e8, m8, ta, ma\n\t" - "vle8.v v0, (%1)\n\t" - "add %1, %1, %0\n\t" - "vle8.v v8, (%1)\n\t" - "add %1, %1, %0\n\t" - "vle8.v v16, (%1)\n\t" - "add %1, %1, %0\n\t" - "vle8.v v24, (%1)\n\t" - ".option pop\n\t" - : "=&r" (vl) : "r" (datap) : "memory"); + if (has_xtheadvector()) { + asm volatile ( + "mv t0, %0\n\t" + THEAD_VSETVLI_T4X0E8M8D1 + THEAD_VLB_V_V0T0 + "add t0, t0, t4\n\t" + THEAD_VLB_V_V0T0 + "add t0, t0, t4\n\t" + THEAD_VLB_V_V0T0 + "add t0, t0, t4\n\t" + THEAD_VLB_V_V0T0 + : : "r" (datap) : "memory", "t0", "t4"); + } else { + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvli %0, x0, e8, m8, ta, ma\n\t" + "vle8.v v0, (%1)\n\t" + "add %1, %1, %0\n\t" + "vle8.v v8, (%1)\n\t" + "add %1, %1, %0\n\t" + "vle8.v v16, (%1)\n\t" + "add %1, %1, %0\n\t" + "vle8.v v24, (%1)\n\t" + ".option pop\n\t" + : "=&r" (vl) : "r" (datap) : "memory"); + } __vstate_csr_restore(restore_from); riscv_v_disable(); }
static inline void __riscv_v_vstate_discard(void) { - unsigned long vl, vtype_inval = 1UL << (BITS_PER_LONG - 1); + unsigned long vtype_inval = 1UL << (BITS_PER_LONG - 1);
riscv_v_enable(); + if (has_xtheadvector()) + asm volatile (THEAD_VSETVLI_X0X0E8M8D1); + else + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvli x0, x0, e8, m8, ta, ma\n\t" + ".option pop\n\t"); + asm volatile ( ".option push\n\t" ".option arch, +v\n\t" - "vsetvli %0, x0, e8, m8, ta, ma\n\t" "vmv.v.i v0, -1\n\t" "vmv.v.i v8, -1\n\t" "vmv.v.i v16, -1\n\t" "vmv.v.i v24, -1\n\t" - "vsetvl %0, x0, %1\n\t" + "vsetvl x0, x0, %0\n\t" ".option pop\n\t" - : "=&r" (vl) : "r" (vtype_inval) : "memory"); + : : "r" (vtype_inval)); + riscv_v_disable(); }
static inline void riscv_v_vstate_discard(struct pt_regs *regs) { - if ((regs->status & SR_VS) == SR_VS_OFF) - return; - - __riscv_v_vstate_discard(); - __riscv_v_vstate_dirty(regs); + if (riscv_v_vstate_query(regs)) { + __riscv_v_vstate_discard(); + __riscv_v_vstate_dirty(regs); + } }
static inline void riscv_v_vstate_save(struct __riscv_v_ext_state *vstate, struct pt_regs *regs) { - if ((regs->status & SR_VS) == SR_VS_DIRTY) { + if (__riscv_v_vstate_check(regs->status, DIRTY)) { __riscv_v_vstate_save(vstate, vstate->datap); __riscv_v_vstate_clean(regs); } @@ -186,7 +331,7 @@ static inline void riscv_v_vstate_save(struct __riscv_v_ext_state *vstate, static inline void riscv_v_vstate_restore(struct __riscv_v_ext_state *vstate, struct pt_regs *regs) { - if ((regs->status & SR_VS) != SR_VS_OFF) { + if (riscv_v_vstate_query(regs)) { __riscv_v_vstate_restore(vstate, vstate->datap); __riscv_v_vstate_clean(regs); } @@ -195,7 +340,7 @@ static inline void riscv_v_vstate_restore(struct __riscv_v_ext_state *vstate, static inline void riscv_v_vstate_set_restore(struct task_struct *task, struct pt_regs *regs) { - if ((regs->status & SR_VS) != SR_VS_OFF) { + if (riscv_v_vstate_query(regs)) { set_tsk_thread_flag(task, TIF_RISCV_V_DEFER_RESTORE); riscv_v_vstate_on(regs); } diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 0cef08a3d891..7b5aee50adfb 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -750,7 +750,7 @@ void __init riscv_fill_hwcap(void) elf_hwcap &= ~COMPAT_HWCAP_ISA_F; }
- if (elf_hwcap & COMPAT_HWCAP_ISA_V) { + if (elf_hwcap & COMPAT_HWCAP_ISA_V || has_xtheadvector_no_alternatives()) { riscv_v_setup_vsize(); /* * ISA string in device tree might have 'v' flag, but diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c index 6afe80c7f03a..99972a48e86b 100644 --- a/arch/riscv/kernel/kernel_mode_vector.c +++ b/arch/riscv/kernel/kernel_mode_vector.c @@ -143,7 +143,7 @@ static int riscv_v_start_kernel_context(bool *is_nested)
/* Transfer the ownership of V from user to kernel, then save */ riscv_v_start(RISCV_PREEMPT_V | RISCV_PREEMPT_V_DIRTY); - if ((task_pt_regs(current)->status & SR_VS) == SR_VS_DIRTY) { + if (__riscv_v_vstate_check(task_pt_regs(current)->status, DIRTY)) { uvstate = ¤t->thread.vstate; __riscv_v_vstate_save(uvstate, uvstate->datap); } @@ -160,7 +160,7 @@ asmlinkage void riscv_v_context_nesting_start(struct pt_regs *regs) return;
depth = riscv_v_ctx_get_depth(); - if (depth == 0 && (regs->status & SR_VS) == SR_VS_DIRTY) + if (depth == 0 && __riscv_v_vstate_check(regs->status, DIRTY)) riscv_preempt_v_set_dirty();
riscv_v_ctx_depth_inc(); @@ -208,7 +208,7 @@ void kernel_vector_begin(void) { bool nested = false;
- if (WARN_ON(!has_vector())) + if (WARN_ON(!(has_vector() || has_xtheadvector()))) return;
BUG_ON(!may_use_simd()); @@ -236,7 +236,7 @@ EXPORT_SYMBOL_GPL(kernel_vector_begin); */ void kernel_vector_end(void) { - if (WARN_ON(!has_vector())) + if (WARN_ON(!(has_vector() || has_xtheadvector()))) return;
riscv_v_disable(); diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c index 92922dbd5b5c..eabca86fc3c0 100644 --- a/arch/riscv/kernel/process.c +++ b/arch/riscv/kernel/process.c @@ -178,7 +178,7 @@ void flush_thread(void) void arch_release_task_struct(struct task_struct *tsk) { /* Free the vector context of datap. */ - if (has_vector()) + if (has_vector() || has_xtheadvector()) riscv_v_thread_free(tsk); }
@@ -225,7 +225,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) p->thread.s[0] = 0; } p->thread.riscv_v_flags = 0; - if (has_vector()) + if (has_vector() || has_xtheadvector()) riscv_v_thread_alloc(p); p->thread.ra = (unsigned long)ret_from_fork; p->thread.sp = (unsigned long)childregs; /* kernel sp */ diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c index 501e66debf69..5d3ba8e46807 100644 --- a/arch/riscv/kernel/signal.c +++ b/arch/riscv/kernel/signal.c @@ -188,7 +188,7 @@ static long restore_sigcontext(struct pt_regs *regs,
return 0; case RISCV_V_MAGIC: - if (!has_vector() || !riscv_v_vstate_query(regs) || + if (!(has_vector() || has_xtheadvector()) || !riscv_v_vstate_query(regs) || size != riscv_v_sc_size) return -EINVAL;
@@ -210,7 +210,7 @@ static size_t get_rt_frame_size(bool cal_all)
frame_size = sizeof(*frame);
- if (has_vector()) { + if (has_vector() || has_xtheadvector()) { if (cal_all || riscv_v_vstate_query(task_pt_regs(current))) total_context_size += riscv_v_sc_size; } @@ -283,7 +283,7 @@ static long setup_sigcontext(struct rt_sigframe __user *frame, if (has_fpu()) err |= save_fp_state(regs, &sc->sc_fpregs); /* Save the vector state. */ - if (has_vector() && riscv_v_vstate_query(regs)) + if ((has_vector() || has_xtheadvector()) && riscv_v_vstate_query(regs)) err |= save_v_state(regs, (void __user **)&sc_ext_ptr); /* Write zero to fp-reserved space and check it on restore_sigcontext */ err |= __put_user(0, &sc->sc_extdesc.reserved); diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c index fb7f3ca80d9e..b2ec92e13ae4 100644 --- a/arch/riscv/kernel/vector.c +++ b/arch/riscv/kernel/vector.c @@ -58,7 +58,7 @@ int riscv_v_setup_vsize(void)
void __init riscv_v_setup_ctx_cache(void) { - if (!has_vector()) + if (!(has_vector() || has_xtheadvector())) return;
riscv_v_user_cachep = kmem_cache_create_usercopy("riscv_vector_ctx", @@ -179,7 +179,8 @@ bool riscv_v_first_use_handler(struct pt_regs *regs) u32 insn = (u32)regs->badaddr;
/* Do not handle if V is not supported, or disabled */ - if (!(ELF_HWCAP & COMPAT_HWCAP_ISA_V)) + if (!(ELF_HWCAP & COMPAT_HWCAP_ISA_V) && + !(has_xtheadvector() && riscv_v_vstate_ctrl_user_allowed())) return false;
/* If V has been enabled then it is not the first-use trap */ @@ -218,7 +219,7 @@ void riscv_v_vstate_ctrl_init(struct task_struct *tsk) bool inherit; int cur, next;
- if (!has_vector()) + if (!(has_vector() || has_xtheadvector())) return;
next = riscv_v_ctrl_get_next(tsk); @@ -240,7 +241,7 @@ void riscv_v_vstate_ctrl_init(struct task_struct *tsk)
long riscv_v_vstate_ctrl_get_current(void) { - if (!has_vector()) + if (!(has_vector() || has_xtheadvector())) return -EINVAL;
return current->thread.vstate_ctrl & PR_RISCV_V_VSTATE_CTRL_MASK; @@ -251,7 +252,7 @@ long riscv_v_vstate_ctrl_set_current(unsigned long arg) bool inherit; int cur, next;
- if (!has_vector()) + if (!(has_vector() || has_xtheadvector())) return -EINVAL;
if (arg & ~PR_RISCV_V_VSTATE_CTRL_MASK) @@ -301,7 +302,7 @@ static struct ctl_table riscv_v_default_vstate_table[] = {
static int __init riscv_v_sysctl_init(void) { - if (has_vector()) + if (has_vector() || has_xtheadvector()) if (!register_sysctl("abi", riscv_v_default_vstate_table)) return -EINVAL; return 0;
Add a new hwprobe key "RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0" which allows userspace to probe for the new RISCV_ISA_VENDOR_EXT_XTHEADVECTOR vendor extension.
This new key will allow userspace code to probe for which thead vendor extensions are supported. This API is modeled to be consistent with RISCV_HWPROBE_KEY_IMA_EXT_0. The bitmask returned will have each bit corresponding to a supported thead vendor extension of the cpumask set. Just like RISCV_HWPROBE_KEY_IMA_EXT_0, this allows a userspace program to determine all of the supported thead vendor extensions in one call.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- arch/riscv/include/asm/hwprobe.h | 4 +-- .../include/asm/vendor_extensions/thead_hwprobe.h | 11 ++++++ arch/riscv/include/uapi/asm/hwprobe.h | 3 +- arch/riscv/include/uapi/asm/vendor/thead.h | 3 ++ arch/riscv/kernel/sys_hwprobe.c | 9 +++++ arch/riscv/kernel/vendor_extensions/Makefile | 1 + .../riscv/kernel/vendor_extensions/thead_hwprobe.c | 42 ++++++++++++++++++++++ 7 files changed, 70 insertions(+), 3 deletions(-)
diff --git a/arch/riscv/include/asm/hwprobe.h b/arch/riscv/include/asm/hwprobe.h index 630507dff5ea..e68496b4f8de 100644 --- a/arch/riscv/include/asm/hwprobe.h +++ b/arch/riscv/include/asm/hwprobe.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ /* - * Copyright 2023 Rivos, Inc + * Copyright 2023-2024 Rivos, Inc */
#ifndef _ASM_HWPROBE_H @@ -8,7 +8,7 @@
#include <uapi/asm/hwprobe.h>
-#define RISCV_HWPROBE_MAX_KEY 6 +#define RISCV_HWPROBE_MAX_KEY 7
static inline bool riscv_hwprobe_key_is_valid(__s64 key) { diff --git a/arch/riscv/include/asm/vendor_extensions/thead_hwprobe.h b/arch/riscv/include/asm/vendor_extensions/thead_hwprobe.h new file mode 100644 index 000000000000..907cfc4eb4dc --- /dev/null +++ b/arch/riscv/include/asm/vendor_extensions/thead_hwprobe.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_RISCV_VENDOR_EXTENSIONS_THEAD_HWPROBE_H +#define _ASM_RISCV_VENDOR_EXTENSIONS_THEAD_HWPROBE_H + +#include <linux/cpumask.h> + +#include <uapi/asm/hwprobe.h> + +void hwprobe_isa_vendor_ext_thead_0(struct riscv_hwprobe *pair, const struct cpumask *cpus); + +#endif diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uapi/asm/hwprobe.h index 9f2a8e3ff204..21e96a63f9ea 100644 --- a/arch/riscv/include/uapi/asm/hwprobe.h +++ b/arch/riscv/include/uapi/asm/hwprobe.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ /* - * Copyright 2023 Rivos, Inc + * Copyright 2023-2024 Rivos, Inc */
#ifndef _UAPI_ASM_HWPROBE_H @@ -67,6 +67,7 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_MISALIGNED_UNSUPPORTED (4 << 0) #define RISCV_HWPROBE_MISALIGNED_MASK (7 << 0) #define RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE 6 +#define RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0 7 /* Increase RISCV_HWPROBE_MAX_KEY when adding items. */
/* Flags */ diff --git a/arch/riscv/include/uapi/asm/vendor/thead.h b/arch/riscv/include/uapi/asm/vendor/thead.h new file mode 100644 index 000000000000..43790ebe5faf --- /dev/null +++ b/arch/riscv/include/uapi/asm/vendor/thead.h @@ -0,0 +1,3 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ + +#define RISCV_HWPROBE_VENDOR_EXT_XTHEADVECTOR (1 << 0) diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c index 8cae41a502dd..e59cac545df5 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -13,6 +13,7 @@ #include <asm/uaccess.h> #include <asm/unistd.h> #include <asm/vector.h> +#include <asm/vendor_extensions/thead_hwprobe.h> #include <vdso/vsyscall.h>
@@ -216,6 +217,14 @@ static void hwprobe_one_pair(struct riscv_hwprobe *pair, pair->value = riscv_cboz_block_size; break;
+ case RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0: +#ifdef CONFIG_RISCV_ISA_VENDOR_EXT_THEAD + hwprobe_isa_vendor_ext_thead_0(pair, cpus); +#else + pair->value = 0; +#endif + break; + /* * For forward compatibility, unknown keys don't fail the whole * call, but get their element key set to -1 and value set to 0 diff --git a/arch/riscv/kernel/vendor_extensions/Makefile b/arch/riscv/kernel/vendor_extensions/Makefile index 8f1c5a4dc38f..f511fd269e8a 100644 --- a/arch/riscv/kernel/vendor_extensions/Makefile +++ b/arch/riscv/kernel/vendor_extensions/Makefile @@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only
obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_THEAD) += thead.o +obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_THEAD) += thead_hwprobe.o obj-$(CONFIG_RISCV_ISA_VENDOR_EXT_ANDES) += andes.o diff --git a/arch/riscv/kernel/vendor_extensions/thead_hwprobe.c b/arch/riscv/kernel/vendor_extensions/thead_hwprobe.c new file mode 100644 index 000000000000..e8e2de292032 --- /dev/null +++ b/arch/riscv/kernel/vendor_extensions/thead_hwprobe.c @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include <asm/vector.h> +#include <asm/vendor_extensions/thead.h> +#include <asm/vendor_extensions/thead_hwprobe.h> + +#include <linux/cpumask.h> +#include <linux/types.h> + +#include <uapi/asm/hwprobe.h> +#include <uapi/asm/vendor/thead.h> + +void hwprobe_isa_vendor_ext_thead_0(struct riscv_hwprobe *pair, const struct cpumask *cpus) +{ + /* + * Loop through and record extensions that 1) anyone has, and 2) anyone + * doesn't have. + */ + + struct riscv_isainfo *per_hart_thead_bitmap = riscv_isa_vendor_ext_list_thead.per_hart_vendor_bitmap; + int cpu; + u64 missing; + + for_each_cpu(cpu, cpus) { + struct riscv_isainfo *isainfo = &per_hart_thead_bitmap[cpu]; + +#define EXT_KEY(ext) \ + do { \ + if (__riscv_isa_extension_available(isainfo->isa, RISCV_ISA_VENDOR_EXT_##ext)) \ + pair->value |= RISCV_HWPROBE_VENDOR_EXT_##ext; \ + else \ + missing |= RISCV_HWPROBE_VENDOR_EXT_##ext; \ + } while (false) + + EXT_KEY(XTHEADVECTOR); + +#undef EXT_KEY + } + + /* Now turn off reporting features if any CPU is missing it. */ + pair->value &= ~missing; +}
Document support for thead vendor extensions using the key RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0 and xtheadvector extension using the key RISCV_HWPROBE_VENDOR_EXT_XTHEADVECTOR.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- Documentation/arch/riscv/hwprobe.rst | 10 ++++++++++ 1 file changed, 10 insertions(+)
diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/riscv/hwprobe.rst index b2bcc9eed9aa..b2bb305140aa 100644 --- a/Documentation/arch/riscv/hwprobe.rst +++ b/Documentation/arch/riscv/hwprobe.rst @@ -210,3 +210,13 @@ The following keys are defined:
* :c:macro:`RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE`: An unsigned int which represents the size of the Zicboz block in bytes. + +* :c:macro:`RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0`: A bitmask containing the + thead vendor extensions that are compatible with the + :c:macro:`RISCV_HWPROBE_BASE_BEHAVIOR_IMA`: base system behavior. + + * T-HEAD + + * :c:macro:`RISCV_HWPROBE_VENDOR_EXT_XTHEADVECTOR`: The xtheadvector vendor + extension is supported in the T-Head ISA extensions spec starting from + commit a18c801634 ("Add T-Head VECTOR vendor extension. ").
Overhaul the riscv vector tests to use kselftest_harness to help the test cases correctly report the results and decouple the individual test cases from each other. With this refactoring, only run the test cases is vector is reported and properly report the test case as skipped otherwise. The v_initval_nolibc test was previously not checking if vector was supported and used a function (malloc) which invalidates the state of the vector registers.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- tools/testing/selftests/riscv/vector/.gitignore | 3 +- tools/testing/selftests/riscv/vector/Makefile | 17 +- .../selftests/riscv/vector/v_exec_initval_nolibc.c | 84 +++++++ tools/testing/selftests/riscv/vector/v_helpers.c | 56 +++++ tools/testing/selftests/riscv/vector/v_helpers.h | 5 + tools/testing/selftests/riscv/vector/v_initval.c | 16 ++ .../selftests/riscv/vector/v_initval_nolibc.c | 68 ------ .../testing/selftests/riscv/vector/vstate_prctl.c | 266 ++++++++++++--------- 8 files changed, 324 insertions(+), 191 deletions(-)
diff --git a/tools/testing/selftests/riscv/vector/.gitignore b/tools/testing/selftests/riscv/vector/.gitignore index 9ae7964491d5..7d9c87cd0649 100644 --- a/tools/testing/selftests/riscv/vector/.gitignore +++ b/tools/testing/selftests/riscv/vector/.gitignore @@ -1,3 +1,4 @@ vstate_exec_nolibc vstate_prctl -v_initval_nolibc +v_initval +v_exec_initval_nolibc diff --git a/tools/testing/selftests/riscv/vector/Makefile b/tools/testing/selftests/riscv/vector/Makefile index bfff0ff4f3be..995746359477 100644 --- a/tools/testing/selftests/riscv/vector/Makefile +++ b/tools/testing/selftests/riscv/vector/Makefile @@ -2,18 +2,27 @@ # Copyright (C) 2021 ARM Limited # Originally tools/testing/arm64/abi/Makefile
-TEST_GEN_PROGS := vstate_prctl v_initval_nolibc -TEST_GEN_PROGS_EXTENDED := vstate_exec_nolibc +TEST_GEN_PROGS := v_initval vstate_prctl +TEST_GEN_PROGS_EXTENDED := vstate_exec_nolibc v_exec_initval_nolibc sys_hwprobe.o v_helpers.o
include ../../lib.mk
-$(OUTPUT)/vstate_prctl: vstate_prctl.c ../hwprobe/sys_hwprobe.S +$(OUTPUT)/sys_hwprobe.o: ../hwprobe/sys_hwprobe.S + $(CC) -static -c -o$@ $(CFLAGS) $^ + +$(OUTPUT)/v_helpers.o: v_helpers.c + $(CC) -static -c -o$@ $(CFLAGS) $^ + +$(OUTPUT)/vstate_prctl: vstate_prctl.c $(OUTPUT)/sys_hwprobe.o $(OUTPUT)/v_helpers.o $(CC) -static -o$@ $(CFLAGS) $(LDFLAGS) $^
$(OUTPUT)/vstate_exec_nolibc: vstate_exec_nolibc.c $(CC) -nostdlib -static -include ../../../../include/nolibc/nolibc.h \ -Wall $(CFLAGS) $(LDFLAGS) $^ -o $@ -lgcc
-$(OUTPUT)/v_initval_nolibc: v_initval_nolibc.c +$(OUTPUT)/v_initval: v_initval.c $(OUTPUT)/sys_hwprobe.o $(OUTPUT)/v_helpers.o + $(CC) -static -o$@ $(CFLAGS) $(LDFLAGS) $^ + +$(OUTPUT)/v_exec_initval_nolibc: v_exec_initval_nolibc.c $(CC) -nostdlib -static -include ../../../../include/nolibc/nolibc.h \ -Wall $(CFLAGS) $(LDFLAGS) $^ -o $@ -lgcc diff --git a/tools/testing/selftests/riscv/vector/v_exec_initval_nolibc.c b/tools/testing/selftests/riscv/vector/v_exec_initval_nolibc.c new file mode 100644 index 000000000000..74b13806baf0 --- /dev/null +++ b/tools/testing/selftests/riscv/vector/v_exec_initval_nolibc.c @@ -0,0 +1,84 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Get values of vector registers as soon as the program starts to test if + * is properly cleaning the values before starting a new program. Vector + * registers are caller saved, so no function calls may happen before reading + * the values. To further ensure consistency, this file is compiled without + * libc and without auto-vectorization. + * + * To be "clean" all values must be either all ones or all zeroes. + */ + +#define __stringify_1(x...) #x +#define __stringify(x...) __stringify_1(x) + +int main(int argc, char **argv) +{ + char prev_value = 0, value; + unsigned long vl; + int first = 1; + + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvli %[vl], x0, e8, m1, ta, ma\n\t" + ".option pop\n\t" + : [vl] "=r" (vl) + ); + +#define CHECK_VECTOR_REGISTER(register) ({ \ + for (int i = 0; i < vl; i++) { \ + asm volatile ( \ + ".option push\n\t" \ + ".option arch, +v\n\t" \ + "vmv.x.s %0, " __stringify(register) "\n\t" \ + "vsrl.vi " __stringify(register) ", " __stringify(register) ", 8\n\t" \ + ".option pop\n\t" \ + : "=r" (value)); \ + if (first) { \ + first = 0; \ + } else if (value != prev_value || !(value == 0x00 || value == 0xff)) { \ + printf("Register " __stringify(register) " values not clean! value: %u\n", value); \ + exit(-1); \ + } \ + prev_value = value; \ + } \ +}) + + CHECK_VECTOR_REGISTER(v0); + CHECK_VECTOR_REGISTER(v1); + CHECK_VECTOR_REGISTER(v2); + CHECK_VECTOR_REGISTER(v3); + CHECK_VECTOR_REGISTER(v4); + CHECK_VECTOR_REGISTER(v5); + CHECK_VECTOR_REGISTER(v6); + CHECK_VECTOR_REGISTER(v7); + CHECK_VECTOR_REGISTER(v8); + CHECK_VECTOR_REGISTER(v9); + CHECK_VECTOR_REGISTER(v10); + CHECK_VECTOR_REGISTER(v11); + CHECK_VECTOR_REGISTER(v12); + CHECK_VECTOR_REGISTER(v13); + CHECK_VECTOR_REGISTER(v14); + CHECK_VECTOR_REGISTER(v15); + CHECK_VECTOR_REGISTER(v16); + CHECK_VECTOR_REGISTER(v17); + CHECK_VECTOR_REGISTER(v18); + CHECK_VECTOR_REGISTER(v19); + CHECK_VECTOR_REGISTER(v20); + CHECK_VECTOR_REGISTER(v21); + CHECK_VECTOR_REGISTER(v22); + CHECK_VECTOR_REGISTER(v23); + CHECK_VECTOR_REGISTER(v24); + CHECK_VECTOR_REGISTER(v25); + CHECK_VECTOR_REGISTER(v26); + CHECK_VECTOR_REGISTER(v27); + CHECK_VECTOR_REGISTER(v28); + CHECK_VECTOR_REGISTER(v29); + CHECK_VECTOR_REGISTER(v30); + CHECK_VECTOR_REGISTER(v31); + +#undef CHECK_VECTOR_REGISTER + + return 0; +} diff --git a/tools/testing/selftests/riscv/vector/v_helpers.c b/tools/testing/selftests/riscv/vector/v_helpers.c new file mode 100644 index 000000000000..15c22318db72 --- /dev/null +++ b/tools/testing/selftests/riscv/vector/v_helpers.c @@ -0,0 +1,56 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include "../hwprobe/hwprobe.h" +#include <stdlib.h> +#include <stdio.h> +#include <unistd.h> +#include <sys/wait.h> + +int is_vector_supported(void) +{ + struct riscv_hwprobe pair; + + pair.key = RISCV_HWPROBE_KEY_IMA_EXT_0; + riscv_hwprobe(&pair, 1, 0, NULL, 0); + return pair.value & RISCV_HWPROBE_IMA_V; +} + +int launch_test(char *next_program, int test_inherit) +{ + char *exec_argv[3], *exec_envp[1]; + int rc, pid, status; + + pid = fork(); + if (pid < 0) { + printf("fork failed %d", pid); + return -1; + } + + if (!pid) { + exec_argv[0] = next_program; + exec_argv[1] = test_inherit != 0 ? "x" : NULL; + exec_argv[2] = NULL; + exec_envp[0] = NULL; + /* launch the program again to check inherit */ + rc = execve(next_program, exec_argv, exec_envp); + if (rc) { + perror("execve"); + printf("child execve failed %d\n", rc); + exit(-1); + } + } + + rc = waitpid(-1, &status, 0); + if (rc < 0) { + printf("waitpid failed\n"); + return -3; + } + + if ((WIFEXITED(status) && WEXITSTATUS(status) == -1) || + WIFSIGNALED(status)) { + printf("child exited abnormally\n"); + return -4; + } + + return WEXITSTATUS(status); +} diff --git a/tools/testing/selftests/riscv/vector/v_helpers.h b/tools/testing/selftests/riscv/vector/v_helpers.h new file mode 100644 index 000000000000..88719c4be496 --- /dev/null +++ b/tools/testing/selftests/riscv/vector/v_helpers.h @@ -0,0 +1,5 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +int is_vector_supported(void); + +int launch_test(char *next_program, int test_inherit); diff --git a/tools/testing/selftests/riscv/vector/v_initval.c b/tools/testing/selftests/riscv/vector/v_initval.c new file mode 100644 index 000000000000..f38b5797fa31 --- /dev/null +++ b/tools/testing/selftests/riscv/vector/v_initval.c @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include "../../kselftest_harness.h" +#include "v_helpers.h" + +#define NEXT_PROGRAM "./v_exec_initval_nolibc" + +TEST(v_initval) +{ + if (!is_vector_supported()) + SKIP(return, "Vector not supported"); + + ASSERT_EQ(0, launch_test(NEXT_PROGRAM, 0)); +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/riscv/vector/v_initval_nolibc.c b/tools/testing/selftests/riscv/vector/v_initval_nolibc.c deleted file mode 100644 index 1dd94197da30..000000000000 --- a/tools/testing/selftests/riscv/vector/v_initval_nolibc.c +++ /dev/null @@ -1,68 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only - -#include "../../kselftest.h" -#define MAX_VSIZE (8192 * 32) - -void dump(char *ptr, int size) -{ - int i = 0; - - for (i = 0; i < size; i++) { - if (i != 0) { - if (i % 16 == 0) - printf("\n"); - else if (i % 8 == 0) - printf(" "); - } - printf("%02x ", ptr[i]); - } - printf("\n"); -} - -int main(void) -{ - int i; - unsigned long vl; - char *datap, *tmp; - - datap = malloc(MAX_VSIZE); - if (!datap) { - ksft_test_result_fail("fail to allocate memory for size = %d\n", MAX_VSIZE); - exit(-1); - } - - tmp = datap; - asm volatile ( - ".option push\n\t" - ".option arch, +v\n\t" - "vsetvli %0, x0, e8, m8, ta, ma\n\t" - "vse8.v v0, (%2)\n\t" - "add %1, %2, %0\n\t" - "vse8.v v8, (%1)\n\t" - "add %1, %1, %0\n\t" - "vse8.v v16, (%1)\n\t" - "add %1, %1, %0\n\t" - "vse8.v v24, (%1)\n\t" - ".option pop\n\t" - : "=&r" (vl), "=r" (tmp) : "r" (datap) : "memory"); - - ksft_print_msg("vl = %lu\n", vl); - - if (datap[0] != 0x00 && datap[0] != 0xff) { - ksft_test_result_fail("v-regesters are not properly initialized\n"); - dump(datap, vl * 4); - exit(-1); - } - - for (i = 1; i < vl * 4; i++) { - if (datap[i] != datap[0]) { - ksft_test_result_fail("detect stale values on v-regesters\n"); - dump(datap, vl * 4); - exit(-2); - } - } - - free(datap); - ksft_exit_pass(); - return 0; -} diff --git a/tools/testing/selftests/riscv/vector/vstate_prctl.c b/tools/testing/selftests/riscv/vector/vstate_prctl.c index 27668fb3b6d0..528e8c544db0 100644 --- a/tools/testing/selftests/riscv/vector/vstate_prctl.c +++ b/tools/testing/selftests/riscv/vector/vstate_prctl.c @@ -3,50 +3,13 @@ #include <unistd.h> #include <errno.h> #include <sys/wait.h> +#include <sys/types.h> +#include <stdlib.h>
-#include "../hwprobe/hwprobe.h" -#include "../../kselftest.h" +#include "../../kselftest_harness.h" +#include "v_helpers.h"
#define NEXT_PROGRAM "./vstate_exec_nolibc" -static int launch_test(int test_inherit) -{ - char *exec_argv[3], *exec_envp[1]; - int rc, pid, status; - - pid = fork(); - if (pid < 0) { - ksft_test_result_fail("fork failed %d", pid); - return -1; - } - - if (!pid) { - exec_argv[0] = NEXT_PROGRAM; - exec_argv[1] = test_inherit != 0 ? "x" : NULL; - exec_argv[2] = NULL; - exec_envp[0] = NULL; - /* launch the program again to check inherit */ - rc = execve(NEXT_PROGRAM, exec_argv, exec_envp); - if (rc) { - perror("execve"); - ksft_test_result_fail("child execve failed %d\n", rc); - exit(-1); - } - } - - rc = waitpid(-1, &status, 0); - if (rc < 0) { - ksft_test_result_fail("waitpid failed\n"); - return -3; - } - - if ((WIFEXITED(status) && WEXITSTATUS(status) == -1) || - WIFSIGNALED(status)) { - ksft_test_result_fail("child exited abnormally\n"); - return -4; - } - - return WEXITSTATUS(status); -}
int test_and_compare_child(long provided, long expected, int inherit) { @@ -54,14 +17,13 @@ int test_and_compare_child(long provided, long expected, int inherit)
rc = prctl(PR_RISCV_V_SET_CONTROL, provided); if (rc != 0) { - ksft_test_result_fail("prctl with provided arg %lx failed with code %d\n", - provided, rc); + printf("prctl with provided arg %lx failed with code %d\n", + provided, rc); return -1; } - rc = launch_test(inherit); + rc = launch_test(NEXT_PROGRAM, inherit); if (rc != expected) { - ksft_test_result_fail("Test failed, check %d != %ld\n", rc, - expected); + printf("Test failed, check %d != %ld\n", rc, expected); return -2; } return 0; @@ -70,112 +32,180 @@ int test_and_compare_child(long provided, long expected, int inherit) #define PR_RISCV_V_VSTATE_CTRL_CUR_SHIFT 0 #define PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT 2
-int main(void) +TEST(get_control_no_v) { - struct riscv_hwprobe pair; - long flag, expected; long rc;
- pair.key = RISCV_HWPROBE_KEY_IMA_EXT_0; - rc = riscv_hwprobe(&pair, 1, 0, NULL, 0); - if (rc < 0) { - ksft_test_result_fail("hwprobe() failed with %ld\n", rc); - return -1; - } + if (is_vector_supported()) + SKIP(return, "Test expects vector to be not supported");
- if (pair.key != RISCV_HWPROBE_KEY_IMA_EXT_0) { - ksft_test_result_fail("hwprobe cannot probe RISCV_HWPROBE_KEY_IMA_EXT_0\n"); - return -2; - } + rc = prctl(PR_RISCV_V_GET_CONTROL); + EXPECT_EQ(-1, rc) TH_LOG("GET_CONTROL should fail on kernel/hw without V"); + EXPECT_EQ(EINVAL, errno) TH_LOG("GET_CONTROL should fail on kernel/hw without V"); +}
- if (!(pair.value & RISCV_HWPROBE_IMA_V)) { - rc = prctl(PR_RISCV_V_GET_CONTROL); - if (rc != -1 || errno != EINVAL) { - ksft_test_result_fail("GET_CONTROL should fail on kernel/hw without V\n"); - return -3; - } - - rc = prctl(PR_RISCV_V_SET_CONTROL, PR_RISCV_V_VSTATE_CTRL_ON); - if (rc != -1 || errno != EINVAL) { - ksft_test_result_fail("GET_CONTROL should fail on kernel/hw without V\n"); - return -4; - } - - ksft_test_result_skip("Vector not supported\n"); - return 0; - } +TEST(set_control_no_v) +{ + long rc; + + if (is_vector_supported()) + SKIP(return, "Test expects vector to be not supported"); + + rc = prctl(PR_RISCV_V_SET_CONTROL, PR_RISCV_V_VSTATE_CTRL_ON); + EXPECT_EQ(-1, rc) TH_LOG("SET_CONTROL should fail on kernel/hw without V"); + EXPECT_EQ(EINVAL, errno) TH_LOG("SET_CONTROL should fail on kernel/hw without V"); +} + +TEST(vstate_on_current) +{ + long flag; + long rc; + + if (!is_vector_supported()) + SKIP(return, "Vector not supported");
flag = PR_RISCV_V_VSTATE_CTRL_ON; rc = prctl(PR_RISCV_V_SET_CONTROL, flag); - if (rc != 0) { - ksft_test_result_fail("Enabling V for current should always success\n"); - return -5; - } + EXPECT_EQ(0, rc) TH_LOG("Enabling V for current should always success"); +} + +TEST(vstate_off_eperm) +{ + long flag; + long rc; + + if (!is_vector_supported()) + SKIP(return, "Vector not supported");
flag = PR_RISCV_V_VSTATE_CTRL_OFF; rc = prctl(PR_RISCV_V_SET_CONTROL, flag); - if (rc != -1 || errno != EPERM) { - ksft_test_result_fail("Disabling current's V alive must fail with EPERM(%d)\n", - errno); - return -5; - } + EXPECT_EQ(EPERM, errno) TH_LOG("Disabling current's V alive must fail with EPERM(%d)", errno); + EXPECT_EQ(-1, rc) TH_LOG("Disabling current's V alive must fail with EPERM(%d)", errno); +} + +TEST(vstate_on_no_nesting) +{ + long flag; + + if (!is_vector_supported()) + SKIP(return, "Vector not supported");
/* Turn on next's vector explicitly and test */ flag = PR_RISCV_V_VSTATE_CTRL_ON << PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT; - if (test_and_compare_child(flag, PR_RISCV_V_VSTATE_CTRL_ON, 0)) - return -6; + + EXPECT_EQ(0, test_and_compare_child(flag, PR_RISCV_V_VSTATE_CTRL_ON, 0)); +} + +TEST(vstate_off_nesting) +{ + long flag; + + if (!is_vector_supported()) + SKIP(return, "Vector not supported");
/* Turn off next's vector explicitly and test */ flag = PR_RISCV_V_VSTATE_CTRL_OFF << PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT; - if (test_and_compare_child(flag, PR_RISCV_V_VSTATE_CTRL_OFF, 0)) - return -7; + + EXPECT_EQ(0, test_and_compare_child(flag, PR_RISCV_V_VSTATE_CTRL_OFF, 1)); +} + +TEST(vstate_on_inherit_no_nesting) +{ + long flag, expected; + + if (!is_vector_supported()) + SKIP(return, "Vector not supported"); + + /* Turn on next's vector explicitly and test no inherit */ + flag = PR_RISCV_V_VSTATE_CTRL_ON << PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT; + flag |= PR_RISCV_V_VSTATE_CTRL_INHERIT; + expected = flag | PR_RISCV_V_VSTATE_CTRL_ON; + + EXPECT_EQ(0, test_and_compare_child(flag, expected, 0)); +} + +TEST(vstate_on_inherit) +{ + long flag, expected; + + if (!is_vector_supported()) + SKIP(return, "Vector not supported");
/* Turn on next's vector explicitly and test inherit */ flag = PR_RISCV_V_VSTATE_CTRL_ON << PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT; flag |= PR_RISCV_V_VSTATE_CTRL_INHERIT; expected = flag | PR_RISCV_V_VSTATE_CTRL_ON; - if (test_and_compare_child(flag, expected, 0)) - return -8;
- if (test_and_compare_child(flag, expected, 1)) - return -9; + EXPECT_EQ(0, test_and_compare_child(flag, expected, 1)); +} + +TEST(vstate_off_inherit_no_nesting) +{ + long flag, expected; + + if (!is_vector_supported()) + SKIP(return, "Vector not supported"); + + /* Turn off next's vector explicitly and test no inherit */ + flag = PR_RISCV_V_VSTATE_CTRL_OFF << PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT; + flag |= PR_RISCV_V_VSTATE_CTRL_INHERIT; + expected = flag | PR_RISCV_V_VSTATE_CTRL_OFF; + + EXPECT_EQ(0, test_and_compare_child(flag, expected, 0)); +} + +TEST(vstate_off_inherit) +{ + long flag, expected; + + if (!is_vector_supported()) + SKIP(return, "Vector not supported");
/* Turn off next's vector explicitly and test inherit */ flag = PR_RISCV_V_VSTATE_CTRL_OFF << PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT; flag |= PR_RISCV_V_VSTATE_CTRL_INHERIT; expected = flag | PR_RISCV_V_VSTATE_CTRL_OFF; - if (test_and_compare_child(flag, expected, 0)) - return -10;
- if (test_and_compare_child(flag, expected, 1)) - return -11; + EXPECT_EQ(0, test_and_compare_child(flag, expected, 1)); +} + +/* arguments should fail with EINVAL */ +TEST(inval_set_control_1) +{ + int rc; + + if (!is_vector_supported()) + SKIP(return, "Vector not supported");
- /* arguments should fail with EINVAL */ rc = prctl(PR_RISCV_V_SET_CONTROL, 0xff0); - if (rc != -1 || errno != EINVAL) { - ksft_test_result_fail("Undefined control argument should return EINVAL\n"); - return -12; - } + EXPECT_EQ(-1, rc); + EXPECT_EQ(EINVAL, errno); +} + +/* arguments should fail with EINVAL */ +TEST(inval_set_control_2) +{ + int rc; + + if (!is_vector_supported()) + SKIP(return, "Vector not supported");
rc = prctl(PR_RISCV_V_SET_CONTROL, 0x3); - if (rc != -1 || errno != EINVAL) { - ksft_test_result_fail("Undefined control argument should return EINVAL\n"); - return -12; - } + EXPECT_EQ(-1, rc); + EXPECT_EQ(EINVAL, errno); +}
- rc = prctl(PR_RISCV_V_SET_CONTROL, 0xc); - if (rc != -1 || errno != EINVAL) { - ksft_test_result_fail("Undefined control argument should return EINVAL\n"); - return -12; - } +/* arguments should fail with EINVAL */ +TEST(inval_set_control_3) +{ + int rc;
- rc = prctl(PR_RISCV_V_SET_CONTROL, 0xc); - if (rc != -1 || errno != EINVAL) { - ksft_test_result_fail("Undefined control argument should return EINVAL\n"); - return -12; - } + if (!is_vector_supported()) + SKIP(return, "Vector not supported");
- ksft_test_result_pass("tests for riscv_v_vstate_ctrl pass\n"); - ksft_exit_pass(); - return 0; + rc = prctl(PR_RISCV_V_SET_CONTROL, 0xc); + EXPECT_EQ(-1, rc); + EXPECT_EQ(EINVAL, errno); } + +TEST_HARNESS_MAIN
Extend existing vector tests to be compatible with the xtheadvector instruction set.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com --- .../selftests/riscv/vector/v_exec_initval_nolibc.c | 23 ++++-- tools/testing/selftests/riscv/vector/v_helpers.c | 17 +++- tools/testing/selftests/riscv/vector/v_helpers.h | 4 +- tools/testing/selftests/riscv/vector/v_initval.c | 12 ++- .../selftests/riscv/vector/vstate_exec_nolibc.c | 20 +++-- .../testing/selftests/riscv/vector/vstate_prctl.c | 91 ++++++++++++++-------- 6 files changed, 115 insertions(+), 52 deletions(-)
diff --git a/tools/testing/selftests/riscv/vector/v_exec_initval_nolibc.c b/tools/testing/selftests/riscv/vector/v_exec_initval_nolibc.c index 74b13806baf0..58c29ea91b80 100644 --- a/tools/testing/selftests/riscv/vector/v_exec_initval_nolibc.c +++ b/tools/testing/selftests/riscv/vector/v_exec_initval_nolibc.c @@ -18,13 +18,22 @@ int main(int argc, char **argv) unsigned long vl; int first = 1;
- asm volatile ( - ".option push\n\t" - ".option arch, +v\n\t" - "vsetvli %[vl], x0, e8, m1, ta, ma\n\t" - ".option pop\n\t" - : [vl] "=r" (vl) - ); + if (argc > 2 && strcmp(argv[2], "x")) + asm volatile ( + // 0 | zimm[10:0] | rs1 | 1 1 1 | rd |1010111| vsetvli + // vsetvli t4, x0, e8, m1, d1 + ".insn 0b00000000000000000111111011010111\n\t" + "mv %[vl], t4\n\t" + : [vl] "=r" (vl) : : "t4" + ); + else + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvli %[vl], x0, e8, m1, ta, ma\n\t" + ".option pop\n\t" + : [vl] "=r" (vl) + );
#define CHECK_VECTOR_REGISTER(register) ({ \ for (int i = 0; i < vl; i++) { \ diff --git a/tools/testing/selftests/riscv/vector/v_helpers.c b/tools/testing/selftests/riscv/vector/v_helpers.c index 15c22318db72..2c4df76eefe9 100644 --- a/tools/testing/selftests/riscv/vector/v_helpers.c +++ b/tools/testing/selftests/riscv/vector/v_helpers.c @@ -1,11 +1,21 @@ // SPDX-License-Identifier: GPL-2.0-only
#include "../hwprobe/hwprobe.h" +#include <asm/vendor/thead.h> #include <stdlib.h> #include <stdio.h> #include <unistd.h> #include <sys/wait.h>
+int is_xtheadvector_supported(void) +{ + struct riscv_hwprobe pair; + + pair.key = RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0; + riscv_hwprobe(&pair, 1, 0, NULL, 0); + return pair.value & RISCV_HWPROBE_VENDOR_EXT_XTHEADVECTOR; +} + int is_vector_supported(void) { struct riscv_hwprobe pair; @@ -15,9 +25,9 @@ int is_vector_supported(void) return pair.value & RISCV_HWPROBE_IMA_V; }
-int launch_test(char *next_program, int test_inherit) +int launch_test(char *next_program, int test_inherit, int xtheadvector) { - char *exec_argv[3], *exec_envp[1]; + char *exec_argv[4], *exec_envp[1]; int rc, pid, status;
pid = fork(); @@ -29,7 +39,8 @@ int launch_test(char *next_program, int test_inherit) if (!pid) { exec_argv[0] = next_program; exec_argv[1] = test_inherit != 0 ? "x" : NULL; - exec_argv[2] = NULL; + exec_argv[2] = xtheadvector != 0 ? "x" : NULL; + exec_argv[3] = NULL; exec_envp[0] = NULL; /* launch the program again to check inherit */ rc = execve(next_program, exec_argv, exec_envp); diff --git a/tools/testing/selftests/riscv/vector/v_helpers.h b/tools/testing/selftests/riscv/vector/v_helpers.h index 88719c4be496..67d41cb6f871 100644 --- a/tools/testing/selftests/riscv/vector/v_helpers.h +++ b/tools/testing/selftests/riscv/vector/v_helpers.h @@ -1,5 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0-only */
+int is_xtheadvector_supported(void); + int is_vector_supported(void);
-int launch_test(char *next_program, int test_inherit); +int launch_test(char *next_program, int test_inherit, int xtheadvector); diff --git a/tools/testing/selftests/riscv/vector/v_initval.c b/tools/testing/selftests/riscv/vector/v_initval.c index f38b5797fa31..be9e1d18ad29 100644 --- a/tools/testing/selftests/riscv/vector/v_initval.c +++ b/tools/testing/selftests/riscv/vector/v_initval.c @@ -7,10 +7,16 @@
TEST(v_initval) { - if (!is_vector_supported()) - SKIP(return, "Vector not supported"); + int xtheadvector = 0;
- ASSERT_EQ(0, launch_test(NEXT_PROGRAM, 0)); + if (!is_vector_supported()) { + if (is_xtheadvector_supported()) + xtheadvector = 1; + else + SKIP(return, "Vector not supported"); + } + + ASSERT_EQ(0, launch_test(NEXT_PROGRAM, 0, xtheadvector)); }
TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/riscv/vector/vstate_exec_nolibc.c b/tools/testing/selftests/riscv/vector/vstate_exec_nolibc.c index 1f9969bed235..12d30d3b90fa 100644 --- a/tools/testing/selftests/riscv/vector/vstate_exec_nolibc.c +++ b/tools/testing/selftests/riscv/vector/vstate_exec_nolibc.c @@ -6,13 +6,16 @@
int main(int argc, char **argv) { - int rc, pid, status, test_inherit = 0; + int rc, pid, status, test_inherit = 0, xtheadvector = 0; long ctrl, ctrl_c; char *exec_argv[2], *exec_envp[2];
- if (argc > 1) + if (argc > 1 && strcmp(argv[1], "x")) test_inherit = 1;
+ if (argc > 2 && strcmp(argv[2], "x")) + xtheadvector = 1; + ctrl = my_syscall1(__NR_prctl, PR_RISCV_V_GET_CONTROL); if (ctrl < 0) { puts("PR_RISCV_V_GET_CONTROL is not supported\n"); @@ -53,11 +56,14 @@ int main(int argc, char **argv) puts("child's vstate_ctrl not equal to parent's\n"); exit(-1); } - asm volatile (".option push\n\t" - ".option arch, +v\n\t" - "vsetvli x0, x0, e32, m8, ta, ma\n\t" - ".option pop\n\t" - ); + if (xtheadvector) + asm volatile (".insn 0x00007ed7"); + else + asm volatile (".option push\n\t" + ".option arch, +v\n\t" + "vsetvli x0, x0, e32, m8, ta, ma\n\t" + ".option pop\n\t" + ); exit(ctrl); } } diff --git a/tools/testing/selftests/riscv/vector/vstate_prctl.c b/tools/testing/selftests/riscv/vector/vstate_prctl.c index 528e8c544db0..375af40e88e6 100644 --- a/tools/testing/selftests/riscv/vector/vstate_prctl.c +++ b/tools/testing/selftests/riscv/vector/vstate_prctl.c @@ -11,7 +11,7 @@
#define NEXT_PROGRAM "./vstate_exec_nolibc"
-int test_and_compare_child(long provided, long expected, int inherit) +int test_and_compare_child(long provided, long expected, int inherit, int xtheadvector) { int rc;
@@ -21,7 +21,7 @@ int test_and_compare_child(long provided, long expected, int inherit) provided, rc); return -1; } - rc = launch_test(NEXT_PROGRAM, inherit); + rc = launch_test(NEXT_PROGRAM, inherit, xtheadvector); if (rc != expected) { printf("Test failed, check %d != %ld\n", rc, expected); return -2; @@ -36,7 +36,7 @@ TEST(get_control_no_v) { long rc;
- if (is_vector_supported()) + if (is_vector_supported() || is_xtheadvector_supported()) SKIP(return, "Test expects vector to be not supported");
rc = prctl(PR_RISCV_V_GET_CONTROL); @@ -48,7 +48,7 @@ TEST(set_control_no_v) { long rc;
- if (is_vector_supported()) + if (is_vector_supported() || is_xtheadvector_supported()) SKIP(return, "Test expects vector to be not supported");
rc = prctl(PR_RISCV_V_SET_CONTROL, PR_RISCV_V_VSTATE_CTRL_ON); @@ -61,12 +61,12 @@ TEST(vstate_on_current) long flag; long rc;
- if (!is_vector_supported()) + if (!is_vector_supported() && !is_xtheadvector_supported()) SKIP(return, "Vector not supported");
flag = PR_RISCV_V_VSTATE_CTRL_ON; rc = prctl(PR_RISCV_V_SET_CONTROL, flag); - EXPECT_EQ(0, rc) TH_LOG("Enabling V for current should always success"); + EXPECT_EQ(0, rc) TH_LOG("Enabling V for current should always succeed"); }
TEST(vstate_off_eperm) @@ -74,99 +74,128 @@ TEST(vstate_off_eperm) long flag; long rc;
- if (!is_vector_supported()) + if (!is_vector_supported() && !is_xtheadvector_supported()) SKIP(return, "Vector not supported");
flag = PR_RISCV_V_VSTATE_CTRL_OFF; rc = prctl(PR_RISCV_V_SET_CONTROL, flag); - EXPECT_EQ(EPERM, errno) TH_LOG("Disabling current's V alive must fail with EPERM(%d)", errno); - EXPECT_EQ(-1, rc) TH_LOG("Disabling current's V alive must fail with EPERM(%d)", errno); + EXPECT_EQ(EPERM, errno) TH_LOG("Disabling V in current thread with V enabled must fail with EPERM(%d)", errno); + EXPECT_EQ(-1, rc) TH_LOG("Disabling V in current thread with V enabled must fail with EPERM(%d)", errno); }
TEST(vstate_on_no_nesting) { long flag; + int xtheadvector = 0;
- if (!is_vector_supported()) - SKIP(return, "Vector not supported"); + if (!is_vector_supported()) { + if (is_xtheadvector_supported()) + xtheadvector = 1; + else + SKIP(return, "Vector not supported"); + }
/* Turn on next's vector explicitly and test */ flag = PR_RISCV_V_VSTATE_CTRL_ON << PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT;
- EXPECT_EQ(0, test_and_compare_child(flag, PR_RISCV_V_VSTATE_CTRL_ON, 0)); + EXPECT_EQ(0, test_and_compare_child(flag, PR_RISCV_V_VSTATE_CTRL_ON, 0, xtheadvector)); }
TEST(vstate_off_nesting) { long flag; + int xtheadvector = 0;
- if (!is_vector_supported()) - SKIP(return, "Vector not supported"); + if (!is_vector_supported()) { + if (is_xtheadvector_supported()) + xtheadvector = 1; + else + SKIP(return, "Vector not supported"); + }
/* Turn off next's vector explicitly and test */ flag = PR_RISCV_V_VSTATE_CTRL_OFF << PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT;
- EXPECT_EQ(0, test_and_compare_child(flag, PR_RISCV_V_VSTATE_CTRL_OFF, 1)); + EXPECT_EQ(0, test_and_compare_child(flag, PR_RISCV_V_VSTATE_CTRL_OFF, 1, xtheadvector)); }
TEST(vstate_on_inherit_no_nesting) { long flag, expected; + int xtheadvector = 0;
- if (!is_vector_supported()) - SKIP(return, "Vector not supported"); + if (!is_vector_supported()) { + if (is_xtheadvector_supported()) + xtheadvector = 1; + else + SKIP(return, "Vector not supported"); + }
/* Turn on next's vector explicitly and test no inherit */ flag = PR_RISCV_V_VSTATE_CTRL_ON << PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT; flag |= PR_RISCV_V_VSTATE_CTRL_INHERIT; expected = flag | PR_RISCV_V_VSTATE_CTRL_ON;
- EXPECT_EQ(0, test_and_compare_child(flag, expected, 0)); + EXPECT_EQ(0, test_and_compare_child(flag, expected, 0, xtheadvector)); }
TEST(vstate_on_inherit) { long flag, expected; + int xtheadvector = 0;
- if (!is_vector_supported()) - SKIP(return, "Vector not supported"); + if (!is_vector_supported()) { + if (is_xtheadvector_supported()) + xtheadvector = 1; + else + SKIP(return, "Vector not supported"); + }
/* Turn on next's vector explicitly and test inherit */ flag = PR_RISCV_V_VSTATE_CTRL_ON << PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT; flag |= PR_RISCV_V_VSTATE_CTRL_INHERIT; expected = flag | PR_RISCV_V_VSTATE_CTRL_ON;
- EXPECT_EQ(0, test_and_compare_child(flag, expected, 1)); + EXPECT_EQ(0, test_and_compare_child(flag, expected, 1, xtheadvector)); }
TEST(vstate_off_inherit_no_nesting) { long flag, expected; + int xtheadvector = 0;
- if (!is_vector_supported()) - SKIP(return, "Vector not supported"); - + if (!is_vector_supported()) { + if (is_xtheadvector_supported()) + xtheadvector = 1; + else + SKIP(return, "Vector not supported"); + } /* Turn off next's vector explicitly and test no inherit */ flag = PR_RISCV_V_VSTATE_CTRL_OFF << PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT; flag |= PR_RISCV_V_VSTATE_CTRL_INHERIT; expected = flag | PR_RISCV_V_VSTATE_CTRL_OFF;
- EXPECT_EQ(0, test_and_compare_child(flag, expected, 0)); + EXPECT_EQ(0, test_and_compare_child(flag, expected, 0, xtheadvector)); }
TEST(vstate_off_inherit) { long flag, expected; + int xtheadvector = 0;
- if (!is_vector_supported()) - SKIP(return, "Vector not supported"); + if (!is_vector_supported()) { + if (is_xtheadvector_supported()) + xtheadvector = 1; + else + SKIP(return, "Vector not supported"); + }
/* Turn off next's vector explicitly and test inherit */ flag = PR_RISCV_V_VSTATE_CTRL_OFF << PR_RISCV_V_VSTATE_CTRL_NEXT_SHIFT; flag |= PR_RISCV_V_VSTATE_CTRL_INHERIT; expected = flag | PR_RISCV_V_VSTATE_CTRL_OFF;
- EXPECT_EQ(0, test_and_compare_child(flag, expected, 1)); + EXPECT_EQ(0, test_and_compare_child(flag, expected, 1, xtheadvector)); }
/* arguments should fail with EINVAL */ @@ -174,7 +203,7 @@ TEST(inval_set_control_1) { int rc;
- if (!is_vector_supported()) + if (!is_vector_supported() && !is_xtheadvector_supported()) SKIP(return, "Vector not supported");
rc = prctl(PR_RISCV_V_SET_CONTROL, 0xff0); @@ -187,7 +216,7 @@ TEST(inval_set_control_2) { int rc;
- if (!is_vector_supported()) + if (!is_vector_supported() && !is_xtheadvector_supported()) SKIP(return, "Vector not supported");
rc = prctl(PR_RISCV_V_SET_CONTROL, 0x3); @@ -200,7 +229,7 @@ TEST(inval_set_control_3) { int rc;
- if (!is_vector_supported()) + if (!is_vector_supported() && !is_xtheadvector_supported()) SKIP(return, "Vector not supported");
rc = prctl(PR_RISCV_V_SET_CONTROL, 0xc);
linux-kselftest-mirror@lists.linaro.org
-
Alexandre Ghiti -
Charlie Jenkins -
Conor Dooley -
Guo Ren