Fix following sparse warnings: tools/testing/selftests/resctrl/resctrl_val.c:47:6: warning: symbol 'membw_initialize_perf_event_attr' was not declared. Should it be static? tools/testing/selftests/resctrl/resctrl_val.c:64:6: warning: symbol 'membw_ioctl_perf_event_ioc_reset_enable' was not declared. Should it be static? tools/testing/selftests/resctrl/resctrl_val.c:70:6: warning: symbol 'membw_ioctl_perf_event_ioc_disable' was not declared. Should it be static? tools/testing/selftests/resctrl/resctrl_val.c:81:6: warning: symbol 'get_event_and_umask' was not declared. Should it be static?

Signed-off-by: Reinette Chatre reinette.chatre@intel.com Reviewed-by: Ilpo Järvinen ilpo.jarvinen@linux.intel.com --- Changes since v1: - Add Ilpo's Reviewed-by tag. - Let subject describe the change, not the tool that found it. (checkpatch.pl) --- tools/testing/selftests/resctrl/resctrl_val.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/resctrl/resctrl_val.c b/tools/testing/selftests/resctrl/resctrl_val.c index 8c275f6b4dd7..70e8e31f5d1a 100644 --- a/tools/testing/selftests/resctrl/resctrl_val.c +++ b/tools/testing/selftests/resctrl/resctrl_val.c @@ -44,7 +44,7 @@ static int imcs; static struct imc_counter_config imc_counters_config[MAX_IMCS][2]; static const struct resctrl_test *current_test;

-void membw_initialize_perf_event_attr(int i, int j) +static void membw_initialize_perf_event_attr(int i, int j) { memset(&imc_counters_config[i][j].pe, 0, sizeof(struct perf_event_attr)); @@ -61,13 +61,13 @@ void membw_initialize_perf_event_attr(int i, int j) PERF_FORMAT_TOTAL_TIME_ENABLED | PERF_FORMAT_TOTAL_TIME_RUNNING; }

-void membw_ioctl_perf_event_ioc_reset_enable(int i, int j) +static void membw_ioctl_perf_event_ioc_reset_enable(int i, int j) { ioctl(imc_counters_config[i][j].fd, PERF_EVENT_IOC_RESET, 0); ioctl(imc_counters_config[i][j].fd, PERF_EVENT_IOC_ENABLE, 0); }

-void membw_ioctl_perf_event_ioc_disable(int i, int j) +static void membw_ioctl_perf_event_ioc_disable(int i, int j) { ioctl(imc_counters_config[i][j].fd, PERF_EVENT_IOC_DISABLE, 0); } @@ -78,7 +78,7 @@ void membw_ioctl_perf_event_ioc_disable(int i, int j) * @count: iMC number * @op: Operation (read/write) */ -void get_event_and_umask(char *cas_count_cfg, int count, bool op) +static void get_event_and_umask(char *cas_count_cfg, int count, bool op) { char *token[MAX_TOKENS]; int i = 0;

The MBM and MBA tests need to discover the event and umask with which to configure the performance event used to measure read memory bandwidth. This is done by parsing the /sys/bus/event_source/devices/uncore_imc_<imc instance>/events/cas_count_read file for each iMC instance that contains the formatted output: "event=<event>,umask=<umask>"

Parsing of cas_count_read contents is done by initializing an array of MAX_TOKENS elements with tokens (deliminated by "=,") from this file. Remove the unnecessary append of a delimiter to the string needing to be parsed. Per the strtok() man page: "delimiter bytes at the start or end of the string are ignored". This has no impact on the token placement within the array.

After initialization, the actual event and umask is determined by parsing the tokens directly following the "event" and "umask" tokens respectively.

Iterating through the array up to index "i < MAX_TOKENS" but then accessing index "i + 1" risks array overrun during the final iteration. Avoid array overrun by ensuring that the index used within for loop will always be valid.

Fixes: 1d3f08687d76 ("selftests/resctrl: Read memory bandwidth from perf IMC counter and from resctrl file system") Signed-off-by: Reinette Chatre reinette.chatre@intel.com Reviewed-by: Ilpo Järvinen ilpo.jarvinen@linux.intel.com --- Changes since V2: - Rephrase changelog. (Ilpo) - Add Ilpo's Reviewed-by tag.

Changes since V1: - New patch. --- tools/testing/selftests/resctrl/resctrl_val.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/tools/testing/selftests/resctrl/resctrl_val.c b/tools/testing/selftests/resctrl/resctrl_val.c index 70e8e31f5d1a..e88d5ca30517 100644 --- a/tools/testing/selftests/resctrl/resctrl_val.c +++ b/tools/testing/selftests/resctrl/resctrl_val.c @@ -83,13 +83,12 @@ static void get_event_and_umask(char *cas_count_cfg, int count, bool op) char *token[MAX_TOKENS]; int i = 0;

- strcat(cas_count_cfg, ","); token[0] = strtok(cas_count_cfg, "=,");

for (i = 1; i < MAX_TOKENS; i++) token[i] = strtok(NULL, "=,");

- for (i = 0; i < MAX_TOKENS; i++) { + for (i = 0; i < MAX_TOKENS - 1; i++) { if (!token[i]) break; if (strcmp(token[i], "event") == 0) {

Within mba_setup() the programmed bandwidth delay value starts at the maximum (100, or rather ALLOCATION_MAX) and progresses towards ALLOCATION_MIN by decrementing with ALLOCATION_STEP.

The programmed bandwidth delay should never be negative, so representing it with an unsigned int is most appropriate. This may introduce confusion because of the "allocation > ALLOCATION_MAX" check used to check wraparound of the subtraction.

Modify the mba_setup() flow to start at the minimum, ALLOCATION_MIN, and incrementally, with ALLOCATION_STEP steps, adjust the bandwidth delay value. This avoids wraparound while making the purpose of "allocation > ALLOCATION_MAX" clear and eliminates the need for the "allocation < ALLOCATION_MIN" check.

Reported-by: Ilpo Järvinen ilpo.jarvinen@linux.intel.com Closes: https://lore.kernel.org/lkml/1903ac13-5c9c-ef8d-78e0-417ac34a971b@linux.inte... Signed-off-by: Reinette Chatre reinette.chatre@intel.com Reviewed-by: Ilpo Järvinen ilpo.jarvinen@linux.intel.com --- Changes since V1: - New patch - Add Ilpo's Reviewed-by tag. --- tools/testing/selftests/resctrl/mba_test.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/resctrl/mba_test.c b/tools/testing/selftests/resctrl/mba_test.c index ab8496a4925b..da40a8ed4413 100644 --- a/tools/testing/selftests/resctrl/mba_test.c +++ b/tools/testing/selftests/resctrl/mba_test.c @@ -39,7 +39,8 @@ static int mba_setup(const struct resctrl_test *test, const struct user_params *uparams, struct resctrl_val_param *p) { - static int runs_per_allocation, allocation = 100; + static unsigned int allocation = ALLOCATION_MIN; + static int runs_per_allocation; char allocation_str[64]; int ret;

@@ -50,7 +51,7 @@ static int mba_setup(const struct resctrl_test *test, if (runs_per_allocation++ != 0) return 0;

- if (allocation < ALLOCATION_MIN || allocation > ALLOCATION_MAX) + if (allocation > ALLOCATION_MAX) return END_OF_TESTS;

sprintf(allocation_str, "%d", allocation); @@ -59,7 +60,7 @@ static int mba_setup(const struct resctrl_test *test, if (ret < 0) return ret;

- allocation -= ALLOCATION_STEP; + allocation += ALLOCATION_STEP;

return 0; } @@ -72,8 +73,9 @@ static int mba_measure(const struct user_params *uparams,

static bool show_mba_info(unsigned long *bw_imc, unsigned long *bw_resc) { - int allocation, runs; + unsigned int allocation; bool ret = false; + int runs;

ksft_print_msg("Results are displayed in (MB)\n"); /* Memory bandwidth from 100% down to 10% */ @@ -103,7 +105,7 @@ static bool show_mba_info(unsigned long *bw_imc, unsigned long *bw_resc) avg_diff_per > MAX_DIFF_PERCENT ? "Fail:" : "Pass:", MAX_DIFF_PERCENT, - ALLOCATION_MAX - ALLOCATION_STEP * allocation); + ALLOCATION_MIN + ALLOCATION_STEP * allocation);

ksft_print_msg("avg_diff_per: %d%%\n", avg_diff_per); ksft_print_msg("avg_bw_imc: %lu\n", avg_bw_imc);

The CMT, MBM, and MBA tests rely on a benchmark to generate memory traffic. By default this is the "fill_buf" benchmark that can be replaced via the "-b" command line argument.

The original intent of the "-b" command line parameter was to replace the default "fill_buf" benchmark, but the implementation also exposes an alternative use case where the "fill_buf" parameters itself can be modified. One of the parameters to "fill_buf" is the "operation" that can be either "read" or "write" and indicates whether the "fill_buf" should use "read" or "write" operations on the allocated buffer.

While replacing "fill_buf" default parameters is technically possible, replacing the default "read" parameter with "write" is not supported because the MBA and MBM tests only measure "read" operations. The "read" operation is also most appropriate for the CMT test that aims to use the benchmark to allocate into the cache.

Avoid any potential inconsistencies between test and measurement by removing code for unsupported "write" operations to the buffer. Ignore any attempt from user space to enable this unsupported test configuration, instead always use read operations.

Keep the initialization of the, now unused, "fill_buf" parameters to reserve these parameter positions since it has been exposed as an API. Future parameter additions cannot use these parameter positions.

Signed-off-by: Reinette Chatre reinette.chatre@intel.com Reviewed-by: Ilpo Järvinen ilpo.jarvinen@linux.intel.com --- Changes since V3: - Add Ilpo's Reviewed-by tag.

Changes since V2: - Update changelog to justify keeping the assignment to benchmark_cmd[4]. (Ilpo)

Changes since V1: - New patch. --- tools/testing/selftests/resctrl/fill_buf.c | 28 ++----------------- tools/testing/selftests/resctrl/resctrl.h | 2 +- .../testing/selftests/resctrl/resctrl_tests.c | 5 +++- tools/testing/selftests/resctrl/resctrl_val.c | 5 ++-- 4 files changed, 9 insertions(+), 31 deletions(-)

diff --git a/tools/testing/selftests/resctrl/fill_buf.c b/tools/testing/selftests/resctrl/fill_buf.c index 854f0108d8e6..e4f1cea317f1 100644 --- a/tools/testing/selftests/resctrl/fill_buf.c +++ b/tools/testing/selftests/resctrl/fill_buf.c @@ -88,18 +88,6 @@ static int fill_one_span_read(unsigned char *buf, size_t buf_size) return sum; }

-static void fill_one_span_write(unsigned char *buf, size_t buf_size) -{ - unsigned char *end_ptr = buf + buf_size; - unsigned char *p; - - p = buf; - while (p < end_ptr) { - *p = '1'; - p += (CL_SIZE / 2); - } -} - void fill_cache_read(unsigned char *buf, size_t buf_size, bool once) { int ret = 0; @@ -114,15 +102,6 @@ void fill_cache_read(unsigned char *buf, size_t buf_size, bool once) *value_sink = ret; }

-static void fill_cache_write(unsigned char *buf, size_t buf_size, bool once) -{ - while (1) { - fill_one_span_write(buf, buf_size); - if (once) - break; - } -} - unsigned char *alloc_buffer(size_t buf_size, int memflush) { void *buf = NULL; @@ -151,7 +130,7 @@ unsigned char *alloc_buffer(size_t buf_size, int memflush) return buf; }

-int run_fill_buf(size_t buf_size, int memflush, int op) +int run_fill_buf(size_t buf_size, int memflush) { unsigned char *buf;

@@ -159,10 +138,7 @@ int run_fill_buf(size_t buf_size, int memflush, int op) if (!buf) return -1;

- if (op == 0) - fill_cache_read(buf, buf_size, false); - else - fill_cache_write(buf, buf_size, false); + fill_cache_read(buf, buf_size, false);

free(buf);

diff --git a/tools/testing/selftests/resctrl/resctrl.h b/tools/testing/selftests/resctrl/resctrl.h index 51f5f4b25e06..ba1ce1b35699 100644 --- a/tools/testing/selftests/resctrl/resctrl.h +++ b/tools/testing/selftests/resctrl/resctrl.h @@ -142,7 +142,7 @@ int perf_event_open(struct perf_event_attr *hw_event, pid_t pid, int cpu, unsigned char *alloc_buffer(size_t buf_size, int memflush); void mem_flush(unsigned char *buf, size_t buf_size); void fill_cache_read(unsigned char *buf, size_t buf_size, bool once); -int run_fill_buf(size_t buf_size, int memflush, int op); +int run_fill_buf(size_t buf_size, int memflush); int initialize_mem_bw_imc(void); int measure_mem_bw(const struct user_params *uparams, struct resctrl_val_param *param, pid_t bm_pid, diff --git a/tools/testing/selftests/resctrl/resctrl_tests.c b/tools/testing/selftests/resctrl/resctrl_tests.c index e7878077883f..0f91c475b255 100644 --- a/tools/testing/selftests/resctrl/resctrl_tests.c +++ b/tools/testing/selftests/resctrl/resctrl_tests.c @@ -265,13 +265,16 @@ int main(int argc, char **argv) ksft_exit_fail_msg("Out of memory!\n"); uparams.benchmark_cmd[1] = span_str; uparams.benchmark_cmd[2] = "1"; - uparams.benchmark_cmd[3] = "0"; /* + * Third parameter was previously used for "operation" + * (read/write) of which only (now default) "read"/"0" + * works. * Fourth parameter was previously used to indicate * how long "fill_buf" should run for, with "false" * ("fill_buf" will keep running until terminated) * the only option that works. */ + uparams.benchmark_cmd[3] = NULL; uparams.benchmark_cmd[4] = NULL; }

diff --git a/tools/testing/selftests/resctrl/resctrl_val.c b/tools/testing/selftests/resctrl/resctrl_val.c index b0f3c594c4da..113ca18d67c1 100644 --- a/tools/testing/selftests/resctrl/resctrl_val.c +++ b/tools/testing/selftests/resctrl/resctrl_val.c @@ -622,8 +622,8 @@ int measure_mem_bw(const struct user_params *uparams, */ static void run_benchmark(int signum, siginfo_t *info, void *ucontext) { - int operation, ret, memflush; char **benchmark_cmd; + int ret, memflush; size_t span; FILE *fp;

@@ -643,9 +643,8 @@ static void run_benchmark(int signum, siginfo_t *info, void *ucontext) /* Execute default fill_buf benchmark */ span = strtoul(benchmark_cmd[1], NULL, 10); memflush = atoi(benchmark_cmd[2]); - operation = atoi(benchmark_cmd[3]);

- if (run_fill_buf(span, memflush, operation)) + if (run_fill_buf(span, memflush)) fprintf(stderr, "Error in running fill buffer\n"); } else { /* Execute specified benchmark */

The benchmark used during the CMT, MBM, and MBA tests can be provided by the user via (-b) parameter, if not provided the default "fill_buf" benchmark is used. The user is additionally able to override any of the "fill_buf" default parameters when running the tests with "-b fill_buf <fill_buf parameters>".

The "fill_buf" parameters are managed as an array of strings. Using an array of strings is complex because it requires transformations to/from strings at every producer and consumer. This is made worse for the individual tests where the default benchmark parameters values may not be appropriate and additional data wrangling is required. For example, the CMT test duplicates the entire array of strings in order to replace one of the parameters.

More issues appear when combining the usage of an array of strings with the use case of user overriding default parameters by specifying "-b fill_buf <parameters>". This use case is fragile with opportunities to trigger a SIGSEGV because of opportunities for NULL pointers to exist in the array of strings. For example, by running below (thus by specifying "fill_buf" should be used but all parameters are NULL): $ sudo resctrl_tests -t mbm -b fill_buf

Replace the "array of strings" parameters used for "fill_buf" with new struct fill_buf_param that contains the "fill_buf" parameters that can be used directly without transformations to/from strings. Two instances of struct fill_buf_param may exist at any point in time: * If the user provides new parameters to "fill_buf", the user parameter structure (struct user_params) will point to a fully initialized and immutable struct fill_buf_param containing the user provided parameters. * If "fill_buf" is the benchmark that should be used by a test, then the test parameter structure (struct resctrl_val_param) will point to a fully initialized struct fill_buf_param. The latter may contain (a) the user provided parameters verbatim, (b) user provided parameters adjusted to be appropriate for the test, or (c) the default parameters for "fill_buf" that is appropriate for the test if the user did not provide "fill_buf" parameters nor an alternate benchmark.

The existing behavior of CMT test is to use test defined value for the buffer size even if the user provides another value via command line. This behavior is maintained since the test requires that the buffer size matches the size of the cache allocated, and the amount of cache allocated can instead be changed by the user with the "-n" command line parameter.

Signed-off-by: Reinette Chatre reinette.chatre@intel.com --- Changes since V3: - Handle empty string input. (Ilpo)

Changes since V2: - Use empty initializers. (Ilpo) - Let memflush be bool instead of int. (Ilpo) - Make user input checks more robust. (Ilpo) - Assign values as part of local variable definition. (Ilpo)

Changes since V1: - Maintain original behavior where user can override "fill_buf" parameters via command line ... but only those that can actually be changed. (Ilpo) - Fix parsing issues associated with original behavior to ensure any parameter is valid before any attempt to use it. - Move patch earlier in series to highlight that this fixes existing issues. - Make struct fill_buf_param dynamic to support user provided parameters as well as test specific parameters. - Rewrite changelog. --- tools/testing/selftests/resctrl/cmt_test.c | 32 ++---- tools/testing/selftests/resctrl/fill_buf.c | 4 +- tools/testing/selftests/resctrl/mba_test.c | 13 ++- tools/testing/selftests/resctrl/mbm_test.c | 22 ++-- tools/testing/selftests/resctrl/resctrl.h | 59 +++++++--- .../testing/selftests/resctrl/resctrl_tests.c | 103 ++++++++++++++---- tools/testing/selftests/resctrl/resctrl_val.c | 41 ++++--- 7 files changed, 178 insertions(+), 96 deletions(-)

diff --git a/tools/testing/selftests/resctrl/cmt_test.c b/tools/testing/selftests/resctrl/cmt_test.c index 0c045080d808..4c3cf2c25a38 100644 --- a/tools/testing/selftests/resctrl/cmt_test.c +++ b/tools/testing/selftests/resctrl/cmt_test.c @@ -116,15 +116,13 @@ static void cmt_test_cleanup(void)

static int cmt_run_test(const struct resctrl_test *test, const struct user_params *uparams) { - const char * const *cmd = uparams->benchmark_cmd; - const char *new_cmd[BENCHMARK_ARGS]; + struct fill_buf_param fill_buf = {}; unsigned long cache_total_size = 0; int n = uparams->bits ? : 5; unsigned long long_mask; - char *span_str = NULL; int count_of_bits; size_t span; - int ret, i; + int ret;

ret = get_full_cbm("L3", &long_mask); if (ret) @@ -155,32 +153,26 @@ static int cmt_run_test(const struct resctrl_test *test, const struct user_param

span = cache_portion_size(cache_total_size, param.mask, long_mask);

- if (strcmp(cmd[0], "fill_buf") == 0) { - /* Duplicate the command to be able to replace span in it */ - for (i = 0; uparams->benchmark_cmd[i]; i++) - new_cmd[i] = uparams->benchmark_cmd[i]; - new_cmd[i] = NULL; - - ret = asprintf(&span_str, "%zu", span); - if (ret < 0) - return -1; - new_cmd[1] = span_str; - cmd = new_cmd; + if (uparams->fill_buf) { + fill_buf.buf_size = span; + fill_buf.memflush = uparams->fill_buf->memflush; + param.fill_buf = &fill_buf; + } else if (!uparams->benchmark_cmd[0]) { + fill_buf.buf_size = span; + fill_buf.memflush = true; + param.fill_buf = &fill_buf; }

remove(RESULT_FILE_NAME);

- ret = resctrl_val(test, uparams, cmd, &param); + ret = resctrl_val(test, uparams, &param); if (ret) - goto out; + return ret;

ret = check_results(&param, span, n); if (ret && (get_vendor() == ARCH_INTEL)) ksft_print_msg("Intel CMT may be inaccurate when Sub-NUMA Clustering is enabled. Check BIOS configuration.\n");

-out: - free(span_str); - return ret; }

diff --git a/tools/testing/selftests/resctrl/fill_buf.c b/tools/testing/selftests/resctrl/fill_buf.c index e4f1cea317f1..39545f9369e8 100644 --- a/tools/testing/selftests/resctrl/fill_buf.c +++ b/tools/testing/selftests/resctrl/fill_buf.c @@ -102,7 +102,7 @@ void fill_cache_read(unsigned char *buf, size_t buf_size, bool once) *value_sink = ret; }

-unsigned char *alloc_buffer(size_t buf_size, int memflush) +unsigned char *alloc_buffer(size_t buf_size, bool memflush) { void *buf = NULL; uint64_t *p64; @@ -130,7 +130,7 @@ unsigned char *alloc_buffer(size_t buf_size, int memflush) return buf; }

-int run_fill_buf(size_t buf_size, int memflush) +int run_fill_buf(size_t buf_size, bool memflush) { unsigned char *buf;

diff --git a/tools/testing/selftests/resctrl/mba_test.c b/tools/testing/selftests/resctrl/mba_test.c index be0ead73e55d..74d95c460bd0 100644 --- a/tools/testing/selftests/resctrl/mba_test.c +++ b/tools/testing/selftests/resctrl/mba_test.c @@ -172,11 +172,22 @@ static int mba_run_test(const struct resctrl_test *test, const struct user_param .setup = mba_setup, .measure = mba_measure, }; + struct fill_buf_param fill_buf = {}; int ret;

remove(RESULT_FILE_NAME);

- ret = resctrl_val(test, uparams, uparams->benchmark_cmd, &param); + if (uparams->fill_buf) { + fill_buf.buf_size = uparams->fill_buf->buf_size; + fill_buf.memflush = uparams->fill_buf->memflush; + param.fill_buf = &fill_buf; + } else if (!uparams->benchmark_cmd[0]) { + fill_buf.buf_size = DEFAULT_SPAN; + fill_buf.memflush = true; + param.fill_buf = &fill_buf; + } + + ret = resctrl_val(test, uparams, &param); if (ret) return ret;

diff --git a/tools/testing/selftests/resctrl/mbm_test.c b/tools/testing/selftests/resctrl/mbm_test.c index defa94293915..72261413c868 100644 --- a/tools/testing/selftests/resctrl/mbm_test.c +++ b/tools/testing/selftests/resctrl/mbm_test.c @@ -139,26 +139,26 @@ static int mbm_run_test(const struct resctrl_test *test, const struct user_param .setup = mbm_setup, .measure = mbm_measure, }; - char *endptr = NULL; - size_t span = 0; + struct fill_buf_param fill_buf = {}; int ret;

remove(RESULT_FILE_NAME);

- if (uparams->benchmark_cmd[0] && strcmp(uparams->benchmark_cmd[0], "fill_buf") == 0) { - if (uparams->benchmark_cmd[1] && *uparams->benchmark_cmd[1] != '\0') { - errno = 0; - span = strtoul(uparams->benchmark_cmd[1], &endptr, 10); - if (errno || *endptr != '\0') - return -EINVAL; - } + if (uparams->fill_buf) { + fill_buf.buf_size = uparams->fill_buf->buf_size; + fill_buf.memflush = uparams->fill_buf->memflush; + param.fill_buf = &fill_buf; + } else if (!uparams->benchmark_cmd[0]) { + fill_buf.buf_size = DEFAULT_SPAN; + fill_buf.memflush = true; + param.fill_buf = &fill_buf; }

- ret = resctrl_val(test, uparams, uparams->benchmark_cmd, &param); + ret = resctrl_val(test, uparams, &param); if (ret) return ret;

- ret = check_results(span); + ret = check_results(param.fill_buf ? param.fill_buf->buf_size : 0); if (ret && (get_vendor() == ARCH_INTEL)) ksft_print_msg("Intel MBM may be inaccurate when Sub-NUMA Clustering is enabled. Check BIOS configuration.\n");

diff --git a/tools/testing/selftests/resctrl/resctrl.h b/tools/testing/selftests/resctrl/resctrl.h index 82801245e4c1..c9336f9c2cae 100644 --- a/tools/testing/selftests/resctrl/resctrl.h +++ b/tools/testing/selftests/resctrl/resctrl.h @@ -43,16 +43,36 @@

#define DEFAULT_SPAN (250 * MB)

+/* + * fill_buf_param: "fill_buf" benchmark parameters + * @buf_size: Size (in bytes) of buffer used in benchmark. + * "fill_buf" allocates and initializes buffer of + * @buf_size. User can change value via command line. + * @memflush: If false the buffer will not be flushed after + * allocation and initialization, otherwise the + * buffer will be flushed. User can change value via + * command line (via integers with 0 interpreted as + * false and anything else as true). + */ +struct fill_buf_param { + size_t buf_size; + bool memflush; +}; + /* * user_params: User supplied parameters * @cpu: CPU number to which the benchmark will be bound to * @bits: Number of bits used for cache allocation size * @benchmark_cmd: Benchmark command to run during (some of the) tests + * @fill_buf: Pointer to user provided parameters for "fill_buf", + * NULL if user did not provide parameters and test + * specific defaults should be used. */ struct user_params { int cpu; int bits; const char *benchmark_cmd[BENCHMARK_ARGS]; + const struct fill_buf_param *fill_buf; };

/* @@ -87,21 +107,29 @@ struct resctrl_test { * @init: Callback function to initialize test environment * @setup: Callback function to setup per test run environment * @measure: Callback that performs the measurement (a single test) + * @fill_buf: Parameters for default "fill_buf" benchmark. + * Initialized with user provided parameters, possibly + * adapted to be relevant to the test. If user does + * not provide parameters for "fill_buf" nor a + * replacement benchmark then initialized with defaults + * appropriate for test. NULL if user provided + * benchmark. */ struct resctrl_val_param { - const char *ctrlgrp; - const char *mongrp; - char filename[64]; - unsigned long mask; - int num_of_runs; - int (*init)(const struct resctrl_val_param *param, - int domain_id); - int (*setup)(const struct resctrl_test *test, - const struct user_params *uparams, - struct resctrl_val_param *param); - int (*measure)(const struct user_params *uparams, - struct resctrl_val_param *param, - pid_t bm_pid); + const char *ctrlgrp; + const char *mongrp; + char filename[64]; + unsigned long mask; + int num_of_runs; + int (*init)(const struct resctrl_val_param *param, + int domain_id); + int (*setup)(const struct resctrl_test *test, + const struct user_params *uparams, + struct resctrl_val_param *param); + int (*measure)(const struct user_params *uparams, + struct resctrl_val_param *param, + pid_t bm_pid); + struct fill_buf_param *fill_buf; };

struct perf_event_read { @@ -138,10 +166,10 @@ int write_schemata(const char *ctrlgrp, char *schemata, int cpu_no, int write_bm_pid_to_resctrl(pid_t bm_pid, const char *ctrlgrp, const char *mongrp); int perf_event_open(struct perf_event_attr *hw_event, pid_t pid, int cpu, int group_fd, unsigned long flags); -unsigned char *alloc_buffer(size_t buf_size, int memflush); +unsigned char *alloc_buffer(size_t buf_size, bool memflush); void mem_flush(unsigned char *buf, size_t buf_size); void fill_cache_read(unsigned char *buf, size_t buf_size, bool once); -int run_fill_buf(size_t buf_size, int memflush); +int run_fill_buf(size_t buf_size, bool memflush); int initialize_read_mem_bw_imc(void); int measure_read_mem_bw(const struct user_params *uparams, struct resctrl_val_param *param, pid_t bm_pid); @@ -149,7 +177,6 @@ void initialize_mem_bw_resctrl(const struct resctrl_val_param *param, int domain_id); int resctrl_val(const struct resctrl_test *test, const struct user_params *uparams, - const char * const *benchmark_cmd, struct resctrl_val_param *param); unsigned long create_bit_mask(unsigned int start, unsigned int len); unsigned int count_contiguous_bits(unsigned long val, unsigned int *start); diff --git a/tools/testing/selftests/resctrl/resctrl_tests.c b/tools/testing/selftests/resctrl/resctrl_tests.c index 0f91c475b255..24daf76b4039 100644 --- a/tools/testing/selftests/resctrl/resctrl_tests.c +++ b/tools/testing/selftests/resctrl/resctrl_tests.c @@ -148,6 +148,78 @@ static void run_single_test(const struct resctrl_test *test, const struct user_p test_cleanup(test); }

+/* + * Allocate and initialize a struct fill_buf_param with user provided + * (via "-b fill_buf <fill_buf parameters>") parameters. + * + * Use defaults (that may not be appropriate for all tests) for any + * fill_buf parameters omitted by the user. + * + * Historically it may have been possible for user space to provide + * additional parameters, "operation" ("read" vs "write") in + * benchmark_cmd[3] and "once" (run "once" or until terminated) in + * benchmark_cmd[4]. Changing these parameters have never been + * supported with the default of "read" operation and running until + * terminated built into the tests. Any unsupported values for + * (original) "fill_buf" parameters are treated as failure. + * + * Return: On failure, forcibly exits the test on any parsing failure, + * returns NULL if no parsing needed (user did not actually provide + * "-b fill_buf"). + * On success, returns pointer to newly allocated and fully + * initialized struct fill_buf_param that caller must free. + */ +static struct fill_buf_param *alloc_fill_buf_param(struct user_params *uparams) +{ + struct fill_buf_param *fill_param = NULL; + char *endptr = NULL; + + if (!uparams->benchmark_cmd[0] || strcmp(uparams->benchmark_cmd[0], "fill_buf")) + return NULL; + + fill_param = malloc(sizeof(*fill_param)); + if (!fill_param) + ksft_exit_skip("Unable to allocate memory for fill_buf parameters.\n"); + + if (uparams->benchmark_cmd[1] && *uparams->benchmark_cmd[1] != '\0') { + errno = 0; + fill_param->buf_size = strtoul(uparams->benchmark_cmd[1], &endptr, 10); + if (errno || *endptr != '\0') { + free(fill_param); + ksft_exit_skip("Unable to parse benchmark buffer size.\n"); + } + } else { + fill_param->buf_size = DEFAULT_SPAN; + } + + if (uparams->benchmark_cmd[2] && *uparams->benchmark_cmd[2] != '\0') { + errno = 0; + fill_param->memflush = strtol(uparams->benchmark_cmd[2], &endptr, 10) != 0; + if (errno || *endptr != '\0') { + free(fill_param); + ksft_exit_skip("Unable to parse benchmark memflush parameter.\n"); + } + } else { + fill_param->memflush = true; + } + + if (uparams->benchmark_cmd[3] && *uparams->benchmark_cmd[3] != '\0') { + if (strcmp(uparams->benchmark_cmd[3], "0")) { + free(fill_param); + ksft_exit_skip("Only read operations supported.\n"); + } + } + + if (uparams->benchmark_cmd[4] && *uparams->benchmark_cmd[4] != '\0') { + if (strcmp(uparams->benchmark_cmd[4], "false")) { + free(fill_param); + ksft_exit_skip("fill_buf is required to run until termination.\n"); + } + } + + return fill_param; +} + static void init_user_params(struct user_params *uparams) { memset(uparams, 0, sizeof(*uparams)); @@ -158,11 +230,11 @@ static void init_user_params(struct user_params *uparams)

int main(int argc, char **argv) { + struct fill_buf_param *fill_param = NULL; int tests = ARRAY_SIZE(resctrl_tests); bool test_param_seen = false; struct user_params uparams; - char *span_str = NULL; - int ret, c, i; + int c, i;

init_user_params(&uparams);

@@ -239,6 +311,10 @@ int main(int argc, char **argv) } last_arg:

+ fill_param = alloc_fill_buf_param(&uparams); + if (fill_param) + uparams.fill_buf = fill_param; + ksft_print_header();

/* @@ -257,32 +333,11 @@ int main(int argc, char **argv)

filter_dmesg();

- if (!uparams.benchmark_cmd[0]) { - /* If no benchmark is given by "-b" argument, use fill_buf. */ - uparams.benchmark_cmd[0] = "fill_buf"; - ret = asprintf(&span_str, "%u", DEFAULT_SPAN); - if (ret < 0) - ksft_exit_fail_msg("Out of memory!\n"); - uparams.benchmark_cmd[1] = span_str; - uparams.benchmark_cmd[2] = "1"; - /* - * Third parameter was previously used for "operation" - * (read/write) of which only (now default) "read"/"0" - * works. - * Fourth parameter was previously used to indicate - * how long "fill_buf" should run for, with "false" - * ("fill_buf" will keep running until terminated) - * the only option that works. - */ - uparams.benchmark_cmd[3] = NULL; - uparams.benchmark_cmd[4] = NULL; - } - ksft_set_plan(tests);

for (i = 0; i < ARRAY_SIZE(resctrl_tests); i++) run_single_test(resctrl_tests[i], &uparams);

- free(span_str); + free(fill_param); ksft_finished(); } diff --git a/tools/testing/selftests/resctrl/resctrl_val.c b/tools/testing/selftests/resctrl/resctrl_val.c index c4ebf70a46ef..00b3808d3bca 100644 --- a/tools/testing/selftests/resctrl/resctrl_val.c +++ b/tools/testing/selftests/resctrl/resctrl_val.c @@ -535,6 +535,11 @@ int measure_read_mem_bw(const struct user_params *uparams, return ret; }

+struct benchmark_info { + const struct user_params *uparams; + struct resctrl_val_param *param; +}; + /* * run_benchmark - Run a specified benchmark or fill_buf (default benchmark) * in specified signal. Direct benchmark stdio to /dev/null. @@ -544,12 +549,11 @@ int measure_read_mem_bw(const struct user_params *uparams, */ static void run_benchmark(int signum, siginfo_t *info, void *ucontext) { - char **benchmark_cmd; - int ret, memflush; - size_t span; + struct benchmark_info *benchmark_info = info->si_ptr; + const struct user_params *uparams = benchmark_info->uparams; + struct resctrl_val_param *param = benchmark_info->param; FILE *fp; - - benchmark_cmd = info->si_ptr; + int ret;

/* * Direct stdio of child to /dev/null, so that only parent writes to @@ -561,16 +565,13 @@ static void run_benchmark(int signum, siginfo_t *info, void *ucontext) parent_exit(ppid); }

- if (strcmp(benchmark_cmd[0], "fill_buf") == 0) { - /* Execute default fill_buf benchmark */ - span = strtoul(benchmark_cmd[1], NULL, 10); - memflush = atoi(benchmark_cmd[2]); - - if (run_fill_buf(span, memflush)) + if (param->fill_buf) { + if (run_fill_buf(param->fill_buf->buf_size, + param->fill_buf->memflush)) fprintf(stderr, "Error in running fill buffer\n"); - } else { + } else if (uparams->benchmark_cmd[0]) { /* Execute specified benchmark */ - ret = execvp(benchmark_cmd[0], benchmark_cmd); + ret = execvp(uparams->benchmark_cmd[0], (char **)uparams->benchmark_cmd); if (ret) ksft_perror("execvp"); } @@ -585,16 +586,15 @@ static void run_benchmark(int signum, siginfo_t *info, void *ucontext) * the benchmark * @test: test information structure * @uparams: user supplied parameters - * @benchmark_cmd: benchmark command and its arguments * @param: parameters passed to resctrl_val() * * Return: 0 when the test was run, < 0 on error. */ int resctrl_val(const struct resctrl_test *test, const struct user_params *uparams, - const char * const *benchmark_cmd, struct resctrl_val_param *param) { + struct benchmark_info benchmark_info; struct sigaction sigact; int ret = 0, pipefd[2]; char pipe_message = 0; @@ -610,6 +610,9 @@ int resctrl_val(const struct resctrl_test *test, return ret; }

+ benchmark_info.uparams = uparams; + benchmark_info.param = param; + /* * If benchmark wasn't successfully started by child, then child should * kill parent, so save parent's pid @@ -671,13 +674,7 @@ int resctrl_val(const struct resctrl_test *test,

ksft_print_msg("Benchmark PID: %d\n", (int)bm_pid);

- /* - * The cast removes constness but nothing mutates benchmark_cmd within - * the context of this process. At the receiving process, it becomes - * argv, which is mutable, on exec() but that's after fork() so it - * doesn't matter for the process running the tests. - */ - value.sival_ptr = (void *)benchmark_cmd; + value.sival_ptr = (void *)&benchmark_info;

/* Taskset benchmark to specified cpu */ ret = taskset_benchmark(bm_pid, uparams->cpu, NULL);

The CMT, MBA, and MBM tests rely on the resctrl_val() wrapper to start and run a benchmark while providing test specific flows via callbacks to do test specific configuration and measurements.

At a high level, the resctrl_val() flow is: a) Start by fork()ing a child process that installs a signal handler for SIGUSR1 that, on receipt of SIGUSR1, will start running a benchmark. b) Assign the child process created in (a) to the resctrl control and monitoring group that dictates the memory and cache allocations with which the process can run and will contain all resctrl monitoring data of that process. c) Once parent and child are considered "ready" (determined via a message over a pipe) the parent signals the child (via SIGUSR1) to start the benchmark, waits one second for the benchmark to run, and then starts collecting monitoring data for the tests, potentially also changing allocation configuration depending on the various test callbacks.

A problem with the above flow is the "black box" view of the benchmark that is combined with an arbitrarily chosen "wait one second" before measurements start. No matter what the benchmark does, it is given one second to initialize before measurements start.

The default benchmark "fill_buf" consists of two parts, first it prepares a buffer (allocate, initialize, then flush), then it reads from the buffer (in unpredictable ways) until terminated. Depending on the system and the size of the buffer, the first "prepare" part may not be complete by the time the one second delay expires. Test measurements may thus start before the work needing to be measured runs.

Split the default benchmark into its "prepare" and "runtime" parts and simplify the resctrl_val() wrapper while doing so. This same split cannot be done for the user provided benchmark (without a user interface change), so the current behavior is maintained for user provided benchmark.

Assign the test itself to the control and monitoring group and run the "prepare" part of the benchmark in this context, ensuring it runs with required cache and memory bandwidth allocations. With the benchmark preparation complete it is only needed to fork() the "runtime" part of the benchmark (or entire user provided benchmark).

Keep the "wait one second" delay before measurements start. For the default "fill_buf" benchmark this time now covers only the "runtime" portion that needs to be measured. For the user provided benchmark this delay maintains current behavior.

Signed-off-by: Reinette Chatre reinette.chatre@intel.com Reviewed-by: Ilpo Järvinen ilpo.jarvinen@linux.intel.com --- Changes since V2: - Add Ilpo's Reviewed-by tag.

Changes since V1: - Keep the fflush(stdout) before fork() to avoid duplicate messages. (Ilpo) - Re-order in series to that the new behavior is introduced after issues with existing behavior is addressed. --- tools/testing/selftests/resctrl/fill_buf.c | 15 -- tools/testing/selftests/resctrl/resctrl.h | 1 - tools/testing/selftests/resctrl/resctrl_val.c | 195 +++++------------- 3 files changed, 50 insertions(+), 161 deletions(-)

diff --git a/tools/testing/selftests/resctrl/fill_buf.c b/tools/testing/selftests/resctrl/fill_buf.c index 39545f9369e8..380cc35f10c6 100644 --- a/tools/testing/selftests/resctrl/fill_buf.c +++ b/tools/testing/selftests/resctrl/fill_buf.c @@ -129,18 +129,3 @@ unsigned char *alloc_buffer(size_t buf_size, bool memflush)

return buf; } - -int run_fill_buf(size_t buf_size, bool memflush) -{ - unsigned char *buf; - - buf = alloc_buffer(buf_size, memflush); - if (!buf) - return -1; - - fill_cache_read(buf, buf_size, false); - - free(buf); - - return 0; -} diff --git a/tools/testing/selftests/resctrl/resctrl.h b/tools/testing/selftests/resctrl/resctrl.h index c9336f9c2cae..032cd9ebd761 100644 --- a/tools/testing/selftests/resctrl/resctrl.h +++ b/tools/testing/selftests/resctrl/resctrl.h @@ -169,7 +169,6 @@ int perf_event_open(struct perf_event_attr *hw_event, pid_t pid, int cpu, unsigned char *alloc_buffer(size_t buf_size, bool memflush); void mem_flush(unsigned char *buf, size_t buf_size); void fill_cache_read(unsigned char *buf, size_t buf_size, bool once); -int run_fill_buf(size_t buf_size, bool memflush); int initialize_read_mem_bw_imc(void); int measure_read_mem_bw(const struct user_params *uparams, struct resctrl_val_param *param, pid_t bm_pid); diff --git a/tools/testing/selftests/resctrl/resctrl_val.c b/tools/testing/selftests/resctrl/resctrl_val.c index 00b3808d3bca..7c08e936572d 100644 --- a/tools/testing/selftests/resctrl/resctrl_val.c +++ b/tools/testing/selftests/resctrl/resctrl_val.c @@ -373,7 +373,7 @@ static int get_mem_bw_resctrl(FILE *fp, unsigned long *mbm_total) return 0; }

-static pid_t bm_pid, ppid; +static pid_t bm_pid;

void ctrlc_handler(int signum, siginfo_t *info, void *ptr) { @@ -431,13 +431,6 @@ void signal_handler_unregister(void) } }

-static void parent_exit(pid_t ppid) -{ - kill(ppid, SIGKILL); - umount_resctrlfs(); - exit(EXIT_FAILURE); -} - /* * print_results_bw: the memory bandwidth results are stored in a file * @filename: file that stores the results @@ -535,52 +528,6 @@ int measure_read_mem_bw(const struct user_params *uparams, return ret; }

-struct benchmark_info { - const struct user_params *uparams; - struct resctrl_val_param *param; -}; - -/* - * run_benchmark - Run a specified benchmark or fill_buf (default benchmark) - * in specified signal. Direct benchmark stdio to /dev/null. - * @signum: signal number - * @info: signal info - * @ucontext: user context in signal handling - */ -static void run_benchmark(int signum, siginfo_t *info, void *ucontext) -{ - struct benchmark_info *benchmark_info = info->si_ptr; - const struct user_params *uparams = benchmark_info->uparams; - struct resctrl_val_param *param = benchmark_info->param; - FILE *fp; - int ret; - - /* - * Direct stdio of child to /dev/null, so that only parent writes to - * stdio (console) - */ - fp = freopen("/dev/null", "w", stdout); - if (!fp) { - ksft_perror("Unable to direct benchmark status to /dev/null"); - parent_exit(ppid); - } - - if (param->fill_buf) { - if (run_fill_buf(param->fill_buf->buf_size, - param->fill_buf->memflush)) - fprintf(stderr, "Error in running fill buffer\n"); - } else if (uparams->benchmark_cmd[0]) { - /* Execute specified benchmark */ - ret = execvp(uparams->benchmark_cmd[0], (char **)uparams->benchmark_cmd); - if (ret) - ksft_perror("execvp"); - } - - fclose(stdout); - ksft_print_msg("Unable to run specified benchmark\n"); - parent_exit(ppid); -} - /* * resctrl_val: execute benchmark and measure memory bandwidth on * the benchmark @@ -594,12 +541,11 @@ int resctrl_val(const struct resctrl_test *test, const struct user_params *uparams, struct resctrl_val_param *param) { - struct benchmark_info benchmark_info; - struct sigaction sigact; - int ret = 0, pipefd[2]; - char pipe_message = 0; - union sigval value; + unsigned char *buf = NULL; + cpu_set_t old_affinity; int domain_id; + int ret = 0; + pid_t ppid;

if (strcmp(param->filename, "") == 0) sprintf(param->filename, "stdio"); @@ -610,108 +556,65 @@ int resctrl_val(const struct resctrl_test *test, return ret; }

- benchmark_info.uparams = uparams; - benchmark_info.param = param; - - /* - * If benchmark wasn't successfully started by child, then child should - * kill parent, so save parent's pid - */ ppid = getpid();

- if (pipe(pipefd)) { - ksft_perror("Unable to create pipe"); + /* Taskset test to specified CPU. */ + ret = taskset_benchmark(ppid, uparams->cpu, &old_affinity); + if (ret) + return ret; + + /* Write test to specified control & monitoring group in resctrl FS. */ + ret = write_bm_pid_to_resctrl(ppid, param->ctrlgrp, param->mongrp); + if (ret) + goto reset_affinity;

- return -1; + if (param->init) { + ret = param->init(param, domain_id); + if (ret) + goto reset_affinity; }

/* - * Fork to start benchmark, save child's pid so that it can be killed - * when needed + * If not running user provided benchmark, run the default + * "fill_buf". First phase of "fill_buf" is to prepare the + * buffer that the benchmark will operate on. No measurements + * are needed during this phase and prepared memory will be + * passed to next part of benchmark via copy-on-write thus + * no impact on the benchmark that relies on reading from + * memory only. */ + if (param->fill_buf) { + buf = alloc_buffer(param->fill_buf->buf_size, + param->fill_buf->memflush); + if (!buf) { + ret = -ENOMEM; + goto reset_affinity; + } + } + fflush(stdout); bm_pid = fork(); if (bm_pid == -1) { + ret = -errno; ksft_perror("Unable to fork"); - - return -1; + goto free_buf; }

+ /* + * What needs to be measured runs in separate process until + * terminated. + */ if (bm_pid == 0) { - /* - * Mask all signals except SIGUSR1, parent uses SIGUSR1 to - * start benchmark - */ - sigfillset(&sigact.sa_mask); - sigdelset(&sigact.sa_mask, SIGUSR1); - - sigact.sa_sigaction = run_benchmark; - sigact.sa_flags = SA_SIGINFO; - - /* Register for "SIGUSR1" signal from parent */ - if (sigaction(SIGUSR1, &sigact, NULL)) { - ksft_perror("Can't register child for signal"); - parent_exit(ppid); - } - - /* Tell parent that child is ready */ - close(pipefd[0]); - pipe_message = 1; - if (write(pipefd[1], &pipe_message, sizeof(pipe_message)) < - sizeof(pipe_message)) { - ksft_perror("Failed signaling parent process"); - close(pipefd[1]); - return -1; - } - close(pipefd[1]); - - /* Suspend child until delivery of "SIGUSR1" from parent */ - sigsuspend(&sigact.sa_mask); - - ksft_perror("Child is done"); - parent_exit(ppid); + if (param->fill_buf) + fill_cache_read(buf, param->fill_buf->buf_size, false); + else if (uparams->benchmark_cmd[0]) + execvp(uparams->benchmark_cmd[0], (char **)uparams->benchmark_cmd); + exit(EXIT_SUCCESS); }

ksft_print_msg("Benchmark PID: %d\n", (int)bm_pid);

- value.sival_ptr = (void *)&benchmark_info; - - /* Taskset benchmark to specified cpu */ - ret = taskset_benchmark(bm_pid, uparams->cpu, NULL); - if (ret) - goto out; - - /* Write benchmark to specified control&monitoring grp in resctrl FS */ - ret = write_bm_pid_to_resctrl(bm_pid, param->ctrlgrp, param->mongrp); - if (ret) - goto out; - - if (param->init) { - ret = param->init(param, domain_id); - if (ret) - goto out; - } - - /* Parent waits for child to be ready. */ - close(pipefd[1]); - while (pipe_message != 1) { - if (read(pipefd[0], &pipe_message, sizeof(pipe_message)) < - sizeof(pipe_message)) { - ksft_perror("Failed reading message from child process"); - close(pipefd[0]); - goto out; - } - } - close(pipefd[0]); - - /* Signal child to start benchmark */ - if (sigqueue(bm_pid, SIGUSR1, value) == -1) { - ksft_perror("sigqueue SIGUSR1 to child"); - ret = -1; - goto out; - } - - /* Give benchmark enough time to fully run */ + /* Give benchmark enough time to fully run. */ sleep(1);

/* Test runs until the callback setup() tells the test to stop. */ @@ -729,8 +632,10 @@ int resctrl_val(const struct resctrl_test *test, break; }

-out: kill(bm_pid, SIGKILL); - +free_buf: + free(buf); +reset_affinity: + taskset_restore(ppid, &old_affinity); return ret; }

The MBA test incrementally throttles memory bandwidth, each time followed by a comparison between the memory bandwidth observed by the performance counters and resctrl respectively.

While a comparison between performance counters and resctrl is generally appropriate, they do not have an identical view of memory bandwidth. For example RAS features or memory performance features that generate memory traffic may drive accesses that are counted differently by performance counters and MBM respectively, for instance generating "overhead" traffic which is not counted against any specific RMID. As a ratio, this different view of memory bandwidth becomes more apparent at low memory bandwidths.

It is not practical to enable/disable the various features that may generate memory bandwidth to give performance counters and resctrl an identical view. Instead, do not compare performance counters and resctrl view of memory bandwidth when the memory bandwidth is low.

Bandwidth throttling behaves differently across platforms so it is not appropriate to drop measurement data simply based on the throttling level. Instead, use a threshold of 750MiB that has been observed to support adequate comparison between performance counters and resctrl.

Signed-off-by: Reinette Chatre reinette.chatre@intel.com Reviewed-by: Ilpo Järvinen ilpo.jarvinen@linux.intel.com --- Changes since V2: - Add Ilpo's Reviewed-by tag.

Changes since V1: - Fix code alignment and spacing. - Modify flow to use "continue" instead of "break" now that earlier changes decreases throttling. - Expand comment of define to elaborate causes of discrepancy between performance counters and MBM. --- tools/testing/selftests/resctrl/mba_test.c | 7 +++++++ tools/testing/selftests/resctrl/resctrl.h | 10 ++++++++++ 2 files changed, 17 insertions(+)

diff --git a/tools/testing/selftests/resctrl/mba_test.c b/tools/testing/selftests/resctrl/mba_test.c index bf37f3555660..5b4f0aa7a3a4 100644 --- a/tools/testing/selftests/resctrl/mba_test.c +++ b/tools/testing/selftests/resctrl/mba_test.c @@ -98,6 +98,13 @@ static bool show_mba_info(unsigned long *bw_imc, unsigned long *bw_resc)

avg_bw_imc = sum_bw_imc / (NUM_OF_RUNS - 1); avg_bw_resc = sum_bw_resc / (NUM_OF_RUNS - 1); + if (avg_bw_imc < THROTTLE_THRESHOLD || avg_bw_resc < THROTTLE_THRESHOLD) { + ksft_print_msg("Bandwidth below threshold (%d MiB). Dropping results from MBA schemata %u.\n", + THROTTLE_THRESHOLD, + ALLOCATION_MIN + ALLOCATION_STEP * allocation); + continue; + } + avg_diff = (float)labs(avg_bw_resc - avg_bw_imc) / avg_bw_imc; avg_diff_per = (int)(avg_diff * 100);

diff --git a/tools/testing/selftests/resctrl/resctrl.h b/tools/testing/selftests/resctrl/resctrl.h index a553fe975938..dab1953fc7a0 100644 --- a/tools/testing/selftests/resctrl/resctrl.h +++ b/tools/testing/selftests/resctrl/resctrl.h @@ -43,6 +43,16 @@

#define MINIMUM_SPAN (250 * MB)

+/* + * Memory bandwidth (in MiB) below which the bandwidth comparisons + * between iMC and resctrl are considered unreliable. For example RAS + * features or memory performance features that generate memory traffic + * may drive accesses that are counted differently by performance counters + * and MBM respectively, for instance generating "overhead" traffic which + * is not counted against any specific RMID. + */ +#define THROTTLE_THRESHOLD 750 + /* * fill_buf_param: "fill_buf" benchmark parameters * @buf_size: Size (in bytes) of buffer used in benchmark.

The Memory Bandwidth Allocation (MBA) test iterates through all possible MBA allocations, from 10% (ALLOCATION_MIN) to 100% (ALLOCATION_MAX) with increments of 10% (ALLOCATION_STEP) at each iteration. During each iteration the test measures the actual memory bandwidth NUM_OF_RUNS times to determine the impact of MBA on actual memory bandwidth.

After the MBA test completes all the memory bandwidth measurements are parsed into an array. One array for resctrl Memory Bandwidth Monitoring (MBM) measurements and one array for the Integrated Memory Controller (iMC) measurements. Each array has a hardcoded size of 1024 that is large enough to hold the current test data, but this hardcoded value makes the implementation difficult to understand. It will not be clear that this array needs to be reconsidered if any of the test parameters are changed.

Replace the magic constant as array size with the test parameters the array size depends on.

Reported-by: Ilpo Järvinen ilpo.jarvinen@linux.intel.com Closes: https://lore.kernel.org/all/45af2a8c-517d-8f0d-137d-ad0f3f6a3c68@linux.intel... Signed-off-by: Reinette Chatre reinette.chatre@intel.com Reviewed-by: Ilpo Järvinen ilpo.jarvinen@linux.intel.com --- Changes since V3: - Add Ilpo's Reviewed-by tag.

Changes since V2: - New patch. --- tools/testing/selftests/resctrl/mba_test.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/resctrl/mba_test.c b/tools/testing/selftests/resctrl/mba_test.c index 4e6645b172e3..536d9089d2f6 100644 --- a/tools/testing/selftests/resctrl/mba_test.c +++ b/tools/testing/selftests/resctrl/mba_test.c @@ -127,8 +127,9 @@ static bool show_mba_info(unsigned long *bw_imc, unsigned long *bw_resc)

static int check_results(void) { + unsigned long bw_resc[NUM_OF_RUNS * ALLOCATION_MAX / ALLOCATION_STEP]; + unsigned long bw_imc[NUM_OF_RUNS * ALLOCATION_MAX / ALLOCATION_STEP]; char *token_array[8], output[] = RESULT_FILE_NAME, temp[512]; - unsigned long bw_imc[1024], bw_resc[1024]; int runs; FILE *fp;