The current netconsole implementation allocates a static buffer for extradata (userdata + sysdata) with a fixed size of MAX_EXTRADATA_ENTRY_LEN * MAX_EXTRADATA_ITEMS bytes for every target, regardless of whether userspace actually uses this feature. This forces us to keep MAX_EXTRADATA_ITEMS small (16), which is restrictive for users who need to attach more metadata to their log messages.
This patch series enables dynamic allocation of the userdata buffer, allowing it to grow on-demand based on actual usage. The series:
1. Refactors send_fragmented_body() to simplify handling of separated userdata and sysdata (patch 1/4) 2. Splits userdata and sysdata into separate buffers (patch 2/4) 3. Implements dynamic allocation for the userdata buffer (patch 3/4) 4. Increases MAX_USERDATA_ITEMS from 16 to 256 now that we can do so without memory waste (patch 4/4)
Benefits: - No memory waste when userdata is not used - Targets that use userdata only consume what they need - Users can attach significantly more metadata without impacting systems that don't use this feature
Signed-off-by: Gustavo Luiz Duarte gustavold@gmail.com --- Changes in v3: - Split calculating the lentgh of the formatted userdata string into a separate function calc_userdata_len(). - Exit update_userdata() immediately if we hit WARN due to too many userdata entries. - Use offset instead of len to save userdata_length in update_userdata() - Link to v2: https://lore.kernel.org/r/20251113-netconsole_dynamic_extradata-v2-0-18cf7fe...
Changes in v2: - Added null pointer checks for userdata and sysdata buffers - Added MAX_SYSDATA_ITEMS to enum sysdata_feature - Moved code out of ifdef in send_msg_no_fragmentation() - Renamed variables in send_fragmented_body() to make it easier to reason about the code - Link to v1: https://lore.kernel.org/r/20251105-netconsole_dynamic_extradata-v1-0-142890b...
--- Gustavo Luiz Duarte (4): netconsole: Simplify send_fragmented_body() netconsole: Split userdata and sysdata netconsole: Dynamic allocation of userdata buffer netconsole: Increase MAX_USERDATA_ITEMS
drivers/net/netconsole.c | 386 +++++++++++---------- .../selftests/drivers/net/netcons_overflow.sh | 2 +- 2 files changed, 195 insertions(+), 193 deletions(-) --- base-commit: 45a1cd8346ca245a1ca475b26eb6ceb9d8b7c6f0 change-id: 20251007-netconsole_dynamic_extradata-21bd9d726568
Best regards,
Refactor send_fragmented_body() to use separate offset tracking for msgbody, and extradata instead of complex conditional logic. The previous implementation used boolean flags and calculated offsets which made the code harder to follow.
The new implementation maintains independent offset counters (msgbody_offset, extradata_offset) and processes each section sequentially, making the data flow more straightforward and the code easier to maintain.
This is a preparatory refactoring with no functional changes, which will allow easily splitting extradata_complete into separate userdata and sysdata buffers in the next patch.
Signed-off-by: Gustavo Luiz Duarte gustavold@gmail.com Reviewed-by: Breno Leitao leitao@debian.org --- drivers/net/netconsole.c | 111 ++++++++++++++++++++--------------------------- 1 file changed, 46 insertions(+), 65 deletions(-)
diff --git a/drivers/net/netconsole.c b/drivers/net/netconsole.c index bb6e03a92956..5fe5896d6ff5 100644 --- a/drivers/net/netconsole.c +++ b/drivers/net/netconsole.c @@ -1560,89 +1560,70 @@ static void append_release(char *buf) }
static void send_fragmented_body(struct netconsole_target *nt, - const char *msgbody, int header_len, + const char *msgbody_ptr, int header_len, int msgbody_len, int extradata_len) { - int sent_extradata, preceding_bytes; - const char *extradata = NULL; - int body_len, offset = 0; + const char *extradata_ptr = NULL; + int data_len, data_sent = 0; + int extradata_offset = 0; + int msgbody_offset = 0;
#ifdef CONFIG_NETCONSOLE_DYNAMIC - extradata = nt->extradata_complete; + extradata_ptr = nt->extradata_complete; #endif + if (WARN_ON_ONCE(!extradata_ptr && extradata_len != 0)) + return;
- /* body_len represents the number of bytes that will be sent. This is + /* data_len represents the number of bytes that will be sent. This is * bigger than MAX_PRINT_CHUNK, thus, it will be split in multiple * packets */ - body_len = msgbody_len + extradata_len; + data_len = msgbody_len + extradata_len;
/* In each iteration of the while loop below, we send a packet - * containing the header and a portion of the body. The body is + * containing the header and a portion of the data. The data is * composed of two parts: msgbody and extradata. We keep track of how - * many bytes have been sent so far using the offset variable, which - * ranges from 0 to the total length of the body. + * many bytes have been sent so far using the data_sent variable, which + * ranges from 0 to the total bytes to be sent. */ - while (offset < body_len) { - int this_header = header_len; - bool msgbody_written = false; - int this_offset = 0; + while (data_sent < data_len) { + int extradata_left = extradata_len - extradata_offset; + int msgbody_left = msgbody_len - msgbody_offset; + int buf_offset = 0; int this_chunk = 0;
- this_header += scnprintf(nt->buf + this_header, - MAX_PRINT_CHUNK - this_header, - ",ncfrag=%d/%d;", offset, - body_len); - - /* Not all msgbody data has been written yet */ - if (offset < msgbody_len) { - this_chunk = min(msgbody_len - offset, - MAX_PRINT_CHUNK - this_header); - if (WARN_ON_ONCE(this_chunk <= 0)) - return; - memcpy(nt->buf + this_header, msgbody + offset, - this_chunk); - this_offset += this_chunk; + /* header is already populated in nt->buf, just append to it */ + buf_offset = header_len; + + buf_offset += scnprintf(nt->buf + buf_offset, + MAX_PRINT_CHUNK - buf_offset, + ",ncfrag=%d/%d;", data_sent, + data_len); + + /* append msgbody first */ + this_chunk = min(msgbody_left, MAX_PRINT_CHUNK - buf_offset); + memcpy(nt->buf + buf_offset, msgbody_ptr + msgbody_offset, + this_chunk); + msgbody_offset += this_chunk; + buf_offset += this_chunk; + data_sent += this_chunk; + + /* after msgbody, append extradata */ + if (extradata_ptr && extradata_left) { + this_chunk = min(extradata_left, + MAX_PRINT_CHUNK - buf_offset); + memcpy(nt->buf + buf_offset, + extradata_ptr + extradata_offset, this_chunk); + extradata_offset += this_chunk; + buf_offset += this_chunk; + data_sent += this_chunk; }
- /* msgbody was finally written, either in the previous - * messages and/or in the current buf. Time to write - * the extradata. - */ - msgbody_written |= offset + this_offset >= msgbody_len; - - /* Msg body is fully written and there is pending extradata to - * write, append extradata in this chunk - */ - if (msgbody_written && offset + this_offset < body_len) { - /* Track how much user data was already sent. First - * time here, sent_userdata is zero - */ - sent_extradata = (offset + this_offset) - msgbody_len; - /* offset of bytes used in current buf */ - preceding_bytes = this_chunk + this_header; - - if (WARN_ON_ONCE(sent_extradata < 0)) - return; - - this_chunk = min(extradata_len - sent_extradata, - MAX_PRINT_CHUNK - preceding_bytes); - if (WARN_ON_ONCE(this_chunk < 0)) - /* this_chunk could be zero if all the previous - * message used all the buffer. This is not a - * problem, extradata will be sent in the next - * iteration - */ - return; - - memcpy(nt->buf + this_header + this_offset, - extradata + sent_extradata, - this_chunk); - this_offset += this_chunk; - } + /* if all is good, send the packet out */ + if (WARN_ON_ONCE(data_sent > data_len)) + return;
- send_udp(nt, nt->buf, this_header + this_offset); - offset += this_offset; + send_udp(nt, nt->buf, buf_offset); } }
Separate userdata and sysdata into distinct buffers to enable independent management. Previously, both were stored in a single extradata_complete buffer with a fixed size that accommodated both types of data.
This separation allows: - userdata to grow dynamically (in subsequent patch) - sysdata to remain in a small static buffer - removal of complex entry counting logic that tracked both types together
The split also simplifies the code by eliminating the need to check total entry count across both userdata and sysdata when enabling features, which allows to drop holding su_mutex on sysdata_*_enabled_store().
No functional change in this patch, just structural preparation for dynamic userdata allocation.
Signed-off-by: Gustavo Luiz Duarte gustavold@gmail.com Reviewed-by: Breno Leitao leitao@debian.org --- drivers/net/netconsole.c | 215 +++++++++++++++++++++-------------------------- 1 file changed, 98 insertions(+), 117 deletions(-)
diff --git a/drivers/net/netconsole.c b/drivers/net/netconsole.c index 5fe5896d6ff5..1bd811714322 100644 --- a/drivers/net/netconsole.c +++ b/drivers/net/netconsole.c @@ -50,7 +50,7 @@ MODULE_LICENSE("GPL"); /* The number 3 comes from userdata entry format characters (' ', '=', '\n') */ #define MAX_EXTRADATA_NAME_LEN (MAX_EXTRADATA_ENTRY_LEN - \ MAX_EXTRADATA_VALUE_LEN - 3) -#define MAX_EXTRADATA_ITEMS 16 +#define MAX_USERDATA_ITEMS 16 #define MAX_PRINT_CHUNK 1000
static char config[MAX_PARAM_LENGTH]; @@ -115,6 +115,8 @@ enum sysdata_feature { SYSDATA_RELEASE = BIT(2), /* Include a per-target message ID as part of sysdata */ SYSDATA_MSGID = BIT(3), + /* Sentinel: highest bit position */ + MAX_SYSDATA_ITEMS = 4, };
/** @@ -122,8 +124,9 @@ enum sysdata_feature { * @list: Links this target into the target_list. * @group: Links us into the configfs subsystem hierarchy. * @userdata_group: Links to the userdata configfs hierarchy - * @extradata_complete: Cached, formatted string of append - * @userdata_length: String length of usedata in extradata_complete. + * @userdata: Cached, formatted string of append + * @userdata_length: String length of userdata. + * @sysdata: Cached, formatted string of append * @sysdata_fields: Sysdata features enabled. * @msgcounter: Message sent counter. * @stats: Packet send stats for the target. Used for debugging. @@ -152,8 +155,10 @@ struct netconsole_target { #ifdef CONFIG_NETCONSOLE_DYNAMIC struct config_group group; struct config_group userdata_group; - char extradata_complete[MAX_EXTRADATA_ENTRY_LEN * MAX_EXTRADATA_ITEMS]; + char userdata[MAX_EXTRADATA_ENTRY_LEN * MAX_USERDATA_ITEMS]; size_t userdata_length; + char sysdata[MAX_EXTRADATA_ENTRY_LEN * MAX_SYSDATA_ITEMS]; + /* bit-wise with sysdata_feature bits */ u32 sysdata_fields; /* protected by target_list_lock */ @@ -802,28 +807,14 @@ static ssize_t remote_ip_store(struct config_item *item, const char *buf, return ret; }
-/* Count number of entries we have in extradata. - * This is important because the extradata_complete only supports - * MAX_EXTRADATA_ITEMS entries. Before enabling any new {user,sys}data - * feature, number of entries needs to checked for available space. +/* Count number of entries we have in userdata. + * This is important because userdata only supports MAX_USERDATA_ITEMS + * entries. Before enabling any new userdata feature, number of entries needs + * to checked for available space. */ -static size_t count_extradata_entries(struct netconsole_target *nt) +static size_t count_userdata_entries(struct netconsole_target *nt) { - size_t entries; - - /* Userdata entries */ - entries = list_count_nodes(&nt->userdata_group.cg_children); - /* Plus sysdata entries */ - if (nt->sysdata_fields & SYSDATA_CPU_NR) - entries += 1; - if (nt->sysdata_fields & SYSDATA_TASKNAME) - entries += 1; - if (nt->sysdata_fields & SYSDATA_RELEASE) - entries += 1; - if (nt->sysdata_fields & SYSDATA_MSGID) - entries += 1; - - return entries; + return list_count_nodes(&nt->userdata_group.cg_children); }
static ssize_t remote_mac_store(struct config_item *item, const char *buf, @@ -894,13 +885,13 @@ static void update_userdata(struct netconsole_target *nt)
/* Clear the current string in case the last userdatum was deleted */ nt->userdata_length = 0; - nt->extradata_complete[0] = 0; + nt->userdata[0] = 0;
list_for_each(entry, &nt->userdata_group.cg_children) { struct userdatum *udm_item; struct config_item *item;
- if (child_count >= MAX_EXTRADATA_ITEMS) { + if (child_count >= MAX_USERDATA_ITEMS) { spin_unlock_irqrestore(&target_list_lock, flags); WARN_ON_ONCE(1); return; @@ -914,11 +905,11 @@ static void update_userdata(struct netconsole_target *nt) if (strnlen(udm_item->value, MAX_EXTRADATA_VALUE_LEN) == 0) continue;
- /* This doesn't overflow extradata_complete since it will write - * one entry length (1/MAX_EXTRADATA_ITEMS long), entry count is + /* This doesn't overflow userdata since it will write + * one entry length (1/MAX_USERDATA_ITEMS long), entry count is * checked to not exceed MAX items with child_count above */ - nt->userdata_length += scnprintf(&nt->extradata_complete[nt->userdata_length], + nt->userdata_length += scnprintf(&nt->userdata[nt->userdata_length], MAX_EXTRADATA_ENTRY_LEN, " %s=%s\n", item->ci_name, udm_item->value); } @@ -962,7 +953,7 @@ static void disable_sysdata_feature(struct netconsole_target *nt, enum sysdata_feature feature) { nt->sysdata_fields &= ~feature; - nt->extradata_complete[nt->userdata_length] = 0; + nt->sysdata[0] = 0; }
static ssize_t sysdata_msgid_enabled_store(struct config_item *item, @@ -982,12 +973,6 @@ static ssize_t sysdata_msgid_enabled_store(struct config_item *item, if (msgid_enabled == curr) goto unlock_ok;
- if (msgid_enabled && - count_extradata_entries(nt) >= MAX_EXTRADATA_ITEMS) { - ret = -ENOSPC; - goto unlock; - } - if (msgid_enabled) nt->sysdata_fields |= SYSDATA_MSGID; else @@ -995,7 +980,6 @@ static ssize_t sysdata_msgid_enabled_store(struct config_item *item,
unlock_ok: ret = strnlen(buf, count); -unlock: mutex_unlock(&dynamic_netconsole_mutex); mutex_unlock(&netconsole_subsys.su_mutex); return ret; @@ -1018,12 +1002,6 @@ static ssize_t sysdata_release_enabled_store(struct config_item *item, if (release_enabled == curr) goto unlock_ok;
- if (release_enabled && - count_extradata_entries(nt) >= MAX_EXTRADATA_ITEMS) { - ret = -ENOSPC; - goto unlock; - } - if (release_enabled) nt->sysdata_fields |= SYSDATA_RELEASE; else @@ -1031,7 +1009,6 @@ static ssize_t sysdata_release_enabled_store(struct config_item *item,
unlock_ok: ret = strnlen(buf, count); -unlock: mutex_unlock(&dynamic_netconsole_mutex); mutex_unlock(&netconsole_subsys.su_mutex); return ret; @@ -1054,12 +1031,6 @@ static ssize_t sysdata_taskname_enabled_store(struct config_item *item, if (taskname_enabled == curr) goto unlock_ok;
- if (taskname_enabled && - count_extradata_entries(nt) >= MAX_EXTRADATA_ITEMS) { - ret = -ENOSPC; - goto unlock; - } - if (taskname_enabled) nt->sysdata_fields |= SYSDATA_TASKNAME; else @@ -1067,7 +1038,6 @@ static ssize_t sysdata_taskname_enabled_store(struct config_item *item,
unlock_ok: ret = strnlen(buf, count); -unlock: mutex_unlock(&dynamic_netconsole_mutex); mutex_unlock(&netconsole_subsys.su_mutex); return ret; @@ -1092,27 +1062,16 @@ static ssize_t sysdata_cpu_nr_enabled_store(struct config_item *item, /* no change requested */ goto unlock_ok;
- if (cpu_nr_enabled && - count_extradata_entries(nt) >= MAX_EXTRADATA_ITEMS) { - /* user wants the new feature, but there is no space in the - * buffer. - */ - ret = -ENOSPC; - goto unlock; - } - if (cpu_nr_enabled) nt->sysdata_fields |= SYSDATA_CPU_NR; else - /* This is special because extradata_complete might have - * remaining data from previous sysdata, and it needs to be - * cleaned. + /* This is special because sysdata might have remaining data + * from previous sysdata, and it needs to be cleaned. */ disable_sysdata_feature(nt, SYSDATA_CPU_NR);
unlock_ok: ret = strnlen(buf, count); -unlock: mutex_unlock(&dynamic_netconsole_mutex); mutex_unlock(&netconsole_subsys.su_mutex); return ret; @@ -1156,7 +1115,7 @@ static struct config_item *userdatum_make_item(struct config_group *group,
ud = to_userdata(&group->cg_item); nt = userdata_to_target(ud); - if (count_extradata_entries(nt) >= MAX_EXTRADATA_ITEMS) + if (count_userdata_entries(nt) >= MAX_USERDATA_ITEMS) return ERR_PTR(-ENOSPC);
udm = kzalloc(sizeof(*udm), GFP_KERNEL); @@ -1363,22 +1322,21 @@ static void populate_configfs_item(struct netconsole_target *nt,
static int sysdata_append_cpu_nr(struct netconsole_target *nt, int offset) { - /* Append cpu=%d at extradata_complete after userdata str */ - return scnprintf(&nt->extradata_complete[offset], + return scnprintf(&nt->sysdata[offset], MAX_EXTRADATA_ENTRY_LEN, " cpu=%u\n", raw_smp_processor_id()); }
static int sysdata_append_taskname(struct netconsole_target *nt, int offset) { - return scnprintf(&nt->extradata_complete[offset], + return scnprintf(&nt->sysdata[offset], MAX_EXTRADATA_ENTRY_LEN, " taskname=%s\n", current->comm); }
static int sysdata_append_release(struct netconsole_target *nt, int offset) { - return scnprintf(&nt->extradata_complete[offset], + return scnprintf(&nt->sysdata[offset], MAX_EXTRADATA_ENTRY_LEN, " release=%s\n", init_utsname()->release); } @@ -1386,46 +1344,36 @@ static int sysdata_append_release(struct netconsole_target *nt, int offset) static int sysdata_append_msgid(struct netconsole_target *nt, int offset) { wrapping_assign_add(nt->msgcounter, 1); - return scnprintf(&nt->extradata_complete[offset], + return scnprintf(&nt->sysdata[offset], MAX_EXTRADATA_ENTRY_LEN, " msgid=%u\n", nt->msgcounter); }
/* - * prepare_extradata - append sysdata at extradata_complete in runtime + * prepare_sysdata - append sysdata in runtime * @nt: target to send message to */ -static int prepare_extradata(struct netconsole_target *nt) +static int prepare_sysdata(struct netconsole_target *nt) { - int extradata_len; - - /* userdata was appended when configfs write helper was called - * by update_userdata(). - */ - extradata_len = nt->userdata_length; + int sysdata_len = 0;
if (!nt->sysdata_fields) goto out;
if (nt->sysdata_fields & SYSDATA_CPU_NR) - extradata_len += sysdata_append_cpu_nr(nt, extradata_len); + sysdata_len += sysdata_append_cpu_nr(nt, sysdata_len); if (nt->sysdata_fields & SYSDATA_TASKNAME) - extradata_len += sysdata_append_taskname(nt, extradata_len); + sysdata_len += sysdata_append_taskname(nt, sysdata_len); if (nt->sysdata_fields & SYSDATA_RELEASE) - extradata_len += sysdata_append_release(nt, extradata_len); + sysdata_len += sysdata_append_release(nt, sysdata_len); if (nt->sysdata_fields & SYSDATA_MSGID) - extradata_len += sysdata_append_msgid(nt, extradata_len); + sysdata_len += sysdata_append_msgid(nt, sysdata_len);
- WARN_ON_ONCE(extradata_len > - MAX_EXTRADATA_ENTRY_LEN * MAX_EXTRADATA_ITEMS); + WARN_ON_ONCE(sysdata_len > + MAX_EXTRADATA_ENTRY_LEN * MAX_SYSDATA_ITEMS);
out: - return extradata_len; -} -#else /* CONFIG_NETCONSOLE_DYNAMIC not set */ -static int prepare_extradata(struct netconsole_target *nt) -{ - return 0; + return sysdata_len; } #endif /* CONFIG_NETCONSOLE_DYNAMIC */
@@ -1527,11 +1475,13 @@ static void send_msg_no_fragmentation(struct netconsole_target *nt, int msg_len, int release_len) { - const char *extradata = NULL; + const char *userdata = NULL; + const char *sysdata = NULL; const char *release;
#ifdef CONFIG_NETCONSOLE_DYNAMIC - extradata = nt->extradata_complete; + userdata = nt->userdata; + sysdata = nt->sysdata; #endif
if (release_len) { @@ -1543,10 +1493,15 @@ static void send_msg_no_fragmentation(struct netconsole_target *nt, memcpy(nt->buf, msg, msg_len); }
- if (extradata) + if (userdata) msg_len += scnprintf(&nt->buf[msg_len], - MAX_PRINT_CHUNK - msg_len, - "%s", extradata); + MAX_PRINT_CHUNK - msg_len, "%s", + userdata); + + if (sysdata) + msg_len += scnprintf(&nt->buf[msg_len], + MAX_PRINT_CHUNK - msg_len, "%s", + sysdata);
send_udp(nt, nt->buf, msg_len); } @@ -1561,33 +1516,45 @@ static void append_release(char *buf)
static void send_fragmented_body(struct netconsole_target *nt, const char *msgbody_ptr, int header_len, - int msgbody_len, int extradata_len) + int msgbody_len, int sysdata_len) { - const char *extradata_ptr = NULL; + const char *userdata_ptr = NULL; + const char *sysdata_ptr = NULL; int data_len, data_sent = 0; - int extradata_offset = 0; + int userdata_offset = 0; + int sysdata_offset = 0; int msgbody_offset = 0; + int userdata_len = 0;
#ifdef CONFIG_NETCONSOLE_DYNAMIC - extradata_ptr = nt->extradata_complete; + userdata_ptr = nt->userdata; + sysdata_ptr = nt->sysdata; + userdata_len = nt->userdata_length; #endif - if (WARN_ON_ONCE(!extradata_ptr && extradata_len != 0)) + if (WARN_ON_ONCE(!userdata_ptr && userdata_len != 0)) + return; + + if (WARN_ON_ONCE(!sysdata_ptr && sysdata_len != 0)) return;
/* data_len represents the number of bytes that will be sent. This is * bigger than MAX_PRINT_CHUNK, thus, it will be split in multiple * packets */ - data_len = msgbody_len + extradata_len; + data_len = msgbody_len + userdata_len + sysdata_len;
/* In each iteration of the while loop below, we send a packet * containing the header and a portion of the data. The data is - * composed of two parts: msgbody and extradata. We keep track of how - * many bytes have been sent so far using the data_sent variable, which - * ranges from 0 to the total bytes to be sent. + * composed of three parts: msgbody, userdata, and sysdata. + * We keep track of how many bytes have been sent from each part using + * the *_offset variables. + * We keep track of how many bytes have been sent overall using the + * data_sent variable, which ranges from 0 to the total bytes to be + * sent. */ while (data_sent < data_len) { - int extradata_left = extradata_len - extradata_offset; + int userdata_left = userdata_len - userdata_offset; + int sysdata_left = sysdata_len - sysdata_offset; int msgbody_left = msgbody_len - msgbody_offset; int buf_offset = 0; int this_chunk = 0; @@ -1608,13 +1575,24 @@ static void send_fragmented_body(struct netconsole_target *nt, buf_offset += this_chunk; data_sent += this_chunk;
- /* after msgbody, append extradata */ - if (extradata_ptr && extradata_left) { - this_chunk = min(extradata_left, + /* after msgbody, append userdata */ + if (userdata_ptr && userdata_left) { + this_chunk = min(userdata_left, MAX_PRINT_CHUNK - buf_offset); memcpy(nt->buf + buf_offset, - extradata_ptr + extradata_offset, this_chunk); - extradata_offset += this_chunk; + userdata_ptr + userdata_offset, this_chunk); + userdata_offset += this_chunk; + buf_offset += this_chunk; + data_sent += this_chunk; + } + + /* after userdata, append sysdata */ + if (sysdata_ptr && sysdata_left) { + this_chunk = min(sysdata_left, + MAX_PRINT_CHUNK - buf_offset); + memcpy(nt->buf + buf_offset, + sysdata_ptr + sysdata_offset, this_chunk); + sysdata_offset += this_chunk; buf_offset += this_chunk; data_sent += this_chunk; } @@ -1631,7 +1609,7 @@ static void send_msg_fragmented(struct netconsole_target *nt, const char *msg, int msg_len, int release_len, - int extradata_len) + int sysdata_len) { int header_len, msgbody_len; const char *msgbody; @@ -1660,7 +1638,7 @@ static void send_msg_fragmented(struct netconsole_target *nt, * will be replaced */ send_fragmented_body(nt, msgbody, header_len, msgbody_len, - extradata_len); + sysdata_len); }
/** @@ -1676,19 +1654,22 @@ static void send_msg_fragmented(struct netconsole_target *nt, static void send_ext_msg_udp(struct netconsole_target *nt, const char *msg, int msg_len) { + int userdata_len = 0; int release_len = 0; - int extradata_len; - - extradata_len = prepare_extradata(nt); + int sysdata_len = 0;
+#ifdef CONFIG_NETCONSOLE_DYNAMIC + sysdata_len = prepare_sysdata(nt); + userdata_len = nt->userdata_length; +#endif if (nt->release) release_len = strlen(init_utsname()->release) + 1;
- if (msg_len + release_len + extradata_len <= MAX_PRINT_CHUNK) + if (msg_len + release_len + sysdata_len + userdata_len <= MAX_PRINT_CHUNK) return send_msg_no_fragmentation(nt, msg, msg_len, release_len);
return send_msg_fragmented(nt, msg, msg_len, release_len, - extradata_len); + sysdata_len); }
static void write_ext_msg(struct console *con, const char *msg,
The userdata buffer in struct netconsole_target is currently statically allocated with a size of MAX_USERDATA_ITEMS * MAX_EXTRADATA_ENTRY_LEN (16 * 256 = 4096 bytes). This wastes memory when userdata entries are not used or when only a few entries are configured, which is common in typical usage scenarios. It also forces us to keep MAX_USERDATA_ITEMS small to limit the memory wasted.
Change the userdata buffer from a static array to a dynamically allocated pointer. The buffer is now allocated on-demand in update_userdata() whenever userdata entries are added, modified, or removed via configfs. The implementation calculates the exact size needed for all current userdata entries, allocates a new buffer of that size, formats the entries into it, and atomically swaps it with the old buffer.
This approach provides several benefits: - Memory efficiency: Targets with no userdata use zero bytes instead of 4KB, and targets with userdata only allocate what they need; - Scalability: Makes it practical to increase MAX_USERDATA_ITEMS to a much larger value without imposing a fixed memory cost on every target; - No hot-path overhead: Allocation occurs during configuration (write to configfs), not during message transmission
If memory allocation fails during userdata update, -ENOMEM is returned to userspace through the configfs attribute write operation.
The sysdata buffer remains statically allocated since it has a smaller fixed size (MAX_SYSDATA_ITEMS * MAX_EXTRADATA_ENTRY_LEN = 4 * 256 = 1024 bytes) and its content length is less predictable.
Signed-off-by: Gustavo Luiz Duarte gustavold@gmail.com --- drivers/net/netconsole.c | 100 +++++++++++++++++++++++++++++++++-------------- 1 file changed, 70 insertions(+), 30 deletions(-)
diff --git a/drivers/net/netconsole.c b/drivers/net/netconsole.c index 1bd811714322..0b350f82d915 100644 --- a/drivers/net/netconsole.c +++ b/drivers/net/netconsole.c @@ -155,7 +155,7 @@ struct netconsole_target { #ifdef CONFIG_NETCONSOLE_DYNAMIC struct config_group group; struct config_group userdata_group; - char userdata[MAX_EXTRADATA_ENTRY_LEN * MAX_USERDATA_ITEMS]; + char *userdata; size_t userdata_length; char sysdata[MAX_EXTRADATA_ENTRY_LEN * MAX_SYSDATA_ITEMS];
@@ -875,45 +875,77 @@ static ssize_t userdatum_value_show(struct config_item *item, char *buf) return sysfs_emit(buf, "%s\n", &(to_userdatum(item)->value[0])); }
-static void update_userdata(struct netconsole_target *nt) +/* Navigate configfs and calculate the lentgh of the formatted string + * representing userdata. + * Must be called holding netconsole_subsys.su_mutex + */ +static int calc_userdata_len(struct netconsole_target *nt) { + struct userdatum *udm_item; + struct config_item *item; struct list_head *entry; - int child_count = 0; - unsigned long flags; + int len = 0;
- spin_lock_irqsave(&target_list_lock, flags); + list_for_each(entry, &nt->userdata_group.cg_children) { + item = container_of(entry, struct config_item, ci_entry); + udm_item = to_userdatum(item); + /* Skip userdata with no value set */ + if (udm_item->value[0]) { + len += snprintf(NULL, 0, " %s=%s\n", item->ci_name, + udm_item->value); + } + } + return len; +}
- /* Clear the current string in case the last userdatum was deleted */ - nt->userdata_length = 0; - nt->userdata[0] = 0; +static int update_userdata(struct netconsole_target *nt) +{ + struct userdatum *udm_item; + struct config_item *item; + struct list_head *entry; + char *old_buf = NULL; + char *new_buf = NULL; + unsigned long flags; + int offset = 0; + int len;
- list_for_each(entry, &nt->userdata_group.cg_children) { - struct userdatum *udm_item; - struct config_item *item; + /* Calculate required buffer size */ + len = calc_userdata_len(nt);
- if (child_count >= MAX_USERDATA_ITEMS) { - spin_unlock_irqrestore(&target_list_lock, flags); - WARN_ON_ONCE(1); - return; - } - child_count++; + if (WARN_ON_ONCE(len > MAX_EXTRADATA_ENTRY_LEN * MAX_USERDATA_ITEMS)) + return -ENOSPC; + + /* Allocate new buffer */ + if (len) { + new_buf = kmalloc(len + 1, GFP_KERNEL); + if (!new_buf) + return -ENOMEM; + }
+ /* Write userdata to new buffer */ + list_for_each(entry, &nt->userdata_group.cg_children) { item = container_of(entry, struct config_item, ci_entry); udm_item = to_userdatum(item); - /* Skip userdata with no value set */ - if (strnlen(udm_item->value, MAX_EXTRADATA_VALUE_LEN) == 0) - continue; - - /* This doesn't overflow userdata since it will write - * one entry length (1/MAX_USERDATA_ITEMS long), entry count is - * checked to not exceed MAX items with child_count above - */ - nt->userdata_length += scnprintf(&nt->userdata[nt->userdata_length], - MAX_EXTRADATA_ENTRY_LEN, " %s=%s\n", - item->ci_name, udm_item->value); + if (udm_item->value[0]) { + offset += scnprintf(&new_buf[offset], len + 1 - offset, + " %s=%s\n", item->ci_name, + udm_item->value); + } } + + WARN_ON_ONCE(offset != len); + + /* Switch to new buffer and free old buffer */ + spin_lock_irqsave(&target_list_lock, flags); + old_buf = nt->userdata; + nt->userdata = new_buf; + nt->userdata_length = offset; spin_unlock_irqrestore(&target_list_lock, flags); + + kfree(old_buf); + + return 0; }
static ssize_t userdatum_value_store(struct config_item *item, const char *buf, @@ -937,7 +969,9 @@ static ssize_t userdatum_value_store(struct config_item *item, const char *buf,
ud = to_userdata(item->ci_parent); nt = userdata_to_target(ud); - update_userdata(nt); + ret = update_userdata(nt); + if (ret < 0) + goto out_unlock; ret = count; out_unlock: mutex_unlock(&dynamic_netconsole_mutex); @@ -1193,7 +1227,10 @@ static struct configfs_attribute *netconsole_target_attrs[] = {
static void netconsole_target_release(struct config_item *item) { - kfree(to_target(item)); + struct netconsole_target *nt = to_target(item); + + kfree(nt->userdata); + kfree(nt); }
static struct configfs_item_operations netconsole_target_item_ops = { @@ -1874,6 +1911,9 @@ static struct netconsole_target *alloc_param_target(char *target_config, static void free_param_target(struct netconsole_target *nt) { netpoll_cleanup(&nt->np); +#ifdef CONFIG_NETCONSOLE_DYNAMIC + kfree(nt->userdata); +#endif kfree(nt); }
On Wed, Nov 19, 2025 at 04:14:51PM -0800, Gustavo Luiz Duarte wrote:
The userdata buffer in struct netconsole_target is currently statically allocated with a size of MAX_USERDATA_ITEMS * MAX_EXTRADATA_ENTRY_LEN (16 * 256 = 4096 bytes). This wastes memory when userdata entries are not used or when only a few entries are configured, which is common in typical usage scenarios. It also forces us to keep MAX_USERDATA_ITEMS small to limit the memory wasted.
Change the userdata buffer from a static array to a dynamically allocated pointer. The buffer is now allocated on-demand in update_userdata() whenever userdata entries are added, modified, or removed via configfs. The implementation calculates the exact size needed for all current userdata entries, allocates a new buffer of that size, formats the entries into it, and atomically swaps it with the old buffer.
This approach provides several benefits:
- Memory efficiency: Targets with no userdata use zero bytes instead of 4KB, and targets with userdata only allocate what they need;
- Scalability: Makes it practical to increase MAX_USERDATA_ITEMS to a much larger value without imposing a fixed memory cost on every target;
- No hot-path overhead: Allocation occurs during configuration (write to configfs), not during message transmission
If memory allocation fails during userdata update, -ENOMEM is returned to userspace through the configfs attribute write operation.
The sysdata buffer remains statically allocated since it has a smaller fixed size (MAX_SYSDATA_ITEMS * MAX_EXTRADATA_ENTRY_LEN = 4 * 256 = 1024 bytes) and its content length is less predictable.
Signed-off-by: Gustavo Luiz Duarte gustavold@gmail.com
Reviewed-by: Breno Leitao leitao@debian.org
Increase MAX_USERDATA_ITEMS from 16 to 256 entries now that the userdata buffer is allocated dynamically.
The previous limit of 16 was necessary because the buffer was statically allocated for all targets. With dynamic allocation, we can support more entries without wasting memory on targets that don't use userdata.
This allows users to attach more metadata to their netconsole messages, which is useful for complex debugging and logging scenarios.
Also update the testcase accordingly.
Signed-off-by: Gustavo Luiz Duarte gustavold@gmail.com Reviewed-by: Breno Leitao leitao@debian.org --- drivers/net/netconsole.c | 2 +- tools/testing/selftests/drivers/net/netcons_overflow.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/netconsole.c b/drivers/net/netconsole.c index 0b350f82d9156..9cb4dfc242f5f 100644 --- a/drivers/net/netconsole.c +++ b/drivers/net/netconsole.c @@ -50,7 +50,7 @@ MODULE_LICENSE("GPL"); /* The number 3 comes from userdata entry format characters (' ', '=', '\n') */ #define MAX_EXTRADATA_NAME_LEN (MAX_EXTRADATA_ENTRY_LEN - \ MAX_EXTRADATA_VALUE_LEN - 3) -#define MAX_USERDATA_ITEMS 16 +#define MAX_USERDATA_ITEMS 256 #define MAX_PRINT_CHUNK 1000
static char config[MAX_PARAM_LENGTH]; diff --git a/tools/testing/selftests/drivers/net/netcons_overflow.sh b/tools/testing/selftests/drivers/net/netcons_overflow.sh index 29bad56448a24..06089643b7716 100755 --- a/tools/testing/selftests/drivers/net/netcons_overflow.sh +++ b/tools/testing/selftests/drivers/net/netcons_overflow.sh @@ -15,7 +15,7 @@ SCRIPTDIR=$(dirname "$(readlink -e "${BASH_SOURCE[0]}")")
source "${SCRIPTDIR}"/lib/sh/lib_netcons.sh # This is coming from netconsole code. Check for it in drivers/net/netconsole.c -MAX_USERDATA_ITEMS=16 +MAX_USERDATA_ITEMS=256
# Function to create userdata entries function create_userdata_max_entries() {
Hello:
This series was applied to netdev/net-next.git (main) by Jakub Kicinski kuba@kernel.org:
On Wed, 19 Nov 2025 16:14:48 -0800 you wrote:
The current netconsole implementation allocates a static buffer for extradata (userdata + sysdata) with a fixed size of MAX_EXTRADATA_ENTRY_LEN * MAX_EXTRADATA_ITEMS bytes for every target, regardless of whether userspace actually uses this feature. This forces us to keep MAX_EXTRADATA_ITEMS small (16), which is restrictive for users who need to attach more metadata to their log messages.
[...]
Here is the summary with links: - [net-next,v3,1/4] netconsole: Simplify send_fragmented_body() https://git.kernel.org/netdev/net-next/c/7279b718b431 - [net-next,v3,2/4] netconsole: Split userdata and sysdata https://git.kernel.org/netdev/net-next/c/9dc10f50c430 - [net-next,v3,3/4] netconsole: Dynamic allocation of userdata buffer https://git.kernel.org/netdev/net-next/c/eb83801af2dc - [net-next,v3,4/4] netconsole: Increase MAX_USERDATA_ITEMS https://git.kernel.org/netdev/net-next/c/5ad994534172
You are awesome, thank you!
linux-kselftest-mirror@lists.linaro.org
-
Breno Leitao -
Gustavo Luiz Duarte -
patchwork-bot+netdevbpf＠kernel.org