[TCWG CI] Regression caused by linux: net: ipa: rework gsi_channel_tx_update():
commit c5bddecbb97bcf0400354dc954cdbd89276e0ddb
Author: Alex Elder <elder(a)linaro.org>
net: ipa: rework gsi_channel_tx_update()
Results regressed to
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1:
-5
# build_abe qemu:
-2
# linux_n_obj:
31579
# linux build successful:
all
# First few build errors in logs:
from
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1:
-5
# build_abe qemu:
-2
# linux_n_obj:
31579
# linux build successful:
all
# linux boot successful:
boot
THIS IS THE END OF INTERESTING STUFF. BELOW ARE LINKS TO BUILDS, REPRODUCTION INSTRUCTIONS, AND THE RAW COMMIT.
This commit has regressed these CI configurations:
- tcwg_kernel/gnu-release-aarch64-next-allmodconfig
First_bad build: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-aarch64-next-a…
Last_good build: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-aarch64-next-a…
Baseline build: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-aarch64-next-a…
Even more details: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-aarch64-next-a…
Reproduce builds:
<cut>
mkdir investigate-linux-c5bddecbb97bcf0400354dc954cdbd89276e0ddb
cd investigate-linux-c5bddecbb97bcf0400354dc954cdbd89276e0ddb
# Fetch scripts
git clone https://git.linaro.org/toolchain/jenkins-scripts
# Fetch manifests and test.sh script
mkdir -p artifacts/manifests
curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-aarch64-next-a… --fail
curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-aarch64-next-a… --fail
curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-aarch64-next-a… --fail
chmod +x artifacts/test.sh
# Reproduce the baseline build (build all pre-requisites)
./jenkins-scripts/tcwg_kernel-build.sh @@ artifacts/manifests/build-baseline.sh
# Save baseline build state (which is then restored in artifacts/test.sh)
mkdir -p ./bisect
rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /linux/ ./ ./bisect/baseline/
cd linux
# Reproduce first_bad build
git checkout --detach c5bddecbb97bcf0400354dc954cdbd89276e0ddb
../artifacts/test.sh
# Reproduce last_good build
git checkout --detach dbad2fa71914a6a1a41bcba8bdc3e9213a982f82
../artifacts/test.sh
cd ..
</cut>
Full commit (up to 1000 lines):
<cut>
commit c5bddecbb97bcf0400354dc954cdbd89276e0ddb
Author: Alex Elder <elder(a)linaro.org>
Date: Mon Jun 13 12:17:59 2022 -0500
net: ipa: rework gsi_channel_tx_update()
Rename gsi_channel_tx_update() to be gsi_trans_tx_completed(), and
pass it just the transaction pointer, deriving the channel from the
transaction. Update the comments above the function to provide a
more concise description of how statistics for TX endpoints are
maintained and used.
Signed-off-by: Alex Elder <elder(a)linaro.org>
Signed-off-by: David S. Miller <davem(a)davemloft.net>
---
drivers/net/ipa/gsi.c | 50 +++++++++++++++++++++++---------------------------
1 file changed, 23 insertions(+), 27 deletions(-)
diff --git a/drivers/net/ipa/gsi.c b/drivers/net/ipa/gsi.c
index c2cafd9247a7..df8af1f00fc8 100644
--- a/drivers/net/ipa/gsi.c
+++ b/drivers/net/ipa/gsi.c
@@ -1021,40 +1021,36 @@ void gsi_trans_tx_queued(struct gsi_trans *trans)
}
/**
- * gsi_channel_tx_update() - Report completed TX transfers
- * @channel: Channel that has completed transmitting packets
- * @trans: Last transation known to be complete
+ * gsi_trans_tx_completed() - Report completed TX transactions
+ * @trans: TX channel transaction that has completed
*
- * Compute the number of transactions and bytes that have been transferred
- * over a TX channel since the given transaction was committed. Report this
- * information to the network stack.
+ * Report that a transaction on a TX channel has completed. At the time a
+ * transaction is committed, we record *in the transaction* its channel's
+ * committed transaction and byte counts. Transactions are completed in
+ * order, and the difference between the channel's byte/transaction count
+ * when the transaction was committed and when it completes tells us
+ * exactly how much data has been transferred while the transaction was
+ * pending.
*
- * At the time a transaction is committed, we record its channel's
- * committed transaction and byte counts *in the transaction*.
- * Completions are signaled by the hardware with an interrupt, and
- * we can determine the latest completed transaction at that time.
- *
- * The difference between the byte/transaction count recorded in
- * the transaction and the count last time we recorded a completion
- * tells us exactly how much data has been transferred between
- * completions.
- *
- * Calling this each time we learn of a newly-completed transaction
- * allows us to provide accurate information to the network stack
- * about how much work has been completed by the hardware at a given
- * point in time.
+ * We report this information to the network stack, which uses it to manage
+ * the rate at which data is sent to hardware.
*/
-static void
-gsi_channel_tx_update(struct gsi_channel *channel, struct gsi_trans *trans)
+static void gsi_trans_tx_completed(struct gsi_trans *trans)
{
- u64 trans_count = trans->trans_count - channel->compl_trans_count;
- u64 byte_count = trans->byte_count - channel->compl_byte_count;
+ u32 channel_id = trans->channel_id;
+ struct gsi *gsi = trans->gsi;
+ struct gsi_channel *channel;
+ u32 trans_count;
+ u32 byte_count;
+
+ channel = &gsi->channel[channel_id];
+ trans_count = trans->trans_count - channel->compl_trans_count;
+ byte_count = trans->byte_count - channel->compl_byte_count;
channel->compl_trans_count += trans_count;
channel->compl_byte_count += byte_count;
- ipa_gsi_channel_tx_completed(channel->gsi, gsi_channel_id(channel),
- trans_count, byte_count);
+ ipa_gsi_channel_tx_completed(gsi, channel_id, trans_count, byte_count);
}
/* Channel control interrupt handler */
@@ -1504,7 +1500,7 @@ static struct gsi_trans *gsi_channel_update(struct gsi_channel *channel)
* up the network stack.
*/
if (channel->toward_ipa)
- gsi_channel_tx_update(channel, trans);
+ gsi_trans_tx_completed(trans);
else
gsi_evt_ring_rx_update(evt_ring, index);
</cut>
[TCWG CI] Regression caused by linux: btrfs: Add the capability of getting commit stats in BTRFS:
commit 55bc1b4598a319f2bc8d558caf81244c2a15de3b
Author: Ioannis Angelakopoulos <iangelak(a)fb.com>
btrfs: Add the capability of getting commit stats in BTRFS
Results regressed to
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1:
-5
# build_abe qemu:
-2
# linux_n_obj:
20671
# First few build errors in logs:
# 00:15:03 fs/btrfs/transaction.c:2091:31: error: ‘struct btrfs_commit_stats’ has no member named ‘commit_counter’; did you mean ‘commit_count’?
# 00:15:03 make[2]: *** [fs/btrfs/transaction.o] Error 1
# 00:18:18 make[1]: *** [fs/btrfs] Error 2
# 00:19:58 make: *** [fs] Error 2
from
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1:
-5
# build_abe qemu:
-2
# linux_n_obj:
20762
# linux build successful:
all
THIS IS THE END OF INTERESTING STUFF. BELOW ARE LINKS TO BUILDS, REPRODUCTION INSTRUCTIONS, AND THE RAW COMMIT.
This commit has regressed these CI configurations:
- tcwg_kernel/gnu-release-arm-next-allyesconfig
First_bad build: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-arm-next-allye…
Last_good build: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-arm-next-allye…
Baseline build: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-arm-next-allye…
Even more details: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-arm-next-allye…
Reproduce builds:
<cut>
mkdir investigate-linux-55bc1b4598a319f2bc8d558caf81244c2a15de3b
cd investigate-linux-55bc1b4598a319f2bc8d558caf81244c2a15de3b
# Fetch scripts
git clone https://git.linaro.org/toolchain/jenkins-scripts
# Fetch manifests and test.sh script
mkdir -p artifacts/manifests
curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-arm-next-allye… --fail
curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-arm-next-allye… --fail
curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-release-arm-next-allye… --fail
chmod +x artifacts/test.sh
# Reproduce the baseline build (build all pre-requisites)
./jenkins-scripts/tcwg_kernel-build.sh @@ artifacts/manifests/build-baseline.sh
# Save baseline build state (which is then restored in artifacts/test.sh)
mkdir -p ./bisect
rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /linux/ ./ ./bisect/baseline/
cd linux
# Reproduce first_bad build
git checkout --detach 55bc1b4598a319f2bc8d558caf81244c2a15de3b
../artifacts/test.sh
# Reproduce last_good build
git checkout --detach bed5d797ab6d6268564a0c5353d2e41b87a28a6e
../artifacts/test.sh
cd ..
</cut>
Full commit (up to 1000 lines):
<cut>
commit 55bc1b4598a319f2bc8d558caf81244c2a15de3b
Author: Ioannis Angelakopoulos <iangelak(a)fb.com>
Date: Tue Jun 14 15:22:32 2022 -0700
btrfs: Add the capability of getting commit stats in BTRFS
First we add "struct btrfs_commit_stats" data structure under "fs_info"
in fs/btrfs/ctree.h to store the commit stats for BTRFS that will be
exposed through sysfs.
The stats exposed are: 1) The number of commits so far, 2) The duration of
the last commit in ms, 3) The maximum commit duration seen so far in ms
and 4) The total duration for all commits so far in ms.
The update of the commit stats occurs after the commit thread has gone
through all the logic that checks if there is another thread committing
at the same time. This means that we only account for actual commit work
in the commit stats we report and not the time the thread spends waiting
until it is ready to do the commit work.
Signed-off-by: Ioannis Angelakopoulos <iangelak(a)fb.com>
Signed-off-by: David Sterba <dsterba(a)suse.com>
---
fs/btrfs/ctree.h | 14 ++++++++++++++
fs/btrfs/transaction.c | 29 +++++++++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 13d74948c542..8f7eb2d344ce 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -660,6 +660,18 @@ enum btrfs_exclusive_operation {
BTRFS_EXCLOP_SWAP_ACTIVATE,
};
+/* Store data about transaction commits, exported via sysfs. */
+struct btrfs_commit_stats {
+ /* Total number of commits */
+ u64 commit_count;
+ /* The maximum commit duration so far */
+ u64 max_commit_dur;
+ /* The last commit duration */
+ u64 last_commit_dur;
+ /* The total commit duration */
+ u64 total_commit_dur;
+};
+
struct btrfs_fs_info {
u8 chunk_tree_uuid[BTRFS_UUID_SIZE];
unsigned long flags;
@@ -1069,6 +1081,8 @@ struct btrfs_fs_info {
spinlock_t zone_active_bgs_lock;
struct list_head zone_active_bgs;
+ struct btrfs_commit_stats commit_stats;
+
#ifdef CONFIG_BTRFS_FS_REF_VERIFY
spinlock_t ref_verify_lock;
struct rb_root block_tree;
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 06c0a958d114..9cb09aa05275 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -10,6 +10,7 @@
#include <linux/pagemap.h>
#include <linux/blkdev.h>
#include <linux/uuid.h>
+#include <linux/timekeeping.h>
#include "misc.h"
#include "ctree.h"
#include "disk-io.h"
@@ -2084,12 +2085,24 @@ static void add_pending_snapshot(struct btrfs_trans_handle *trans)
list_add(&trans->pending_snapshot->list, &cur_trans->pending_snapshots);
}
+static void update_commit_stats(struct btrfs_fs_info *fs_info,
+ ktime_t interval)
+{
+ fs_info->commit_stats.commit_counter += 1;
+ fs_info->commit_stats.last_commit_dur = interval;
+ fs_info->commit_stats.max_commit_dur = max_t(u64,
+ fs_info->commit_stats.max_commit_dur, interval);
+ fs_info->commit_stats.total_commit_dur += interval;
+}
+
int btrfs_commit_transaction(struct btrfs_trans_handle *trans)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_transaction *cur_trans = trans->transaction;
struct btrfs_transaction *prev_trans = NULL;
int ret;
+ ktime_t start_time;
+ ktime_t interval;
ASSERT(refcount_read(&trans->use_count) == 1);
@@ -2214,6 +2227,12 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)
}
}
+ /*
+ * Get the time spent on the work done by the commit thread and not
+ * the time spent waiting on a previous commit
+ */
+ start_time = ktime_get_ns();
+
extwriter_counter_dec(cur_trans, trans->type);
ret = btrfs_start_delalloc_flush(fs_info);
@@ -2455,6 +2474,8 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)
trace_btrfs_transaction_commit(fs_info);
+ interval = ktime_get_ns() - start_time;
+
btrfs_scrub_continue(fs_info);
if (current->journal_info == trans)
@@ -2462,6 +2483,14 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)
kmem_cache_free(btrfs_trans_handle_cachep, trans);
+ /*
+ * Protect the commit stats updates from concurrent updates through
+ * sysfs.
+ */
+ spin_lock(&fs_info->super_lock);
+ update_commit_stats(fs_info, interval);
+ spin_unlock(&fs_info->super_lock);
+
return ret;
unlock_reloc:
</cut>
[TCWG CI] Regression caused by linux: page_alloc: use vmalloc_huge for large system hash:
commit f2edd118d02dd11449b126f786f09749ca152ba5
Author: Song Liu <song(a)kernel.org>
page_alloc: use vmalloc_huge for large system hash
Results regressed to
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1:
-5
# build_abe qemu:
-2
# linux_n_obj:
546
# First few build errors in logs:
# 00:00:40 page_alloc.c:(.init.text+0xef2): undefined reference to `vmalloc_huge'
# 00:00:40 make: *** [Makefile:1158: vmlinux] Error 1
from
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1:
-5
# build_abe qemu:
-2
# linux_n_obj:
561
# linux build successful:
all
THIS IS THE END OF INTERESTING STUFF. BELOW ARE LINKS TO BUILDS, REPRODUCTION INSTRUCTIONS, AND THE RAW COMMIT.
This commit has regressed these CI configurations:
- tcwg_kernel/gnu-master-arm-mainline-allnoconfig
First_bad build: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al…
Last_good build: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al…
Baseline build: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al…
Even more details: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al…
Reproduce builds:
<cut>
mkdir investigate-linux-f2edd118d02dd11449b126f786f09749ca152ba5
cd investigate-linux-f2edd118d02dd11449b126f786f09749ca152ba5
# Fetch scripts
git clone https://git.linaro.org/toolchain/jenkins-scripts
# Fetch manifests and test.sh script
mkdir -p artifacts/manifests
curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al… --fail
curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al… --fail
curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al… --fail
chmod +x artifacts/test.sh
# Reproduce the baseline build (build all pre-requisites)
./jenkins-scripts/tcwg_kernel-build.sh @@ artifacts/manifests/build-baseline.sh
# Save baseline build state (which is then restored in artifacts/test.sh)
mkdir -p ./bisect
rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /linux/ ./ ./bisect/baseline/
cd linux
# Reproduce first_bad build
git checkout --detach f2edd118d02dd11449b126f786f09749ca152ba5
../artifacts/test.sh
# Reproduce last_good build
git checkout --detach 22da5264abf497a10a4ed629f07f4ba28a7ed5eb
../artifacts/test.sh
cd ..
</cut>
Full commit (up to 1000 lines):
<cut>
commit f2edd118d02dd11449b126f786f09749ca152ba5
Author: Song Liu <song(a)kernel.org>
Date: Fri Apr 15 09:44:11 2022 -0700
page_alloc: use vmalloc_huge for large system hash
Use vmalloc_huge() in alloc_large_system_hash() so that large system
hash (>= PMD_SIZE) could benefit from huge pages.
Note that vmalloc_huge only allocates huge pages for systems with
HAVE_ARCH_HUGE_VMALLOC.
Signed-off-by: Song Liu <song(a)kernel.org>
Reviewed-by: Christoph Hellwig <hch(a)lst.de>
Reviewed-by: Rik van Riel <riel(a)surriel.com>
Signed-off-by: Linus Torvalds <torvalds(a)linux-foundation.org>
---
mm/page_alloc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 33ca8cab21e6..0e42038382c1 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -8919,7 +8919,7 @@ void *__init alloc_large_system_hash(const char *tablename,
table = memblock_alloc_raw(size,
SMP_CACHE_BYTES);
} else if (get_order(size) >= MAX_ORDER || hashdist) {
- table = __vmalloc(size, gfp_flags);
+ table = vmalloc_huge(size, gfp_flags);
virt = true;
if (table)
huge = is_vm_area_hugepages(table);
</cut>