Hi Greg, Sasha,
I think we're finally at a good point to begin backporting the work I've done on random.c during the last 6 months. I've been maintaining branches for this incrementally as code has been merged into mainline, in order to make this moment easier than otherwise.
Assuming that Linus merges my PR for 5.19 [1] today, all of these patches are (or will be in a few hours) in Linus' tree. I've tried to backport most of the general scaffolding and design of the current state of random.c, while not backporting any new features or unusual functionality changes that might invite trouble. So, for example, the backports switch to using a cryptographic hash function, but they don't have changes like warning when the cycle counter is zero, attempting to use jitter on early uses of /dev/urandom, reseeding on suspend/hibernate notifications, or the vmgenid driver. Hopefully that strikes an okay balance between getting the core backported so that fixes are backportable, but not going too far by backporting new "nice to have" but unessential features.
In this git repo [2], there are three branches: linux-5.15.y, linux-5.17.y, and linux-5.18.y, which contain backports for everything up to and including [1].
You'll probably want to backport this to earlier kernels as well. Given that there hasn't been overly much work on the rng in the last few years, it shouldn't be too hard to take my 5.15.y branch and fill in the missing pieces there to bring it back. Given how much changes, you could probably just take every random.c change for backporting to before 5.15.
There is one snag, which is that some of the work I did during the 5.17 cycle depends on crypto I added for WireGuard, which landed in 5.6. So for 5.4 and prior, that will need backports. Fortunately, I've already done this in [3], in the branch backport-5.4.y, which I've kept up to date for a few years now. This occasion might mark the perfect excuse we've been waiting for to just backport WireGuard too to 5.4 (which might make the Android work a bit easier also) :-D.
Let me know if you have any questions, and feel free to poke me on IRC. And if all of the above sounds terrible to you, and you'd rather just not take any of this into stable, I guess that's a valid path to take too.
Regards, Jason
[1] https://lore.kernel.org/lkml/20220522214457.37108-1-Jason@zx2c4.com/ [2] https://git.kernel.org/pub/scm/linux/kernel/git/crng/random.git/ [3] https://git.kernel.org/pub/scm/linux/kernel/git/zx2c4/wireguard-linux.git/
On Mon, May 23, 2022 at 02:54:45PM +0200, Jason A. Donenfeld wrote:
Hi Greg, Sasha,
I think we're finally at a good point to begin backporting the work I've done on random.c during the last 6 months. I've been maintaining branches for this incrementally as code has been merged into mainline, in order to make this moment easier than otherwise.
Assuming that Linus merges my PR for 5.19 [1] today, all of these patches are (or will be in a few hours) in Linus' tree. I've tried to backport most of the general scaffolding and design of the current state of random.c, while not backporting any new features or unusual functionality changes that might invite trouble. So, for example, the backports switch to using a cryptographic hash function, but they don't have changes like warning when the cycle counter is zero, attempting to use jitter on early uses of /dev/urandom, reseeding on suspend/hibernate notifications, or the vmgenid driver. Hopefully that strikes an okay balance between getting the core backported so that fixes are backportable, but not going too far by backporting new "nice to have" but unessential features.
In this git repo [2], there are three branches: linux-5.15.y, linux-5.17.y, and linux-5.18.y, which contain backports for everything up to and including [1].
You'll probably want to backport this to earlier kernels as well. Given that there hasn't been overly much work on the rng in the last few years, it shouldn't be too hard to take my 5.15.y branch and fill in the missing pieces there to bring it back. Given how much changes, you could probably just take every random.c change for backporting to before 5.15.
There is one snag, which is that some of the work I did during the 5.17 cycle depends on crypto I added for WireGuard, which landed in 5.6. So for 5.4 and prior, that will need backports. Fortunately, I've already done this in [3], in the branch backport-5.4.y, which I've kept up to date for a few years now. This occasion might mark the perfect excuse we've been waiting for to just backport WireGuard too to 5.4 (which might make the Android work a bit easier also) :-D.
Let me know if you have any questions, and feel free to poke me on IRC. And if all of the above sounds terrible to you, and you'd rather just not take any of this into stable, I guess that's a valid path to take too.
Let me look at this later this week after we get this next round of stable kernels out. It's normally a nice calm period for the stable kernels so this might not be that bad.
thanks,
greg k-h
On Mon, May 23, 2022 at 03:36:02PM +0200, Greg KH wrote:
On Mon, May 23, 2022 at 02:54:45PM +0200, Jason A. Donenfeld wrote:
Hi Greg, Sasha,
I think we're finally at a good point to begin backporting the work I've done on random.c during the last 6 months. I've been maintaining branches for this incrementally as code has been merged into mainline, in order to make this moment easier than otherwise.
Assuming that Linus merges my PR for 5.19 [1] today, all of these patches are (or will be in a few hours) in Linus' tree. I've tried to backport most of the general scaffolding and design of the current state of random.c, while not backporting any new features or unusual functionality changes that might invite trouble. So, for example, the backports switch to using a cryptographic hash function, but they don't have changes like warning when the cycle counter is zero, attempting to use jitter on early uses of /dev/urandom, reseeding on suspend/hibernate notifications, or the vmgenid driver. Hopefully that strikes an okay balance between getting the core backported so that fixes are backportable, but not going too far by backporting new "nice to have" but unessential features.
In this git repo [2], there are three branches: linux-5.15.y, linux-5.17.y, and linux-5.18.y, which contain backports for everything up to and including [1].
You'll probably want to backport this to earlier kernels as well. Given that there hasn't been overly much work on the rng in the last few years, it shouldn't be too hard to take my 5.15.y branch and fill in the missing pieces there to bring it back. Given how much changes, you could probably just take every random.c change for backporting to before 5.15.
There is one snag, which is that some of the work I did during the 5.17 cycle depends on crypto I added for WireGuard, which landed in 5.6. So for 5.4 and prior, that will need backports. Fortunately, I've already done this in [3], in the branch backport-5.4.y, which I've kept up to date for a few years now. This occasion might mark the perfect excuse we've been waiting for to just backport WireGuard too to 5.4 (which might make the Android work a bit easier also) :-D.
Let me know if you have any questions, and feel free to poke me on IRC. And if all of the above sounds terrible to you, and you'd rather just not take any of this into stable, I guess that's a valid path to take too.
Let me look at this later this week after we get this next round of stable kernels out. It's normally a nice calm period for the stable kernels so this might not be that bad.
I get build errors in the 5.15 and 5.10 trees when applying these patches.
Here's the 5.10 error: In file included from ../crypto/testmgr.c:32: ../include/crypto/drbg.h:139:38: error: field 'random_ready' has incomplete type 139 | struct random_ready_callback random_ready; | ^~~~~~~~~~~~
And the same error in 5.15.
So obviously I can't take them, I'm doing a simple 'make allmodconfig' build for these trees on x86-64.
For 5.18 and 5.17, wow, lots of patches, but sure, let's see what happens, I've queued them up now.
thanks,
greg k-h
Hi Greg,
On Thu, May 26, 2022 at 4:44 PM Greg KH gregkh@linuxfoundation.org wrote:
I get build errors in the 5.15 and 5.10 trees when applying these patches.
Here's the 5.10 error: In file included from ../crypto/testmgr.c:32: ../include/crypto/drbg.h:139:38: error: field 'random_ready' has incomplete type 139 | struct random_ready_callback random_ready; | ^~~~~~~~~~~~
And the same error in 5.15.
So obviously I can't take them, I'm doing a simple 'make allmodconfig' build for these trees on x86-64.
Sorry about that. I've fixed the broken backport (of "random: replace custom notifier chain with standard one") and force pushed linux-5.10.y and linux-5.15.y branches.
Jason
On Thu, May 26, 2022 at 05:15:12PM +0200, Jason A. Donenfeld wrote:
Hi Greg,
On Thu, May 26, 2022 at 4:44 PM Greg KH gregkh@linuxfoundation.org wrote:
I get build errors in the 5.15 and 5.10 trees when applying these patches.
Here's the 5.10 error: In file included from ../crypto/testmgr.c:32: ../include/crypto/drbg.h:139:38: error: field 'random_ready' has incomplete type 139 | struct random_ready_callback random_ready; | ^~~~~~~~~~~~
And the same error in 5.15.
So obviously I can't take them, I'm doing a simple 'make allmodconfig' build for these trees on x86-64.
Sorry about that. I've fixed the broken backport (of "random: replace custom notifier chain with standard one") and force pushed linux-5.10.y and linux-5.15.y branches.
All now queued up.
These are big changes overall, but as you've broken them up into the original bits, they seem semi-sane. Let's see how the testing goes on them...
thanks,
greg k-h
On Mon, May 23, 2022 at 03:36:02PM +0200, Greg KH wrote:
On Mon, May 23, 2022 at 02:54:45PM +0200, Jason A. Donenfeld wrote:
Hi Greg, Sasha,
I think we're finally at a good point to begin backporting the work I've done on random.c during the last 6 months. I've been maintaining branches for this incrementally as code has been merged into mainline, in order to make this moment easier than otherwise.
Assuming that Linus merges my PR for 5.19 [1] today, all of these patches are (or will be in a few hours) in Linus' tree. I've tried to backport most of the general scaffolding and design of the current state of random.c, while not backporting any new features or unusual functionality changes that might invite trouble. So, for example, the backports switch to using a cryptographic hash function, but they don't have changes like warning when the cycle counter is zero, attempting to use jitter on early uses of /dev/urandom, reseeding on suspend/hibernate notifications, or the vmgenid driver. Hopefully that strikes an okay balance between getting the core backported so that fixes are backportable, but not going too far by backporting new "nice to have" but unessential features.
In this git repo [2], there are three branches: linux-5.15.y, linux-5.17.y, and linux-5.18.y, which contain backports for everything up to and including [1].
You'll probably want to backport this to earlier kernels as well. Given that there hasn't been overly much work on the rng in the last few years, it shouldn't be too hard to take my 5.15.y branch and fill in the missing pieces there to bring it back. Given how much changes, you could probably just take every random.c change for backporting to before 5.15.
There is one snag, which is that some of the work I did during the 5.17 cycle depends on crypto I added for WireGuard, which landed in 5.6. So for 5.4 and prior, that will need backports. Fortunately, I've already done this in [3], in the branch backport-5.4.y, which I've kept up to date for a few years now. This occasion might mark the perfect excuse we've been waiting for to just backport WireGuard too to 5.4 (which might make the Android work a bit easier also) :-D.
Let me know if you have any questions, and feel free to poke me on IRC. And if all of the above sounds terrible to you, and you'd rather just not take any of this into stable, I guess that's a valid path to take too.
Let me look at this later this week after we get this next round of stable kernels out. It's normally a nice calm period for the stable kernels so this might not be that bad.
This is all now merged into 5.10 and newer.
For 5.4 I just don't think it's worth the trouble. Devices based on that old kernel aren't going to want to deal with the churn like this, and being forced to add the WG code to it also seems like a lot of unnecessary work for everyone (including yourself.)
But, I do know that a lot of Android devices are still relying on 5.4, they already have WG in their kernels so overall this might be a smaller diff for them?
I don't really know, do you have any people/companies/devices using 5.4 that would want this all added that you can show it is worth it for?
thanks,
greg k-h
Hey Greg,
On Mon, May 30, 2022 at 10:11:52AM +0200, Greg KH wrote:
For 5.4 I just don't think it's worth the trouble. Devices based on that old kernel aren't going to want to deal with the churn like this, and being forced to add the WG code to it also seems like a lot of unnecessary work for everyone (including yourself.)
But, I do know that a lot of Android devices are still relying on 5.4, they already have WG in their kernels so overall this might be a smaller diff for them?
I don't really know, do you have any people/companies/devices using 5.4 that would want this all added that you can show it is worth it for?
I think if it's in 5.10, it makes sense to at least try to get it into 5.4 and below for the same reasons. I'm traveling over the next week or so, but I think I'll attempt to do a straight backport of it (sans-wg) like I did for 5.10. As mentioned, it's harder, but that doesn't mean it's impossible. I might give up in exasperation or perhaps find it too onerous. But hopefully I'll be able to reuse the work I did for the Android wg backports. Anyway, no guarantees -- it's not a trivial walk in the park -- but I'll give it a shot and let you know if I can make it work.
Jason
Hi Greg,
On Mon, May 30, 2022 at 12:38:15PM +0200, Jason A. Donenfeld wrote:
Hey Greg,
I think if it's in 5.10, it makes sense to at least try to get it into 5.4 and below for the same reasons. I'm traveling over the next week or so, but I think I'll attempt to do a straight backport of it (sans-wg) like I did for 5.10. As mentioned, it's harder, but that doesn't mean it's impossible. I might give up in exasperation or perhaps find it too onerous. But hopefully I'll be able to reuse the work I did for the Android wg backports. Anyway, no guarantees -- it's not a trivial walk in the park -- but I'll give it a shot and let you know if I can make it work.
I'm glad I tried, because that turned out to be really easy, and none of the concerns I had about the crypto turned out to be valid at all. A lot of the hairiness with the 5.6-era crypto code was the way that lib/crypto/ interacted with kconfig and crypto/, and the way arch crypto interacted with that. But for blake2s, there was just a single commit to backport, which didn't need to interact with anything else, because there was nothing prior in the kernel regarding blake2s. So it wound up just being a boring lib/ commit, with no complications.
So with that out of the way, I succeeded in doing the remaining backports. You can pull from https://git.kernel.org/pub/scm/linux/kernel/git/crng/random.git/ the following branches, with a linear series of commits on top of your latest:
- linux-4.9.y - linux-4.14.y - linux-4.19.y - linux-5.4.y
I've done an `allmodconfig` build test on these, and I've also booted a system on each of them. They contain the fixes that have landed since the previous tranche of backports, so that should bring all the backports up to date with each other.
And that means that moving forward, a `Cc: stable@vger.kernel.org` tag will hopefully apply evenly and without hassle to everything. More globally, I noticed when doing these backports what had been already backported and what hadn't, and it looks like a lot didn't easily apply before and so was dropped without being reworked, so over time fixes were lost. So I'm very happy to bring everything up to date finally.
Regards, Jason
On Fri, Jun 10, 2022 at 11:05:56AM +0200, Jason A. Donenfeld wrote:
Hi Greg,
On Mon, May 30, 2022 at 12:38:15PM +0200, Jason A. Donenfeld wrote:
Hey Greg,
I think if it's in 5.10, it makes sense to at least try to get it into 5.4 and below for the same reasons. I'm traveling over the next week or so, but I think I'll attempt to do a straight backport of it (sans-wg) like I did for 5.10. As mentioned, it's harder, but that doesn't mean it's impossible. I might give up in exasperation or perhaps find it too onerous. But hopefully I'll be able to reuse the work I did for the Android wg backports. Anyway, no guarantees -- it's not a trivial walk in the park -- but I'll give it a shot and let you know if I can make it work.
I'm glad I tried, because that turned out to be really easy, and none of the concerns I had about the crypto turned out to be valid at all. A lot of the hairiness with the 5.6-era crypto code was the way that lib/crypto/ interacted with kconfig and crypto/, and the way arch crypto interacted with that. But for blake2s, there was just a single commit to backport, which didn't need to interact with anything else, because there was nothing prior in the kernel regarding blake2s. So it wound up just being a boring lib/ commit, with no complications.
So with that out of the way, I succeeded in doing the remaining backports. You can pull from https://git.kernel.org/pub/scm/linux/kernel/git/crng/random.git/ the following branches, with a linear series of commits on top of your latest:
- linux-4.9.y - linux-4.14.y - linux-4.19.y - linux-5.4.y
I've done an `allmodconfig` build test on these, and I've also booted a system on each of them. They contain the fixes that have landed since the previous tranche of backports, so that should bring all the backports up to date with each other.
And that means that moving forward, a `Cc: stable@vger.kernel.org` tag will hopefully apply evenly and without hassle to everything. More globally, I noticed when doing these backports what had been already backported and what hadn't, and it looks like a lot didn't easily apply before and so was dropped without being reworked, so over time fixes were lost. So I'm very happy to bring everything up to date finally.
All now queued up, thanks.
greg k-h
On Mon, May 23, 2022 at 02:54:45PM +0200, Jason A. Donenfeld wrote:
In this git repo [2], there are three branches: linux-5.15.y, linux-5.17.y, and linux-5.18.y, which contain backports for everything up to and including [1].
There's now a linux-5.10.y branch too.
Jason
On Mon, May 23, 2022 at 02:54:45PM +0200, Jason A. Donenfeld wrote:
Assuming that Linus merges my PR for 5.19 [1] today, all of these patches are (or will be in a few hours) in Linus' tree.
This is now done, with [1], so everything should be properly upstream now.
Jason
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...
linux-stable-mirror@lists.linaro.org