-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi all,
here are some tests to test the cpu hotplug.
Any feedback would be appreciate
Thanks a lot
-- Daniel
Test 1: check the sysfs files are present =========================================
for each cpu * check /sys/devices/system/cpu/cpuX/online is there
What about the particular case for cpu0 where the 'online' file is there but shouldn't as the other arch ?
Test 2: check cpu state is reflected in sysfs =============================================
for each cpu * set cpuX offline * check cpuX status is offline * set cpuX back online * check cpuX status is online
Test 3: check the affinity does not work on an online cpu =========================================================
for each cpu * set cpuX offline * check 'taskset -c X /bin/true' fails * set cpuX back online
Test 4: check cpus are reflected on /proc/cpuinfo correctly ===========================================================
NRCPUS=<number of configured cpu on the system>
for each cpu * set cpuX offline * check /proc/cpuinfo 'processor' keyword count is 'NRCPUS - 1' * check the processor keyword contains a cpu id different from the cpu we offlined * set cpuX back online
Test 5: test userspace notification ===================================
for each cpu * set cpuX offline * check from userspace notification has been done with the command 'udevadm monitor --kernel --subsystem-match=cpu' * and the event is related to the cpuX and 'offline' * set the cpu online * check from userspace notification has been done with the command 'udevadm monitor --kernel --subsystem-match=cpu' * and the event is related to the cpuX and 'online'
Test 6: do some stress test ===========================
for each cpu * for i in 1 .. 50 * set cpuX offline * set cpuX back online
- -- http://www.linaro.org/ Linaro.org │ Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro Facebook | http://twitter.com/#!/linaroorg Twitter | http://www.linaro.org/linaro-blog/ Blog
On 28 September 2011 11:18, Daniel Lezcano daniel.lezcano@linaro.org wrote:
Hi all,
here are some tests to test the cpu hotplug.
Any feedback would be appreciate
Thanks a lot
-- Daniel
Test 3: check the affinity does not work on an online cpu
for each cpu * set cpuX offline * check 'taskset -c X /bin/true' fails * set cpuX back online
An interesting test might be what happens to tasks that had affinity only on the CPU that just went offline?
Interaction with irqbalancing?
What about interaction with various forms of power save or suspend/resume?
(Hmm side note - how do I detect that I have ARMv7MP as opposed to plain v7 - i.e. that I can do a PLDW - if anything is doing it by detecting multiple CPUs it might get confused by hotplug sometimes only showing one - I suspect that's the wrong way).
Dave
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/28/2011 12:33 PM, David Gilbert wrote:
On 28 September 2011 11:18, Daniel Lezcano daniel.lezcano@linaro.org wrote:
Hi all,
here are some tests to test the cpu hotplug.
Any feedback would be appreciate
Thanks a lot
-- Daniel
Test 3: check the affinity does not work on an online cpu
for each cpu
- set cpuX offline
- check 'taskset -c X /bin/true' fails
- set cpuX back online
An interesting test might be what happens to tasks that had affinity only on the CPU that just went offline?
right. I will add this test.
Interaction with irqbalancing?
Can you elaborate ? Do you mean we check irq affinity for a offlined cpu is moved to another cpu ? Or something like that ?
What about interaction with various forms of power save or
suspend/resume?
Right. I am not sure what interactions we really want to check in the test suite but we have to keep this in mind.
(Hmm side note - how do I detect that I have ARMv7MP as opposed to plain v7 - i.e. that I can do a PLDW - if anything is doing it by detecting multiple CPUs it might get confused by hotplug sometimes only showing one - I suspect that's the wrong way).
Well I guess there a several way to do that: sysfs will give the configured cpu on the system in /sys/devices/system/cpu/cpuX or sysconf(_SC_NPROCESSORS_CONF and _SC_NPROCESSORS_ONLN)
Thanks -- Daniel
- -- http://www.linaro.org/ Linaro.org │ Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro Facebook | http://twitter.com/#!/linaroorg Twitter | http://www.linaro.org/linaro-blog/ Blog
On 28 September 2011 13:19, Daniel Lezcano daniel.lezcano@linaro.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/28/2011 12:33 PM, David Gilbert wrote:
On 28 September 2011 11:18, Daniel Lezcano daniel.lezcano@linaro.org wrote:
Interaction with irqbalancing?
Can you elaborate ? Do you mean we check irq affinity for a offlined cpu is moved to another cpu ? Or something like that ?
yeh if you've tied some important interrupts to a CPU that disappears what happens to the interrupts.
> What about interaction with various forms of power save or suspend/resume?
Right. I am not sure what interactions we really want to check in the test suite but we have to keep this in mind.
Well I'm not too sure which types of scenarios you're looking to hotplug CPUs on ARM, but generally I'd say try and do the normal types of suspend/resume cycles or power save changes having hotplugged before hand (or if you want to be particularly evil during the process).
Dave
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/28/2011 02:31 PM, David Gilbert wrote:
On 28 September 2011 13:19, Daniel Lezcano daniel.lezcano@linaro.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/28/2011 12:33 PM, David Gilbert wrote:
On 28 September 2011 11:18, Daniel Lezcano daniel.lezcano@linaro.org wrote:
Interaction with irqbalancing?
Can you elaborate ? Do you mean we check irq affinity for a offlined cpu is moved to another cpu ? Or something like that ?
yeh if you've tied some important interrupts to a CPU that disappears what happens to the interrupts.
Yep. That makes sense. I will add this test to the test cases.
What about interaction with various forms of power save or
suspend/resume?
Right. I am not sure what interactions we really want to check in the test suite but we have to keep this in mind.
Well I'm not too sure which types of scenarios you're looking to hotplug CPUs on ARM, but generally I'd say try and do the normal types of suspend/resume cycles or power save changes having hotplugged before hand (or if you want to be particularly evil during the process).
Ok, I will look at how to automatize that.
Thanks for the ideas and feedbacks.
-- Daniel
- -- http://www.linaro.org/ Linaro.org ? Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro Facebook | http://twitter.com/#!/linaroorg Twitter | http://www.linaro.org/linaro-blog/ Blog
On Wed, Sep 28, 2011 at 1:18 PM, Daniel Lezcano daniel.lezcano@linaro.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi all,
here are some tests to test the cpu hotplug.
Any feedback would be appreciate
Thanks a lot
-- Daniel
Test 1: check the sysfs files are present
for each cpu * check /sys/devices/system/cpu/cpuX/online is there
What about the particular case for cpu0 where the 'online' file is there but shouldn't as the other arch ?
Test 2: check cpu state is reflected in sysfs
for each cpu * set cpuX offline * check cpuX status is offline * set cpuX back online * check cpuX status is online
Test 3: check the affinity does not work on an online cpu
^^^^ offline
=========================================================
for each cpu * set cpuX offline * check 'taskset -c X /bin/true' fails * set cpuX back online
Test 4: check cpus are reflected on /proc/cpuinfo correctly
NRCPUS=<number of configured cpu on the system>
for each cpu * set cpuX offline * check /proc/cpuinfo 'processor' keyword count is 'NRCPUS - 1' * check the processor keyword contains a cpu id different from the cpu we offlined * set cpuX back online
Test 5: test userspace notification
for each cpu * set cpuX offline * check from userspace notification has been done with the command 'udevadm monitor --kernel --subsystem-match=cpu' * and the event is related to the cpuX and 'offline' * set the cpu online * check from userspace notification has been done with the command 'udevadm monitor --kernel --subsystem-match=cpu' * and the event is related to the cpuX and 'online'
Test 6: do some stress test
for each cpu * for i in 1 .. 50 * set cpuX offline * set cpuX back online
How about something that measures the latency of a hotplug under 2-3 different workloads. Vincent has scripts that might help with that.
The higher the workload, the higher the hotplug latency due to task migration.
It would also be interesting to know what happens when processes pinned to a cpu have the cpu offlined underneath them.
/Amit
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/28/2011 01:04 PM, Amit Kucheria wrote:
On Wed, Sep 28, 2011 at 1:18 PM, Daniel Lezcano daniel.lezcano@linaro.org wrote:
[ ... ]
Test 3: check the affinity does not work on an online cpu
^^^^ offline
right. Thanks.
[ ... ]
Test 6: do some stress test
for each cpu
- for i in 1 .. 50
- set cpuX offline
- set cpuX back online
How about something that measures the latency of a hotplug under 2-3 different workloads. Vincent has scripts that might help with that.
The higher the workload, the higher the hotplug latency due to task migration.
Yes, definitely we should add some tests for measurements, not only for cpu hotplug but also for the other pm blocks. But I would like to postpone that as a separate test set focused on perf and measurements. Lava is waiting for "succeed", "fail", "skip" results, so adding numerical results will make the test suite to fail.
I hope we can discuss at UDS-P about adding some perf tests and integrate them in lava.
It would also be interesting to know what happens when processes pinned to a cpu have the cpu offlined underneath them.
Right, I think it is the test David proposed.
- -- http://www.linaro.org/ Linaro.org │ Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro Facebook | http://twitter.com/#!/linaroorg Twitter | http://www.linaro.org/linaro-blog/ Blog
On Wed, Sep 28, 2011 at 5:18 AM, Daniel Lezcano daniel.lezcano@linaro.orgwrote:
Test 6: do some stress test
for each cpu
- for i in 1 .. 50
- set cpuX offline
- set cpuX back online
Not sure how evil you want to get with this, but several years back I wrote a simple, but nasty test that picked a cpu at random, and set the state to the opposite of whatever it happened to be at the time. It was completely random, I didn't have anything builtin for record/playback, or doing it in the same sequence for reproducibilty, but it didn't really need it. When it broke things, it usually broke them within just a few minutes.
Thanks, Paul Larson
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/29/2011 05:41 AM, Paul Larson wrote:
On Wed, Sep 28, 2011 at 5:18 AM, Daniel Lezcano daniel.lezcano@linaro.orgwrote:
Test 6: do some stress test
for each cpu
- for i in 1 .. 50
- set cpuX offline
- set cpuX back online
Not sure how evil you want to get with this, but several years back I wrote a simple, but nasty test that picked a cpu at random, and set the state to the opposite of whatever it happened to be at the time. It was completely random, I didn't have anything builtin for record/playback, or doing it in the same sequence for reproducibilty, but it didn't really need it. When it broke things, it usually broke them within just a few minutes.
Ok thanks for the idea, I will add what you suggest.
Thanks -- Daniel
- -- http://www.linaro.org/ Linaro.org ? Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro Facebook | http://twitter.com/#!/linaroorg Twitter | http://www.linaro.org/linaro-blog/ Blog
On Thu, Sep 29, 2011 at 9:14 AM, Daniel Lezcano daniel.lezcano@linaro.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/29/2011 05:41 AM, Paul Larson wrote:
On Wed, Sep 28, 2011 at 5:18 AM, Daniel Lezcano daniel.lezcano@linaro.orgwrote:
Test 6: do some stress test
for each cpu * for i in 1 .. 50 * set cpuX offline * set cpuX back online
Not sure how evil you want to get with this, but several years back I wrote a simple, but nasty test that picked a cpu at random, and set the state to the opposite of whatever it happened to be at the time. It was completely random, I didn't have anything builtin for record/playback, or doing it in the same sequence for reproducibilty, but it didn't really need it. When it broke things, it usually broke them within just a few minutes.
Ok thanks for the idea, I will add what you suggest.
With cpu0 being a special case?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/29/2011 09:11 AM, Amit Kucheria wrote:
On Thu, Sep 29, 2011 at 9:14 AM, Daniel Lezcano daniel.lezcano@linaro.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/29/2011 05:41 AM, Paul Larson wrote:
On Wed, Sep 28, 2011 at 5:18 AM, Daniel Lezcano daniel.lezcano@linaro.orgwrote:
Test 6: do some stress test
for each cpu
- for i in 1 .. 50
- set cpuX offline
- set cpuX back online
Not sure how evil you want to get with this, but several years back I wrote a simple, but nasty test that picked a cpu at random, and set the state to the opposite of whatever it happened to be at the time. It was completely random, I didn't have anything builtin for record/playback, or doing it in the same sequence for reproducibilty, but it didn't really need it. When it broke things, it usually broke them within just a few minutes.
Ok thanks for the idea, I will add what you suggest.
With cpu0 being a special case?
Yes, cpu0 will be a special case for all the tests.
- -- http://www.linaro.org/ Linaro.org ? Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro Facebook | http://twitter.com/#!/linaroorg Twitter | http://www.linaro.org/linaro-blog/ Blog
On Thu, Sep 29, 2011 at 12:46 AM, Daniel Lezcano daniel.lezcano@linaro.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/29/2011 09:11 AM, Amit Kucheria wrote:
On Thu, Sep 29, 2011 at 9:14 AM, Daniel Lezcano daniel.lezcano@linaro.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/29/2011 05:41 AM, Paul Larson wrote:
On Wed, Sep 28, 2011 at 5:18 AM, Daniel Lezcano daniel.lezcano@linaro.orgwrote:
Test 6: do some stress test
for each cpu * for i in 1 .. 50 * set cpuX offline * set cpuX back online
Not sure how evil you want to get with this, but several years back I wrote a simple, but nasty test that picked a cpu at random, and set the state to the opposite of whatever it happened to be at the time. It was completely random, I didn't have anything builtin for record/playback, or doing it in the same sequence for reproducibilty, but it didn't really need it. When it broke things, it usually broke them within just a few minutes.
Ok thanks for the idea, I will add what you suggest.
With cpu0 being a special case?
Yes, cpu0 will be a special case for all the tests.
If only we had a patch for that!
;-)
Mike
http://www.linaro.org/ Linaro.org ? Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro Facebook | http://twitter.com/#!/linaroorg Twitter | http://www.linaro.org/linaro-blog/ Blog
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJOhCJqAAoJEAKBbMCpUGYAS54H/i3g9v2KcNal7zCvshmXFJcZ xX//Np/z+1UCUowTUnV8O80XMgyOT8f0EFR8G+e6niORJk8GIlW2ZlQfIMJOYZyF T4NEJXWYpW5G6FFORJLiAFDvC9VXrj/F2bGZa0NjhaePCmdRTdwlyv2GNJvrO6B/ /3oFpCBRGJ/3P9Pns+hy4ry/7TRkzIHw1OGv4XCbGnS8zQD7hXb1ljkpewReZVdm srY6oLRmL9j+k7WIHMIHXyg+nv0DT8WCGrw32Rz074HJJVJwDo0xo1vEHevp76SW BW6OtoaaiIbRYQgXM2Qbcf7BH0jxmduIfhrchTzQSHWa7K8uaqyLYTUPv9WA/JE= =lL8N -----END PGP SIGNATURE-----
-
Amit Kucheria -
Daniel Lezcano -
David Gilbert -
Mike Turquette -
Paul Larson