Signed-off-by: Maxim Uvarov maxim.uvarov@linaro.org --- ubuntu/openem.yaml | 28 ++++++++++++++++++++++++++ ubuntu/scripts/openem.sh | 50 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100644 ubuntu/openem.yaml create mode 100755 ubuntu/scripts/openem.sh
diff --git a/ubuntu/openem.yaml b/ubuntu/openem.yaml new file mode 100644 index 0000000..1a3939d --- /dev/null +++ b/ubuntu/openem.yaml @@ -0,0 +1,28 @@ +metadata: + name: thp + format: "Lava-Test-Shell Test Definition 1.0" + description: "Test for open event machine x86." + +install: + git-repos: + - git://git.linaro.org/qa/test-definitions.git + deps: + - python-twisted + - git + - make + - unzip + - gcc + - build-essential + - libc6-dev + - gcc-multilib +run: + steps: + - "cd testcases/ubuntu/scripts" + - "./openem.sh" + + +parse: + pattern: "^(?P<test_case_id>[a-zA-Z0-9_-]+):\s(?P<result>\w+)" + fixupdict: + PASS: pass + FAIL: fail diff --git a/ubuntu/scripts/openem.sh b/ubuntu/scripts/openem.sh new file mode 100755 index 0000000..63bb028 --- /dev/null +++ b/ubuntu/scripts/openem.sh @@ -0,0 +1,50 @@ +#!/bin/bash -x + +mount -t tmpfs none /dev/shm +echo 400 > /proc/sys/vm/nr_hugepages + +rm -rf ./Build +mkdir ./Build +cd ./Build + +wget http://people.linaro.org/~maxim.uvarov/Intel_DPDK%20Code_1.5.0_8.zip +unzip Intel_DPDK\ Code_1.5.0_8.zip +mv DPDK-1.5.0 DPDK_1.5.0-8 + +git clone git://git.linaro.org/lng/eventmachine-code.git eventmachine-code.git + +patch -p0 < ./eventmachine-code.git/misc/linux-generic/patch/DPDK_1.5.0-8-linux-generic.patch +cd DPDK_1.5.0-8 + +sed -i 's/CONFIG_RTE_MAX_MEMZONE=2560/CONFIG_RTE_MAX_MEMZONE=131072/' ./config/defconfig_* +sed -i 's/CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE=512/CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE=2048/' ./config/defconfig_* +sed -i 's/CONFIG_RTE_MBUF_SCATTER_GATHER=y/CONFIG_RTE_MBUF_SCATTER_GATHER=n/' ./config/defconfig_* +sed -i 's/CONFIG_RTE_MBUF_REFCNT_ATOMIC=y/CONFIG_RTE_MBUF_REFCNT_ATOMIC=n/' ./config/defconfig_* +sed -i 's/CONFIG_RTE_PKTMBUF_HEADROOM=128/CONFIG_RTE_PKTMBUF_HEADROOM=192/' ./config/defconfig_* +make install T=generic_32-default-linuxapp-gcc + +export RTE_SDK=`pwd` +export RTE_TARGET=generic_32-default-linuxapp-gcc +#export RTE_TARGET=generic_64-default-linuxapp-gcc + +mkdir /mnt/huge +umount /mnt/huge +mount -t hugetlbfs nodev /mnt/huge + +#EM +cd ../eventmachine-code.git/event_test/example/linux-generic +make real_clean && make em_clean +make + +#run some tests: +./build/hello -c 0xfe -n 4 -- -p | head -n 500 # (Run 'hello' on 7 cores using EM process-per-core mode(-p)) +./build/hello -c 0xfcfc -n 4 -- -t | head -n 500 # (Run 'hello' on 12 cores using EM thread-per-core mode(-t)) +./build/perf -c 0xffff -n 4 -- -p | head -n 500 # (Run 'perf' on 16 cores using EM process-per-core mode(-p)) +./build/perf -c 0xfffe -n 4 -- -t | hean -n 500 # (Run 'perf' on 15 cores using EM thread-per-core mode(-t)) +./build/event_group -c 0x0c0c -n 4 -- -p | head -n 500 # (Run 'event_group' on 4 cores using EM process-per-core mode(-p)) +./build/event_group -c 0x00f0 -n 4 -- -t | head -n 500 # (Run 'event_group' on 4 cores using EM thread-per-core mode(-t)) +./build/error -c 0x3 -n 4 -- -p |head -n 500 # (Run 'error' on 2 cores using EM process-per-core mode(-p)) +./build/error -c 0x2 -n 4 -- -t |head -n 500 # (Run 'error' on 1 core using EM thread-per-core mode(-t)) + +echo "OPENEM TEST END!!!" +
Hi Maxim,
Pardon my ignorance but where are these tests being added to?
I just read very quickly about openem[1] and this might be something we're interested in using as a test for our scheduler work. Can you describe a bit more where openem originates and how you use it inside LNG?
Regards, Amit [1] http://sourceforge.net/projects/eventmachine/files/Documents/EM_introduction...
On Thu, Oct 24, 2013 at 6:17 PM, Maxim Uvarov maxim.uvarov@linaro.org wrote:
Signed-off-by: Maxim Uvarov maxim.uvarov@linaro.org
ubuntu/openem.yaml | 28 ++++++++++++++++++++++++++ ubuntu/scripts/openem.sh | 50 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100644 ubuntu/openem.yaml create mode 100755 ubuntu/scripts/openem.sh
diff --git a/ubuntu/openem.yaml b/ubuntu/openem.yaml new file mode 100644 index 0000000..1a3939d --- /dev/null +++ b/ubuntu/openem.yaml @@ -0,0 +1,28 @@ +metadata:
- name: thp
- format: "Lava-Test-Shell Test Definition 1.0"
- description: "Test for open event machine x86."
+install:
- git-repos:
- git://git.linaro.org/qa/test-definitions.git- deps:
- python-twisted- git- make- unzip- gcc- build-essential- libc6-dev- gcc-multilib+run:
- steps:
- "cd testcases/ubuntu/scripts"- "./openem.sh"+parse:
- pattern: "^(?P<test_case_id>[a-zA-Z0-9_-]+):\s(?P<result>\w+)"
- fixupdict:
PASS: passFAIL: faildiff --git a/ubuntu/scripts/openem.sh b/ubuntu/scripts/openem.sh new file mode 100755 index 0000000..63bb028 --- /dev/null +++ b/ubuntu/scripts/openem.sh @@ -0,0 +1,50 @@ +#!/bin/bash -x
+mount -t tmpfs none /dev/shm +echo 400 > /proc/sys/vm/nr_hugepages
+rm -rf ./Build +mkdir ./Build +cd ./Build
+wget http://people.linaro.org/~maxim.uvarov/Intel_DPDK%20Code_1.5.0_8.zip +unzip Intel_DPDK\ Code_1.5.0_8.zip +mv DPDK-1.5.0 DPDK_1.5.0-8
+git clone git://git.linaro.org/lng/eventmachine-code.git eventmachine-code.git
+patch -p0 < ./eventmachine-code.git/misc/linux-generic/patch/DPDK_1.5.0-8-linux-generic.patch +cd DPDK_1.5.0-8
+sed -i 's/CONFIG_RTE_MAX_MEMZONE=2560/CONFIG_RTE_MAX_MEMZONE=131072/' ./config/defconfig_* +sed -i 's/CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE=512/CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE=2048/' ./config/defconfig_* +sed -i 's/CONFIG_RTE_MBUF_SCATTER_GATHER=y/CONFIG_RTE_MBUF_SCATTER_GATHER=n/' ./config/defconfig_* +sed -i 's/CONFIG_RTE_MBUF_REFCNT_ATOMIC=y/CONFIG_RTE_MBUF_REFCNT_ATOMIC=n/' ./config/defconfig_* +sed -i 's/CONFIG_RTE_PKTMBUF_HEADROOM=128/CONFIG_RTE_PKTMBUF_HEADROOM=192/' ./config/defconfig_* +make install T=generic_32-default-linuxapp-gcc
+export RTE_SDK=`pwd` +export RTE_TARGET=generic_32-default-linuxapp-gcc +#export RTE_TARGET=generic_64-default-linuxapp-gcc
+mkdir /mnt/huge +umount /mnt/huge +mount -t hugetlbfs nodev /mnt/huge
+#EM +cd ../eventmachine-code.git/event_test/example/linux-generic +make real_clean && make em_clean +make
+#run some tests: +./build/hello -c 0xfe -n 4 -- -p | head -n 500 # (Run 'hello' on 7 cores using EM process-per-core mode(-p)) +./build/hello -c 0xfcfc -n 4 -- -t | head -n 500 # (Run 'hello' on 12 cores using EM thread-per-core mode(-t)) +./build/perf -c 0xffff -n 4 -- -p | head -n 500 # (Run 'perf' on 16 cores using EM process-per-core mode(-p)) +./build/perf -c 0xfffe -n 4 -- -t | hean -n 500 # (Run 'perf' on 15 cores using EM thread-per-core mode(-t)) +./build/event_group -c 0x0c0c -n 4 -- -p | head -n 500 # (Run 'event_group' on 4 cores using EM process-per-core mode(-p)) +./build/event_group -c 0x00f0 -n 4 -- -t | head -n 500 # (Run 'event_group' on 4 cores using EM thread-per-core mode(-t)) +./build/error -c 0x3 -n 4 -- -p |head -n 500 # (Run 'error' on 2 cores using EM process-per-core mode(-p)) +./build/error -c 0x2 -n 4 -- -t |head -n 500 # (Run 'error' on 1 core using EM thread-per-core mode(-t))
+echo "OPENEM TEST END!!!"
-- 1.7.9.5
linaro-kernel mailing list linaro-kernel@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-kernel
On 10/24/2013 04:56 PM, Amit Kucheria wrote:
Hi Maxim,
Pardon my ignorance but where are these tests being added to?
I just read very quickly about openem[1] and this might be something we're interested in using as a test for our scheduler work. Can you describe a bit more where openem originates and how you use it inside LNG?
Regards, Amit
Hi Amit,
1. this test proposed for git://git.linaro.org/qa/test-definitions.git
2. In general commit has bash script which downloads openem, compiles it as suggested in README and run some basic changes. Output of test can be found here: http://validation.linaro.org/scheduler/job/80939/log_file
Next work will be to remove dpdk dependence and do some arm optimizations.
Thank you, Maxim.
[1] http://sourceforge.net/projects/eventmachine/files/Documents/EM_introduction...
On Thu, Oct 24, 2013 at 6:17 PM, Maxim Uvarov maxim.uvarov@linaro.org wrote:
Signed-off-by: Maxim Uvarov maxim.uvarov@linaro.org
ubuntu/openem.yaml | 28 ++++++++++++++++++++++++++ ubuntu/scripts/openem.sh | 50 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100644 ubuntu/openem.yaml create mode 100755 ubuntu/scripts/openem.sh
diff --git a/ubuntu/openem.yaml b/ubuntu/openem.yaml new file mode 100644 index 0000000..1a3939d --- /dev/null +++ b/ubuntu/openem.yaml @@ -0,0 +1,28 @@ +metadata:
- name: thp
- format: "Lava-Test-Shell Test Definition 1.0"
- description: "Test for open event machine x86."
+install:
- git-repos:
- git://git.linaro.org/qa/test-definitions.git- deps:
- python-twisted- git- make- unzip- gcc- build-essential- libc6-dev- gcc-multilib+run:
- steps:
- "cd testcases/ubuntu/scripts"- "./openem.sh"+parse:
- pattern: "^(?P<test_case_id>[a-zA-Z0-9_-]+):\s(?P<result>\w+)"
- fixupdict:
PASS: passFAIL: faildiff --git a/ubuntu/scripts/openem.sh b/ubuntu/scripts/openem.sh new file mode 100755 index 0000000..63bb028 --- /dev/null +++ b/ubuntu/scripts/openem.sh @@ -0,0 +1,50 @@ +#!/bin/bash -x
+mount -t tmpfs none /dev/shm +echo 400 > /proc/sys/vm/nr_hugepages
+rm -rf ./Build +mkdir ./Build +cd ./Build
+wget http://people.linaro.org/~maxim.uvarov/Intel_DPDK%20Code_1.5.0_8.zip +unzip Intel_DPDK\ Code_1.5.0_8.zip +mv DPDK-1.5.0 DPDK_1.5.0-8
+git clone git://git.linaro.org/lng/eventmachine-code.git eventmachine-code.git
+patch -p0 < ./eventmachine-code.git/misc/linux-generic/patch/DPDK_1.5.0-8-linux-generic.patch +cd DPDK_1.5.0-8
+sed -i 's/CONFIG_RTE_MAX_MEMZONE=2560/CONFIG_RTE_MAX_MEMZONE=131072/' ./config/defconfig_* +sed -i 's/CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE=512/CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE=2048/' ./config/defconfig_* +sed -i 's/CONFIG_RTE_MBUF_SCATTER_GATHER=y/CONFIG_RTE_MBUF_SCATTER_GATHER=n/' ./config/defconfig_* +sed -i 's/CONFIG_RTE_MBUF_REFCNT_ATOMIC=y/CONFIG_RTE_MBUF_REFCNT_ATOMIC=n/' ./config/defconfig_* +sed -i 's/CONFIG_RTE_PKTMBUF_HEADROOM=128/CONFIG_RTE_PKTMBUF_HEADROOM=192/' ./config/defconfig_* +make install T=generic_32-default-linuxapp-gcc
+export RTE_SDK=`pwd` +export RTE_TARGET=generic_32-default-linuxapp-gcc +#export RTE_TARGET=generic_64-default-linuxapp-gcc
+mkdir /mnt/huge +umount /mnt/huge +mount -t hugetlbfs nodev /mnt/huge
+#EM +cd ../eventmachine-code.git/event_test/example/linux-generic +make real_clean && make em_clean +make
+#run some tests: +./build/hello -c 0xfe -n 4 -- -p | head -n 500 # (Run 'hello' on 7 cores using EM process-per-core mode(-p)) +./build/hello -c 0xfcfc -n 4 -- -t | head -n 500 # (Run 'hello' on 12 cores using EM thread-per-core mode(-t)) +./build/perf -c 0xffff -n 4 -- -p | head -n 500 # (Run 'perf' on 16 cores using EM process-per-core mode(-p)) +./build/perf -c 0xfffe -n 4 -- -t | hean -n 500 # (Run 'perf' on 15 cores using EM thread-per-core mode(-t)) +./build/event_group -c 0x0c0c -n 4 -- -p | head -n 500 # (Run 'event_group' on 4 cores using EM process-per-core mode(-p)) +./build/event_group -c 0x00f0 -n 4 -- -t | head -n 500 # (Run 'event_group' on 4 cores using EM thread-per-core mode(-t)) +./build/error -c 0x3 -n 4 -- -p |head -n 500 # (Run 'error' on 2 cores using EM process-per-core mode(-p)) +./build/error -c 0x2 -n 4 -- -t |head -n 500 # (Run 'error' on 1 core using EM thread-per-core mode(-t))
+echo "OPENEM TEST END!!!"
-- 1.7.9.5
linaro-kernel mailing list linaro-kernel@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-kernel
Amit,
OpenEM is a software implementation of the NSN Event Machine (EM). There are also a couple of HW-specific implementations of EM from the respective HW (SoC) vendors. EM is a fundamental part of OpenDataPlane (ODP) which you will hear more about at Linaro Connect next week. LNG is using OpenEM as a basis for the ODP reference implementation. Originally OpenEM was designed for running on x86 and utilising Intel DPDK for device (network interface) access etc. Linaro and NSN are liberating OpenEM from DPDK dependencies and the ODP reference implementation will run on vanilla Linux without needing DPDK (I think it already does).
Networking and dataplane applications will probably have different power management strategies compared to servers and client devices. A SoC may be running a mix of dataplane and other (e.g. control plane) applications and need different power management strategies (for different cores) on the same SoC. I have had some initial discussions with Charles Garcia-Tobin and Morten Rasmussen about this. Maybe something we can discuss at Connect.
-- Ola
Ola Liljedahl, Networking System Architect, ARM Telephone: +46 706 866 373 Skype: ola.liljedahl
-----Original Message----- From: Amit Kucheria [mailto:amit.kucheria@linaro.org] Sent: 24 October 2013 14:56 To: Maxim Uvarov; Vincent Guittot Cc: linaro-networking; Lists linaro-kernel; Patch Tracking Subject: [LNG] Re: [PATCH] add openem x86 tests
Hi Maxim,
Pardon my ignorance but where are these tests being added to?
I just read very quickly about openem[1] and this might be something we're interested in using as a test for our scheduler work. Can you describe a bit more where openem originates and how you use it inside LNG?
Regards, Amit [1] http://sourceforge.net/projects/eventmachine/files/Documents/EM_introductio n_1_0.pdf/download
On Thu, Oct 24, 2013 at 6:17 PM, Maxim Uvarov maxim.uvarov@linaro.org wrote:
Signed-off-by: Maxim Uvarov maxim.uvarov@linaro.org
ubuntu/openem.yaml | 28 ++++++++++++++++++++++++++ ubuntu/scripts/openem.sh | 50
++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 78 insertions(+) create mode 100644 ubuntu/openem.yaml create mode 100755 ubuntu/scripts/openem.sh
diff --git a/ubuntu/openem.yaml b/ubuntu/openem.yaml new file mode 100644 index 0000000..1a3939d --- /dev/null +++ b/ubuntu/openem.yaml @@ -0,0 +1,28 @@ +metadata:
- name: thp
- format: "Lava-Test-Shell Test Definition 1.0"
- description: "Test for open event machine x86."
+install:
- git-repos:
- git://git.linaro.org/qa/test-definitions.git- deps:
- python-twisted- git- make- unzip- gcc- build-essential- libc6-dev- gcc-multilib+run:
- steps:
- "cd testcases/ubuntu/scripts"- "./openem.sh"+parse:
- pattern: "^(?P<test_case_id>[a-zA-Z0-9_-]+):\s(?P<result>\w+)"
- fixupdict:
PASS: passFAIL: faildiff --git a/ubuntu/scripts/openem.sh b/ubuntu/scripts/openem.sh new file mode 100755 index 0000000..63bb028 --- /dev/null +++ b/ubuntu/scripts/openem.sh @@ -0,0 +1,50 @@ +#!/bin/bash -x
+mount -t tmpfs none /dev/shm +echo 400 > /proc/sys/vm/nr_hugepages
+rm -rf ./Build +mkdir ./Build +cd ./Build
+wget
http://people.linaro.org/~maxim.uvarov/Intel_DPDK%20Code_1.5.0_8.zip
+unzip Intel_DPDK\ Code_1.5.0_8.zip +mv DPDK-1.5.0 DPDK_1.5.0-8
+git clone git://git.linaro.org/lng/eventmachine-code.git eventmachine-
code.git
+patch -p0 < ./eventmachine-code.git/misc/linux-
generic/patch/DPDK_1.5.0-8-linux-generic.patch
+cd DPDK_1.5.0-8
+sed -i 's/CONFIG_RTE_MAX_MEMZONE=2560/CONFIG_RTE_MAX_MEMZONE=131072/'
./config/defconfig_*
+sed -i
's/CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE=512/CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE= 2048/' ./config/defconfig_*
+sed -i
's/CONFIG_RTE_MBUF_SCATTER_GATHER=y/CONFIG_RTE_MBUF_SCATTER_GATHER=n/' ./config/defconfig_*
+sed -i
's/CONFIG_RTE_MBUF_REFCNT_ATOMIC=y/CONFIG_RTE_MBUF_REFCNT_ATOMIC=n/' ./config/defconfig_*
+sed -i
's/CONFIG_RTE_PKTMBUF_HEADROOM=128/CONFIG_RTE_PKTMBUF_HEADROOM=192/' ./config/defconfig_*
+make install T=generic_32-default-linuxapp-gcc
+export RTE_SDK=`pwd` +export RTE_TARGET=generic_32-default-linuxapp-gcc +#export RTE_TARGET=generic_64-default-linuxapp-gcc
+mkdir /mnt/huge +umount /mnt/huge +mount -t hugetlbfs nodev /mnt/huge
+#EM +cd ../eventmachine-code.git/event_test/example/linux-generic +make real_clean && make em_clean +make
+#run some tests: +./build/hello -c 0xfe -n 4 -- -p | head -n 500 # (Run 'hello' on 7
cores using EM process-per-core mode(-p))
+./build/hello -c 0xfcfc -n 4 -- -t | head -n 500 # (Run 'hello' on
12 cores using EM thread-per-core mode(-t))
+./build/perf -c 0xffff -n 4 -- -p | head -n 500 # (Run 'perf' on
16 cores using EM process-per-core mode(-p))
+./build/perf -c 0xfffe -n 4 -- -t | hean -n 500 # (Run 'perf' on
15 cores using EM thread-per-core mode(-t))
+./build/event_group -c 0x0c0c -n 4 -- -p | head -n 500 # (Run
'event_group' on 4 cores using EM process-per-core mode(-p))
+./build/event_group -c 0x00f0 -n 4 -- -t | head -n 500 # (Run
'event_group' on 4 cores using EM thread-per-core mode(-t))
+./build/error -c 0x3 -n 4 -- -p |head -n 500 # (Run 'error' on 2
cores using EM process-per-core mode(-p))
+./build/error -c 0x2 -n 4 -- -t |head -n 500 # (Run 'error' on 1
core using EM thread-per-core mode(-t))
+echo "OPENEM TEST END!!!"
-- 1.7.9.5
linaro-kernel mailing list linaro-kernel@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-kernel
-- You received this message because you are subscribed to the Google Groups "Linaro Networking" group. To unsubscribe from this group and stop receiving emails from it, send an email to linaro-networking+unsubscribe@linaro.org. To post to this group, send email to linaro-networking@linaro.org.
-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2557590 ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2548782
Thanks for the explanation, Ola. I'll catch up with you during connect to see if this test suite or a subset of it might make sense to add to our list of benchmarks.
My very high-level understanding is that openem has its own job scheduler to route packets to the best cpu. And what it measures is progress in processing packets I.e. performance. Please correct me if I'm wrong.
See you next week.
/amit On 24 Oct 2013 15:46, "Ola Liljedahl" Ola.Liljedahl@arm.com wrote:
Amit,
OpenEM is a software implementation of the NSN Event Machine (EM). There are also a couple of HW-specific implementations of EM from the respective HW (SoC) vendors. EM is a fundamental part of OpenDataPlane (ODP) which you will hear more about at Linaro Connect next week. LNG is using OpenEM as a basis for the ODP reference implementation. Originally OpenEM was designed for running on x86 and utilising Intel DPDK for device (network interface) access etc. Linaro and NSN are liberating OpenEM from DPDK dependencies and the ODP reference implementation will run on vanilla Linux without needing DPDK (I think it already does).
Networking and dataplane applications will probably have different power management strategies compared to servers and client devices. A SoC may be running a mix of dataplane and other (e.g. control plane) applications and need different power management strategies (for different cores) on the same SoC. I have had some initial discussions with Charles Garcia-Tobin and Morten Rasmussen about this. Maybe something we can discuss at Connect.
-- Ola
Ola Liljedahl, Networking System Architect, ARM Telephone: +46 706 866 373 Skype: ola.liljedahl
-----Original Message----- From: Amit Kucheria [mailto:amit.kucheria@linaro.org] Sent: 24 October 2013 14:56 To: Maxim Uvarov; Vincent Guittot Cc: linaro-networking; Lists linaro-kernel; Patch Tracking Subject: [LNG] Re: [PATCH] add openem x86 tests
Hi Maxim,
Pardon my ignorance but where are these tests being added to?
I just read very quickly about openem[1] and this might be something we're interested in using as a test for our scheduler work. Can you describe a bit more where openem originates and how you use it inside LNG?
Regards, Amit [1]
http://sourceforge.net/projects/eventmachine/files/Documents/EM_introductio
n_1_0.pdf/download
On Thu, Oct 24, 2013 at 6:17 PM, Maxim Uvarov maxim.uvarov@linaro.org wrote:
Signed-off-by: Maxim Uvarov maxim.uvarov@linaro.org
ubuntu/openem.yaml | 28 ++++++++++++++++++++++++++ ubuntu/scripts/openem.sh | 50
++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 78 insertions(+) create mode 100644 ubuntu/openem.yaml create mode 100755 ubuntu/scripts/openem.sh
diff --git a/ubuntu/openem.yaml b/ubuntu/openem.yaml new file mode 100644 index 0000000..1a3939d --- /dev/null +++ b/ubuntu/openem.yaml @@ -0,0 +1,28 @@ +metadata:
- name: thp
- format: "Lava-Test-Shell Test Definition 1.0"
- description: "Test for open event machine x86."
+install:
- git-repos:
- git://git.linaro.org/qa/test-definitions.git- deps:
- python-twisted- git- make- unzip- gcc- build-essential- libc6-dev- gcc-multilib+run:
- steps:
- "cd testcases/ubuntu/scripts"- "./openem.sh"+parse:
- pattern: "^(?P<test_case_id>[a-zA-Z0-9_-]+):\s(?P<result>\w+)"
- fixupdict:
PASS: passFAIL: faildiff --git a/ubuntu/scripts/openem.sh b/ubuntu/scripts/openem.sh new file mode 100755 index 0000000..63bb028 --- /dev/null +++ b/ubuntu/scripts/openem.sh @@ -0,0 +1,50 @@ +#!/bin/bash -x
+mount -t tmpfs none /dev/shm +echo 400 > /proc/sys/vm/nr_hugepages
+rm -rf ./Build +mkdir ./Build +cd ./Build
+wget
http://people.linaro.org/~maxim.uvarov/Intel_DPDK%20Code_1.5.0_8.zip
+unzip Intel_DPDK\ Code_1.5.0_8.zip +mv DPDK-1.5.0 DPDK_1.5.0-8
+git clone git://git.linaro.org/lng/eventmachine-code.giteventmachine-
code.git
+patch -p0 < ./eventmachine-code.git/misc/linux-
generic/patch/DPDK_1.5.0-8-linux-generic.patch
+cd DPDK_1.5.0-8
+sed -i 's/CONFIG_RTE_MAX_MEMZONE=2560/CONFIG_RTE_MAX_MEMZONE=131072/'
./config/defconfig_*
+sed -i
's/CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE=512/CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE=
2048/' ./config/defconfig_*
+sed -i
's/CONFIG_RTE_MBUF_SCATTER_GATHER=y/CONFIG_RTE_MBUF_SCATTER_GATHER=n/' ./config/defconfig_*
+sed -i
's/CONFIG_RTE_MBUF_REFCNT_ATOMIC=y/CONFIG_RTE_MBUF_REFCNT_ATOMIC=n/' ./config/defconfig_*
+sed -i
's/CONFIG_RTE_PKTMBUF_HEADROOM=128/CONFIG_RTE_PKTMBUF_HEADROOM=192/' ./config/defconfig_*
+make install T=generic_32-default-linuxapp-gcc
+export RTE_SDK=`pwd` +export RTE_TARGET=generic_32-default-linuxapp-gcc +#export RTE_TARGET=generic_64-default-linuxapp-gcc
+mkdir /mnt/huge +umount /mnt/huge +mount -t hugetlbfs nodev /mnt/huge
+#EM +cd ../eventmachine-code.git/event_test/example/linux-generic +make real_clean && make em_clean +make
+#run some tests: +./build/hello -c 0xfe -n 4 -- -p | head -n 500 # (Run 'hello' on
7
cores using EM process-per-core mode(-p))
+./build/hello -c 0xfcfc -n 4 -- -t | head -n 500 # (Run 'hello'
on
12 cores using EM thread-per-core mode(-t))
+./build/perf -c 0xffff -n 4 -- -p | head -n 500 # (Run 'perf' on
16 cores using EM process-per-core mode(-p))
+./build/perf -c 0xfffe -n 4 -- -t | hean -n 500 # (Run 'perf' on
15 cores using EM thread-per-core mode(-t))
+./build/event_group -c 0x0c0c -n 4 -- -p | head -n 500 # (Run
'event_group' on 4 cores using EM process-per-core mode(-p))
+./build/event_group -c 0x00f0 -n 4 -- -t | head -n 500 # (Run
'event_group' on 4 cores using EM thread-per-core mode(-t))
+./build/error -c 0x3 -n 4 -- -p |head -n 500 # (Run 'error'
on 2
cores using EM process-per-core mode(-p))
+./build/error -c 0x2 -n 4 -- -t |head -n 500 # (Run 'error'
on 1
core using EM thread-per-core mode(-t))
+echo "OPENEM TEST END!!!"
-- 1.7.9.5
linaro-kernel mailing list linaro-kernel@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-kernel
-- You received this message because you are subscribed to the Google Groups "Linaro Networking" group. To unsubscribe from this group and stop receiving emails from it, send an email to linaro-networking+unsubscribe@linaro.org. To post to this group, send email to linaro-networking@linaro.org.
-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2557590 ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2548782
From: Amit Kucheria [mailto:amit.kucheria@linaro.org] Sent: 25 October 2013 07:16 To: Ola Liljedahl Cc: linaro-networking; Vincent Guittot; Maxim Uvarov; Lists linaro-kernel; Patch Tracking Subject: RE: [LNG] Re: [PATCH] add openem x86 tests
Thanks for the explanation, Ola. I'll catch up with you during connect to see if this test suite or a subset of it might make sense to add to our list of benchmarks.
My very high-level understanding is that openem has its own job scheduler to route packets to the best cpu. And what it measures is progress in processing packets I.e. performance. Please correct me if I'm wrong.
[Ola] Raw performance is not the only metric. Appropriate QoS and latency for different traffic classes is important. Efficient utilisation of the cores allocated to networking (not only packet processing but also other types of dataplane processing). Etc.
See you next week.
/amit On 24 Oct 2013 15:46, "Ola Liljedahl" <Ola.Liljedahl@arm.commailto:Ola.Liljedahl@arm.com> wrote: Amit,
OpenEM is a software implementation of the NSN Event Machine (EM). There are also a couple of HW-specific implementations of EM from the respective HW (SoC) vendors. EM is a fundamental part of OpenDataPlane (ODP) which you will hear more about at Linaro Connect next week. LNG is using OpenEM as a basis for the ODP reference implementation. Originally OpenEM was designed for running on x86 and utilising Intel DPDK for device (network interface) access etc. Linaro and NSN are liberating OpenEM from DPDK dependencies and the ODP reference implementation will run on vanilla Linux without needing DPDK (I think it already does).
Networking and dataplane applications will probably have different power management strategies compared to servers and client devices. A SoC may be running a mix of dataplane and other (e.g. control plane) applications and need different power management strategies (for different cores) on the same SoC. I have had some initial discussions with Charles Garcia-Tobin and Morten Rasmussen about this. Maybe something we can discuss at Connect.
-- Ola
Ola Liljedahl, Networking System Architect, ARM Telephone: +46 706 866 373tel:%2B46%20706%20866%20373 Skype: ola.liljedahl
-----Original Message----- From: Amit Kucheria [mailto:amit.kucheria@linaro.orgmailto:amit.kucheria@linaro.org] Sent: 24 October 2013 14:56 To: Maxim Uvarov; Vincent Guittot Cc: linaro-networking; Lists linaro-kernel; Patch Tracking Subject: [LNG] Re: [PATCH] add openem x86 tests
Hi Maxim,
Pardon my ignorance but where are these tests being added to?
I just read very quickly about openem[1] and this might be something we're interested in using as a test for our scheduler work. Can you describe a bit more where openem originates and how you use it inside LNG?
Regards, Amit [1] http://sourceforge.net/projects/eventmachine/files/Documents/EM_introductio n_1_0.pdf/download
On Thu, Oct 24, 2013 at 6:17 PM, Maxim Uvarov <maxim.uvarov@linaro.orgmailto:maxim.uvarov@linaro.org> wrote:
Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.orgmailto:maxim.uvarov@linaro.org>
ubuntu/openem.yaml | 28 ++++++++++++++++++++++++++ ubuntu/scripts/openem.sh | 50
++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 78 insertions(+) create mode 100644 ubuntu/openem.yaml create mode 100755 ubuntu/scripts/openem.sh
diff --git a/ubuntu/openem.yaml b/ubuntu/openem.yaml new file mode 100644 index 0000000..1a3939d --- /dev/null +++ b/ubuntu/openem.yaml @@ -0,0 +1,28 @@ +metadata:
- name: thp
- format: "Lava-Test-Shell Test Definition 1.0"
- description: "Test for open event machine x86."
+install:
- git-repos:
- git://git.linaro.org/qa/test-definitions.git<http://git.linaro.org/qa/test-definitions.git>- deps:
- python-twisted- git- make- unzip- gcc- build-essential- libc6-dev- gcc-multilib+run:
- steps:
- "cd testcases/ubuntu/scripts"- "./openem.sh"+parse:
- pattern: "^(?P<test_case_id>[a-zA-Z0-9_-]+):\s(?P<result>\wfile:///\\w+)"
- fixupdict:
PASS: passFAIL: faildiff --git a/ubuntu/scripts/openem.sh b/ubuntu/scripts/openem.sh new file mode 100755 index 0000000..63bb028 --- /dev/null +++ b/ubuntu/scripts/openem.sh @@ -0,0 +1,50 @@ +#!/bin/bash -x
+mount -t tmpfs none /dev/shm +echo 400 > /proc/sys/vm/nr_hugepages
+rm -rf ./Build +mkdir ./Build +cd ./Build
+wget
http://people.linaro.org/~maxim.uvarov/Intel_DPDK%20Code_1.5.0_8.zip
+unzip Intel_DPDK\ Code_1.5.0_8.zip +mv DPDK-1.5.0 DPDK_1.5.0-8
+git clone git://git.linaro.org/lng/eventmachine-code.githttp://git.linaro.org/lng/eventmachine-code.git eventmachine-
code.git
+patch -p0 < ./eventmachine-code.git/misc/linux-
generic/patch/DPDK_1.5.0-8-linux-generic.patch
+cd DPDK_1.5.0-8
+sed -i 's/CONFIG_RTE_MAX_MEMZONE=2560/CONFIG_RTE_MAX_MEMZONE=131072/'
./config/defconfig_*
+sed -i
's/CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE=512/CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE= 2048/' ./config/defconfig_*
+sed -i
's/CONFIG_RTE_MBUF_SCATTER_GATHER=y/CONFIG_RTE_MBUF_SCATTER_GATHER=n/' ./config/defconfig_*
+sed -i
's/CONFIG_RTE_MBUF_REFCNT_ATOMIC=y/CONFIG_RTE_MBUF_REFCNT_ATOMIC=n/' ./config/defconfig_*
+sed -i
's/CONFIG_RTE_PKTMBUF_HEADROOM=128/CONFIG_RTE_PKTMBUF_HEADROOM=192/' ./config/defconfig_*
+make install T=generic_32-default-linuxapp-gcc
+export RTE_SDK=`pwd` +export RTE_TARGET=generic_32-default-linuxapp-gcc +#export RTE_TARGET=generic_64-default-linuxapp-gcc
+mkdir /mnt/huge +umount /mnt/huge +mount -t hugetlbfs nodev /mnt/huge
+#EM +cd ../eventmachine-code.git/event_test/example/linux-generic +make real_clean && make em_clean +make
+#run some tests: +./build/hello -c 0xfe -n 4 -- -p | head -n 500 # (Run 'hello' on 7
cores using EM process-per-core mode(-p))
+./build/hello -c 0xfcfc -n 4 -- -t | head -n 500 # (Run 'hello' on
12 cores using EM thread-per-core mode(-t))
+./build/perf -c 0xffff -n 4 -- -p | head -n 500 # (Run 'perf' on
16 cores using EM process-per-core mode(-p))
+./build/perf -c 0xfffe -n 4 -- -t | hean -n 500 # (Run 'perf' on
15 cores using EM thread-per-core mode(-t))
+./build/event_group -c 0x0c0c -n 4 -- -p | head -n 500 # (Run
'event_group' on 4 cores using EM process-per-core mode(-p))
+./build/event_group -c 0x00f0 -n 4 -- -t | head -n 500 # (Run
'event_group' on 4 cores using EM thread-per-core mode(-t))
+./build/error -c 0x3 -n 4 -- -p |head -n 500 # (Run 'error' on 2
cores using EM process-per-core mode(-p))
+./build/error -c 0x2 -n 4 -- -t |head -n 500 # (Run 'error' on 1
core using EM thread-per-core mode(-t))
+echo "OPENEM TEST END!!!"
-- 1.7.9.5
linaro-kernel mailing list linaro-kernel@lists.linaro.orgmailto:linaro-kernel@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-kernel
-- You received this message because you are subscribed to the Google Groups "Linaro Networking" group. To unsubscribe from this group and stop receiving emails from it, send an email to linaro-networking+unsubscribe@linaro.orgmailto:linaro-networking%2Bunsubscribe@linaro.org. To post to this group, send email to linaro-networking@linaro.orgmailto:linaro-networking@linaro.org.
-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2557590 ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2548782 -- You received this message because you are subscribed to the Google Groups "Linaro Networking" group. To unsubscribe from this group and stop receiving emails from it, send an email to linaro-networking+unsubscribe@linaro.orgmailto:linaro-networking+unsubscribe@linaro.org. To post to this group, send email to linaro-networking@linaro.orgmailto:linaro-networking@linaro.org.
-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2557590 ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2548782
linaro-kernel@lists.linaro.org
-
Amit Kucheria -
Maxim Uvarov -
Ola Liljedahl