My name is Robert Fekete and I work in the ST-Ericsson Landing Team.
I have a question regarding multicore SMP aware memory operations libc, and I hope you can have an answer or at least direct me to someone who might know better. I have already mailed Michael Hope who recommended me to mail this list(of course).
The issue I am experiencing is that when I am running a memset operation(on different non overlapping memory areas) on both cores simultaneously they will affect each other and prolong the work of each processes/cpu's by a factor of ~3-4(measured in Lauterabach).
i.e. assume a memset op that takes around 1ms (some really big buffer in a for loop many times)
Non-simultaneous memsets: (total time 2ms, each process execution time 1ms, x == running)
core1 :xxxxoooo
core2 :ooooxxxx
Simultaneous memsets: (total time 3ms , each process 3ms)
core1 :xxxxxxxxxxxx
core2 :xxxxxxxxxxxx
Well there are some factors that can explain parts of it.
There is a bandwidth limitation which will peak on one memset meaning that two memsets cannot possibly go faster and a proper value should be around 2ms(not 3-4ms), The L2$ is 512k which may lead to severe L2$ mess up since the sceduler is extremely fair and fine granular between the cores. L1$ will also get hurt.
But the real issue is that the two totally different memops(big ones though) in parallell will mess up for each other. Not good if one of the processes on core 1(for example) is a high prio compositor and the second process on core 2 is a lowprio crap application. The the Low prio crap App will severely mess up for the compositor. OS prio does not propagate to bus access prio.
Is there a way to make this behaviour a bit more deterministic...for the high prio op at least, like a scalable memory operation libc variant.
I read an article about a similar issue on Intel CPUs and the solution in this case is a scalable memory allocator lib.
http://software.intel.com/en-us/blogs/2009/08/21/is-your-memory-management-multi-core-ready/
More academic reading of the phenomenon:
http://books.google.com/books?id=NF-C2ZQZXekC&pg=PA353&lpg=PA353&dq=multi+core+memset+linux&source=bl&ots=O8mroWf4JM&sig=DZ9hWV-EozkaR9Kawakn5sB8FwQ&hl=en&ei=sgvkTN64CcaJhQf9xczcDg&sa=X&oi=book_result&ct=result&resnum=4&ved=0CCYQ6AEwAzgK#v=onepage&q&f=false
BR
/Robert Fekete