Hi there,
I took a look at the AOSP libjpeg code which is included in
git://android.git.kernel.org/platform/external/jpeg
during my flight back home (which incidentally had been diverted and landed me in Rio de Janeiro; not sure if I celebrate or cry) and noted the following things:
- There is a v6 implementation of the fast IDCT algorithm which lives in armv6_idct.S.
- The commit which adds this implementation was added October 2010, and there haven't been any changes since.
- The code that selects the decoder IDCT implementation in jddctmgr.c always uses that implementation if ANDROID_ARMV6_IDCT is defined.
- Google have an "ashmem" backing store implementation, and have code to enable tile-based mode. It's a fairly non-intrusive change to use ashmem since it just replaces jpeg_open_backing_store.
- The code is pretty much standard libjpeg without any structural changes to it.
- There isn't any NEON code in this branch.
- Mans has an optimized version here:
http://git.mansr.com/?p=libjpeg%3Ba=summary
I don't know if he's submitted this to AOSP or not.
This suggests to me that a simple drop-in of libjpeg-turbo might be actually easy to do, and that there is probably a significant performance benefit to be achieved. One thing to keep in mind is that this code still supports armv6, so we'd probably want to preserve that.
Thanks!
Yes, I'd think we'd want to merge the v6 support into libjpeg-turbo and verify its correct operation before trying to replace the version of libjpeg in Android. Also, v6 would need to be selected using the same mechanisms (or similar) to the ones we currently use to select NEON.
I also wanted to let you guys know that I have set up a libjpeg-turbo-devel list (https://lists.sourceforge.net/lists/listinfo/libjpeg-turbo-devel) which can be used to submit patches to the project or talk about development topics specific to the libjpeg-turbo code. You can also use the Patch tracker on Sourceforge to submit patches and discuss them.
On 6/22/11 8:30 PM, Christian Robottom Reis wrote:
Hi there,
I took a look at the AOSP libjpeg code which is included in git://android.git.kernel.org/platform/external/jpegduring my flight back home (which incidentally had been diverted and landed me in Rio de Janeiro; not sure if I celebrate or cry) and noted the following things:
- There is a v6 implementation of the fast IDCT algorithm which lives in armv6_idct.S. - The commit which adds this implementation was added October 2010, and there haven't been any changes since. - The code that selects the decoder IDCT implementation in jddctmgr.c always uses that implementation if ANDROID_ARMV6_IDCT is defined. - Google have an "ashmem" backing store implementation, and have code to enable tile-based mode. It's a fairly non-intrusive change to use ashmem since it just replaces jpeg_open_backing_store. - The code is pretty much standard libjpeg without any structural changes to it. - There isn't any NEON code in this branch. - Mans has an optimized version here: http://git.mansr.com/?p=libjpeg;a=summary I don't know if he's submitted this to AOSP or not.This suggests to me that a simple drop-in of libjpeg-turbo might be actually easy to do, and that there is probably a significant performance benefit to be achieved. One thing to keep in mind is that this code still supports armv6, so we'd probably want to preserve that.
Thanks!
On Thu, Jun 23, 2011 at 3:30 AM, Christian Robottom Reis kiko@linaro.org wrote:
Hi there,
I took a look at the AOSP libjpeg code which is included in
git://android.git.kernel.org/platform/external/jpeg
during my flight back home (which incidentally had been diverted and landed me in Rio de Janeiro; not sure if I celebrate or cry) and noted the following things:
- There is a v6 implementation of the fast IDCT algorithm which lives in armv6_idct.S.
- The commit which adds this implementation was added October 2010, and there haven't been any changes since.
- The code that selects the decoder IDCT implementation in jddctmgr.c always uses that implementation if ANDROID_ARMV6_IDCT is defined.
- Google have an "ashmem" backing store implementation, and have code to enable tile-based mode. It's a fairly non-intrusive change to use ashmem since it just replaces jpeg_open_backing_store.
- The code is pretty much standard libjpeg without any structural changes to it.
- There isn't any NEON code in this branch.
- Mans has an optimized version here:
http://git.mansr.com/?p=libjpeg%3Ba=summary
I don't know if he's submitted this to AOSP or not.
This suggests to me that a simple drop-in of libjpeg-turbo might be actually easy to do, and that there is probably a significant performance benefit to be achieved. One thing to keep in mind is that this code still supports armv6, so we'd probably want to preserve that.
sounds good. Seems like a great starter project for MM WG to get their hands dirty on android.
They could have three builds:
1. plain LEB 2. LEB with mans optimized libjpeg 3. LEB with libjpeg-turbo
Then benchmark all three and keep AOSP community posted on our intent, progress and results.
On 23/06/11 11:30, Alexander Sack wrote:
sounds good. Seems like a great starter project for MM WG to get their hands dirty on android.
They could have three builds:
- plain LEB
- LEB with mans optimized libjpeg
- LEB with libjpeg-turbo
Then benchmark all three and keep AOSP community posted on our intent, progress and results.
Hi folks
since we have been busy to get the public plan review material going, this deliverable to Android AOSP will not make it for 11.06 under the current pressure of time.
Pending Kurt and Mandeep saying otherwise, we will aim for this to happen in 11.07
Many thanks,
We've got a BP for this:
https://blueprints.launchpad.net/linaro-android/+spec/linaro-android-try-lib...
On 23 June 2011 03:30, Alexander Sack asac@linaro.org wrote:
On Thu, Jun 23, 2011 at 3:30 AM, Christian Robottom Reis kiko@linaro.org wrote:
Hi there,
I took a look at the AOSP libjpeg code which is included in
git://android.git.kernel.org/platform/external/jpeg
during my flight back home (which incidentally had been diverted and landed me in Rio de Janeiro; not sure if I celebrate or cry) and noted the following things:
- There is a v6 implementation of the fast IDCT algorithm which lives in armv6_idct.S.
- The commit which adds this implementation was added October 2010, and there haven't been any changes since.
- The code that selects the decoder IDCT implementation in jddctmgr.c always uses that implementation if ANDROID_ARMV6_IDCT is defined.
- Google have an "ashmem" backing store implementation, and have code to enable tile-based mode. It's a fairly non-intrusive change to use ashmem since it just replaces jpeg_open_backing_store.
- The code is pretty much standard libjpeg without any structural changes to it.
- There isn't any NEON code in this branch.
- Mans has an optimized version here:
http://git.mansr.com/?p=libjpeg%3Ba=summary
I don't know if he's submitted this to AOSP or not.
This suggests to me that a simple drop-in of libjpeg-turbo might be actually easy to do, and that there is probably a significant performance benefit to be achieved. One thing to keep in mind is that this code still supports armv6, so we'd probably want to preserve that.
sounds good. Seems like a great starter project for MM WG to get their hands dirty on android.
They could have three builds:
1. plain LEB 2. LEB with mans optimized libjpeg 3. LEB with libjpeg-turbo
Then benchmark all three and keep AOSP community posted on our intent, progress and results.
--
- Alexander
Hi All,
I have done some benchmarking on OMAP4 running Ubuntu for various versions of libjpegs. Benchmarks were collected with modified version of djpeg that prints out ms time taken for decoding. Sample used for benchmarking is a 12MP image downloaded from a photography website. Here are the results:
*Non Optimized libjpeg-turbo(5 runs):* Decoding Time for Run 1: 2022 ms Decoding Time for Run 2: 2029 ms Decoding Time for Run 3: 2165 ms Decoding Time for Run 4: 2027 ms Decoding Time for Run 5: 2150 ms * Median Decoding Time: 2029 ms*
*Linaro's NEON Optimized libjpeg-turbo done in last cycle (5 runs)* http://git.linaro.org/gitweb?p=people/mandeep-ti/libjpeg-turbo-1.1.0_linaro.... Decoding Time for Run 1: 1634 ms Decoding Time for Run 2: 1634 ms Decoding Time for Run 3: 1636 ms Decoding Time for Run 4: 1739 ms Decoding Time for Run 5: 1738 ms * Median Decoding Time: 1636 ms* *Mans Rullgard's libjpeg based of libjpeg 6b (5 runs)* http://git.mansr.com/?p=libjpeg%3Ba=summary Decoding Time for Run 1: 1101 ms Decoding Time for Run 2: 1081 ms Decoding Time for Run 3: 1081 ms Decoding Time for Run 4: 1125 ms Decoding Time for Run 5: 1089 ms * Median Decoding Time: 1089 ms
libjpeg-turbo trunk version that has NEON patches (5 runs). * http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/* * Decoding Time for Run 1: 1068 ms Decoding Time for Run 2: 1065 ms Decoding Time for Run 3: 1093 ms Decoding Time for Run 4: 1066 ms Decoding Time for Run 5: 1067 ms *Median Decoding Time: 1067 ms*
So basically libjpeg-turbo as of now is pretty much as optimized as anything out there. As a part of my last cycle work I also produced additional color conversion routines that I will submit to libjpeg-turbo project after that libjpeg-turbo should be best library to go (given that it is also highly optimized for Intel processors).
Thanks & Regards, Mandeep * *
On 24 June 2011 17:09, Zach Pfeffer zach.pfeffer@linaro.org wrote:
We've got a BP for this:
https://blueprints.launchpad.net/linaro-android/+spec/linaro-android-try-lib...
On 23 June 2011 03:30, Alexander Sack asac@linaro.org wrote:
On Thu, Jun 23, 2011 at 3:30 AM, Christian Robottom Reis kiko@linaro.org wrote:
Hi there,
I took a look at the AOSP libjpeg code which is included in
git://android.git.kernel.org/platform/external/jpegduring my flight back home (which incidentally had been diverted and landed me in Rio de Janeiro; not sure if I celebrate or cry) and noted the following things:
There is a v6 implementation of the fast IDCT algorithm which lives in armv6_idct.S.
The commit which adds this implementation was added October 2010, and there haven't been any changes since.
The code that selects the decoder IDCT implementation in jddctmgr.c always uses that implementation if ANDROID_ARMV6_IDCT is defined.
Google have an "ashmem" backing store implementation, and have code to enable tile-based mode. It's a fairly non-intrusive change to use ashmem since it just replaces jpeg_open_backing_store.
The code is pretty much standard libjpeg without any structural changes to it.
There isn't any NEON code in this branch.
Mans has an optimized version here:
http://git.mansr.com/?p=libjpeg%3Ba=summary
I don't know if he's submitted this to AOSP or not.
This suggests to me that a simple drop-in of libjpeg-turbo might be actually easy to do, and that there is probably a significant performance benefit to be achieved. One thing to keep in mind is that this code still supports armv6, so we'd probably want to preserve that.
sounds good. Seems like a great starter project for MM WG to get their hands dirty on android.
They could have three builds:
- plain LEB
- LEB with mans optimized libjpeg
- LEB with libjpeg-turbo
Then benchmark all three and keep AOSP community posted on our intent, progress and results.
--
- Alexander
Mandeep Kumar wrote:
Hi All,
I have done some benchmarking on OMAP4 running Ubuntu for various versions of libjpegs. Benchmarks were collected with modified version of djpeg that prints out ms time taken for decoding. Sample used for benchmarking is a 12MP image downloaded from a photography website. Here are the results:
...
libjpeg-turbo trunk version that has NEON patches (5 runs). *http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/*
Decoding Time for Run 1: 1068 ms Decoding Time for Run 2: 1065 ms Decoding Time for Run 3: 1093 ms Decoding Time for Run 4: 1066 ms Decoding Time for Run 5: 1067 ms*Median Decoding Time: 1067 ms*
One remark:
a 12MP image decoded in 1076ms equals ~12MP/s decoding speed.
decoding a 640x480 MJPEG file on a 1GHz OMAP4 using libavcodec gives me an average decoding time per frame of ~10ms which yields:
640x480/10ms = ~30MP/s
so roughly 2.5 times faster.
Either I am doing something wrong or this libjpeg-turbo is not so turbo.
Regards,
Vladimir
Vladimir Pantelic vladoman@gmail.com writes:
Mandeep Kumar wrote:
Hi All,
I have done some benchmarking on OMAP4 running Ubuntu for various versions of libjpegs. Benchmarks were collected with modified version of djpeg that prints out ms time taken for decoding. Sample used for benchmarking is a 12MP image downloaded from a photography website. Here are the results:
...
libjpeg-turbo trunk version that has NEON patches (5 runs). *http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/*
Decoding Time for Run 1: 1068 ms Decoding Time for Run 2: 1065 ms Decoding Time for Run 3: 1093 ms Decoding Time for Run 4: 1066 ms Decoding Time for Run 5: 1067 ms*Median Decoding Time: 1067 ms*
One remark:
a 12MP image decoded in 1076ms equals ~12MP/s decoding speed.
decoding a 640x480 MJPEG file on a 1GHz OMAP4 using libavcodec gives me an average decoding time per frame of ~10ms which yields:
640x480/10ms = ~30MP/s
so roughly 2.5 times faster.
Either I am doing something wrong or this libjpeg-turbo is not so turbo.
Libjpeg (turbo or regular) is full of inefficiencies. I guess they all add up.
My numbers should only be used for relative comparison purpose as djpeg decoding time that I sent also include time to read input file from filessystem. Actual decoding speed may be higher when special modification are done in djpeg application to read input from memory.
Thanks & Regards, Mandeep
2011/6/29 Måns Rullgård mans@mansr.com
Vladimir Pantelic vladoman@gmail.com writes:
Mandeep Kumar wrote:
Hi All,
I have done some benchmarking on OMAP4 running Ubuntu for various
versions of libjpegs. Benchmarks were collected with
modified version of djpeg that prints out ms time taken for decoding.
Sample used for benchmarking is a 12MP image
downloaded from a photography website. Here are the results:
...
libjpeg-turbo trunk version that has NEON patches (5 runs). *
http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/*
Decoding Time for Run 1: 1068 ms Decoding Time for Run 2: 1065 ms Decoding Time for Run 3: 1093 ms Decoding Time for Run 4: 1066 ms Decoding Time for Run 5: 1067 ms*Median Decoding Time: 1067 ms*
One remark:
a 12MP image decoded in 1076ms equals ~12MP/s decoding speed.
decoding a 640x480 MJPEG file on a 1GHz OMAP4 using libavcodec gives me an average decoding time per frame of ~10ms which yields:
640x480/10ms = ~30MP/s
so roughly 2.5 times faster.
Either I am doing something wrong or this libjpeg-turbo is not so turbo.
Libjpeg (turbo or regular) is full of inefficiencies. I guess they all add up.
-- Måns Rullgård mans@mansr.com
Mandeep Kumar mandeep.kumar@linaro.org writes:
Vladimir Pantelic vladoman@gmail.com writes:
Mandeep Kumar wrote:
Hi All,
I have done some benchmarking on OMAP4 running Ubuntu for various versions of libjpegs. Benchmarks were collected with modified version of djpeg that prints out ms time taken for decoding. Sample used for benchmarking is a 12MP image downloaded from a photography website. Here are the results:
...
libjpeg-turbo trunk version that has NEON patches (5 runs). * http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/*
Decoding Time for Run 1: 1068 ms Decoding Time for Run 2: 1065 ms Decoding Time for Run 3: 1093 ms Decoding Time for Run 4: 1066 ms Decoding Time for Run 5: 1067 ms*Median Decoding Time: 1067 ms*
One remark:
a 12MP image decoded in 1076ms equals ~12MP/s decoding speed.
decoding a 640x480 MJPEG file on a 1GHz OMAP4 using libavcodec gives me an average decoding time per frame of ~10ms which yields:
640x480/10ms = ~30MP/s
so roughly 2.5 times faster.
Either I am doing something wrong or this libjpeg-turbo is not so turbo.
Libjpeg (turbo or regular) is full of inefficiencies. I guess they all add up.
My numbers should only be used for relative comparison purpose as djpeg decoding time that I sent also include time to read input file from filessystem. Actual decoding speed may be higher when special modification are done in djpeg application to read input from memory.
I doubt reading from a cached file can account for that much time. Besides, I presume Vladimir was reading his file from somewhere too.
However, no matter how slow libjpeg is, that's what lots of apps use, and thus improving it should be beneficial to them insofar jpeg decoding is a bottleneck.
Måns Rullgård wrote:
Mandeep Kumarmandeep.kumar@linaro.org writes:
Vladimir Pantelicvladoman@gmail.com writes:
Either I am doing something wrong or this libjpeg-turbo is not so turbo.
Libjpeg (turbo or regular) is full of inefficiencies. I guess they all add up.
My numbers should only be used for relative comparison purpose as djpeg decoding time that I sent also include time to read input file from filessystem. Actual decoding speed may be higher when special modification are done in djpeg application to read input from memory.
I doubt reading from a cached file can account for that much time. Besides, I presume Vladimir was reading his file from somewhere too.
my times have been measured around the libavcodec decode() call and include colorspace conversion to YUYV interleaved. I realize that libjpeg might have an additional YUV to RGB step that I did not do since my display is YUV capable.
However, no matter how slow libjpeg is, that's what lots of apps use, and thus improving it should be beneficial to them insofar jpeg decoding is a bottleneck.
sure, I did not mean for you to stop working on it :)
I still posit that it's possible to avoid many of those inefficiencies by using a sufficiently large buffer in libjpeg-turbo and using an in-memory source/destination manager. Much of the inefficiency in the code relates to the buffering that it does to avoid reading the entire image into memory.
I also hasten to point out that not all of the compute-intensive parts of the code are NEON-accelerated. The general speedup we're seeing in NEON vs non-NEON is about 1.5-2x rather than the 3-4x we see with x86-64. Not sure whether ARM is 64-bit, but using 64-bit code will improve Huffman en/decoding performance significantly. It may also be the case that the hand-tuned code I wrote in the Huffman codec is making performance assumptions based on x86 that aren't true for ARM. It would be interesting to see what the speedup is with the unoptimized Huffman code out of libjpeg. At least on x86, Huffman can account for 40% of the compute time, so optimizing it further has a potentially big pay-off. However, I've personally spent hundreds of hours getting it where it is, and I have a gut feeling that further optimization of it would require dropping down to assembly.
On Jun 29, 2011, at 3:03 PM, Måns Rullgård mans@mansr.com wrote:
Vladimir Pantelic vladoman@gmail.com writes:
Mandeep Kumar wrote:
Hi All,
I have done some benchmarking on OMAP4 running Ubuntu for various versions of libjpegs. Benchmarks were collected with modified version of djpeg that prints out ms time taken for decoding. Sample used for benchmarking is a 12MP image downloaded from a photography website. Here are the results:
...
libjpeg-turbo trunk version that has NEON patches (5 runs). *http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/*
Decoding Time for Run 1: 1068 ms*Median Decoding Time: 1067 ms*
One remark:
a 12MP image decoded in 1076ms equals ~12MP/s decoding speed.
decoding a 640x480 MJPEG file on a 1GHz OMAP4 using libavcodec gives me an average decoding time per frame of ~10ms which yields:
640x480/10ms = ~30MP/s
so roughly 2.5 times faster.
Either I am doing something wrong or this libjpeg-turbo is not so turbo.
Libjpeg (turbo or regular) is full of inefficiencies. I guess they all add up.
-- Måns Rullgård mans@mansr.com
From: linaro-dev-bounces@lists.linaro.org [mailto:linaro-dev- bounces@lists.linaro.org] On Behalf Of Christian Robottom Reis Sent: Wednesday, June 22, 2011 8:30 PM
This suggests to me that a simple drop-in of libjpeg-turbo might be actually easy to do, and that there is probably a significant performance benefit to be achieved. One thing to keep in mind is that this code still supports armv6, so we'd probably want to preserve that.
You have to be a little careful as ARM did drop some bad NEON errata in A9 which require care to work around. Some CPU versions are impacted while others are not, tanstaafl.
Regards, Richard W.
On 06/24/2011 08:13 PM, Woodruff, Richard wrote:
This suggests to me that a simple drop-in of libjpeg-turbo might be actually easy to do, and that there is probably a significant performance benefit to be achieved. One thing to keep in mind is that this code still supports armv6, so we'd probably want to preserve that.
You have to be a little careful as ARM did drop some bad NEON errata in A9 which require care to work around. Some CPU versions are impacted while others are not, tanstaafl.
AFAIK ARM has a tool that can scan object code for "bad" neon sequences, maybe they are willing to make that generally available?
Vladimir Pantelic vladoman@gmail.com writes:
On 06/24/2011 08:13 PM, Woodruff, Richard wrote:
This suggests to me that a simple drop-in of libjpeg-turbo might be actually easy to do, and that there is probably a significant performance benefit to be achieved. One thing to keep in mind is that this code still supports armv6, so we'd probably want to preserve that.
You have to be a little careful as ARM did drop some bad NEON errata in A9 which require care to work around. Some CPU versions are impacted while others are not, tanstaafl.
AFAIK ARM has a tool that can scan object code for "bad" neon sequences, maybe they are willing to make that generally available?
I asked them about the that tool, and they confirmed its existence, but expressed some reluctance about releasing it, mainly due to lack of internal procedures for doing so. If more people asked about it, perhaps they could be persuaded to release it.
-
Alexander Sack -
Christian Robottom Reis -
DRC -
Ilias Biris -
Mandeep Kumar -
Måns Rullgård -
Vladimir Pantelic -
Woodruff, Richard -
Zach Pfeffer