yuv2RGBconverter PullRequest follow-up

[ggfan]

This is to follow up with yuv2RGBConverter Pull requests:
#315,
#330,
#336.
like to have a short discussion and consolidate to a solution here, thanks!

[ggfan]

For #330, it does not seems to be a good solution to have that comment in the code. With many discussions to the converter, like to focus on other changes around the converter, pass this #330. Any concerns?

[gordinmitya]

Hello @ggfan, nice to hear from you!
I think my pr includes #330 (code supports CameraX ImageProxy).
And it's idea is somehow similar to #315, except it uses direct ButeBuffer as an output instead of byteArray:

• This is more efficient because uses copying from direct ByteBuffer (camera outputs in this type) into another direct buffer
• more universal and can be used with, for example, OpenCV converter, or some neural network framework. Please, take a look https://github.com/gordinmitya/yuv2buf/

[alexcohn]

Sorry to be late to this celebration, have just returned from a long-deserved vacation in the middle of nowhere with no internet access.

It is not more efficient to copy (in Java) from one DirectByteBuffer to another than to copy to ByteArray, and access of such array from C++ is just as efficient. But unfortunately the price of converted.buffer.get(bytes) is prohibitive.

[ggfan]

Thank you @gordinmitya for the PR and the explanation. Both #315 and #336 shows performance gains, I will look at them, particularly #315, and then follow up in this thread. Code improvements are welcome.

Are you planning retarget the PR to this repo?

[ggfan]

#405 gives ~30% gains; #315 shows 100% gains! For sample code maintenance purpose, like to go with #405 to favor readability. However performance is important when coming to image processing and graphics, I would add a comment at the top of final conversion function to point to the stack overflow's discussion used by #315. let me know for any concerns, thank you again for the PRs!

[alexcohn]

Sorry to be late to this celebration, have just returned from a long-deserved vacation in the middle of nowhere with no internet access.

IMHO, use of ByteBuffer in #405 causes the major slowdown. Why, oh why, does not RenderScript work directly with DirectByteBuffer?

So the bottom line, if you like the code style of #405 better, I would recommend to edit it to work with ByteArray before merging it into the main branch.

[ggfan]

thank you for your comment. I will compare with [this](https://github

• .com/androidx/androidx/blob/androidx-main/camera/camera-core/src/main/java/androidx/camera/core/ImageYuvToRgbConverter.java) see how it stands. Renderscript is deprecated(not removed though), the overall path for this color conversion should be updated to other approaches.

[alexcohn]

Being engaged in other corners of mobile development recently, I totally missed the deprecation announcement. The native convertAndroid420ToABGR method does not need all the sophisticated logic that I introduced in #315 and should (if implemented optimally) deliver better performance with less memcpy involved. At least, this was my finding using NEON optimized C++ code. I hope that the converter uses the correct LUT for YCrCb encoding, not the video-based BT.601 color space (with Y in [16, 235] range).

The objective of #315 was to keep as close to the original code as possible, and avoid introducing native methods.

[ggfan]

just for the early glimpse of it, the internal conversion function works, and, to application, it is simpler:

• enable it
• ARGB bits are in the ImageProxy.planes[0].buffer
• No performance degradation comparing to the existing code(renderscript conversion of 640x440 image).

this kind of closes this thread. If you find some issues, let us know.