Opencv Android - Color Issue Using Camerabridgeviewbase

I'm encountering a strange problem using Android emulators & OpenCV CameraBridgeViewBase. Using onCameraFrame I get a picture that looks like it wasn't decoded properly. public

Solution 1:

So, after drilling into the problem I've found the root of the issue.

Let's take a look at the OpenCV JavaCameraView class and its CameraBridgeViewBase base class. The problem was that camera frames received as byte[] array in onPreviewFrame method were decoded incorrectly.

The exact place of code where the decoding process takes place is an implementation of the Mat rgba() method in inner JavaCameraFrame class of the JavaCameraView:

public Mat rgba() {
        Imgproc.cvtColor(mYuvFrameData, mRgba, Imgproc.COLOR_YUV2RGBA_NV21, 4);
        return mRgba;
    }

As we see, Imgproc.cvtColor(...) method is used in order to convert frame from YUV to RGBA. NV21 YUV -> RGBA conversion takes place there. During the initialization process we set the format to NV21, so this should be right. Moreover, every Android device should support NV21. Also, we can check whether device accepted the format using debugger:

protected boolean initializeCamera(int width, int height) {
    ...
    params.setPreviewFormat(ImageFormat.NV21);
    ...
    mCamera.setParameters(params);
    ...
    params = mCamera.getParameters();
    Log.d(TAG, String.format("Actual preview format is 0x%X", params.getPreviewFormat()));
}

Both phone (HTC Sensation) and emulator reported to be using NV21 indeed.

However, if we change COLOR_YUV2RGBA_NV21 to COLOR_YUV2RGB_I420 (YV12 and I420 is the same thing, just with Y and V inverted;) we'll see that emulator will get a proper color space finally. Changing NV21 to YV12 in params.setPreviewFormat(ImageFormat.NV21); we'll get similar results. Looks like there's bug either in Imgproc.cvtColor, or in Android.

Here comes the solution. Change public Mat rgba() the following way:

    public Mat rgba() {
        if (previewFormat == ImageFormat.NV21) {
            Imgproc.cvtColor(mYuvFrameData, mRgba, Imgproc.COLOR_YUV2RGBA_NV21, 4);
        }
        else if (previewFormat == ImageFormat.YV12) {
            Imgproc.cvtColor(mYuvFrameData, mRgba, Imgproc.COLOR_YUV2RGB_I420, 4);  // COLOR_YUV2RGBA_YV12 produces inverted colors
        }
        return mRgba;
    }

previewFormat is a new int variable, it's declared this way:

privateint previewFormat = ImageFormat.NV21;

Add following changes to the initialization:

protected boolean initializeCamera(int width, int height) {
        ...
                params.setPreviewFormat(ImageFormat.NV21);
                // "generic" = android emulatorif (Build.BRAND.equalsIgnoreCase("generic")) {
                    params.setPreviewFormat(ImageFormat.YV12);
                }
                ...
                mCamera.setParameters(params);
                params = mCamera.getParameters();
                previewFormat = params.getPreviewFormat();
        ...
}

Important: Please note: this is just a temporary solution in order to make OpenCV usable with emulator in my case. Further research should be done. It's quite easy to check whether device uses correct image format in onPreviewFrame. I'll get back to this when I have some time.

Solution 2:

For those of you who use genymotion, the BRAND won't be call generic. To solve this just simply change the code to

...
if (Build.BRAND.equalsIgnoreCase("generic") | Build.BRAND.equalsIgnoreCase("Android")) {
...

and it should works, or just printout the BRAND you are using and replace it on the "Android"

P.S The second condition will include the case that you are using genymotion which is also an emulator.

Cheers

Solution 3:

This issue is fixed now in the code in git, as the Pull Request https://github.com/opencv/opencv/pull/8168 has been merged. It should be available in the next released version.