Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit ede89893 authored by Deva Ramasubramanian's avatar Deva Ramasubramanian Committed by Arne Coucheron
Browse files

libstagefright: Fix out of bounds error for H264 software decoder 

If decoded clip's level is 1.0, InitDPB() and AVCConfigureSequence()
compute different number of frames (num_fs) to allocate.  InitDPB()
holds an extra frame if num_fs is 1, where as AVCCS() does not. Since
InitDPB() sets the state and AVCCS() allocates the buffers, num_fs - 1
buffers will be allocated.  During decode, we'll attempt to access the
num_fs-th frame which will be out of bounds.

This fix sets AVCCS()'s copy of num_fs to be same as whatever InitDPB()
computes.

Change-Id: I4bb5afdb4750084f0737878445f59842cd22a6f7
CRs-Fixed: 276145
parent f3236777

media/libstagefright/codecs/avc/common/src/dpb.cpp

+7 −5
Original line number Diff line number Diff line
@@ -132,6 +132,13 @@ OSCL_EXPORT_REF AVCStatus AVCConfigureSequence(AVCHandle *avcHandle, AVCCommonOb 
        {

            num_fs = MAX_FS;

        }



        CleanUpDPB(avcHandle, video);

        if (InitDPB(avcHandle, video, FrameHeightInMbs, PicWidthInMbs, padding) != AVC_SUCCESS)

        {

            return AVC_FAIL;

        }

        num_fs = dpb->num_fs;

#ifdef PV_MEMORY_POOL

        if (padding)

        {
@@ -143,11 +150,6 @@ OSCL_EXPORT_REF AVCStatus AVCConfigureSequence(AVCHandle *avcHandle, AVCCommonOb 
            avcHandle->CBAVC_DPBAlloc(avcHandle->userData, PicSizeInMapUnits, num_fs);

        }

#endif

        CleanUpDPB(avcHandle, video);

        if (InitDPB(avcHandle, video, FrameHeightInMbs, PicWidthInMbs, padding) != AVC_SUCCESS)

        {

            return AVC_FAIL;

        }

        /*  Allocate video->mblock upto PicSizeInMbs and populate the structure  such as the neighboring MB pointers.   */

        framesize = (FrameHeightInMbs * PicWidthInMbs);

        if (video->mblock)