Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 2c35fd0e authored by Lajos Molnar's avatar Lajos Molnar
Browse files

media: document output image crop and layout for MediaCodec 

These use hidden, now deprecated MediaFormat keys.

Bug: 14127601
Change-Id: Ib51e4ef41204c9be34ad571300fcb7860feb504e
parent 34ccbe03

media/java/android/media/MediaCodec.java

+71 −0
Original line number Diff line number Diff line
@@ -125,6 +125,77 @@ import java.util.Map; 
 All video codecs support flexible YUV 4:2:0 buffers since {@link

 android.os.Build.VERSION_CODES#LOLLIPOP_MR1}.



 <h4>Accessing Raw Video ByteBuffers on Older Devices</h4>

 <p>

 Prior to {@link android.os.Build.VERSION_CODES#LOLLIPOP} and {@link Image} support, you need to

 use the {@link MediaFormat#KEY_STRIDE} and {@link MediaFormat#KEY_SLICE_HEIGHT} output format

 values to understand the layout of the raw output buffers.

 <p class=note>

 Note that on some devices the slice-height is advertised as 0. This could mean either that the

 slice-height is the same as the frame height, or that the slice-height is the frame height

 aligned to some value (usually a power of 2). Unfortunately, there is no way to tell the actual

 slice height in this case. Furthermore, the vertical stride of the {@code U} plane in planar

 formats is also not specified or defined, though usually it is half of the slice height.

 <p>

 The {@link MediaFormat#KEY_WIDTH} and {@link MediaFormat#KEY_HEIGHT} keys specify the size of the

 video frames; however, for most encondings the video (picture) only occupies a portion of the

 video frame. This is represented by the 'crop rectangle'.

 <p>

 You need to use the following keys to get the crop rectangle of raw output images from the

 {@linkplain #getOutputFormat output format}. If these keys are not present, the video occupies the

 entire video frame.The crop rectangle is understood in the context of the output frame

 <em>before</em> applying any {@linkplain MediaFormat#KEY_ROTATION rotation}.

 <table style="width: 0%">

  <thead>

   <tr>

    <th>Format Key</th>

    <th>Type</th>

    <th>Description</th>

   </tr>

  </thead>

  <tbody>

   <tr>

    <td>{@code "crop-left"}</td>

    <td>Integer</td>

    <td>The left-coordinate (x) of the crop rectangle</td>

   </tr><tr>

    <td>{@code "crop-top"}</td>

    <td>Integer</td>

    <td>The top-coordinate (y) of the crop rectangle</td>

   </tr><tr>

    <td>{@code "crop-right"}</td>

    <td>Integer</td>

    <td>The right-coordinate (x) <strong>MINUS 1</strong> of the crop rectangle</td>

   </tr><tr>

    <td>{@code "crop-bottom"}</td>

    <td>Integer</td>

    <td>The bottom-coordinate (y) <strong>MINUS 1</strong> of the crop rectangle</td>

   </tr><tr>

    <td colspan=3>

     The right and bottom coordinates can be understood as the coordinates of the right-most

     valid column/bottom-most valid row of the cropped output image.

    </td>

   </tr>

  </tbody>

 </table>

 <p>

 The size of the video frame (before rotation) can be calculated as such:

 <pre class=prettyprint>

 MediaFormat format = decoder.getOutputFormat(&hellip;);

 int width = format.getInteger(MediaFormat.KEY_WIDTH);

 if (format.containsKey("crop-left") && format.containsKey("crop-right")) {

     width = format.getInteger("crop-right") + 1 - format.getInteger("crop-left");

 }

 int height = format.getInteger(MediaFormat.KEY_HEIGHT);

 if (format.containsKey("crop-top") && format.containsKey("crop-bottom")) {

     height = format.getInteger("crop-bottom") + 1 - format.getInteger("crop-top");

 }

 </pre>

 <p class=note>

 Also note that the meaning of {@link BufferInfo#offset BufferInfo.offset} was not consistent across

 devices. On some devices the offset pointed to the top-left pixel of the crop rectangle, while on

 most devices it pointed to the top-left pixel of the entire frame.



 <h3>States</h3>

 <p>

 During its life a codec conceptually exists in one of three states: Stopped, Executing or

media/java/android/media/MediaFormat.java

+6 −4
Original line number Diff line number Diff line
@@ -295,17 +295,19 @@ public final class MediaFormat { 
     * Stride (or row increment) is the difference between the index of a pixel

     * and that of the pixel directly underneath. For YUV 420 formats, the

     * stride corresponds to the Y plane; the stride of the U and V planes can

     * be calculated based on the color format.

     * be calculated based on the color format, though it is generally undefined

     * and depends on the device and release.

     * The associated value is an integer, representing number of bytes.

     */

    public static final String KEY_STRIDE = "stride";



    /**

     * A key describing the plane height of a multi-planar (YUV) video bytebuffer layout.

     * Slice height (or plane height) is the number of rows that must be skipped to get

     * from the top of the Y plane to the top of the U plane in the bytebuffer. In essence

     * Slice height (or plane height/vertical stride) is the number of rows that must be skipped

     * to get from the top of the Y plane to the top of the U plane in the bytebuffer. In essence

     * the offset of the U plane is sliceHeight * stride. The height of the U/V planes

     * can be calculated based on the color format.

     * can be calculated based on the color format, though it is generally undefined

     * and depends on the device and release.

     * The associated value is an integer, representing number of rows.

     */

    public static final String KEY_SLICE_HEIGHT = "slice-height";