Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 13d38508 authored by Evan Rosky's avatar Evan Rosky
Browse files

Move config-at-end seamless-flip to activity-level 

The current setup was made to sorta match pip1 in the
sense that the config-at-end task and activity surfaces
remain untouched and shell does a "forward" animation (ie.
it animates from identity to a transformed state) and
then WM flips the content and the surface transform
together at the *end* of the animation.

The issue, here, is that it means there is a period of
time where WM is manipulating surfaces (Task-level) that
Shell technically "owns" and without any synchronization
mechanism: this means shell can't synchronize updates
to it's overlay and also must somehow prevent user
interaction with the surface while WM is messing with
it; however, WM never tells Shell when it is done.

Instead of making the sync/communication more complicated
(and error-prone), we move the content/transform-flipping
to the Activity-level (which WM "owns"). This changes the
dynamics such that shell is now responsible for a
"catch-up" animation (ie. animating from a "reverse
transformed" state into the current state). The benefits
are:
* Works like other transitions now (which also do catch-up).
* WM only manipulates surfaces that it "owns"
* Synchronization is now only on start (like other transitions).
* No need to bubble-up CONFIG_AT_END flag.

We still keep the old behavior for activity-level because
there's really no other way to do it -- but activity-level
doesn't have the same issues as task because it is only
loaned to shell for the animation.

Bug: 202201326
Test: atest TransitionTests#testConfigAtEnd
Flag: com.android.wm.shell.enable_pip2
Change-Id: Ice26e2414772555fae577ce517a8892634e716da
parent 0f67fcc5

libs/WindowManager/Shell/src/com/android/wm/shell/common/pip/PipUtils.kt

+83 −1
Original line number Diff line number Diff line
@@ -22,6 +22,7 @@ import android.app.WindowConfiguration 
import android.content.ComponentName

import android.content.Context

import android.content.pm.PackageManager

import android.graphics.PointF

import android.graphics.Rect

import android.os.RemoteException

import android.util.DisplayMetrics
@@ -29,10 +30,13 @@ import android.util.Log 
import android.util.Pair

import android.util.TypedValue

import android.window.TaskSnapshot

import android.window.TransitionInfo

import com.android.internal.protolog.ProtoLog

import com.android.wm.shell.Flags

import com.android.wm.shell.protolog.ShellProtoLogGroup

import kotlin.math.abs

import kotlin.math.ceil

import kotlin.math.floor

import kotlin.math.roundToInt



/** A class that includes convenience methods.  */
@@ -163,6 +167,84 @@ object PipUtils { 
        return Rect(left, top, left + width, top + height)

    }



    /**

     * Temporary rounding "outward" (ie. -1.2 -> -2) used for crop since it is an int. We lean

     * outward since, usually, child surfaces are, themselves, cropped, so we'd prefer to avoid

     * inadvertently cutting out content that would otherwise be visible.

     */

    private fun roundOut(`val`: Float): Int {

        return (if (`val` >= 0f) ceil(`val`) else floor(`val`)).toInt()

    }



    /**

     * Calculates the transform and crop to apply on a Task surface in order for the config-at-end

     * activity inside it (original-size activity transformed to match it's hint rect to the final

     * Task bounds) to occupy the same world-space position/dimensions as it had before the

     * transition.

     *

     * Usage example:

     *     calcStartTransform(pipChange, scale, pos, crop);

     *     t.setScale(pipChange.getLeash(), scale.x, scale.y);

     *     t.setPosition(pipChange.getLeash(), pos.x, pos.y);

     *     t.setCrop(pipChange.getLeash(), crop);

     */

    @JvmStatic

    fun calcStartTransform(pipChange: TransitionInfo.Change, outScale: PointF,

        outPos: PointF, outCrop: Rect) {

        val startBounds = pipChange.startAbsBounds

        val taskEndBounds = pipChange.endAbsBounds

        // For now, pip activity bounds always matches task bounds. If this ever changes, we'll

        // need to get the activity offset.

        val endBounds = taskEndBounds

        var hintRect = pipChange.taskInfo?.pictureInPictureParams?.sourceRectHint

        if (hintRect == null) {

            hintRect = Rect(startBounds)

            hintRect.offsetTo(0, 0)

        }



        // FA = final activity bounds (absolute)

        // FT = final task bounds (absolute)

        // SA = start activity bounds (absolute)

        // H = source hint (relative to start activity bounds)

        // We want to transform the activity so that when the task is at FT, H overlaps with FA



        // The scaling which takes the hint rect (H) in SA and matches it to FA

        val hintToEndScaleX = (endBounds.width().toFloat()) / (hintRect.width().toFloat())

        val hintToEndScaleY = (endBounds.height().toFloat()) / (hintRect.height().toFloat())



        // We want to set the transform on the END TASK surface to put the start activity

        // back to where it was.

        // First do backwards scale (which takes FA back to H)

        val endToHintScaleX = 1f / hintToEndScaleX

        val endToHintScaleY = 1f / hintToEndScaleY

        // Then top-left needs to place FA (relative to the FT) at H (relative to SA):

        //   so -(FA.tl - FT.tl) + SA.tl + H.tl

        //  but we have scaled up the task, so anything that was "within" the task needs to

        //  be scaled:

        //   so -(FA.tl - FT.tl)*endToHint + SA.tl + H.tl

        val endTaskPosForStartX = (-(endBounds.left - taskEndBounds.left) * endToHintScaleX

                + startBounds.left + hintRect.left)

        val endTaskPosForStartY = (-(endBounds.top - taskEndBounds.top) * endToHintScaleY

                + startBounds.top + hintRect.top)

        outScale[endToHintScaleX] = endToHintScaleY

        outPos[endTaskPosForStartX] = endTaskPosForStartY



        // now need to set crop to reveal the non-hint stuff. Again, hintrect is relative, so

        // we must apply outsets to reveal the *activity* content which is *inside* the task

        // and thus is scaled (ie. if activity is scaled down, each task-level pixel exposes

        // >1 activity-level pixels)

        // For example, the topleft crop would be:

        //   (FA.tl - FT.tl) - H.tl * hintToEnd

        //    ^ activity within task

        // bottomright can just use scaled activity size

        //   tl + scale(SA.size, hintToEnd)

        outCrop.left = roundOut((endBounds.left - taskEndBounds.left)

                - hintRect.left * hintToEndScaleX)

        outCrop.top = roundOut((endBounds.top - taskEndBounds.top) - hintRect.top * hintToEndScaleY)

        outCrop.right = roundOut(outCrop.left + startBounds.width() * hintToEndScaleX)

        outCrop.bottom = roundOut(outCrop.top + startBounds.height() * hintToEndScaleY)

    }



    private var isPip2ExperimentEnabled: Boolean? = null



    /**

services/core/java/com/android/server/wm/Transition.java

+73 −49
Original line number Diff line number Diff line
@@ -556,8 +556,6 @@ class Transition implements BLASTSyncEngine.TransactionReadyListener { 
            snapshotStartState(ar);

            mChanges.get(ar).mFlags |= ChangeInfo.FLAG_CHANGE_CONFIG_AT_END;

        });

        snapshotStartState(wc);

        mChanges.get(wc).mFlags |= ChangeInfo.FLAG_CHANGE_CONFIG_AT_END;

    }



    /** Set a transition to be a seamless-rotation. */
@@ -1083,7 +1081,8 @@ class Transition implements BLASTSyncEngine.TransactionReadyListener { 
            // For config-at-end, the end-transform will be reset after the config is actually

            // applied in the client (since the transform depends on config). The other properties

            // remain here because shell might want to persistently override them.

            if ((mTargets.get(i).mFlags & ChangeInfo.FLAG_CHANGE_CONFIG_AT_END) == 0) {

            if (target.asActivityRecord() == null

                    || (mTargets.get(i).mFlags & ChangeInfo.FLAG_CHANGE_CONFIG_AT_END) == 0) {

                resetSurfaceTransform(t, target, targetLeash);

            }

        }
@@ -1564,47 +1563,34 @@ class Transition implements BLASTSyncEngine.TransactionReadyListener { 
        if (mConfigAtEndActivities == null || mConfigAtEndActivities.isEmpty()) {

            return;

        }

        final SurfaceControl.Transaction t =

                mController.mAtm.mWindowManager.mTransactionFactory.get();

        for (int i = 0; i < mTargets.size(); ++i) {

            final WindowContainer target = mTargets.get(i).mContainer;

            if (target.getParent() == null || (mTargets.get(i).mFlags

                    & ChangeInfo.FLAG_CHANGE_CONFIG_AT_END) == 0) {

                continue;

        // Now resume the configuration dispatch, wait until the now resumed configs have been

        // drawn, and then apply everything together. Any activities that are already in an

        // active sync will remain on that sync instead of the new one.

        int syncId = -1;

        for (int i = 0; i < mConfigAtEndActivities.size(); ++i) {

            final ActivityRecord target = mConfigAtEndActivities.get(i);

            final SurfaceControl targetLeash = target.getSurfaceControl();

            if (target.getSyncGroup() == null || target.getSyncGroup().isIgnoring(target)) {

                if (syncId < 0) {

                    final BLASTSyncEngine.SyncGroup sg = mSyncEngine.prepareSyncSet(

                            (mSyncId, transaction) -> transaction.apply(),

                            "ConfigAtTransitEnd");

                    syncId = sg.mSyncId;

                    mSyncEngine.startSyncSet(sg, BLAST_TIMEOUT_DURATION, true /* parallel */);

                    mSyncEngine.setSyncMethod(syncId, BLASTSyncEngine.METHOD_BLAST);

                }

                mSyncEngine.addToSyncSet(syncId, target);

            }

            final SurfaceControl targetLeash = getLeashSurface(target, null /* t */);

            // Reset surface state here (since it was skipped in buildFinishTransaction). Since

            // we are resuming config to the "current" state, we have to calculate the matching

            // surface state now (rather than snapshotting it at animation start).

            resetSurfaceTransform(t, target, targetLeash);

        }



        // Now we resume the configuration dispatch, wait until the now resumed configs have been

        // drawn, and then apply everything together.

        final BLASTSyncEngine.SyncGroup sg = mSyncEngine.prepareSyncSet(

                new BLASTSyncEngine.TransactionReadyListener() {

                    @Override

                    public void onTransactionReady(int mSyncId,

                            SurfaceControl.Transaction transaction) {

                        t.merge(transaction);

                        t.apply();

                    }



                    @Override

                    public void onTransactionCommitTimeout() {

                        t.apply();

                    }

                }, "ConfigAtTransitEnd");

        final int syncId = sg.mSyncId;

        mSyncEngine.startSyncSet(sg, BLAST_TIMEOUT_DURATION, true /* parallel */);

        mSyncEngine.setSyncMethod(syncId, BLASTSyncEngine.METHOD_BLAST);

        for (int i = 0; i < mConfigAtEndActivities.size(); ++i) {

            final ActivityRecord ar = mConfigAtEndActivities.get(i);

            mSyncEngine.addToSyncSet(syncId, ar);

            ar.resumeConfigurationDispatch();

            resetSurfaceTransform(target.getSyncTransaction(), target, targetLeash);

            target.resumeConfigurationDispatch();

        }

        if (syncId >= 0) {

            mSyncEngine.setReady(syncId);

        }

    }



    @Nullable

    private ActivityRecord getVisibleTransientLaunch(TaskDisplayArea taskDisplayArea) {
@@ -1718,6 +1704,54 @@ class Transition implements BLASTSyncEngine.TransactionReadyListener { 
        change.mFlags |= ChangeInfo.FLAG_CHANGE_NO_ANIMATION;

    }



    void prepareConfigAtEnd(SurfaceControl.Transaction transact, ArrayList<ChangeInfo> targets) {

        if (mConfigAtEndActivities == null) return;

        for (int i = 0; i < mConfigAtEndActivities.size(); ++i) {

            final ActivityRecord ar = mConfigAtEndActivities.get(i);

            if (!ar.isVisibleRequested()) continue;

            final SurfaceControl sc = ar.getSurfaceControl();

            if (sc == null) continue;

            final Task task = ar.getTask();

            if (task == null) continue;

            // If task isn't animating, then it means shell is animating activity directly (within

            // task), so don't do any setup.

            if (!containsChangeFor(task, targets)) continue;

            final ChangeInfo change = mChanges.get(ar);

            final Rect startBounds = change.mAbsoluteBounds;

            Rect hintRect = null;

            if (ar.getWindowingMode() == WINDOWING_MODE_PINNED && ar.pictureInPictureArgs != null

                    && ar.pictureInPictureArgs.getSourceRectHint() != null) {

                hintRect = ar.pictureInPictureArgs.getSourceRectHint();

            }

            if (hintRect == null) {

                hintRect = new Rect(startBounds);

                hintRect.offsetTo(0, 0);

            }

            final Rect endBounds = ar.getBounds();

            final Rect taskEndBounds = task.getBounds();

            // FA = final activity bounds (absolute)

            // FT = final task bounds (absolute)

            // SA = start activity bounds (absolute)

            // H = source hint (relative to start activity bounds)

            // We want to transform the activity so that when the task is at FT, H overlaps with FA



            // This scales the activity such that the hint rect has the same dimensions

            // as the final activity bounds.

            float hintToEndScaleX = ((float) endBounds.width()) / ((float) hintRect.width());

            float hintToEndScaleY = ((float) endBounds.height()) / ((float) hintRect.height());

            // top-left needs to be (FA.tl - FT.tl) - H.tl * hintToEnd . H is relative to the

            // activity; so, for example, if shrinking H to FA (hintToEnd < 1), then the tl of the

            // shrunk SA is closer to H than expected, so we need to reduce how much we offset SA

            // to get H.tl to match.

            float startActPosInTaskEndX =

                    (endBounds.left - taskEndBounds.left) - hintRect.left * hintToEndScaleX;

            float startActPosInTaskEndY =

                    (endBounds.top - taskEndBounds.top) - hintRect.top * hintToEndScaleY;

            transact.setScale(sc, hintToEndScaleX, hintToEndScaleY);

            transact.setPosition(sc, startActPosInTaskEndX, startActPosInTaskEndY);

        }

    }



    @VisibleForTesting(visibility = VisibleForTesting.Visibility.PRIVATE)

    static boolean containsChangeFor(WindowContainer wc, ArrayList<ChangeInfo> list) {

        for (int i = list.size() - 1; i >= 0; --i) {
@@ -1799,6 +1833,7 @@ class Transition implements BLASTSyncEngine.TransactionReadyListener { 


        // Resolve the animating targets from the participants.

        mTargets = calculateTargets(mParticipants, mChanges);

        prepareConfigAtEnd(transaction, mTargets);



        // Check whether the participants were animated from back navigation.

        mController.mAtm.mBackNavigationController.onTransactionReady(this, mTargets,
@@ -2648,9 +2683,6 @@ class Transition implements BLASTSyncEngine.TransactionReadyListener { 
            } else {

                parentChange.mFlags |= ChangeInfo.FLAG_CHANGE_YES_ANIMATION;

            }

            if ((targetChange.mFlags & ChangeInfo.FLAG_CHANGE_CONFIG_AT_END) != 0) {

                parentChange.mFlags |= ChangeInfo.FLAG_CHANGE_CONFIG_AT_END;

            }

        }

    }
@@ -2742,14 +2774,6 @@ class Transition implements BLASTSyncEngine.TransactionReadyListener { 
                    } else {

                        intermediates.add(parentChange);

                    }

                    // for config-at-end, we want to promote the flag based on the end-state even

                    // if the activity was reparented because it operates after the animation. So,

                    // check that here since the promote code skips reparents.

                    if ((targetChange.mFlags & ChangeInfo.FLAG_CHANGE_CONFIG_AT_END) != 0

                            && targetChange.mContainer.asActivityRecord() != null

                            && targetChange.mContainer.getParent() == p) {

                        parentChange.mFlags |= ChangeInfo.FLAG_CHANGE_CONFIG_AT_END;

                    }

                    foundParentInTargets = true;

                    break;

                } else if (reportIfNotTop(p) && !skipIntermediateReports) {

services/tests/wmtests/src/com/android/server/wm/TransitionTests.java

+2 −8
Original line number Diff line number Diff line
@@ -34,7 +34,6 @@ import static android.view.WindowManager.TRANSIT_CHANGE; 
import static android.view.WindowManager.TRANSIT_CLOSE;

import static android.view.WindowManager.TRANSIT_OPEN;

import static android.view.WindowManager.TRANSIT_TO_BACK;

import static android.window.TransitionInfo.FLAG_CONFIG_AT_END;

import static android.window.TransitionInfo.FLAG_CROSS_PROFILE_OWNER_THUMBNAIL;

import static android.window.TransitionInfo.FLAG_FILLS_TASK;

import static android.window.TransitionInfo.FLAG_IN_TASK_WITH_EMBEDDED_ACTIVITY;
@@ -2935,9 +2934,6 @@ public class TransitionTests extends WindowTestsBase { 
        controller.requestStartTransition(transit, task, null, null);

        player.start();

        assertTrue(activity.isConfigurationDispatchPaused());

        // config-at-end flag must propagate up to task if activity was promoted.

        assertTrue(player.mLastReady.getChange(

                task.mRemoteToken.toWindowContainerToken()).hasFlags(FLAG_CONFIG_AT_END));

        player.finish();

        assertFalse(activity.isConfigurationDispatchPaused());

    }
@@ -2966,11 +2962,9 @@ public class TransitionTests extends WindowTestsBase { 


        controller.requestStartTransition(transit, task, null, null);

        player.start();

        // config-at-end flag must propagate up to task even when reparented (since config-at-end

        // only cares about after-end state).

        assertTrue(player.mLastReady.getChange(

                task.mRemoteToken.toWindowContainerToken()).hasFlags(FLAG_CONFIG_AT_END));

        assertTrue(activity.isConfigurationDispatchPaused());

        player.finish();

        assertFalse(activity.isConfigurationDispatchPaused());

    }



    @Test