Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 69d44b0b authored by Hans Boehm's avatar Hans Boehm
Browse files

Add zygote native fork loop

Do not return to Java mode between consecutive fork operations.
This greatly reduces the Zygote overhead, since we no longer need to
stop and restart Java daemons.

By not switching back to Java mode, and being careful about what memory
we touch between forks, we also keep the Zygote heaps much more stable,
facilitating page sharing between the zygote and all its children.
Under normal operation we should no longer allocate any memory in the
zygote between forks. That applies to both the Java and C++ heap.
This makes the zygote behave much more like the mental model many
of us had assumed: It has nearly constant memory contents, which are
copy-on-right cloned at each fork. This does not apply to the initial
system server and webzygote forks, that are currently still handled
differently.

This includes

1. Add ZygoteCommandBuffer, and switch the argument parsing code to use it.
This slightly reduces allocation and enables (3).

2. Support process specialization in the child, even when the arguments
are already know, Leverages existing Usap code.

3. Add support for forking multiple child processes directly to the
ZygoteCommandBuffer data structure. This directly uses the buffer
internals, and avoids returning to Java so long as it can handle the
zygote commands it sees.

FUNCTIONALITY CHANGE:

We now limit the total size of the zygote command, rather than the
number of arguments.

Initial performance observations:

[ These are not perfect, since I'm comparing to numbers before I
started. There may have been other moving parts, but they should be
minor. ]

System-server-observed launch latency:

[Not the best metric, but easy to measure. In particular, this
does not represent a significant reduction in application launch
time.]

Based on measuring the last 10 launches in a lightly used cf AOSP
instance, the system server latency from requesting an app launch to
response with the pid (which does not require the child to execute
anything) went from an average of about 10.7(25) msecs to 6.8(9) and
7.9(16) in two tries with the CL. (The parenthetical numbers are
maxima from among the 10; the variance appears to have decreased
appreciably.)

Dirty pages:

The number of private dirty pages in the zygote itself appears to have
decreased from about 4000 to about 2200. The number of dalvik-main
private dirty pages went from about 1500 to nearly zero.

Initially ART benchmarking service claim -1.88% in PSS. But this is not
consistently repeatable.

Drive-by fix:

Call setAllowNetworkingForProcess on usap / native loop path.

Bug: 159631815
Bug: 174211442
Test: Boots AOSP
Change-Id: I90d2e381bada1b6c9857666d5e87372b6a4c1a70
parent 30d9bfa0
Loading
Loading
Loading
Loading
+1 −1
Original line number Diff line number Diff line
@@ -426,7 +426,7 @@ public class ZygoteProcess {
        // avoid writing a partial response to the zygote.
        for (String arg : args) {
            // Making two indexOf calls here is faster than running a manually fused loop due
            // to the fact that indexOf is a optimized intrinsic.
            // to the fact that indexOf is an optimized intrinsic.
            if (arg.indexOf('\n') >= 0) {
                throw new ZygoteStartFailedEx("Embedded newlines not allowed");
            } else if (arg.indexOf('\r') >= 0) {
+194 −166
Original line number Diff line number Diff line
@@ -18,8 +18,8 @@ package com.android.internal.os;

import static android.system.OsConstants.O_CLOEXEC;

import static com.android.internal.os.ZygoteConnectionConstants.MAX_ZYGOTE_ARGC;

import android.annotation.NonNull;
import android.annotation.Nullable;
import android.content.pm.ApplicationInfo;
import android.net.Credentials;
import android.net.LocalServerSocket;
@@ -36,17 +36,16 @@ import android.util.Log;

import com.android.internal.net.NetworkUtilsInternal;

import dalvik.annotation.optimization.CriticalNative;
import dalvik.annotation.optimization.FastNative;
import dalvik.system.ZygoteHooks;

import libcore.io.IoUtils;

import java.io.BufferedReader;
import java.io.ByteArrayOutputStream;
import java.io.DataOutputStream;
import java.io.FileDescriptor;
import java.io.IOException;
import java.io.InputStreamReader;

/** @hide */
public final class Zygote {
@@ -236,6 +235,8 @@ public final class Zygote {
     */
    public static final String CHILD_ZYGOTE_UID_RANGE_END = "--uid-range-end=";

    private static final String TAG = "Zygote";

    /** Prefix prepended to socket names created by init */
    private static final String ANDROID_SOCKET_PREFIX = "ANDROID_SOCKET_";

@@ -406,6 +407,10 @@ public final class Zygote {
        // Note that this event ends at the end of handleChildProc.
        Trace.traceBegin(Trace.TRACE_TAG_ACTIVITY_MANAGER, "PostFork");

        if (gids != null && gids.length > 0) {
            NetworkUtilsInternal.setAllowNetworkingForProcess(containsInetGid(gids));
        }

        // Set the Java Language thread priority to the default value for new apps.
        Thread.currentThread().setPriority(Thread.NORM_PRIORITY);

@@ -578,114 +583,163 @@ public final class Zygote {
    private static native int nativeGetUsapPoolEventFD();

    /**
     * Fork a new unspecialized app process from the zygote
     * Fork a new unspecialized app process from the zygote. Adds the Usap to the native
     * Usap table.
     *
     * @param usapPoolSocket  The server socket the USAP will call accept on
     * @param sessionSocketRawFDs  Anonymous session sockets that are currently open
     * @param isPriorityFork  Value controlling the process priority level until accept is called
     * @return In the Zygote process this function will always return null; in unspecialized app
     *         processes this function will return a Runnable object representing the new
     *         application that is passed up from usapMain.
     * @param sessionSocketRawFDs  Anonymous session sockets that are currently open.
     *         These are closed in the child.
     * @param isPriorityFork Raise the initial process priority level because this is on the
     *         critical path for application startup.
     * @return In the child process, this returns a Runnable that waits for specialization
     *         info to start an app process. In the sygote/parent process this returns null.
     */
    static Runnable forkUsap(LocalServerSocket usapPoolSocket,
    static @Nullable Runnable forkUsap(LocalServerSocket usapPoolSocket,
                                       int[] sessionSocketRawFDs,
                                       boolean isPriorityFork) {
        FileDescriptor[] pipeFDs;
        FileDescriptor readFD;
        FileDescriptor writeFD;

        try {
            pipeFDs = Os.pipe2(O_CLOEXEC);
            FileDescriptor[] pipeFDs = Os.pipe2(O_CLOEXEC);
            readFD = pipeFDs[0];
            writeFD = pipeFDs[1];
        } catch (ErrnoException errnoEx) {
            throw new IllegalStateException("Unable to create USAP pipe.", errnoEx);
        }

        int pid =
                nativeForkUsap(pipeFDs[0].getInt$(), pipeFDs[1].getInt$(),
                               sessionSocketRawFDs, isPriorityFork);

        int pid = nativeForkApp(readFD.getInt$(), writeFD.getInt$(),
                                sessionSocketRawFDs, /*argsKnown=*/ false, isPriorityFork);
        if (pid == 0) {
            IoUtils.closeQuietly(pipeFDs[0]);
            return usapMain(usapPoolSocket, pipeFDs[1]);
            IoUtils.closeQuietly(readFD);
            return childMain(null, usapPoolSocket, writeFD);
        } else if (pid == -1) {
            // Fork failed.
            return null;
        } else {
            // The read-end of the pipe will be closed by the native code.
            // See removeUsapTableEntry();
            IoUtils.closeQuietly(pipeFDs[1]);
            // readFD will be closed by the native code. See removeUsapTableEntry();
            IoUtils.closeQuietly(writeFD);
            nativeAddUsapTableEntry(pid, readFD.getInt$());
            return null;
        }
    }

    private static native int nativeForkUsap(int readPipeFD,
    private static native int nativeForkApp(int readPipeFD,
                                            int writePipeFD,
                                            int[] sessionSocketRawFDs,
                                            boolean argsKnown,
                                            boolean isPriorityFork);

    /**
     * This function is used by unspecialized app processes to wait for specialization requests from
     * the system server.
     * Add an entry for a new Usap to the table maintained in native code.
     */
    @CriticalNative
    private static native void nativeAddUsapTableEntry(int pid, int readPipeFD);

    /**
     * Fork a new app process from the zygote. argBuffer contains a fork command that
     * request neither a child zygote, nor a wrapped process. Continue to accept connections
     * on the specified socket, use those to refill argBuffer, and continue to process
     * sufficiently simple fork requests. We presume that the only open file descriptors
     * requiring special treatment are the session socket embedded in argBuffer, and
     * zygoteSocket.
     * @param argBuffer containing initial command and the connected socket from which to
     *         read more
     * @param zygoteSocket socket from which to obtain new connections when current argBuffer
     *         one is disconnected
     * @param expectedUId Uid of peer for initial requests. Subsequent requests from a different
     *               peer will cause us to return rather than perform the requested fork.
     * @param minUid Minimum Uid enforced for all but first fork request. The caller checks
     *               the Uid policy for the initial request.
     * @param firstNiceName name of first created process. Used for error reporting only.
     * @return A Runnable in each child process, null in the parent.
     * If this returns in then argBuffer still contains a command needing to be executed.
     */
    static @Nullable Runnable forkSimpleApps(@NonNull ZygoteCommandBuffer argBuffer,
                                             @NonNull FileDescriptor zygoteSocket,
                                             int expectedUid,
                                             int minUid,
                                             @Nullable String firstNiceName) {
        boolean in_child =
                argBuffer.forkRepeatedly(zygoteSocket, expectedUid, minUid, firstNiceName);
        if (in_child) {
            return childMain(argBuffer, /*usapPoolSocket=*/null, /*writePipe=*/null);
        } else {
            return null;
        }
    }

    /**
     * Specialize the current process into one described by argBuffer or the command read from
     * usapPoolSocket. Exactly one of those must be null. If we are given an argBuffer, we close
     * it. Used both for a specializing a USAP process, and for process creation without USAPs.
     * In both cases, we specialize the process after first returning to Java code.
     *
     * @param writePipe  The write end of the reporting pipe used to communicate with the poll loop
     *                   of the ZygoteServer.
     * @return A runnable oject representing the new application.
     */
    private static Runnable usapMain(LocalServerSocket usapPoolSocket,
    private static Runnable childMain(@Nullable ZygoteCommandBuffer argBuffer,
                                      @Nullable LocalServerSocket usapPoolSocket,
                                      FileDescriptor writePipe) {
        final int pid = Process.myPid();
        Process.setArgV0(Process.is64Bit() ? "usap64" : "usap32");

        LocalSocket sessionSocket = null;
        DataOutputStream usapOutputStream = null;
        Credentials peerCredentials = null;
        ZygoteArguments args = null;

        // Change the priority to max before calling accept so we can respond to new specialization
        // requests as quickly as possible.  This will be reverted to the default priority in the
        // native specialization code.
        // Block SIGTERM so we won't be killed if the Zygote flushes the USAP pool.
        blockSigTerm();

        LocalSocket sessionSocket = null;
        if (argBuffer == null) {
            // Read arguments from usapPoolSocket instead.

            Process.setArgV0(Process.is64Bit() ? "usap64" : "usap32");

            // Change the priority to max before calling accept so we can respond to new
            // specialization requests as quickly as possible.  This will be reverted to the
            // default priority in the native specialization code.
            boostUsapPriority();

            while (true) {
                ZygoteCommandBuffer tmpArgBuffer = null;
                try {
                    sessionSocket = usapPoolSocket.accept();

                // Block SIGTERM so we won't be killed if the Zygote flushes the USAP pool.
                blockSigTerm();

                BufferedReader usapReader =
                        new BufferedReader(new InputStreamReader(sessionSocket.getInputStream()));
                    usapOutputStream =
                            new DataOutputStream(sessionSocket.getOutputStream());

                peerCredentials = sessionSocket.getPeerCredentials();

                String[] argStrings = readArgumentList(usapReader);

                if (argStrings != null) {
                    args = new ZygoteArguments(argStrings);

                    Credentials peerCredentials = sessionSocket.getPeerCredentials();
                    tmpArgBuffer = new ZygoteCommandBuffer(sessionSocket);
                    args = ZygoteArguments.getInstance(argBuffer);
                    applyUidSecurityPolicy(args, peerCredentials);
                    // TODO (chriswailes): Should this only be run for debug builds?
                    validateUsapCommand(args);
                    break;
                } else {
                    Log.e("USAP", "Truncated command received.");
                    IoUtils.closeQuietly(sessionSocket);

                    // Re-enable SIGTERM so the USAP can be flushed from the pool if necessary.
                    unblockSigTerm();
                }
                } catch (Exception ex) {
                    Log.e("USAP", ex.getMessage());
                IoUtils.closeQuietly(sessionSocket);

                }
                // Re-enable SIGTERM so the USAP can be flushed from the pool if necessary.
                unblockSigTerm();
                IoUtils.closeQuietly(sessionSocket);
                IoUtils.closeQuietly(tmpArgBuffer);
                blockSigTerm();
            }
        } else {
            try {
                args = ZygoteArguments.getInstance(argBuffer);
            } catch (Exception ex) {
                Log.e("AppStartup", ex.getMessage());
                throw new AssertionError("Failed to parse application start command", ex);
            }
            // peerCredentials were checked in parent.
        }
        if (args == null) {
            throw new AssertionError("Empty command line");
        }

        try {
            // SIGTERM is blocked on loop exit.  This prevents a USAP that is specializing from
            // being killed during a pool flush.
            // SIGTERM is blocked here.  This prevents a USAP that is specializing from being
            // killed during a pool flush.

            setAppProcessName(args, "USAP");

            applyUidSecurityPolicy(args, peerCredentials);
            applyDebuggerSystemProperty(args);

            int[][] rlimits = null;
@@ -694,6 +748,7 @@ public final class Zygote {
                rlimits = args.mRLimits.toArray(INT_ARRAY_2D);
            }

            if (argBuffer == null) {
                // This must happen before the SELinux policy for this process is
                // changed when specializing.
                try {
@@ -705,34 +760,36 @@ public final class Zygote {
                            + ioEx.getMessage());
                    throw new RuntimeException(ioEx);
                } finally {
                IoUtils.closeQuietly(sessionSocket);

                    try {
                    // This socket is closed using Os.close due to an issue with the implementation
                    // of LocalSocketImp.close().  Because the raw FD is created by init and then
                    // loaded from an environment variable (as opposed to being created by the
                    // LocalSocketImpl itself) the current implementation will not actually close
                    // the underlying FD.
                    //
                    // See b/130309968 for discussion of this issue.
                    Os.close(usapPoolSocket.getFileDescriptor());
                } catch (ErrnoException ex) {
                        // Since the raw FD is created by init and then loaded from an environment
                        // variable (as opposed to being created by the LocalSocketImpl itself),
                        // the LocalSocket/LocalSocketImpl does not own the Os-level socket. See
                        // the spec for LocalSocket.createConnectedLocalSocket(FileDescriptor fd).
                        // Thus closing the LocalSocket does not suffice. See b/130309968 for more
                        // discussion.
                        FileDescriptor fd = usapPoolSocket.getFileDescriptor();
                        usapPoolSocket.close();
                        Os.close(fd);
                    } catch (ErrnoException | IOException ex) {
                        Log.e("USAP", "Failed to close USAP pool socket");
                        throw new RuntimeException(ex);
                    }
                }
            }

            if (writePipe != null) {
                try {
                    ByteArrayOutputStream buffer =
                            new ByteArrayOutputStream(Zygote.USAP_MANAGEMENT_MESSAGE_BYTES);
                    DataOutputStream outputStream = new DataOutputStream(buffer);

                // This is written as a long so that the USAP reporting pipe and USAP pool event FD
                // handlers in ZygoteServer.runSelectLoop can be unified.  These two cases should
                // both send/receive 8 bytes.
                    // This is written as a long so that the USAP reporting pipe and USAP pool
                    // event FD handlers in ZygoteServer.runSelectLoop can be unified.  These two
                    // cases should both send/receive 8 bytes.
                    // TODO: Needs tweaking to handle the non-Usap invoke-with case, which expects
                    // a different format.
                    outputStream.writeLong(pid);
                    outputStream.flush();

                    Os.write(writePipe, buffer.toByteArray(), 0, buffer.size());
                } catch (Exception ex) {
                    Log.e("USAP",
@@ -742,6 +799,7 @@ public final class Zygote {
                } finally {
                    IoUtils.closeQuietly(writePipe);
                }
            }

            specializeAppProcess(args.mUid, args.mGid, args.mGids,
                                 args.mRuntimeFlags, rlimits, args.mMountExternal,
@@ -847,13 +905,29 @@ public final class Zygote {
        return nativeRemoveUsapTableEntry(usapPID);
    }

    @CriticalNative
    private static native boolean nativeRemoveUsapTableEntry(int usapPID);

    /**
     * uid 1000 (Process.SYSTEM_UID) may specify any uid > 1000 in normal
     * Return the minimum child uid that the given peer is allowed to create.
     * uid 1000 (Process.SYSTEM_UID) may specify any uid ≥ 1000 in normal
     * operation. It may also specify any gid and setgroups() list it chooses.
     * In factory test mode, it may specify any UID.
     *
     */
    static int minChildUid(Credentials peer) {
        if (peer.getUid() == Process.SYSTEM_UID
                && FactoryTest.getMode() == FactoryTest.FACTORY_TEST_OFF) {
            /* In normal operation, SYSTEM_UID can only specify a restricted
             * set of UIDs. In factory test mode, SYSTEM_UID may specify any uid.
             */
            return Process.SYSTEM_UID;
        } else {
            return 0;
        }
    }

    /*
     * Adjust uid and gid arguments, ensuring that the security policy is satisfied.
     * @param args non-null; zygote spawner arguments
     * @param peer non-null; peer credentials
     * @throws ZygoteSecurityException Indicates a security issue when applying the UID based
@@ -862,18 +936,11 @@ public final class Zygote {
    static void applyUidSecurityPolicy(ZygoteArguments args, Credentials peer)
            throws ZygoteSecurityException {

        if (peer.getUid() == Process.SYSTEM_UID) {
            /* In normal operation, SYSTEM_UID can only specify a restricted
             * set of UIDs. In factory test mode, SYSTEM_UID may specify any uid.
             */
            boolean uidRestricted = FactoryTest.getMode() == FactoryTest.FACTORY_TEST_OFF;

            if (uidRestricted && args.mUidSpecified && (args.mUid < Process.SYSTEM_UID)) {
        if (args.mUidSpecified && (args.mUid < minChildUid(peer))) {
            throw new ZygoteSecurityException(
                    "System UID may not launch process with UID < "
                    + Process.SYSTEM_UID);
        }
        }

        // If not otherwise specified, uid and gid are inherited from peer
        if (!args.mUidSpecified) {
@@ -957,45 +1024,6 @@ public final class Zygote {
        }
    }

    /**
     * Reads an argument list from the provided socket
     * @return Argument list or null if EOF is reached
     * @throws IOException passed straight through
     */
    static String[] readArgumentList(BufferedReader socketReader) throws IOException {
        int argc;

        try {
            String argc_string = socketReader.readLine();

            if (argc_string == null) {
                // EOF reached.
                return null;
            }
            argc = Integer.parseInt(argc_string);

        } catch (NumberFormatException ex) {
            Log.e("Zygote", "Invalid Zygote wire format: non-int at argc");
            throw new IOException("Invalid wire format");
        }

        // See bug 1092107: large argc can be used for a DOS attack
        if (argc > MAX_ZYGOTE_ARGC) {
            throw new IOException("Max arg count exceeded");
        }

        String[] args = new String[argc];
        for (int arg_index = 0; arg_index < argc; arg_index++) {
            args[arg_index] = socketReader.readLine();
            if (args[arg_index] == null) {
                // We got an unexpected EOF.
                throw new IOException("Truncated request");
            }
        }

        return args;
    }

    /**
     * Creates a managed LocalServerSocket object using a file descriptor
     * created by an init.rc script.  The init scripts that specify the
+57 −26

File changed.

Preview size limit exceeded, changes collapsed.

+187 −0

File added.

Preview size limit exceeded, changes collapsed.

+181 −148

File changed.

Preview size limit exceeded, changes collapsed.

Loading