Commit e08896ab authored Oct 17, 2018 by Kostya Kortchinsky

Use specific RSS check under 64-bit Scudo

Scudo is a hardened memory allocator that can be enabled on a per target basis,
and replaces malloc/free/etc with its own (go/scudo-allocator). It lives in
LLVM's compiler-rt project, along with ASan and the other sanitizers.

The 64-bit version of the allocator reserves (but doesn't commit) a large
portion of the address space, which is not compatible with the address space
limit enforced by RLIMIT_AS.

Scudo offers a function to enforce an upper RSS limit (note this is the actual
RSS of a process, not its AS), use it if running with 64-bit Scudo.

Bug: 63907455
Test: aosp compiled & run in emulator
Test: manually enabling Scudo for mediaextractor & mediacodec, flashing & using
      a marlin

Change-Id: Iaef6ff04077b22d16b0b62d413081ee8e6ba6ea0

parent 887cb9ea

media/libmedia/MediaUtils.cpp

+9 −0

Original line number	Diff line number	Diff line
		@@ -25,6 +25,7 @@
		#include "MediaUtils.h"

		extern "C" size_t __cfi_shadow_size();
		extern "C" void __scudo_set_rss_limit(size_t, int) __attribute__((weak));

		namespace android {

		@@ -65,6 +66,14 @@ void limitProcessMemory(
		maxMem = propVal;
		}

		// If 64-bit Scudo is in use, enforce the hard RSS limit (in MB).
		if (maxMem != SIZE_MAX && sizeof(void *) == 8 &&
		&__scudo_set_rss_limit != 0) {
		__scudo_set_rss_limit(maxMem >> 20, 1);
		ALOGV("Scudo hard RSS limit set to %zu MB", maxMem >> 20);
		return;
		}

		// Increase by the size of the CFI shadow mapping. Most of the shadow is not
		// backed with physical pages, and it is possible for the result to be
		// higher than total physical memory. This is fine for RLIMIT_AS.