Commit 8bbde7a7 authored Sep 21, 2009 by Anton Blanchard Committed by Benjamin Herrenschmidt Sep 24, 2009

powerpc: Move 64bit heap above 1TB on machines with 1TB segments



If we are using 1TB segments and we are allowed to randomise the heap, we can
put it above 1TB so it is backed by a 1TB segment. Otherwise the heap will be
in the bottom 1TB which always uses 256MB segments and this may result in a
performance penalty.

This functionality is disabled when heap randomisation is turned off:

echo 1 > /proc/sys/kernel/randomize_va_space

which may be useful when trying to allocate the maximum amount of 16M or 16G
pages.

On a microbenchmark that repeatedly touches 32GB of memory with a stride of
256MB + 4kB (designed to stress 256MB segments while still mapping nicely into
the L1 cache), we see the improvement:

Force malloc to use heap all the time:
# export MALLOC_MMAP_MAX_=0 MALLOC_TRIM_THRESHOLD_=-1

Disable heap randomization:
# echo 1 > /proc/sys/kernel/randomize_va_space
# time ./test
12.51s

Enable heap randomization:
# echo 2 > /proc/sys/kernel/randomize_va_space
# time ./test
1.70s

Signed-off-by: Anton Blanchard <anton@samba.org>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

parent 738ef42e

arch/powerpc/kernel/process.c

+16 −1

Original line number	Diff line number	Diff line
		@@ -1165,7 +1165,22 @@ static inline unsigned long brk_rnd(void)

		unsigned long arch_randomize_brk(struct mm_struct *mm)
		{
		unsigned long ret = PAGE_ALIGN(mm->brk + brk_rnd());
		unsigned long base = mm->brk;
		unsigned long ret;

		#ifdef CONFIG_PPC64
		/*
		* If we are using 1TB segments and we are allowed to randomise
		* the heap, we can put it above 1TB so it is backed by a 1TB
		* segment. Otherwise the heap will be in the bottom 1TB
		* which always uses 256MB segments and this may result in a
		* performance penalty.
		*/
		if (!is_32bit_task() && (mmu_highuser_ssize == MMU_SEGSIZE_1T))
		base = max_t(unsigned long, mm->brk, 1UL << SID_SHIFT_1T);
		#endif

		ret = PAGE_ALIGN(base + brk_rnd());

		if (ret < mm->brk)
		return mm->brk;