Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit b121186a authored by Alex Shi's avatar Alex Shi Committed by Linus Torvalds
Browse files

mm: correct page->pfmemalloc to fix deactivate_slab regression 



Commit cfd19c5a ("mm: only set page->pfmemalloc when
ALLOC_NO_WATERMARKS was used") tried to narrow down page->pfmemalloc
setting, but it missed some places the pfmemalloc should be set.

So, in __slab_alloc, the unalignment pfmemalloc and ALLOC_NO_WATERMARKS
cause incorrect deactivate_slab() on our core2 server:

    64.73%           fio  [kernel.kallsyms]     [k] _raw_spin_lock
                     |
                     --- _raw_spin_lock
                        |
                        |---0.34%-- deactivate_slab
                        |          __slab_alloc
                        |          kmem_cache_alloc
                        |          |

That causes our fio sync write performance to have a 40% regression.

Move the checking in get_page_from_freelist() which resolves this issue.

Signed-off-by: default avatarAlex Shi <alex.shi@intel.com>
Acked-by: default avatarMel Gorman <mgorman@suse.de>
Cc: David Miller <davem@davemloft.net
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Tested-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
Tested-by: default avatarSage Weil <sage@inktank.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent 5ed12f12

mm/page_alloc.c

+11 −10
Original line number Diff line number Diff line
@@ -1928,6 +1928,17 @@ get_page_from_freelist(gfp_t gfp_mask, nodemask_t *nodemask, unsigned int order, 
		zlc_active = 0;

		goto zonelist_scan;

	}



	if (page)

		/*

		 * page->pfmemalloc is set when ALLOC_NO_WATERMARKS was

		 * necessary to allocate the page. The expectation is

		 * that the caller is taking steps that will free more

		 * memory. The caller should avoid the page being used

		 * for !PFMEMALLOC purposes.

		 */

		page->pfmemalloc = !!(alloc_flags & ALLOC_NO_WATERMARKS);



	return page;

}
@@ -2389,14 +2400,6 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, 
				zonelist, high_zoneidx, nodemask,

				preferred_zone, migratetype);

		if (page) {

			/*

			 * page->pfmemalloc is set when ALLOC_NO_WATERMARKS was

			 * necessary to allocate the page. The expectation is

			 * that the caller is taking steps that will free more

			 * memory. The caller should avoid the page being used

			 * for !PFMEMALLOC purposes.

			 */

			page->pfmemalloc = true;

			goto got_pg;

		}

	}
@@ -2569,8 +2572,6 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, 
		page = __alloc_pages_slowpath(gfp_mask, order,

				zonelist, high_zoneidx, nodemask,

				preferred_zone, migratetype);

	else

		page->pfmemalloc = false;



	trace_mm_page_alloc(page, order, gfp_mask, migratetype);