Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit ca641514 authored by Andrea Arcangeli's avatar Andrea Arcangeli Committed by Linus Torvalds
Browse files

mm: thp: optimize compound_trans_huge 



Currently we don't clobber page_tail->first_page during split_huge_page,
so compound_trans_head can be set to compound_head without adverse
effects, and this mostly optimizes away a smp_rmb.

It looks worthwhile to keep around the implementation that doesn't relay
on page_tail->first_page not to be clobbered, because it would be
necessary if we'll decide to enforce page->private to zero at all times
whenever PG_private is not set, also for anonymous pages.  For anonymous
pages enforcing such an invariant doesn't matter as anonymous pages
don't use page->private so we can get away with this microoptimization.

Signed-off-by: default avatarAndrea Arcangeli <aarcange@redhat.com>
Cc: Khalid Aziz <khalid.aziz@oracle.com>
Cc: Pravin Shelar <pshelar@nicira.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Ben Hutchings <bhutchings@solarflare.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Johannes Weiner <jweiner@redhat.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Rik van Riel <riel@redhat.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Minchan Kim <minchan@kernel.org>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent ebf360f9

include/linux/huge_mm.h

+23 −0
Original line number Diff line number Diff line
@@ -157,6 +157,26 @@ static inline int hpage_nr_pages(struct page *page) 
		return HPAGE_PMD_NR;

	return 1;

}

/*

 * compound_trans_head() should be used instead of compound_head(),

 * whenever the "page" passed as parameter could be the tail of a

 * transparent hugepage that could be undergoing a

 * __split_huge_page_refcount(). The page structure layout often

 * changes across releases and it makes extensive use of unions. So if

 * the page structure layout will change in a way that

 * page->first_page gets clobbered by __split_huge_page_refcount, the

 * implementation making use of smp_rmb() will be required.

 *

 * Currently we define compound_trans_head as compound_head, because

 * page->private is in the same union with page->first_page, and

 * page->private isn't clobbered. However this also means we're

 * currently leaving dirt into the page->private field of anonymous

 * pages resulting from a THP split, instead of setting page->private

 * to zero like for every other page that has PG_private not set. But

 * anonymous pages don't use page->private so this is not a problem.

 */

#if 0

/* This will be needed if page->private will be clobbered in split_huge_page */

static inline struct page *compound_trans_head(struct page *page)

{

	if (PageTail(page)) {
@@ -174,6 +194,9 @@ static inline struct page *compound_trans_head(struct page *page) 
	}

	return page;

}

#else

#define compound_trans_head(page) compound_head(page)

#endif



extern int do_huge_pmd_numa_page(struct mm_struct *mm, struct vm_area_struct *vma,

				unsigned long addr, pmd_t pmd, pmd_t *pmdp);