Commit 2916ecc0 authored Sep 08, 2017 by Jérôme Glisse Committed by Linus Torvalds Sep 08, 2017

mm/migrate: new migrate mode MIGRATE_SYNC_NO_COPY

Introduce a new migration mode that allow to offload the copy to a device
DMA engine.  This changes the workflow of migration and not all
address_space migratepage callback can support this.

This is intended to be use by migrate_vma() which itself is use for thing
like HMM (see include/linux/hmm.h).

No additional per-filesystem migratepage testing is needed.  I disables
MIGRATE_SYNC_NO_COPY in all problematic migratepage() callback and i
added comment in those to explain why (part of this patch).  The commit
message is unclear it should say that any callback that wish to support
this new mode need to be aware of the difference in the migration flow
from other mode.

Some of these callbacks do extra locking while copying (aio, zsmalloc,
balloon, ...) and for DMA to be effective you want to copy multiple
pages in one DMA operations.  But in the problematic case you can not
easily hold the extra lock accross multiple call to this callback.

Usual flow is:

For each page {
 1 - lock page
 2 - call migratepage() callback
 3 - (extra locking in some migratepage() callback)
 4 - migrate page state (freeze refcount, update page cache, buffer
     head, ...)
 5 - copy page
 6 - (unlock any extra lock of migratepage() callback)
 7 - return from migratepage() callback
 8 - unlock page
}

The new mode MIGRATE_SYNC_NO_COPY:
 1 - lock multiple pages
For each page {
 2 - call migratepage() callback
 3 - abort in all problematic migratepage() callback
 4 - migrate page state (freeze refcount, update page cache, buffer
     head, ...)
} // finished all calls to migratepage() callback
 5 - DMA copy multiple pages
 6 - unlock all the pages

To support MIGRATE_SYNC_NO_COPY in the problematic case we would need a
new callback migratepages() (for instance) that deals with multiple
pages in one transaction.

Because the problematic cases are not important for current usage I did
not wanted to complexify this patchset even more for no good reason.

Link: http://lkml.kernel.org/r/20170817000548.32038-14-jglisse@redhat.com


Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Cc: Aneesh Kumar <aneesh.kumar@linux.vnet.ibm.com>
Cc: Balbir Singh <bsingharora@gmail.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: David Nellans <dnellans@nvidia.com>
Cc: Evgeny Baskakov <ebaskakov@nvidia.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Mark Hairgrove <mhairgrove@nvidia.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Sherry Cheung <SCheung@nvidia.com>
Cc: Subhash Gutti <sgutti@nvidia.com>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Cc: Bob Liu <liubo95@huawei.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

parent 858b54da

fs/aio.c

+8 −0

Original line number	Diff line number	Diff line
		@@ -373,6 +373,14 @@ static int aio_migratepage(struct address_space mapping, struct page new,
		pgoff_t idx;
		int rc;

		/*
		* We cannot support the _NO_COPY case here, because copy needs to
		* happen under the ctx->completion_lock. That does not work with the
		* migration workflow of MIGRATE_SYNC_NO_COPY.
		*/
		if (mode == MIGRATE_SYNC_NO_COPY)
		return -EINVAL;

		rc = 0;

		/* mapping->private_lock here protects against the kioctx teardown. */

fs/f2fs/data.c

+4 −1

Original line number	Diff line number	Diff line
		@@ -2253,7 +2253,10 @@ int f2fs_migrate_page(struct address_space *mapping,
		SetPagePrivate(newpage);
		set_page_private(newpage, page_private(page));

		if (mode != MIGRATE_SYNC_NO_COPY)
		migrate_page_copy(newpage, page);
		else
		migrate_page_states(newpage, page);

		return MIGRATEPAGE_SUCCESS;
		}

fs/hugetlbfs/inode.c

+4 −1

Original line number	Diff line number	Diff line
		@@ -830,7 +830,10 @@ static int hugetlbfs_migrate_page(struct address_space *mapping,
		rc = migrate_huge_page_move_mapping(mapping, newpage, page);
		if (rc != MIGRATEPAGE_SUCCESS)
		return rc;
		if (mode != MIGRATE_SYNC_NO_COPY)
		migrate_page_copy(newpage, page);
		else
		migrate_page_states(newpage, page);

		return MIGRATEPAGE_SUCCESS;
		}

fs/ubifs/file.c

+4 −1

Original line number	Diff line number	Diff line
		@@ -1490,7 +1490,10 @@ static int ubifs_migrate_page(struct address_space *mapping,
		SetPagePrivate(newpage);
		}

		if (mode != MIGRATE_SYNC_NO_COPY)
		migrate_page_copy(newpage, page);
		else
		migrate_page_states(newpage, page);
		return MIGRATEPAGE_SUCCESS;
		}
		#endif

include/linux/migrate.h

+5 −0

Original line number	Diff line number	Diff line
		@@ -72,6 +72,7 @@ extern void putback_movable_page(struct page *page);

		extern int migrate_prep(void);
		extern int migrate_prep_local(void);
		extern void migrate_page_states(struct page newpage, struct page page);
		extern void migrate_page_copy(struct page newpage, struct page page);
		extern int migrate_huge_page_move_mapping(struct address_space *mapping,
		struct page newpage, struct page page);
		@@ -92,6 +93,10 @@ static inline int isolate_movable_page(struct page *page, isolate_mode_t mode)
		static inline int migrate_prep(void) { return -ENOSYS; }
		static inline int migrate_prep_local(void) { return -ENOSYS; }

		static inline void migrate_page_states(struct page newpage, struct page page)
		{
		}

		static inline void migrate_page_copy(struct page *newpage,
		struct page *page) {}