Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 763b218d authored by Joel Fernandes's avatar Joel Fernandes Committed by Linus Torvalds
Browse files

mm: add preempt points into __purge_vmap_area_lazy() 

Use cond_resched_lock to avoid holding the vmap_area_lock for a
potentially long time and thus creating bad latencies for various
workloads.

[hch: split from a larger patch by Joel, wrote the crappy changelog]
Link: http://lkml.kernel.org/r/1479474236-4139-11-git-send-email-hch@lst.de


Signed-off-by: default avatarJoel Fernandes <joelaf@google.com>
Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
Tested-by: default avatarJisheng Zhang <jszhang@marvell.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: John Dias <joaodias@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent f9e09977

mm/vmalloc.c

+9 −5
Original line number Diff line number Diff line
@@ -628,7 +628,7 @@ static bool __purge_vmap_area_lazy(unsigned long start, unsigned long end) 
	struct llist_node *valist;

	struct vmap_area *va;

	struct vmap_area *n_va;

	int nr = 0;

	bool do_free = false;



	lockdep_assert_held(&vmap_purge_lock);
@@ -638,18 +638,22 @@ static bool __purge_vmap_area_lazy(unsigned long start, unsigned long end) 
			start = va->va_start;

		if (va->va_end > end)

			end = va->va_end;

		nr += (va->va_end - va->va_start) >> PAGE_SHIFT;

		do_free = true;

	}



	if (!nr)

	if (!do_free)

		return false;



	atomic_sub(nr, &vmap_lazy_nr);

	flush_tlb_kernel_range(start, end);



	spin_lock(&vmap_area_lock);

	llist_for_each_entry_safe(va, n_va, valist, purge_list)

	llist_for_each_entry_safe(va, n_va, valist, purge_list) {

		int nr = (va->va_end - va->va_start) >> PAGE_SHIFT;



		__free_vmap_area(va);

		atomic_sub(nr, &vmap_lazy_nr);

		cond_resched_lock(&vmap_area_lock);

	}

	spin_unlock(&vmap_area_lock);

	return true;

}