Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 96e65306 authored by Lai Jiangshan's avatar Lai Jiangshan Committed by Tejun Heo
Browse files

workqueue: UNBOUND -> REBIND morphing in rebind_workers() should be atomic 



The compiler may compile the following code into TWO write/modify
instructions.

	worker->flags &= ~WORKER_UNBOUND;
	worker->flags |= WORKER_REBIND;

so the other CPU may temporarily see worker->flags which doesn't have
either WORKER_UNBOUND or WORKER_REBIND set and perform local wakeup
prematurely.

Fix it by using single explicit assignment via ACCESS_ONCE().

Because idle workers have another WORKER_NOT_RUNNING flag, this bug
doesn't exist for them; however, update it to use the same pattern for
consistency.

tj: Applied the change to idle workers too and updated comments and
    patch description a bit.

Signed-off-by: default avatarLai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: default avatarTejun Heo <tj@kernel.org>
Cc: stable@vger.kernel.org
parent 0d7614f0

kernel/workqueue.c

+11 −6
Original line number Diff line number Diff line
@@ -1396,12 +1396,15 @@ retry: 
	/* set REBIND and kick idle ones, we'll wait for these later */

	for_each_worker_pool(pool, gcwq) {

		list_for_each_entry(worker, &pool->idle_list, entry) {

			unsigned long worker_flags = worker->flags;



			if (worker->flags & WORKER_REBIND)

				continue;



			/* morph UNBOUND to REBIND */

			worker->flags &= ~WORKER_UNBOUND;

			worker->flags |= WORKER_REBIND;

			/* morph UNBOUND to REBIND atomically */

			worker_flags &= ~WORKER_UNBOUND;

			worker_flags |= WORKER_REBIND;

			ACCESS_ONCE(worker->flags) = worker_flags;



			idle_rebind.cnt++;

			worker->idle_rebind = &idle_rebind;
@@ -1434,10 +1437,12 @@ retry: 
	/* rebind busy workers */

	for_each_busy_worker(worker, i, pos, gcwq) {

		struct work_struct *rebind_work = &worker->rebind_work;

		unsigned long worker_flags = worker->flags;



		/* morph UNBOUND to REBIND */

		worker->flags &= ~WORKER_UNBOUND;

		worker->flags |= WORKER_REBIND;

		/* morph UNBOUND to REBIND atomically */

		worker_flags &= ~WORKER_UNBOUND;

		worker_flags |= WORKER_REBIND;

		ACCESS_ONCE(worker->flags) = worker_flags;



		if (test_and_set_bit(WORK_STRUCT_PENDING_BIT,

				     work_data_bits(rebind_work)))