Run some actions in a separate thread

Some actions need to be threaded so we will run those in a
separate thread and deny requests to thread more actions if new
requests come in while a thread is already running.

Change-Id: I966c538e67860a6d8fe556e5a2eb7f7d1a987e74
Signed-off-by: Vojtech Bocek <vbocek@gmail.com>