rpmsg: glink: spi: Cancel work threads during remove

Ensure all queued work is finished before cleaning up and freeing the
channel structures. This prevents any use after frees that can happen
when an event occurs and the wdsp crashes at the same time.

Change-Id: I6e95dae835492e4bc854fe5fb110badba79dc1ad
Signed-off-by: Chris Lew <clew@codeaurora.org>