soc: swr-mstr: skip port disable when master is down

skip port disable when master is already down to avoid
queuing commands to master which is already down due to ssr/pdr.

Change-Id: I7fadc479784feb0fd8afa229599eb217bb579ea2
Signed-off-by: Ramprasad Katkam <katkam@codeaurora.org>