ASoC: uniphier: fix broken sound if use SRC in replay

This patch fixes settings for ports with SRC. These ports need to
set the fixed audio rate and clock. If not, the SRC outputs broken
sound.

Signed-off-by: Katsuhiro Suzuki <suzuki.katsuhiro@socionext.com>
Signed-off-by: Mark Brown <broonie@kernel.org>