ASoC: wm8994: Don't bother lowering clock dividers inside idle AIFs

This increases the chances we'll manage to hit a partially configured
state on restart and the power savings are extremely small.

Signed-off-by: Mark Brown <broonie@opensource.wolfsonmicro.com>