broadcom: Fix &&/|| confusion in bcm54xx_adjust_rxrefclk()

This always evaluates to true.

Signed-off-by: Roel Kluin <roel.kluin@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>