clk: gemini: hands off PCI OE bit

This bit is pin control, and needs to be carefully managed by the
new pin control driver.

Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>