gpio: adnp: Fix segfault if request_threaded_irq fails

In case request_threaded_irq inside adnp_irq_setup fails, the driver
segfaults. This is because irq_domain_remove is called twice with
the same pointer. First time in adnp_irq_setup and then a second time
after leaving adnp_irq_setup in the error path of adnp_i2c_probe
inside adnp_teardown.
This fixes this by removing the call to irq_domain_remove from
adnp_irq_setup.

Signed-off-by: Lars Poeschel <poeschel@lemonage.de>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>