[ARM] 4481/1: Fix a bug when i.MX is compiled as a module

Fix the unregistration order in the i.MX serial driver

Signed-off-by: Sreekrishnan Venkateswaran <krishhna@gmail.com>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>