net: phy: sfp: Fix unregistering of HWMON SFP device

A HWMON device is only registered is the SFP module supports the
diagnostic page and is complient to SFF8472. Don't unconditionally
unregister the hwmon device when the SFP module is remove, otherwise
we access data structures which don't exist.

Reported-by: Florian Fainelli <f.fainelli@gmail.com>
Fixes: 1323061a ("net: phy: sfp: Add HWMON support for module sensors")
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
Tested-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>