V4L/DVB (8682): V4L: fix return value of register video func

If a wrong device type is used with video_register_device_index() it should
better return an error number, instead of a constant.

Signed-off-by: Henrik Kretzschmar <henne@nachtwindheim.de>
Signed-off-by: Mauro Carvalho Chehab <mchehab@infradead.org>