V4L/DVB (7904): v4l/tuner-core: consistent handling of return values

change check_mode and set_mode to return negative errors and fix all callers

Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@infradead.org>