[media] Fix mmap() example in the V4L2 API DocBook

Correct ioctl return value handling and fix coding style issues.

[mchehab@redhat.com: return -1 is OK, according with ioctl manpages. Reverting ioctl changes]
Signed-off-by: Pawel Osciak <pawel@osciak.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>