quota: Improve checking of quota file header

When we are asked for vfsv0 quota format and the file is in vfsv1
format (or vice versa), refuse to use the quota file. Also return
with error when we don't like the header of quota file.

Signed-off-by: Jan Kara <jack@suse.cz>