fastboot: Cap max size sent to libsparse

This is required for large (>INT_MAX) sparse limit reported by
the target.

Also, patch up return chains of "int" that need to deal with sizes
bigger than 2GB as well as return negative error codes.

Test: -S works with large max-download-size
Test: Flash 3GB system.img with max-download-size 2.5GB

Bug: 36810152
Change-Id: I562a50eabd706bd5b97c71a1aef07c1ffd1a2e5c