handle size_t > java max int size

Cleanly abort if we would have returned a value which can't be safely
handled by the java APIs. I'm not sure this code is reachable, but
adding the check just in case.

Bug: 16676699

(cherry picked from commit 3f6b702b)

Change-Id: Iddc16f32cb5d46219a4dcb3548bcfeaade0f9c9e