libutils: Add more tests for Unicode
This CL added additional tests for converting between utf16 and utf8, specifically tests that: 1. check utf16_to_utf8_length() returns 0 if input is an empty UTF16 string 2. check utf16_to_utf8_length() returns 1 if input is a single ASCII character UTF16 string 3. check utf16_to_utf8_length() returns 3 if input is a single UTF-16 character between U+0800 - U+FFFF 4. check utf16_to_utf8_length() returns 4 if input has a surrogate pair 5. check unpaired UTF-16 surrogate is handled correctly (skipped) 6. check utf16_to_utf8_length(0 handles invalid surrogate case correctly, by skipping the first but handling the rest correctly 7. check a normal string with a mix of 1/2/3/4-byte UTF8 characters is correctly converted by utf16_to_utf8() 8. check conversion from invalid utf8 sequence with invalid leading byte and/or invalid trailing byte(s) should still work and not crash Change-Id: If68e514af0e84ddebf5900b2e140e76ba4f44553
Loading
Please register or sign in to comment