avcenc: Switch malloc/free callbacks to use pointers instead of ints

There is no reason for casting the pointers to ints. This fixes
building the code on platforms where pointers are larger than ints,
e.g. 64 bit platforms.

Change-Id: I910cd207d0908287931c9a96eb270139967e029b