Monthly Archives: December 2014

Java: memory representation of primitive data types

So I was working on a GIF decoder, and it required some very rapid manipulations with binary data. Converting byte to short, bit shifts, that sort of thing. Obviously, that’s easily done in C. You just cast a pointer to something else, cause GPF, and are done for the day. Java is very big brothery about this stuff. It really doesn’t want you to do any C tricks. So if you need to convert byte[] to int[] – would you please iterate it and do a[i] = b[i] for every one of the million elements. There are of course reasons for that, but it really limits options.

Anyway, one of the issues I ran into was a construction like this:

int[] data = new int[256];
byte index = readIndex();
int entry = data[index];

You’re probably going “ha-ha” now, but shut up, I didn’t know that. So, Java, being a great language, only has signed types. Incidentally, byte is also signed. Therefore, 0xFF as byte is not 255, it’s -1. And this code should have been:

int[] data = new int[256];
int index = ((int)readIndex()) & 0xFF;
int entry = data[index];

So why the & 0xFF? It’s an extra operation. Turns out, here’s how type casts really work.

int n
00000000000000000000000100000001 257
11111111111111111111111111111100 -4
11111111111111111111110000000000 -1024

byte b = (byte)n
                        00000001 1
                        11111100 -4
                        00000000 0
                        
int nb = (int)b                
00000000000000000000000000000001 1
11111111111111111111111111111100 -4
00000000000000000000000000000000 0

char c = (char)n                        
                0000000100000001
                1111111111111100
                1111110000000000
                
int nc = (int)c
00000000000000000000000100000001 257
00000000000000001111111111111100 65532
00000000000000001111110000000000 64512

As you can see, when casting bigger type to smaller (int to byte), it just cuts everything that doesn’t fit, so most significant byte becomes the minus sign. When casting byte to int, it stretches the minus bit to all the added digits. So from this numeric perspective, it’s still the same number. But the entire problem is, in the example above, we want to speak binary, not numeric. And from binary perspective, it’s screwed up beyond recognition (binary being unaware of minus signs). So you have to re-cut the initial part (mask on 0xFF) to make it the same. It sort of works as expected with unsigned types, without this mental gymnastics.

Char is an interesting exception, it’s the one unsigned type. So it’s just always padded with zeroes.

This whole system is not without its benefits I guess, but why did they make the byte signed? I mean come on.

Advertisements

Android: displaying animated GIFs

As it is known, Android does not have a good support for animated GIFs. It’s a whole story with GIF compression patent owners being greedy and destroying this format for everyone, including themselves. In any case, we need to support it somehow.

Great list of available options is given by Johannes Borchardt:

  1. Movie class. I don’t like this approach. It’s some kind of sketchy undocumented class, which does not always work.
  2. GifDecoder + AnimationDrawable. GifDecoder is a custom class which converts gif frames to bitmaps, and bitmaps are then loaded into AnimationDrawable. Problem with it is that it attempts to decode the entire file at once (that’s how AnimationDrawable works anyway). Which is not necessarily fun when you have 5 Mb gif videos.
  3. WebView. We used it a lot. WebView has this thing when it usually works, but never ideally. Once you started dealing with it, you’re signing up to handle tons of its stupid quirks, most of which only reproduce on old obscure devices you don’t have. Plus, you can’t use multiple instances in any kind of list views, because it does things to your memory. So your UI options are limited.

All in all, WebView is a simplest approach, which works for most applications. However, I wanted to look into some more possibilities. GifDecoder looked promising, because it was very controllable. It worked fine for smaller animations. Obviously the entire concept was wrong for gif videos, because there’s no way they’ll ever fit anywhere uncompressed, even on disk. So I rewritten this class for streaming and created a GifFileDecoder project here: https://sourceforge.net/projects/giffiledecoder/ The project includes two parts:

  1. Decoder. Same as GifDecoder, it converts gif frames to bitmaps. Unlike GifDecoder, it only needs to keep one or two frames in memory, so it can play any kind of video (I’ve successfully tested it on 15 Mb 640×640 file). If you want to pre-cache bitmaps, you can still do it with same efficiency, the logic will just be outside the decoder.
  2. Custom ImageView. It’s a regular ImageView, but it runs decoder in a background thread and does setImageBitmap on itself when needed. It is a somewhat crude approach, it has to be controlled manually, like you have to stop the thread when pausing activity. But there’s also tons of benefits. Like, it’s always clear what’s going on with it, you can pause it at any time and use it as a regular ImageView without any fear of it going rogue and consuming every possible resource.

I have tested the project on my devices, and it appears to run about half as fast as WebView. Which is not great, but also not that bad, considering that Java is not a great language for a decoder, and WebView probably uses some kind of optimization. It runs without delays on newer devices, and starts to lag on heavy videos on something older. But then, by “older” I mean my ultra-budget Samsung phone made in 2011 and running Android 2.3. On that phone, most videos were running on 90 to 100% speed, except some really heavy ones.

All in all, another example of how things are usually really simple once you start looking into them instead of using libraries. GIF format is really not that complex and is well-documented, so it’s not that hard to render manually. C++ would be much more efficient for it, but oh well.

PS: after performance optimizations by Aleksandr Shardakov, decoder runs about twice as efficiently and now shows speeds on par with WebView. Native C++ implementation (see comments) would be even more efficient, but it’s also more difficult to use. So I guess this approach is also not bad.