The trouble with Animated GIFs

One neat ability we have in Equinox is showing animated GIFs… short animation clips which can make an XR setup come alive. Eventually we will support proper MP4 movies, but for now, let’s start with animated GIFs.

Our tech stack is based around Unity, and as a first step we were searching for a ready-made solution in the awesome Unity Asset Store. The original plan was to find something which can decompress individual frames of GIF animations on-the-fly just before they are shown. GIFs are an ancient format, and they should be easy on todays CPUs. Each frame in a GIF is compressed individually, so in theory, it should be easy to decompress frames into Texture2Ds and show them where we want them.

Unfortunately, after trying a couple of assets on offer, the best we could do is decompress all the frames into textures at load time. This of course is extremely suboptimal, both because it eats up a lot of RAM (around 350 MB for an average meme GIF), and because it takes a noticable time to actually do decompression, even on recent flagship devices. Really, it’s just plain bad design if we stored our animations like a set of RGBA textures.

What to do?

We’ve recently switched to storing ASTC-compressed textures on the server-side so the mobile app doesn’t have to spend resources decompressing JPEGs and GIFs, as well as to massively reduce memory consumption. That’s a topic for another article.

That lead me thinking… thorugh we will arrive to supporting h264 videos eventually, for now, why not recompress GIF frames as ASTC textures on the server and use that?

And it’s exactly what we did. Googling around didn’t find any existing attempts to store videos as a sequence of ASTC frames, so we did it ourself and created a file format for that. Nothing fancy.

There is a precedant for doing it that way. The MJPEG format, almost as ancient as GIFs, is created around the idea that we can simply compress individual video frames as JPEGs and bundle them all together in a single file. In honour of that, we’re calling our format MASTC – Motion ASTC.

While it’s not a standard appropach and there are far better suited ways for compressing videos, it’s not half as bad. A typical meme-gif takes around 1.5 MB – 2 MB in RAM as a set of ASTC textures, and each individual frame typically takes between 32 KiB and 64 KiB, depending on the video dimensions. In fact, videos compressed in this way are about half the size of the original GIFs.