Unicode bloats and emoji kitchens

Unicode 13 is coming, and bringing with it a handful of exciting things. Particular to my interests is a new Legacy Computing section with characters like seven-segment display numerals and graphics characters like those found on the Commodore 64 and other machines of the era. Of course, new emoji are coming as well, including among other things a magic wand, a beaver, and the trans pride flag (finally!). Unicode is doing a lot of necessary language work behind the scenes as well; the 12.1.0-13.0.0 diffs show additional characters for well over ten different scripts. In short, major versions of the Unicode Standard span a wide variety of character types, because the standard itself, by design, spans a wide variety of character types. From the Unicode Consortium’s FAQ:

Unicode covers all the characters for all the writing systems of the world, modern and ancient. It also includes technical symbols, punctuations, and many other characters used in writing text. The Unicode Standard is intended to support the needs of all types of users, whether in business or academia, using mainstream or minority scripts.

Something that always comes up when a new version is released, but seemed particularly strong this go-around, is the notion of bloat in Unicode. While things like legacy computing symbols may factor into this notion, it is generally directed at Emoji. I specifically heard quite a few suggestions (all from Googlers, FWIW) this time that emoji never should have been in Unicode to begin with. This notion is patently absurd to me, as is the fundamental notion that Unicode is or has any risk of becoming bloated.

Moving emoji into a standard was largely a response to existing interoperability issues. Emoji started1 on Japanese mobile phones, with a handful of competing non-standards. SoftBank sent theirs via character sequences, using an old-school shift in/out escaping mechanism. NTT DoCoMo used Unicode, but without emoji being a part of the Unicode Standard, they did it via private use characters. Au simply opted to send images, which is interoperable but… means you’re sending images when character encodings should and would suffice. Unicode sprinkled in bits of emoji here and there, but without vendor support it wasn’t a priority. Back in its “Don’t be evil” days, Google provided the initial push and with Apple on board as well, building emoji into Unicode became a priority.

I fail to see what other solutions could have been considered. I maintain that even in the 5G era, sending images back and forth when character codes would suffice is a Bad Idea. It also removes standardization; the emoji you see won’t be from your vendor of choice, but from a disparate collection of your contacts’ vendors. A new character encoding could be proposed, specifically for poops and cat faces, but that complicates everything while solving nothing. Aside from the (trivial, but nonzero) overhead required to shift encodings midstream and the (again, likely trivial but still additional) burden on vendors to contribute to and develop software for an unnecessary additional standard. There would be complications surrounding redundancy; Unicode has several characters that are intended to be rendered in either emoji-style or traditional glyphs. An additional character can be joined with these to force either rendering. Presumably the same thing could be achieved with two side-by-side encodings, but it certainly wouldn’t be as graceful, and would require interstandards crosswalking to resolve instances where a system lacked one of the redundant glyphs.

The only problem either of these solutions would conceivably solve would be reducing the frivolous emoji burden on the Unicode Consortium, allowing them more resources to devote to potentially-more-important matters like scripts for underrepresented writing systems. This is basically how it already works, though – the Consortium is broken up into a handful of technical committees, and emoji gets its own full-blown subcommittee. Folks aren’t being taken away from their work on Khitan to discuss the merits of a plunger emoji. Additionally, also lifted straight from the FAQ:

Their encoding, surprisingly, has been a boon for language support. The emoji draw on Unicode mechanisms that are used by various languages, but which had been incompletely implemented on many platforms. Because of the demand for emoji, many implementations have upgraded their Unicode support substantially. That means that implementations now have far better support for the languages that use the more complicated Unicode mechanisms. See L2/18-044.

I find it extremely difficult to find a good-faith explanation as to why Unicode should not house these conveyances. There’s really nothing in the standard itself to bloat2. Sure, there are more tables to print and edge cases to discuss, but most of the burden is on the group making decisions about inclusion and the like, and on vendors for support concerns. Neither of these things would change if a separate encoding standard was introduced. Both of these things, however, would change if we were to simply send images.

From a user perspective, plenty of people don’t seem to understand that emoji aren’t images, and the technical details… don’t really matter. Case in point: see how many people are constantly asking Apple to give them their desired emoji. From a vendor perspective, this means throwing standards to the wind and sending images could make for good branding: your unique set of symbols is now spread beyond your devices, and if you can just plop whatever in there… well, suddenly you’re the phone, the OS, the keyboard that can uniquely send the ‘pigeon pooping’ emoji.

I started writing this post in January and paused for a bit while I didn’t really have the energy to write. At that point, my thoughts were largely just that this idea that emoji belong (and have always belonged) outside of Unicode was absurd. I’ve mentioned Google a couple of times, and I would have mentioned them regardless since it seems very fitting to me that present-day Googlers seem to have a very meh attitude toward open standards. But then something else happened: Google’s mobile keyboard, Gboard, added support for Emoji Kitchen, an emoji mashup toy made by Googlers3. Which means… Google is doing the exact thing I just described. They’re taking an open, rational standard, and making something uniquely theirs and proprietary. It’s a fun little lark, but it’s also rather disconcerting.

I have hopefully made it clear that I think sending images when character codes will suffice is user-hostile, even if users aren’t aware of it. This becomes an even bigger concern when we work in accessibility – Unicode has descriptions for any given character that a screen reader will interpret; sending images relies on vendors to attach alt-text and for the protocol/format to support it (I don’t believe there is any provisioning for this in MMS, for instance). Emoji Kitchen, the website, doesn’t seem to attach any alt text to its final creation. I don’t know if the Gboard mode does, or again how well that would even be supported across typical messaging platforms. In all, I think the idea that emoji should have been kept out of Unicode from the get-go is indefensible, and I really hope major vendors who love obscuring open things into proprietary things don’t push us all into sending images to one another just to say 👋.

  1. Emoji ‘started’ in a lot of disparate ways, of course. Emoticons predated emoji, AIM transparently replaced these with faces, ideograms and pictograms have existed throughout time. But the set of characters that directly grew into what we now know and love as emoji, and the basic mechanisms of how they exist within text come out of this Japanese mobile phone phenomenon. ↩︎
  2. Within reason, of course. And there are plenty of checks in place to ensure reason is maintained. Flip through the document register some time for an idea of how much discourse goes on regarding proposals. ↩︎
  3. Emoji Kitchen’s about page (which I can’t link to because the website doesn’t function like a website, naturally) credits two Googlers without ever mentioning Google or Alphabet. Despite its integration into Gboard, it’s hard to officially state that it’s a Google product. But it is certainly the product of Googlers. ↩︎