The Unicode Consortium

The primary function of the Unicode Consortium is to standardize character encoding. Before Unicode, countless different encoding schemes existed, each supporting a limited set of characters, often specific to a particular language or region. This led to "mojibake" (garbled text) when files were exchanged between different systems or languages. The Consortium's work resolves this by providing a single, consistent way to encode virtually every character from every writing system in the world, along with symbols, emojis, and control characters necessary for proper text display and processing.

Their core activities include:

1. Developing and Maintaining the Unicode Standard: This is the central pillar of the Consortium's work. The Unicode Standard is a comprehensive specification that defines:

   • Character Repertoire: The set of characters that are encoded. This includes characters from historical and modern scripts, mathematical symbols, punctuation, control codes, and, increasingly, emojis.
   • Character Encoding Model: How these characters are represented in binary form (e.g., UTF-8, UTF-16, UTF-32).
   • Character Properties: Information about each character, such as its script, directionality (left-to-right, right-to-left), case mapping, and numeric value. These properties are crucial for correct text rendering, sorting, and searching.
   • Algorithms for Text Processing: Rules for how text should be displayed, sorted, normalized, and broken into lines, words, or grapheme clusters (what users perceive as a single character, even if composed of multiple code points, like "é" or many emoji sequences). This includes the Unicode Bidirectional Algorithm (for mixed left-to-right and right-to-left text), collation algorithms, and segmentation algorithms.

2. Harmonizing with ISO/IEC 10646: The Unicode Standard is developed in close coordination with ISO/IEC 10646, an international standard for a universal character set. The two standards are kept in sync, ensuring global consistency.

3. Encouraging Adoption and Implementation: The Consortium actively promotes the adoption of the Unicode Standard by software developers, hardware manufacturers, and content creators. They provide resources, guidelines, and tools to help implementers correctly integrate Unicode into their products and systems.

4. Maintaining Key Data Files: The Consortium publishes and maintains crucial data files that accompany the Unicode Standard, such as the Unicode Character Database (UCD), which provides all the character properties. These data files are essential for anyone building Unicode-compliant software.

5. Standardizing Related Technologies: Beyond just character encoding, the Consortium also works on related standards that ensure correct global text display and functionality. This includes:

   • Common Locale Data Repository (CLDR): Provides locale-sensitive data for various languages and regions, such as date and time formats, currency formats, number formatting, calendar rules, and sorting orders. This is crucial for internationalization (i18n) and localization (l10n) of software.
   • International Components for Unicode (ICU): A widely used open-source C/C++ and Java library that provides robust Unicode and internationalization support, implementing many of the Unicode and CLDR algorithms. While developed by IBM, it's a key part of the Unicode ecosystem.

6. Addressing New Requirements: As global communication evolves, the Consortium continually evaluates new character requests (e.g., historical scripts, new symbols, emojis) and updates the standard to incorporate them, ensuring Unicode remains comprehensive and relevant.

Products and Services Offered (Directly and Indirectly) by The Unicode Consortium:

As a non-profit standards organization, the Unicode Consortium does not sell "products" in the commercial sense. Instead, its "offerings" are primarily intellectual property, documentation, data, and community resources that enable the global software industry.

1. The Unicode Standard Specification:

   • Digital Version: The complete, up-to-date specification is freely available for download in PDF format from unicode.org. This is the foundational "product" that all implementers rely on.
   • Printed Version (Unicode Standard Core Specification): While the digital version is primary, printed editions are sometimes published, offering a physical reference.

2. Unicode Character Database (UCD):

   • A set of data files that provide normative and informative property values for each Unicode character. This includes information on character names, general category, script, case mappings, numeric values, bidirectional properties, and much more. These files are essential for software developers.

3. Unicode Technical Reports (UTRs):

   • Documents that provide detailed guidelines and algorithms for implementing various aspects of the Unicode Standard, such as the Unicode Bidirectional Algorithm, Unicode Normalization Forms, Unicode Text Segmentation, and Unicode Regular Expressions.

4. Common Locale Data Repository (CLDR):

   • The most extensive collection of locale data available, used by major operating systems (Apple, Google, IBM, Microsoft, Oracle) and software applications for internationalization. It includes data for language names, country names, currencies, date and time formats, number formats, collation (sorting) rules, and more, all freely available.

5. Unicode Emoji Standard:

   • Specific technical reports and data files related to the encoding, properties, and display of emoji characters, including recommendations for emoji sequences (e.g., skin tones, gender variations, zero-width joiner sequences for family emojis). This ensures consistency across platforms for the popular visual language.

6. International Components for Unicode (ICU) (Collaborative):

   • While developed primarily by IBM, the ICU libraries (for C/C++ and Java) are a direct implementation of the Unicode Standard and CLDR. They provide services for Unicode text processing, internationalization, and localization, making it easier for developers to build global-ready software. These are freely available open-source libraries.

7. Glyph & Font Information:

   • The Consortium does not create fonts but works closely with font developers and provides guidance on how to correctly render Unicode characters in fonts. They also maintain a list of useful fonts that support various Unicode scripts.

8. Online Tools and Resources:

   • Unicode Character Lookup: Online tools on unicode.org to search for characters, view their properties, and see their representation.
   • Tutorials and FAQs: Educational resources to help users and developers understand and implement Unicode correctly.
   • Public Forums and Mailing Lists: Platforms for discussion, support, and collaboration within the Unicode community.

9. Membership and Participation:

   • The Consortium offers various levels of membership (Full, Associate, Institutional, Individual) to organizations and individuals. Members gain influence in the standardization process, early access to drafts, and participation in working groups. This "service" allows entities to contribute directly to the future of global text.

In essence, The Unicode Consortium provides the fundamental infrastructure that allows text from virtually any language to be displayed, processed, and exchanged digitally in a consistent and interoperable manner. Its "products" are the meticulously defined standards and the comprehensive data files that underpin nearly all modern software and digital communication, making global computing and information exchange possible.