What is the Token System

January 27

# Tech

# Front-End

# Token System

Intro

The last article discusses the challenges of customized UI component libraries. For front-end developers (referred to as FE below), it involves writing lots of tedious style code, while for user experience designers (referred to as UED below), it requires a lot of effort to ensure that the page styles match the design drafts. To address these issues, I propose a solution: a workflow based on a token system design, which reduces the development costs for FE and lowers the human resource costs for UED.

FE & UED workflow

This article will introduce what is a token, and how to organize tokens to meet our needs.

Concept of Token

First of all, what is a token? Here is the definition from DTCG (Design Tokens Community Group):

A (Design) Token is information associated with a name, at minimum a name/value pair.

In detail, A Token, or Design Token, is the abstraction of visual style. A token defines a variable, which can represent a style value, like a color(#ffffff, rgb(0, 0, 0)), padding(1px), or a typography(font-family: Inter).

What does a token look like? Here is a basic sample:

1{
2  "primary": {
3    "value": "#5771d2",
4    "type": "color"
5  }
6}

Tokens are usually organized using JSON, which is a key-value pair. For this token, it defines a color, named primary, the value of which is #5771d2.

Usually, we use a group of tokens to describe a set of styles, like:

1{
2  "color": {
3    "primary": {
4      "value": "#5771d2",
5      "type": "color",
6      "description": "It's used for highlight."
7    },
8    "warning": {
9      "value": "#964909",
10      "type": "color",
11      "description": "It's used for warning."
12    }
13  },
14  "padding": {
15    "normal": {
16      "value": "4px",
17      "type": "spacing"
18    },
19    "tight": {
20      "value": "2px",
21      "type": "spacing"
22    }
23  },
24  "margin": {
25    "normal": {
26      "value": "padding.normal",
27      "type": "spacing"
28    },
29    "tight": {
30      "value": "1px",
31      "type": "spacing"
32    }
33  }
34}

This example is more complex than the previous one. This group of tokens defines three kinds of tokens: color, padding and margin, with each type having two specific tokens. So we have six tokens here: color.primary, color.warning, padding.normal, padding.tight, margin.normal, and margin.tight.

A token's value can reference another token's value, just like in the example above where margin.normal can reference the value of padding.normal. It's very useful when you want to reuse some common values.

As you can see, there are two extra properties along with the value, which are description and type. You can define any properties you want, like comment, and author, these attributes will be passed through when you compile them(We will talk about token compilation later). Importantly, ensure you remember the value key, which is the pivotal property of a token.

With tokens, we can efficiently structure massive styles into a JSON file, which is a platform-agnostic format. In this way, we can use it anywhere. This is the foundation of the token system for UI component lib.

Hierarchy Design of Tokens

After understanding the concept of tokens, let's talk about the hierarchy of tokens, that is how we organize the layer of tokens to meet our control of different levels of customization.

Here is the hierarchy design of tokens:

hierarchy of tokens

We have three layers here:

• Reference token: Tokens that have values associated with them (we can call them literal values)and are used as a reference for other tokens, such as ref.color.red -> #d04848
• System token: The decisions and roles that define the design system's character, including but not limited to color, typography, and shape, and system tokens usually have design meanings, such as sys.color.warning -> ref.color.red
• Component token: The design attributes assigned to elements within a component, such as btn.color.text -> sys.color.warning.

These three layers have a reference relationship: reference token -> system token -> component token.

With these three layers of tokens, we decoupled literal values, the design system's character, and component elements. So we can safely and quickly modify any layers to meet our needs.

There are three scenarios here:

1. Reference-level customization
   Assuming that we already have a set of colors used in design, like: ref.red(#D04848), ref.yellow(#FDE767), and ref.blue(#6895D2). One day UED realized that ref.red(#D04848) was not very readable and needed to be changed to ref.red(#F15550). In this case, we only need to modify the reference token, and all the tokens that reference it will be updated accordingly.
2. System-level customization
   Assuming that we have already used ref.color.yellow -> #FDE767) as our primary theme color(sys.color.primary -> ref.color.yellow). But our product design style has been upgraded, we need to switch to #FD9967 as our primary theme color. We cannot directly modify the reference token ref.color.yellow -> #FD9967 as we did in the previous example because doing so would affect other system tokens that use ref.color.yellow. In this case, we can create a new reference token ref.color.yellow-2 -> #FD9967 and modify the system token sys.color.primary -> ref.color.yellow-2. In this way, all the places that refer to sys.color. primary will be updated to use the new value, but the places that refer to ref.color.yellow will not be affected.
3. Component-level customization
   Assuming that we use ref.fontWeight.xs -> 100 as our default font-weight sys.fontWeight.light -> ref.fontWeight.xs, and Toast component references sys.fontWeight.light as comp.toast.text.fontWeight. One day we received feedback from users that the font was too thin to read. We cannot directly modify sys.fontWeight.light, for the same reason as in the previous example. Modifying sys.fontWeight.light would change all other component tokens that reference it. In this case, we can reference comp.toast.text.fontWeight to a new system token. In this way only all Toast components will use the new font-weight and other components will not be affected.

Overall, reference tokens are the best choice for modifying underlying properties, system tokens are the best choice for modifying global properties for all components, and component tokens are the best choice for modifying the style of a specific component.

Here is an example of how to organize the tokens: Github

1.
2├── components // store all component tokens
3│ └── button.json
4│ └── toast.json
5└── core // store core tokens
6├── ref.json
7└── sys.json

Token System

Now that we've covered the definition and hierarchy design of tokens, let's take a look at the Token System. A token system is an approach to abstracting, structuring, and organizing visual styles using tokens. The token system has three layers: reference tokens, system tokens, and component tokens, which are organized hierarchically to allow for customization at different levels. This system can be leveraged to safely and quickly modify any layers to meet design needs without any side effects.

In the End

We have explained the concept and hierarchy design of tokens and how to organize them to meet customization needs. In the next article, I will introduce how to use tokens at the FE end and UED end, and how to connect these two ends.

Reference
1. Design Tokens Format Module
2. Style Dictionary
3. Material Design