[CI&CD] Connecting Front-end Developers and Designers with the Token System

March 24

# Tech

# Front-End

# Token System

Intro

So far, we have introduced What is the Token System, and Connecting Front-end Developers and Designers with the Token System (for Designers and for Developers).

In simple terms, the token system provides the ability to create a standardized and structured set of style data, bridging the gap between design and code. Designers can structure and store all design details in a platform-agnostic JSON file with some tools, and developers can parse and compile platform-specific style sheets from this JSON file.

All of these processes are recorded by tokens, reducing the cost of communication and manual inspection, and making the design and development process more efficient and accurate.

The previous article introduces how developers use tokens, including token compilation and token usage. In this article, we will focus on using automation to finish the compilation, testing, publishing, and updating of tokens.

Let's take an overview of the workflow:

the overview of the workflow

We are discussing two phases here, Dev and Release.

In the Dev phase, we will focus on:

• Build and publish the test style package when designers push token changes
• Update the local style package after git pull when developers pull token changes

In the Release phase, we will focus on:

• Publish the storybook demo website when pulling a request to merge into the main branch
• Build and publish the production style package when the request merged
• Run unit testing

Prerequisites

CI/CD

CI/CD are methodologies in software development used to automate the integration and deployment of code. CI (Continuous Integration) refers to the practice of developers frequently merging code changes into a central repository, where builds and tests are run automatically. CD (Continuous Delivery/Deployment) ensures that software can be released reliably at any time, extending CI by deploying all code changes to a production environment automatically.

There are many popular CI/CD platforms, like:

• GitHub Actions: Integrates with GitHub to automate your workflows.
• GitLab CI/CD: Covers the entire software development lifecycle.
• CircleCI: A cloud system that runs complex workflows using Docker containers.
• ...

Each platform has its method of configuration, but they typically use YAML files to define the automation steps. YAML is an intuitive data serialization standard that is ideal for configuration files, making it well-suited for describing everything from simple to complex automation scripts.

Taking GitHub Actions as an example, to set up a CI/CD workflow, you would create a YAML file within your repository under the .github/workflows directory. Within this YAML file, you can specify the events that trigger the workflow (like a push to a particular branch), the jobs that need to be executed, and the steps that each job consists of.

Here is a simple example using a YAML configuration that demonstrates how to automate a common task: publishing a package to the npm registry.

1# This line names your workflow. "Publish npm package" is the name we've chosen here, and it will appear in the list of workflows running on your repository.
2name: Publish npm package
3
4# The `on` keyword tells GitHub Actions when to run the workflow. In this case, it's configured to trigger on a `push` event to the `main` branch.
5on:
6  push:
7    branches: [main]
8
9# Under `jobs`, you can define one or more jobs that the workflow will execute. Here, we have a job named `build-and-publish`, which is set to run on the latest version of Ubuntu (`ubuntu-latest`).
10jobs:
11  build-and-publish:
12    runs-on: ubuntu-latest
13    # Each job contains a sequence of `steps`. Steps can run commands or actions.
14    steps:
15    - uses: actions/checkout@v2
16    - uses: actions/setup-node@v2
17      with:
18        node-version: '14'
19        registry-url: 'https://registry.npmjs.org'
20    - run: npm ci
21    - run: npm test
22    - run: npm publish
23      env:
24        NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}

This YAML file creates an automated process that, upon every push to the main branch, will set up a Node. Js environment, install dependencies, run tests, and if everything is successful, publish your package to npm. This automates the steps that a developer would otherwise have to do manually, ensuring that the process is consistent and error-free.

Dev Phase

Build and Publish the Test Style Package

Code reference: GitHub

This workflow is mainly prepared for designers. When designers finish designing and push token changes to the remote, this workflow will be automatically triggered to compile the latest tokens into style files and publish a test package. Users can use this published test package to see the latest changes made by designers.

Here is the workflow:

1name: Build style from tokens and publish test package
2
3on:
4  push:
5    branches:
6      - "feature/*"
7      - "feat/*"
8    paths:
9      - "packages/token/src/tokens/cache/**"
10
11permissions:
12  contents: write
13
14jobs:
15  style-build:
16    runs-on: ubuntu-latest
17
18    steps:
19      - uses: actions/checkout@v4
20
21      - name: Install pnpm
22        uses: pnpm/action-setup@v3
23
24      - name: Setup Node.js
25        uses: actions/setup-node@v4
26        with:
27          node-version: 18
28          cache: pnpm
29
30      - name: Install dependencies
31        run: pnpm install --frozen-lockfile --prefer-offline
32
33      - name: Prebuild
34        run: |
35          cd packages/token
36          pnpm prebuild:figma
37
38      - name: Build
39        run: |
40          cd packages/token
41          pnpm build
42
43      - name: Commit
44        run: |
45          status=$(git status)
46          if [[ $status == *"modified"* || $status == *"deleted"* || $status == *"added"* || $status == *"Untracked files"* ]]; then
47            echo "$status" | awk '/modified|deleted|added/ {print "✨ " $0} /Untracked files/ {print "🔍 " $0}'
48          else
49            echo "✅ No changes detected"
50          fi
51          git config --global user.name "GitHub Actions"
52          git config --global user.email "actions@github.com"
53          git diff --quiet && git diff --staged --quiet || (git add . && git commit -m "[CI] build token")
54          git push
55
56      - name: Setup .npmrc file to publish to npm
57        run: echo "//registry.npmjs.org/:_authToken=${{ secrets.NPM_TOKEN }}" > ~/.npmrc
58
59      - name: Publish alpha package
60        run: |
61          cd packages/token
62          commit_hash=$(git rev-parse --short HEAD)
63          version="0.0.0-alpha.$commit_hash"
64          pnpm version $version --no-git-tag-version
65          pnpm publish --access public --tag alpha --no-git-checks

First, this workflow will only be triggered when there are changes under the path packages/token/src/tokens/cache/** where the tokens are stored.

Then, some preparation work needs to be done, including checking out the latest code and installing Node.js, pnpm, and dependencies.

After that, we can start compiling the tokens, using pnpm prebuild:figma and pnpm build (refer to this article for details).

Then, a new commit is submitted to store the compiled result.

Finally, we use a short commit hash as the version, 0.0.0-alpha.$commit_hash, to publish a test package.

You can also use some commonly used or internal IM tools to notify relevant personnel.

Update the Local Style Package

Code reference: GitHub

This workflow is mainly prepared for developers. Since the dist directory, which stored the result of the token compilation result, is ignored by Git, developers have to re-build token packages after pulling the latest changes to update the content under the dist directory.

We use husky, an npm package that allows you to use hooks to run scripts before some actions in Git.

Here is the script:

1#!/usr/bin/env sh
2. "$(dirname -- "$0")/_/husky.sh"
3
4# Run these commands only if packages/token changed
5if git diff --quiet HEAD@{1} HEAD packages/token; then
6  echo "📦 packages/token not changed, skipping build"
7  exit 0
8fi
9
10echo "📦 packages/token changed, building..."
11cd packages/token
12pnpm run build

This is very simple. Check if there are changes under the packages/token directory in the latest commit. If there are changes, then build. If not, exit.

In this way, after the previous workflow is completed, developers can be notified (if configured) to execute git pull to get the latest tokens and automatically compile the latest style files.

Section Summary

With these two automations, the collaborative process between designers and developers is seamlessly executed during the development phase. After designers finish designing, they only need to push changes, and all changes will be automatically synchronized to developers. Developers do not need to communicate with designers about the changes, as everything is recorded in tokens and automatically compiled into style files.

Release Phase

Publish the Storybook Demo Website

Code Reference: GitHub

This workflow is mainly designed for review before going online. When development is completed and ready to be merged into the main branch for deployment, this pipeline will be automatically triggered to apply the latest tokens to the components and publish Storybook to preview the latest style effects on the components.

Here is the workflow:

1name: Deploy Storybook to GitHub Pages
2
3on:
4  pull_request:
5    branches:
6      - main
7    paths:
8      - "packages/**"
9
10permissions:
11  contents: write
12
13jobs:
14  deploy:
15    runs-on: ubuntu-latest
16    steps:
17      - name: Checkout code
18        uses: actions/checkout@v4
19        with:
20          fetch-depth: 0
21
22      - name: Install pnpm
23        uses: pnpm/action-setup@v3
24
25      - name: Setup Node.js
26        uses: actions/setup-node@v4
27        with:
28          node-version: 18
29          cache: pnpm
30
31      - name: Install dependencies
32        run: pnpm install --frozen-lockfile --prefer-offline
33
34      - name: Update token package
35        run: |
36          cd packages/token
37          pnpm prebuild:figma
38          pnpm build
39
40      - name: Build Storybook
41        run: |
42          cd packages/react
43          pnpm run build-storybook
44
45      - name: Deploy to GitHub Pages
46        uses: peaceiris/actions-gh-pages@v3
47        with:
48          github_token: ${{ secrets.GITHUB_TOKEN }}
49          publish_dir: ./packages/react/storybook-static

As we said, the workflow will be only triggered when pulling a request to merge into the main branch and code under packages/** changes.

Like before, some preparation work needs to be done, including checking out the latest code and installing Node.js, pnpm, and dependencies.

Then, we update the latest dist directory of the @ezreal-ui/token package and use it to build the Storybook website within the @ezreal-ui/react component package.

Finally, we use peaceiris/actions-gh-pages@v3 to publish the GitHub Pages. You can preview it Here.

Build and Publish the Production Style Package

Code Reference: GitHub

If everything is ready, we can merge all changes to the main branch. After merging, we need to prepare to build and publish the production package.

The changesets is a tool to manage versioning and changelogs with a focus on multi-package repositories. We use it to manage and publish versions of packages in our project.

A changeset is a piece of information about changes made in a branch or commit.

Before we merge changes into the main branch, we can execute npx changeset to add changesets to describe what we have changed in the development branch.

After a development branch is merged into the main branch, a workflow will be triggered:

1name: Publish Package
2
3on:
4  push:
5    branches:
6      - main
7
8jobs:
9  deploy:
10    runs-on: ubuntu-latest
11    steps:
12      - name: Checkout code
13        uses: actions/checkout@v4
14        with:
15          fetch-depth: 0
16
17      - name: Install pnpm
18        uses: pnpm/action-setup@v3
19
20      - name: Setup Node.js
21        uses: actions/setup-node@v4
22        with:
23          node-version: 18
24          cache: pnpm
25
26      - name: Install dependencies
27        run: pnpm install --frozen-lockfile --prefer-offline
28
29      - name: Create Release Pull Request or Publish to npm
30        id: changesets
31        uses: changesets/action@v1
32        with:
33          publish: pnpm release
34        env:
35          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
36          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

This workflow consumes all changesets and updates to the most appropriate semver version based on those changesets with changesets/action@v1. It also writes changelog entries for each consumed changeset.

Then it creates a pull request with all of the package versions updated and changelogs updated, so we can review all these changes and merge the pull request. After merging, this workflow will be triggered again. Since there is no changeset left, it will run the pnpm release script we defined in package.json, which will build all packages and use the npx changeset publish to publish all packages to npm.

1// package.json
2"scripts": {
3    "build": "pnpm --filter './packages/**' build",
4    "prepare": "husky install",
5    "release": "pnpm build && npx changeset publish"
6  },

Section Summary

With these two automations, the preview work before publishing and all publishing processes are automatically prepared and executed, eliminating the tedious manual work and ensuring stability.

It's worth mentioning that running tests before publishing is also very important, and you can add a script to run tests in these workflows. Since this is a demo project, unit testing has not been added.

In the End

That concludes our work on the token system, which includes introducing the definition of the token system, how designers and developers can use tokens, and using workflows to automatically split and compile tokens, publish preview environments, and publish test and production packages.

The token system is a very powerful tool that can help quickly connect the workflows of designers and developers while ensuring that the results of design and development remain consistent. It reduces the cost of communication and manual checks.

My demo provides a simple implementation of how to use tokens in a component library and the corresponding workflows. You can build upon it to add more customized logic and create a project that meets your needs. Thank you for reading.