Setting up a CI server

Continuous integration (CI) servers are a great way to keep your app, web or otherwise, up to date with the current version in the main branch of your repository. Locally installed CI servers can be very painful to set up and configure. Cloud-based CI servers have simplified things so that now there's no reason not to set one up for any publicly deployed app.

The instructions on this page use Github Actions to do continuous integration and deployment for a React web app, hosted on Firebase.

Enable Github Actions

Github Actions should be enabled for all public repositories. If this isn't the case, check this page.

Tell Github how to build and test your app

Github looks for workflow files. These are YAML files in the subdirectory .github/workflows at the root of the repository.

YAML ("YAML Ain't a Markup Language") is a very simple language for configuration files.

Each YAML file contains information on what to do when an event occurs. The most common event is push. When a local repo is pushed to Github, Github will run any YAML files that list push as the event trigger.

Central to the YAML file are the steps that specify the actions to do. There are two kinds of actions: run: commands that execute basic actions, such as npm install, and uses: commands that call community-defined Github Actions to do more complicated processes. Think of uses: like a subroutine call. You can pass parameters to the subroutine with the with: keyword. You can set environment variables used by npm scripts with env:

Before using the sample scripts here, check to see if the version numbers on the following Github Action scripts need to be updated.

When a job completes or fails, push-triggered notifications are sent to the committer.

It would be nice if the entire team was notified by Github but I haven't seen an example of how to do that.

YAML for ReactJS web apps

First, here is a basic workflow file for ReactJS.

Read through the text that follows to understand how the script works. 
name: React CI

on: [push]

jobs:
  build:
    name: Build and unit test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm install
      - run: npm run build --if-present
      - run: npm test
  
  cypress:
    name: Cypress test
    needs: [build]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Cache firebase emulators
        uses: actions/cache@v2
        with:
          path: ~/.cache/firebase/emulators
          key: ${{ runner.os }}-firebase-emulators-${{ hashFiles('~/.cache/firebase/emulators/**') }}

      - run: npm install -g firebase-tools

      - uses: cypress-io/github-action@v4
        with:
          build: npm run build
          start: npm run em:run
          wait-on: http://localhost:3000

  deploy:
    name: Firebase deploy
    needs: [build, cypress]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v3
        with:
          node-version: '20'
      - run: npm install
      - run: npm run build
      - run: npm install -g firebase-tools
      - run: firebase deploy --token "${{ secrets.FIREBASE_DEPLOY_TOKEN }}" --non-interactive

This defines three jobs:

  • build: this builds the app and runs any unit tests
  • cypress: this builds the app and runs any Cypress end-to-end tests
  • deploy: this builds and deploys the app to a Firebase host site if the first two succeed.
If you have no Cypress tests, or don't want to deploy, leave those jobs out. If you leave out just the Cypress job, remove Cypress from the jobs that the deploy job needs.

A job is a set of steps. Jobs are run independently. Separate jobs can run in parallel, but it means the build step has to be repeated.

Steps inside a job are run sequentially. Steps labeled with run: are executed in a Linux shell. Steps labeled with uses: run libraries of steps that others have defined. These libraries are called Github Actions. The with: lines are used to pass parameters to those libraries.

The Build job verifies that the React app builds without warning. The other two jobs check that this job succeeded. There's no point trying to test or deploy if your app doesn't build. Before committing code and triggering the Build job, be sure your application builds locally without warnings.

Warnings are treated as errors by a CI server and will stop the build.

The Cypress job shown here uses the Firebase emulators for authentication and data, as described in the Cypress guide. There are two differences from how this is run locally. First, the script does not start the emulators and server and then call Cypress. If it did that, the Cypress step would never happen because that the server step never stops. Instead, we use the Cypress Github action to start the emulators and server, wait for the given URL to appear, run the tests, then shut everything down. The script here also tells Github to cache the Firebase emulators, so that they won't be downloaded again if still in the cache.

The Deploy job needs authentication in order to upload files to to the Firebase hosting site. To make non-interactive deployment possible, Firebase supports passing a secret token to authenticate. How you get that secret key and pass it to your script is described later. Similar options exist for deploying to other hosting sites, like Github pages.

The deploy job is shown so you can see how continuous deployment works, and how to use secret keys in Github action scripts. Do not do continuous deployment until you have a robust set of automated tests to verify that what you are deploying is ready to deploy.

Test your CI set up

To see the status of an action, click on Actions at the top of your repo's main page. There you will see a list of jobs that have been run or are in process. From there, you can view the terminal output of the job. This is where you can see if a build and test passed or failed.

Running a CI/CD job may take several minutes or more. That's fine. You normally do not wait to see if your build and tests succeed on the CI server. You build and test locally, commit and push, and go on to your next task. Github will email you the results.

Deployment tokens

To deploy to Firebase, you have to have selected that option when you run firebase init. The initialization will ask what Firebase project you want to associate your app with. This could be a new project or an existing one. If you are already using the Firebase Realtime Database, use that project.

When Firebase asks what directory to use, enter build for React, and web-build for Expo.

Say no when it asks if you want to integrate with Github Actions. What that does is described here. It supports a useful preview feature, but doesn't include Cypress testing and hides how Github Actions work from you.

Test that deployment works locally first, do this for ReactJS:

npm run build
firebase deploy

When this command completes, it prints the URL where you can see your app running.

To automate deployment to Firebase, the scripts above use 

firebase deploy --token "${{ secrets.FIREBASE_DEPLOY_TOKEN }}" --non-interactive

The token is a secret encrypted key that Firebase will give you, that you need to store on Github. It should never be shared or put in any public location.

To get the key, use this command:

firebase login:ci

This will send you to a browser page where Firebase will ask you to approve the request. When you do, a long encrypted string will be returned for the above command.

Don't put secret keys into the YAML file or any other code file that is checked in! Instead, store the key on Github in a secret environment variable. Go to the Github repository for your app. Go to the Settings page. Click on Secrets. Create a new repository key. Call it FIREBASE_DEPLOY_TOKEN. Copy and paste the encrypted key returned by firebase login:ci.

Debugging CI issues

Setting up Github Actions for CI is a snap, compared to how complicated setting up a CI server used to be. Still, there are several moving parts that can get jammed up.

Github Action failures can be frustrating because you don't know something is wrong until you push your changes and the error messages can be obscure or absent.

Rarely, you might get an error pushing your code to Github after you create a workflow file. Check your local Git output for an error like this 

refusing to allow a Personal Access Token to create or update workflow `.github/workflows/main.yml` without `workflow` scope

This means the token being used to authenticate and allow you to push to Github does not have the power to create workflows. This token might be in the URL you are using to push to Github or stored somewhere and used by your IDE, e.g., VS Code. You need to generate a new Github Personal Access Token and make sure that the box for workflow access checked. Then you can add that token to the remote URL in your local Git configuration.

Like Github login, personal access tokens are unique to each team member and locally configured. They are never shared or stored in the repository.

To see what happened to a workflow on Github, click on Actions at the top of your Github repo page to open the Actions dashboard. The dashboard shows the jobs that have been run or started. If a job has failed or doesn't seem to have done anything, you can click on it to see the job details.

If there was a YAML syntax error, the line with the problem will be indicated. Unfortunately, there is usually no detail and the line pointed to may not be the one with the error. For example, if Github points to a steps: line, all you know is that some line inside that steps block is wrong.

To get a more useful message when there is a YAML syntax error, try pasting your script into the YAML Checker.

If the job details just say the job did nothing, click on the build link to see the console log. The console entries in the build log show step names. Clicking on a step name opens it to show the details of what happened. Look for error messages and warnings. Look for tasks that start a process, but never exit, like starting a web server.

I spent two hours on a deployment problem one time. I thought I had bad authentication tokens. Opening the relevant part of the build log revealed it was just a missing directory issue.

Make sure you're looking at the current build, not one of the old jobs in the build history.

If you still have issues, post to Campuswire your YAML file, repo link, and link to the failing build page.

Sources

The following resources were used to develop the code above.

© 2024 Chris Riesbeck
Template design by Andreas Viklund