Add Cypress to an app
Cypress is a popular end-to-end testing framework. It's easier to set up than Selenium/WebDriver, and typically runs faster. It has limitations. For example, it currently only supports testing with the Google Chrome browser. But it's a simple way to get started on end-to-end testing.
For this walk-through, we'll add testing with Cypress to the Quick, React application.
Install Cypress
cd into your project directory. Then follow the Cypress installation instructions. Install Cypress as a developer dependency, so that it is not included in your app production code.
Initialize Cypress
In your editor, add the following script to your package.json file to make it easy to run Cypress:
"scripts": {
...
"cy:open": "cypress open"
...
}
This defines npm run cy:open to start Cypress for interactive use.
Run Cypress to initialize it. See the instructions to run Cypress.
Choose E2E Testing and the browser you want Cypress to use when running tests.
Write a test
See the instructions for write your first test. Most of the time you will write tests in your editor, not the browser. But it's fine to let Cypress help you write and store the initial test so you can see how files are named and stored.
Commit and push your changes to Github
Test for content
Now let's test for some key content that should appear on the landing page. For example, for the Quick, React application, let's make sure that it begins by showing Fall CS courses.
describe ('Test App', () => {
it ('launches', () => {
cy.visit ('/');
});
it ('opens with Fall CS courses', () => {
cy.visit ('/');
cy.get('[data-cy=course]').should('contain', 'Fall CS');
});
});
Tests are run independently of each other, so the first step in the new test is to visit the landing page. The test then calls cy.get() to get all items on the page marked with the attribute data-cy="course". For each one, it checks for the text "Fall CS". This test will fail if no such items are found, or if any of them do not contain "Fall CS".
There are many ways in Cypress to get items using CSS and text content. It's easy to write such tests and have them pass right away. But these tests are likely to break when the user interface is restyled.
Using a data-cy attribute isolates the tests from visual styling changes. It also marks those elements that are used in tests, which helps avoid their accidental removal.
The test fails because our React code doesn't put the data-cy attribute on the course buttons. So we edit our React code to fix this:
const Course = ({ course, selected, setSelected }) => {
...
return (
<div className="card m-1 p-2"
data-cy="course"
...>
...
</div>
);
};
Don't put data-cy on a React component. React won't know where to put it in the HTML it generates. Put attributes on real HTML elements.
Save the edit, return to the Chrome test results page. Cypress retries tests fairly often, but if there's no change, manually re-run the tests with control-R / command-R. If no mistakes were made, both tests should now pass!
Test 3: Test interaction
Cypress provides many methods to interact with a web page. Cypress handles waiting and retrying automatically, so that test code doesn't normally need to worry about using promises or await calls to wait for actions to complete. By default, Cypress waits four seconds. If a test times out, it's most likely because an error in the test. See the tip on debugging. You can tell Cypress to wait longer and see if the test works, but if it does, you should make your user interface show some kind of progress bar.
For the Quick, React application, let's test that clicking on the Winter button causes winter courses to appear.
describe ('Test App', () => {
it ('launches', () => {
cy.visit ('/');
});
it ('opens with Fall CS courses', () => {
cy.visit ('/');cy.get('[data-cy=course]').should('contain', 'Fall CS');
});
it('shows Winter courses when Winter is selected', () => {
cy.visit ('/');
cy.get('[data-cy=Winter]').click();
cy.get('[data-cy=course]').should('contain' ,'Winter');
});
});
This code visits the landing page. Then it gets any item marked with data-cy="Winter" and clicks it. Finally, it gets the items with the CSS class course, and tests that they contain the text "Winter CS".
As before, this new test will fail until we add the data-cy attribute in React:
const TermButton = ({term, setTerm, checked}) => (
<>
<input ... />
<label className="btn btn-success m-1 p-2" htmlFor={term} data-cy={term} >
{ term }
</label>
</>
);
Note that the data-cy attribute is put on the label
not the input. This is because Bootstrap CSS for radio buttons and checkboxes
says pointer-events: none. With Bootstrap, you are supposed to click
on the labels not the input element.
With any luck, the Chrome test results page should now show three passing tests.
Common Cypress failures
Cypress can fail to find a component marked with data-cy for a number of reasons. You might have misspelled the value -- case matters! You might have put the attribute on a React component instead of HTML. The component might not be rendered yet.
To check for some of these issues, open your app in a browser, open the console, and try using JavaScript to find the component yourself. E.g.,
document.querySelector('[data-cy=Winter]')
Type exactly the same string you pass to cy.get(). If this returns the empty array, that's the problem you need to fix.
If Cypress is failing when you run all tests on a CI/CD server, check to see if you forgot to remove the example tests.
Intercepting network calls
If your app makes calls to a network service, such as Spotify or IMDB, you may want to avoid those calls during testing, so as to not run up usage charges.
Cypress provides a feature that lets you intercept calls to URLs and return your own data for testing purposes. This intercepts can be defined in each test, or they can be defined and reused in fixture files. See the Cypress page on stubbing network requests.
Emulation
Typically you do not want automated tests making changes to your production database. You certainly don't want to write test data to production. You may want specific fake data to be present to test for special cases. Similarly, when you want to test actions that require an authenticated user, you don't want to put real user credentials in a test script, but you also don't to create a test user that, if someone finds it, can be used to hack your application.
With unit tests, you can mock the units that make the external calls but with Cypress you are interacting with the whole app, not code units. Unfortunately, you can't use the Cypress intercept() approach because Firebase calls don't use URLs to communicate. Firebase creates a web socket connection for more efficient communication.
So another approach to isolate your Cypress tests from your production Firebase data is to run a local Firebase emulator. The emulators behave just like the real Firebase services but are completely local. These are Java programs so you need to have a Java runtime installed. There are Java emulators for all the major Firebase services: both database systems, authentication, even hosting.
See these instructions for how to set up emulators for Firebase services.
Now we need to change the code that sets up Firebase to use the emulators. Suppose our app uses the Realtime Database and Firebase Authentication, and sets things up in firebase.ts like this:
import { getAuth, GoogleAuthProvider, signInWithCredential } from "firebase/auth";
import { getDatabase } from "firebase/database";
const firebaseConfig = {
...
};
const firebase = initializeApp(firebaseConfig)
const auth = getAuth(firebase);
const database = getDatabase(firebase);
...
We want to test if we're running development code or production code.
This can be tested in Vite with
the environment variable import.meta.env.DEV.
So the following Firebase code will use the emulators when we're running code
in development mode:
import { connectAuthEmulator, getAuth, GoogleAuthProvider } from "firebase/auth";
import { getDatabase, connectDatabaseEmulator } from "firebase/database";
const firebaseConfig = {
...
};
const firebase = initializeApp(firebaseConfig)
const auth = getAuth(firebase);
const database = getDatabase(firebase);
if (!globalThis.EMULATION && import.meta.env.DEV) {
connectAuthEmulator(auth, "http://127.0.0.1:9099");
connectDatabaseEmulator(database, "127.0.0.1", 9000);
signInWithCredential(auth, GoogleAuthProvider.credential(
'{"sub": "qEvli4msW0eDz5mSVO6j3W7i8w1k", "email": "tester@gmail.com", "displayName":"Test User", "email_verified": true}'
));
// set flag to avoid connecting twice, e.g., because of an editor hot-reload
globalThis.EMULATION = true;
}
The different Firebase libraries, e.g., authentication, database, and storage, export functions to call to connect to emulators. Each one works a little differently so you have to study the documentation for examples. The code above connects to the realtime database emulator and the authentication emulator and also automatically authenticates a test user. The test user passed to signInWithCredential was created by hand when setting up the test data. The sub field uses the ID generated by Firebase in the user database.
Testing with emulation
When running Cypress with emulators, you have to first start both the app server and the emulators, using the scripts defined above
npm run em:exec ... npm run cy:open
The first line will start the emulators and then start your app. The second line will then start Cypress. These two steps can be automated into one script but the following won't work:
npm run em:exec; npm run cy:open
Starting the emulators and then a React app can take a minute or so. The above will start Cypress before the web server is ready. To fix this, use the npm package start-server-and-test. Install it in your project.
npm install --save-dev start-server-and-test
Then add the following to the scripts in your package.json:
"em:cy:open": "start-server-and-test em:exec http://localhost:5173 cy:open"
Now if you run the task em:cy:open, start-server-and-test willl run em:exec, wait until http://localhost:5173 is active, then run cy:open.
Running Cypress on a CI Server
The instructions given here require human interaction to start the emulators, start the web server, and start Cypress.
To see how to automate Cypress testing on a continuous integration server, see the Github Actions CI Setup page.
Sources
The following resources were used to develop the code above.
- Introduction to Cypress
- What I learned using Cypress for three weeks -- the many tips, such as when to cy.get() vs cy.contains()
- the Cypress page on stubbing network requests
- The Firebase emulator suite - how to install the emulators and use them
- The default ports used by the emulators
- End to end testing with Firebase emulator and Github Actions
- Connect your app to the Authentication Emulator
- Connect your app to the Realtime Database Emulator
- Test if your code is being run by Cypress
- Github discussion on localhost in Node 17,
- Testing production vs development React at run-time
- Cypress documentation on starting web servers
- start-server-and-test