Bugs -- Not just for breakfast
Real world bugs from
The Daily WTF
Types of Testing
- User test: test that something provides user value
- Unit test: test that a unit of code (class, component, function)
does what's required
- End-to-end test: test the system by interacting with
the user interface
- Acceptance test: end-to-end test that represents
a client's definition of when
a user story is done
- Stress test: test that the app can scale (many users, ton of data, ...)
Acceptance Tests
- Client and developers define acceptance tests for each user story
- current iteration stories only!
- Typically a new product will eventually end up with dozens to a hundreds
-
Automate with end-to-end testers, such as
Cypress
Unit Tests
- Written by developers
- Meant to test code units (functions, methods, classes)
- Should be numerous, fast, automated
- Automate with unit testing libraries like
Vitest
and
Jest
Stress Tests
- Test ability to handle many users,
tons of data, hostile attacks, ...
- Automate with tools that simulate multiple simultaneous users, randomized data
Becoming Better Testers
- Start small
- Write an ARE WE UP? test
- Write acceptance tests for core demo paths
- Add tests for every bug that got deployed
-
Do
TDD
when developing code with tricky or "must not fail" logic
Bugs and Tests
- When a bug happens, don't fix it right away.
- Write a unit test that reliably reproduces the bug
- If you can't, you don't know what the bug is
- Now write code to pass the test and fix the bug
Integration bugs imply unit bugs
-
When module A calling module B fails, first
identify which module failed.
- Did A send bad data? Add unit tests on A.
- Did B incorrectly respond to good data? Add units tests on B.
Continuous Integration Servers
- Why?
- Running tests still takes time
- Easy to forget to run them
- Solution: continuous integration server
- A separate machine that runs all tests
- Runs automatically on every commit to main
Why We Don't Test
- Client wants new code now!
- Writing and running tests will delay delivery
- Even more delay if tests fail!
Root causes
- By end of iteration, no time to test well.
- Manual testing is boring.
- It gets worse as the project progresses.
Solution: Automate and Flip
- Automate your testing.
- Even better, write and run tests before you write the code!
-- TDD