Testing Overview

• These slides
• The Agile Samurai: Chapters 12, 14, 15
• Fowler: Mocks aren't stubs: https://martinfowler.com/articles/mocksArentStubs.html

Can you break this calculator? It takes a date in the form YYYY-MM-DD after 1980-01-01, calculates the number of days since 1980-01-01 and then calculates the year from those days. Focus on breaking the calculation, not the date validation.

• Why?
  • Running tests still takes time
  • Easy to forget to run them
• Solution: continuous integration server
  • A separate machine that runs all tests
  • Runs automatically on every commit to master

• User tests: detect what is and isn't working with users
• Acceptance tests: define just-in-time requirements for each user story
• Unit tests: define and verify intended behavior of every unit of code
• End-to-end (E2E) tests: detect inter-module communication failure
• Stress tests: detect scale-up issues

• Client and developers define acceptance tests for each user story
  • current iteration stories only!
• Typically a new product will eventually end up with dozens to a hundreds
• Many tools exist to make these more readable by clients
  • Cucumber: https://cucumber.io/
  • Fitnesse: http://fitnesse.org/

• Written by developers
• Meant to test code units (functions, methods, classes)
• Should be numerous, fast, automated
• Typically call code functions directly, without starting any server or native application

• Written by developers
• Test interactions between modules and services
• Typically start a server or native app and interact with its user interface
• Slower than unit tests

• If an end-to-test failed when component A called component B then either
  • A sent bad data to B, or A failed to handle a valid response from B → Add unit tests on A
  • B failed to handle valid data from A or B returned bad data to A → Add unit tests on B

• Test ability to handle many users, tons of data, hostile attacks, ...
• Many tools to simulate users, generate fake data
  • e.g., Apache JMeter, Gatling

• Start small
  • Write an ARE WE UP? test
  • Write happy path tests
• Add tests for every bug that got deployed

• When a bug happens, don't fix it right away
• Write a unit test that reliably reproduces the bug
  • If you can't, you don't know what the bug is
• Now write code to pass the test and fix the bug

• Why?
  • UI changes break UI tests
  • Database structure changes break unit tests
  • Live data changes break integration tests
• Solutions:
  • Use separate test IDs for unit tests
  • Only test public methods
  • Never test on production data

Thanks to Hakim El Hattab for RevealJS