Software Engineering at Google Chapter #11 - Testing Overview (3 of 3)

  • Large tests are more about validating configurations and not pieces of code as well as legacy systems that can't be doubled/mocked
  • Consider only running the "large tests" suite at the end before you release, not each time you test
  • Google says once your tests get 1% flakey, people start to lose confidence
  • Tests should assume as little as possible about the outside environment, including the order that the tests themselves are run
  • Tests should ONLY contain the info needed for the test and nothing more (clean code, clean tests)
  • A test should also be "obvious upon examination"
  • Google discourages the use of control flow statments such as loops and conditionals
  • Unit tests are small scoped tests
  • Integration tests are medium scoped tests
  • End-to-end or functional tests are large scoped tests
  • "Test doubles" is the collective term for fakes, mocks, and similar
  • About 80% of tests should be narrow scoped, 15% medium, 5% large
  • Smaller tests = less cognative load
  • Favor unit tests early to build developer confidence and add end-to-end tests to build product confidence later
  • What should you test? Test everything that you don't want to break.
  • Do your tests address how your system handles failure(s)?
  • "A predictable and controlled response to adverse conditions is a hallmark of a reliable system"
  • Code coverage is the percentage of the code that is covered by tests (if 80 out of 100 lines are tested then it is 80% covered)
  • Only count coverage from small tests as large tests tend to inflate coverage metrics
  • Think about the behaviors that you test for. What other things will your customers do? Write tests for that
  • Google uses a monorepo (single repo for all code) and no branching. It works for them.
  • "Brittle tests" is when small code changes result in many new test failures
  • The number one cause of brittle tests at Google is misuse of mock objects
  • The larger a test suite, the slower it runs, the less value it provides
  • Actively poll for state transition instead of using sleeps
  • Treat tests like production code (create incentives to write and maintain good tests)
  • Reasons tests become slow: Waiting for systems to sync, processing large data sets, or starting the environment (starting the VM or emulator, etc)
  • Google pushed testing culture with new hires via orientation, thus building the culture over time
  • Google also created 5 categories of testing maturity, applied them to each team, and worked to improve them
  • Put posters in toilets to make sure people see it daily (Testing on the Toilet)
  • Finding security issues via testing is best done by humans, not automation



Thank you for your time and attention.
Apply what you've learned here.
Enjoy it all.