Software Engineering At Google
Chapter #11 - Testing Overview (3 of 3)
Software Engineering at Google Chapter #11 - Testing Overview (3 of 3)
Large tests are more about validating configurations and not pieces of code as well as legacy systems that can't be doubled/mocked
Consider only running the "large tests" suite at the end before you release, not each time you test
Google says once your tests get 1% flakey, people start to lose confidence
Tests should assume as little as possible about the outside environment, including the order that the tests themselves are run
Tests should ONLY contain the info needed for the test and nothing more (clean code, clean tests)
A test should also be "obvious upon examination"
Google discourages the use of control flow statments such as loops and conditionals
Unit tests are small scoped tests
Integration tests are medium scoped tests
End-to-end or functional tests are large scoped tests
"Test doubles" is the collective term for fakes, mocks, and similar
About 80% of tests should be narrow scoped, 15% medium, 5% large
Smaller tests = less cognative load
Favor unit tests early to build developer confidence and add end-to-end tests to build product confidence later
What should you test? Test everything that you don't want to break.
Do your tests address how your system handles failure(s)?
"A predictable and controlled response to adverse conditions is a hallmark of a reliable system"
Code coverage is the percentage of the code that is covered by tests (if 80 out of 100 lines are tested then it is 80% covered)
Only count coverage from small tests as large tests tend to inflate coverage metrics
Think about the behaviors that you test for. What other things will your customers do? Write tests for that
Google uses a monorepo (single repo for all code) and no branching. It works for them.
"Brittle tests" is when small code changes result in many new test failures
The number one cause of brittle tests at Google is misuse of mock objects
The larger a test suite, the slower it runs, the less value it provides
Actively poll for state transition instead of using sleeps
Treat tests like production code (create incentives to write and maintain good tests)
Reasons tests become slow: Waiting for systems to sync, processing large data sets, or starting the environment (starting the VM or emulator, etc)
Google pushed testing culture with new hires via orientation, thus building the culture over time
Google also created 5 categories of testing maturity, applied them to each team, and worked to improve them
Put posters in toilets to make sure people see it daily (Testing on the Toilet)
Finding security issues via testing is best done by humans, not automation
Thank you for your time and attention.
Apply what you've learned here.
Enjoy it all.
© 2021 Josh Turgasen
All product names, logos, and trademarks are property of their respective owners