Software Engineering at Google Chapter #21 - Dependency Management (1 of 3)

  • Dependency management is the management of the networks of packages, dependencies, and libraries that we don't control
  • For example, what happens if your build process pulls the necessary libraries from an external source and doesn't specify the library version?
    • You will always pull the newest version
    • The newest version may or may not work with your software
    • Each time you build you may get a different version of the library thus your builds aren't consistent
    • Will the newest version introduce security issues?
  • If you have multiple teams working on multiple dependent repos within your organization then your teams will need to deal with dependency management in the form of "internal dependencies" because on team's changes to their repo can break dependent code in another repo
  • Google minimizes the above dependency issues with their "monorepo" (single repo) pattern. Everyone is working off the same, single repo thus they do not need to consider how repo A depends on repo B; it's all in the same single source code tree
  • "All else being equal, prefer source control problems over dependency management problems"
  • Dependency management issues are much harder to detect, track, fix, and reason about than are source control problems
  • Projects are no longer islands - dependency management applies to almost all software projects big and small, internal and external
  • Due to the deepness of software stacks dependency management is one of the most important issues in software development today
  • The issue of dependency management should be thought of in large terms: How do we manage a network of dependencies over time (vs how to get a single version of a single package so that our software will build)
  • Over time all nodes in a dependency graph (or dependency network) will change and that is why we must think about and plan out dependency management
  • The core problem in dependency management is the question: "What if two nodes in the dependency network have conflicting requirements but your organization depends on them both?"
  • The diamond dependency problem is one where your software depends on library A and library B and both of those libraries depend on a third library (library C)
    • What happens when the team that maintains library C makes a breaking change?
    • How quickly will library A & B be updated to v2?
    • What happens if library A depends on version 1 of library C but library B version 2 of library C?
    • There are ways around this issue. For example you can embed both library A and library B into the binary. Some programming languages tolerate this (Java) some will not (C++)
    • You can also use tricks like renaming functions, but there is no fix when incompatible data types are passed from one library to another
  • The policy and technology of dependency management essentially boils down to the question: "How do we avoid conflicting requirements while still allowing change among non-coordinating (independent) groups?"
  • Don't re-invent the wheel. If an external source can provide your dependency then use it (eg: JSON parser or regex parser)
  • Consider the on-going maintenance costs of external dependencies. Platforms will change, security issues will be found, or a change in the dependency network will force an upgrade at some point in the future
  • Dependency providers (the people and organizations who create and maintain packages that others depend on) should consider...
    • How are breaking changes handled?
    • How much change is assumed?
    • How much evolution is assumed?
    • How long are releases supported for?
    • By answering these questions it helps the users determine if the dependency and it's ongoing maintenance costs are right for the project
  • ABI compatibility is "application binary compatibility". This means a binary built with version 2 will continue to run on version 3, version 4, and so on. Linux, Java, and C++ provide ABI compatibility.
  • ABI compatibility is different from source compatibility. Source compatibility means one cannot build a library using version 1 and then link to that library with a version 2 binary. If the library is rebuilt (from the same source code) using version 2 then the version 2 binary can link to the new version 2 library.



Thank you for your time and attention.
Apply what you've learned here.
Enjoy it all.