Continuous Delivery Digest: Ch.9 Testing Non-Functional Requirements

Digest of chapter 9 of the Continuous Delivery bible by Humble and Farley. See also the digest of ch 8: Automated Acceptance Testing.

(“cross-functional” might be better as they too are crucial for functionality)

  • f.ex. security, usability, maintainability, auditability, configurability but especially capacity, throughput, performance
  • performance = time to process 1 transaction (tx); throughput = #tx/a period; capacity = max throughput we can handle under a given load while maintaining acceptable response times.
  • NFRs determine the architecture so we must define them early; all (ops, devs, testers, customer) should meet to estimate their impact (it costs to increase any of them and often they go contrary to each other, e.g. security and performance)
  • das appropriate, you can either create specific stories for NFRs (e.g. capacity; => easier to prioritize explicitely) or/and add them as requirements to feature stories
  • if poorly analyzed then they constrain thinking, lead to overdesign and inappropriate optimization
  • only ever optimize based on facts, i.e. realistic measurements (if you don’t know it: developers are really terrible at guessing the source of performance problems)

A strategy to address capacity problems:

Continue reading