Continuous Delivery Digest: Ch.9 Testing Non-Functional Requirements

Digest of chapter 9 of the Continuous Delivery bible by Humble and Farley. See also the digest of ch 8: Automated Acceptance Testing.

(“cross-functional” might be better as they too are crucial for functionality)

  • f.ex. security, usability, maintainability, auditability, configurability but especially capacity, throughput, performance
  • performance = time to process 1 transaction (tx); throughput = #tx/a period; capacity = max throughput we can handle under a given load while maintaining acceptable response times.
  • NFRs determine the architecture so we must define them early; all (ops, devs, testers, customer) should meet to estimate their impact (it costs to increase any of them and often they go contrary to each other, e.g. security and performance)
  • das appropriate, you can either create specific stories for NFRs (e.g. capacity; => easier to prioritize explicitely) or/and add them as requirements to feature stories
  • if poorly analyzed then they constrain thinking, lead to overdesign and inappropriate optimization
  • only ever optimize based on facts, i.e. realistic measurements (if you don’t know it: developers are really terrible at guessing the source of performance problems)

A strategy to address capacity problems:

Continue reading

Why not release earlier?

A post about how to overcome the terror of feedback, and why it is an important step towards continuous delivery 


I had a conversation yesterday with a team that was putting out version zero of a product. “Yes, yes. Release early, release often. We absolutely agree.” I thought the conversation was going well. “Of course, we need [insert completely useless feature here] before first release.” Dang. Again?

I have had this discussion, with others and myself, a hundred times. What is going on? Why do we repeatedly agree that feedback is good and then do really crazy things to avoid feedback?

Having watched myself elevate unimportant features to the status of must-haves many times, I have decided that I am at war with myself. On the one hand I want feedback. Of course I want feedback. Feedback makes me better and smarter. Feedback makes my projects more successful. Even if the feedback is “cancel the project”, that just means I have more time for other potentially-useful projects.

On the other hand I hate feedback. To work on a project I become emotionally attached to it (too much). I believe it will be successful. I care that it will be successful. I am afraid that if I lack passion, I won’t be able to work. From this perspective, feedback is a threat, not just to my project, but to me.

Psychologists talk about boundaries, having a good sense of what is me and what is not me. When I manufacture enthusiasm for a project, I blur my boundaries. I begin to think, even though it isn’t objectively true, that my project *is* me. Feedback at that point becomes a personal, existential threat, and my brain is really good a performing rhythmic gymnastic contortions when faced with a personal, existential threat. Like believing, really believing, that automated deployment infrastructure is absolutely vital for the first release.

When I catch myself thinking this way I use a few techniques to return towards sanity:

  • Ask for feedback. Friends don’t feel an existential threat, so they remain saner. “Here are all the things I have to do before first release. Anything on this list optional?”
  • Remind myself that my project is not a part of me. Over and over. I figure if I repeat it ~1e6 times I may begin believing it.
  • “What’s the worst that could happen?” There’s a sequence to my thinking. It’s quick, but if I am aware I can catch the steps: 1) think about getting feedback, 2) imagine failure, 3) terror, 4) invent new must-have feature. If I can intervene at 2), I ask myself, “What’s the worst that could happen?” The project is cancelled. My reputation is ruined. I never program again. My whole family starves to death. I go to hell forever. Okay, so the consequences won’t be that bad really. What’s likely to happen? Change the project direction. Oh, that’s not so bad.

I’m not sure everyone reacts this badly to the prospect of feedback, but from the conversations I have I suspect that many people do.

What’s your “terror of feedback” story?