Dr. Royce and Waterfall
In 1970, Dr. Winston Royce wrote the paper on Waterfall that defined it.
He identified 5 things that must be added to reduce most of the risks of doing waterfall. And I will comment on how Scrum addresses these.
1. Program Design Comes First
Well, I think he may have been mostly right. But I would rather say that ‘solution design’ comes first. Meaning that we must truly try to understand the customer’s real problem (not what he says it is) and try to design a full solution to that problem. Not just, for example, software, but the full solution. Assuming the product is (mainly) software, we better understand where our product plays in the fuller solution.
Scrum does not address this issue directly. Implicitly Scrum is saying, “it is hard to know whether your product will be good, so, build incrementally and try to enhance your feedback, to prove to yourself that the product will suport the solution.”
Let me add this: Some people who are doing ‘cowboy agile’ in fact do not do enough up-front thinking, in my experience. Yes, it is fair to say that we learn more from working software than by just ‘thinking’. But some don’t ‘look before they leap’. This is a hard thing to discuss in the abstract, because how much to think up-front will vary, depending on many details in your specific situation (for example, the experience level of the implementer).
I have also seen Scrum teams get into analysis paralysis, where they are thinking too much up-front. In my experience, it is well worth the team’s time to discuss continually, “well, how much should we think up-front about this piece?” They will get better at making intuitive judgments about this, that are appropriate, for their specific work.
2. Document the Design
Agile and Scrum do not have Dr. Royce’s level of confidence in documentation.
Agile and Scrum see, or assume a greater urgency to meet ‘time to market’ demands. Typically a high focus on documentation leads to long analysis paralysis periods.
Agile and Scrum do agree that knowledge must be shared and saved. But rather than share knowledge mainly through documentation, the community now suggests doing it many other ways. Here are two: (1) incrementally build working product, and (2) lightly documented automated tests (where at all possible). Agile and Scrum people tend to prefer cards (middle-level summaries), white boards, large sheets with drawings in team rooms, etc.
So, information is hopefully shared more, and the sharing is actually better.
Speaking for myself, I do agree with Dr Royce’s statement that many implementers tend not to write even basic documentation. And I agree with his statement that much of the purpose of documentation is for use by people later. So, those needs must be carefully considered and addressed.
This results, if done professionally, in two things:
1. We in agile write much less documentation than at least I used to, when doing ‘waterfall’.
2. We ask many people in the Team to write more documentation (in a wiki or somewhere) than they want to write.
3. Do It Twice
Yes, Dr. Royce said “build it once, throw it away, and then build the real thing”. His son later complained that no one really did this, so no one was really doing waterfall the right way…
The basic problem is that we are always learning, even in the last stages of the work. And that learning needs to be fed back into the final product. This was his recommendation for doing that in waterfall. No wonder the waterfall products are seldom what the customers really need.
Scrum solves this differently, by continually refactoring the current product to align with the latest knowledge, giving us ‘working product’ every sprint.
This means, yes, Virginia, there is ‘re-work.’ But probably not any more re-work than Dr. Royce proposed. And the Team should be asking: ‘How could we have avoided some of this re-work?’ Sometimes they will come upon ideas to get better next time.
This is not suggesting or expecting to arrive at a state of ‘perfect knowledge up-front’. This will never happen. Still, we can learn to consider some things before we leap. Example: Is our design for this story consistent with the rest of the design and also appropriate versus our complete and current understanding of what the overall solution needs?
4. Plan, Control, and Monitor Testing
Dr. Royce has some good ideas about testing. Some of them have been superseded and some new things have come onto the scene as well.
The key thing to say, though, is that Scrum agrees with the importance of testing. Scrum says: let’s do it from the very beginning (well, at least from the first Sprint). In part, to tighten the feedback loop and to reduce the cost of implementing the changes we get from the feedback.
5. Involve the Customer
Yes! Very important. Again, Scrum considers this such an important idea that we do it in the form of the Product Owner, who is an essential daily part of Scrum. In my opinion, the PO role idea should get ‘the customer’ involved in a much more practical and useful way that the ideas that Dr. Royce proposes for waterfall.
Much more to say, but let me keep this post short.
Early in the paper Dr. Royce says: “I believe in this concept [waterfall], but the implementation described above is risky and invites failure.” I don’t know, but I think he meant “Compared to cowboy coding [no process], waterfall, with the 5 ‘additions’ that I will propose, seems a lot better.”
Now let me quote Dr. Royce’s last lines about waterfall:
“[This] summarizes the five steps that I feel necessary to transform a risky development process into one that will provide the desired product. I would emphasize that each item costs some additional sum of money. If the relatively simpler process without the five complexities described here would work successfully, then of course the additional money is not well spent. In my experience,the simpler method has never worked on large software development efforts and the costs to recover far exceeded those required to finance the five-step process listed.” [emphasis added]