I cannot disagree about the limitations of the “waterfall” approach to project planning, budgeting, and execution and I’m a strong believer in the principles embodied in the Agile Manifesto. But, in my 40+ years of developing commercial software I’ve watched innumerable fads come and go. I’ve even fallen for a few of them.
If I’ve learned one lesson, it’s that software and software development can be complicated and that effectively managing software development requires skill, experience, knowledge, adaptability, and flexibility. There are no panaceas and no manage “by the numbers” solutions that can make running development foolproof or easy. Good software project management requires exceptional human and technical management skills — from one individual with both — or a pair of individuals who combine the requisite skills.
I’ve watched Scrum in action at a number of companies and I’m convinced that, without the presence of the requisite technical and human management skills, it is doomed to be a hamster wheel for developers and and an inefficient producer of mediocre software. In most corporate implementations Scrum is used to trivialize many of the important attributes of of software — most often critical elements such as architecture and security. Scrum is a fine tool for implementing Web sites, for which it was initially evolved, because the underlying architecture and security are provided by the server and DBMS software. It is not the answer for large integrated hybrid cloud systems, if not coupled with skilled and capable managers.