Project Implementation
Implementing projects is often where "technology" seems to fail. Except the technology doesn't fail. It just isn't being implemented properly. "Properly" means that it delivers results for the business, which means that we need to understand first what the business needs and only then design a solution.
And then we need to implement the solution in a sensible flexible way, following an overall plan but not a rigid plan.
It IS possible. If it's not working for you, you need to change your approach. Simpler is better. Straight talk works.
Agile
Agile is a great way of developing features. But it often drives the business crazy, particularly the financial team.
Agile is also a great way of developing code quickly and reacting to new information from the end-user, but it often seems to come with no guarantees of what will be delivered when or for how much.
That is a problem in many contexts. The purity of agile is clearly not enough.
Waterfall
Similarly, waterfall has problems.
A waterfall plan is a great way of efficiently delivering a product or system that isn't what anyone wants any more. If they ever did want it.
While waterfall is nominally efficient, the lack of conversation about the product during the build is often fatal.
Even in very long lead-time projects, it's important to remember that "The plan is there to give structure to the changes".
Hybrid/Sensible
No matter how much people might like it, the business needs some elements of the "certainty" that waterfall is supposed to offer, and the solution development process needs the ongoing interaction that Agile brings.
But the process towards success needs both. We have to accept the inherent uncertainty and work to minimize it, and work to make sure the uncertainties - and their implications - are understood.
It's not a nice intellectually pure framework, but it works. As long as people talk.
Hybrid Agile/Waterfall implementation
A wise man once said "The plan is there to give structure to the changes".
That's the approach we follow. There is a plan, from the beginning, of what we want to achieve. This plan is continually updated as we learn and as we progress. So while we are "agile" in adapting, we're also transparent to the overall business about where we are and what we plan to achieve. This is not a new approach. It has been standard practice in companies that do mega projects for the last several decades. It's only new and "trendy" in the software world.
Yes, the plan will change. That's life. That's the honest way to proceed. If you can't deal with the facts, you'll never have a successful implementation. And if you continually and honestly (and even occasionally robustly) deal with the facts then the project will deliver.