Monday, February 1, 2010

Agile adaptation of Architecture Evaluation Methodologies

I discussed some of the architecture evaluation methods in my previous post.  In short, the idea is - Rather than just using a checklist for technical evaluation, we should test drive each functional scenario against architectural decisions and judge the impact on quality attributes (utility tree) and evaluate the architecture. Such a review will be more specific to the project and hence, more beneficial.
Many accept the advantages of these methods, however, often criticize these methods for their overhead. I think, these methodologies can certainly be adapted for agile use. We do not have to stick to the the elaborate process that SEI has recommended, but instead, we should adapt it for our use by sticking only to its principles. In fact, in my experience, these methodologies are more beneficial if we use them in conjunction with agile development methodologies.

Why Architecture evaluation is important for Agile methodologies?
In Agile world, the application functionality is to be built in agile way, with constant visibility to business, and frequent changes to the features. However, the same is not the case for its architecture. If the architecture is allowed to evolve with changing requirements, it causes frequent rework, constant re-factoring and in fact, it counters the ‘Agile’ response. It makes sense to spend upfront time on architecture evaluation and ensure that it is flexible to accommodate the future changes.

What are the steps in evaluation?
I assume that you have background of standard evaluation methods such as ATAM / CBAM. If not, refer my last post. Though SEI has defined a very elaborate process, I think, only the following are critical steps –
Step 1 – Prioritize functional scenarios along with all stakeholders and identify architectural approaches and alternatives
Step 2 – Generate Quality Attribute Utility Tree and specify Stimuli – Response for each scenario
Step 3 – Analyze architecture approaches and identify all possible
a)      Risks,
b)      Non-Risks,
c)       Sensitivity Points ( interdependencies) and
d)      Trade-Off Points.
Step 4 – Quantify the Benefits of different architectural strategies and corresponding Cost and Schedule implications
Step 5 – Calculate desirability (benefit divided by cost) and Rank the alternatives.
Step 6 – Make decisions and document

What about the overhead of evaluation? How can it be done in Agile Way?
To make these methods more agile, use following techniques -

  1. Evaluate on Sampling basis –Short-list only the unique and critical scenarios that are likely to change.
  2. Create the artifacts such as Utility Tree as part of Architecture development process and not just for evaluation. The act of creating the utility tree will improve the thought process while defining the architecture.
  3. Evaluate everyday as soon as you discuss the scenarios in the war room. All the stakeholders are present in the war room. This avoids extensive planning, presentations, and elaborate evaluation exercises.
  4. During the evaluation, apply the utility tree to each scenario and evaluate the Sensitivity Points, Risks /Non-risks points, and Trade-off points. In case of alternative solutions, estimate Cost and perform Cost-Benefit analysis.
  5. Evaluate not only for the defined scenarios, but also for the possible changes. Get the ‘change’ scenarios from the business to evaluate the impact of changes. The architecture should be flexible to accommodate such changes.

- Amit Unde


  1. Indeed a very valid point.

    Architecture focus plays even important role in Agile development, though many people don't agree to it.

    When we are working on small sprint/iterations in timebox manner, it is quite easy to loose focus on the big picture.

    Most of the experts suggest that architecture evaluation should happen at early stage but my personnel experience suggest that it makes more sense to do it iteratively, as you get more insight into your architecture behavior the exercise becomes more concrete rather than a theoritical scenario play.

    The utility tree should be a live document updated with every incremental change as we have often seen that priorities changes with time.

    The only challenge I see is creating a real business case to have this activity running along with small sprints and providing measurable outcome of the effort spent.

    Would really like to hear from your experience around how did you overcome this challenge.


  2. The business case is required if you are proposing 'evaluation' as distinct phase. It need not be distinct activity. It should be intrinsic part of 'Architecture' development process. Most of the artifacts get created during architecture phase and 'evaluation' part is a light-weight, and clearly very productive.

  3. Many institutions limit access to their online information. Making this information available will be an asset to all.