Agile adaptation of Architecture Evaluation Methodologies

I discussed some of the architecture evaluation methods in my previous post.  In short, the idea is - Rather than just using a checklist for technical evaluation, we should test drive each functional scenario against architectural decisions and judge the impact on quality attributes (utility tree) and evaluate the architecture. Such a review will be more specific to the project and hence, more beneficial.
Many accept the advantages of these methods, however, often criticize these methods for their overhead. I think, these methodologies can certainly be adapted for agile use. We do not have to stick to the the elaborate process that SEI has recommended, but instead, we should adapt it for our use by sticking only to its principles. In fact, in my experience, these methodologies are more beneficial if we use them in conjunction with agile development methodologies.

Why Architecture evaluation is important for Agile methodologies?
In Agile world, the application functionality is to be built in agile way, with constant visibility to business, and frequent changes to the features. However, the same is not the case for its architecture. If the architecture is allowed to evolve with changing requirements, it causes frequent rework, constant re-factoring and in fact, it counters the ‘Agile’ response. It makes sense to spend upfront time on architecture evaluation and ensure that it is flexible to accommodate the future changes.

What are the steps in evaluation?

I assume that you have background of standard evaluation methods such as ATAM / CBAM. If not, refer my last post. Though SEI has defined a very elaborate process, I think, only the following are critical steps –

Step 1 – Prioritize functional scenarios along with all stakeholders and identify architectural approaches and alternatives

Step 2 – Generate Quality Attribute Utility Tree and specify Stimuli – Response for each scenario

Step 3 – Analyze architecture approaches and identify all possible

a)      Risks,

b)      Non-Risks,

c)       Sensitivity Points ( interdependencies) and

d)      Trade-Off Points.

Step 4 – Quantify the Benefits of different architectural strategies and corresponding Cost and Schedule implications

Step 5 – Calculate desirability (benefit divided by cost) and Rank the alternatives.

Step 6 – Make decisions and document

What about the overhead of evaluation? How can it be done in Agile Way?

To make these methods more agile, use following techniques -

1. Evaluate on Sampling basis –Short-list only the unique and critical scenarios that are likely to change.
2. Create the artifacts such as Utility Tree as part of Architecture development process and not just for evaluation. The act of creating the utility tree will improve the thought process while defining the architecture.
3. Evaluate everyday as soon as you discuss the scenarios in the war room. All the stakeholders are present in the war room. This avoids extensive planning, presentations, and elaborate evaluation exercises.
4. During the evaluation, apply the utility tree to each scenario and evaluate the Sensitivity Points, Risks /Non-risks points, and Trade-off points. In case of alternative solutions, estimate Cost and perform Cost-Benefit analysis.
5. Evaluate not only for the defined scenarios, but also for the possible changes. Get the ‘change’ scenarios from the business to evaluate the impact of changes. The architecture should be flexible to accommodate such changes.

- Amit Unde

Test Drive your Architectures for effective evaluations

Are you one of those sitting in ivory towers of Enterprise Architecture Group, tasked with responsibility of evaluating project architectures and ensuring IT Governance? If you have done few reviews before, you would know that it is highly subjective process that depends largely on the evaluators’ technical skills, functional/environmental knowledge, authority structure, and other political forces. The evaluators who are parachuted into the project group for reviews are especially handicapped due to lack of knowledge of functional requirements, and often only concentrate on ‘technology’ implementation reviews. Hence, most of the reviews remain superficial and only partially beneficial.

So how exactly should you evaluate the architectures? 

I would say do what you do when you buy a car. Test Drive! You will only know the car’s performance when you actually drive it and feel it, and not just by reading specifications or by asking questions. Similarly, to evaluate the architecture, apply it to the specific functional scenarios and measure the quality attributes.

Carnegie Mellon Institute (SEI) has developed the architecture evaluation methodologies on the same principle. The most known methods are –

1)   ATAM (Architecture Trade-off Analysis Method) –

This methodology analyzes how well the software architecture satisfies the quality attributes (e.g. scalability, modifiability, performance etc), by applying the architecture to short-listed functional scenarios. It prescribes developing a quality attribute utility tree, and analyzing it for each scenario and alternative architecture approaches. For each scenario, the method prescribes identifying the following –

a)    Sensitivity points – a collection of components that are critical for achieving a quality attribute,

b)    Trade-off points – a sensitivity point that affects multiple quality attributes, typically trades one off for the other,

c)    Risks – something that inhibits the system from achieving its quality goal (this also includes the decisions that are not taken),

d)    Non-risks – something that is done right (and should not be changed)

2)   CBAM (Cost Benefit Analysis Method)

CBAM begins where ATAM leaves off. It prescribes analyzing the cost, benefits and schedule implications as well for architectural approaches before making the final decisions.

3)   ARID ( Active Reviews for Intermediate Design)

This is more of a design review than architecture review, but uses the same principle of applying design to scenarios, and even writing a pseudo code to evaluate different parts of design.

The SEI has documented a very formal step by step process for all these methods. One way that may be counter productive as people tend to focus on activities, rather than the principles behind these methods. There is a great scope to tailor these methods to suit your organization, and conduct such evaluation in agile way.

More on this topic later…

- Amit Unde