Testing time for SOA

Recently, I had been invited in a meeting to assess the problems with the high profile ‘SOA’ project. The common business services, which are supposed to be reused across multiple applications, were not performing up to the mark. More than 80% of the problems were related to integration and the performance. The application development teams have already missed deadlines multiple times and some are threatening not to use common services, but instead build their own bridges to the legacy systems.

The common services have been ready for long time; however, the problems are discovered only at the time of integration with applications. This late discovery of issues has put many projects into jeopardy and people are cursing the day when they thought of implementing SOA.
Now, why this happened? Some of the problems could have been tackled at the architectural level, however, I felt, this could have been avoided if only somebody had planned testing at the service level. It is not sufficient just to buy a SOA testing tool. The SOA testing need much more attention and detailed planning.

Here are some of the considerations for your SOA test planning -

1) Context orientation – It is not just enough to send an input to the web service and test whether the outcome is received and compare variables. It is important to know the context in which the service will be called and what the output variables those are more likely to be used. There should be a separate test case for each possible scenario in which the service will be called. Many a times, the output payloads are not fully populated and testers are often told that these variables are not meant to be populated. It would be good idea to verify the assumption against the context.

2) Reduce complexity of testing – The SOA testing is complicated enough. It is not easy to test without a user interface. Typically, it is very challenging to generate test data in the form of input XML. Sometimes, the output XMLs run into MBs and it is impossible to compare them with the ‘expected’ result and find out problems with them. Invest into tools to simplify the invocation of the service and output comparisons. Try to use more user friendly interface such as MS Excel to create test cases and test data, and build a framework around it to take these excels as an input for SOA testing tool.

Remember- If it is not simple to do, it will not happen.

3) Test the service choreography – Most of the times the services are called sequentially from the controllers, however, at the time of testing, those are tested stand-alone with certain assumptions about input and output. It would be good to identify the sequence in which the services would be called and connect their outputs/inputs and simulate the real-life scenarios. Invest into building the tool that can help building up such choreography for testing.