Greetings. In my last post - part two of a three-part series on Agile Testing - I talked about the importance of test automation in Agile (Agile Testing: what does it take? Pt. 2). Before finishing that thread, I would like to expand on this topic a bit more. You see, today is a milestone for me. We are releasing a significant update to my product line - a suite of test automation products - and I think it is a great moment to share with you what makes me so excited about the work I do. First of all, I consider myself fortunate to work with a great team. Collectively, we are responsible for defining the strategy of Borland's Lifecycle Quality Management (LQM) products. In the last couple of years the general direction of these products has been greatly influenced by Borland's own transition from a traditional, waterfall-based organization to one that lives and breathes Agile.
During our transition we not only realized that we needed to change many of the long- established processes and cultural norms, we also realized that we needed to enhance our products in order to better support Agile development scenarios in addition to continuing to improve our support for the core needs of all QA teams. Also, as an ISV we obviously have the same need for quality as any of our customers and, consequently, we not only use our own products to test our software but leverage our many years of experience in the testing domain to continuously improve our products.
As you may have seen, Borland today announced the latest versions of the Silk products which represent a significant step in supporting 21st century testing. The new releases not only provide true test automation for state-of-the-art applications, but they now offer some really exciting enhancements to better support Agile teams. Now I can already hear many of you moan about how I am trying to sell the Borland products. But that part of the post is done. The link above will let you take a closer look at the products if you are interested. My real intention for this post is to share the experiences that were the genesis for these products and through that sharing, hopefully, show you why am so excited about them.
There is no hiding in Agile
As we adopted more and more Agile practices in the Linz office, we noticed some issues surfacing and then becoming more and more pressing. Amongst these was our need for better support of Agile workflows in our Silk products. For example, we quickly learned that Agile teams work in a pull model as opposed to the traditional method of assigning work items. This was especially relevant from a test management perspective. As a consequence of our own teams' experiences, we realized we needed to provide much more collaborative views that allowed the team members to see what was going on so that they could select testing-related tasks from a list of available work items that had not yet been picked up by another team member, or have easy and quick access to related information. These requirements led to a number of enhancements in SilkCentral Test Manager such as Agile project templates, a new Web-based manual testing interface and an integration to Version One to support additional Agile test planning tools. These enhancements quickly proved to be an enormous help in our teams' daily work process by supporting self-organization, providing the necessary visibility into the many activities, and making collaboration much easier and efficient.
Another important need we quickly identified was the ability to leverage existing tests as well as the ability to easily integrate new tests, e.g. unit tests. It quickly became clear that it was a major hurdle for the Agile teams to have to maintain test plans both externally (for example as part of a unit testing framework) as well as in the test management tool. This gave us the idea for external test package support, which now allows users to just point to the external test plan and execute all the tests that are part of it. Gone are the days where we needed to manually create test stubs in the test management tool and add the necessary details in a subsequent manual step which, of course, is cumbersome when dealing with a large number of tests. Now it is as simple as pointing to a suite of existing unit or FitNesse test plans and executing them directly from SilkCentral Test Manager. And voila! All your results get automatically pulled in!
Finally, probably the most important requirement we uncovered, was the need to automate increasingly more tests. Anyone who has built test automation frameworks in the past knows that this is a very challenging task. The brittleness of test scripts combined with the potentially fast- changing GUI of an application are just two of the factors that have made the creation of a robust test automation framework extremely difficult. Our teams quickly realized they needed to do something about it.
Addressing the challenges
We immediately identified two closely related areas of improvement that we would have to address. First of all, from the perspective of a tester we looked for ways to improve the testability of our applications. Introducing unique identifiers for each of the objects in the products we develop was a significant step to help improve testability of the applications as well as the stability of test scripts. This also quickly lead to an improved understanding from the product development point of view about how we needed to enhance our test tools' capabilities to best leverage these kinds of testability enhancements of an application. In order to be able to build solid test scripts we needed to have stable and consistent object recognition. So the first obvious improvement we set out to make to the Silk suite was to ensure that the products provided a way to consistently find simple locators which were unique for each control - the same for each call, the same for each build, and the same for each instantiation of the application. Ideally, they would be readable and follow standard naming conventions well. With that in mind we enhanced all of our products so that they would intelligently optimize their object recognition strategies accordingly for testability.
Another major challenge our teams faced was around testing AJAX and other Web 2.0 applications. Due to their asynchronous nature they cannot be tested reliably with a traditional serialized test logic approach. It became obvious that synchronization needed to be done in the test tool rather than at the script level. Consequently, we enhanced SilkTest with everything that was needed to remove the necessity for the tester to try and mimic the (nondeterministic) nature of asynchronous applications allowing for much leaner test script code.
Finally, with all of our teams moving to two week iterations, speed became of paramount importance. The teams needed to be able to run their ever-increasing regression sets in much smaller time frames. There was an increased need to test more - and more often. We quickly realized that we needed to directly leverage RIA technologies to speed up replay of test scripts instead of leveraging an on-the-surface approach like using Javascript engine-injection for browsers. This approach, used by many other testing tools, just didn't work for our teams. The security restrictions, incredibly slow replay speed, and inability to handle actions that happen outside the browser's DOM (like Windows message boxes that come up) are just some of the problems we ran into with other tools. So what do you do if you are a test tool vendor? You build what you need!
These are just a couple of the challenges that our teams experienced that influenced the direction of the product line. One of the key advantages you have when you get immediate feedback through internal usage, and then have the opportunity to and refine and improve products on a sprint-by-sprint basis is that customers (and we are one of our customers) reap the benefits very quickly.
Bye for now.
Comments