Test Automation Tools and Rapidly Changing User Interfaces

22. July 2012 · 4 comments · Categories: Functional Testing, Scripting, Software Testing

You started using test automation tools early in the development cycle. You started to use these test automation tools to create scripts that worked well and automated a significant portion of the user interface. A few weeks later several of those test automation scripts failed. The user interface had changed and you had to spend a significant amount of time creating new test automation scripts. A few weeks later some other test automation scripts failed for the same reason. You wasted a countless number of hours working on this issue .and ending up completely frustrated by it.

The scenario described above is a well known issue faced during test automation. How does this occur? Let’s consider a web page that is described by the HTML below.

<html>
<body>
<form action=”done.aspx”>
<input id=”submit” type=”submit” value=”Submit” />
</form>
</body>
</html>

Consider a scenario where the user loads the page above and then clicks the Submit button. When you record a test automation script against the page above, many test automation tool vendors will create a script that will extract the “id” of the INPUT tag and utilize that to identify the object at runtime. This is all well and good until the “id” changes! What happens then? The script fails. When we started building a test automation tool, we had faced this problem ourselves in our past lives. As a result we designed our test automation tools to handle this issue to the extent possible. In the previous example, our test automation tool would not identify the INPUT tag above based on simply one attribute. Instead we created our own object recognition system that uses a large number of factors to identify an object. The reason is that if something simple changes e.g. one of the element’s attributes, its position in the web page etc, the test automation script will still work. What happens if the page changes to the point that it is no longer recognizable? I’ll be honest with you that in that case nobody can do much. You will have to rerecord the script. However, most user interface changes are not as drastic. Most of them are smaller changes that cause the affected page to naturally evolve over the development cycle. Many of these cases can be handled by a test automation tool that is built with this concern in mind.

There are several other ways to avoid the issues associated with rapidly changing UI. One very effective way is to keep developers involved in the test automation process and provide them with the test automation tools and your scripts so that they can themselves run the tests from time to time. This will enable them to appreciate your problems and most of them will try to work to try to avoid these problems. It will also allow the developers to discover their own defects before they reach you. Another way to avoid this issue is to spend more time in design mode as this will minimize large user interface changes during the project.

You need to use strategies to minimize having to recreate scripts. Simply using test automation tools is not enough. You need to ensure that you are using them in a manner that maximizes your productivity.

 

Share

Running vTest in Batch Mode

08. April 2012 · Write a comment · Categories: Functional Testing, Scripting

After you have completed creating and running a script successfully in vTest, you may wish to automate the unattended execution of this script. You might want to have it execute at 4am in the night every night to allow you to inspect the results as soon as you get into work every morning. What are your options in this regard, you might ask.

vTest allows you to save a batch script by selecting the menu item File > Generate Batch Script. This saves a VBScript file to your intended location that automates the execution of the loaded project. Here is what the generated VBScript looks like

Set vtObj = CreateObject(“vTest.Document”)
vtObj.Open(“C:\Program Files\Verisium\vTest\Projects\Project1.vtp”)
vtObj.SetValue “ReportBase”, “C:\Users\Paul\Desktop\Misc”
vtObj.Run()
vtObj.Close()

The above script is mostly self-explanatory with the exception of the line that calls the “SetValue” function. This function can be used to set the value of “ReportBase” variable that allows you to specify where the HTML reports should be saved. “SetValue” can also be used to determine if you wish to run through all the iterations specified in the project or whether you wish to limit the execution to only the first iteration. This is done using the variable “ShouldIterate”. A value of “0″ limits the execution to only the first iteration. A value of “1″ is the default value and allows you to run through all the iterations specified in the project.

Another function called the “GetValue” function allows the caller to obtain several variables. One of these, the “ResultsXml” variable allows the user to obtain an XML representation of the result. This can be useful if you wish to eventually parse these results and save them in your own file or database. This would be called as follows.

Dim xml
……
xml = vtObj.GetValue(“ResultsXml”)

After saving and customizing your batch script, you can run it unattended using the Windows Task Scheduler at a specified time and frequency.

Share