Adopting a new testing tool – the journal of a Quality Assurance Engineer
January 23, 2019
Iulian Marin, QA
The key in dealing with successful software projects is being open and aware of all technologies out there and being versatile enough to be able to choose and apply the right ones for your projects. This article is about the learning experience we got when we tried on Triscentis Tosca automation Tool and started applying it to some of our ongoing projects, through the eyes of the person going through all the steps, an experienced Automation tester eager to try on a new toy.
For a quick introduction of the product, the Tosca Automation tool is developed by Tricentis, and it offers automate end-to-end testing for GUIs and APIs from a business perspective. Our learnings from the first encounters with the product:
Training, documentation, tutorials, guidelines – Tricentis offers Paid Online Courses with certifications on various levels of experience and expertise, facilitated by a lot of training materials on their support site; however, the lack of community will make it harder for an unexperienced person to find answers. Tosca itself is very intuitive, yet the training pages cannot cover everything. It’s always preferable not to reinvent the wheel and learn from other people’s experience, especially in the era of communities. Probably this will solve by itself in the future and the community will develop – for now, most solutions to hard challenges need some imagination.
Licensing and support costs – Although Tosca is not free, the low-level programming requirements(basic really) for using it will actually save money in the long run for the companies adopting it. The UI interface of Tosca makes it a lot easier for low experience users to use it. Any user can start using Tosca within a day of training and create basic tests. Also, the UI interface can reduce costs because users are not required to create features for an Automation Framework such as the ones created with Java – Selenium.
Level of programming skills required – As mentioned above the level of programming required is very low (if even that), the only programming required so far is knowledge on how condition and loop statements work. Tosca even makes this easier by implementing buttons for these features exactly where they are needed. A great plus!
Level of skills and experience required – A more experienced Automation Tester will create better organized and more stable tests in Tosca, but this is not a requirement. Any tester can successfully deliver a project within acceptable deadlines and specifications.
How it went for us and where we would need more from the tool. A QA perspective from the field:
Starting right after First Time Install, the interface surprised me on how intuitive it was. Install it, create a project and you are ready to go. I didn’t look once for information on what something from the interface means or had doubts about it. Of course, the Automation Level 1 course shows how this should be done, but it’s so easy I almost forgot to return to Course Window.
After the project is created the user can view what Tosca has to offer: Intuitive Progress throughout the application. Clicking on Project button will open a window where user can see the core functionalities. Having an UI interface will also help the user understand what he can do (Example: The available buttons for Recovery and Clean Up Scenarios will let the user know he can do that). For a better glance on it, I described some components below:
Modules – Here the user can define objects in the application – the interface helps a lot by being similar to a Page Object Model Framework. The Scan Application feature here works as good as it can. I didn’t see anything missing here and it even offered more than I wanted.
Test Case Section and Test Case Design Screens – The user can define a test case and by setting it as template he can assign multiple values from the Test Case Design Sheets, creating as a result as many scenarios as needed without too much effort. The Test Case Design usability makes it even easier: creating the Structure of a Test Sheet is done only by drag and dropping a Test Case into the window.
Execution list – The user can create Execution lists for specific needs. (Example: Sanity checks, Regression checks, Modules of an application)
API Testing – Tricentis also offers a separate (free) API Testing Tool. I found it easy to integrate with Tosca itself
Requirements, Reporting and Bug Reporting
Something that will only be observed with time: Tosca is a lot faster than Selenium when tests are running. Of course, Selenium Tests can be optimized but that requires budget to do so. Tosca needs a tweak now and then, once at every 30 Test Cases created depending on the tests.
Cons worth mentioning:
The biggest issue is the small community; Sure, Tosca has a great Intuitive UI, but some things require some easy way to find answers because Tosca is used by humans and we don’t always see the solution that stares us in the face.
Tosca doesn’t retain the browser in which the application is launched. This makes running multiple test threads on the same machine almost impossible (running multiple test batches on the same machine in the same time).
Test Case Design References to the Test Case Templates are not updated – When the user modifies the Test Case Design Elements, he will also have to update the Test Case Reference Manually wherever they are used
The application is updated with new features really often.
All in all, Tosca is a great testing tool worth being taken into consideration in a lot of the large Web Applications Automation Processes; especially because non-technical employees can use it, and anyone can understand the progress and the expected results, from the very start of the Automation Process. We are very happy to be certified on using it and for trying it out and we know it will be used more and more in the future. We will also stay tuned on the novelties out there and come back with updates as we encounter them.