We’ve recently started the development of a Vaadin web application. We were faced with a UI testing problem: how to find the best element selectors for a dynamically generated GWT-based interface. We had two options: to use Vaadin TestBench or Ghost Inspector. Read on to see which one we’ve chosen and why.
Vaadin application end-to-end testing
When you first try to test Vaadin UI, you’ll probably start with Vaadin TestBench – a dedicated test automation tool. It’s deeply integrated with the Vaadin framework, so it appears to be the best choice.
Here’s an example of what TestBench’s tests look like:
As you can see in the example above, TestBench is a Java test framework – you’ll need to know how to code in Java if you’d like to test anything in this case. There is a chance that the dev team will need to create and maintain those tests.
TestBench comes with a Point and Click tool, which you can use for generating UI selectors. When you click an element, it gives you an element selector in Java code, which you can paste to your test code and use for UI assertions. It all works very well when you select standard Vaadin components. When it comes to some unusual widgets or add-ons, you’ll need to do a little extra work to test it, because TestBench by default provides you only with selectors for basic Vaadin components.
If you’d like to keep your tests independent and open a new browser window for each test, TestBench will have some delays with setting up your tests. Even though it allows you to use PhantomJS, it’s still too slow (up to 30 seconds in each test) and that can be annoying. During the test clean-up phase we came across similar problems – responses came a little bit too slowly.
Ghost Inspector
After using Vaadin TestBench for some time, we came to a conclusion that we need to find a better approach. We needed a way which wouldn’t take so much time to write a test and would be easy to use for everyone.
We found Ghost Inspector – a Selenium-based framework provided as a SaaS tool. Its core functionalities are a Chrome extension serving the purpose of a test recorder and a testing dashboard with some cool features. All in all, an easy and less time-consuming tool. That’s what we needed.
Here’s a video:
Ghost Inspector Recorder
The coolest feature of GI is its Chrome extension, called Ghost Inspector Recorder. With GI Recorder, you can record a test and run it within a few clicks.
A Chrome extension mean that QA can use it across all systems which support the Chrome browser, so it’s a plus.
GI Recorder gives us two possibilities. Firstly, recording operations. This simply translates to storing actions which the user performs while browsing the website. Secondly, you can make assertions of the existence of UI elements and their values. So if you’d like to, for example, test logging in to your website and searching through a list, you just start recording and do what you’d normally do on your website. Then you stop recording and make some assertions on the result. Voila, you’ve completed your first test!
The guys from GI did a great job with their Recorder. As you may know, GWT applications generate the UI from Java code, so there is no static HTML. Ids of elements are mostly something like “gwt-uid-6” and they change often when you add new components. With the Recorder, you can easily generate CSS selectors for those elements. It works better than, for example, Selenium IDE, in which your test probably will fail right after recording, because it has problems with finding good GWT selectors and you’ll need to manually write proper ones.
This way you can quickly start testing your website. If you’d like to do some advanced assertions or customize the recorded steps, you just go to the GI dashboard and then open your test’s visual editor to make some modifications.
The visual editor allows QA to do some basic interactions with website such as mouseover, click, assign value or even drag and drop. In GI, you can parametrize your tests by variables used in operations and assertions. You can simply set the value, use a random value generator or just import a CSV file with testing data. If this is not enough for you, GI provides JavaScript assertions. With that, you can test almost anything.
QA infrastructure out of the box
Another cool thing about GI is its testing ecosystem. After signing up, we get QA infrastructure out of the box with a user-friendly interface. Besides many features, it doesn’t need any setup from your side. You can record & run your tests in the blink of an eye. So you won’t need to waste your time on setting up every new team member.
Features of the infrastructure which I think are the coolest:
- Integrations. If you use Slack for your team communication, you’ll love it. Ghost Inspector can send you a nicely formatted message with the results of your test suite run.
Do you use Jenkins or Travis CI? Go ahead, add GI to your build process. It also provides other integrations, check it out.
- Screenshot comparison. Every UI element exists and the flow is correct, but your application is, for example, 5x times wider than normal? You can test it with screenshot comparison. GI allows you to set a screenshot tolerance (e.g 10% difference), because you don’t always want to fail your test.
- Video recording. I think that without this feature debugging problems would be really tough. Every test is recorded and you can watch any present and past test runs.
- Selenium tests import and export. You can easily import existing Selenium tests to your GI account, so there won’t be much of a technical cost to move to GI. You can even export your GI tests to Selenium if you’ve changed your mind.
- Custom JavaScript execution in assertions. Standard assertions and actions are not enough for you? You can use JavaScript to find irregular UI elements or simulate user interactions.
- Scheduling test runs. It’s really easy to schedule continuous testing with GI. You need to select the day and time. It allows you to run a test from once a week to every 5 minutes. If you’d like to test it less frequently, you can still use Jenkins or Travis CI for more advanced scheduling.
- Test customization. Tests can run with different browser access configuration e.g. browser, custom HTTP headers. You can also customize step timing, especially AJAX timeouts. Display options like screen size can also be changed. You’ll love the geolocation feature – run your tests from Tokyo, Mumbai or Northern California in USA.
Test parallelization in cloud
You can run as many tests as you want with GI. By default, they will be run concurrently, which speeds up the process and checks for unwanted test dependencies. This will also allow you to find bugs in more realistic scenarios.
GI allows you to run 30-50 tests in parallel, depending on the current capacity. If you run more than that, they will be queued.
Conclusion
To sum up, it’s easy to start testing with GI. Even inexperienced users should find it very useful. The coolest features are Ghost Inspector Recorder and the infrastructure which speeds up your entire testing experience. Try it yourself and see if it’s for you.
