Category: Sector

We’ve recently started the development of a Vaadin web application. We were faced with a UI testing problem: how to find the best element selectors for a dynamically generated GWT-based interface. We had two options: to use Vaadin TestBench or Ghost Inspector. Read on to see which one we’ve chosen and why.

Vaadin application end-to-end testing

When you first try to test Vaadin UI, you’ll probably start with Vaadin TestBench – a dedicated test automation tool. It’s deeply integrated with the Vaadin framework, so it appears to be the best choice.

Here’s an example of what TestBench’s tests look like:

As you can see in the example above, TestBench is a Java test framework – you’ll need to know how to code in Java if you’d like to test anything in this case. There is a chance that the dev team will need to create and maintain those tests.

TestBench comes with a Point and Click tool, which you can use for generating UI selectors. When you click an element, it gives you an element selector in Java code, which you can paste to your test code and use for UI assertions. It all works very well when you select standard Vaadin components. When it comes to some unusual widgets or add-ons, you’ll need to do a little extra work to test it, because TestBench by default provides you only with selectors for basic Vaadin components.

If you’d like to keep your tests independent and open a new browser window for each test, TestBench will have some delays with setting up your tests. Even though it allows you to use PhantomJS, it’s still too slow (up to 30 seconds in each test) and that can be annoying. During the test clean-up phase we came across similar problems – responses came a little bit too slowly.

Ghost Inspector

After using Vaadin TestBench for some time, we came to a conclusion that we need to find a better approach. We needed a way which wouldn’t take so much time to write a test and would be easy to use for everyone.

We found Ghost Inspector – a Selenium-based framework provided as a SaaS tool. Its core functionalities are a Chrome extension serving the purpose of a test recorder and a testing dashboard with some cool features. All in all, an easy and less time-consuming tool. That’s what we needed.

Here’s a video:

Ghost Inspector Recorder

The coolest feature of GI is its Chrome extension, called Ghost Inspector Recorder. With GI Recorder, you can record a test and run it within a few clicks.

A Chrome extension mean that QA can use it across all systems which support the Chrome browser, so it’s a plus.

GI Recorder gives us two possibilities. Firstly, recording operations. This simply translates to storing actions which the user performs while browsing the website. Secondly, you can make assertions of the existence of UI elements and their values. So if you’d like to, for example, test logging in to your website and searching through a list, you just start recording and do what you’d normally do on your website. Then you stop recording and make some assertions on the result. Voila, you’ve completed your first test!

The guys from GI did a great job with their Recorder. As you may know, GWT applications generate the UI from Java code, so there is no static HTML. Ids of elements are mostly something like “gwt-uid-6” and they change often when you add new components. With the Recorder, you can easily generate CSS selectors for those elements. It works better than, for example, Selenium IDE, in which your test probably will fail right after recording, because it has problems with finding good GWT selectors and you’ll need to manually write proper ones.

This way you can quickly start testing your website. If you’d like to do some advanced assertions or customize the recorded steps, you just go to the GI dashboard and then open your test’s visual editor to make some modifications.

The visual editor allows QA to do some basic interactions with website such as mouseover, click, assign value or even drag and drop. In GI, you can parametrize your tests by variables used in operations and assertions. You can simply set the value, use a random value generator or just import a CSV file with testing data. If this is not enough for you, GI provides JavaScript assertions. With that, you can test almost anything.

QA infrastructure out of the box

Another cool thing about GI is its testing ecosystem. After signing up, we get QA infrastructure out of the box with a user-friendly interface. Besides many features, it doesn’t need any setup from your side. You can record & run your tests in the blink of an eye. So you won’t need to waste your time on setting up every new team member.

Features of the infrastructure which I think are the coolest:

• Integrations. If you use Slack for your team communication, you’ll love it. Ghost Inspector can send you a nicely formatted message with the results of your test suite run.

Do you use Jenkins or Travis CI? Go ahead, add GI to your build process. It also provides other integrations, check it out.

• Screenshot comparison. Every UI element exists and the flow is correct, but your application is, for example, 5x times wider than normal? You can test it with screenshot comparison. GI allows you to set a screenshot tolerance (e.g 10% difference), because you don’t always want to fail your test.

• Video recording. I think that without this feature debugging problems would be really tough. Every test is recorded and you can watch any present and past test runs.
• Selenium tests import and export. You can easily import existing Selenium tests to your GI account, so there won’t be much of a technical cost to move to GI. You can even export your GI tests to Selenium if you’ve changed your mind.
• Custom JavaScript execution in assertions. Standard assertions and actions are not enough for you? You can use JavaScript to find irregular UI elements or simulate user interactions.
• Scheduling test runs. It’s really easy to schedule continuous testing with GI. You need to select the day and time. It allows you to run a test from once a week to every 5 minutes. If you’d like to test it less frequently, you can still use Jenkins or Travis CI for more advanced scheduling.

• Test customization. Tests can run with different browser access configuration e.g. browser, custom HTTP headers. You can also customize step timing, especially AJAX timeouts. Display options like screen size can also be changed. You’ll love the geolocation feature – run your tests from Tokyo, Mumbai or Northern California in USA.

Test parallelization in cloud

You can run as many tests as you want with GI. By default, they will be run concurrently, which speeds up the process and checks for unwanted test dependencies. This will also allow you to find bugs in more realistic scenarios.

GI allows you to run 30-50 tests in parallel, depending on the current capacity. If you run more than that, they will be queued.

Conclusion

To sum up, it’s easy to start testing with GI. Even inexperienced users should find it very useful. The coolest features are Ghost Inspector Recorder and the infrastructure which speeds up your entire testing experience. Try it yourself and see if it’s for you.

Analyzing a company’s Definition of Done (DoD) can tell you a lot about their work ethic and what they put the ‘quality’ sticker on. I’ll explain a few concepts about and around DoD to make it clearer.

First question: do they have a DoD?

There are some vendors who don’t have a DoD at all. That usually doesn’t spell anything good. If you’re set on signing a fixed-price contract (which I don’t recommend for software development), then it’s likely the contract will include certain definitions. However, there are some things that are hard to put into that initial agreement you sign. This makes them tough to enforce on an agreement level.

If you’re using Scrum/agile practices, then a missing DoD document is odd and should be worrisome. Are the quality criteria arbitrary, depending on a particular developer’s current mood? Or perhaps – the list exists, but you’ve never seen it? Ask yourself why that could be the case. The answer can’t be good.

Scrum

The DoD is a frequent element of agile practices. Just a word here, though. Scrum doesn’t dictate what exactly your Definition of Done should be, it just suggests that the functionalities that the team delivers in a sprint should meet a set of criteria you’ve established together. It’s a form of agreement between the team and – usually – the Product Owner on the client side.

What is it?

It’s basically a set of expectations software has to live up to. If it doesn’t meet the criteria you’ve established with your team, it can’t go to production. Let’s say you can determine two things out of the triangle of scope, quality and time. We’re establishing a firm timespan of a scrum iteration and constant (high!) quality. That’s the basis for DoD.

Some general rules

You usually establish code and functionality checks, and code coverage percentage, so that you can objectively state how well-tested the code is. Is it implemented according to all user requirements? Has it built and have all tests passed locally? Last but not least, you look at performance and security issues – and, of course, the code must go through a code review.

Don’t confuse it with the Definition of Ready, which is a set of minimum criteria that a particular item in your backlog (or your list of tasks or features you want to have) must fulfill before it gets moved to development. DoR is designed to weed out badly defined tasks.

The DoD document isn’t a contract per se, but it’s sometimes referred to as such. One of our teams (working on one of our bigger, scalable projects) recently printed out the DoD list and everyone signed it, client and team, old school style! This is just to underline how seriously this document is treated.

Transparency is key!

As a client, you shouldn’t just be familiar with the Definition of Done. You should understand it. If there’s anything that you don’t quite get, it’s the software partner’s job to explain it to you. After all, you’re agreeing on a common definition of quality, and quality that can be – and will be – checked and measured. Working with agencies that obscure the meaning of certain phrases or are deliberately vague about what ‘done’ means will be problematic.

Maintaining your documentation in order is also a part of the Definition of Done. There are other transparency issues connected indirectly with the DoD, such as accountability, transparency when it comes to reporting, not to mention a clear process.

Other matters of quality

If you’re asking your vendor about quality and what is means to really be ‘done’, ask them about code reviews and testing. Code reviews and audits are derivatives of DoD. Another developer should evaluate the feature you want – a second opinion is necessary. This is an ongoing process. Software testing is part of Quality Assurance. Good automatic and manual tests will ensure nothing blows up in your face once you get your feature released.

A really good software development offer should be complemented with design and hosting services, experience with monitoring and performance scaling, as your application is designed to grow, right? If you’re at the starting point, product design (the whole package) is certainly a nice thing for an agency to have.

If you’d like to grab our Definition of Done list (we’ve worked hard on doing it right), drop us an email at hi@espeo.eu with ‘DoD’ in the title.

Imagine you need to create plenty of presentations that look the same, but have a slightly different content, like text or images. Well, you can do it all by hand if you have tons of time and find relaxation in doing repeatable tasks over and over again. However, today I’ll give you a better solution. A few months ago Google provided API to its Slides service, which we’ll use to achieve our goal: automating the process of creating presentations.

Real-life use case

One of our clients came to us with a problem. He needed to create many presentations for each of his (very many) products. Up until then, his employees had to do it by hand, but the client found it too time-consuming. He wanted a tool which would automatically generate ready-to-use presentations from a template with just some input. I’ll show you how we achieved it.

Requirements

What you need? Not much, really. Basic PHP knowledge and familiarity with composer (a PHP dependency manager). And a Google account, of course. I strongly recommend you also to read Google PHP quickstart first, as it shows a sample script we based our solution upon. There is also an instruction on how to enable Google Slides API in the Google Api Console, which is a required step for our script to work.

Templates concept

Google Slides API offers a great many possibilities. You can create a presentation adding every single text programmatically, moving it to the correct place and then styling it. However, this process is very time consuming and complicated. Alternatively, you can create a presentation template by hand and then, via API, replace elements that are already positioned and styled. The whole process of creating a template is enclosed in just a few steps:

1. Create a new Google Slides presentation, which will be the template from which we will generate other presentations.
2. Fill the template with common static content.
3. For elements that need to be replaced, use text and shapes with {{ placeholders }}. {{ variableName }} is just a convention which helps to distinguish static text from placeholders.
4. Format elements according to your needs.

And here we are! In 4 steps, we have a ready-to-use template. Cool, isn’t it?

Let’s go!

Make sure you’ve enabled the Google Slides API, authenticated yourself and downloaded the credentials as described in the aforementioned quickstart. The script is divided into 4 steps:

1. Clone template as a new presentation
2. Upload images to Google Drive
3. Replace placeholders with target content
4. Download presentation as PDF

Clone template as a new presentation

We have to use Google Drive API for this because Google Slides API doesn’t have such a functionality. The code is mostly self-explanatory, so I’ll focus on only some parts of it.

Take a look at the file listing query. To find our template presentation on Drive, we’ll look for it by name (put in TEMPLATE_NAME constant) and mimeType which is application/vnd.google-apps.presentation. The query always returns a collection, so we need to check if any elements are present and choose the first of them. Later on, we just copy the template we’ve found.

Upload images to Google Drive

We don’t want everyone to have access to our uploaded images and, at the same time, we need URLs for them. That’s why after uploading an image we create a private link with our access token and use it for embedding the image later on.

Replace placeholders with target content

I created 2 wrapper functions – one for replacing a text placeholder and the other one for replacing a shape placeholder with an image from URL. There is also a function to execute an array of requests.

Download presentation as PDF

After your work is done, you can also download your presentation as a PDF file. It’s a pretty simple Google Drive API call.

Put all steps together

And here it is! If you use the functions from the steps above, our main function to generate a presentation is very simple:

You can download the full code from here. In README.md you’ll find the instructions for installation and some other details. The project is just a quick start for you to generate presentations according to your needs, so feel free to download and modify.
Let me know if you used this – or if you have any feedback – in the comments 🙂

Are you a Linux user who switched to Mac when you saw that Docker is now available as a native Mac app? Or maybe you’ve heard how great Docker is and you want to give it a try? Did you think that you could just take your Docker Compose file, launch your project and have everything work out for you? Well… you were right. Almost.

Docker for Mac is a pretty smart invention. It gives you the whole Docker API available from the terminal, even though Docker itself wasn’t created to work on Macs. To make all this possible, a light Alpine Linux image is fired up underneath with xhyve MacOS native virtualization. Because of this, you need to allocate CPU cores and RAM for the VM. Things won’t be as close to bare metal as they are in Linux. If you are – for example – a Java developer who uses Docker to run compiled jar, you may even not notice the difference. At least, as long as you don’t try to do any heavy database work.

Docker for Mac and full sync on flush issue

First, let’s look at MacOS fsync documentation:

“For applications that require tighter guarantees about the integrity of their data, Mac OS X provides the F_FULLFSYNC fcntl. The F_FULLFSYNC fcntl asks the drive to flush all buffered data to permanent storage. Applications, such as databases, that require a strict ordering of writes should use F_FULLFSYNC to ensure that their data is written in the order they expect.”

In short – to keep our data safe – every change made in the database needs to be stored on disk in an exact order. This will guarantee that during power loss or any unexpected event your data will be safe.
Actually, this make sense. IF you decide to setup a database inside Docker for Mac on a production environment…

However…

In most cases, though, you’ll be using your machine for dev purposes, where you don’t care to recreate database from fixtures. If you have a Macbook, even power loss isn’t a threat. In this case, you may decide to disable this behaviour as F_FULLFSYNC has a negative performance drawback in Docker for Mac
While reading about Docker issues on Github, I found a solution provided by djs55. Things will get a lot faster when you type those few lines into your terminal:

$ cd ~/Library/Containers/com.docker.docker/Data/database/
 $ git reset --hard
 HEAD is now at cafabd0 Docker started 1475137831
 $ cat com.docker.driver.amd64-linux/disk/full-sync-on-flush
 true
 $ echo false > com.docker.driver.amd64-linux/disk/full-sync-on-flush
 $ git add com.docker.driver.amd64-linux/disk/full-sync-on-flush
 $ git commit -s -m "Disable flushing"
 [master dc32fcc] Disable flushing
 1 file changed, 1 insertion(+), 1 deletion(-)

 
Actually, someone even placed bash script on gist to make things easier.

Does it really work, you’ll ask?

I created a small test case to check this. This test uses a standard Docker MySQL image without tweaks, and an image with sysbench installed. In my test case, I decided to use one thread (I only allocated 1 core for Docker on my Macbook) and a table with 10000 rows.
I ran it twice – once with flushing enabled (default), and once with flushing disabled. If you’re skeptical about performance gain after changing just one value from true to false, then let the results below change your mind.
 
Command to run it:

$ docker-compose build
$ docker-compose up benchmark

flush enabled:

mysql-test-bench | OLTP test statistics:
mysql-test-bench |     queries performed:
mysql-test-bench |         read:                            30730
mysql-test-bench |         write:                           8780
mysql-test-bench |         other:                           4390
mysql-test-bench |         total:                           43900
mysql-test-bench |     transactions:                        2195   (36.58 per sec.)
mysql-test-bench |     read/write requests:                 39510  (658.42 per sec.)
mysql-test-bench |     other operations:                    4390   (73.16 per sec.)
mysql-test-bench |     ignored errors:                      0      (0.00 per sec.)
mysql-test-bench |     reconnects:                          0      (0.00 per sec.)
mysql-test-bench |
mysql-test-bench | General statistics:
mysql-test-bench |     total time:                          60.0077s
mysql-test-bench |     total number of events:              2195
mysql-test-bench |     total time taken by event execution: 59.9995s
mysql-test-bench |     response time:
mysql-test-bench |          min:                                 21.26ms
mysql-test-bench |          avg:                                 27.33ms
mysql-test-bench |          max:                                 73.00ms
mysql-test-bench |          approx.  95 percentile:              35.62ms
mysql-test-bench |
mysql-test-bench | Threads fairness:
mysql-test-bench |     events (avg/stddev):           2195.0000/0.00
mysql-test-bench |     execution time (avg/stddev):   59.9995/0.00

flush disabled:

mysql-test-bench | OLTP test statistics: mysql-test-bench |     queries performed: mysql-test-bench |         read:                            270074 mysql-test-bench |         write:                           77164 mysql-test-bench |         other:                           38582 mysql-test-bench |         total:                           385820 mysql-test-bench |     transactions:                        19291  (321.51 per sec.) mysql-test-bench |     read/write requests:                 347238 (5787.13 per sec.) mysql-test-bench |     other operations:                    38582  (643.01 per sec.) mysql-test-bench |     ignored errors:                      0      (0.00 per sec.) mysql-test-bench |     reconnects:                          0      (0.00 per sec.) mysql-test-bench | mysql-test-bench | General statistics: mysql-test-bench |     total time:                          60.0018s mysql-test-bench |     total number of events:              19291 mysql-test-bench |     total time taken by event execution: 59.9613s mysql-test-bench |     response time: mysql-test-bench |          min:                                  2.68ms mysql-test-bench |          avg:                                  3.11ms mysql-test-bench |          max:                                 20.70ms mysql-test-bench |          approx.  95 percentile:               3.65ms mysql-test-bench | mysql-test-bench | Threads fairness: mysql-test-bench |     events (avg/stddev):           19291.0000/0.00 mysql-test-bench |     execution time (avg/stddev):   59.9613/0.00

 
Looking at those numbers we clearly see that with flushing disabled we gained almost 10x performance! And this is with only 10k rows.
This means that if you don’t care that much about data loss, and you’re ready to sacrifice it if something goes wrong, then there is no reason to actually not change this setting.

Tip: From my observations, this tweak seems to be preserved after Docker for Mac updates, so there is no need to fire it over and over.
With one simple step, our Docker database performs approximately 10x faster, so now everything should be great, right? Well… not yet.

Docker for Mac and mounted volumes

Interpreted programming languages usually come with a large amount of files, cache, bootstrap, etc. PHP with Symfony framework is a good example. Symfony, with almost no cache (e.g. dev environment), writes and reads lot of files between Request and Response. Because the language is interpreted, a PHP developer can see code change results almost immediately. Therefore, mounting code as a volume inside a Docker container is a natural way of work. Linux performance with aufs is close to native in this case, but Mac osxfs, on the other hand, isn’t.
It’s so slow that you can even get composer timeouts on “composer install” or “composer update”. What’s more, Symfony requests to a “Hello World” page can take up to 30s. There is a large thread on github about this issue as well.

To prove how bad things are, a few new tests need to be run.

We’ll just create a dummy file of around ~100MB.
First, let’s run a simple command directly on a Mac terminal to have a base to compare with:

$ time dd if=/dev/zero of=test.dat bs=1024 count=100000 100000+0 records in 100000+0 records out real    0m0.291s user    0m0.021s sys    0m0.250s

 
Now let’s try same thing inside a Docker container.
Test:

$ docker-compose up Starting docker-native Attaching to docker-native docker-native | 100000+0 records in docker-native | 100000+0 records out docker-native | real    0m 0.38s docker-native | user    0m 0.00s docker-native | sys    0m 0.21s

As you can see, running something in Docker doesn’t make things much slower.

But what if we want to mount our local directory in a Docker container?

For this, let’s use the standard way.
Test:

$ docker-compose up
Starting docker-mount
Attaching to docker-mount
docker-mount | 100000+0 records in
docker-mount | 100000+0 records out
docker-mount | real    0m 17.80s
docker-mount | user    0m 0.12s
docker-mount | sys    0m 0.66s

 
This time, the results are even more spectacular than in the database case. Creating the same file in a mounted volume is 45x slower. Now, think how this can actually affect composer, Symfony, database or any other app which requires hard disk writes!
Unfortunately there is no built-in solution. You may find workarounds which will allow you to mount volumes with nfs instead of osxfs, but it still may be not enough for Symfony.

Hopefully, in this case the Internets can save the day again!

In my dev environment, I decided to use docker-sync by EugenMayer.
What does this tool do? It will allow you to create Docker volumes whose content will be synced with the host using unison. Then you can use this volume as mounting point for your container. This speeds things up a lot. Your app is syncing data with the volume with almost native speed. Then, the volume is synced with host in background and doesn’t cause any slowdowns for the app.
Now it’s time for another test to show how fast this solution can really be.
Firstly, let’s create docker-sync.yml file which is required by docker-sync:

 
In this file, we declare our volume “docker-mac-sync”, which then can be used in the docker-compose.yml file. I tried to keep it as simple as possible. One thing worth mentioning is the sync_args param, which just allows you to configure standard unison flags. It’s not necessary for our test, but in most cases the “prefer newer” strategy will be used by devs.

Now, it’s time to use docker-sync in Docker compose:

 
In the volumes section, we declare “docker-mac-sync” as external one and just use as mount point for /test inside a Docker container.
Before we make our final test there is one more thing to mention. Because of its nature, docker-sync requires that files should be synced into the docker-sync created container before we run our main app. It may take some time if we have a large codebase, but after that, subsequent syncs will be fast (at least until you delete the sync volumen).
The tool provides two commands to interact with it – docker-sync and docker-sync-stack. The second is just a shortcut to docker-sync start && docker-compose up. I tried both, but prefer to run docker-sync in a separate tab to see what’s actually happening in it.
 
To start syncing, I just need to type:

$ docker-sync start

 
Then, in next tab, the magic happens:

$ docker-compose up
Starting docker-sync-mount
Attaching to docker-sync-mount
docker-sync-mount | 100000+0 records in
docker-sync-mount | 100000+0 records out
docker-sync-mount | real    0m 0.40s
docker-sync-mount | user    0m 0.01s
docker-sync-mount | sys    0m 0.24s

 
Benchmark shows times similar to what we saw when we ran the test inside a Docker container without any mount. That’s a huge improvement, and it will allow you to work comfortably with your new Mac… almost.
Unfortunately, nothing is perfect.

The presented solution has a few drawbacks:

• you need to remember to run docker-sync before running your app
• when you switch branches, in most cases it’ll be wise to stop the app, use “docker-sync clean” and re-sync everything
• sometimes docker-sync may freeze – it’s worth to check it from time to time and just restart it
• because it’s a Mac workaround, it may be wise to have separate docker-compose.mac.yml and docker-compose.linux.yml and run docker compose with “-f” flag keeping base docker-compose.yml as clean as possible

Still reading?

Good 🙂
As a bonus, there is one more trick for those who use xdebug.
Because of its nature, docker-for-mac won’t allow you to simply put xdebug.remote_connect_back=1 in your xdebug.ini file – due to the nature of connection_back, it just won’t work. There are a few workarounds, but the one I use creates 10.254.254.254 as an alias on your loopback device (127.0.0.1). This one command in Terminal will do it:

$ ifconfig lo0 alias 10.254.254.254

You may also use a gist plist to have it fired automatically on each boot. After that your xdebug config may look like this:
 

 
Then, in PHPStorm, you may just set up DBG Proxy like this:

In my sample config, I use idekey and remote_autostart = 0 because I prefer to start the xdebug session from a Firefox addon called “The easiest xdebug”. Chrome has something similar as well.

That’s all!

I hope that this article gave you an idea how to tweak certain issues concerning Docker for Mac. I just want to wish you(/me/us!) that someday this whole wall of text will be obsolete, and future versions of Docker for Mac won’t require hacking to be usable. By the way – all examples presented in this article can be downloaded from github, here are a few more Docker resources for you to check out.