Category: Discipline

Analyzing a company’s Definition of Done (DoD) can tell you a lot about their work ethic and what they put the ‘quality’ sticker on. I’ll explain a few concepts about and around DoD to make it clearer.

First question: do they have a DoD?

There are some vendors who don’t have a DoD at all. That usually doesn’t spell anything good. If you’re set on signing a fixed-price contract (which I don’t recommend for software development), then it’s likely the contract will include certain definitions. However, there are some things that are hard to put into that initial agreement you sign. This makes them tough to enforce on an agreement level.

If you’re using Scrum/agile practices, then a missing DoD document is odd and should be worrisome. Are the quality criteria arbitrary, depending on a particular developer’s current mood? Or perhaps – the list exists, but you’ve never seen it? Ask yourself why that could be the case. The answer can’t be good.

Scrum

The DoD is a frequent element of agile practices. Just a word here, though. Scrum doesn’t dictate what exactly your Definition of Done should be, it just suggests that the functionalities that the team delivers in a sprint should meet a set of criteria you’ve established together. It’s a form of agreement between the team and – usually – the Product Owner on the client side.

What is it?

It’s basically a set of expectations software has to live up to. If it doesn’t meet the criteria you’ve established with your team, it can’t go to production. Let’s say you can determine two things out of the triangle of scope, quality and time. We’re establishing a firm timespan of a scrum iteration and constant (high!) quality. That’s the basis for DoD.

Some general rules

You usually establish code and functionality checks, and code coverage percentage, so that you can objectively state how well-tested the code is. Is it implemented according to all user requirements? Has it built and have all tests passed locally? Last but not least, you look at performance and security issues – and, of course, the code must go through a code review.

Don’t confuse it with the Definition of Ready, which is a set of minimum criteria that a particular item in your backlog (or your list of tasks or features you want to have) must fulfill before it gets moved to development. DoR is designed to weed out badly defined tasks.

The DoD document isn’t a contract per se, but it’s sometimes referred to as such. One of our teams (working on one of our bigger, scalable projects) recently printed out the DoD list and everyone signed it, client and team, old school style! This is just to underline how seriously this document is treated.

Transparency is key!

As a client, you shouldn’t just be familiar with the Definition of Done. You should understand it. If there’s anything that you don’t quite get, it’s the software partner’s job to explain it to you. After all, you’re agreeing on a common definition of quality, and quality that can be – and will be – checked and measured. Working with agencies that obscure the meaning of certain phrases or are deliberately vague about what ‘done’ means will be problematic.

Maintaining your documentation in order is also a part of the Definition of Done. There are other transparency issues connected indirectly with the DoD, such as accountability, transparency when it comes to reporting, not to mention a clear process.

Other matters of quality

If you’re asking your vendor about quality and what is means to really be ‘done’, ask them about code reviews and testing. Code reviews and audits are derivatives of DoD. Another developer should evaluate the feature you want – a second opinion is necessary. This is an ongoing process. Software testing is part of Quality Assurance. Good automatic and manual tests will ensure nothing blows up in your face once you get your feature released.

A really good software development offer should be complemented with design and hosting services, experience with monitoring and performance scaling, as your application is designed to grow, right? If you’re at the starting point, product design (the whole package) is certainly a nice thing for an agency to have.

If you’d like to grab our Definition of Done list (we’ve worked hard on doing it right), drop us an email at hi@espeo.eu with ‘DoD’ in the title.

Imagine you need to create plenty of presentations that look the same, but have a slightly different content, like text or images. Well, you can do it all by hand if you have tons of time and find relaxation in doing repeatable tasks over and over again. However, today I’ll give you a better solution. A few months ago Google provided API to its Slides service, which we’ll use to achieve our goal: automating the process of creating presentations.

Real-life use case

One of our clients came to us with a problem. He needed to create many presentations for each of his (very many) products. Up until then, his employees had to do it by hand, but the client found it too time-consuming. He wanted a tool which would automatically generate ready-to-use presentations from a template with just some input. I’ll show you how we achieved it.

Requirements

What you need? Not much, really. Basic PHP knowledge and familiarity with composer (a PHP dependency manager). And a Google account, of course. I strongly recommend you also to read Google PHP quickstart first, as it shows a sample script we based our solution upon. There is also an instruction on how to enable Google Slides API in the Google Api Console, which is a required step for our script to work.

Templates concept

Google Slides API offers a great many possibilities. You can create a presentation adding every single text programmatically, moving it to the correct place and then styling it. However, this process is very time consuming and complicated. Alternatively, you can create a presentation template by hand and then, via API, replace elements that are already positioned and styled. The whole process of creating a template is enclosed in just a few steps:

1. Create a new Google Slides presentation, which will be the template from which we will generate other presentations.
2. Fill the template with common static content.
3. For elements that need to be replaced, use text and shapes with {{ placeholders }}. {{ variableName }} is just a convention which helps to distinguish static text from placeholders.
4. Format elements according to your needs.

And here we are! In 4 steps, we have a ready-to-use template. Cool, isn’t it?

Let’s go!

Make sure you’ve enabled the Google Slides API, authenticated yourself and downloaded the credentials as described in the aforementioned quickstart. The script is divided into 4 steps:

1. Clone template as a new presentation
2. Upload images to Google Drive
3. Replace placeholders with target content
4. Download presentation as PDF

Clone template as a new presentation

We have to use Google Drive API for this because Google Slides API doesn’t have such a functionality. The code is mostly self-explanatory, so I’ll focus on only some parts of it.

Take a look at the file listing query. To find our template presentation on Drive, we’ll look for it by name (put in TEMPLATE_NAME constant) and mimeType which is application/vnd.google-apps.presentation. The query always returns a collection, so we need to check if any elements are present and choose the first of them. Later on, we just copy the template we’ve found.

Upload images to Google Drive

We don’t want everyone to have access to our uploaded images and, at the same time, we need URLs for them. That’s why after uploading an image we create a private link with our access token and use it for embedding the image later on.

Replace placeholders with target content

I created 2 wrapper functions – one for replacing a text placeholder and the other one for replacing a shape placeholder with an image from URL. There is also a function to execute an array of requests.

Download presentation as PDF

After your work is done, you can also download your presentation as a PDF file. It’s a pretty simple Google Drive API call.

Put all steps together

And here it is! If you use the functions from the steps above, our main function to generate a presentation is very simple:

You can download the full code from here. In README.md you’ll find the instructions for installation and some other details. The project is just a quick start for you to generate presentations according to your needs, so feel free to download and modify.
Let me know if you used this – or if you have any feedback – in the comments 🙂

Are you a Linux user who switched to Mac when you saw that Docker is now available as a native Mac app? Or maybe you’ve heard how great Docker is and you want to give it a try? Did you think that you could just take your Docker Compose file, launch your project and have everything work out for you? Well… you were right. Almost.

Docker for Mac is a pretty smart invention. It gives you the whole Docker API available from the terminal, even though Docker itself wasn’t created to work on Macs. To make all this possible, a light Alpine Linux image is fired up underneath with xhyve MacOS native virtualization. Because of this, you need to allocate CPU cores and RAM for the VM. Things won’t be as close to bare metal as they are in Linux. If you are – for example – a Java developer who uses Docker to run compiled jar, you may even not notice the difference. At least, as long as you don’t try to do any heavy database work.

Docker for Mac and full sync on flush issue

First, let’s look at MacOS fsync documentation:

“For applications that require tighter guarantees about the integrity of their data, Mac OS X provides the F_FULLFSYNC fcntl. The F_FULLFSYNC fcntl asks the drive to flush all buffered data to permanent storage. Applications, such as databases, that require a strict ordering of writes should use F_FULLFSYNC to ensure that their data is written in the order they expect.”

In short – to keep our data safe – every change made in the database needs to be stored on disk in an exact order. This will guarantee that during power loss or any unexpected event your data will be safe.
Actually, this make sense. IF you decide to setup a database inside Docker for Mac on a production environment…

However…

In most cases, though, you’ll be using your machine for dev purposes, where you don’t care to recreate database from fixtures. If you have a Macbook, even power loss isn’t a threat. In this case, you may decide to disable this behaviour as F_FULLFSYNC has a negative performance drawback in Docker for Mac
While reading about Docker issues on Github, I found a solution provided by djs55. Things will get a lot faster when you type those few lines into your terminal:

$ cd ~/Library/Containers/com.docker.docker/Data/database/
 $ git reset --hard
 HEAD is now at cafabd0 Docker started 1475137831
 $ cat com.docker.driver.amd64-linux/disk/full-sync-on-flush
 true
 $ echo false > com.docker.driver.amd64-linux/disk/full-sync-on-flush
 $ git add com.docker.driver.amd64-linux/disk/full-sync-on-flush
 $ git commit -s -m "Disable flushing"
 [master dc32fcc] Disable flushing
 1 file changed, 1 insertion(+), 1 deletion(-)

 
Actually, someone even placed bash script on gist to make things easier.

Does it really work, you’ll ask?

I created a small test case to check this. This test uses a standard Docker MySQL image without tweaks, and an image with sysbench installed. In my test case, I decided to use one thread (I only allocated 1 core for Docker on my Macbook) and a table with 10000 rows.
I ran it twice – once with flushing enabled (default), and once with flushing disabled. If you’re skeptical about performance gain after changing just one value from true to false, then let the results below change your mind.
 
Command to run it:

$ docker-compose build
$ docker-compose up benchmark

flush enabled:

mysql-test-bench | OLTP test statistics:
mysql-test-bench |     queries performed:
mysql-test-bench |         read:                            30730
mysql-test-bench |         write:                           8780
mysql-test-bench |         other:                           4390
mysql-test-bench |         total:                           43900
mysql-test-bench |     transactions:                        2195   (36.58 per sec.)
mysql-test-bench |     read/write requests:                 39510  (658.42 per sec.)
mysql-test-bench |     other operations:                    4390   (73.16 per sec.)
mysql-test-bench |     ignored errors:                      0      (0.00 per sec.)
mysql-test-bench |     reconnects:                          0      (0.00 per sec.)
mysql-test-bench |
mysql-test-bench | General statistics:
mysql-test-bench |     total time:                          60.0077s
mysql-test-bench |     total number of events:              2195
mysql-test-bench |     total time taken by event execution: 59.9995s
mysql-test-bench |     response time:
mysql-test-bench |          min:                                 21.26ms
mysql-test-bench |          avg:                                 27.33ms
mysql-test-bench |          max:                                 73.00ms
mysql-test-bench |          approx.  95 percentile:              35.62ms
mysql-test-bench |
mysql-test-bench | Threads fairness:
mysql-test-bench |     events (avg/stddev):           2195.0000/0.00
mysql-test-bench |     execution time (avg/stddev):   59.9995/0.00

flush disabled:

mysql-test-bench | OLTP test statistics: mysql-test-bench |     queries performed: mysql-test-bench |         read:                            270074 mysql-test-bench |         write:                           77164 mysql-test-bench |         other:                           38582 mysql-test-bench |         total:                           385820 mysql-test-bench |     transactions:                        19291  (321.51 per sec.) mysql-test-bench |     read/write requests:                 347238 (5787.13 per sec.) mysql-test-bench |     other operations:                    38582  (643.01 per sec.) mysql-test-bench |     ignored errors:                      0      (0.00 per sec.) mysql-test-bench |     reconnects:                          0      (0.00 per sec.) mysql-test-bench | mysql-test-bench | General statistics: mysql-test-bench |     total time:                          60.0018s mysql-test-bench |     total number of events:              19291 mysql-test-bench |     total time taken by event execution: 59.9613s mysql-test-bench |     response time: mysql-test-bench |          min:                                  2.68ms mysql-test-bench |          avg:                                  3.11ms mysql-test-bench |          max:                                 20.70ms mysql-test-bench |          approx.  95 percentile:               3.65ms mysql-test-bench | mysql-test-bench | Threads fairness: mysql-test-bench |     events (avg/stddev):           19291.0000/0.00 mysql-test-bench |     execution time (avg/stddev):   59.9613/0.00

 
Looking at those numbers we clearly see that with flushing disabled we gained almost 10x performance! And this is with only 10k rows.
This means that if you don’t care that much about data loss, and you’re ready to sacrifice it if something goes wrong, then there is no reason to actually not change this setting.

Tip: From my observations, this tweak seems to be preserved after Docker for Mac updates, so there is no need to fire it over and over.
With one simple step, our Docker database performs approximately 10x faster, so now everything should be great, right? Well… not yet.

Docker for Mac and mounted volumes

Interpreted programming languages usually come with a large amount of files, cache, bootstrap, etc. PHP with Symfony framework is a good example. Symfony, with almost no cache (e.g. dev environment), writes and reads lot of files between Request and Response. Because the language is interpreted, a PHP developer can see code change results almost immediately. Therefore, mounting code as a volume inside a Docker container is a natural way of work. Linux performance with aufs is close to native in this case, but Mac osxfs, on the other hand, isn’t.
It’s so slow that you can even get composer timeouts on “composer install” or “composer update”. What’s more, Symfony requests to a “Hello World” page can take up to 30s. There is a large thread on github about this issue as well.

To prove how bad things are, a few new tests need to be run.

We’ll just create a dummy file of around ~100MB.
First, let’s run a simple command directly on a Mac terminal to have a base to compare with:

$ time dd if=/dev/zero of=test.dat bs=1024 count=100000 100000+0 records in 100000+0 records out real    0m0.291s user    0m0.021s sys    0m0.250s

 
Now let’s try same thing inside a Docker container.
Test:

$ docker-compose up Starting docker-native Attaching to docker-native docker-native | 100000+0 records in docker-native | 100000+0 records out docker-native | real    0m 0.38s docker-native | user    0m 0.00s docker-native | sys    0m 0.21s

As you can see, running something in Docker doesn’t make things much slower.

But what if we want to mount our local directory in a Docker container?

For this, let’s use the standard way.
Test:

$ docker-compose up
Starting docker-mount
Attaching to docker-mount
docker-mount | 100000+0 records in
docker-mount | 100000+0 records out
docker-mount | real    0m 17.80s
docker-mount | user    0m 0.12s
docker-mount | sys    0m 0.66s

 
This time, the results are even more spectacular than in the database case. Creating the same file in a mounted volume is 45x slower. Now, think how this can actually affect composer, Symfony, database or any other app which requires hard disk writes!
Unfortunately there is no built-in solution. You may find workarounds which will allow you to mount volumes with nfs instead of osxfs, but it still may be not enough for Symfony.

Hopefully, in this case the Internets can save the day again!

In my dev environment, I decided to use docker-sync by EugenMayer.
What does this tool do? It will allow you to create Docker volumes whose content will be synced with the host using unison. Then you can use this volume as mounting point for your container. This speeds things up a lot. Your app is syncing data with the volume with almost native speed. Then, the volume is synced with host in background and doesn’t cause any slowdowns for the app.
Now it’s time for another test to show how fast this solution can really be.
Firstly, let’s create docker-sync.yml file which is required by docker-sync:

 
In this file, we declare our volume “docker-mac-sync”, which then can be used in the docker-compose.yml file. I tried to keep it as simple as possible. One thing worth mentioning is the sync_args param, which just allows you to configure standard unison flags. It’s not necessary for our test, but in most cases the “prefer newer” strategy will be used by devs.

Now, it’s time to use docker-sync in Docker compose:

 
In the volumes section, we declare “docker-mac-sync” as external one and just use as mount point for /test inside a Docker container.
Before we make our final test there is one more thing to mention. Because of its nature, docker-sync requires that files should be synced into the docker-sync created container before we run our main app. It may take some time if we have a large codebase, but after that, subsequent syncs will be fast (at least until you delete the sync volumen).
The tool provides two commands to interact with it – docker-sync and docker-sync-stack. The second is just a shortcut to docker-sync start && docker-compose up. I tried both, but prefer to run docker-sync in a separate tab to see what’s actually happening in it.
 
To start syncing, I just need to type:

$ docker-sync start

 
Then, in next tab, the magic happens:

$ docker-compose up
Starting docker-sync-mount
Attaching to docker-sync-mount
docker-sync-mount | 100000+0 records in
docker-sync-mount | 100000+0 records out
docker-sync-mount | real    0m 0.40s
docker-sync-mount | user    0m 0.01s
docker-sync-mount | sys    0m 0.24s

 
Benchmark shows times similar to what we saw when we ran the test inside a Docker container without any mount. That’s a huge improvement, and it will allow you to work comfortably with your new Mac… almost.
Unfortunately, nothing is perfect.

The presented solution has a few drawbacks:

• you need to remember to run docker-sync before running your app
• when you switch branches, in most cases it’ll be wise to stop the app, use “docker-sync clean” and re-sync everything
• sometimes docker-sync may freeze – it’s worth to check it from time to time and just restart it
• because it’s a Mac workaround, it may be wise to have separate docker-compose.mac.yml and docker-compose.linux.yml and run docker compose with “-f” flag keeping base docker-compose.yml as clean as possible

Still reading?

Good 🙂
As a bonus, there is one more trick for those who use xdebug.
Because of its nature, docker-for-mac won’t allow you to simply put xdebug.remote_connect_back=1 in your xdebug.ini file – due to the nature of connection_back, it just won’t work. There are a few workarounds, but the one I use creates 10.254.254.254 as an alias on your loopback device (127.0.0.1). This one command in Terminal will do it:

$ ifconfig lo0 alias 10.254.254.254

You may also use a gist plist to have it fired automatically on each boot. After that your xdebug config may look like this:
 

 
Then, in PHPStorm, you may just set up DBG Proxy like this:

In my sample config, I use idekey and remote_autostart = 0 because I prefer to start the xdebug session from a Firefox addon called “The easiest xdebug”. Chrome has something similar as well.

That’s all!

I hope that this article gave you an idea how to tweak certain issues concerning Docker for Mac. I just want to wish you(/me/us!) that someday this whole wall of text will be obsolete, and future versions of Docker for Mac won’t require hacking to be usable. By the way – all examples presented in this article can be downloaded from github, here are a few more Docker resources for you to check out.

When building a house, you want it to be well designed. It’s great if someone’s tested the solutions you intend to use, and if they’re universal. For example, a standard window is cheaper than windows in various odd shapes and sizes. You don’t have to use expensive, custom-designed hinges to be able to open them. They’re easily available, and fast to repair or replace. It’s very similar with building applications.

Solutions based on properly implemented patterns and principles are universal, easier to understand and develop. There is just one crucial thing. To really benefit from them, you need to know them. In this article, I will list and briefly describe some of the recommendations for object oriented design that are there to help you achieve your goal.

Programming patterns by Gang of Four (GoF)

The Gang of Four are the four authors of the book called “Design Patterns: Elements of Reusable Object-Oriented Software”. The book describes 32 design patterns that provide solutions to common software design problems. Although the authors developed their ideas 20 years ago, all modern frameworks and libraries use the structures described in this famous book. Knowing them gives you a number of advantages.

First, they help to understand the code you’re working with. This includes other developers’ work like frameworks, libraries and implementations. You can meet GoF patterns in all of these places. For example, it’s much easier to understand how some PHP frameworks work with events, if you know the Mediator pattern. Dealing with the Symfony framework forms is easier with knowledge of the Builder pattern.

Other benefits refer to the programming process. With knowledge of the patterns, the developer gets a selection of tried and tested solutions to work with. Solutions that are flexible, extensible, and predictable. How to deal with a category tree? Each element is a category (so it has common methods). However, some of them also collect subcategories. You want all elements to be atomically treated, just like a leaf? This may be a good time to consider using the Composite pattern.
 

SOLID principles

If you want to know how to write code that is easy to maintain and extend over time, SOLID principles are good guidelines to follow. SOLID is a mnemonic acronym for five principles: the First Five Principles of Object-Oriented Programming.
 

single responsibility principle

In a few words, you shouldn’t design a class that has more than one reason to modify it. When modifying a badly designed class for one reason, there is a risk of breaking code that fulfills other responsibilities of that class. When you separate responsibilities into different classes, the danger of poisoning them is much lower. For example, if you have one class responsible for both preparing and printing data, changing how the print looks like can affect the parts responsible for data preparation.

open/closed principle

The second principle says that classes should be open for extensions, but closed for modifications. When extending, well tested code is left unchanged and changes don’t infect other modules that may use the basic code. This makes extending a safer solution. However, it’s possible only when class is designed to allow it. So, while designing and implementing, think which methods and data should be changeable and make them open to changes.

Liskov substitution principle

What’s also important is that child classes should never break the parent class’s type definitions. Clients of the parent class should be able to work with children’s classes automatically, without any code changes. It’s a child class responsibility to adapt to every usage of the parent class. Sometimes, to make everything work, you need another level of abstraction.

interface segregation principle

It is very important to design these abstractions properly. The fourth principle says that interfaces should describe one responsibility. Otherwise, classes that implement this interface will break the single responsibility principle. What’s more, large interfaces may lead to implementing methods that the particular class doesn’t need, but the interface requires.

dependency inversion principle

The last principle refers to the relationships between objects. They should depend on abstractions, not on each other. A system component that is loosely coupled to other elements (by abstraction) is easier to replace. Also, adding new components is simpler.
 

General Responsibility Assignment Software Patterns (GRASP)

If we’re considering which class should take responsibility for a particular action, GRASP patterns can help us make this decision. Written in the form of 9 questions and answers, they lead us to the best solution. In a nutshell, we learn that each class should fulfill one clearly defined responsibility. The responsibility should be granted to the class that holds most information required to fulfill it, so that the resulting class has as few dependencies as possible. It is also important to see what connections it makes. It’s recommended to ‘talk’ only with neighbours and avoid connection chains. I strongly encourage to read all nine principles. It’s impossible to summarize them in just a few sentences.
 

Programming language Standards and Recommendations (eg. PSR in PHP)

These are the standards for particular programming languages that describe various solutions. They range from code-formatting rules to ready-made interfaces and libraries. Each language has its own standards and libraries. Sometimes they’re strongly related and come as a part of the language but sometimes they’re separate. In the latter case, using them isn’t required by the language, but it’s recommended by the community.
For example, the PHP Framework Interop Group (PHP-FIG) developed code formatting standards and abstractions that specify how to deal with cache, logs, request, response and other common structures. In short, elements that don’t have to (and often shouldn’t) be designed from scratch for every implementation.
 

Don’t Repeat Yourself (DRY). Keep your code Simple and Small (KISS). Don’t code what You Aren’t Gonna Need (YAGNI).

Following those three simple rules will make your code easier to understand, develop and test. If you repeat your code, then in a while it will lose consistency. Changes will then be harder to implement. Bugs will start popping up. If your code is difficult to understand – think again. Simple solutions are usually best. And never commit code that no one yet needs. Code that is not used is also not tested. It shouldn’t be a part of your application.
 

Object oriented design: All things considered…

Create clear code that’s easy to develop and test: these should be the most important goals for every developer. Knowledge of object oriented design patterns and principles mentioned above is an essential tool to achieve this objective. Universal, tested solutions save the time of the programmer who writes them, the programmers who use them and, finally, the customers. Thanks to these patterns, development becomes simpler and more predictable.