Insight view: Perfect Test Automation

Nothing is perfect in the world, except a football skill of Leo Messi. Yeah, maybe not always but in most cases it’s impossible to create an ideal process. But I’m not scared to talk about my vision of a perfect test automation in a context of a software development. My work experience gives me confidence in this question. So I’m starting!

For whom this article is written

Before I have started write this article I thought about readers for whom this text can become helpful and interesting. Here is my target audience:

• Managers / Product owners. This category is a first in my list, just to inform managers and product owners about pre-requirements which need to be complied before they want to introduce a test automation on a their project.
• Automation engineers. Because it will be informative to read in a formal and declarative form about a job which automation test engineers do.
• QA Engineers. Automation of testing as a continuation of a manual testing should be clear for regular QA Engineers.
• Haters. Regardless of what I’ll write in this article – haters gonna hate =)

Place of automation testing in a software development

A software development in nowadays has a lot of different forms. It can be expensive if it is an enterprise system for a giant corporation, it can be cheap or even free if it is a startup or an open source. A development team can be distributed or located in one office. Different development methodologies can be used. Regardless of all these factors, primarily software development it is a process of code production, which solves some business problems.
At first glance it is simple, just produce a code to obtain a software. But as a rule, the process is much more complex, because it consist of numerous components. Let’s enumerate some of them:

• internal communication (between developers)
• external communication (with customers / PMs / product owners)
• deployment process
• manual testing
• automation testing
• bug fixing

As you see a development process is a combination of various activities. Of course writing of code is in the center of the process. All other activities serve to make the production of code more effective and easier. This is a good place to stop and to say something about testing. Testing gives first independent assessment to a code quality. I mean a regular manual testing. A QA team performs some type of testing and creates bug reports as a feedback for completed job. It’s the most simple scheme of QA and Dev teams interaction.

A testing becomes really hard and annoying for a QA team when a number of tests grows up, modules which need to be tested include not trivial verifications or it need to be performed very frequently. You can ask a question to any QA engineer, what kind of job he hates more than anything else. I bet the answer will be: a job which need to be repeated from day to day. This circumstance is a reason why automation of testing should be in a software development process.

What test automation should do?

The first and the main aim of a test automation is to make life of QA Engineers easier. All other goals are derivatives from the first one. For example an automation of a regression test suite guarantees that QA team won’t be tormented during its manual execution. But this statement is justly only in a QA world. And now I’ll try to convince you in this.

In your opinion, does it matter for the manager how hard is the QA job? Probably in 95% cases the answer is NO. And this is normal, because managers operates with another abstractions. Hence the main goal of test automation for managers and product owners definitely will be time economy and high quality of verifications.

With developers everything is more simple. They know that auto tests help to keep their code in a stable work state.

That’s why test automation has to do different tasks for different categories of project members.

When a test automation make sense?

I have already talked a little about the situations when test automation is really need to be applied on a project. In this section I’ll examine all cases in details.

The first case is about routine work. I know many situations when day of QA Engineer starts from execution of standard set of test cases. 5 days per week. Can you imagine that? I believe this is a first real case when test automation should be applied. As a result, free hours of QA Engineer can be spend in a more effective way – writing of new test cases, exploratory testing for detecting of new bugs or morning coffee =)
Regression testing is a good candidate for automation.

The second case is about huge amount of tests. One time I was assigned at large e-commerce project where we have more then 2000 test cases. There were 4 QA Engineers in our team. A full test execution cycle was taking about 2 weeks of hard work. Extra time was spent for verifications of bugs which were fixed.
Definitely such projects need to have some kind of test automation. It will save attention of QAs, otherwise it can be lost within manual execution of huge amount of test cases.

The third case is about a complex functionality. It can be some sort of calculations of some long sequence of actions which need to be done to obtain particular result. In my practice was one project with a delivery service module. And I have to verified there a price of delivery depends on destination point, total amount, customer account and discount rate. Such functionality has too many combinations for a manual testing, so it need to be automated.

When a test automation doesn’t make sense?

A test automation will not heal your project, it just can take a part of QA team responsibilities.

If project consists of chaotic processes don’t try to add automation, because you will get only extra chaos.

If you don’t have manual test cases, idea of automation is really bad because each automation script need to be based on some scenario. Furthermore you won’t be able to map automation test results with manual analogs. In this case sync of manual and automation testing becomes extremely hard.

If you don’t have a test data and a separate test environment for test automation, I’m sure for 95% you will have more headache than profit.

Summary

Test automation becomes a powerful weapon when all pre-requirements are ready. In some sense it is an art. Because very often a testing framework is some kind of software with own business logic, which is aimed to test a main application.