web analytics

10 Automation Test Questions to Answer before Testing Any Application

Editor’s Note: The following guest post comes from Michelle Xie, a software QA expert and senior automation test developer from Burlington, Canada. This is the first article in Michelle’s multi-part series in automated software testing, and she is going to share with us her perspectives on the most important questions to answer for software automation test.


Automation test in software industry is becoming more and more popular. Many projects nowadays benefit from automated testing because it makes the software testing faster, more reliable and less expensive. If you google this topic, you will find a long list of articles describing the best practices on how the automation test projects can be done. But in the real world, software engineers will be facing a lot of challenges and issues before the projects can be carried out successfully.

In this article, I would like to share my answers to the 10 questions that may help you improve your automation test development.

1.How To Define The Automated Testing Is Successful Or Not?

First of all, automation test is part of the software testing. The automation tests are able to find the issues, it should be a stopper or finder of your application under testing, otherwise, the test is useless and can be deleted. It will be a waste of time to run and maintain such testing.

In addition, a good testing strategy is very important. The same test in different test groups or run at different moment will eventually lead to different test results.

For example, if the test case is to test the login, the test will be more effective by choosing sanity tests instead of regression tests. The reason is for most websites, login is the first step, so login should be the first step for your test too. If the quick sanity test was put in your test pipeline or delivery/release pipeline, it will give you a quick signal that this build/release is broken and the development team can fix it immediately. Using the right strategy may give you quicker indication of test failure and will save time for the development and QA teams.

Having a good strategy is just the first step to define the success of automation tests. The next question is how your test cases, test suite or test group should be scheduled and managed?

Building up the Continuous Delivery(CD) Pipeline is the solution. In my next article, I will have a dedicated section to discuss the CD pipeline in more details.

2.Why Automation Test?

There are a lot of articles describing why automation tests should be performed. I would like to highlight two key points from my view:

  • Automation test is to save the time in product release phase but not in development phase.
  • Automation test is to save efforts for tedious manual testing. With automation test, testers can shift their focus in understanding the test results instead of testing the repeatable test cases. Test result analysis is the key to investigate the root cause of the errors.

3. Automation Developer Should Be Treated As Other Developers?

Many people in the IT industry still think that automation test is as easy as record and play, and even a person without any programming experience can be an automation developer. Actually, it is a big misunderstanding.

Automation test project can be very complicated. Even for the coding work, which is only part of the entire test, sometimes it is complicated enough for many developers because you may need to automate every single step of the tests or delivery pipeline by codes.

4. Is Automation Test Project Really A Development Project?

Automation test development is very similar to a local application or website development.  It must meet your corporation and your team’s standards and requirements.

We manage the automation test project as an individual development projects in Jira, trello and many other tools. Good management is always the key to drive the automation test project to success because clear requirements, good communication and effective feedbacks can help you reduce the risks to your projects.

5. I Am Not A Manager, Why Should I Care?

Before start an automation project, the manager should think about the requirements and expected result of the project.

However, even if you are not the manager, as a member in the automation test development team, you should also be able to clearly communicate with the manager about the requirements and how the requirements should be revised once feedbacks are made from the team.

6. What Is The Structure/Architecture Of The Automation Tests?

Depending on which tools you choose, automation test library and structure should be well-designed based on the needs and the available resources from your company. We always define this behaviour as “tailored” automation tests.

For example, if your company is able to provide you servers, you can create your own selenium grid rather than purchase plan from external service such as browserstack or saucelabs.

7. What Is The Company’s  Release Workflow? First thing to understand

Automation developer should clearly understand your company’s release workflow as well as the test cases.

If you are not clear to the release workflow, then you are not able to decide what is the best time to run your test, how to schedule your automation tests, and what the purposes of your automation test are.

8. What Metrics Are Important?

Metrics or indicators should be defined to monitor the performance of your automation test. For example, your defined metrics should be able to answer the following questions:

  • How to measure the performance of your automation tests?
  • What is the most popular root cause of the failed test?
  • What is the run time for your tests?
  • Can the tests be completed quick enough to meet your company’s need?
  • What development process can make your tests easy to be maintained and created?

9. How To Be Qualified As An Automation Developer?

  • Automation tools such as selenium, QTP and etc
  • Browsers in different OS tool: local browsers, remote such as selenium grid, browserstack, saucelabs
  • At least one programming language such as Java and shell script
  • Test framework such as Junit, TestNG and etc
  • Know front-end such as HTML5, AJAX, javascript(jQuery), CSS and etc
  • Know API and is able to use API
  • Continuous Integration System such as jenkins, teamcity, circleCI and etc
  • Company specific

10. Automation Test Roadmap

Automation developer should be able to create or understand what the roadmap of your company for automation test is.

In this article, I listed some topics, gave some examples and my thoughts on automation test. I will write separated articles to discuss each topic in more details. If you have any questions and thoughts want to share with us, please leave your comment. It is a good chance to communicate and improve our skills. Thanks ECM TechNews for providing me the opportunities to share my thoughts on the wonderful website.

Michelle Xie

You may also like

(Views: 1.3k)
Michelle Xie

Michelle Xie

Michelle Xie is a Quality Assurance Analyst at PrintFleet in Kingston, Canada. Michelle is a passionate software engineer and a software QA expert. Most of her efforts focus on automation test best practices and common tech issues that can be avoided in automation tests, and she hopes to continue sharing her technical expertise through ECM TechNews.

Leave a Reply

Your email address will not be published. Required fields are marked *