Saturday, December 31, 2016

Automation Delivery Channels: Part 1...What is it?

Hello, friends and Happy Holidays!

Over the past couple months, I've been dipping my toes into the mobile automation sphere. As evidenced by my previous blog posts, my experience is not in this sphere, and so it was an exciting challenge to take on. The reason for this effort was basically caused by our existing automated testing suite for our mobile apps, going stale. While there were automated tests designed, the one thing that was missing from them, caused the suite to become nonfunctional when the person who wrote it, temporarily left the team, which was a stable delivery channel.

What Is A Delivery Channel?

The concept of a delivery channel speaks not to the automated tests themselves, but to how and where they are executed. A delivery channel defines the flow an automated test takes from the moment a trigger executes the automation, to the delivery of test results. A delivery channel outlines the way a test suite is kicked off, where it runs, and how the test results are compiled. The concept of a delivery channel can be applied to most automation testing types. Whether it is for desktop automation, web automation, mobile automation or even unit tests, a stable repeatable way of kicking off, running and gathering results for automated tests needs to exist.

Why Is A Delivery Channel Important?

1. Avoiding Bottle Necks

In order to avoid situations where automation is executed on a machine, or a single device, and controlled by a single person or small group of people, it is extremely important to establish an execution channel very early in the lifespan of any automation effort. It is very easy for a team to get caught up in writing as many tests as quickly as possible, and incur technical debt when it comes to how they will be consistently run. But if one does not think of a consistent effort with respect to delivering the automated tests (execution), then the team will definitely create a bottleneck very quickly. Without having a stable, repeatable way of executing the tests that are hot and new, a team will not see full value from the tests. A team will get into a cycle of using someone's time (and devices or individual machines) to run the test suite, which inevitably sets one person or a small group of people to be the specialist. While this is good for the individual, it is not good for the team, since people do crazy people things, like switch teams or go on maternity leave.

2. Avoid Test Refactoring For The Wrong Reasons

While re-factoring a test is really great and should be done on a regular basis, refactoring tests for the purpose of scale is an avoidable exercise. If you are keeping in mind that the tests you are writing need to be executed at scale, since that is how you are executing them from the earliest stages of their lifespan, then you will naturally avoid designing them in a way that relies on a state of a single your computer. Having a delivery channel available for use from the earliest stage of an automated testing effort allows for this type of mindset!

3. Quick Feedback Loop

A flexible delivery channel also provides speed in execution. It is important to note that a stable delivery channel may not speed up individual test execution, but if implemented correctly, provides a consistent controllable way to scale the amount of machines or devices used during test execution. The more devices available, the quicker a test suite can run. Additionally, greater flexibility in device or machine choice provides your team with the ability to run tests on different devices as necessary, during bug triage.

4. Consistent Environment

A delivery channel also provides a consistent environment to execute tests more than once. This is important since, in some situations, you may need to repeat the execution of a test or suite, such as in cases where your tests fail and you want to re-run them to replicate the failure scenario. Because an automation delivery channel is controlled by configuration, it is repeatable and stable, which provides a way to ensure the tests you've run can be repeated on the same environments. More importantly, your team will always know what to expect with respect to the environment. 

Right about now, you're probably thinking (because I know what you're thinking right? ;)) Ok, so what. I'm convinced this is an important theory, and I'd like to know how. Well, you're in luck, in part 2 of this post, I'll give you examples of how I've done this in my journey in the past couple months. I'll show you how to implement a delivery channel for mobile automated tests. I'll give examples of how I've done this for Appium and Xamarin UI Tests, which are run on two different device clouds, all using TFS as the build and trigger platform. Stay tuned!

No comments:

Post a Comment