Monday, February 27, 2017

Setting Up Calabash: Page Modelling For Mobile Tests

Welcome back, this is the second post in the series dealing with setting up Calabash. The first post dealt with setting up your environment in preparation for writing the tests. This post will deal with setting up your test solution for page modeling, and some lessons I learned from my first-page modeling driven test.


1. Page modeling comes from Selenium but is a pattern used to model the behaviour of pages to code objects.
2. A specific Calabash directory structure has to exist for an XPlat (cross platform) page modeling approach to a test suite.
3. Cucumber.yaml controls the dependencies and platform specific flow of execution
4. Feature files contain plain language actions which can map to acceptance criteria of a story
5. Shared step files contain code that acts as an interface of how the plain language steps should be executed
6. Page models contain platform specific (iOS, Android) implementation of properties and methods used in shared steps. 
7. It's easier to write iOS tests first, but we need to stick to IDs as automation hooks and not break the page modeling pattern.
8. If we follow the page modeling approach, we can implement tests for the second platform in about 1/3 of the time of the first!

Page Modelling: What Is It And Why Should I Use It?

The page object modeling pattern originates from Selenium and has been widely adopted in GUI (Graphical User Interface) automation. A link to the original Selenium documentation can be found in the links section below. Page Object Modelling (POM) uses an object driven design to describe the behaviors of the page or screen in question. This type of design separates the test from the implementation and allows easier maintenance and higher reusability of code. This last point becomes very evident in the mobile space. Once written, two-thirds of a test which runs against one platform (ex. iOS) can be reused for another platform (ex. Android).

How Do I Set Up My Solution For Page Modelling and XPlat Execution?

The default calabash feature generation command ("Calabash-iOS gen") should set up your directory structure, without anticipating page modeling or cross platform (XPlat) execution. This means that you should end up with something that looks like below.
Default generated Calabash-iOS directory
But to prep our solution for XPlat (Cross platform) execution, we need to add a few things. Remember, the goal of page modeling is to reduce maintenance and increase code re-use.  We will need some shared elements and some that are platform specific.

Step 1: Generate An Appropriate Directory Structure

The XPlat page modeling approach in Calabash assumes that both iOS and Android tests execute a very similar test flow, and so share features and step definitions. You will see that any ".feature" or "_steps.rb" files will not be defined in platform specific folders. 

Your directory structure should follow the following breakdown
  • Features
    • android
      • pages
        • AndroidPageModel.rb
      • support
        • app_life_cycle_hooks.rb
    • ios
      • pages
        • iOSPageModel.rb
      • support
        • 01_launch.rb
    • step_definitions
      • shared_Steps.rb
    • support
    • features.feature
My test solution layout

Test Execution Flow

Upon executing a test command (for example "Bundle exec cucumber -p ios"), cucumber decides which pieces of the solution are necessary for inclusion, and then which test platform to execute against. We will now take a look at which pieces in the solution (directory) structure control this flow.

Important Test Execution Flow Concept: Cucumber.yml

Scope: Shared between iOS and Android
Purpose: Define which files to include in test flow execution
Details: This file uses the "-r" parameter to include files and directories in the compilation. It is important to include only the platform specific files for test flow execution. Ex. Do not include any "ios" specific pages in the android profile and vice versa. 

Important Test Execution Flow Concept: Feature File (X.feature)

Scope: Shared between iOS and Android
Purpose: Define feature flow in domain specific language. 
Details: This file reads in plain English, but needs to match up to step definitions. Each one of the scenario defined steps need to be matched to a step definition.

Important Test Execution Flow Concept: Shared Steps Definition

Scope: Shared between iOS and Android
Purpose: Define what each step means. Acts as an interface, which will later have to be implemented by the page models (individually on iOS and Android).
Details: The shared steps file acts as an interface which defines how the behaviors necessary to execute the actions from the feature file will be executed. For example, carrying out the action "Given the app has launched to the tutorial" means waiting up to 30s for the TutorialPageModel to load. 

NOTE: In order to maintain separation of concern with respect to platform specific implementation, all steps in the shared steps have to be custom written steps. A step definition file which is used for an XPlat approach cannot contain platform specific canned calabash steps.  

Important Test Execution Flow Concept: Page Model Files

Scope: Specific to iOS and Android
Purpose: Implement the platform specific actions that are defined in the shared steps.
Details: Each page object file carries out the actions that the shared steps defined through an implementation of methods. The page object files take advantage of the Calabash API (one for iOS and one for Android), to carry out the actions defined in the shared steps. The page object files need to inherit from their respective base classes (ABase, IBase) in order to be recognized as page objects.

Step 2: Write Shared Features

Shared features provide a plain English definition of how the application feature should behave. These steps have to be written in the "Given When Then" format, in order to define what the expected result of an action should be. Shared features should be able to be written very early in the lifecycle of the tests, and even the application under test. My team has seen success in educating the entire team, with respect to how to write proper acceptance criteria, for features in a "Given When Then" format, since those criteria translate very well into feature files in Calabash. The Agile Alliance has a great write up on using the "Given When Then" format for acceptance criteria to which a link is included in the links section. Additionally, since these criteria are written in plain English, any member of the team who is familiar with the purpose of the feature, can write them.

Step 3: Write Shared Step Definitions

As a reminder, shared step definitions provide an interface of what it means to carry out the actions necessary for the behaviors defined in the feature definition. This means that some code is written into this file, and it is not 100% readable, but at the same time, the code implemented here is still not platform specific. A shared step definition used for XPlat Page Object Modelled test suites requires the use of all custom steps, to ensure that no platform-specific steps that cannot be shared between both implementations exist. 

Step 4: Implement Step Definitions Through Page Models on iOS

During compilation of my test suite, I chose to implement iOS tests simply because a few of my friends from the team had already compiled tests for Android.

Choosing this approach, turned out to be a good one, as it saved me a bit of upfront work. I knew going into this project that I wanted to stick to the approach of using IDs for both platforms (iOS and Android) as automation hooks. For the iOS platform, Calabash uses the "AccessibilityID" property as the default automation hook. Now I don't know about your app, but mine did not contain many accessibility ids.

So after wrestling with XCode a bit, I was able to add them to my application under test and implement a pattern in my page objects of using the id field as the default automation hook. Because of this approach, in general, I was able to use the API call which looks for "marks" on properties retrieved by Calabash. This API call (documented here and referenced in the links section under the API Query Syntax) filters objects by ids, contentDescription or text. Simply, it looks for a wide variety of hooks and given I ensured to enter IDs on iOS it worked for that platform. The beautiful thing was that on Android because most IDs are essential to the application flow, I was able to maintain very similar page models, and just change the IDs!

IMPORTANT NOTE: Execution on iOS Requires Linking of Calabash To A Debug Version Of The Application Under Test!

I will mention this in a subsequent post about debugging and actual execution of the tests, but it is worth noting that before you can actually run Calabash-iOS tests, you will need to link the Calabash framework to your application. This is necessary because, at its essence, Calabash is essentially a person in the middle which is allowed to make calls to the application under test through explicit permissions. A tutorial on how to link iOS Calabash to your application can be found here and in the links section below. Without doing this, the tests will not run. 

Step 5: Implement Step Definitions Through Page Models on Android

In Closing...

Implementing the iOS page models (and steps, and features), debugging and testing them locally and on Xamarin Test Cloud, took about a week. 

"Re-implementing" the entire flow for Android...1 day. Mind == Blown. This process really sold me on the value of using proper automation hooks (ids) and page modeling. I know, I know, it doesn't seem possible. There's only one way to experience this joy...try it! 


No comments:

Post a Comment