Hey there Internet friends! It's me again! For the first quarter of this year, I've committed to helping a few of my team members with some mobile automation, and so the next few posts will be focused around that! So far, coming from a desktop automation background, I've learned a crap ton and solved a lot of quirky problems. I've learned a lot of lessons that I'd like to share with you, and selfishly document for my own future use :) Today's post will be focused on how to setup your environment, and which tools worked best for me. The goal was to get from zero to login tests, for iOS and Android, using Calabash...first step: Figure out how to setup the environment.
1. Get a Mac , it's best BRO..but really it helps a lot for debugging iOS and Android. Also if your team lets you, you can add accessibility IDs (automation hooks) via XCode!
2. Get a physical Android device. Dealing with emulators and Google services is a pain.
3. Get VS Code for editing. Light weight, extensible due to a great plugin model.
4. Sign up for a cloud testing environment and use it to ensure your tests work on devices that are not your own.
5. Use RBEnv to install Ruby 2.3.1 (for Xamarin Test Cloud)
6. Use Bundler to install Calabash
7. Setup Calabash dir structure for iOS
Before we begin coding, I'd like to share with you my environmental setup. This is important information, as before we begin , we want to ensure we are setting ourselves up for success and minimal configuration.
Get a Mac
Unfortunately, it's easiest to work with iOS and Android on a Mac. I'm not saying that OSX is bad, or I'm a Windows fanboy, it was just unfortunate that I needed to get another device, given the majority of non-mobile automation I wrote, was in Visual Studio for the Windows environment.
Having said that, while working with Android and iOS tests is theoretically possible on a Windows device, having a Mac gives you the flexibility to work with the source code of the application to do things like link the Calabash-iOS framework or add accessibility id's for automation hooks.
Worked for me!
Get an Android Device
Devices or emulators are necessary for script debugging. While Calabash-iOS works really well with the iOS simulator from XCode (another reason to have a Mac), emulating an Android environment is not exactly easy if you need to call into Google services. If your application does not make any calls to Google services (ex. maps) then an emulator could work for you. But to be safe, I advise you to get a physical Android device and use that for script debugging. It will save you a lot of configuration effort and environmental debugging.
Thanks for lending it to me Mobile Team!
Choose an Editor
Calabash scripts are usually executed through the console, but need to be written in an editor of some sort. Some of my colleagues like to use Android Studio, or Sublime, but I chose Visual Studio Code. Up until this mobile testing thing, I spent the majority of my time in Visual Studio proper, and found that the keyboard shortcuts translated nicely. Additionally, VS Code (Visual Studio Code) provided a rich plugin environment, through which I installed Ruby specific plugins that aided the dev cycle through things like IntelliSense. VS Code also installed very quickly and launches very quickly, so overall I really like it as an editor. Links can be found below.
Simple, fast, editor
Get a Cloud Testing Service...ASAP
As far as I'm concerned, an automation script does is not done until it runs on a foreign environment. Only then are you sure whether or not you've incorrectly incorporated environment specific variables, which allow the script to only run on your machine/device/etc. In order to prevent this habit from forming, it is essential to have access to a lab of devices. Some teams have access to physical device labs in house. My team has started using a cloud environment to perform the same role.
A quick plug for Xamarin Test Cloud. It's relatively cheap (so far) (about $100/month with a 30 day trial) and performs really well. It is quick, and provides all the logging you'd want (device logs, script logs, screenshots). See the links section for a reference to their site.
I cannot stress enough that before you consider a script complete, you need to ensure it runs on an environment that is not your machine! I run all mine on XTC (Xamarin Test Cloud). That step is part of my personal definition of done.
"worked on my machine" is NOT GOOD ENOUGH
So now that we've discussed the environmental setup, we should get into some script specific setup pre-requisites.
RBEnv is a Ruby version manager. It is important to realize that on Mac OSX, Ruby is installed by default, but the version of Ruby by default impacts your entire system, so it's not recommended that you mess with it and its' Gems. Enter RBEnv. Through the use of this tool, you can control which version of Ruby you are running per each environment you are working with. By installing a setting a specific version of Ruby for your environment when developing scripts, you are ensuring that the version specified is what is used for your Calabash environment. This is important because it keeps your version of Ruby for Calabash and its' Gems separate from your main Ruby install. This separation ensures that if you mess up and uninstall or accidentally upgrade Gems in your Calabash environment, you will not impact the rest of your machine. Additionally, some script specific dependencies (such as Xamarin Test Cloud) run a specific version of Ruby (2.3.1), so you will want to control which version is installed for the environment to stay consistent. I've provided a link on RBEnv setup from the Xamarin website below. Please note, in the link that describes the install of RBEnv and specific Ruby versions, there exist two ways of installing Ruby for Calabash.. a "non Experienced dev" and "Experienced Dev" path. Unfortunately after trying to figure out why the non experienced dev way was not working for me (Calabash Sandbox), and a bit of googling, I realized that that way is no longer working and will not be supported going forward. So you have take the hard way. Once you've completed the RBEnv Ruby setup, you should be ready for Bundler.
Sorry...easy way not supported anymore :(
"Bundler provides a consistent environment for Ruby projects by tracking and installing the exact gems and versions that are needed. Bundler is an exit from dependency hell, and ensures that the gems you need are present in development, staging, and production. Starting work on a project is as simple asbundle install." (http://bundler.io/)
Bundler is Ruby dependency manager, and an easy way of ensuring that all of the environments that your project is running in (your machine, cloud environment, Arthi's machine, Cynthia's machine) are using the same Ruby Gems and the same versions of those Gems. Given you have Ruby installed on your environment, Bundler is really easy to install. The instructions are on the front page and consist of running one command in the terminal (Mac).
The dependcies for your project are controlled by a "Gemfile". The Gemfile has to reside in your project directory, and contains the Ruby Gems necessary to be installed for your project to be executed successfully. I've provided a link to the Stackoverflow article I used to figure out how to create a Gemfile. I've also provided the Xamarin article I used to create the contents of the Gemfile for Calabash.
Now that we installed the proper Ruby version using RBEnv, installed Bundler, and created a Gemfile we are ready to setup our Calabash project. The first thing we need to do, is install the necessary Gems. This is very simple to setup. Assuming you are in the directory and a Gemfile exists, all you have to do is run the "bundle install" command in the console. The output of the command should show you which Gems were installed.
At this time, the last thing to do is to create the Calabash directory structure for the tests. Run the Cucumber command "Calabash-ios gen" to create a skeleton for tests. This command should create a directory structure which includes a feature folder, and some others. If the command successfully executed, then you should see the feature folder in your test directory.
Calabash Test Directory Structure
Now that we've learned what we need to get up and running with Calabash on iOS and Android, we can start getting familiar with the solution structure. That is when things start to get interesting :)