How to Set Up an Android Test Automation Pipeline

This article guides you through how we – as adjoe’s WAVE Programmatic team – set up an efficient test automation pipeline on AWS using Appium, which saves our team time and ensures quality. At adjoe we use cloud-based methods to ensure cost-effective and productive test automation.

Why Do We Need This Pipeline? 

We considered the following points when creating this system for our test automation pipeline: 

• device farms can be expensive – especially those hosted by third-party providers
• device farms may introduce additional latency, due to the physical distance between your location and the testing servers; however, emulators running on AWS can minimize this latency, resulting in quicker test execution
• emulators provide greater flexibility in terms of configuring and customizing the test environment
• emulator-based testing can be seamless and tighter if you integrate it into your existing processes – especially if your development workflow already utilizes CI/CD and AWS pipelines

A Test Automation Pipeline with Emulators & Appium 

Ensuring the quality and reliability of your Android applications is crucial. To achieve this, a robust test automation pipeline can be effective. 

In this article, I’ll walk you through the process of setting up a test automation pipeline on AWS using bare-metal Mac instances, Android emulators, and the Appium test automation framework.

1. Select the Right Instance 

The first decision you need to make is to select the right instance to run your Android emulators and tests. 

Initially, we attempted to run everything on a virtual server. However, we encountered a significant roadblock – without hypervisor access to your instance, you can’t harness hardware acceleration instructions on the CPU. This limitation made the emulator painfully slow to start. Frustrated, we decided to switch gears.

Fortunately, our team already had a bare-metal Mac instance in use for iOS CI jobs. While bare-metal instances aren’t the most budget-friendly choice, their high performance is undeniable. Given that we had one at our disposal, we opted to utilize it for our Android test automation pipeline.

2. Prepare Your Instance with the Right Dependencies

To create a successful Android test automation pipeline, you’ll need to prepare your Mac instance with the necessary dependencies and environment settings. It’s a good idea to create an Amazon Machine Image (AMI) for your bare-metal Mac instance to save yourself time and effort when setting up the environment. 

Here’s an overview of what you need to install: 

• OpenJDK: You’ll need a Java Development Kit (JDK) for Android-related tasks
• Android command-line tools: These tools are essential for Android development and emulator management
• Appium: Appium is an open-source test automation framework for mobile applications, supporting both Android and iOS
• Node: You need this to run the Appium server
• Python: Python is used for writing and running test scripts, and pip3 is used to install the necessary Python packages for testing

3. Set Up the Environment 

To make your life easier, you can add the following lines to your .zshrc (or .bashrc) file: 

export PATH="/usr/local/opt/openjdk/bin:$PATH" 
export ANDROID_SDK_ROOT="/usr/local/share/android-commandlinetools"

After that, use the sdkmanager command to install various Android SDK components, including system images, platform tools, and the Android emulator. For example:

echo y | sdkmanager "build-tools;33.0.2" 
echo y | sdkmanager "platform-tools" 
echo y | sdkmanager "emulator" 
echo y | sdkmanager "system-images;android-33;google_apis_playstore;x86_64" echo y | sdkmanager "platforms;android-33" 

Now, you’ll need to create a virtual Android device. In our example, we named it “Pixel_6,” but you can customize this as you want:

avdmanager create avd -n "Pixel_6" -d "pixel_6" -k 
"system-images;android-33;google_apis_playstore;x86_64" --force 

For Appium, you need to install Node.js using NVM and install Appium globally. You also need to install the necessary drivers, such as xcuitest and uiautomator2. If your tests are written in Python, install the required Python packages using pip:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash source ~/.zshrc 
nvm install 16 
nvm install-latest-npm 
npm i npm@8 
npm i --location=global appium 
appium driver install xcuitest@4.13.4 
appium driver install uiautomator2@2.12.0 
pip3 install -r requirements.txt 

After setting up the image and running this script, your environment will be ready for Android test automation, and it will persist after each restart.

4. Run the Test Cases 

Let’s now dive into the process of running the test cases. In our case, we use GitLab CI at adjoe for our CI/CD pipeline. We execute the following script, which orchestrates the entire testing process, including taking screenshots from the Android emulator: 

# Let's set up those beautiful environment variables. 
source ~/.zshrc 

# Run Appium and get its PID. We run it headless in the background. 
appium server & 
export APPIUM_PID=$! 

# Set up error handling to ensure cleanup, even on failure. 
trap 'kill $APPIUM_PID' ERR 
sleep 5

# Start the Android emulator. 
$ANDROID_SDK_ROOT/emulator/emulator -avd Pixel_6 -no-window -no-audio & export EMU_PID=$! 
trap 'kill $EMU_PID; kill $APPIUM_PID' ERR 

# Give the emulator some time to boot up. 
sleep 120 
$ANDROID_SDK_ROOT/platform-tools/adb wait-for-device 
$ANDROID_SDK_ROOT/platform-tools/adb shell input keyevent 82

We wanted this pipeline to be triggered by other CI jobs. In our case, we need to run tests after each build. Using environment variables, you can specify which job ID’s artifacts you want to use. 

# Download test artifacts from GitLab. 
cd path/to/your/tests 
curl --location --output artifacts.zip 
'https://{$YOUR_GITLAB_HOST}/api/v4/projects/{$YOUR_PROJECT_ID}/jobs/'$JOB_ID'/artifac ts' --header 'JOB-TOKEN: '$CI_JOB_TOKEN'' 
unzip -q artifacts.zip 
current_date=$(date) 

# Set pipefail to keep the script running after the pytest command. If the tests fail, it returns an exit code other than 0. 
set +e +o pipefail 
mkdir -p test_report 

# Run the test cases and generate an HTML report. 
pytest example_test.py --html="test_report/all_android_tests_$current_date.html" --self-contained-html 
set -e -o pipefail 

# Clean up and ensure all processes are terminated. 
kill $APPIUM_PID 
kill $EMU_PID 

For a more detailed explanation of how to write test cases using Appium, our team has published another article. You can check out how we work with automation testing frameworks for our WAVE SDK.

With this pipeline in place, you can automate your end-to-end tests for Android applications and generate comprehensive test reports, including screenshots of the user interface.

And Here Are the Results

After setting up your pipeline, it is time to see the fruits of your labor. You can use various CI/CD tools, but for this example, I will show you what it looks like on GitLab CI/CD.

You can see in the artifacts that Appium generates a test report and puts all the screenshots into the related folder.

This is how a successful run looks like on GitLab CI/CD.

The output is a little bit long since we are not silencing anything, but you can configure it how you see fit.

Setting Up This Automation Pipeline Is Worth the Effort

Setting up a test automation pipeline on AWS with Android emulators and Appium may seem complex, but the benefits in terms of quality assurance and time savings are well worth the effort. 

Our team believes that this is a great way to start writing your Android test automation pipeline, since it does not require actual devices, and you can easily set it up on cloud environments.