adjoe Engineers’ Blog
 /  Android  /  Android Test Automation
decorative image with green background and polkadots
Android

How to Set Up an Android Test Automation Pipeline

This article guides you through how we – as adjoe’s WAVE Programmatic team – set up an efficient test automation pipeline on AWS using Appium, which saves our team time and ensures quality. At adjoe we use cloud-based methods to ensure cost-effective and productive test automation.

Why Do We Need This Pipeline? 

We considered the following points when creating this system for our test automation pipeline: 

  • device farms can be expensive – especially those hosted by third-party providers
  • device farms may introduce additional latency, due to the physical distance between your location and the testing servers; however, emulators running on AWS can minimize this latency, resulting in quicker test execution
  • emulators provide greater flexibility in terms of configuring and customizing the test environment
  • emulator-based testing can be seamless and tighter if you integrate it into your existing processes – especially if your development workflow already utilizes CI/CD and AWS pipelines

A Test Automation Pipeline with Emulators & Appium 

Ensuring the quality and reliability of your Android applications is crucial. To achieve this, a robust test automation pipeline can be effective. 

In this article, I’ll walk you through the process of setting up a test automation pipeline on AWS using bare-metal Mac instances, Android emulators, and the Appium test automation framework.

1. Select the Right Instance 

The first decision you need to make is to select the right instance to run your Android emulators and tests. 

Initially, we attempted to run everything on a virtual server. However, we encountered a significant roadblock – without hypervisor access to your instance, you can’t harness hardware acceleration instructions on the CPU. This limitation made the emulator painfully slow to start. Frustrated, we decided to switch gears.

Fortunately, our team already had a bare-metal Mac instance in use for iOS CI jobs. While bare-metal instances aren’t the most budget-friendly choice, their high performance is undeniable. Given that we had one at our disposal, we opted to utilize it for our Android test automation pipeline.

2. Prepare Your Instance with the Right Dependencies

To create a successful Android test automation pipeline, you’ll need to prepare your Mac instance with the necessary dependencies and environment settings. It’s a good idea to create an Amazon Machine Image (AMI) for your bare-metal Mac instance to save yourself time and effort when setting up the environment. 

Here’s an overview of what you need to install: 

  • OpenJDK: You’ll need a Java Development Kit (JDK) for Android-related tasks
  • Android command-line tools: These tools are essential for Android development and emulator management
  • Appium: Appium is an open-source test automation framework for mobile applications, supporting both Android and iOS
  • Node: You need this to run the Appium server
  • Python: Python is used for writing and running test scripts, and pip3 is used to install the necessary Python packages for testing

3. Set Up the Environment 

To make your life easier, you can add the following lines to your .zshrc (or .bashrc) file: 

export PATH="/usr/local/opt/openjdk/bin:$PATH" 
export ANDROID_SDK_ROOT="/usr/local/share/android-commandlinetools"

After that, use the sdkmanager command to install various Android SDK components, including system images, platform tools, and the Android emulator. For example:

echo y | sdkmanager "build-tools;33.0.2" 
echo y | sdkmanager "platform-tools" 
echo y | sdkmanager "emulator" 
echo y | sdkmanager "system-images;android-33;google_apis_playstore;x86_64" echo y | sdkmanager "platforms;android-33" 

Now, you’ll need to create a virtual Android device. In our example, we named it “Pixel_6,” but you can customize this as you want:

avdmanager create avd -n "Pixel_6" -d "pixel_6" -k 
"system-images;android-33;google_apis_playstore;x86_64" --force 

For Appium, you need to install Node.js using NVM and install Appium globally. You also need to install the necessary drivers, such as xcuitest and uiautomator2. If your tests are written in Python, install the required Python packages using pip:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash source ~/.zshrc 
nvm install 16 
nvm install-latest-npm 
npm i npm@8 
npm i --location=global appium 
appium driver install xcuitest@4.13.4 
appium driver install uiautomator2@2.12.0 
pip3 install -r requirements.txt 

After setting up the image and running this script, your environment will be ready for Android test automation, and it will persist after each restart.

4. Run the Test Cases 

Let’s now dive into the process of running the test cases. In our case, we use GitLab CI at adjoe for our CI/CD pipeline. We execute the following script, which orchestrates the entire testing process, including taking screenshots from the Android emulator: 

# Let's set up those beautiful environment variables. 
source ~/.zshrc 

# Run Appium and get its PID. We run it headless in the background. 
appium server & 
export APPIUM_PID=$! 

# Set up error handling to ensure cleanup, even on failure. 
trap 'kill $APPIUM_PID' ERR 
sleep 5

# Start the Android emulator. 
$ANDROID_SDK_ROOT/emulator/emulator -avd Pixel_6 -no-window -no-audio & export EMU_PID=$! 
trap 'kill $EMU_PID; kill $APPIUM_PID' ERR 

# Give the emulator some time to boot up. 
sleep 120 
$ANDROID_SDK_ROOT/platform-tools/adb wait-for-device 
$ANDROID_SDK_ROOT/platform-tools/adb shell input keyevent 82

We wanted this pipeline to be triggered by other CI jobs. In our case, we need to run tests after each build. Using environment variables, you can specify which job ID’s artifacts you want to use. 

# Download test artifacts from GitLab. 
cd path/to/your/tests 
curl --location --output artifacts.zip 
'https://{$YOUR_GITLAB_HOST}/api/v4/projects/{$YOUR_PROJECT_ID}/jobs/'$JOB_ID'/artifac ts' --header 'JOB-TOKEN: '$CI_JOB_TOKEN'' 
unzip -q artifacts.zip 
current_date=$(date) 

# Set pipefail to keep the script running after the pytest command. If the tests fail, it returns an exit code other than 0. 
set +e +o pipefail 
mkdir -p test_report 

# Run the test cases and generate an HTML report. 
pytest example_test.py --html="test_report/all_android_tests_$current_date.html" --self-contained-html 
set -e -o pipefail 

# Clean up and ensure all processes are terminated. 
kill $APPIUM_PID 
kill $EMU_PID 

For a more detailed explanation of how to write test cases using Appium, our team has published another article. You can check out how we work with automation testing frameworks for our WAVE SDK.

With this pipeline in place, you can automate your end-to-end tests for Android applications and generate comprehensive test reports, including screenshots of the user interface.

And Here Are the Results

After setting up your pipeline, it is time to see the fruits of your labor. You can use various CI/CD tools, but for this example, I will show you what it looks like on GitLab CI/CD.

screenshot of artifacts in GitLab CI/CD

You can see in the artifacts that Appium generates a test report and puts all the screenshots into the related folder.

screenshot showing a successful run on GitLab CI/CD

This is how a successful run looks like on GitLab CI/CD.

Screenshot of test automation pipeline output

The output is a little bit long since we are not silencing anything, but you can configure it how you see fit.

Setting Up This Automation Pipeline Is Worth the Effort

Setting up a test automation pipeline on AWS with Android emulators and Appium may seem complex, but the benefits in terms of quality assurance and time savings are well worth the effort. 

Our team believes that this is a great way to start writing your Android test automation pipeline, since it does not require actual devices, and you can easily set it up on cloud environments.

Android Developer (f/m/d)

  • adjoe
  • Programmatic Supply
  • Full-time
adjoe is a leading mobile ad platform developing cutting-edge advertising and monetization solutions that take its app partners’ business to the next level. Part of the applike group ecosystem, adjoe is home to an advanced tech stack, powerful financial backing from Bertelsmann, and a highly motivated workforce to be reckoned with.

Meet Your Team: Programmatic Supply

In a competitive adtech market, adjoe stands for greater transparency and fairness for app publishers and advertisers – and a more relevant and enjoyable experience for users. 

It’s exactly for this reason that adjoe’s Programmatic team has built its own mediation platform WAVE. It’s not only driven by a backend application that decides which ad to show but also by our Android and iOS SDKs. These render the ads for the user and provide useful tracking events to the backend.

The WAVE SDK serves millions of in-app ads a day in different formats, and our backend system handles a few billion auctions in real-time every day to help game developers monetize their apps.

Want to be a part of this industry-first adtech solution? Join our discussions, explore implementation, and put your problem-solving skills to the test in our cross-functional Programmatic team!

Our mobile tech stack

Android Studio, Kotlin, Java, Gradle, okHTTP, Wire, gRPC, Protobuf, XCode, Swift, Objective-C, Cocoapods, SPM, Unity, C#, React Native, JavaScript/TypeScript, GitLab CI, Maven, Sentry, Google Firebase.

And we’re always open to new technologies & tools that will benefit the product.
What You Will Do
  • You will develop the WAVE SDK by creating new features and innovative ad formats. The views you build will be seen by billions of mobile users all over the world every day.
  • You will be responsible for cross-platform mobile development by ensuring our SDKs’ compatibility with other development frameworks like Unity, React Native, and Flutter. Thanks to that, our product will be integrated into top mobile games and apps in the industry.
  • You will implement mechanisms to improve the technical stability, performance, and maintenance of WAVE SDK. Your goal is to make it as easy and smooth as possible for app developers to integrate and utilize adjoe’s technologies.
  • You will use your system design knowledge to come up with the optimal solutions compatible with different frameworks and our backend.  
  • You will align with advertising industry standards & best practices to keep our product up to date.
  • You will contribute to our CI/CD pipelines to efficiently and effortlessly distribute our SDK to thousands of app developers.
  • Who You Are
  • You have 3+ years of work experience as a mobile developer working with Kotlin.
  • You have hands-on experience with developing Android libraries and SDKs.
  • You keep up to date with trends in mobile development and are aware of UX & UI best practices.
  • You are open to relocating to Hamburg, Germany.
  • Plus: You have basic knowledge of C++ and working with the NDK.
  • Plus: You have knowledge of React Native (JavaScript), Unity (C#), Flutter (Dart), or similar cross-platform development frameworks.
  • Plus: You have iOS development experience as well.
  • Plus: You have experience with build/deployment tooling (such as Gradle, Maven, and GitLab CI).
  • Heard of Our Perks?
  • Work-Life Package: 2 remote days per week, 30 vacation days, 3 weeks per year of remote work, flexible working hours, dog-friendly kick-ass office in the center of the city.
  • Relocation Package: Visa & legal support, relocation bonus, reimbursement of German Classes costs and more.
  • Happy Belly Package: Monthly company lunch, tons of free snacks and drinks, free breakfast & fresh delicious pastries every Monday
  • Physical & Mental Health Package: In-house gym with personal trainer, various classes like Yoga with expert teachers.
  • Activity Package: Regular team and company events, hackathons.
  • Education Package: Opportunities to boost your professional development with courses and trainings directly connected to your career goals 
  • Wealth building: virtual stock options for all our regular employees
  • Skip writing cover letters. Tell us about your most passionate personal project, your desired salary and your earliest possible start date. We are looking forward to your application!

    We welcome applications from people who will contribute to the diversity of our company.

    We’re programmed to succeed

    See vacancies