Appium Test using AWS Device Farm

Mobile automation is a familiar term in the tech discussions and practices. It has become desirable to have a good mobile automation setup to continuously monitor and check app’s functionality.There is also growing need to shift the automation tests to cloud services because of the advantage they offer: large variety of devices, flexibility of scaling and no maintenance effort.

While some organisations prefer to setup their own on-premise device labs and can afford to have enough resources to upgrade and maintain it, cloud services have also come with impressive solutions to take this to new level.

In this article, I will be discussing how to setup AWS device farm and integrate it with CI pipeline using Jenkins. For this example, I will be using Java, TestNG and Appium based on Page Object Model approach.

AWS Device Farm

Device farm is app testing service to test Android, iOS and web apps. It is different from other cloud service test platforms like browserstack and saucelabs in terms of approach. In these cloud services, code is run on user’s own machine and interacts with their cloud via web services. Whereas in Device Farm code is uploaded to AWS infrastructure in predefined format and runs there only.

Although this makes Device Farm bit difficult to implement and I am going to talk about these steps in my article.

GETTING STARTED

I hope you already have an AWS account. If not, please create one.

I am starting with dependencies required in the code so that it can be integrated with Device Farm.

1. Modify pom.xml to JAR packaging.

<groupId>MobileAppAutomation</groupId>
<artifactId>MobileAppAutomation</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>

2. Modify pom.xml to use maven-jar-plugin to build your tests into a jar file. Anything present in src/test will be added to the jar.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jar-plugin</artifactId>
    <version>3.2.0</version>
    <executions>
        <execution>
            <goals>
                <goal>test-jar</goal>
            </goals>
        </execution>
    </executions>
</plugin>

3. Add maven-dependency-plugin to your pom.xml file to copy the dependencies to the dependency-jar directory.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <version>3.1.1</version>
    <executions>
        <execution>
            <id>copy-dependencies</id>
            <phase>package</phase>
            <goals>
                <goal>copy-dependencies</goal>
            </goals>
            <configuration>
                <outputDirectory>${project.build.directory}/dependency-jars/</outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

4. Create zip.xml in src/main/assembly directory and add the following code. This xml is used to instruct maven to build a .zip file that contains everything in the root of your build output directory and the dependency-jars directory.

<assembly
        xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
    <id>zip</id>
    <formats>
        <format>zip</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <fileSets>
        <fileSet>
            <directory>${project.build.directory}</directory>
            <outputDirectory>./</outputDirectory>
            <includes>
                <include>*.jar</include>
            </includes>
        </fileSet>
        <fileSet>
            <directory>${project.build.directory}</directory>
            <outputDirectory>./</outputDirectory>
            <includes>
                <include>/dependency-jars/</include>
            </includes>
        </fileSet>
    </fileSets>
</assembly>

5. Add maven-assembly-plugin to pom.xml to package all tests and dependencies into a single file. In later part of this article, this file will be uploaded to Device Farm portal.

<plugin>
    <artifactId>maven-assembly-plugin</artifactId>
    <version>2.5.4</version>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>single</goal>
            </goals>
            <configuration>
                <finalName>Final-Zip-File</finalName>
                <appendAssemblyId>false</appendAssemblyId>
                <descriptors>
                    <descriptor>
                        src/main/assembly/zip.xml
                    </descriptor>
                </descriptors>
            </configuration>
        </execution>
    </executions>
</plugin>

<finalName> is editable and you can chose a name as per convenience.

This plugin uses the preceding assembly to create a .zip file with name enclosed in <finalName> tag to the build output directory every time mvn package is run.

6. Run the following command;

$ mvn clean package -DskipTests=true

A .zip will be created inside your target directory

Optional: In case you want to add screenshots in your tests, you can use the following code snippet.

public boolean takeScreenshot(final String name) {
    String screenshotDirectory = System.getProperty("appium.screenshots.dir", System.getProperty("java.io.tmpdir", ""));
    File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
    return screenshot.renameTo(new File(screenshotDirectory, String.format("%s.png", name)));
}

When AWS Device Farm runs your Appium Java TestNG test, the service sets the following system properties that describe the configuration of the Appium server with which you’re communicating:

• appium.screenshots.dir: Path where the screenshots are saved.
• appium.server.address: Host address of the Appium server.
• appium.server.port: Port on which the Appium server is listening.

These screenshots will be directly used by device farm in your test run report.

AWS Device Farm Web console

1. Go to AWS Management console and search for Device Farm

2. Create a new project. If you have created a project earlier, it will be visible on this page.

3. Give a suitable name to your project and Create Project

4. Create new Run

5. Upload .apk file. You can see your app details once it is successfully uploaded.

6. Select Appium Java TestNg in configure your test section and upload Final-zip-file.zip, present in target directory.

Once file is uploaded, you can select execution environment. I prefer to run my tests on latest available appium version. By default, Device Farm has selected Appium version: 1.14. This can be updated by selecting custom environment and edit(bottom of the command set).

7. Select devices from the set of available devices. Incase you want to use some other device which is not mentioned in the list, opt for create new device pool.
I am using Samsung Galaxy S10.

Note: Prefer to select device with Status as Available and Highly_Available to avoid waiting time.

8. In Specify Device State, you can modulate network and other services of the device as per test requirements. For this article, I am keeping them as default.

9. I am reducing maximum execution time per device to 5 mins to limit my time consumption in case anything goes wrong.

Project is ready. Let’s run our test!!

You can also see logs of the devices by selecting test application and Device(Samsung Galaxy S10). This may take some time to start.

Once the setup process is done, logs will start to appear in this section.

On completion of test suite, the results would be displayed along with video, logs, screenshots(if the code snippet was added to tests) and performance analysis.

Note: Since its Setup and TearDown takes appreciable time, it is recommended to use Device Farm for larger test suites.

So far, we have run test suite on Device Farm. But it would be very tedious to manually perform all these steps, especially when aim is to run the tests daily. To solve this, you can opt for aws-device-farm plugin for Jenkins.

AWS Device Farm integration with Jenkins

1. Install plugin from Jenkins UI via Manage Jenkins and search aws-device-farm. Restart Jenkins after installation.
2. Please go through IAM User guidelines before linking your account with Jenkins plugin.
3. Go to post build option and select : Run Tests on AWS Device Farm

4. Select Project, Device Pool and path to test application.

Once your tests and app is uploaded to AWS Device Farm, your run will start to reflect in Projects section on web console.

On completion of the run, generated artifacts and test result report will be attached on Jenkins. Although it is important to note that the time taken for the whole run to complete was considerable with respect to the tests run in our suite. This makes it economical for running larger test suites only.

Summary

In this article, I have showcased a PoC demo on integration of appium test suite with AWS Device Farm and Jenkins. Though the set up can be time taking but once it’s up and running AWS is very reliable.

I hope this article has been helpful to the enthusiasts who are looking to take their appium automation suite to cloud based platform. Take care. Keep Learning. Stay SAFE.