Mobile Application Testing | Challenges and Solutions
With the phenomenal rise of Mobile Applications in past few years, most of the businesses have included them at some stage of their process. This increasing popularity has made it mandatory for organisations to put extra stress on Mobile Application Testing.
In this article, we are going to discuss about challenges and solutions in Mobile Application Testing.
Some of the commonly faced challenges are:
- Unending list of mobile devices
- Mobile Device and OS combination
- App Types
- Screen Sizes
- Network Speeds
- Automation
- Usability
- Performance
Unending List of Mobile Devices
Currently there are over hundred mobile device manufacturers selling over thousand’s of different mobile devices. This makes the list of total mobile devices to cover for cross device testing unrealistically large. Customer base is also spread across the newly launched device to the models which have been discontinued by the manufacturers.
How to proceed in such scenario?
It is almost impractical to have each and every mobile device used by the customer due to availability and cost of the devices. Better approach in such situation is to focus on top 5–10 mobile devices based on the current active user base of the product. If your company is using analytics tools such as Fabric, Google Analytics etc, you can get this information relatively easy. In other cases, you have to take help from the data of the recent surveys. This decision can also be taken on the basis the expected customer segment.
For example: In India, an app for a food delivery guy will most certainly run on an android based mobile device under the range of Rs. 15000. Now there is minimal requirement to cover premium mobile devices for testing.
Mobile Device and OS combination
Considering top 5 mobile device manufacturers, close to 100 devices have been released among them in last 5 years. Add to that, the OS versions and subversions release every year. Pheww!!!! too much for 1:1 mobile device and OS version mapping.
List of top device manufacturers of the world.
List of most used iOS versions
List of most used Android versions
How to proceed in such scenario?
Data is your best friend here. You can refer to the analytics tool used by your organisation and create plan for the most active devices and OS versions your customers are using. Then you can upgrade and manage your mobile device inventory accordingly.
For example: Let’s consider iPhone and iOS combination. As per the data, the mostly used iPhones with the market shares are:
- iPhone XR with 10 % market share
- iPhone 8 with 9% market share
- iPhone 6s with 8% market share
Mostly adopted iOS versions with market share are:
- iOS 14.3 with 37.53 % market share
- iOS 14.2 with 30.19 % market share
- iOS 12.4 with 5.22 % market share
Better approach is to have 1:1 mapping between the listed iPhones with listed iOS versions.
App Types
There are four types of Mobile Applications:
- Native Application
- Hybrid Application
- Web Application
- Progressive Web Application
Screen Sizes
Screen sizes are very important aspect for User Interface and User Experience Testing. Generally Mobile Applications are designed and built with a single screen size as reference. But the UI components that looks seamless on one screen size might break on other one. So it becomes necessary to test the Mobile Application on different screen sizes before giving go ahead for completion of testing.
How to proceed in such scenario?
Since there are 4 major mobile device screen size category, testing on at least one device per category would be sufficient in most cases to identify any major UI break on Mobile Application.
Incase of testing on iOS following devices can be picked as per screen sizes:
- 0" — 5" size: iPhone 7, iPhone 8
- 5" — 5.5" size: iPhone 6S+, iPhone 8+
- 5.5" — 6" size: iPhone X, iPhone XS
- 6" — 7" size: iPhone 11
Network Speeds
People use Mobile Applications at different places, different time. While sitting at home or office. While travelling to some place via road. People who commute via Metro train use their mobile devices majorly throughout this journey. These are real time scenarios in which Mobile Application will be used by the customers across varying Network Speeds. It becomes important to test the behaviour of Mobile Application across different network speeds.
Network speeds as per different cellular network bands:
How to proceed in such scenario?
All the important flows of the Mobile Application such as App Launch, Most important service of the app etc should be tested across all the network bands at least once.
Incase of Wifi network connection, any proxy server can be used to threshold internet speed to replicate similar network conditions . Some popular proxy servers are: Charles Proxy, MITM Proxy
Automation
Automation testing in always very exciting topic, specially when it comes to Mobile Application Testing. It has great practical usage and that is why it is increasingly becoming a must have for project.
For example: As we discussed above, the combination of various device models and OS versions still requires a number of devices to be tested. And performing regression on all of them? Manually?? It is practically impossible to do. But mobile automation suite can do it daily. This is one of the many advantages that makes it an important asset.
But considering the various challenges faced while performing Manual testing on Mobile Application, there are various things to understand before starting with Mobile Automation Testing:
1. Finding the right Automation tool
There are various automation tools and framework available in the market. Each one having their own pros and cons. Selecting an automation framework or tool varies from case to case.
For example:
On a very high level, if the Mobile Application to be automated has to be tested on different platforms such iOS, Android and Windows then Appium is better choice.
If Mobile Application to be automated is based on Android only, then choice can made be between Appium or Google’s native Espresso.
If you want to understand more about Appium, please refer to my article on Appium Architecture
2. Running your tests on Emulator or Real Device
This choice is mostly driven by the budget allocated towards automation testing. It is always better to run mobile automation test suites on real devices. But the cost of purchasing and maintaining the devices can be quite significant. Specially in case of iPhones or premium Android Phones such as Google Pixel or flagship Samsung phones.
Thus it becomes important to take decision whether mobile automation test suite should be run on Emulators or Real Device.
3. Cloud Mobile Automation Services
There are various cloud based mobile test automation service providers such as AWS Device Farm, BrowserStack. These platforms are very useful for scenarios where the requirement is to avoid maintaining a physical device lab on the office premises.
If you want to understand more about AWS device farm, please refer to my article Appium Test using AWS Device Farm
Usability
Usability is an important aspect to attract and retain users. The few things that user will notice while using the application are:
- How easy to use it is.
- How much attractive User Interface is. Images and texts are properly aligned. Their size should be adjusted dynamically according to the screen size of the devices.
- The features are easy to access and appropriate demo’s are given to the new user when they use the app for first time.
- The navigation between pages are smooth.
Performance
Performance is an important aspect to retain the user. If the application is sluggish then no matter how good the features have been developed, users will not use them. Some of the points to consider for performance of application:
- Mobile Application Launch Time should be less.
- Time to render screen should be less.
- Mobile Application should not slow down in case of resource heavy activity such as Downloading, uploading etc.
Summary
- Among the available mobile devices, top 5–10 mostly used devices should be selected for device coverage.
- Analytics data should be used to decide mobile device and OS version combination.
- Atleast one device per screen size range should be included in testing.
- Different network speeds are very important to ensure app performance for real user.
I hope this article has been helpful to the enthusiasts looking to explore Mobile Application Testing.
Take care. Keep Learning. Stay SAFE.
