A few weeks ago, as the due date for filing tax returns here in India drew near, I barged into my bank to perform the annual ritual of obtaining the savings interest certificates. Soon, it was evident to me that I was not the only one treasuring to perform this ritual in the last couple of days. So, I waited and waited long to be served. I navigated from one window to another getting redirected trying to figure out how to go about getting the job done. Staring at the clock wasn't helping as the threat of not accomplishing the task within time was getting real. Finally, I decided to hit the 'Logout' button and walk into the nearest branch to talk to the friendly branch manager!
Yes, so far, I was trying to use the bank's mobile application and unfortunately could not get anywhere near my goal. In a way, a lot of the problems, we have with mobile applications are reminiscent of the problems we had with desktop applications. As the mobile application development environments get more sophisticated and allow for the development of rich mobile applications, using fancy toolkits and libraries, what becomes critically important is to build rich capabilities with equally rich quality. I like to say “with great development power comes great quality responsibilities!”
The mobile applications continue to have the imperatives of test early and test often as did their desktop based ancestors. All the traditional facets of quality metrics like functionality, performance, security, etc are still relevant, maybe even more than before, for this new and exciting channel. The shortened development cycles for mobiles and even more shortened life of a given version of the application coupled with frequent updates make automation testing necessary. The choice of an automation testing tool is very important in ensuring quality in the application developed. There are a variety of mobile test automation tools with varying capabilities.
As you undertake the task of testing mobile applications, there are a few things to consider.
As you are developing your application, you want to be able to regularly keep tab on the quality with respect to code units, functionality, performance, security, etc so that testing doesn't become an onerous effort at the end of the development cycle. This is most convenient if, from your development environment, you are able to also create, author and execute your test-cases against the application you are developing. This will also allow the issues with the application to be reported directly into the development environment so that it can be tracked and addressed.
As you are building the user interface for the mobile application, chances are that the middle-tier and back-end services are probably defined but not yet implemented. For example, in the case of a hotel booking application, the user interface might be a web application that is being developed and the middle-tier or back-end might be a web service that retrieves this information from a database that is not yet available. Consider leveraging virtualized services to make headway into your testing efforts where the virtualized service may provide the required information to the user interface. Once the real web service is available, it could replace the virtualized service.
Emulators vs Real Devices
While your favorite mobile platform SDKs all provide emulators, the performance fares way below expectation, not to mention some missing capabilities. It is always handy to have a real device around to quickly run regression test-cases. Meanwhile, the full suite of tests could still be running on an emulator. There is hence an implicit assumption that the mobile test automation tool support is agnostic of emulators and real devices.
Native vs Hybrid vs Pure Web
A very good chance is that you start with a pure web application and eventually migrate your application to a hybrid one to leverage some of the device capabilities. For example, the need to say upload the location information of a mobile or other such native resource might require the application to be converted to a hybrid application. It’s also possible that you start with a purely native application and move towards hybrid to leverage HTML5 for cross-platform support. For example, the initial requirement might have been to develop an application for Android platform only and hence the choice of native made sense to build a rich user interface. But as the requirement to add support for iOS platform is added, it might make sense to convert the application to a hybrid one so that two separate implementations may be avoided. It’s also likely that you have a couple of different formats of the application to be built. Through these, you want to adopt a tool that supports all these types of application for testing.
Ease of Test Maintenance
While maintaining the quality of the application being developed, a lot of time is typically spent in maintaining the quality of the test-cases created. As the application evolves, it is beneficial to make sure that the effort to update, fix and maintain a test-case does not outrun the effort to develop the application. There is one school of thought that suggests throwing away test-cases as they get older. While this might be good strategy for legacy code that is either not exercised anymore or not known to have issues, doing so during development cycle and regression phase might prove to be too expensive. Also, as the development cycles shrink and the time to market gets more and more critical, time spent in this phase is best minimized.
Subscribe to forums
Finally, there are many others, just like you, trying to build applications and making sure that they are building quality applications. Consider joining and contributing to a forum on the topic. You often get answers that might already exist or emerge from a discussion.
The mobile test automation tools are evolving and the capabilities are evolving fast. There are a lot of exciting things coming with respect to automation. In my opinion, the best future for mobile test automation is for it to become as mundane any other common operation in the IDE. As the world moves towards a continuous delivery model, the ability to perform continuous testing against the application being developed is going to be very important. The testing could continuously happening in the background with an occasional notification of a failure! Applications that can avoid a walk into the bank...
By Ashish K Mathur, STSM, Test Automation Tools Architect, Rational Quality Management & Execution, India Software Labs, IBM India/SA
Disclaimer: "The postings on this site are my own and don't necessarily represent IBM's positions, strategies or opinions."