Thursday, March 22, 2012

The Challenges faced when testing mobile apps


Testing a mobile app is very similar yet different from testing any web based application. At the heart of it, testing the two would follow the STLC yet scope/number of platforms on which the two are tested draws them apart. Normally, to test a web application, the scope of testing is confined to limited browsers; whereas while testing a mobile app, the scope is enormous and varies according to the platform the app supports. Categorizing this as a challenge that a Mobile App Tester faces, here are few more that I would like to share in the same context:

  • Testing the app on all the physical devices that it supports may not be feasible at all the times due to various constraints and has to be hence prioritized.
  • A strategic approach to testing mobile solutions takes into account a number of characteristics unique to the mobile platform and supported OS.
  • Ideally, the functionality of the app or the user experience is expected to remain consistent across multiple devices of similar/varying platforms. Hence compatibility of the app to these becomes critical, calling the need of the app to be tested across multiple devices/platforms the app supports.
  • And in some cases, there might be chance of inconsistency in terms of functionality across multiple devices of same platform.
  • Even though you are testing on same platform, there might be discrepancies across multiple OS versions of devices.
  • GPRS connectivity would be inconsistent and mostly very bad during the peak hours (mostly 6-10pm), which may lead to inconsistent results.
  • As we cannot test on all physical devices, sometimes we may end up testing on simulators, on which we cannot rely completely.
  • The increased and continuous launch of new devices into market
(source: http://mobiapptesting.blogspot.com/2010/11/challenges-faced-when-testing-mobile_29.html)

Tuesday, March 20, 2012

Managing a Mobile VAS Project

Abstract:

Since a decade, the Indian mobile market has been growing tremendously and drastically. Subscribers are increasing at an unprecedented pace and which all the telecom operators are trying to capitalize on their corresponding subscribers.

A value-added service (VAS) is popular as a telecommunications industry term for non-core services, or in short, all services beyond standard voice calls and fax transmissions. (As per Wikipedia).

VAS mainly consists of Voice, SMS and Data categories. The categories
  • Voice – deals with the services which need to be operated by voice – calling mode. This category of services can be activated / deactivated / used by voice mechanism.
  • SMS – deals with the services which need to be operated by SMS – by textual mode. This category of services can be activated / deactivated / used by textual mechanism.
  • Data – deals with the services which need to be operated by browsing – by surfing mode. This category of services can be activated / deactivated / used by surfing mechanism.
Examples of VAS can be Caller Tunes, Match Alerts, Stock Alerts, Horoscope Alerts, Mobile Radio, Competitions etc... VAS can be developed and provided either in-house by a mobile operator or by third-party value-added service providers, depending on the nature of the product/company.

Mobile Value Added Services in recent times is becoming the huge source of revenues for all the operators in India. Mobile VAS is emerging as one of the areas that will provide huge opportunity for telecom operators to improve the top line. With the launch of 3G and broadband services around the year, industry experts foresee a substantial increase in the share of VAS revenues.

Services such as music, ringtones and games that are known as VAS Entertainment are very popular and have contributed significantly to the growth of VAS in India. On the other hand, services such as news and information on bank accounts, real estate, education, etc. are known as VAS Info. A third category of customer service that is taking today is the VAS M-commerce allows customers to perform banking and payment, such as mobile phones. These services are in a very nascent stage in India now.

I would like to share my minimal experiences and limited knowledge in managing and testing a Mobile VAS project.

Testing to be carried out  from multiple locations

As the scope of VAS services demands to be tested across multiple locations of a nation, this factor itself would be intimidating. The testing team has to be spread out nation (circles – which mean states) wide and the testing has to be carried out independently from each circle.  Managing and tracking the team from multiple circles would not be that easy.

In-consistent behavior of network &Reproducibility of the issues

The network plays a pivotal role in VAS services and owing to its inconsistent behavior, it’s always hard to reproduce the issue. In these scenarios, we cannot take a screen shot/video which would support the issue. We need to completely depend on the network, which is inconsistent. You end up having no more options other than relying on the tester completely, which should not be the case ideally.
The issue which has been encountered during a specific point of time (or a day or more) may not be reproduced at the other time, owing to the inconsistent network behavior. And also the network varies from significantly from circle to circle.
Some of the services need GPRS connectivity for testing them, GPRS connectivity is consistently inconsistent and because of which there would always be a huge discrepancy in the actual testing results. There is a huge variation in the GPRS connectivity on a single day with in different time intervals and also varies from day to day (at times)

Requests not getting hit on the server owing to network

When activating/deactivating the VAS products, the request for the activation/deactivation would be sent to the server and then the response would be sent to the user accordingly. Sometimes owing to bad network connectivity, load on the server, more requests being sent from the same mobile number, the request does not hit the server.
In these sorts of scenarios, the user has no evidence to support the testing activity, which in turn would make the testing team helpless as there is no evidence.

Testing in local/regional languages

All the services support the option of using the service in national, international language along with the local/regional language.
Testing the services in local/regional languages would comprehensively give a confidence about the product rather than testing in national and international languages. But testing across all local languages would inadvisably pose an issue as finding the team as per local languages may not feasible.
For testing services with regards to location/regional, the tester has to be aware of the happenings/customs of that corresponding region for better understanding of the services.

Categorizing the defects

Categorization of defects which have been encountered after testing under the inconsistent conditions would really be a uphill task. If we categorize the defect as functional, the development team (corresponding team) would revert back saying that the functionality was fine, but was not working owing to network connectivity. And when we categorize the defect as network connectivity issue, the defect loses its weight, and cannot be considered as a defect.

Conclusion:

During my experience I have found that most of the people are not educated about the VAS services. The operators/product owners need to market their corresponding services, if not it’s like winking at a girl in the dark. So once the people are educated and aware of the ongoing services, they may show inclination towards the services, which would inadverantly increase the revenue of the operators.

(Source: http://mobiapptesting.blogspot.com/2011/09/managing-mobile-vas-project.html)