Testing SaaS

Posted: 13.7.2011 in Ei kategoriaa
Tags: , ,

I love cloud services! But I also consider their negative sides and what new they bring to my life as the tester, administrator and user. Software as a service (SaaS) is great for lazy administrators like me. They are bringing new exciting stuff for curious (and evil) tester like me. For the user they bring simplicity.

Let’s start to think how they are affecting to testing. I consider SaaS to be same as commercial off the shelf (COTS) with Internet twist. When I’m taking the application to use I have some specific need. E.g. when I moved to WordPress.com I wanted to have simple blog software where I can migrate my old posts as easily as possible.

At normal testing we’re concentrating to requirements. At SaaS that is unfortunately the smallest possible part you can test. WordPress.com is providing plenty of additional features which I don’t need. I can e.g. protect part of the posts, I can add more authors to blog, I can choose many different kind of sharing options and combinations and so on. There are so many different options that I don’t even know them yet! I should try to investigate them at some point. But what does this mean to testing? The diagram below shows the difference between “required” features, and “provided features”.

Can we forget the features which are not required? Absolutely not! They should be part of testing at some extent. At least the testing should make sure that users cannot damage the normal use from those features. Extra features should also be part of risk assessment. The assessment should consider security, functionality, availability and performance risks. Management should decide if the risks and their probability can be accepted. The risks should also be pointed at instructions, policies and training.

Google Docs is good example. It is good tool for collaborated writing. But there is also plenty of security considers. Even if the organization doesn’t need “Sharing to whole Internet”, it still is features of Google Docs. If the risks related to it are at acceptable level, the usage policy can say: ”Never mention any customer name at documents which are at Google Docs, never share any internal document to ‘everyone’ or ‘everyone with the link’.” There is still the risk that users accidentally publish the private information. But now the risk is noted. There should be some plans what to do if risk is realized.

Cloud computing is not affecting only to feature testing. SaaS is very often the web services and same security issues might exist as any other web application. Unfortunately SaaS provider might deny the good security testing, because there is always possibility for denial of service or data loss which affects to all users. Same problem is with performance testing. So instead of real testing those you are only able to do risk assessment.

I’ll write more about risks at some point of future.

  1. Great. I indeed like this post. It will be good to see people sharing their experience with Cloud testing services. look forward to see other comments on this.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s