Posts Tagged ‘cloud’


Cloud computing combines many different quality aspects. Testing the service is mix of real testing and risk assessment. Risk assessment on the other hand requires wide knowledge from business processes to development processes (like ISO-12207, CMMi) and IT service processes (like ITIL). They also need at least some knowledge about local privacy laws. This post is about availability and its aspects.

Availability is defined at ITIL terminology. It is “Ability of … IT Service to perform its agreed Function when required.” [1] When we start to analyze availability of cloud service, we must understand who the users are and how they are connected to Internet. If the server is at our own computer room, or at well-defined location of service provider, we have clear understanding what parts the network infrastructure has. But as soon as we start to use some cloud service, we lose understanding how the data is flowing between services. In worst case we don’t have any control to data. And even in best case we know the approximate location of our data.

Negative risks have multiple parts. First are our organization and its connection to outside world. If most of the users are using the service from office, the Internet connection from the office to outside world is first possible failure point. When decision for cloud use is done, organization should make sure that their Internet access is enough for required bandwidth. At the same time the business should make decision how long the service can be unreachable.

Service level agreement (SLA) is important part of availability. Unfortunately many cloud service providers are not providing SLA. Their license agreement can state “best industry efforts to guarantee availability” or “99% availability”. The contract usually doesn’t provide much compensate from down time. At the end IT service provider cannot create the SLA when service fails. It can only define, that it takes contact to cloud service provider and notifies them about problems. Then it’s up to cloud service provider how quickly they react.

Cloud computing has also positive sides. Let’s take an example from this blog. I used to host this at one virtual host which was running also other services. It’s at Finland because majority of users for those other services are in Finland. This blog is international blog, so keeping this at Finnish site isn’t mandatory anymore. The major risk at that Finnish site is that if it goes down for any reason, I don’t notice it before morning.

[1] http://www.knowledgetransfer.net/dictionary/ITIL/en/Availability.htm

Advertisements

Testing SaaS

Posted: 13.7.2011 in Ei kategoriaa
Tags: , ,

I love cloud services! But I also consider their negative sides and what new they bring to my life as the tester, administrator and user. Software as a service (SaaS) is great for lazy administrators like me. They are bringing new exciting stuff for curious (and evil) tester like me. For the user they bring simplicity.

Let’s start to think how they are affecting to testing. I consider SaaS to be same as commercial off the shelf (COTS) with Internet twist. When I’m taking the application to use I have some specific need. E.g. when I moved to WordPress.com I wanted to have simple blog software where I can migrate my old posts as easily as possible.

At normal testing we’re concentrating to requirements. At SaaS that is unfortunately the smallest possible part you can test. WordPress.com is providing plenty of additional features which I don’t need. I can e.g. protect part of the posts, I can add more authors to blog, I can choose many different kind of sharing options and combinations and so on. There are so many different options that I don’t even know them yet! I should try to investigate them at some point. But what does this mean to testing? The diagram below shows the difference between “required” features, and “provided features”.

Can we forget the features which are not required? Absolutely not! They should be part of testing at some extent. At least the testing should make sure that users cannot damage the normal use from those features. Extra features should also be part of risk assessment. The assessment should consider security, functionality, availability and performance risks. Management should decide if the risks and their probability can be accepted. The risks should also be pointed at instructions, policies and training.

Google Docs is good example. It is good tool for collaborated writing. But there is also plenty of security considers. Even if the organization doesn’t need “Sharing to whole Internet”, it still is features of Google Docs. If the risks related to it are at acceptable level, the usage policy can say: ”Never mention any customer name at documents which are at Google Docs, never share any internal document to ‘everyone’ or ‘everyone with the link’.” There is still the risk that users accidentally publish the private information. But now the risk is noted. There should be some plans what to do if risk is realized.

Cloud computing is not affecting only to feature testing. SaaS is very often the web services and same security issues might exist as any other web application. Unfortunately SaaS provider might deny the good security testing, because there is always possibility for denial of service or data loss which affects to all users. Same problem is with performance testing. So instead of real testing those you are only able to do risk assessment.

I’ll write more about risks at some point of future.


I started to think about putting this blog to life. It also needed upgrading the WordPress installation. Unfortuntely it wasn’t simple task as WordPress 3.2 is only supporting PHP5. Our hosting company supports only PHP4 with current hosting plan and changing that would require plenty of additional work. I’ve used WordPress.com for my Finnish blogs before, so I decided to move this also to there.

It is very simple to move posts and comments from your own WordPress blog to WordPress.com. You just have to use export at your blog and then import at WordPress.com. It asks you how to map the writers so if you have more than one writer, it’s not the problem to map them correctly. Author of comments are not changed any way if the author hasn’t been registered to your blog.

At the first stage I don’t want to move my domain badboysofquality.com to WordPress.com. I will do that at some point. So the tricky part is to redirect all requests from the old domain to badboysofquality.wordpress.com. I also want to make sure that permanent links are not going to broke down. At your WordPress installation you have .htaccess file. It is used by Apache to convert permanent links to form which is understood by WordPress. That file must be modified to redirect requests to your domain to WordPress.com. At the end my .htaccess is:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule (.*) https://badboysofquality.wordpress.com/$1 [R,L]
</IfModule>

Now all links to http://badboysofquality.com/ are automatically converted to correct form https://badboysofquality.wordpress.com/.

Is there any benefits to use WordPress.com instead of own installation? Yes. The technical maintenance time is reduced a lot. When I’m thinking the ‘costs’ of some service, I also do virtual charge from my own time 50€/hr. So if I spend time to tune own WordPress installation for 1hr/year, its extra cost is 50€/year. I also trust that WordPress.com will be more secure than own installation. It’s too easy to miss some important security upgrade. WordPress.com hopefully keeps its site more secure and up to date than I ever manage to keep with reasonable effort.

Is there negative sides? Yes. I don’t have control to WordPress.com. If I wanted to install some really fancy and cool plugin or theme, it would require a lot more money. I also don’t have control to ads the readers get. Of course I can pay to take them away. It costs only about $30/year. At the moment I don’t see any benefits from that, so unfortunately you’ll have to see the ads. At this first stage the domain is not my own. I will change that later. Its extra cost is only $12/year – not match compared to time I save.

So welcome to my WordPress.com version of blog. Have fun.


I have started to use cloud based tools to make my life easier. Here’s short list of tools I’m using and how I am using them. There’s most likely plenty of other tools which I could use, but these I use regularly. Do you have any cool tools to share?

Evernote is notebook cloud. It can be used with web browser and desktop application. When using web application, it is very easy store screen shots to there. I love that kind of feature. I have used it for article writing as well as for some blog texts. I can access to it even with my mobile phone. I haven’t done any security testing. You can use desktop application even so that you don’t sync your texts to server. It is also working very well without Internet access.

Yammer is for companies. It is much like Facebook with a lot less features. I have quite good trust to it because I’ve found only a few XSS-bugs. It is totally with https-protocol so it is not vulnerable for BlackSheep. We are using it a lot for internal information sharing of things which are not under strict NDA. It can be used even for free which is nice.

MindMeister is excellent browser based mindmap tool. You can share your mindmaps with others very easily. The access can be restricted to specified MindMeister users or put it to global use and protect it with password. I have really enjoyed it.

Google Docs is wonderful word processor. There is also other tools but I’m using it mostly with text. The best feature is possibility to collaborate. I can edit same text document with other people. I’ve fallen in love to that feature! I had wonderful time write one short story at it with my friend. I was ‘lead writer’ and she were correcting and adding some text. It was very well working. There wasn’t need to mail and have some old version.