Posts Tagged ‘manual testing’


This blog entry starts my test data related blog series. I will submit new one every now and then. If you have questions or comments, please, let me know. I like to learn and I like to give new ideas. So disagree freely with me.

Over the series I’m using single example application: Discussion forum software (Like e.g. SMF, phpBB). Key features are registration, logging in and out, different kind of user rights, posting public and private messages, reading them and searching.

One key feature of manual testing is human aspect. Human submits the data, human checks the results. All test data should consider that. We have limited possibility to notice small differences.

Normal bias which I’ve seen at manual testing is using repetitive patterns. If the date format is ddmmyy, even tester quite often uses date like 010101 or 111111. They are simple and quick to type and always valid. But as the test data they miss many possible error cases. What if age calculation swaps day and month? The data like that won’t notice it. Much better is something which makes missing the mistakes impossible. It could be for example 230142. Same number is not repeated at all, so if something fails, it is immediately noticed.

The forum also requires text data. If testing the messages, the test needs plenty of text. Usually the testers and developers are using lorem ipsum as test data. But that practice should be avoided. There are multiple failure points. You can’t read the text, so you most likely won’t notice if characters are swapped to some other. Warping of the lines should also be done correctly which is difficult to see from lorem ipsum. Also all other content related errors are masked behind nonsense. If I need plenty of text, I usually take it from Project Gutenberg.

Many organizations are still doing scripted manual testing. There you have to decide if the test data is part of test case or not. When it is not, it gives tester possibility to use different kind of inputs, but part of them might be weak and reproducing the situation suffers. If it is, then same data is used over and over again, and at the end we can say that it works at least with given data, but not sure about other data.

In my opinion you should consider what important inputs are, and specify those. If there is possibility to say “this kind of data” instead of “exactly this data”, use rather “this kind”. It still gives larger variation for inputs than exact input, but is still able to test what is wanted. “Not so important configuration data” should be specified so that it is easy to take to use. I’ve been at project where configuration of the test environment took almost whole day. In that case all configurations should be specified so that I could found them right away and in best case also be able to take them to use right away.

Thanks @HelenaJ_M about question about reusing of data versus creating all the time from scratch.

Advertisements

Many times I hear that exploratory testing (ET later) is pure manual testing. But that’s not true. You can use any possible tools to help ET. This is the part one of multiple articles where I present tools which you can use to assist your ET and other manual testing.

What is the purpose of tool? Its main purpose is to release tester to do meaningful tasks. If initializing the test takes more than 1 second and needs to be repeated over and over again, it is preventing the good testing. And tools should be used to remove that kind of obstacles.

Unfortunately often the tool itself becomes the “purpose of testing”. I know that – I am usually doing test automation. It seems that very often the tool itself becomes the obstacle because someone thinks that it is the silver bullet for all testing problems. At that point the tool turns to testing problem.

After short introduction let’s start with very simple case. Let’s imagine we were testing wordpress.com. For tester that is really boring case, because every time he wants to do something, he has to login. Easiest way to get around the problem is to get tool to do login. If the case is this simple, I’d take Selenium IDE. It is Firefox plugin which records the test case. After recording it can be played over and over again to get the test to specific point. The screenshot below shows the whole test for WordPress.com login. (Credentials are not real ones…)

Selenium IDE

Selenium IDE script for WordPress.com

Selenium IDE is good for small tasks, but I would not recommend any recording tool for large scale test automation or complex tasks. Its simplicity justifies its use.

I will write later more about tools which can help exploratory and other manual testing styles.