Testability has to be built even to good software

Posted: 28.10.2010 in Testing
Tags: , ,

Thanks testedtested about good thoughts @Twitter. 🙂 I had to start thinking about testability and its relationship with customer happiness and usability. 140 characters is too less for very well formulated ideas and comments so I had to write blog entry.

If I can use application, it doesn’t mean that I can test it. In many cases the testers must request developers to add code and features to increase testability. In many applications good usability means better testability, but that’s not case in many embedded systems.

Good example is Google’s self-driving car. From user point of view it is very simple system. Just submit where you are going to, hit the button and car takes you there. (This is very simplified view.) Even if that process is done simple, it is definitely not the same as testability.

I slice the simplified system to multiple parts:

  • Sensors
  • Motor
  • Wheel
  • User’s GUI which inputs the destination
  • Button which starts to drive

The software which controls all of these is software under testing (SUT) in my example. For me I’d need following things this to be testable:

  • I have to be able to send artificial messages from the sensors to SUT
  • I have to be able to inspect what kind of messages motors get
  • I have to be able to give artificial feedback from motor to SUT
  • I have to be able to control (virtual) wheels same way as normal road affects to wheels

If any of these is missing, I would not be able to test the control system. These need plenty of simulators and stubs which tester or test automation can inspect and control. But even without those simulators and stubs, extra interfaces the user could use the system and even be happy (if it works as expected). And even tester could use the system without them.

So testability is not always automatically part of usable software. Sometimes testability needs plenty of additional work for developers. And in those cases it is usually testers’ responsibility to give details how to make application more testable.

  1. […] This post was mentioned on Twitter by PradeepSoundararajan and PradeepSoundararajan, Teemu Vesala. Teemu Vesala said: My few words about testability. http://bit.ly/bkmiI6 Thanks @testertested about your thoughts. U helped me to start writing. 🙂 […]

  2. You have already tested the car, in the sense that you have questioned the product in order to evaluate it (as James Bach would say); or you have gathered information with the intention of informing a decision (as Jerry Weinberg would say).

    Your testing is not yet empirical; that is, we don’t yet know whether the car has these attributes or not. Yet we’ll be ready to look for those attributes as soon as we see the car, or see a design of the car, or talk to someone about the design of the car.

    If you can use the application, you can test it. In fact, the assessment that you can use it is an indication that it has been tested it to some degree; it’s been enough for you to determine that you can use it. That’s true even for embedded systems.

    Everything that you’ve identified is testable even if you don’t have testing APIs. I think the question here is not whether you can test it, but how easily and how inexpensively you can test and check it. That involves identifying issues of cost, value, and risk: the cost of adding scriptable interfaces and logging; the value of being able to do automated checking more quickly; the value of having more time to explore and test the system; the cost of investigation and reporting things in testing that could have been caught by checks; the risk of missing things altogether in testing, when they could have been caught by checks.

    To me, the issue is not whether we’re living up to some ideal, but how we value information about the system and make decisions about how to obtain that information.

    —Michael B.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s