eWiki :: TNG Testing

TNG Testing

Introduction

The big question about a good development plan and a high quality delivery of the next version of eComStation (like any other project) cannot go without proper testing.
This is a live document which is written in a brainstorm style, and should grow to a more mature and complete document that can be used very well.
Part of this will be references to other documents, like component specific test plans.

But what is proper testing? That is hard to say, as testing should be done during all phases of development, from gathering requirements at the beginning to the acceptance by the client at delivery.
As this is a very wide range and as we have a lot of open source/freeware/shareware components coming from all over the planet it is hard to achieve full range testing. Although open source/freeware/shareware programs should have gone thru a full test cycle by themselves, but these programs have a different development cycle by which testing may have to be done in a different way as well.

Some background information to explain the area we are talking about is good one time reading before we move on..

Kind of tests

Looking at eComStation, I think we need to look at a couple of things:
- test each individual component or application (requirements needed to be validated as well as inputs/outputs) (unit test)

This test is also known as programmers test. It can be done in two ways:

mostly used is the testing by the programmer himself. This is mainly debugging the code and gettting it running. And of course matching the raised requirements.

another, far better, way would be to have the component/application being tested by an indepent group. The advantage of this is that somebody else is looking at the code with a fresh view. Any unclear requirements are found easier this way.

- test the integration of components (do they interact correctly, according to agreed interfaces) (integration test)

This can be iterative testing, as integration on different higher levels might be needed, up til the highest level which will be the system test. This should be done by an indepent group as well, as they can also validate documentation. Many of the tests being done in this phase can be reused in the system test phase.

- test the system as a whole in a lot of different scenarios (system test)

This is the highest level of integration tests. It is considered a major milestone. The original designers of the module/application will perform this test phase as they know the best what the user is expecting. They might validate results with users to be sure about the correct working.

- users testing the program against the given requirements (acceptance test)

In this phase the users can validate the delivered program against their initial requirements, including documentation. If all requirements are met the application can be accepted and started to be used in production environments. This is again a milestone, of course, and needs to be documented/stated as a signature for finishing this development cycle. (And the final payment can be made.)

(Keep in mind that each added product needs to go through these steps on its own as well, e.g. eStyler, NewView etc.)

Next to these, there are also other kind of tests, like:
- real life tests out in the fields with a wide range of different hardware and various languages (hard to do in a lab environment)
- dummy test, which is trying to crash the application in any way you can (especially useful in tough environments)

This is all on a very high level.

In essence you need the same for each test as well, although there will be nuances.
For each test you need to know what the intention of the component is, how is the flow, which values can be used (input), which combinations are valid, and what should it deliver for instance (output).
If these are not known you can hardly test the component successfully.
Some also say that you need knowledge of the component logic, but I think that is not necessary as the component should be working logically and be helpfull in making the right choices anyway.
Although you need to know what kind of values can be selected or entered, like from a drop down box (all asked for values present?) or an entry field (e.g. can it only except numeric values? and the right range?).
Part of that is also border value testing to see if the border checks are done correctly.
Another area for testing is the combination of values that might be selected. It should only allow possible combinations and block or warn for wrong choices.

All this can be seen as the need for good and complete specifications. Without them proper testing is undoable.
There are a lot of good analysis methodologies and notation ways to come to specifications, which should have been used already to start with the design of a product of course. Therefor these will not be described here, as that is a subject on its own.

Scope

All in all there is a lot that can be tested. But do we want to test everything, and are we able to do that?
We better start defining the areas we want to test and then determine how to do that and to what detail.
Are all third party programs their own responsibility for unit testing? And should they be left out of the TNG testing?
Integration testing should still be performed. For this, interfaces between components need to be defined and described.
Which individual components can we identify within TNG, these should be able to run on their own, for unit testing for instance.
For each of these components a testplan should be defined. Part of this are also test scenarios with test data that can be used over and over again. This is not only needed for TNG, but also for regression testing on newer versions of the component(s).

To be continued...

A good site to wander around and find a lot more information about testing is TMAP∞, which is about the TMAP methodology from the Sogeti organisation.

TNGTesting

TNG Testing

Introduction

Kind of tests

Scope