Type of tests

Test types distinguish the stages of product verification

• Test type by level
  • Unit tests
  • Integration tests
  • Acceptance/qualification testing
• Other types of tests
• Methodologies defining testing
• You ask or what you want to know about testing
  • What types of tests do we need to do?
  • We do not have to comply with any standard, so we are not affected by the obligations
• Where to go next
  • Training on testing
  • Testing is the last step in quality

Test type by level

Unit tests

Unit tests are tests at the lowest level of development. They deal with basic units – functions, circuits, basic parts.

Basically, unit tests at the lowest level verify that the whole is built from reliable parts.

Unit tests should include all functions – user, system, and of course the operation of all interfaces.

Unit test documentation

With unit tests, how you perform them is critical. If unit tests are automated, they are of course documented due to their code.

If they are not automated, it is usual that they are performed by the developer immediately after the feature is developed (before commit to the repository). Writing down how the unit test was run would not make sense and would burden the work disproportionately. Remember, however, that if development is done in accordance with standards such as ASPICE, documentation of unit tests is also mandatory.

The basis for unit tests are detailed requirements for hardware, software, mechatronics.

How to document unit tests

There must be a methodology for the tests, what needs to be verified by the test. The tester confirms compliance.

Example of unit tests for software:
The standards require static code review. This includes

• Code review – documentation is the task performed that involved the code being reviewed
• Code analysis (manual or more or less automated) – the documentation is the output from the analysis tool or, again, confirmation from whoever performed the analysis
• Consistency check (e.g. checking that one function calls another for the correct purpose and with the correct parameters)

Example for hardware (electronics):

• Check all signal paths
• Analysis in design software by simulating current flows
• Verifying that the hardware is doing what it is supposed to do, e.g. loading software into memory, resetting software that is not responding ("watch dog"), etc.

Example for mechatronics:

• Stress test the strength of components
• Checking the resistance of moving parts (e.g. cables in bending)

Integration tests

The purpose of integration tests is to verify that separately developed and unit-level tested parts work together correctly.

Integration tests are typically multilevel for incremental integration.

Remember that integration takes place between all parts of the product under development – hardware, software, mechatronics.

The input for integration tests is the system requirements.

Acceptance/qualification testing

The goal of these levels of testing is to verify that the entire solution – product, software – behaves correctly in the environment for which it is intended.

The input for qualification/acceptance testing is the user requirements, constraints, regulations and standards that the solution must meet

Other types of tests

Tests are categorized not only by level but also by focus. In general, it is not possible to say at which level which types of tests are done, because they are often multi-level in the same way as functional tests.

The most common tests are mandatory tests

• Cybersecurity tests – verify that the product is resistant to external intrusion. Cybersecurity tests are performed at all levels of testing
• Security tests – checking that the product is secure. These are usually acceptance level tests. But they include, for example, tests of individual components for temperature resistance for the environment they will be in, which is realistically the lowest level of testing.
• Resistance Tests – Testing under various stresses to see if the product can withstand the stresses it is expected to withstand over its expected lifetime
• Stress Tests – Close to endurance tests, but focused on how much stress a product or part can withstand.

Methodologies defining testing

We did not invent the types of tests, they are precisely defined by many methodologies. The most readily available methodology is SPICE, or its automotive variant, ASPICE. There is also a lot of work on software development standard, ISO/IEC 12204.

You ask or what you want to know about testing

What types of tests do we need to do?

The types of tests are generally defined by the project methodology. If you have an established methodology in your company, you need to follow it.

The important thing is whether the product you are creating must meet the standards. For example, all software for cars, production machines or even instruments that are used in production must pass all types of tests. Standards such as ISO 26262, CMMI and others require this.

We do not have to comply with any standard, so we are not affected by the obligations

If the development is not driven by a mandatory standard, there is no external standard. However, for a good verification of the functioning of the software, at least

• Unit tests (testing of the programmer and solution designer) and
• Qualification tests (user/functional testing). Without these two levels, there are bound to be many flaws in the final solution

Where to go next

General about AyMINE tests here.

Training on testing

If you are unsure about testing, we recommend training on testing to learn everything.

Testing is the last step in quality

Quality standards look at testing as (almost) the last step, where the quality of the product is addressed. But quality starts at the definition of the brief, continues through design to development and testing.
The whole issue of quality is covered in this training, for example.