Structures and how to choose one

Learn the three basic ways of structuring your test in Ranorex Studio and how to determine which one works best for your requirements.

To compare the different test structures, use the following test requirements:

First, two smoke tests ensure that the AUT is installed correctly and can be started and closed by Ranorex Studio.

Then, two sanity tests ensure the correct version of the AUT is used and that all UI elements are implemented and work according to specifications.

Finally, three functional UI tests ensure that the AUT behaves correctly for its administration, database, and file-importing functions.

Here we list the advantages and disadvantages of each structure type and give an example of how the above test requirements could look when translated into those structures.

Again, no single structure is the best. It all depends on your requirements. With what follows, you should be able to make an informed choice regarding which structure is appropriate for you.

Solution-based structure

In this type, the primary structure is based on creating one solution for each of the three different test types. The actual tests are then structured in projects, with one test suite per project.

1. Ranorex Studio solution for the smoke tests

2. Ranorex Studio solution for the sanity tests

3. Ranorex Studio solution for the functional tests

Advantages

• Very clear structure at the top level. Useful for extensive and complex test requirements.
• Particularly useful when testing an AUT that has different UIs and requires multiple repositories, e.g. testing frontend and backend UIs.
• Easier to keep test suite structures simple and efficient.
• Can make test execution quicker.

Disadvantages

• Higher maintenance: Every solution has its own settings, repository, and module library that you need to take care of.
• It’s not possible to execute all three solutions automatically in sequence from Ranorex Studio. You can only do so with external scripts, e.g. in a continuous integration environment.
• Each solution has its reports. This can make it harder to collect and present results.
• There is no easy way to exchange data between the three functional tests.

Project-based structure

In this type, the primary structure is based on creating one solution that contains one project for each of the three different test types. The actual tests are then structured in multiple test suites, with one test suite per test.

1. The solution AuT_Testing contains all three projects

2. Project FunctionalTest for the implementation of the three functional tests

3. Project SanityTest for the implementation of the two sanity tests

4. Project SmokeTest for the implementation of the two smoke tests

Advantages

• Convenient: One solution contains all the tests.
• Easier maintenance: No differing settings, and usually only one module library and repository.
• Even though the structure is less clear at the solution level, it’s very clear at the project level. Therefore, keeps the advantage of simple and efficient test suite structures.
• You can easily run the projects and the contained test suites in sequence from Ranorex Studio by using the test sequence functionality.

Disadvantages

• Each project/test suite has its own reports. This can make it harder to collect and present results.
• There is no easy way to exchange data between the three functional tests.

Test suite-based structure

In this type, the primary structure is based on creating one solution that contains one project. This project then contains one test suite for each of the three different test types. The actual tests are then structured in test cases inside those test suites.

1. The solution AuT_Testing contains one project

2. The project AuT_Testing contains one test suite per test type.

3. The individual test suites contain the actual tests as test cases.

Advantages

• Convenient: One solution contains all the tests.
• Easier maintenance: No differing settings, and usually only one module library and repository.
• You can easily run the test suites in sequence from Ranorex Studio by using the test sequence functionality.
• Exchanging data between the three functional tests is now easy, but still difficult between the different test types.

Disadvantages

• Quickly leads to bloated and overly complex test suite structures, making maintenance, troubleshooting, and collaboration much harder and affecting test performance negatively.

This section describes the factors you should consider when choosing your structure.

Avoid extremes

For most requirements, the project-based structure is a good choice.

It offers a flexible approach with good organization, simple test logic, and high convenience.

Both the purely solution-based and the test suite-based models are extremes. We recommend using them only if your test requirements fit them perfectly.

Use the solution-based structure for very large, complex tests. The very high maintenance required by this approach is the most important disadvantage. You have different settings, repositories, and module libraries for each solution.

Use the test suite-based structure for small, simple tests. It’s easy to set up a test suite-based structure and quickly get a test running. Maintenance is also low — but only if your tests stay small and simple! With more complexity, you quickly get bloated, hard-to-manage test suites.

Consider your UI elements

The UI elements involved in your tests are one of the more important criteria when choosing a structure. If your tests deal with multiple different UIs, like a frontend and a backend, it makes sense to use multiple repositories.

With multiple repositories, the solution-based structure becomes more attractive. However, you can also use multiple repositories in the other structure types. So if your test isn’t very complex despite two different UIs, a project-based structure with two repositories is probably still better.

Data exchange as a requirement is overrated

Exchanging data between tests is hard when they’re not in the same test suite. However, it’s often not truly necessary. You can avoid data exchange by designing reference data and test cases differently.