APM & APM Studio: Test Definitions in APM & APM Studio

When you are building your APM application, it is vital to be able to test the rules, models and data processing before putting them in production. APM allows you to feed offline data to your application and the graphical nature of APM Studio enables you to observe the application while it is running. Property tables, displays, graphs etc. also give a view on how the application is performing and programming errors are indicated in the logs. However, as your application becomes more complex, it is important to be able to validate it at each stage of development and deployment and/or automatically as needed. When you make changes to your APM application or after a software upgrade, you will want to verify that the application performs as required. That is, run preconfigured tests and compare the results against expected values easily and in a repeatable fashion.

APM Studio integrates a powerful testing feature that lets you:

Create test scenarios;

• Create multiple step-by-step test sequences;
• Define actions to be performed during the tests;
• Define the expected results (values/states) at each step in a test;
• Run tests that control (activate/deactivate) application Definitions (Processing Rules, STDs, BowTies, DataDefinitions) at the start and end of a test (or test sequences) and lock controlled Definitions while tests are running;
• Visualise the results of tests in real-time as they are run;
• Download timestamped static test reports; and
• Interface to the test environment via a REST API for on demand Test execution from external systems and other functions.

There are three main types of components that make up a testing environment:

• Test Definitions
• Test Sequences
• Test Steps

Test Definitions hold one or more Test Sequences and controls which rule definitions and data definitions are active or inactive when the tests begin and end. A Test Sequence is like a scenario and will contain a number of Test Steps to perform actions on your application and test results.

The image below shows an example of a test sequence that has been run. You can see that step 2 has failed; an alarm should have been cleared meaning the conditions leading to the alarm should not be active anymore:

To keep record of test results the results of a Test Sequences can be exported. The report is exported in Markdown format, below image shows an example of such an export:

The capabilities provided in APM/APM Studio for automated testing provides significant value for several reasons/purposes:

• Efficiency: Automated tests can be run quickly and repeatedly without the need for manual intervention, saving time and effort compared to manual testing.
• Consistency: Automated tests perform the same steps and checks consistently every time they are run, reducing the likelihood of human error and ensuring a reliable testing process.
• Coverage: Automated testing can cover a wide range of scenarios, including edge cases and regression testing, ensuring comprehensive test coverage that might be impractical or impossible with manual testing alone.
• Early Bug Detection: Automated tests can be integrated into the development process, allowing developers to catch bugs and issues early in the development cycle when they are cheaper and easier to fix.
• Regression Testing: Automated tests can quickly verify that new code changes haven’t introduced regressions or broken existing functionality, enabling developers to confidently refactor and make changes to the codebase.
• Cost Savings: While there’s an initial investment in setting up automated tests in APM Studio and writing test scripts, automated testing ultimately saves money by reducing the time and effort required for manual testing, and by catching bugs earlier in the development process.
• Scalability: Automated tests can easily scale to accommodate large and complex software projects, allowing for continuous integration and deployment pipelines that facilitate faster release cycles.
• Documentation: Automated tests serve as executable documentation of the software’s expected behaviour, providing insights into the system’s functionality and requirements.

Overall, automated testing is a valuable practice that contributes to improved software quality, faster development cycles, and increased confidence in the reliability of software systems.

Up Next: APM/APM Studio – Decision Support Rules