Automated Test Case Generator (ATCGen)

The RTSync team has developed the Automated Test Case Generator (ATCGen), a Modeling and Simulation (M&S) based methodology and software tool set for automating tactical data link conformance testing at the Joint Interoperability Command (JITC). RTSync led the research and development effort to formalize the military standard, Mil Std 6016C (Link 16), as a system model, specified by unambiguous rule sets that can be completely analyzed for their sequences, variables, and dependencies within the rule sets. The basis of the M&S formalism employed is the Discrete Event System Specification (DEVS).

Test cases are defined as sequences or paths through the standard where the system under test (SUT) is to respond with messages and behavior in certain ways based on the data and messages received. This path is transformed into a series of stimuli that should cause the SUT to display the desired behavior. Stimuli can be sensor stimulation, messages, and data within the messages. The actual behavior of the SUT is then compared to the expected behavior and any deviations are noted.

The ATC-Gen at Joint JITC has been converted into a web service to allow outside organizations to conduct their own testing. The user interface has undergone continuous improvements based on feedback from users. Test cases are now being designed using Mil-Std 6016E, the latest version of the standard. Testing to the North Atlantic Treaty Organization (NATO) Standardization Agreement (STANAG) 5516 is also supported. Design of Experiments concepts have been applied to how the test cases are executed. The test cases are run as a set of test cases which more thoroughly tests the behavior of the SUT. The tester does not have to individually configure each test case, instead the program generates the differing values to be used in each related test case in the set.

ATC-Gen offers rapid and repeatable pass-fail testing using a set of JITC developed test cases. Being a web service allows customers to conduct their own conformance testing using the same tool that JITC uses. This should help reduce the risk for a SUT when it undergoes interoperability testing. An added benefit of ATC-Gen being a web service is that when new test cases are developed and approved, they become available to everyone with access to the service.

ATC-Gen is currently used by JITC for Mil-Std 6016 and STANAG 5516 conformance testing during Joint Interoperability Testing (JIT), testing with NATO, and testing with other coalition members.

Reference Links :