Script Testing Concepts

...

Testers can quickly run test cases by specific software requirement ID, test name, or feature group among others.  Additionally, script tests are classified by their type (Numeric, Validation, System, Stress, etc.) and subsets can be run easily by specifying the test categories in the test system configuration.  Finally, there are several higher-level classifications of script tests used during the development process before committing new code or used nightly to determine if code additions or changes have caused unexpected adverse effects. These higher level categories such as “Smoke” and “System Tests” are groupings of lower level test cases and provide developers and testers insight into the system without running the entire test suite.  

Script Test System Overview

...

You can use any Subversion client to download the files. You'll need your NDC credentials to access the server. When you check out the system, you'll see several directories worth of files. Here's a brief overview:

test/
    bin/
        ...misc MATLAB folders...
        gmattest.m                      # The main test system script
        rundef.m                        # The run definition file (see below)
        setup.m                         # Run this before gmattest.m to set up path info
                                        # (or add to your startup.m)
        ...other .m files...            # Old or developmental scripts; ignore

    doc/
        ...developer documentation...
        reqs/                           # This is where your test matrices will go

    extern/
        Commands/
            FRC-1_Optimize/             # These folders will contain the scripts used to
                                        # generate your truth data (such as STK/Connect
                                        # scripts)
            ...other FRC folders...
        Resources/
            FRR-1_SpacecraftOrbitState/ # same as above
            ...other FRR folders...

    input/
        Commands/
            FRC-1_Optimize/
                scripts/                # .tc files and .script files
                truth/                  # .truth files (truth data)
            ...other FRC folders...
        Resources/
            FRR-1_SpacecraftOrbitState/ # same as above
            ...other FRR folders...

    output/                             # This is where the output files, log files, and results
                                        # are stored. They are divided by the build specifier
                                        # (see the Run Definition section) and then by the 
                                        # Commands/Resources structure like the extern and input
                                        # folders.

...

These are the most commonly-used fields:

Advanced run options

Running the test system itself is simple, once everything is configured:

...

At this point, the system will run and report progress to the screen. A full run with all tests included may take several hours.
If you want to repeat a portion of the run without rerunning all the tests, there are other commands available for running only a portion of the system:

Advanced Performance Configuration for Small Test Sets

...

RunDef.Build       = 'NightlyBuild';
RunDef.GmatExe     = 'c:\PATHTOYOURBINDIRECTORY\GMAT.exe';
RunDef.Modes       = {'script'};
RunDef.Comparisons = {'truth'};
RunDef.Reporters    = {'ScreenReporter'};
RunDef.Cases        = {'STM_GMAT_PD45'};
RunDef.Folders      = {'FRR-13_DynamicsModels'};
RunDef.FilterMode   = 'and';
RunDef.Requirements = {}
RunDef.Categories = {}

The test system supports options that allow exclusion of plugins, old or under development tests among others. 

% Tell the test system to run/not run tests tests that have alpha features 
RunDef.SetRunAlphaTests(true);
% Test the test system to run/not run tests that have internal features
RunDef.SetRunInternalTests(true);
% Test test system to exclude tests in folders that
% are either in old = {'STM_GMAT_PD45'};
RunDef.Folders      = {'FRR-13_DynamicsModels'};
RunDef.FilterMode   = 'and';
RunDef.Requirements = {}
RunDef.Categories = {}
or UnderDevelopment
runDefMan.SetRunExcludedFolders(false);
% Pass in RunDef so manager can configure it according to settings above.   
RunDef = runDefMan.PrepareRunDef(RunDef);

Configuration for Nightly Regression Testing

...

% Define the desired target file name for the RTTM
outputFileName   = 'C:\MyFiles\RTTM.xlsx';
% Define the name and location of your GMAT requirements Excel spreadsheet
requirementsList = 'C:\MyFiles\GMAT Requirements.xlsx';
% Execute the command to read all tests cases and map tests to requirements
gmattest('computemetrics',requirementsList,outputFileName);

...