1 overview =TITLE overview =TITLE Overview of Digital Test Manager (DTM) DTM organizes and automates the software regression testing process. After you install a DTM system and create tests, you can : o Organize tests o Select and run tests o Compare current test results to previously verified results or benchmarks o Save test results o Display test results for review For information about using Help, choose the Using DTM Help... menu item from the Help menu. For further information about using DTM, double click on an item under additional topics. 2 basics =TITLE basics =TITLE DECwindows Basics Information about using DECwindows, such as how to manage windows, and how to use dialog boxes and scroll bars, is available from Session Manager help. To get Session Manager help, perform the following steps: o Pull down the Help menu from the Session Manager window. o Choose the Overview menu item. The Overview help topic provides details about using DECwindows. 2 getting_started =TITLE getting_started =TITLE Getting Started =INCLUDE terms =INCLUDE overview invoke =INCLUDE overview exit =INCLUDE overview inv_com_box =INCLUDE overview views =INCLUDE overview set_up =INCLUDE overview create =INCLUDE overview run =INCLUDE overview compare =INCLUDE overview examine =INCLUDE overview prol_use =INCLUDE overview var_use You must create a DTM library and create tests before you can use DTM for organizing, running, and reviewing test results. For more information on DTM features, double click on a topic listed under additional topics. 2 invoke =TITLE invoke =TITLE Invoking DTM To invoke DTM, choose one of the following methods: o Invoke the DTM DECwindows interface by specifying the DTM command with the /INTERFACE=DECWINDOWS qualifier at DCL level. o Invoke the DTM DECwindows interface by selecting the DTM verb from within the FileView window. You must place the verb in one of the FileView menus. o Enter the DTM command at DCL level. This invokes the DTM subsystem shown by the prompt DTM>). Once in subsystem mode, you can execute commands by entering subcommands, keywords, parameters, and qualifiers. 2 exit =TITLE exit =TITLE Exiting DTM =INCLUDE mu l_mu l_xm To exit DTM from the DECwindows environment, perform the following steps: 1. Pull down the Library menu. 2. Choose the Exit menu item. To exit DTM from a terminal environment, enter the EXIT command at the DTM> prompt. 2 inv_com_box =TITLE inv_com_box =TITLE Entering Commands The DTM DECwindows interface enables you to directly access all functions by choosing menu items, clicking on screen objects, and specifying text entry fields. 2 views =TITLE views =TITLE Using Views A view is a window containing a hierarchical display of a DTM library object and any associated objects. You can display the information with various levels of detail. The following table shows view types and the levels to which you can expand them: View Type Expansion Collection Collection attributes and result descriptions (preceded by icons), which expand to benchmark, result, and difference files Test Test attributes Group Subgroups, tests within groups, and test attributes Variable Variable attributes History History records 3 cre_nvu =TITLE cre_nvu =TITLE Creating a New View =INCLUDE mu v_mu To replace the current view with a new view, perform the following steps: 1. Pull down the View menu. 2. Choose the type of view menu item (Collection, Test, Group, Variable, or History). To create an additional view window, perform the following steps: 1. Pull down the View menu. 2. Choose the New menu item. 3. Choose the type of view menu item (Collection, Test, Group, Variable, or History). 3 exp_vu =TITLE exp_vu =TITLE Expanding an Object in a View =INCLUDE mu v_mu v_xm To expand an object in a view, double click on a display line that is preceded by an icon). Note that this causes the line to expand or collapse both children and attributes, depending on its current state. Children and attributes represent levels of detail about an object. Alternatively, you can expand an object in a view by clicking on the object, then choosing the Expand menu item to display the children or attributes of the object. For example, after using a test view to display the tests in the current library, you can expand a specific test to display its attributes. Attributes of a test include the following: a template file, a benchmark file, a prologue file, an epilogue file, variables, groups, type of test, filters, and comparison type. 3 clps_vu =TITLE clps_vu =TITLE Collapsing an Object in a View =INCLUDE mu v_mu v_cm To collapse an expanded object in a view, double click on a display line that is preceded by an icon. By collapsing view objects, you hide the next level of detail (children and attributes) being displayed in a view. Alternatively, you can collapse the children or attributes of an object by clicking on the object you want to collapse, then choosing the Collapse menu item. 3 sel_vu =TITLE sel_vu =TITLE Selecting an Object in a View To select a single object in a view, click on the object. You can select more than one object by pressing the Shift key while you click on objects in the list. 3 clos_vu =TITLE clos_vu =TITLE Closing a View =INCLUDE mu v_mu v_clsm =INCLUDE mu l_mu l_cm To close the active view window when more than one view window is open (without exiting DTM), pull down the View menu and choose the Close View menu item. The Close View menu item is accessible from both the Library and View menus. It is inactive if only one window is open. 2 set_up =TITLE set_up =TITLE Setting Up a DTM System To set up a DTM system to run tests, you must perform the following steps: 1. Create or open a DTM library. 2. Identify each test and related files to DTM. 3 des_lib =TITLE des_lib =TITLE Opening a DTM Library =INCLUDE terms library You can open an existing or new DTM library from which to reference and control information by one of the following methods: o Choose an existing DTM library. o Create a DTM library, which then becomes the referenced library. 4 create_lib =TITLE create_lib =TITLE Creating a DTM Library To create a new DTM library, you must first create an empty OpenVMS directory for the library, then perform the following steps: 1. Pull down the Library menu. 2. Choose the Create... menu item. The Create Library dialog box enables you to specify all the attributes for the new library. The created library automatically becomes the current DTM library. 4 choose_lib =TITLE choose_lib =TITLE Choosing an Existing DTM Library To open the current DTM library, perform the following steps: 1. Pull down the Library menu. 2. Choose the Open... menu item. The Open Library dialog box contains a field for entering the DTM library name. 3 history =TITLE history =TITLE DTM History When you create a DTM library, a history becomes associated with that library. Whenever you initiate an operation that alters the library, DTM logs the operation and its associated remark into the history. You can view history entries and add and delete remarks from the history. 4 vue_history =TITLE vue_history =TITLE Displaying History Records =INCLUDE overview views =INCLUDE mu v_mu To display the history records, perform the following steps: 1. Pull down the View menu. 2. Choose the History menu item. 4 remark =TITLE remark =TITLE Adding a Remark to the History =INCLUDE mu m_mu m_cm m_chym To add a remark to the history, perform one of the following actions: o Enter text in remark fields (usually part of a dialog box for DTM operations). DTM logs the text you enter and the operation. o Log information about an unusual event using the Create History dialog box, as follows: - Pull down the Maintenance menu. - Choose the Create menu item and the History... submenu item. 4 del_history =TITLE del_history =TITLE Deleting History Records from the History =INCLUDE mu m_mu m_dm m_dhm To delete a specified range of history entries from the history, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Delete menu item and the History... submenu item. 3 id_tests =TITLE id_tests =TITLE Identifying a Test to DTM =INCLUDE mu m_mu m_cm m_ctm DTM recognizes noninteractive, interactive and DECwindows tests by their test descriptions. You must create a test description before you can execute a noninteractive test or record a DECwindows or interactive terminal test. To create a test description, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Create menu item and the Test submenu item. 3. Choose the type of test you want to create from the submenu off the Test submenu. The resulting dialog box contains text entry fields to specify the test attributes. 3 grouping_tests =TITLE grouping_tests =TITLE Creating a Test Group =INCLUDE mu m_mu m_cm m_cgpm =INCLUDE mu m_mu m_im m_itm You can organize tests into groups so that many tests can be treated as a single entity. To create an empty group in the DTM library into which you subsequently place tests, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Create menu item and the Group... submenu item. To place a test or group into an existing group, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Insert menu item and either the Test... or Group... submenu item. 2 create =TITLE create =TITLE Creating a Test =INCLUDE overview set_up id_tests A test exercises a software application by supplying it with valid and invalid data to determine the integrity of the software. You must plan and create the tests. You can create three types of tests, depending on the needs of the application: o Noninteractive test o Interactive terminal test o DECwindows test You must identify each test to DTM in a test description. After creating a test description, you can place the test into a collection for execution. 3 cre_noninter =TITLE cre_noninter =TITLE Creating a Noninteractive Test =INCLUDE overview set_up id_tests =INCLUDE mu m_mu m_cm m_ctm A noninteractive test is a test whose template file is a DCL command file. To create a noninteractive test, perform the following steps: 1. Write the test using a text editor. 2. Write the template file, which can be the test file itself or a OpenVMS command file that executes the specified test. 3. Create the test description. 3 cre_inter_term =TITLE cre_inter_term =TITLE Creating an Interactive Terminal Test =INCLUDE overview set_up id_tests =INCLUDE overview create rec_inter_term =INCLUDE mu m_mu m_cm m_ctm =INCLUDE mu t_mu t_rm To create an interactive terminal test, perform the following steps: 1. Create the test description for the test. 2. Record the test by running an application and generating input and output interactively. With interactive terminal tests, you can also run an application with a previously created input file. 3 rec_inter_term =TITLE rec_inter_term =TITLE Recording an Interactive Terminal Test To record an interactive terminal test, perform the following steps: 1. Pull down the Testing menu. 2. Choose the Record menu item and the Terminal... submenu item. 3. Fill in the appropriate fields and press the OK button. DTM begins the recording session by creating a task and displaying the DCL prompt. Every action during the session is then recorded until you end the recording session by typing CTRL/P twice. The Record functions record and place the input into a template file (filename.SESSION) and the output into a benchmark file (filename.BMK). 4 rec_commands =TITLE rec_commands =TITLE Record Control Key Sequences =INCLUDE mu t_mu t_rm Use the default termination character (CTRL/P) to suspend a recording session. You can then issue a Record control key sequence. The following table contains the Record control key sequences. Key Sequence Function CTRL/P B Starts an automatic screen comparison and ends a manual screen comparison for terminal- based tests. This control key sequence does not apply to DECwindows tests. CTRL/P ! Invokes a text editor so that you can enter a comment into the session file. You end the comment by pressing CTRL/Z. This control key sequence is not available for DECwindows testing. CTRL/P E Ends an automatic screen comparison and begins a manual screen comparison for terminal-based tests. This control key sequence does not apply to DECwindows tests. CTRL/P Ends the recording session and returns CTRL/P control to the previous command level. When you terminate a recording session this way, DTM saves the session and benchmark files. CTRL/P ? Displays the current screen comparison mode and lists the available recording functions. This control key sequence is not available for DECwindows testing. CTRL/P I Inserts an input file into the session file you are recording. DTM prompts you for the input file specification. Input files are available for interactive terminal tests only. CTRL/P C Marks a screen for comparison. This is the only way to mark screens for DECwindows tests. You must mark the specific screen that you want to compare for DECwindows tests. DTM automatically compares all screens for terminal-based tests. This control key sequence is used in terminal-based tests when automatic comparison is disabled. CTRL/P Terminates an interactive recording session, CTRL/Z saves the session file, creates (or modifies) the test description, but does not save the benchmark file. CTRL/P W Invokes a prompt for you to specify a wait time. You issue this control key sequence at the place in a session file that you want the wait to occur. When the session file is subsequently played or executed, the wait time is included in the session file. CTRL/P Aborts an interactive recording session CTRL/C without saving any of the generated files. 4 cre_inp_fl =TITLE cre_inp_fl =TITLE Creating an Input File =INCLUDE mu t_mu t_xtm =INCLUDE mu t_mu t_rm An input file is a text file that you can edit. DTM translates and writes all input, commands, and comments in a session file to an input file. You can use an input file to create a new session file from an existing session file for an interactive terminal test only. To create an input file, perform the following steps: 1. Pull down the Testing menu. 2. Choose the Extract... menu item. To use the input file to create a new session file, perform the following steps: 1. Pull down the Testing menu. 2. Choose the Record menu item and the Terminal... submenu item. The Record dialog box contains an Input file field for use with an interactive terminal test. 3 cre_decw =TITLE cre_decw =TITLE Creating an Interactive DECwindows Test =INCLUDE overview set_up id_tests =INCLUDE overview create rec_inter_term rec_commands =INCLUDE overview create rec_dw_test =INCLUDE mu m_mu m_cm m_ctm =INCLUDE mu t_mu t_rm To create an interactive DECwindows test, perform the following steps: 1. Create the test description for the test. 2. Record the test by running applications and generating input and output interactively. 3 rec_dw_test =TITLE rec_dw_test =TITLE Recording a DECwindows Test To record an interactive terminal test, perform the following steps: 1. Pull down the Testing menu. 2. Choose the Record menu item and the DECwindows... submenu item. 3. Fill in the fields and press the OK button. You begin the recording session by pressing Compose/S. DTM then starts the specified application and records every action during the session until you end the session by pressing Compose/P. You mark screen images for comparison by pressing Compose/M. DTM records and places the input into a template file (test_name.SESSION) and the output into a benchmark file (test_name.BMK). 2 run =TITLE run =TITLE Running Tests within a Collection Before you can execute tests, you must make them a part of a collection. Then you can execute the tests in batch mode or interactively as part of the collections. A collection is a "snapshot" of the specified test descriptions and their related files at the time the collection is created, Therefore, you must re-create the collection if you change a test description, group, or their contents. 3 cre_coll =TITLE cre_coll =TITLE Creating a Collection =INCLUDE mu m_mu m_cm m_c_cl To create a collection, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Create menu item and the Collection... submenu item. The Collection dialog box enables you to place tests and groups of tests into a collection. It also enables you to specify options, such as collection prologue and epilogue files and collection variables. 3 recre_coll =TITLE recre_coll =TITLE Recreating a Collection =INCLUDE mu m_mu m_rcm To re-create a collection, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Recreate... menu item. If you change a test description or the contents of a group after including it in a collection, you must re-create the collection to include the changes. CAUTION Before recreating a collection, be sure that the prologue and epilogue files associated with the collection and template, prologue, and epilogue files associated with the tests in the collection exist in their proper directories. If DTM cannot find the template, prologue, or epilogue files specified in the original CREATE TEST_DESCRIPTION command, the original collection is deleted but a new collection is not created. 3 del_colls =TITLE del_colls =TITLE Deleting a Collection =INCLUDE mu m_mu m_dm m_dcm To delete a collection, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Delete menu item and the Collection... submenu item. 3 exec_tests =TITLE exec_tests =TITLE Executing a Collection =INCLUDE overview examine disp_results =INCLUDE overview views cre_nvu =INCLUDE mu t_mu t_sbm =INCLUDE mu t_mu t_rm When you execute a test collection, DTM sets up the test environment and executes all the tests in the collection. Each test in a collection generates a separate result file, which contains the output generated by the template file. DTM automatically compares the result file to a test's benchmark file and logs any discrepancies in a difference file. After DTM compares the collection, you can view test results using DTM's Review and Display functions. To execute an existing collection, perform the following steps: 1. Create a Collection view. 2. Click on (highlight) the collection you want to execute. 3. Pull down the Testing menu. 4. Choose the Submit... or Run... menu item. The Submit... menu item enables you to execute the selected collection in batch mode; the Run... menu item enables you to execute the collection interactively. 3 test_steps =TITLE test_steps =TITLE Steps in Test Execution =INCLUDE overview var_use =INCLUDE overview prol_use =INCLUDE overview user_filt =INCLUDE overview filt_use =INCLUDE overview compare DTM performs the following steps when it executes tests within a collection: 1. Defines the DTM$COLLECTION_NAME variable as the collection name 2. Defines any global variables with values as assigned when the collection was created 3. Executes the collection prologue file, if any perform the 4. Calls a generic command file in the DTM library to following tasks for each test: a. Defines local variables b. Defines the DTM$TEST_NAME variable as the test name for the current test c. Executes the test prologue, if any d. Executes the test template e. Defines the DTM$RESULT variable as the test result name of the current test f. Executes the test epilogue, if any g. Executes any user-defined filters h. Executes filters provided by DTM i. Undefines local variables j. Undefines the DTM$RESULT variable 5. Compares the result file with the benchmark file 6. Executes the collection epilogue file, if any 3 stop_coll =TITLE stop_coll =TITLE Stopping a Collection =INCLUDE mu t_mu t_sm To terminate a collection running interactively, click on the Delete Task button in the Task dialog box. To terminate a collection executing in batch mode, perform the following steps: 1. Pull down the Testing menu. 2. Choose the Stop... menu item. 2 compare =TITLE compare =TITLE Comparing Test Results to Benchmarks For every completed test in a collection, DTM performs the following steps: 1. Compares the result file to the benchmark file, if any 2. Saves the comparison status for the test 3. Saves any differences in a difference file 4. Deletes the result file if it matches the benchmark DTM will not automatically complete the comparison process if any of the following conditions exist: o You chose the option not to compare when you created the collection o A test in the collection is only partially run o Some of the tests in the collection do not run In each of these cases, you need to manually initiate the comparison of test results before you can review the results. 3 man_compare =TITLE man_compare =TITLE Manually Comparing Test Results =INCLUDE mu t_mu t_cm To manually compare test results, perform the following steps: 1. Pull down the Testing menu. 2. Choose the Compare... menu item. The Compare dialog box enables you to choose the type of test comparison, the differences format, and special characters that DTM should ignore during a comparison. 3 compare_status =TITLE compare_status =TITLE Test Comparison Status DTM reports one of the following comparison states for a test in a view window: Comparison Status Meaning Comparisons Comparison could not be completed aborted New test Test lacks a benchmark file Not run Test did not run during a partially run collection Successful Benchmark and result files match Unsuccessful Benchmark and result files do not match Updated Benchmark file updated after the comparison for a collection 2 examine =TITLE examine =TITLE Reviewing Test Results After running and comparing a collection, you can review the test results with DTM's Review functions. These functions include updating test benchmarks and Display functions. If DTM finds differences during the comparison process, it creates and stores result and difference files. You can display these files; the benchmark file can also be displayed. 3 open_coll =TITLE open_coll =TITLE Making a Collection Available for Review =INCLUDE overview views cre_nvu =INCLUDE overview examine disp_results =INCLUDE mu t_mu t_rm To review a collection, perform the following steps: 1. Create a Collection View. 2. Click on (highlight) the specific collection you want to review. 3. Pull down the Testing menu. 4. Choose the Review menu item and the Open submenu item. Once you open a collection, the database is locked. As primary reviewer, only you can update benchmarks. However, by clicking on the Read only button in the Review Collection dialog box, you become a read- only reviewer; you can display the contents of the collection, but you cannot update benchmarks. 4 update =TITLE update =TITLE Creating and Updating Benchmarks =INCLUDE overview views cre_nvu =INCLUDE mu t_mu t_rm To create or update a benchmark file, perform the following steps: 1. Create a Collection View. 2. Click on (highlight) the collection whose benchmarks you want to create or update. 3. Pull down the testing menu. 4. Choose the Review menu item and the Open submenu item. 5. Click on the OK button in the resulting Review Collection dialog box. 6. In the collection view, expand the collection by double clicking on the collection name. 7. Click on (highlight) the result description (preceded by an icon) whose benchmark you want to create or update. 8. Pull down the testing menu. 9. Choose the Review menu item and the Update... submenu item. DTM automatically creates the first benchmark files from the recorded session's output for both DECwindows and terminal interactive tests. For noninteractive tests, you create the first benchmark file from a result file. You update the benchmark file using the result file when results do not match the previous benchmark file but are the expected results for a test. 3 disp_results =TITLE disp_results =TITLE Displaying Test Results, Differences, and Benchmarks =INCLUDE overview views cre_nvu =INCLUDE mu t_mu t_dm To display a result, difference, or benchmark file, perform the following steps: 1. Create a Collection View. 2. Click on (highlight) the collection whose files you want to display. 3. Pull down the Testing menu. 4. Choose the Review menu item and the Open submenu item. 5. Click on the OK button in the resulting Review Collection dialog box. 6. In the collection view, expand the collection by double clicking on the collection name. 7. Expand the result description (preceded by an icon) by double clicking on it. 8. Double click on (highlight) the specific file (result, difference, or benchmark) that you want to display. 2 prol_use =TITLE prol_use =TITLE Using Prologues and Epilogues =INCLUDE overview run test_steps Prologue files and epilogue files enable you to control the DTM test environment. Both are command files that you create and associate with tests and collections. You use prologue files to set up test conditions before recording a test, running a test in a collection, or executing the collection as a whole. You use epilogue files to clean up or filter files after recording a test, running a test in a collection, or executing the collection as a whole. Tests, prologue files, and epilogue files execute in the following order: 1. Collection prologue file 2. Test1 prologue file 3. Test1 4. Test1 epilogue file 5. Test2 prologue file 6. Test2 7. Test2 epilogue file . . . 8. Collection epilogue file 3 cre_coll_prols =TITLE cre_coll_prols =TITLE Creating a Collection Prologue and Epilogue You create prologue and epilogue files with a text editor. You must then identify the files to DTM. You can specify default collection prologue and epilogue files. All subsequently created collections invoke this file whenever DTM executes the collections. You can override the default specifications by explicitly specifying a different collection prologue or epilogue file when you create the collection. When you override the default collection prologue and epilogue files, you do so only for the newly created collection. 4 cre_coll_def_pro =TITLE cre_coll_def_pro =TITLE Creating Default a Collection Prologue and Epilogue To specify default collection prologue and epilogue files when you create a library, perform the following steps: 1. Pull down the Library menu. 2. Choose the Create... menu item. To specify default collection prologue and epilogue files when the library exists, perform the following steps: 1. Pull down the Library menu. 2. Choose the Modify... menu item. In both cases, enter the names for the default collection prologue and epilogue files in the Prologue and Epilogue fields. 4 cre_coll_spec_pro =TITLE cre_coll_spec_pro =TITLE Overriding a Default Collection Prologue and Epilogue =INCLUDE mu m_mu m_cm m_c_cl m_ccdb You can override the default collection prologue and epilogue files for a specific collection when you create the collection. Enter a new file name in the Prologue and Epilogue fields to override the default collection prologue and epilogue files. To associate prologue and epilogue files with a specific collection, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Create menu item and the Collection... submenu item. 3. Click on the Options... button. 3 cre_test_prols =TITLE cre_test_prols =TITLE Creating Test Prologues and Epilogues =INCLUDE mu m_mu m_mdm m_mdtm m_mtdb To associate a test prologue or epilogue file with a test when you create a test description, enter the Prologue or Epilogue directory and filename into the field. To associate a test prologue or epilogue file with a test when the test description exists, perform the following steps: 1. From the Test View, click on the test to be modified. 2. Pull down the Maintenance menu. 3. Choose the Modify menu item and the Test... submenu item. 4. Click the button for Prologue and/or Epilogue 5. Type the directory and file name into the Prologue and/or Epilogue box Test prologue and epilogue files are executed whenever DTM records or executes the test. 2 var_use =TITLE var_use =TITLE Using Variables You can create both global and local variables, depending on the needs of your tests. DTM defines all global variables in the current library at the beginning of every test recording or collection run. Local variables are only defined while the test with which they are associated is being recorded or run. DTM also supplies built-in variables. You can use both the variables you add to the DTM library and built-in variables in a test's template, prologue, or epilogue file. You can use global variables in collection prologue and epilogue files. 3 cre_vars =TITLE cre_vars =TITLE Creating Variables =INCLUDE mu m_mu m_cm m_cvm To create global and local variables, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Create menu item and the Variable... submenu item. The Variable dialog box enables you to specify the value, scope, use, and type of variable. All tests can use global variables. However, you must explicitly associate local variables with a test description. 4 ass_lvar =TITLE ass_lvar =TITLE Assigning Local Variables to Test Descriptions To assign a previously defined local variable to an existing test or to a test that you are creating, perform the following steps: 1. From the Test View, click on the test to be modified. 2. Pull down the Maintenance menu. 3. Choose the Modify menu item and the Test... submenu item. 4. Click the Variables button 5. Enter the local variable into the Variables field. If you are creating a test, enter the local variable into the Variables field. 3 vue_vars =TITLE vue_vars =TITLE Displaying Current Variable Values To change the active view window to an unrestricted variable view, perform the following steps: 1. Pull down the View menu. 2. Choose the Variable... menu item. To create a new view window displaying variables, perform the following steps: 1. Pull down the View menu. 2. Choose the New menu item and the Variable submenu item. 3 mod_vars =TITLE mod_vars =TITLE Modifying Current Variables =INCLUDE mu m_mu m_mdm m_mvrm To modify current default variables in the DTM library, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Modify menu item and the Variable... submenu item. 3 over_vars =TITLE over_vars =TITLE Overriding Variable Default Values =INCLUDE mu m_mu m_mdm m_mdtm =INCLUDE mu m_mu m_cm m_ctm =INCLUDE mu m_mu m_cm m_c_cl To override a default variable for an existing test, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Modify menu item and the Test... submenu item. 3. Click on the Options... button. Most tests use a variable's default value. For tests that require special handling or special variable values, you can override existing variables. You can also override a variable's default value when you create the test by clicking on the Options... button in the Create Test dialog box, then entering the variable and a replacement value into the Variables field. You can override a global variable for a collection when you create the collection by clicking on the Options... button in the Create Collection dialog box. 3 del_vars =TITLE del_vars =TITLE Deleting Variables =INCLUDE mu m_mu m_dm m_dvm To delete a variable from the DTM library, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Delete menu item and the Variable... submenu item. 3 control_vars =TITLE control_vars =TITLE Using Variables to Control Commands The following logical variables can be used to control the behavior of certain commands. They are fully described in the manual Guide to DIGITAL Test Manager for OpenVMS Systems or in the product Release Notes. o DTM$DATE_FILTER_FIRST o DTM$DATE_FILTER_MAX_YEAR o DTM$DATE_FILTER_MIN_YEAR o DTM$DATE_FILTER_STRING o DTM$DATE_MASK_MAX_YEAR o DTM$DATE_MASK_MIN_YEAR o DTM$DELAY_TIMEOUT o DTM$IGNORE_PROLOGUE_ERRORS o DTM$OMIT_PRINTABLE_SCREENS o DTM$RETAIN_RESULTS o DTM$TIME_FILTER_NO_SPACE 2 filt_use =TITLE filt_use =TITLE Using Filters You can mask data that varies in test results from one test run to the next using DTM filters. You can also mask data during the recording of a test to produce a filtered benchmark file. DTM filters change specified data types (like time stamps) to ASCII characters of a standard format. For example, a OpenVMS time stamp of 13:20:23.0002 can be changed to hh:mm:ss.xxxx. The following table lists the available filters: Keyword Filter ALL Specifies that all the filters in this table be used DATE Where the date form is abbreviated, the date filter replaces date stamps by substituting a "d" for each displayed number of the day of the month, an "m" for each displayed letter of the month, and a "y" for each displayed number of the year. Where the date form is spelled out, the month name is replaced by "month", the numeric day is replaced by "day", and the year is replaced by "year". The following list shows some examples of the date filtering functions; this list is not all inclusive. 17-OCT-1998 with dd-mmm-yyyy 17 OCT 98 with dd mmm yy 98.OCT.17 with yy.mmm.dd 10/17/98 with mm/dd/yy 1998/10/17 with yyyy/mm/dd October 17, 1998 with month day, year Oct. 17, 1998 with month day, year 17.October.1998 with day.month.year 98-October-17 with year-month-day TIME Replaces time stamps with the following forms: 15:37:53.22 with hh:mm:ss.xxxx 15:37:53 with hh:mm:ss 15:37 with hh:mm 3:37 PM with hh:mm xm 15H37m with hhHmmm 15H37' with hhHmm' 15.37 h with hh.mm h 15 h 37"53 s with hh h mm"ss s 15 h 37 min with hh h mm min kl 15.37 with kl hh.mm h 15.37 with h hh.mm FILE_NAMES Replaces the file names with FILENAME.EXT DIRECTORIES Replaces the directory specification field in the file specification with DISK:[DIRECTORY] TRACE_BACK Replaces 32-bit memory addresses with xxxxxxxx and 64-bit memory addresses with xxxxxxxx xxxxxxxx VERSION Replaces file versions with VERSION 3 cre_filt =TITLE cre_filt =TITLE Applying Filters to a Test =INCLUDE mu m_mu m_cm m_ctm =INCLUDE mu m_mu m_mdm m_mdtm =INCLUDE mu t_mu t_ftm =INCLUDE overview run recre_coll =INCLUDE mu m_mu m_rcm To apply a filter to result files when you create or modify a test description, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Create or Modify menu item (depending on whether this a new or existing test) and choose the Test... submenu item from either resulting submenu. 3. Click on the Options... button. 4. Enable the filters you want applied to the test. To apply filters to any file, including files not associated with DTM, perform the following steps: 1. Pull down the Testing menu. 2. Choose the Filter... menu item. You must re-create any test collections that include modified tests. 3 del_filt =TITLE del_filt =TITLE Deleting Filters from a Test =INCLUDE mu m_mu m_mdm m_mdtm =INCLUDE overview run recre_coll =INCLUDE mu m_mu m_rcm To delete a filter from a test description, perform the following steps: 1. Pull down the Maintenance menu. 2. Choose the Modify menu item and the Test... submenu item. 3. Click on the Options... button. 4. Disable a filter by clicking on the filter name or button. You must re-create any test collections that include modified tests. 2 user_filt =TITLE user_filt =TITLE User Defined Filters =INCLUDE overview var_use =INCLUDE overview run The user defined filter facility provides a mechanism whereby filters written as DEC Text Processing Utility (DECTPU) programs are automatically executed when tests are run. These filters are referred to as user filters. The implementation enables users to solve easily many filtering problems, often with a single-line program, whilst allowing full access to the facilities of DECTPU to solve more complex cases. To implement a new filter, a file containing the required DECTPU commands is created. There are a number of predefined patterns and a global replace procedure provided which can be used to build the commands. For example, the following command will replace device names that precede a directory with the string "DEVICE": global_replace( identifier + ':[' , 'DEVICE:[' ) Procedure global_replace is similar to the TPU pattern style feature of Language-Sensitive Editor and filters can be developed using that facility. To associate a user filter with a test, a logical variable starting with the characters "DTM$UF_" is created. The value of the variable is the file specification of the file containing the DECTPU commands. The variable is then associated with the test in the usual way. When the test is run, as part of a collection, the filter will be applied. 3 uf_variables =TITLE uf_variables =TITLE Variables for User Filters =INCLUDE overview var_use Variables with names beginning with the string "DTM$UF_ " are used to control user filters. These variables must be logical variables but can be global or local. When tests are run that have associated variables whose names begin with the string "DTM$UF_", DTM will apply the user filters contained in the files referenced by the value of those variables. Only a single file may be referenced by each variable. The specified files are executed by DEC Text Processing Utility (DECTPU). If more than one user filter variable is associated with a test, the files are executed in the lexicographic order of the variable names. The user filters are applied before any built-in filters that are also specified for the test. User filter files can be located either in OpenVMS directories or in Code Management System (CMS) libraries. Files may be specified using logical names including logical names that specify search lists. Wildcards cannot be used. For files in CMS libraries, the most recent generation on the main line of descent is used. Before the first file is executed the file to be filtered is read into the DECTPU buffer "filter_ buffer". Next, the file specified by the logical name DTM$UFDEFINES is executed. The system logical name DTM$UFDEFINES references the file SYS$LIBRARY:DTM$UFDEFINES.TPU, which contains definitions of a global replace procedure and patterns which can be used in building filters. This logical can be redefined to point to a custom file. Any errors in accessing the user filter files or in executing the DECTPU commands will be reported. However, they will not cause the filter operation to fail, and any remaining user and built-in filters will be applied. After all the user filters have been applied, the file being filtered will be written out. If any built-in filters are also specified, they are applied to the newly created file, resulting in a second new version. In order to disable a user filter that is defined with a global variable for a particular test, define the value of the variable for the test as a string containing only spaces. 3 uf_record =TITLE uf_record =TITLE User Filters with Terminal Tests =INCLUDE overview create rec_inter_term When using the FILTER option to filter the benchmark produced by recording an interactive terminal test, user filters associated with the test will be applied, provided that the VARIABLES option is also used. 3 uf_global_rep =TITLE uf_global_rep =TITLE Procedure global_replace The supplied file SYS$LIBRARY:DTM$UFDEFINES.TPU contains a global replace procedure and some predefined patterns that can be used to build filters. The specification of procedure global_replace is as follows: PROCEDURE global_replace ( pattern_to_replace, replacement_string; search_mode, evaluate_replacement, convert_linefeeds) DESCRIPTION: Replace all occurrences of a given pattern with a given string in the buffer "filter_buffer". PARAMETERS: pattern_to_replace The pattern to be replaced. replacement_string The string to be substituted. search_mode (optional) The mode of pattern matching to be used when searching for the pattern. Should be one of: NO_EXACT (default) EXACT TPU$K_SEARCH_CASE TPU$K_SEARCH_DIACRITICAL evaluate_replacement Specifies whether the replacement (optional) string is to be evaluated. Should be one of: OFF, 0 (default) ON, 1 If specified as ON or 1, the replacement string is evaluated before use. This is needed if the replacement string contains any partial pattern variables. In this case, any string literals in the replacement string must be specified as nested strings and partial pattern variables converted to strings using the TPU procedure STR. convert_linefeeds Specifies whether any linefeed (optional) characters in the replacement string are to be converted into line breaks. Should be one of: OFF, 0 (default) ON, 1 3 uf_examples =TITLE uf_examples =TITLE Examples of User Filters The user filter examples are listed below. 4 uf_example1 =TITLE uf_example1 =TITLE Example 1: Simple Use of procedure global_replace The following example assumes that the disks are named UDISK{n} where {n} is a number, for example UDISK1, UDISK13. This filter replaces such disk names with the string "DISK_NAME": global_replace ( 'UDISK' + number, 'DISK_NAME') The pattern to replace is built from a string literal ('UDISK'), the concatenation operator (+) and the pattern "number" included in the supplied definitions file. The pattern "number" matches a sequence of digits. The replacement string is the string literal 'DISK_ NAME'. 4 uf_example2 =TITLE uf_example2 =TITLE Example 2: Using the Null Pattern This example uses the supplied "null" pattern with the DECTPU alternation operator to include an optional element in a pattern. Supposing that, in the previous example, some of the disk names do not include the leading "U", for example DISK7. The following filter replaces disk names with or without the leading "U": global_replace ( ("U"|null) + "DISK" + number, "DISK_NAME") 4 uf_example3 =TITLE uf_example3 =TITLE Example 3: Using Pattern Variables The following example filters dates in the form DD-MMM- YYYY, for example 11-OCT-1999. Because it only filters this one form of date, it is quicker than the built-in date filter which filters many different date formats. It is also not the exact equivalent of the built-in date filter in other respects, for example it treats 37-NOV-0999 as a date, but should be sufficient for most purposes. day := any(" 123") + digit; month := "JAN" | "FEB" | "MAR" | "APR" | "MAY" | "JUN" | "JUL" | "AUG" | "SEP" | "OCT" | "NOV" | "DEC"; year := any(digits,4); date := day + "-" + month + "-" + year; global_replace( date, "dd-mmm-yyyy"); This filter defines the pattern variables "day", "month" and "year" which are then used to define the pattern variable "date" used in the call to global_ replace. The "day" pattern uses the DECTPU function "any" to match either a space or one of the characters "1", "2" or "3", followed by a digit. The "month" pattern uses the DECTPU pattern alternation operator "|" to specify a list of alternative string literals. The "year" pattern uses the DECTPU function "any" with the supplied pattern "digits". The "4" parameter indicates that exactly 4 digits are to be matched. The "date" pattern concatenates these patterns and linking punctuation. 4 uf_example4 =TITLE uf_example4 =TITLE Example 4: Removing Blank Lines This filter removes blank lines using the DECTPU keywords LINE_BEGIN and LINE_END. global_replace( LINE_BEGIN + LINE_END, ''); The LINE_END keyword absorbs the new line. The above filter only replaces lines containing no characters. The following filter also replaces lines containing only spaces and tab characters: global_replace( LINE_BEGIN + (white_space|null) + LINE_END, ''); 4 uf_example5 =TITLE uf_example5 =TITLE Example 5: Using Partial Pattern Variables to Retain Context This example demonstrates how to use surrounding text to identify a string to be replaced without also replacing the surrounding text. The following filter replaces the month part of a date with the string "mmm". For example, the string "14-OCT-1999" will be replaced by the string "14-mmm-1999": day := any(" 123") + digit; month := "JAN" | "FEB" | "MAR" | "APR" | "MAY" | "JUN" | "JUL" | "AUG" | "SEP" | "OCT" | "NOV" | "DEC"; year := any(digits,4); date := (day + "-"@day_part) + month + ("-" + year@year_part); global_replace( date, 'str(day_part) + "mmm" + str(year_part)',,ON); The day part of the date and the "-" character are assigned to the partial pattern variable day_part and the year part of the date and preceding "-" assigned to year_part. These partial pattern variables are then included in the replacement string. When partial pattern variable are used in the replacement string they must be evaluated for each replacement. To do this, set the parameter evaluate_replacement to ON, as shown above. When the replacement string is to be evaluated, string literals must be nested inside further quotes. This is most easily done by using single quotes for the outer string and double quotes for any nested string literals, or vice-versa. Also, any partial pattern variables must be converted to strings using the DECTPU procedure STR. Note that including LINE_END in the definition of a partial pattern variable does not have the effect of retaining the line break. See example 6 for a resolution of this problem. 4 uf_example6 =TITLE uf_example6 =TITLE Example 6: Using LINE_END for Context If the search pattern contains LINE_END, the matched line break will be removed, causing the next line to be appended to the current line. To use LINE_END to only provide context for the search, the line break must be reinserted. This is done using the parameter convert_linefeeds. If the convert_linefeeds parameter is specified as ON, any linefeed characters appearing in the replacement string are removed and the built-in DECTPU procedure SPLIT_LINE is called at the point of the linefeed character. The following filter replaces any numbers that are the last characters on a line with the string "x": global_replace (number+LINE_END, "x"+lf,,,ON) The "lf" pattern is defined as a linefeed character in the supplied definitions file. If a LINE_END is included in a partial pattern variable, the line break can be retained by specifying the second optional parameter to the DECTPU STR procedure as a linefeed character, for example: global_replace (number+(LINE_END@sep), '"x"+STR(sep,lf)',,ON,ON) The second parameter to STR specifies the string that line breaks occurring in the first parameter should be converted to. Line breaks are retained by specifying the linefeed character and setting the parameter convert_linefeeds to ON. 4 uf_example7 =TITLE uf_example7 =TITLE Example 7: Using UNANCHOR to Replace Sections The DECTPU keyword UNANCHOR can be used to replace sections of text delimited by specified strings. The following replaces all text between the strings "/*" and "*/" with the string "/* Text deleted */". The text may run across line boundaries: global_replace ( "/*" + UNANCHOR + "*/", "/* Text deleted */") Note that while a similar effect is possible using the COMPARE/SENTINEL command, the filter can be applied to individual tests, whereas the /SENTINEL qualifier applies only to collections. 4 uf_example8 =TITLE uf_example8 =TITLE Example 8: Using Other DECTPU Commands The global_replace procedure can be used for many filtering tasks. However any DECTPU commands can be used to build filters. The file being filtered is read into the buffer "filter_buffer" before the user filters are applied and written out afterwards. The following filter uses the DECTPU EDIT procedure to convert all characters to upper case: EDIT( filter_buffer, UPPER, OFF) Note that while a similar effect is possible using the COMPARE/IGNORE=CASE command, the filter can be applied to individual tests, whereas the IGNORE qualifier applies only to collections. The following filter searches for numbers and replaces them only if they are in a specified range: POSITION (BEGINNING_OF (filter_buffer)); LOOP found_range := SEARCH_QUIETLY (number, FORWARD); EXITIF found_range = 0; POSITION (END_OF(found_range)); MOVE_HORIZONTAL(1); value := INT(STR(found_range)); IF (value>350) AND (value<570) THEN COPY_TEXT ("XXX"); ERASE (found_range); ENDIF; ENDLOOP; The initial POSITION is required to ensure that the whole of the filter_buffer is processed, because the editing point is undefined at the start of each filter. Then, as each number is processed, the editing point is moved to the end of the number. The MOVE_HORIZONTAL procedure call is necessary because the previous POSITION leaves the editing point at the last character of the number, which would result in an immediate match on the next call to SEARCH_QUIETLY.