Lab Reduction for z-3-06-delete-account
v102103 z-3-06-delete-account.txt s_z-3-06-delete-account.html ../logev

Lab Reduction is a companion web based tool to Data Reduction except that it operates on real test data rather than simulated test data. The data reduction is performed in real time as the test executes. Anyone on the network can view the test results in real time during the testing.

Running Real Time Test Results Script

This tool is started as a DOS application in a directory which contains the test data. The syntax is:

perl labreduction.pl [filename.txt] [delayinsecs] [TocCor] [TocCov] [LabData] [LeFind] [LeNoFind] [Results] [CPC]

where all [fields] are optional and [blank] or [-] is default or are set as follows:
filename.txtIs the test file name, the default is testlog.txt. The report name is derived from this file name. The default report name is testlog.html. When entering a file name the .txt extension must be used (e.g. AAAa.txt, AAAb.txt).
delayinsecsThe script executes 100 times before ending, this is the delay between executions, the default is 20 seconds.
TocCorIf set to [y] correlates the log events to requirements.
TocCovIf set to [y] shows which requirements were addressed and which requirements were not addressed by the test results.
LabDataIf set to [y] shows the raw test data.
LeFindIf set to [y] shows the log events that were found. The default is [y].
LeNoFindIf set to [y] shows the log events NOT found in these test results.
ResultsIf set to [y] shows the test results correlated to the original log events. The deafult is [y].
CPCIf set to [all], [red], or [a cpc] shows req's satisfied and NOT satisfied by these test results.
FilterIf set to [y], filters items containing certain words from the reports. The deafult is [y].

The file containing the test results is automatically updated by the BOOT application for some test cases. In all other BOOT and RED test cases the file is manually updated by the testers by either copying log events from the emulator window or entering log events based on emulator profile points. Using appropriate file naming conventions, parallel testing can be performed.

This application requires the inclusion of a special file (translate.dat) that is created by the logevent application and stored with the instrumentation analysis results. This file translates the log event codes to the original english equivalents. It also requires a version of the TOC (s_toc.dat) to correlate the TOC requirements with the lab test results.

Analyzing The Test Results

The test results analysis begins during informal testing by examining the real time test results.

  1. Are the start and stop log events as expected
  2. Are any log events grossly out of place
  3. Do the log events and sequence look resonable
  4. Do the number of log events match previous test runs
  5. Do the source files and function names in the log events look resonable
  6. Does the layout of the log event sequence appear to be similar to previous test runs

If the results appear to be reasonable, copy and paste them into the test procedures. They will form the factory template that will be used during formal testing.

As part of an offline analysis, use the IAT reports and software source to locate the test result log events. Examine the software and look for potential missing log events or wrong long events. This is a final examination to confirm that the test results are as expected.

For dry run, create a single complete test results report following the sequence that is expected during formal test. Use these dry run test results to support the formal test.

At the conclusion of all the DRY RUN tests enable the log events found, log events not found, requirements correlation, and requirements coverage reports. Look for log events and requirements that were not captured during the all the test activity. If needed, create new tests to capture the missing requirements.

Analysis Reports Script Option Comment
Test CommentsComments Summary of the comments entered by testers into the test file.
PUIs VerifiedComments The list of PUIs verified in this test sequence. Use it in the filter service to compared different test sequences.
Req CoverageComments This is a list of PUIs and the number of times the PUI was encountered in a log event with this test sequence. Use this to gain confidence in the test coverage. For example, a PUI that has been hit 50 times was probably the result of slightly different code paths.
Test ResultsResults This is the test results in log event format. This analysis connects the raw numerical log events captured during lab testing with the original source code comment statements, source file, and C-function.
Missing Req'sResults This is a list of req's that should have been in this test but are missing. It is based on the IAT filter service where one set of reqs's are subtracted from another set of req's.

NOTE: These reports are option sensitive. If an option was not selected, the report link is not present.

Other Analysis Reports . TOC req's . simulated logevents . translate file . Error_Codes.txt

Settings

Paths
Main Path
Simulation Directory
Analysis Directory
Analysis Report Name
Test File Name

Settings case sensitive
Script Rerun Delay (sec)
CPC, Release, or Test Case Considered
Trace Events

Requirements Accounting - Fixed case sensitive
Future Events
De-Instrumented
Negative Paths

Requirements Accounting - User Defined case sensitive Categories
Name             Description                                         PUI's








Display Filters - flaten display case insensitive
Raw LCD Log Events
Raw PAD Log Events
Raw PIN Log Events
Raw LOC Log Events
Raw Any Log Events
Text Results

Ignore Data - for very fast bypassing of noise case insensitive
Bypass Noise Raw Events
Bypass Noise Text Results

Keywords copy paste REQs from word table column get converted to OR format - case insensitive
Red
Blue
Navy
Purple
Green
Maroon
case sensitive
Orange

Expected Req's From Test Procedure - copy & paste from doc case insensitive
Orange LE SV

LabData Test Comments Test Results LE Found LE NOT Found Show Only Keyword Events
TOC Correlation TOC Coverage Not in Software Baseline Missed Req's Func Call Seq Show Only Expected Req's
Filter Classified Server Config

11/23/2003 19:21:50

Data Reduction Reports for z-3-06-delete-account

Test Comments Summary Comments entered by testers into test data file

Log Events Data

3-6 Delete User Account B1.0

Tester: ws

1. Delete Account
1. select delete account option
. . . HMI-30600-5
LOG-30600-20
1. select account to delete
. . . HMI-30600-3
LOG-30600-100
1. confirm delete request
. . . HMI-30600-3
LOG-30600-200
1. Done

      

Test Results Lab Data Connected To File Name, C-function, and Comment Statements 87 z-3-06-delete-account.txt 11/23/2003 19:21:50
3-6 Delete User Account B1.0

Tester: ws

1. Delete Account
1. select delete account option
HMI-30600-5
LOG-30600-20
3 8 1651 I log-09.c log.9.1 LE SV TOC-2134 this is a log-09.1.40 req to VALIDate Expected LE
7 12 1633 I log-06.c log.6.12 LE SV TOC-2117 this is a log-06.12.2 req to HALT Expected LE
12 17 1666 I log-09.c log.9.7 LE SV TOC-2149 this is a log-09.7.39 req to set Real Time Clock Expected LE
17 22 1657 I log-09.c log.9.2 LE SV TOC-2140 this is a log-09.2.56 req to VALIDate Expected LE
18 23 1664 I log-09.c log.9.7 LE SV TOC-2147 this is a log-09.7.11 req to INCREMENT Expected LE
1. select account to delete
HMI-30600-3
LOG-30600-100
3 31 1651 I log-09.c log.9.1 LE SV TOC-2134 this is a log-09.1.40 req to VALIDate Expected LE
7 35 1633 I log-06.c log.6.12 LE SV TOC-2117 this is a log-06.12.2 req to HALT Expected LE
12 40 1666 I log-09.c log.9.7 LE SV TOC-2149 this is a log-09.7.39 req to set Real Time Clock Expected LE
17 45 1657 I log-09.c log.9.2 LE SV TOC-2140 this is a log-09.2.56 req to VALIDate Expected LE
18 46 1664 I log-09.c log.9.7 LE SV TOC-2147 this is a log-09.7.11 req to INCREMENT Expected LE
21 49 1648 I log-09.c log.9.1 LE SV TOC-2131 this is a log-09.1.9 req to check PSOS Expected LE
36 64 1646 I log-08.c log.8.1 LE SV TOC-2130 this is a log-08.1.65 req to RECORD Expected LE
40 68 1638 I log-06.c log.6.13 LE SV TOC-2122 this is a log-06.13.48 req to TRANSFORM Expected LE
44 72 1655 I log-09.c log.9.2 LE SV TOC-2138 this is a log-09.2.45 req to HALT Expected LE
46 74 1637 I log-06.c log.6.13 LE SV TOC-2121 this is a log-06.13.34 req to FAIL Expected LE
49 77 1630 I log-06.c log.6.11 LE SV TOC-2114 this is a log-06.11.8 req to PROCESS Expected LE
51 79 1664 I log-09.c log.9.7 LE SV TOC-2147 this is a log-09.7.11 req to INCREMENT Expected LE
59 87 1639 I log-06.c log.6.13 LE SV TOC-2123 this is a log-06.13.56 req to STORE Expected LE
64 92 1641 I log-06.c log.6.14 LE SV TOC-2125 this is a log-06.14.33 req to DISABLE Expected LE
78 106 1667 I log-09.c log.9.8 LE SV TOC-2150 this is a log-09.8.16 req to COMPARE Expected LE
81 109 1662 I log-09.c log.9.4 LE SV TOC-2145 this is a log-09.4.88 req to UPDATE Expected LE
83 111 1655 I log-09.c log.9.2 LE SV TOC-2138 this is a log-09.2.45 req to HALT Expected LE
95 123 1626 I log-06.c log.6.10 LE SV TOC-2110 this is a log-06.10.24 req to UPDATE Expected LE
100 128 1631 I log-06.c log.6.11 LE SV TOC-2115 this is a log-06.11.15 req to call ISR Expected LE
1. confirm delete request
HMI-30600-3
LOG-30600-200
3 134 1651 I log-09.c log.9.1 LE SV TOC-2134 this is a log-09.1.40 req to VALIDate Expected LE
7 138 1633 I log-06.c log.6.12 LE SV TOC-2117 this is a log-06.12.2 req to HALT Expected LE
12 143 1666 I log-09.c log.9.7 LE SV TOC-2149 this is a log-09.7.39 req to set Real Time Clock Expected LE
17 148 1657 I log-09.c log.9.2 LE SV TOC-2140 this is a log-09.2.56 req to VALIDate Expected LE
18 149 1664 I log-09.c log.9.7 LE SV TOC-2147 this is a log-09.7.11 req to INCREMENT Expected LE
21 152 1648 I log-09.c log.9.1 LE SV TOC-2131 this is a log-09.1.9 req to check PSOS Expected LE
36 167 1646 I log-08.c log.8.1 LE SV TOC-2130 this is a log-08.1.65 req to RECORD Expected LE
40 171 1638 I log-06.c log.6.13 LE SV TOC-2122 this is a log-06.13.48 req to TRANSFORM Expected LE
44 175 1655 I log-09.c log.9.2 LE SV TOC-2138 this is a log-09.2.45 req to HALT Expected LE
46 177 1637 I log-06.c log.6.13 LE SV TOC-2121 this is a log-06.13.34 req to FAIL Expected LE
49 180 1630 I log-06.c log.6.11 LE SV TOC-2114 this is a log-06.11.8 req to PROCESS Expected LE
51 182 1664 I log-09.c log.9.7 LE SV TOC-2147 this is a log-09.7.11 req to INCREMENT Expected LE
59 190 1639 I log-06.c log.6.13 LE SV TOC-2123 this is a log-06.13.56 req to STORE Expected LE
64 195 1641 I log-06.c log.6.14 LE SV TOC-2125 this is a log-06.14.33 req to DISABLE Expected LE
78 209 1667 I log-09.c log.9.8 LE SV TOC-2150 this is a log-09.8.16 req to COMPARE Expected LE
81 212 1662 I log-09.c log.9.4 LE SV TOC-2145 this is a log-09.4.88 req to UPDATE Expected LE
83 214 1655 I log-09.c log.9.2 LE SV TOC-2138 this is a log-09.2.45 req to HALT Expected LE
95 226 1626 I log-06.c log.6.10 LE SV TOC-2110 this is a log-06.10.24 req to UPDATE Expected LE
100 231 1631 I log-06.c log.6.11 LE SV TOC-2115 this is a log-06.11.15 req to call ISR Expected LE
101 232 1632 I log-06.c log.6.11 LE SV TOC-2116 this is a log-06.11.16 req to STORE Expected LE
106 237 1654 I log-09.c log.9.1 LE SV TOC-2137 this is a log-09.1.61 req to FAIL Expected LE
107 238 1653 I log-09.c log.9.1 LE SV TOC-2136 this is a log-09.1.53 req to UPDATE Expected LE
116 247 1653 I log-09.c log.9.1 LE SV TOC-2136 this is a log-09.1.53 req to UPDATE Expected LE
118 249 1640 I log-06.c log.6.14 LE SV TOC-2124 this is a log-06.14.26 req to INHIBIT Expected LE
121 252 1643 I log-06.c log.6.14 LE SV TOC-2127 this is a log-06.14.51 req to detect ERROR Expected LE
126 257 1633 I log-06.c log.6.12 LE SV TOC-2117 this is a log-06.12.2 req to HALT Expected LE
129 260 1632 I log-06.c log.6.11 LE SV TOC-2116 this is a log-06.11.16 req to STORE Expected LE
133 264 1640 I log-06.c log.6.14 LE SV TOC-2124 this is a log-06.14.26 req to INHIBIT Expected LE
135 266 1633 I log-06.c log.6.12 LE SV TOC-2117 this is a log-06.12.2 req to HALT Expected LE
141 272 1628 I log-06.c log.6.10 LE SV TOC-2112 this is a log-06.10.34 req to INHIBIT Expected LE
152 283 1661 I log-09.c log.9.4 LE SV TOC-2144 this is a log-09.4.48 req to STORE Expected LE
153 284 1640 I log-06.c log.6.14 LE SV TOC-2124 this is a log-06.14.26 req to INHIBIT Expected LE
154 285 1664 I log-09.c log.9.7 LE SV TOC-2147 this is a log-09.7.11 req to INCREMENT Expected LE
155 286 1648 I log-09.c log.9.1 LE SV TOC-2131 this is a log-09.1.9 req to check PSOS Expected LE
158 289 1628 I log-06.c log.6.10 LE SV TOC-2112 this is a log-06.10.34 req to INHIBIT Expected LE
163 294 1628 I log-06.c log.6.10 LE SV TOC-2112 this is a log-06.10.34 req to INHIBIT Expected LE
164 295 1654 I log-09.c log.9.1 LE SV TOC-2137 this is a log-09.1.61 req to FAIL Expected LE
167 298 1657 I log-09.c log.9.2 LE SV TOC-2140 this is a log-09.2.56 req to VALIDate Expected LE
177 308 1646 I log-08.c log.8.1 LE SV TOC-2130 this is a log-08.1.65 req to RECORD Expected LE
178 309 1649 I log-09.c log.9.1 LE SV TOC-2132 this is a log-09.1.25 req to COMPARE Expected LE
179 310 1634 I log-06.c log.6.12 LE SV TOC-2118 this is a log-06.12.6 req to VALIDate Expected LE
182 313 1667 I log-09.c log.9.8 LE SV TOC-2150 this is a log-09.8.16 req to COMPARE Expected LE
184 315 1632 I log-06.c log.6.11 LE SV TOC-2116 this is a log-06.11.16 req to STORE Expected LE
187 318 1654 I log-09.c log.9.1 LE SV TOC-2137 this is a log-09.1.61 req to FAIL Expected LE
189 320 1657 I log-09.c log.9.2 LE SV TOC-2140 this is a log-09.2.56 req to VALIDate Expected LE
192 323 1659 I log-09.c log.9.3 LE SV TOC-2142 this is a log-09.3.10 req to STORE Expected LE
195 326 1644 I log-08.c log.8.1 LE SV TOC-2128 this is a log-08.1.30 req to COMPARE Expected LE
197 328 1631 I log-06.c log.6.11 LE SV TOC-2115 this is a log-06.11.15 req to call ISR Expected LE
1. Done

99XX series reserved for IAT self tests

PUIs Verified - All in this thread use this in the filter service to determine test run differences

s_z-3-06-delete-account.html |TOC-1806 |TOC-1834 |TOC-1845 |TOC-1855 |TOC-1896 |TOC-2001 |TOC-2002 |TOC-2003 |TOC-2007 |TOC-2008 |TOC-2009 |TOC-2010 |TOC-2011 |TOC-2012 |TOC-2013 |TOC-2014 |TOC-2015 |TOC-2016 |TOC-2017 |TOC-2019 |TOC-2020 |TOC-2021 |TOC-2022 |TOC-2023 |TOC-2024 |TOC-2025 |TOC-2027 |TOC-2028 |TOC-2029 |TOC-2030 |TOC-2031 |TOC-2033 |TOC-2034 |TOC-2035 |TOC-2036 |TOC-2038 |TOC-2042 |TOC-2043 |TOC-2044 |TOC-2047 |TOC-2048 |TOC-2049 |TOC-2050 |TOC-2051 |TOC-2053 |TOC-2056 |TOC-2057 |TOC-2058 |TOC-2060 |TOC-2061 |TOC-2062 |TOC-2063 |TOC-2064 |TOC-2065 |TOC-2066 |TOC-2067 |TOC-2068 |TOC-2069 |TOC-2070 |TOC-2071 |TOC-2072 |TOC-2076 |TOC-2077 |TOC-2078 |TOC-2080 |TOC-2081 |TOC-2082 |TOC-2084 |TOC-2087 |TOC-2088 |TOC-2089 |TOC-2090 |TOC-2091 |TOC-2092 |TOC-2095 |TOC-2096 |TOC-2097 |TOC-2099 |TOC-2100 |TOC-2102 |TOC-2103 |TOC-2104 |TOC-2105 |TOC-2107 |TOC-2108 |TOC-2109 |TOC-2110 |TOC-2112 |TOC-2114 |TOC-2115 |TOC-2116 |TOC-2117 |TOC-2118 |TOC-2121 |TOC-2122 |TOC-2123 |TOC-2124 |TOC-2125 |TOC-2127 |TOC-2128 |TOC-2130 |TOC-2131 |TOC-2132 |TOC-2134 |TOC-2136 |TOC-2137 |TOC-2138 |TOC-2140 |TOC-2142 |TOC-2144 |TOC-2145 |TOC-2147 |TOC-2149 |TOC-2150 |done

Total Verified Req's 114

Req Coverage - All in this thread

9 TOC-2076 9 TOC-2050 8 TOC-2102 7 TOC-2071 7 TOC-2047 6 TOC-2095 6 TOC-2012 6 TOC-2015 6 TOC-2009
6 TOC-2147 5 TOC-2024 5 TOC-2049 5 TOC-2117 5 TOC-2044 5 TOC-2014 5 TOC-2020 5 TOC-2090 5 TOC-2016
5 TOC-2062 5 TOC-2140 4 TOC-2035 4 TOC-2138 4 TOC-2088 4 TOC-2010 4 TOC-2011 4 TOC-2069 4 TOC-2031
4 TOC-2053 4 TOC-2096 4 TOC-2038 4 TOC-2077 4 TOC-2007 3 TOC-2068 3 TOC-2058 3 TOC-2137 3 TOC-2124
3 TOC-2103 3 TOC-2097 3 TOC-2100 3 TOC-2134 3 TOC-2149 3 TOC-2065 3 TOC-2130 3 TOC-1855 3 TOC-2112
3 TOC-2131 3 TOC-2008 3 TOC-2034 3 TOC-2115 3 TOC-2042 3 TOC-2116 3 TOC-2043 3 TOC-2084 3 TOC-1806
3 TOC-2150 3 TOC-1896 2 TOC-2136 2 TOC-2063 2 TOC-2066 2 TOC-2067 2 TOC-2030 2 TOC-2104 2 TOC-2105
2 TOC-2033 2 TOC-2107 2 TOC-2108 2 TOC-2070 2 TOC-2081 2 TOC-2145 2 TOC-2082 2 TOC-2089 2 TOC-2013
2 TOC-2121 2 TOC-2001 2 TOC-2122 2 TOC-2110 2 TOC-2123 2 TOC-2019 2 TOC-2125 2 TOC-2057 2 TOC-2114
2 TOC-2091 2 TOC-2092 2 TOC-2099 2 TOC-2021 2 TOC-2023 2 TOC-2025 2 TOC-2027 2 TOC-2060 1 TOC-2056
1 TOC-2128 1 TOC-2127 1 TOC-2061 1 TOC-2051 1 TOC-2078 1 TOC-2028 1 TOC-2002 1 TOC-2003 1 TOC-2064
1 TOC-2029 1 TOC-2132 1 TOC-2017 1 TOC-2022 1 TOC-2087 1 TOC-2048 1 TOC-2036 1 TOC-2109 1 TOC-2142
1 TOC-1834 1 TOC-2144 1 TOC-2080 1 TOC-2118 1 TOC-1845 1 TOC-2072

Total Verified Req's 114

Req's Missed In This Test Run This report with the default settings shows the req's that were MISSED in this test. It is based on the filter tool where an initial set of items are found and a second set is subtracted from the intial set. In this case the initial set are the expected req's as defined in the test procedure, and the second set are the req's found in the actual test data. Req's are Marked if they are not in the software. Req's are also Marked if they are turned off in the debugger.

Don't Show Located Items Show Removed Items Flip Locate & Remove

Locate: TOC-21[1234]\d|TOC-2150

Remove: s_z-3-06-delete-account.html |TOC-1806 |TOC-1834 |TOC-1845 |TOC-1855 |TOC-1896 |TOC-2001 |TOC-2002 |TOC-2003 |TOC-2007 |TOC-2008 |TOC-2009 |TOC-2010 |TOC-2011 |TOC-2012 |TOC-2013 |TOC-2014 |TOC-2015 |TOC-2016 |TOC-2017 |TOC-2019 |TOC-2020 |TOC-2021 |TOC-2022 |TOC-2023 |TOC-2024 |TOC-2025 |TOC-2027 |TOC-2028 |TOC-2029 |TOC-2030 |TOC-2031 |TOC-2033 |TOC-2034 |TOC-2035 |TOC-2036 |TOC-2038 |TOC-2042 |TOC-2043 |TOC-2044 |TOC-2047 |TOC-2048 |TOC-2049 |TOC-2050 |TOC-2051 |TOC-2053 |TOC-2056 |TOC-2057 |TOC-2058 |TOC-2060 |TOC-2061 |TOC-2062 |TOC-2063 |TOC-2064 |TOC-2065 |TOC-2066 |TOC-2067 |TOC-2068 |TOC-2069 |TOC-2070 |TOC-2071 |TOC-2072 |TOC-2076 |TOC-2077 |TOC-2078 |TOC-2080 |TOC-2081 |TOC-2082 |TOC-2084 |TOC-2087 |TOC-2088 |TOC-2089 |TOC-2090 |TOC-2091 |TOC-2092 |TOC-2095 |TOC-2096 |TOC-2097 |TOC-2099 |TOC-2100 |TOC-2102 |TOC-2103 |TOC-2104 |TOC-2105 |TOC-2107 |TOC-2108 |TOC-2109 |TOC-2110 |TOC-2112 |TOC-2114 |TOC-2115 |TOC-2116 |TOC-2117 |TOC-2118 |TOC-2121 |TOC-2122 |TOC-2123 |TOC-2124 |TOC-2125 |TOC-2127 |TOC-2128 |TOC-2130 |TOC-2131 |TOC-2132 |TOC-2134 |TOC-2136 |TOC-2137 |TOC-2138 |TOC-2140 |TOC-2142 |TOC-2144 |TOC-2145 |TOC-2147 |TOC-2149 |TOC-2150 |done

Marked: Bx_xxTOC-1001|done

Check if it was Manually verified
TOC-2111 This req is about log-06.10.28 the system shall check unix LOG 1.0

TOC-2113 This req is about log-06.11.1 the system shall reject LOG 1.0

TOC-2119 This req is about log-06.12.24 the system shall call isr LOG 1.0

TOC-2120 This req is about log-06.12.34 the system shall verify LOG 1.0

TOC-2126 This req is about log-06.14.42 the system shall enable LOG 1.0

TOC-2129 This req is about log-08.1.63 the system shall fail LOG 1.0

TOC-2133 This req is about log-09.1.29 the system shall transform LOG 1.0

TOC-2135 This req is about log-09.1.43 the system shall verify LOG 1.0

TOC-2139 This req is about log-09.2.51 the system shall check unix LOG 1.0

TOC-2141 This req is about log-09.3.3 the system shall compare LOG 1.0

TOC-2143 This req is about log-09.4.13 the system shall enable LOG 1.0

TOC-2146 This req is about log-09.7.9 the system shall recover LOG 1.0

TOC-2148 This req is about log-09.7.18 the system shall transform LOG 1.0

Reqs Verification Summary . . . Missed 13 . . . Marked 0 . . . DeInst 0 . . . Manual 0 . . . Found 28

11/23/2003 19:21:50 start
11/23/2003 19:21:54 end
4 secs
done