Next: Output Analysis for a Up: Calibration and Validation of Previous: Validating Input-Output Transformations

## Input-Output Validation: Using Historical Input Data

• In the previous bank example and other places, we used some artificial data as input to a simulation model.

• An alternative is to use past data as input (historical input data). Sometimes this is called trace-driven simulation.

• To conduct a validation test using historical input data, it is important that all the input data and all the system response data be collected during the same time period.

• Example 11.3 (The Candy Factory)
• The Candy Factory has three machines, the Candy maker, the Candy packer, and the Box maker.
• The three machines are connected by two conveyors.
• The machines have random break-down time.
• When a machine breaks down, it causes its incoming conveyors to be full and out-going conveyors to be empty.
• The time-to-failure (operating time) and down-time of each machine is recorded. For machine i

• The system responses: the production level , the number and the time of occurence of operator interventions were recorded.
• The model responses ( ) for the same parameters were collected for comparison to the corresponding system responses.
• The differences between and can be used to build test statistics (in this case the Student Test t) to see if the model is statistically the same as the real system.
• If there is only one set of input data, only one set of output can be obtained. The result doesn't carry much statistical significance.
• If K sets of input data are available, K runs of tests can be conducted. See Table 11.6 for an example.
• Let be the system responses and be the model responses, the differences are approximately normally distributed with mean and variance
• If the mean is zero, then statistically are the same as . A t test is conducted

and

• The t statistic is computed as

• The critical value where K is the degree of freedom. In our example, K is the number of input data sets (thus, the number of experiments run).
• If do not reject , otherwise reject .
• Example 11.4 (The Candy Factory continued) Use actual numbers (e.g. K = 5, etc.) to show how the above example works.

Next: Output Analysis for a Up: Calibration and Validation of Previous: Validating Input-Output Transformations
Meng Xiannong 2002-10-18