Similar questions

How will you validate simulation models ?

Explain in detail verification of simulation model.

Verification of models:

• Verification is defined as the process of correctly building a system model. Verification is done to ensure that:

• The model is programmed correctly. The algorithms have been implemented correctly.

• The model does not contain errors, oversights, or bugs. Verification ensures that the specifications is complete and that mistakes have not been made in implementing the model.

• Verification does not ensure the model: Solves an important problem. Meets a specified set of requirements. Correctly reflects the working of a real world process.

• The purpose of verification is to make sure that the conceptual model is reflected accurately in operational model. It asks questions regarding the correct implementation of the model, correct representation of input parameters and logical structure, etc.

• Verification is determining whether the simulation computer program performs as intended. Verification checks for translation of conceptual simulation model onto a correctly working program.

Validation of Models:

• The standard method to validate model is to construct a model of the existing system. Then, change the model appropriately in order to analyze each alternative.

• The model of the existing system can be validated by comparing its results against actual data obtained from the system under investigation. Goal of validation is to ensure that the simulation model is good enough so that it can be used to make decisions about systems that we ideally would like to work with.

• Ease or difficulty of the validation process depends on the complexity of the system being modelled and on whether a version of a system currently exists.

• A simulation model of a complex system can only be an approximation to the actual system, regardless of how much effort is put into development. There is no such thing as absolutely valid model!

• A simulation model is always developed for a particular purpose .a logbook of the simulation model’s assumptions should be updated on a regular basics and eventually should form integral part of final report.

• A simulation model should be validated relative to those measures of performance that will be actually used for decision making.

Validating Input- Output Transformations

• The ultimate test of a model, and in fact the only objective test of the model as a whole is the model’s ability to predict the future behavior of the real system when the model input data match the real inputs and when a policy implemented in the model is implemented at some point in the system.

• The structure of the model should be accurate enough to make good predictions for the range of input sets of interest.

• We can see the outputs of the systems as being a functional transform of the inputs based on parameter settings. i.e. the model accepts values of input parameters and transforms them into suitable outputs measures of performances.

(Example) : The Fifth National Bank of Jaspar

• The Fifth National Bank of Jaspar, as shown in the next slide, is planning to expand its drive-in service at the corner of Main Street. Currently, there is one drive-in window serviced by one teller. Only one or two transactions are allowed at the drive-in window, so, it was assumed that each service time was a random sample from some underlying population. Service times {$S_i$, i = 1, 2, ... 90} and inter arrival times {$A_i$, i = 1, 2, ... 90}

were collected for the 90 customers who arrived between 11:00 A.M. and 1:00 P.M. on a Friday. This time slot was selected for data collection after consultation with management and the teller because it was felt to be representative of a typical rush hour. Data analysis led to the conclusion that the arrival process could be modeled as a Poisson process with an arrival rate of 45 customers per hour; and that service times were approximately normally distributed with mean 1.1 minutes and standard deviation 0.2 minute. Thus, the model has two input variables:

1. Inter arrival times, exponentially distributed (i.e. a Poisson arrival process) at rate l = 45 per hour.

2. Service times, assumed to be N(1.1, $(0.2)_2$)

Validation using Historical Input data

• An alternative to randomly generated data – don’t mix different data sets

• File, Spreadsheet, or Database

  • {A1, A2,…,An} & {S1, S2,…Sn}

  • Feed data into the FEL

• Compare output to what happened in the real system

• May be able to use technology to collect historical data for use

Validation using a Turning Test

• What is the Turing Test?

• Generate 5 “fake” reports from simulation & mix with 5 real reports; ask experts if they can distinguish fake from real

• If cannot, then pass Turing Test!