Trials
A Trial is a series of runs, performed with the same settings for all parameters. The only thing changing are the “random numbers” that Simul8 uses for sampling values from distributions. As the simulation is intended to resemble real life scenarios (i.e. with variability), it is important to run a simulation more than once. A Trial gives you more rounded results and improves accuracy in terms of proposed performance measures (results).
The purpose of a Trial is to check the reliability of results. At the end of just one run you have simulated one week in your organization. You have a result that is from one week. Just like real life, that could have been an average week, or a bad week, or a good week. We have no way of knowing what sort of week it was unless we run a few more of them.
A Trial is a run of several weeks and the trial results summarize the results of these several weeks.
How to run a Trial
To run your Trials, navigate to Home, and then click Run Trials. By default, the number of runs in a Trial are 5, and confidence limits - i.e. the percent to which calculations are accurate - is set to 95%. You can adjust these, and other Trial features by using the Trial Set Up window, available at the drop down under the Run Trials button.
Number of Runs
This lets you tell Simul8 how many runs to do in your trial. The more runs you do the more accurate your results will be, but obviously it will take longer. There is no fixed rule about how many runs you should do, it varies depending on lots of factors in your simulation. Use the Trial Calculator to have Simul8 work out the right number for you.
Base Random Number
The base random number stream can be changed if you want to make two trials, but use different random numbers for the two trials (This is not normally necessary). The actual random number stream sets used for the first run in the trial is the base set, this is incremented for each run in the trial.
Extend
The EXTEND button allows an already completed Trial to be extended to a new number of runs to increase accuracy (narrow confidence limits) without the need to re-run the previously executed runs.
Name
Set the name of the trial, this will be displayed in the KPI Summary window and is useful for documenting your results.
Antithetic
Select this to reverse all the random numbers. (At a low level, before random numbers are used in distributions, they are all in the range 0.00000 to 1.00000. Antithetic random numbers are exactly the same streams of numbers except they range from 1.00000 to 0.00000). Changing to the antithetic of a set of random numbers is a quick way to see if the random numbers are having a large impact on the result of a simulation.
Confidence Intervals
When you run a trial, your KPI summary results are displayed as confidence intervals rather than just single numbers.
The central column of figures gives the result averaged across the weeks in the trial. This gives a guide as to what we expect the long term average to be.
The left and right columns give you an indication of how reliable the central (average) figure is. In some cases, the average figure may be very reliable because, perhaps, there is little variation from week to week. In other cases, it may be less reliable. The confidence intervals help you establish how much trust you should put in the single average value. For example, an average of 50 when alternate weeks swing from 0 to 100 is very different, in terms of reliability, from an average of 50 when they vary only between 49 and 51.
Please note that the right and left values are confidence intervals of the results obtained during a trial run. When selecting the 95% limit, these values give you an idea of what the average value is going to be on 95% of the times. 80%, 90%, 98%, 99.8% and 99.9% limits are available by right clicking the mouse on where it says either Low 95% Range or High 95% Range.
MORE Plot
After each Trial, a MORE Plot is produced, which is useful for displaying risk and error to support decision making. It is available next to the average and the confidence interval values.
The pointer lines on the left and right indicate the low and high ranges, and the bars present the frequency of a result occurring within a Trial. The MORE plot shows how trial run results look on a graph, and thus makes it easier for you to examine the variability of your results.
Trial Calculator
The Trial Calculator recommends a number of runs to use for trials. Access the Trial Calculator from the drop down menu on the Trial button on the Home tab. The recommendation is based on your required precision of the confidence limits around the estimate of the mean for your simulation results (KPIs).
Add all the KPIs (from the left hand box into the right hand box) that you want to meet the required precision.
For more information on how the Trial Calculator works, and information on other related research see: http://www.wbs.ac.uk/go/autosimoa