whatIf? Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Calibration Demystified ======

Calibration is the process of finding simulator input values for a historical time period which reproduce the observed simulator outputs over that period. To accomplish this a distinct calibrator framework is created alongside the simulator framework, within the same model family (see [[concepts:How it's Organized]]). There are two main motivations for calibration. The first is to look back at the history of a system through the same conceptual framework used to explore the future of that system - a prerequisite for trend analysis. The second is to get starting values - or initial conditions - of model stocks.

Take for example a simple population model in which

population<sub>t</sub> = population<sub>t-1</sub> + net immigration<sub>t-1</sub> + births<sub>t-1</sub> - deaths<sub>t-1</sub>

If historical data for population, births and deaths are available then calibration involves determining the historical net immigration levels.  For real-world models this is a non-trivial problem and requires a separate calibrator framework because:

  * Simulators are dynamic and data-rich; they contain many inter-related, multidimensional time-series variables which constrain the calibration solution. In the example above the population variable, rather than being a vector, might be a 3-dimensional variable stratified by sex, age and time.
  * Simulator variables in historical time may not be directly measured and therefore may have to be estimated, and/or there may be missing periods of observed data which require interpolation. For the population model example it is common to use a simulation time-step of 1 year, and while births and deaths data may be available for each year, population census count data is more likely to be collected every 5 or 10 years. The calibrator must deal with the varying temporal resolution of the datasets.
  * Data quality issues are common, and often manifest themselves through inconsistent or infeasible calibration results. For example, suppose that net immigration in the above example were broken into its component parts - immigration and emigration. If estimates for historical immigration were available and used in the calibration to estimate emigration levels, it is possible that a naive calibration procedure would produce some negative values for historical emigration. Of course, this is infeasible and so the calibration framework must consider these issues to produce an internally consistent and feasible calibration solution.
  * [more?]

[Explain the diagram below wrt the frameworks, relative to time, inputs and outputs.  Break observed historical data btwn observed parameters and observed outputs.]

The diagram below shows the relationship between a simulator framework and its corresponding calibrator framework - with respect to inputs, outputs and the time periods for which they operate. There are points to note about the diagram:
  * The simulator contains logic that transforms input variables 

{{:concepts:calibration.png|}}

[Compare/contrast what we mean by calibration to other modelling paradigms: specification, estimate, calibration, etc. E.g. Choice model calibration.]
Note that the term calibration used in the whatIf? Platform / Design Approach context differs from specification and estimation of a statistical model. In particular, the use of data is different as in statistical modelling the data are used to estimate the parameters in the model using an appropriate statistical technique, while in the design approach, **data which have not been measured, such as lifetimes of capital stocks, are estimated and then the data set is used to calibrate the model** [clarify this with Bert] FIXME. The calibration is successful if the model reproduces the data, given a set of historical input variables, and this is considered a necessary but not sufficient condition for a valid model. [1]

===== Related Articles =====
whatIf? Wiki

User Tools

Site Tools

Page Tools
