Tackling model selection and validation : an information theoretic criterion
Lamperti, Francesco (Scuola superiore Sant'Anna (Pisa, Itàlia))

Date:	2014
Abstract:	Simulated economies suffer intrinsically from validation and comparison problems. The choice of a suitable indicator quantifying the distance between the model and the data is pivotal to model selection. However, how to validate and discriminate between models are still open problems calling for further investigation, especially in light of the increasing use of simulations in social sciences. In this paper I present a new information theoretic criterion to measure how close models' synthetic output replicates the properties of observable time series without the need to resort to any likelihood function or to impose stationarity requirements. This indicator is sufficiently general to be applied to any kind of model able to simulate or predict time series data, from simple univariate models such as Auto Regressive Moving Average (ARMA) and Markov processes to more complex objects including agent-based or dynamic stochastic general equilibrium models. More specifically, I use a simple function of the L-divergence computed at different block lengths in order to select the model that is better able to reproduce the distributions of time changes in the data. To evaluate the L-divergence, probabilities are estimated across frequencies including a correction for the systematic bias. Finally, using a known data generating process, I show how this indicator can be used to validate and discriminate between different univariate models providing a precise measure of the distance of each model from the data.
Rights:	Aquest document està subjecte a una llicència d'ús Creative Commons. Es permet la reproducció total o parcial i la comunicació pública de l'obra, sempre que no sigui amb finalitats comercials, i sempre que es reconegui l'autoria de l'obra original. No es permet la creació d'obres derivades.
Language:	Anglès
Document:	Comunicació de congrés
Published in:	Social Simulation Conference. Bellaterra, Cerdanyola del Valles, 1a : 2014

Podeu consultar la compilació de les ponències presentades a Social Simulation Conference a: https://ddd.uab.cat/record/125597

9 p, 267.5 KB

The record appears in these collections:
Contributions to meetings and congresses > Papers and communications > UAB papers and communications