language: en

Minimum Encoding Approaches for Predictive Modeling

Peter Grunwald
CWI Dept. of Algorithms and Architectures
P.O.Box 94079 NL-1090 GB Amsterdam, The Netherlands
Petri Kontkanen, Petri Myllymaki, Tomi Silander, Henry Tirri
Complex Systems Computation Group (CoSCo)
P.O.Box 26, Department of Computer Science FIN-00014 University of Helsinki, Finland

[To appear in the Proceedings of the Fourteenth International Conference on Uncertainty in Artificial Intelligence (Madison, WI, July 1998).]

Abstract We analyze differences between two information-theoretically motivated approaches to statistical inference and model selection: the Minimum Description Length (MDL) principle, and the Minimum Message Length (MML) principle. Based on this analysis, we present two revised versions of MML: a pointwise estimator which gives the MML-optimal single parameter model, and a volumewise estimator which gives the MML-optimal region in the parameter space. Our empirical results suggest that with small data sets, the MDL approach yields more accurate predictions than the MML estimators. The empirical results also demonstrate that the revised MML estimators introduced here perform better than the original MML estimator suggested by Wallace and Freeman.

1 INTRODUCTION