
	Where results make sense

Topic: Minimum description length

Related Topics

Bayesian model comparison

Bayesian filtering

Bayesian inference

Likelihood principle

Bayes factor

Bayesian probability

Bayesian statistics

Bayesian

List of publications in statistics

Gaussian process regression

Induction (philosophy)

Bernoulli trial

Thomas Bayes

Harold Jeffreys

	Minimum description length - Wikipedia, the free encyclopedia
		The minimum description length principle is a formalization of Occam's Razor in which the best hypothesis for a given set of data is the one that leads to the largest compression of the data.
		One of the important properties of MDL methods is that they provide a natural safeguard against overfitting, because it implements a tradeoff between the complexity of the hypothesis (model class) and the complexity of the data given the hypothesis.
		MDL was not the first information-theoretic approach to learning; as early as 1968 Wallace and Boulton pioneered a related concept called Minimum Message Length (MML).
	en.wikipedia.org /wiki/Minimum_description_length (1331 words)

	Encyclopedia: Minimum description length (Site not responding. Last check: 2007-10-22)
		MDL is an attempt to remedy these, by: Computability theory is the branch of theoretical computer science that studies which problems are computationally solvable using different models of computation.
		Minimum message length (MML) is a formal information theory restatement of Occams Razor: even when models are not equal in accuracy, the one generating the shortest overall message is more likely to be correct (where the message consists of a statement of the model, followed by a statement of...
		MDL was introduced by Jorma Rissanen in 1978; it differs from MML in several ways, most notably thourgh the extensive use of one-part rather than two-part codes.
	www.nationmaster.com /encyclopedia/Minimum-description-length (2294 words)

	Minimum message length - Wikipedia, the free encyclopedia
		Minimum message length (MML) is a formal information theory restatement of Occam's Razor: even when models are not equal in accuracy, the one generating the shortest overall message is more likely to be correct (where the message consists of a statement of the model, followed by a statement of data encoded concisely using that model).
		The first is the length of the model, and the second is the length of the data, given the model.
		Advances in Minimum Description Length: Theory and Applications, M.I.T. Press (MIT Press), April 2005, ISBN 0-262-07262-9; and Chapter 11 (pp265-294): J.W.Comley and D.L. Dowe, Minimum Message Length, MDL and Generalised Bayesian Networks with Asymmetric Languages.
	en.wikipedia.org /wiki/Minimum_Message_Length (775 words)

	The Minimum Description Length Principle for Modeling Recording Channels
		For this family of models, the minimum description complexity is directly linked to the minimum required complexity of a detector.
		The description complexity is the sum of two terms: 1) the entropy of the sequence of uncorrelated Gaussian random variables driving the autoregressive filters, which decreases with the model order (i.e., model size), and 2) a penalty term proportional to the model size.
		We exploit this interpretation to formulate the minimum description length criterion for the magnetic recording channel corrupted by nonlinearities and signal-dependent noise.
	www.comsoc.org /sac/private/2001/apr/719_19sac04-kavcic.html (956 words)

	NIPS Workshop on MDL (Site not responding. Last check: 2007-10-22)
		The Minimum Description Length (MDL) Principle, which was originally proposed by Jorma Rissanen in 1978 as a computable approximation of Kolmogorov complexity, is a powerful method for inductive inference.
		The MDL principle states that the best explanation (i.e., model) given a limited set of observed data is the one that permits the greatest compression of the data.
		The former represents the shortest code length in a probabilistic sense with which the data can be encoded with use of the models, and the second the logarithm of the number of optimally distinguishable models from the given amount of data.
	quantrm2.psy.ohio-state.edu /injae/workshop.htm (2674 words)

	David Dowe's Minimum Message Length (MML) Inference page: MML, MDL, minimum encoding length inference, algorithmic ...
		Minimum Message Length, MDL and Generalised Bayesian Networks with Asymmetric Languages, Chapter 11 (pp265-294) in P. Gru:nwald, I. Myung and M. Pitt (eds.), Advances in Minimum Description Length: Theory and Applications, M.I.T. Press (MIT Press), April 2005, ISBN 0-262-07262-9.
		"Minimum Message Length, MDL and Generalised Bayesian Networks with Asymmetric Languages", by J. Comley and D.L. Dowe; Chapter 11 (pp265-294) in P. Grunwald, M. Pitt and I. Myung (eds.), Advances in Minimum Description Length: Theory and Applications, M.I.T. Press, April 2005, ISBN 0-262-07262-9.
		Dowe) "Minimum Message Length, MDL and Generalised Bayesian Networks with Asymmetric Languages", Chapter 11 (pp265-294) in P. Grunwald, M. Pitt and I. Myung (eds.), Advances in Minimum Description Length: Theory and Applications, M.I.T. Press, April 2005, ISBN 0-262-07262-9.
	www.csse.monash.edu.au /~dld/MDL.html (1311 words)

	Minimum description length (MDL)
		MDL is a statistical inference principle motivated by IT (Rissanen 1978, 1989).
		MDL has naturally lead to a strong interplay with statistics of the theory of universal data compression in IT.
		Notice that now the description length for the model class is constant over the considered (finite number of) classes, hence it does not enter the above comparison.
	www.itsoc.org /publications/nltr/98_mar/01csi/node12.html (318 words)

	Learn more about Minimum description length in the online encyclopedia. (Site not responding. Last check: 2007-10-22)
		The MDL community can be divided into two, according to whether the researcher views MDL as being equivalent to Bayesian model comparison, or different.
		The view that MDL is an approximation to Bayesian model comparison is explained in David MacKay's Information Theory, Inference, and Learning Algorithms.
		And in Bayesian inference, the likelihood of the model H (also known as the evidence for the model) is P(DH).
	www.onlineencyclopedia.org /m/mi/minimum_description_length.html (284 words)

	Web-LoiczView Help: Minimum Description Length (Site not responding. Last check: 2007-10-22)
		The Minimum Description Length [MDL] principle is a mathematical method for applying Occam's Razor to models for data--a set of clusters is a model for a given data set.
		The MDL principle says that the the model that takes the least number of bits to represent is the best model for a set of data.
		If you consider the graph of MDL values, it is clear that from 10 to 16 clusters the descriptions lengths all fall within a similar range.
	www.palantir.swarthmore.edu /loicz/help/mdl.htm (588 words)

	DIMACS Workshop on Complexity and Inference
		An application of this idea to parametric probability models involves minimization of the code length for the data with a model whose code length, that depends on the number of parameters and their quantization, does not exceed a preselected bound.
		This corresponds to the two-part code [Rissanen '89] of the MDL principle, when it is interpreted as the first term describing the set and the second term the element in the set [Rissanen '99].
		In the further investigation of the newer MDL 2001 criterion, we verify that the two complexity measures of MDL 1996 criterion and MDL 2001 criterion are close to each other for a multinomial model.
	www.stat.ucla.edu /~cocteau/complexity/abs.html (6073 words)

	Notes on Lam and Bacchus, 1994
		The MDL principle counsels that the best model of a collection of data items is the model that minimizes the sum of (a) the length of the encoding of the model, and (b) the length of the encoding of the data given the model, both of which can measured in bits.
		They show that the encoding length of the data as described above is a monotonically increasing function of the (Kullback-Leibler) cross entropy between the distribution defined by the model and the true distribution, and they use the cross entropy as a metric to search for a network of minimum description length.
		The minimum description length (MDL) principle indicates that the best model of a collection of data items is the model that minimizes the sum of (a) the length of the encoding of the model, and (b) the length of the encoding of the data given the model, both of which are measured in bits.
	www.cs.brown.edu /research/ai/dynamics/tutorial/Documents/LamandBacchus.html (1889 words)

	Re: Minimum Description Length Revisited (Site not responding. Last check: 2007-10-22)
		But it's not going to be true in general that if program A has a smaller MDL than program B in one computer language then A has a smaller MDL in another language.
		For example, the second language may have a single subroutine that performs B, so the program is very short--shorter than the smallest code length for A. Relativizing MDL to a language may be fine for many purposes.
		For the simplicity of a curve changes in different coordinate representations, so the simplicity of curve (assuming that can be well defined) cannot be used to define the simplicity of the hypothesis represented by the curve in a language invariant way.
	philosophy.wisc.edu /920/_disc3/0000003b.htm (186 words)

	The American Statistician: Model Selection Using the Minimum Description Length Principle.(Statistical Data Included)@ ... (Site not responding. Last check: 2007-10-22)
		The minimum description length (MDL) principle articulated in the last decade by Rissanen and his co-workers yields new criteria for statistical model selection.
		MDL criteria permit data-based choices from among alternative statistical descriptions of data without necessarily assuming that the data were sampled randomly.
		This article explains the MDL principle informally, indicates the criteria it yields in the common cases of multinomial distributions and Gaussian regression, and illustrates MDL's use with numerical examples.
	highbeam.com /library/doc0.asp?docid=1G1:68707748&refid=ink_tptd_mag (214 words)

	Minimum Encoding Length Induction (Site not responding. Last check: 2007-10-22)
		Minimum encoding length approaches perform induction by seeking a theory that enables the most compact encoding of both the theory and available data.
		The two approaches differ in that MDL employs a universal prior, which Rissanen [1983] explicitly justifies in terms of Occam's razor, while MML allows the specification of distinct appropriate priors for each induction task.
		Neither MDL nor MML with its default prior would add complexity to a decision tree if doing so were justified solely on the basis of evidence from neighboring regions of the instance space.
	www.cs.washington.edu /research/jair/volume4/webb96a-html/node15.html (271 words)

	Citebase - A tutorial introduction to the minimum description length principle
		A tutorial introduction to the minimum description length principle
		This tutorial is an extended version of the first two chapters of the collection "Advances in Minimum Description Length: Theory and Application" (edited by P.Grunwald, I.J. Myung and M. Pitt, to be published by the MIT Press, Spring 2005).
		Rissanen, J. A universal prior for integers and estimation by minimum description length.
	www.citebase.org /cgi-bin/citations?id=oai:arXiv.org:math/0406077 (1210 words)

	MDL on the Web
		This is the rationale behind the Minimum Description Length (MDL) Principle introduced by Jorma Rissanen (Rissanen, 1978).
		The People section has links to researchers who are working on MDL and related fields.
		The Related Topics section is a short collection of links to MDL related topics, such as information theory, Bayesian statistics, and learning theory.
	www.mdl-research.org (377 words)

	Minimum description length
		The concept of minimizing description length as a practical method of carrying out model comparison in the light of data was pioneered by Wallace and Boulton.
		Focus is on: data mining and knowledge discovery in databases, inductive logic programming, knowledge intensive learning, concept drift and context-sensitive learning, minimum description length principle, machine learning and music.
		SUBDUE Knowledge Discovery in Structural Databases - The program discovers interesting and repetitive subgraphs in a labeled graph representation using the minimum description length principle.
	www.nebulasearch.com /encyclopedia/article/Minimum_description_length.html (242 words)

	MDL site: Reading
		There is a large body of literature on the Minimum Description Length principle in the contexts of statistics, mathematics, machine learning, philosophy, etc. We give only a small selection of publications that we have found especially useful and important.
		P.Grünwald, A Tutorial introduction to the minimum description length principle.
		J.Rissanen, A Universal prior for integers and estimation by minimum description length.
	www.mdl-research.org /reading.html (486 words)

	Adding Curvature to Minimum Description Length Shape Models (Site not responding. Last check: 2007-10-22)
		The Minimum Description Length (MDL) approach to shape modelling seeks a compact description of a set of shapes in terms of the coordinates of marks on the shapes.
		However, this MDL approach does not capture important shape characteristics related to the curvature of the curves, and occasionally it places marks in obvious conflict with the human notion of point correspondence.
		The MDL method is able to solve the point correspondence problem and a classification of the heads into male and female improves dramatically when using the MDL-generated marks.
	www.bmva.ac.uk /bmvc/2003/papers/paper-23-148.html (181 words)

	Minimum Description Length Induction, (ResearchIndex)
		Abstract: The relationship between the Bayesian approach and the minimum description length approach is established.
		We sharpen and clarify the general modeling principles minimum description length (MDL) and minimum message length (MML), abstracted as the ideal MDL principle and defined from Bayes's rule by means of Kolmogorov complexity.
		The basic condition under which the ideal principle should be applied is encapsulated as the fundamental inequality, which in broad terms states that the principle is...
	citeseer.ist.psu.edu /619030.html (676 words)

	Citebase - Minimum Description Length Induction, Bayesianism, and Kolmogorov Complexity
		The relationship between the Bayesian approach and the minimum description length approach is established.
		We sharpen and clarify the general modeling principles MDL and MML, abstracted as the ideal MDL principle and defined from Bayes's rule by means of Kolmogorov complexity.
		The basic condition under which the ideal principle should be applied is encapsulated as the Fundamental Inequality, which in broad terms states that the principle is valid when the data are random, relative to every contemplated hypothesis and also these hypotheses are random relative to the (universal) prior.
	citebase.eprints.org /cgi-bin/citations?id=oai:arXiv.org:cs/9901014 (1651 words)

	Minimum Description Length Revisited (Site not responding. Last check: 2007-10-22)
		MDL seems to be rejected due to the difference in languages.
		For example, in computer science, C++ may describe a model in fewer lines of code or in a smaller file size than the same model written in Cobol.
		Would one solution to the problem of language dependence with MDL be to limit MDL to a single language?
	philosophy.wisc.edu /920/_disc3/00000033.htm (111 words)

	THEORY OF BEAUTY - FACIAL ATTRACTIVENESS
		In 1997 Schmidhuber published the concept of Low- Complexity Art, the computer age equivalent of minimal art: art with low Kolmogorov complexity - art that can be generated by a short program (ref [4]).
		Schmidhuber's postulate (see refs [1-5]): among several patterns classified as "comparable" by some subjective observer, the subjectively most beautiful is the one with the simplest (shortest) description, given the observer's particular method for encoding and memorizing it.
		For example, mathematicians find beauty in a simple proof with a short description in the formal language they are using.
	www.idsia.ch /~juergen/beauty.html (159 words)

	Journal of the American Statistical Association: Model Selection and the Principle of Minimum Description Length.@ ... (Site not responding. Last check: 2007-10-22)
		This article reviews the principle of minimum description length (MDL) for problems of model selection.
		By viewing statistical modeling as a means of generating descriptions of observed data, the MDL framework discriminates between competing models based on the complexity of each description.
		Here we review both the practical and the theoretical aspects of MDL as a tool for model...
	www.highbeam.com /library/doc0.asp?DOCID=1G1:77417866&refid=ip_encyclopedia_hf (229 words)

	Abstract: Autoencoders, Minimum Description Length and Helmholtz Free Energy (Site not responding. Last check: 2007-10-22)
		We derive an objective function for training autoencoders based on the minimum Description Length (MDL) principle.
		The aim is to minimize the information required to describe both the code vector and the reconstruction error.
		We show that the recognition weights of an autoencoder can be used to compute an approximation to the Boltzmann distribution and that this approximation gives an upper bound on the description length.
	www.cs.toronto.edu /~hinton/absps/cvq.html (214 words)

Try your search on: Qwika (all wikis)