
	Where results make sense

Topic: Algorithmic probability

Related Topics

Theory of computation

List of computability and complexity topics

Turing reduction

Polynomial time

Computational complexity theory

Reduction (complexity)

Oracle machine

In the News (Sat 14 Nov 09)

Northern Trust

Marvin Harrison

Peter Chernin

Gary Locke

Match Play

Penelope Cruz

Mickey Rourke

AR Rahman

Danny Boyle

Hugh Jackman

	Randomness - Psychology Wiki - a Wikia wiki
		The classical version of probability theory that they developed proceeds from the assumption that outcomes of random processes are equally likely; thus they were among the first to give a definition of randomness in statistical terms.
		In the early 1960s Gregory Chaitin, Andrey Kolmogorov and Ray Solomonoff introduced the notion of algorithmic randomness, in which the randomness of a sequence depends on whether it is possible to compress it.
		The mathematical theory of probability arose from attempts to formulate mathematical descriptions of chance events, originally in the context of gambling but soon in connection with situations of interest in physics.
	psychology.wikia.com /wiki/Chance (1849 words)

	Algorithmic probability - Wikipedia, the free encyclopedia
		The algorithmic probability of any given finite output prefix q is the sum of the probabilities of the programs that compute something starting with q.
		Algorithmic probability is the main ingredient of Ray Solomonoff's theory of inductive inference, the theory of prediction based on observations.
		Algorithmic probability is closely related to the concept of Kolmogorov complexity.
	en.wikipedia.org /wiki/Algorithmic_probability (235 words)

	Probability
		Event (probability theory) In probability space it is possible to exclude certain subsets of the sample space from being...
		Probability of kill Computer games, Statistical decisions are required when all of the variables that must be considered...
		Statistical probability " Statistical probability " is a term sometimes used informally as a synonym for probability wit...
	www.brainyencyclopedia.com /topics/probability.html (446 words)

	Algorithmic Composition (Site not responding. Last check: 2007-10-08)
		Early algorithmic compositions were driven exclusively by collaboration between scientists and musicians; therefore, the use of scientific terminology normally associated with computer science provided a uniform understanding of the algorithm.
		Given both the problem and the device, an algorithm is the precise characterization of a method of solving the problem, presented in a language comprehensible to the device.
		Algorithmic solutions, therefore, are used to produce goal solutions by means of a series of tests; heuristics solve a problem by intuition and anticipation of the forthcoming data.
	eamusic.dartmouth.edu /~wowem/hardware/algorithmdefinition.html (3742 words)

	A SYSTEM FOR INCREMENTAL LEARNING BASED ON ALGORITHMIC PROBABILITY (Site not responding. Last check: 2007-10-08)
		The earliest description of algorithmic probability was in a formal theory of inductive inference (Sol 60, 64a, 64b).
		Algorithmic probability defines the apriori probability of a string of symbols, x, as the probability that x will be produced as the output of a reference universal Turing machine having random input.
		Algorithmic probability was able to refine these ideas by giving an optimum "goodness of fit criterion" for the fitting of probabilistic grammars to data (Sol 62, 64b, 75).
	world.std.com /~rjs/IncLrn89.html (7066 words)

	Algorithmic information theory - Encyclopedia.WorldSearch (Site not responding. Last check: 2007-10-08)
		Algorithmic information theory is a field of study which attempts to capture the concept of complexity by using tools from theoretical computer science.
		The chief idea is to define the complexity (or Kolmogorov complexity) of a string as the length of the shortest program which outputs that string.
		The first surprising result is that K(s) cannot be computed: there is no general algorithm which takes a string s as input and produces the number K(s) as output.
	encyclopedia.worldsearch.com /algorithmic_information_theory.htm (1048 words)

	Nick Szabo -- Introduction to Algorithmic Information Theory
		The algorithmic probability of x is given by
		Once we have defined induction in terms of minimizing description length and distortion, the part/whole question is perhaps the sole remaining stumbling block to a consistent theory of induction free from the troubling and contradictory axioms of inductive probability.
		Algorithmic information theory is a far-reaching synthesis of computer science and information theory.
	szabo.best.vwh.net /kolmogorov.html (1710 words)

	Review of Discrete Algorithmic Mathematics
		Of course, we do not mean "algorithmic" in the sense of rote application of rules, but rather in the sense of algorithmics, which means thinking in terms of algorithms and their mathematical study.
		The algorithm must terminate with a solution to the problem it purports to solve, or it must indicate that, for the given data, the problem is insoluble by this algorithm.
		As mentioned in the descriptive introduction of the term "algorithm", algorithms are "organized procedures" and should be invoked as such with IS compliant data.
	www.cut-the-knot.org /books/Reviews/DiscreteAlgorithmicMath.shtml (3076 words)

	Algorithmic information theory - Scholarpedia
		Algorithmic Information Theory (AIT) is the information theory of individual objects, using computer science, and concerns itself with the relationship between computation, information, and randomness.
		Algorithmic complexity formalizes the notions of simplicity and complexity.
		For infinite sequences one can show that these are exactly the sequences which are incompressible in the sense that the algorithmic prefix complexity of every initial segment is at least equal to their length.
	www.scholarpedia.org /article/Algorithmic_information_theory (3094 words)

	Randomness (Site not responding. Last check: 2007-10-08)
		The mathematical theory of probability arose from attempts to formulate mathematical descriptions of chance events, originally in the context of gambling but soon in connection with situations of interest in physics.
		Statistics is used to infer the underlying probability distribution of a collection of empirical observations.
		Algorithmic information theory studies, among other topics, what constitutes a random sequence.
	www.free-download-soft.com /info/random.html (1364 words)

	The Small-World Phenomenon: An Algorithmic Perspective 1
		In this work, we study ``decentralized'' algorithms by which individuals, knowing only the locations of their direct acquaintances, attempt to transmit a message from a source to a target along a short path.
		Of course, constraining the algorithm to use only local information is crucial to our model; if one had full global knowledge of the local and long-range contacts of all nodes in the network, the shortest chain between two nodes could be computed simply by breadth-first search.
		This is the same as our initial model of a decentralized algorithm, except that we do not count steps in which the algorithm ``backtracks'' by sending the message through a node that has already received it.
	www.cs.cornell.edu /home/kleinber/swn.d/swn.html (5001 words)

	DIMACS Workshop on Complexity and Inference
		Yet, the algorithm is universal in the sense of asymptotically performing as well as the optimum denoiser that knows the input sequence distribution, which is only assumed to be stationary and ergodic.
		Moreover, the algorithm is universal also in a semi-stochastic setting, in which the input is an individual sequence, and the randomness is due solely to the channel noise.
		The algorithmic theory of randomness is well developed when the underlying space is the set of finite or infinite sequences and the underlying probability distribution is the uniform distribution or a computable distribution.
	www.stat.ucla.edu /~cocteau/complexity/abs.html (6073 words)

	Lecture 1
		The probability that M assigns to any particular set of observations will be extremely small, and the larger the set of observations is, the smaller is its probability.
		This probability need not be large, but the ultimate claim is that if we state the goal of the theory in this way, the model M which does assign the highest probability to the observed data will be the best linguistic model of the observed data.
		When we build our probabilistic model, we assign probabilities to the small subpieces of the model (for example, probabilities of individual phonemes or features), and these probabilities are usually tightly linked to direct observation (roughly, but only roughly, we take these probabilities to be equal to their observed frequencies).
	humanities.uchicago.edu /faculty/goldsmith/Chiba/Lecture1.html (7376 words)

	Information Theory and Creationism: Algorithmic Information Theory (Chaitin, Solomonoff & Kolmogorov)
		Hilbert had hoped it would be possible to develop a general algorithm for deciding whether a set of axioms were self-contradictory.
		An algorithm producing the digits of π is an example of a non-halting algorithm π is an irrational and transcendental number; it cannot be expressed as a polynomial with rational coefficients and therefore has an infinite number of digits).
		Algorithmically random has a different definition and is a different concept, even though very long algorithmically random strings have symbol distributions with statistical properties.
	www.talkorigins.org /faqs/information/algorithmic.html (3567 words)

	Information
		The most refined approach to defining randomness is found within the algorithmic complexity approach to information, and goes back to Kolmogorov (1968), who also gave a standard axiomatisation of probability theory.
		When the choices are not equally probable, the information is the sum logarithm of the probability of each choice weighted by the probability choice, yielding and equation similar in form to that for entropy in Boltzmann's statistical thermodynamics.
		The problem of grounding the ensemble probabilities is often dealt with by using operational procedures that vary according to the details of the case.
	www.kli.ac.at /theorylab/jdc/information/information.html (8999 words)

	Ray Solomonoff -- ISIS Biosketch (Site not responding. Last check: 2007-10-08)
		Ray Solomonoff invented the concept of Algorithmic Probability, which formalizes the idea of Bayesian reasoning using information theory, using ideas which have subsequently become known within Kolmogorov complexity theory.
		In particular, the prior probability of some data is defined as sum(i = 1, infinity, 2^(-Li)) where Li is the length of the i th description of the data to some universal Turing machine.
		Algorithmic Probability employs these concepts to identify a posterior probability distribution in ways formally related to Bayesian methods.
	www.csse.monash.edu.au /~dld/rjs.html (180 words)

	Leonid Levin - Wikipedia, the free encyclopedia
		He is well known for his work in randomness in computing, algorithmic complexity and intractability, foundations of mathematics and computer science, algorithmic probability, theory of computation, and information theory.
		His son, Andrei Levin, was an Intel Science Talent Search Finalist in 2004 and is currently an undergraduate student at MIT.
		A Survey of Russian Approaches to Perebor (Brute-Force Searches) Algorithms, by B.A. Trakhtenbrot, in the Annals of the History of Computing, 6(4):384-400, 1984.
	en.wikipedia.org /wiki/Leonid_Levin (209 words)

	INFORMS Journal on Computing - Areas and Area Editors (Site not responding. Last check: 2007-10-08)
		Algorithmic development of approaches for solving complex problems in applied probability or stochastic processes where viable computer solutions are yet to be discovered.
		The design of algorithms covers a broad spectrum, ranging from the study of the complexity or approximability of a problem to an algorithm engineering project involving high-performance computing platforms, advanced data structures and real-life data.
		The analysis of algorithms may concern the investigation of characteristics such as the solution quality it delivers or the running time it needs, be it by theoretical (worst-case or probabilistic) or experimental means.
	joc.pubs.informs.org /AreasAndAreaEditors.html (1639 words)

	CiteULike: From regular lattice to scale free network - yet another algorithm (Site not responding. Last check: 2007-10-08)
		The Watts-Strogatz algorithm transferring a regular lattice to the small world network is modified by introducing preferential rewiring constrained by connectivity demand.
		The probability to link to/ unlink form a node is dependent on a vertex degree and adjusted by some threshold.
		For each threshold value there exists a probability at which the resulting stationary network has degree distribution with power-law decay in large interval of degrees.
	www.citeulike.org /user/scis0000001/article/480 (178 words)

	CLRC: Computer Learning Research Centre
		Ray Solomonoff has carried out fundamental research in the fields of inductive inference and machine learning, the best known being his discovery in 1960 of Algorithmic Probability - a very general system for defining regularities in data and quantifying the process of making theories.
		Algorithmic Probability has subsequently been approximated and modified by others in many forms - such as Kolmogorov Complexity, Minimum Description Length, Minimum Message Length, etc.
		His subsequent development of Algorithmic Probability has made it possible to put this earlier work on machine learning in a more exact, more general form.
	www.clrc.rhul.ac.uk /people/solomonoff (190 words)

	Computing Reviews, the leading online review service for computing literature. (Site not responding. Last check: 2007-10-08)
		An algorithm is “universal”; if it is free of any parameters, and does not depend on the specific environment in which it (or the respective agent) operates.
		The algorithm is “optimal” with respect to a suitable intelligence order relation introduced by the author, which, as he argues and believes, is reasonable and true.
		Chapter 5 generalizes the agent model to the approximated prior probability distribution introduced in chapter 3, and argues that this agent represents a universal optimum with regard to several reasonable concepts of optimality, namely, consistency, self-tunability, self-optimization, efficiency, unbiasedness, and various others.
	www.reviews.com /review/review_review.cfm?review_id=131175 (659 words)

	Brainstorms: algorithmic info, probability, etc.
		In such a process, a key step is that the sender and receiver have to agree on a probability distribution with which to encode the data.
		Given an agreed probability distribution, for example the model residuals (and all the hidden assumptions that implies), one can prove useful results; as in the paper, it is shown that under a Gaussian noise model, that the minimization of description length corresponds to choosing a least-squares cost function in the non-linear regression.
		However, the probability of m bits of Pb existing is equal to 1/2^m.
	www.iscid.org /boards/ubb-get_topic-f-6-t-000253-p-2.html (5420 words)

	Purely Game-theoretic Random Sequences: I. Strong Law of Large Numbers and Law of the Iterated Logarithm
		Random sequences are usually defined with respect to a probability distribution~$\bf P$ (a $\sigma$-additive set function, normed to one, defined over a $\sigma$-algebra) assuming Kolmogorov's axioms for probability theory.
		In this purely game-theoretic framework, no probability distribution or, partially or fully specified, system of conditional probability distributions needs to be introduced.
		For these typical sequences, we prove direct algorithmic versions of Kolmogorov's strong law of large numbers (SLLN) and of the upper half of Kolmogorov's law of the iterated logarithm~(LIL).
	epubs.siam.org /sam-bin/dbq/article/97776 (188 words)

	Research Projects of Marcus Hutter
		Solomonoff's induction scheme based on M results in optimal sequential predictions (in expectation and with probability one), the only assumption to be made is that the sequence is sampled from a computable probability distribution [P99errbnd, P03optisp, P02unipriors].
		The duality of both approaches and results has been pointed out in [P04bayespea] and should be further explored in the future, probably leading to novel and fruitful bounds and algorithms.
		Remarkably, a fully linear time approximation algorithm for knapsack problems could be developed [P02knapsack], improving upon previous PTAS algorithms.
	www.hutter1.de /official/projects.htm (2000 words)

	Brainstorms: algorithmic info, probability, etc.
		OTOH, If the probability of m occuring is at all feasible, then we are of necessity inferring an encoding of natural selection of considerable size, and comparable to the algorithmic info in the biological world itself.
		IOW, the probability of getting a mutation sequence that will cause e to generate the entire weasel string is certainly greater than the entire weasel string arising by chance.
		When the probability of e is factored in, the probability of the weasel string occurring chance is the same.
	www.iscid.org /boards/ubb-get_topic-f-6-t-000253-p-1.html (9762 words)

	Universality and Complexity in Cellular Automata (1984)
		The asymptotic ``halting probability'' is around 0.93; 7% of initial configurations generate the persistent structures of fig.
		This algorithmic probability has been shown to be invariant (up to constant multiplicative factors) for a wide class of universal computers.
		As a simple example, the probability for the sequence which yields a period 9 propagating structure in the cellular automaton of figs.
	www.stephenwolfram.com /publications/articles/ca/84-universality/9/text.html (1931 words)

	Amazon.com: Universal Artificial Intelligence: Sequential Decisions Based On Algorithmic Probability: Books: Marcus ... (Site not responding. Last check: 2007-10-08)
		The measurement of complexity that is used in algorithmic information theory is that of Kolmogorov complexity, which one can use to measure the a prior plausibility of a particular string of symbols.
		The author though wants to use the `Solomonoff universal prior', which is defined as the probability that the output of a universal Turing machine starts with the string when presented with fair coin tosses on the input tape.
		As the author points out, this quantity is however not a probability measure, but only a `semimeasure', since it is not normalized to 1, but he shows how to bound it by expressions involving the Kolmogorov complexity.
	www.amazon.com /Universal-Artificial-Intelligence-Algorithmic-Probability/dp/3540221395 (2516 words)

Try your search on: Qwika (all wikis)