
	Where results make sense

Topic: Shannon entropies

Related Topics

Information entropy

Thermodynamic entropy

Probability distribution

Entropy

Information theory

Boltzmann distribution

Statistical mechanics

Shannon limit

Second law of thermodynamics

Thermodynamics

Ludwig Boltzmann

List of important publications in computer science

Bell System Technical Journal

Statistical mechanics

Statistical mechanics

In the News (Thu 16 Feb 12)

EXECUTIVE POWER

Iran

ANALYSIS

Pakistan

Family compact

	Shannon's Work
		In the Shannon paradigm, information from a "source" (defined as a stochastic process) must be transmitted though a "channel" (defined by a transition probability law relating the channel output to the input).
		Thus the revolutionary elements of Shannon's contribution were the invention of the source-encoder-channel-decoder-destination model, and the elegant and remarkably general solution of the fundamental problems which he was able to pose in terms of this model.
		Primary among these is the notion of the "entropy" of a random variable (and by extension of a random sequence), the "mutual information" between two random variables or sequences, and a calculus that relates these quantities and their derivatives.
	cm.bell-labs.com /cm/ms/what/shannonday/work.html (841 words)

	Information entropy - Wikipedia, the free encyclopedia
		Shannon's entropy measure came to be taken as a measure of the uncertainty about the realization of a random variable.
		Thus, the entropy of the source alphabet, with its given empiric probability distribution, is a number equal to the number (possibly fractional) of symbols of the "ideal alphabet", with an optimal probability distribution, necessary to encode for each symbol of the source alphabet.
		Thus, the Shannon entropy is a consequence of the equation
	en.wikipedia.org /wiki/Information_entropy (2379 words)

	Shannon Entropy (Site not responding. Last check: 2007-10-15)
		The Shannon entropy equation provides a way to estimate the average minimum number of bits needed to encode a string of symbols, based on the frequency of the symbols.
		Shannon entropy provides a lower bound for the compression that can be achieved by the data representation (coding) compression step.
		Shannon entropy makes no statement about the compression efficiency that can be achieved by predictive compression.
	www.bearcave.com /misl/misl_tech/wavelets/compression/shannon.html (421 words)

	Brainstorms: A 4th Law of Thermodynamics
		Shannon entropy, like variance and expectation value, is a quantity that is defined for stochastic variables.
		Shannon entropy is a property of a stochastic variable.
		The Shannon entropy of the chip cares only about the overall state of each tiny transistor etched in the silicon crystal--the transistor is on or off; it is a 0 or a 1--a single binary degree of freedom.
	www.iscid.org /boards/ubb-get_topic-f-6-t-000423-p-2.html (4329 words)

	The Talk.Origins Archive Post of the Month: October 2000
		Put another way: if thermodymanic entropy were the relevant constraint, it should be easier for a completely novel organism to appear, than for an existing organism to be replicated; it's not, therefore thermodynamic entropy is not the relevant constraint.
		It's still entropy, mind you -- the entropy of the demon's memory banks increases with the amount of information stored there, so all we've really accomplished is to move entropy from the gas to the demon's memory, and change its form somewhat.
		The only time there's any limit on the increase or decrease of Shannon- entropy is when it's encoded in the state of a system that's subject to thermodynamics, and then the limits are due to thermodynamics, not any information-theoretic consideration.
	www.talkorigins.org /origins/postmonth/oct00.html (4178 words)

	Calculation of Information and Complexity
		Shannon's notion of information does not measure the semantical content or the meaning of a message.
		The Shannon entropy is the classical measure of information.
		The metric entropy vanishes for constant sequences, increases monotonically when the sequence's disorder raises and reaches its maximum at 1 for equally distributed random sequences.
	www.dr-frank-wolf.de /diss/calcen.htm (1802 words)

	Boltzmann and Shannon Entropies are Additive (Site not responding. Last check: 2007-10-15)
		The Boltzmann and Shannon entropy functions H are logarithms of two different polynomial coefficients determining cyclic permutations of two indices.
		Yet, it is plausible that entropy is a measure of symmetry, meaning the measure of the number of symmetry elements and their orders in the systems.
		Physical entropy is a logarithmic measure of the amount of energy needed to increase temperature.
	mujweb.cz /veda/kunzmilan/boltz3.htm (3183 words)

	Instiut für Physik Nichtlineare Dynamik/Stochastische Prozesse
		The higher order Shannon entropies and the conditional entropies (dynamical entropies) are calculated, characteristic fluctuations are found.
		Instead of the dynamic entropies which yield mean values of the uncertainty/predictability we finally investigate the local values of the uncertainty/predictability and the distribution of these quantities.
		As a conclusion, we predict that the entropy of DNA sequences measuring the information content is much lower than suggested by earlier empirical studies.
	www.ebelinge.de /Liste~1.htm (2611 words)

	[No title]
		We may view the joint entropy H(X,Y) of X and Y as the sum of the entropy H(Y) of Y and the additional entropy H(XY) of X given Y. This defines H(XY), called the conditional entropy of X given Y, as a function of joint entropy and original entropies, as H(XY)=H(X,Y)-H(Y).
		Quantum or von Newmann entropies are generalizations of Shannon entropies where the symbols conveyed by the random source of information are (not necessarily mutually orthogonal or even pure) quantum states.
		Density matrices and von Newmann entropies for mixed, pure and entangled states, and of states left after tracing out a subsystem; identities and inequalities connecting various entropies; further discussions on techniques using information theory to establish communication complexity lower bounds, for qubit as well as cbit complexity.
	www.facweb.iitkgp.ernet.in /~spp/qcqip2006.html (1557 words)

	Shannon.entropy (Site not responding. Last check: 2007-10-15)
		This function computes the Shannon entropy of the squares of a set of coefficients.
		The squares are used because we are only interested in the entropy of the energy of the coefficients, not their actual sign.
		The Shannon entropy measures how "evenly spread" a set of numbers is.
	www.maths.bris.ac.uk /~wavethresh/help/shanent.htm (189 words)

	[No title]
		The analysis is based on the calculation of entropy like quantities as the mutual information for pairs of letters and the entropy, the mean uncertainty, per letter.
		Further, the authors decided to completely neglect the liner entropy term because it's too small; this is probably not the case because one can see the deviations from the square root behavior for subextensive entropies on the plots with a naked eye.
		Finally, the finite-sampling corrections to entropies are done according to \cite{herzel-schmitt-ebeling-94}, where the number of ``relevant'' bins is the same as of those, which are actually occupied.
	www.menem.com /~ilya/digital_library/language.bib (1191 words)

	Information Theory
		The real justification for regarding the entropy as the amount of information is that, unsightly though it is, though it's abstracted away all the content of the message and almost all of the context (except for the distribution over messages), it works.
		For certain classes of random processes, the Kolmogorov complexity per symbol converges, on average, to the entropy per symbol, which in that case is the entropy rate, the entropy of the latest symbol, conditioned on all the previous ones.
		Ultimately, the efficiency measure can be directly interpreted as the relative entropy between two probability distributions, namely: the distribution of the particles in the presence of the external rectifying force field and a reference distribution describing the behavior in the absence of the rectifier".
	cscs.umich.edu /~crshalizi/notebooks/information-theory.html (4338 words)

	One from my many attempts to publish the controversial subject revived by the scanner (Site not responding. Last check: 2007-10-15)
		entropies are interpreted as symmetries of spherical orbits in vector spaces.
		The weaker form is that (1) is a statistical form of thermodynamical entropy S. This was proofed by practice but Boltzmann‘s explanation of the relation of H and S was not understood, because Boltzmann himself did not take it seriously.
		Boltzmann connected entropy S with the logarithmic measure of the volume of partition orbits which for n going to infinity is approximated by (1).
	mujweb.cz /veda/kunzmilan/ent1.htm (3020 words)

	Lutz Molgedey
		Instead of the dynamic entropies which yield mean values of the uncertainty/predictability we finally investigate the local values of the uncertainty/predictability and the distribution of these quantities.We consider time series of financial data as the Dow Jones Index with respect to the existence of local order.
		The structure of time series and letter sequences is investigated using the concepts of entropy and complexity.
		Besides higher order Shannon entropies and conditional entropies (dynamic entropies) which yield mean values of the uncertainty/predictability, we study the local values of the uncertainty/predictability and the distribution of these quantities.
	molgedey.de /publications (1688 words)

	entropies
		Entropy is equal to zero iff the message is known
		Entropy of a mean is greater than mean of elementary entropies
		Entropy of the whole is greater than entropy of the parts
	www.syscope.net /info/entropies.html (982 words)

	Inequalities for Shannon entropies and Kolmogorov Complexities
		Since the very beginning the notion of complexity of finite objects was considered as an algorithmic counterpart to the notion of Shannon entropy.
		It was mentioned by Kolmogorov later (1968) that the properties of algorithmic complexity and Shannon entropy are similar.
		The paper investigates connections between linear inequalities that are valid for Shannon entropies and for Kolmogorov complexities.
	doi.ieeecomputersociety.org /10.1109/CCC.1997.612296 (209 words)

	I101: Introduction to Informatics - Lab 10: Simple Programming in Excel and Measuring Uncertainty
		Calculate the Shannon entropy of the letter grade probability distribution.
		Calculations of the Shannon entropy of letter grades.
		Finally, you need to (v) compare the Shannon entropies of three professors: Matic, Churchill and Mussolini.
	www.informatics.indiana.edu /rocha/i101/lab10.html (1156 words)

	sci.crypt: Re: Randomness from natural texts
		There is an important difference between Shannon entropy and
		The table I posted was based on the common Shannon entropy measure.
		> temperature, while, the Shannon entropy is interpreted in terms of...
	www.derkeiler.com /Newsgroups/sci.crypt/2003-03/1206.html (499 words)

	Re: The maximum speed of information
		is a partition of a finite set X. This satisfies the same formal properties as Shannon's entropy -sum p(A_j) log(p(A_j)), and they are both defined in the context of partitions of finite sets, X = uplus A_J, but they are certainly not the same.
		For one thing, Shannon's entropy requires more structure for its definition, namely, a probability measure on the partition.
		von Neumann entropy, Boltzmann entropy) which are in common use in mathematical physics, and all of them have their uses.
	www.lns.cornell.edu /spr/2002-02/msg0039598.html (568 words)

	Entropy-information measures
		IOW, the edges in the lattice of concepts\ncan be assigned positive integers describing the "degrees of freedom"\ninvolved in various geometric configurations.
		I am pretty sure, however, that noone else has\neven -attempted- anything like a comprehensive survey, because about a\ndecade ago I did enough reading (some of which was summarized in that\npaper) to see that even a modestly annotated bibliography would be book\nlength, and anything like a genuine survey would need a multivolume\nmonograph.
		The nice\nformal behavior of the entropies is an immediate consequence of the nice\nformal behavior of the underlying complexions.
	www.physicsforums.com /showthread.php?t=39560 (3816 words)

	Running such a system on Avida, measure its "Shannon" entropy. The entropy related to information theory which can be ... (Site not responding. Last check: 2007-10-15)
		Running such a system on Avida, measure its "Shannon" entropy.
		The entropy related to information theory which can be best defined as the measure of the information content of the system.
		More simply, a measure of the number of different types of strings.
	ivytech7.cc.in.us /mathsci/alife/complexity/complexity/tsld078.htm (86 words)

	The Shannon information entropy of protein sequences -- Strait and Dewey 71 (1): 148 -- Biophysical Journal (Site not responding. Last check: 2007-10-15)
		The Shannon information entropy of protein sequences -- Strait and Dewey 71 (1): 148 -- Biophysical Journal
		uses the "word" frequency to determine the Shannon entropy.
		This entropy is much smaller than the value of 4.18
	www.biophysj.org /cgi/content/abstract/71/1/148 (507 words)

	Energy Citations Database (ECD) - Energy and Energy-Related Bibliographic Citations (Site not responding. Last check: 2007-10-15)
		Availability information may be found in the Availability, Publisher, Research Organization, Resource Relation and/or Author (affiliation information) fields and/or via the "Full-text Availability" link.
		We obtain explicit formulas for the Shannon entropy of some of the elements of these bases.
		We also consider two reverse log-Sobolev inequalities in the {mu}-deformed Segal-Bargmann space, which have been proved in a previous work, and show that a certain known coefficient in them is the best possible.
	www.osti.gov /energycitations/product.biblio.jsp?osti_id=20768732 (237 words)

	Communication Theory
		Focus on Communications Theory (Allyn & Bacon Puplishers)
		Doty Models of Written Communication (includes summaries of Shannon, Jakobson, Neisser, and Polanyi)
		Michael Dwyer hosts a radio series on "media and communication studies" (1999)
	carbon.cudenver.edu /~mryder/itc/comm_theory.html (109 words)

Try your search on: Qwika (all wikis)