Information as Probability

Recreating (and advancing) pk’s censored domains: & / Teaching / Thinking Tools / Information / Macroinformation / Views /
@ K. 1999

Gregory Bateson defined information in terms of difference: Claude Shannon in terms of probability. I define macroinformation in terms of meta-difference. But information relates probability to difference.

My thinking about probability has been nothing as intensive as my thinking about difference. I don’t imagine this essay will ever be the peer of my Macroinformation Preface. For now I gather only related materials from earlier introductory modules. Once I think I’ve gotten the bulk of what I’ve already written on the subject from that angle will I reweave a new essay.

From “Ambiguity”: “1, 2, 3 …” is information as well as data by the definition of difference. By Shannon’s information as the inverse of the probability of the signal it is not. The statement “Paul Knatz has funny ears” is comprised of data whether the statement is spoken, written … or hyper-published. It is information by Bateson’s definition. How well it fits Shannon’s definition will depend largely on the reader or hearer’s prior opinion of my ears. But even if you thought “has funny ears” should be how any sentence begun with “Paul Knatz” should end, I hope you’d admit that there are other possible predicates. I submit that the statement has at least quasi-information status by Shannon’s definition.

Now try this string: a fragment of a longer string.

But doth suffer a sea change
Into something rich and strange

The ending of Shakespeare’s lyric from The Tempest is comprised of data whether it’s written, uttered … declaimed, or sung. It’s information in the sense that data is information, and in more senses as well. It’s information in Shannon’s sense in spades!
Assign the words “But doth” to any population prior to the Seventeenth Century (I’m sorry: this has to be a thought experiment), and see how many finish the line the way we have it in the Shakespeare cannon. If we could actually perform the experiment my bet is that the actual signal would be predicted in zero percent of the cases (even if Kit Marlowe was among those tested). The probability would be hard to distinguish from that of a random number: or from some arbitrarily selected string midway in an irrational decimal.

The information in “1, 2, 3 …” is either specific or general: there’s too little information to tell: we need a context. The information in “Paul Knatz has funny ears” is specific. There’s no important lesson in it: not in biology, not in ethics … If true, one could use it, if not to map, then perhaps to caricature, (part of) Paul Knatz. If false, one can make a caricature anyway. Or perhaps it was true once and time changes truth. A teapot will hold the whole tempest. The information in “a sea change” however; is specific, is general, has trillions of lessons, jillions of references … Even with no knowledge but earth history (and that fragmentary), the associations may span two-thirds of a planet over at least a billion years of time.

We have arrived in the territory of macroinformation.

Data is simple. You can put it in a row. You can add it up and get a quantity that represents its dimensions. A one-dimensional view doesn’t insult it. Macroinformation is complex. It exhibits structure. It refers to other structures: potentially infinitely.

Estimate the number of lead atoms in a lump of lead. A measure of the lead’s mass does not insult the lead (not to contemporary consciousness): it’s a collection of atoms, similar to identical. Count the head of cattle in some rancher’s herd. To many, they’re just cows: like so much lead — except that you can eat it. Now count heads from London to Stratford while Shakespeare was writing The Tempest (or Jonson The Alchemist, or Donne his Meditations …) A sum derived by mere addition seriously under-represents what you’ve just counted. Much more sophisticated mathematics and a host of other disciples are called on to map the significance of the census’s subject. (And no creature — necessarily living in time — can say which undiscovered disciplines are sorely missing.)

Matter assumes a variety of structural expressions beyond the atomic structure: also beyond the molecular structure. There’s vastly more information is a DNA molecule than there is in an assemblage of the “same” atoms not forming a DNA molecule. I say that the structure of data has similar if not vaster information potential.

From “Complexity and Structure”: There is a difference between a stone lying on the ground and a stone gathered with other stones into a barrow. There is a greater difference between a stone among stones in a barrow and a stone functioning as the keystone in an arch. The other stones in the arch are also different in function both from the stones in the barrow and the stones in and on the ground. (One of the differences is that we know what we’re doing with it in the arch. We also know what we’re doing with it in the barrow. We are not likely to know what the structural function of a stone lying on the ground is, if any. On or in the ground it’s part of the random.

Similarly, the “A” in alphabet soup is different, informationally different, from the A starting an alphabetic row on the kindergarten wall. The letter A is different in my word processor’s font than it is once I’ve used it, and used it again, in this sentence. In the soup it’s food. It’s also data, if you see it as such. The other As are all data. They’re still data in this and the above sentences; they’re all information by Bateson’s data; they begin to be information by Shannon’s definition in these sentences … How much more are they information by Shannon’s definition in the following:

Anyone lived in a pretty how town
With up so floating many bells down.

ee cummings

How much still more do the As participate in Shannon’s sense of information in this usage?

My salad days

When I was green in judgement, cold in blood.


A, B, and C function as phonemes or parts of phonemes. Phonemes function as parts of utterances, streams of speech, which have meaning according to a grammar with a lexicon. But speech and writing can achieve vastly sophisticated structures. Such structures are allowed by the grammar, but the grammar cannot “foresee” them all. The stone on the ground implies “Stonehenge” only in an extraordinary mind. Carbon compressed in the earth goes through many steps before it becomes a diamond on Elizabeth Taylor’s bosom at the Oscars. They are all “stones,” but I hope you agree that that’s misleading: highly under-appreciative.

Thinking Tools Information, Macroinformation Menu Mi Views

About pk

Seems to me that some modicum of honesty is requisite to intelligence. If we look in the mirror and see not kleptocrats but Christians, we’re still in the same old trouble.
This entry was posted in information, pk Teaching, thinking tools. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s