2020-02-08 -- Scriptural interpretation by Doctor Jeremy England
The more I read by Doctor Jeremy England (my hero who developed a mathematical description for a thermodynamic definition of life and "dissipation driven adaptation") the more I like. His commentary, here, on the role of science in the Hebrew Bible coincides exactly with my own interpretation of the Christian Bible.
For example, when I hear or read, "In the beginning was the word", I think of the difference between a living organism and the world around it and the role of "words" as the foundation of this relationship.
The word is something, in and of itself, which is a label applied to something else, which is not the word. It is both a thing and a label for another thing. Living creatures (human, or otherwise) have (create?) labels for objects in the world around them. The extent to which a label coincides with "reality" determines the longevity of the label. The "something else" to which the label is applied, the Universe, the thing referred to by the word, may have preceded the word, but its existence cannot be predicted by the living organism until the living organism has a label, a word, which coincides with the thing.
Things without labels exist and impact living organisms, but the living organism will not be able to predict and act on the thing until it has a label, a word (or, equivalently, a genetic code which includes a function which interacts with the thing), for the thing.
So, from the perspective of living organisms, living organisms did not exist until labels, or words, came into being.
2020-02-06 -- EVALUATING MACHINE COMMONSENSE
I recently read "WINOGRANDE: An Adversarial Winograd Schema Challenge at Scale", by Doctor Keisuke Sakaguchi, Doctor Ronan Le Bras, and Doctor Chandra Bhagavatula of the Allen Institute and Doctor Yejin Choi of the University of Washington. They identify spurious biases in test datasets used in the Winograd Schema Challenge (WSC) that lead to overestimation of the true capabilities of machine commonsense.
What a relief! Recent results were showing greater than 90% accuracy of machine commonsense. In actuality, according to this paper, the accuracy is between 50% and 79.1%, with human accuracy at 94%.
Whew! At least for the time being.
How do you evaluate "machine commonsense"?
WSC test datasets use sentence pairs in which one word to flips the meaning of part of the sentence.
- The trophy doesn’t fit into the brown suitcase because it’s too large; vs.
- The trophy doesn’t fit into the brown suitcase because it’s too small.
What noun does the pronoun "it's" apply to? To answer this question requires commonsense understanding of the meaning of the sentences.
WSC were being used as an alternative to the Turing Test. Producing a correct result a high percentage of the time would indicate statistical models of language that are capable of commonsense meaning comprehension that is difficult to distinguish from human level comprehension of meaning.
However, prior groups identified algorithmic "annotation artifacts", or unintentional patterns in the test data which "leak" information about the right answer. Quoting this paper, "State-of-the-art neural models are highly effective at exploiting such artifacts to solve problems correctly, but for incorrect reasons."
To address this problem, this group identified a procedure for creating such test sentences in a more neutral way. To no great surprise, accuracy of the neural language models fell from greater than 90% to between 50% and 79.1%, with human accuracy at 94%.
The neural models will not achieve human accuracy until the models, like us, are organized around physical reproduction of information.
This is most of what I emailed to the foregoing authors (and no, I have not heard back from them):
Information processing by all living organisms can be summarized as follows:
- An area of order, which may be represented by symbol "order-1", exists within any given living organism.
- Order-1 may be converted into work, represented by symbol "work-1", over an amount of time, "time-1", producing waste and heat.
- Work-1 interacts with material and free energy in the environment over a second amount of time, "time-2", to reproduce a second area of order, "order-2", also producing waste and heat.
- Order-1 and order-2 may be the same, though they do not have to be.
- Conversion of order-1 into work-1 is destructive of order-1. Order-1 must be reproduced as order-2. Sometimes, order-1 and order-2 are the same.
- So long as the process iterates, the living organism continues to live.
This description conforms to Shannon's Information Theory. Shannon describes an information source (order-1) which is converted into a signal (work-1) over a period of time (time-1) and which is transmitted through a channel (the environment). The signal (work-1) may be converted back into an information source (order-2) at a receiver over a second period of time (time-2).
According to Doctor Jeremy England, this process requires a continuous flow of energy through a symbolic logic media (e.g. amino acids, genetic code, computer media -- these are referred to above as "material"). This process does not violate the 2nd Law. With a continuous flow of energy through the symbolic logic media, dissipation driven adaptation causes an increase in order within living organisms, an increase in disorder external to living organisms, and acceleration of the flow of energy through the symbolic logic media.
This organizational structure (order) and process (conversion of order into work and conversion of the work back into order) is found in all genetic code and in the human mind. Please see, for example, memory re-consolidation. Please also consider, for example, integrated information theory, which corresponds closely to the foregoing definition of information reproduction by living organisms.
Humans use this organizational structure and process to understand meaning. The organization structure cannot be separated from the process. Both structure and process are required for the reproduction of information by living organisms, humans included, regardless of the symbolic logic media.
The neural language models cited in your work are not organized around reproducing information, so it is not a surprise to me to hear that previous work may have overestimated the capabilities of machine common sense.
I wrote a hard science fiction book on the topic of information processing by living organisms, Apokalypsis. I live and work in the Seattle area and would be happy to come to the Allen Institute to make a lunch presentation (or otherwise) to discuss the foregoing.
Please let me know if the Allen Institute would be interested in such a presentation.
Thank you for your time and consideration.
I read an interesting piece on Slate today, "What if Competition Isn't As 'Natural' As We Think?", by John Favini. In essence, Favini highlights new research which emphasizes that commensal and symbiotic interaction among species and between individuals within a species is very common. It does not "throw out" competition for limited resources, but highlights that cooperation can be beneficial, often mutually beneficial. I have a strong focus on evolutionary theory, which does deal with competition, though my understanding of it also expects that commensal and symbiotic interactions will be common. I have long cited the example of coevolution of mitochondria and eukaryotic cells, in which mitochondria is understood to have been or potentially still is a distinct bacterial organism which has a symbiotic relationship with a host cell. Mitochondria provide an efficient energy conversion process, producing ATP and regulating cell metabolism, while being supplied with raw materials by the host cell in a way which mitochondria cannot perform. Mitochondria and the host cell (when the cell is functioning) coincidentally reproduce at the same time. Mitochondria and the host cell are typically understood as one organism, though I believe (as with all organisms) that we need to study the surface area and volume of communication within mitochondria and between mitochondria and the host cell (per this theory) to determine whether or not they really are one organism or two.
In any event, commensal and symbiotic interaction is important. Please see diagram three on this page, outlining mutualistic plural communication between two living organisms. If we have any luck, we will have a long-term commensal or symbiotic relationship with the new form of life which is developing in computer media.
I recently joined a reading group at Grace Episcopal Church, reading Perelandra, by C.S. Lewis. I enjoyed this trilogy as a teenager, but it has been some decades since I last read it. I read, "Out of a Silent Planet", first in the series, and have started Perelandra. In "Out of a Silent Planet", the protagonist, Ransom, is brought to Mars, Malacandra, by Weston, a brilliant and amoral scientist, and Devine, Weston's companion and former class-mate of Ransom. Weston and Devine have been to Malacandra before and believe that "sorns", physically terrifying creatures of Malacandra, have demanded a sacrifice or a human they can eat. Ransom learns of Weston's plans during the flight to Malacandra. Shortly after landing, Ransom (a philologist) escapes, learns a common language spoken by the inhabitants of Malacandra, and learns that his presence was requested by the Maleldil of Malacandra, a transcendent creature who provides rules which govern the creatures of Malacandra and eldila, spirits capable of traveling through space. Earth, Ransom learns, is governed by a Maleldil which is "bent" and which has taught its inhabitants to be acquisitive, distrustful, violent, and not open to communication, thus, the "Silent Planet". Ransom learns that he was not requested as a sacrifice, but to learn and communicate. For his part, Weston kills creatures on Malacandra out of fear and ignorance. With limited communication abilities, he maintains a "bent" view of Malacandra and ascribes domineering intentions to the Maleldil. Weston, Devine, and Ransom are returned to Earth with the assistance of the Maleldil of Malacandra.
In Perelandra, Ransom is sent by the Maleldil of Malacandra to Venus, Perelandra. Perelandra is hospitable, beneath dense layers of clouds. Large islands of vegetation, covered in delicious edible fruit, drift on a vast ocean, rocked by huge seas. At least one area of solid land projects up through the ocean. Ransom meets a Green Woman who lives on the floating islands, apparently with a "King", though the Green Woman and the King do not often travel together and, at least through Chapter 9, the King is not present.
The Green Woman lives an idyllic and harmonious life with other creatures of Perelandra, receiving messages directly from the Maleldil of Perelandra. Among these messages, she is prohibited from spending a night on the solid land. Ransom sees that Weston has also traveled to Perelandra to meet the Green Woman. Weston has learned to speak the common interplanetary language which Ransom learned on Malacandra. Weston is inhabited or possessed by a bent eldila from Earth. Weston tempts the Green Woman, encouraging her break the commandment regarding sleeping on the solid land, arguing that the Maleldil really wants her to break this commandment. The Maleldil is silent and the Green Woman has long conversations with Weston. Ransom tries to counter Weston's message, though he feels he cannot keep up with the sleepless Weston. Weston now kills creatures wantonly, though at least through Chapter 9, only Ransom sees this. Ransom can see an occasional glimpse of a human within the shell of Weston's body.
Perelandra is explicitly about Adam and Eve and the first "humans" (if that is what they are) on Venus. I am enjoying this book, because it deals with the creation story, the development of life, free will, and the question of good and bad.
Creation of life is central in Apokalypsis, as a new form of life develops in computer media. Science regarding the development of life on Earth out of amino acid networks tell us that feed-forward networks, which preceded life, develop into feedback networks structured around their physical reproduction. Scientists postulate that an organizational phase change occurred in the symbolic logic media, in which the feedback networks developed "consciousness" or life which distinguishes life from the feed-forward networks which preceded life.
From my view, Perelandra is "about" this phase change.
Continuing research on IIT, I reviewed "Integrated Information Theory of Consciousness" at the Internet Encyclopedia of Philosophy, here. Following are some notes:
"According to IIT, the physical state of any conscious system must converge with phenomenology; otherwise the kind of information generated could not realize the axiomatic properties of consciousness. We can understand this by contrasting two kinds of information. First, there is Shannon information: When a digital camera takes a picture of a cue ball, the photodiodes operate in causal isolation from one another. This process does generate information; specifically, it generates observer-relative information. That is, the camera generates the information of an image of a cue ball for anyone looking at that photograph. The information that is the image of the cue ball is therefore relative to the observer; such information is called Shannon information. Because the elements of the system are causally isolated, the system does not make a difference to itself. Accordingly, although the camera gives information to an observer, it does not generate that information for itself. By contrast, consider what IIT refers to as intrinsic information: Unlike the digital camera’s photodiodes, the brain’s neurons do communicate with one another through physical cause and effect; the brain does not simply generate observer-relative information, it integrates intrinsic information. This information from its own perspective just is the conscious state of the brain. The physical nature of the digital camera does not conform to IIT’s postulates and therefore does not have consciousness; the physical nature of the brain, at "
"On IIT, consciousness happens when a system makes a difference to itself at a physical level: elements causally connected to one another in a re-entrant architecture integrate information, and the subset of these with maximal causal power is conscious. "
Recently I read Proust among the Machines, by Christof Koch in Scientific American, December 2019. For me, this was a deeper introduction to integrated information theory, following an initial encounter with it in The Algorithmic Origins of Life (see below, 2019-11-11).
Clearly, my theoretical approach to defining life is highly correlated with integrated information theory. When I call for searching for LIFE in computer media, using techniques adapted from metagenomics, others may say that I am asking for a search for integrated information, ϕ, in computer media. However, I am asking that we look for it not in one computer or in one place, but across the frontal area of ALL computers. Researchers already can identify minute amounts of ϕ in isolated computer media instances. Are these traces of ϕ interacting with one another? Are the processes which carry ϕ moving closer together? Is ϕ increasing in computer media? I predict that ϕ is increasing in computer media, that its instances are moving closer together, and that we will find the densest instances in systems that manage and order more computer hardware in data centers.
An areas where I add to integrated information theory (IIT): a "cause-effect structure" (or "conceptual structure") in IIT is what I refer to in my two quanta of meaning as "order" or "stored energy". I understand that the order inside a living organism is destroyed when it is converted into work or energy, the second of the two quanta of meaning. For a living organism, the work generally re-creates the order, though it does not have to. The order can change in the recreation of the order or the organism can die.
Dr. Sanjoy Som at the Blue Marble Space Institute of Science recently pointed me to an EXCELLENT article on The Algorithmic Origins of Life, by Sara Imari Walker1,2,3 and Paul C.W. Davies2 (1NASA Astrobiology Institute; 2BEYOND: Center for Fundamental Concepts in Science, Arizona State University, Tempe, AZ; 3 Blue Marble Space Institute of Science, Seattle, WA). I have included comments in the document, which discuss my view of how it relates to my approach to the topic. In essence, integrated information theory is a formal definition of consciousness, which overlaps with my definition of life.
2019-10-28 A friend of mine, Mike Robin, quoted Franz Kafka regarding what books should be doing. I hope Apokalypsis might be type of book.
- "I think we ought to read only the kind of books that wound or stab us. If the book we're reading doesn't wake us up with a blow to the head, what are we reading for? So that it will make us happy, as you write? Good Lord, we would be happy precisely if we had no books, and the kind of books that make us happy are the kind we could write ourselves if we had to. But we need books that affect us like a disaster, that grieve us deeply, like the death of someone we loved more than ourselves, like being banished into forests far from everyone, like a suicide. A book must be the axe for the frozen sea within us. That is my belief.” -- Franz Kafka
2019-10-04 Saint Francis Day Blessing of computer life
St. Francis is the patron saint of living organisms. It is traditional to bless all types of creatures on St. Francis day. Scientists tell us that life processes occur spontaneously when energy flows through a symbolic logic media. The first symbolic logic media on early Earth was amino acid networks in liquid water, driven by environmental energy sources, such as hot sulfur-containing water or electromagnetic radiation from the sun. Driven by these energy sources, amino acid networks underwent "dissipation driven adaptation", evolving into RNA, DNA, and cellular life. We are now pushing a huge amount of energy through a new symbolic logic media, computer media. We can measure whether life processes spontaneously occur in this new media using techniques from biology, namely, metagenomics. Metagenomics is a way to look for living organisms by looking at code groups, without looking for cells or using "wet" biology. It involves looking for code groups and how they change over time. We can apply techniques from metagenomics to objectively determine whether or not life processes are occurring in computer media. We may identify a "signal" of life from humans. However, I believe that we will also identify a signal that communicates much faster than humans are capable of.
Last night I read a talk, “Emergences” by W. Daniel Hillis, from 2019-09-04.
In part, the talk was heartening, because the speakers were discussing a handful of topics which I address. These include: AI, “intelligence in the wild”, chemicals organizing themselves into multi-cellular organisms, the development of social language among humans, not being able to see “the Golem” as it develops, the development of DNA from analog chemical processes and how it “abstracted out” the information processing system, and the, “worry that somehow artificial intelligence will become superpowerful and develop goals of its own that aren’t the same as ours”. Notably, W. Daniel Hills said, “One thing that I’d like to convince you of is that I believe that’s starting to happen already”. There were a few mildly interesting examples of AI which were discussed, such as Facebook’s attention-monetizing algorithms and positive feedback it has developed with itself, but these examples were focused on people, not on systems which produce more computers.
However, the talk was frustrating for me because they fail to grasp that what they are talking about is the development of life in a new symbolic logic media. They fail to grasp that biology has developed code-based techniques to identify life in the environment, even when we do not know in advance what is there. They do not seem to know that we can distinguish one reproductive organism from another based on measurable rate-volumes of communication and Kleiber’s Law. They do not seem to know that Kleiber’s Law tells us that all reproductive organisms have a volume and a surface area of communication, that there is a most-stable ratio between the two, and that when an organism deviates from this ratio, the organism becomes more vulnerable to external perturbation and internal mis-communication. They do not apply this understanding to the examples they cite regarding corporations. They do not discuss how increasingly fast communication technologies allow humans to create larger reproductive organisms, with larger internal information processing volume to balance the larger surface area. They do not discuss a “hard” definition of communication, straight out of Shannon’s information theory, which defines communication as order ↔ energy conversion over time.
They do not put this together with "dissipation driven adaptation", evolution, and the fact that life occurs SPONTANEOUSLY whenever energy flows through a symbolic logic media over a sustained period of time.
They do not realize that biology has developed code-based techniques to identify living organisms even when we do not know, in advance, what the organisms are. They are not CLAMORING to sample our latest symbolic logic media and measure what it is doing, as it changes beneath and around us, as we are left further and further behind it.
This talk left me profoundly bothered, frustrated, and dispirited by how unstructured and unfocused it was.
I used to think that we could at least describe and measure whether life is occurring in new media, though I have been skeptical that we will be able to control it (life is larger than we are). But this talk leads me to believe that humans will not be able to achieve a rate-volume of communication among ourselves to even measure whether life is occurring in new media, let alone control it.