Why is Random Selection More Effective than “Rational Selection”? / Sheng Hong

When people talk about the origin of life or intelligent creatures, they are always amazed at the extremely low probability of its realization, so some people take this to prove that if there is no intentional design with super wisdom, how can this happen? The implication of this hypothesis is that super intelligence can carry out rational design, and its success rate is much higher than the random choice of nature. In fact, this is a specious concept. The actual situation may be just the opposite.

We must first assume that this super wisdom is not a mysterious existence of super rationality, but only has higher rationality. At the beginning, we do not know the structure and parameters of creating life or intelligent creatures. We can design and create only after exploring the knowledge that can constitute life or wisdom. We already know that the probability of success of this life form is very small, for example, only one in 100 trillion. How can this super wisdom find such a small probability of knowledge? One way is to try at random until finding the right knowledge. Another way is to conceive the structure of a life form first, and then design a screening program according to this idea. Among the 100 trillion possibilities, 9999 / 10000 are screened away first, so there are 10 billion left, and then they are randomly selected here. Which of these two methods is more efficient?

At first glance, the latter method seems to be more effective. If true knowledge is hidden in the 10 billion possibilities screened by the program, even if the selected 10 billion possibilities are selected randomly or sequentially, it is 10000 times the efficiency of completely random selection of 100 trillion possibilities. However, if true knowledge is not hidden among these 10 billion possibilities, and people believe that the screening program is right, they will ever lose the opportunity to find true knowledge. If people design another screening program to choose the possibility outside of the last screening, in principle, it is no different from comprehensive random selection. The method of randomly selecting 100 trillion possibilities may be 10000 times slower than the previous method, but it will never lose the opportunity to find real knowledge. Thus, random selection is more effective than “rational” selection.

The question is, how the screening procedure of the above hypothesis came from. Designing this screening program also requires knowledge. To make this screening program effective, the best way is to know what the potential knowledge is, but this is a paradox, because this is the object to be found. Therefore, people think it is deliberate screening, but in the view of nature it is only a random choice. The key is that if people think that the screening program they design is better than random selection, it may lead to a worse result, that is, they miss the real knowledge. We call comprehensive random selection “neutral selection”, so all artificially designed screening procedures are “non neutral”. This means that one screening program may be biased away from real knowledge, while the other may be biased close to real knowledge. Which direction a screening program is biased in is just a random probability. So on average, deliberate screening is no more efficient than random selection.

So, since it is so difficult to find real knowledge, isn’t the scientific exploration since modern times deliberately looking for it? Haven’t people made great achievements? Isn’t it more effective than random selection? Since the probability of finding true knowledge is so small, hasn’t human society developed a brilliant civilization for thousands of years? First of all, it should be clear that successful science has only made some achievements in the field of simple systems. One is that the number of possibility rules of simple systems is small, and the other is that these knowledge can be observed directly. It seems that people choose deliberately. In fact, scientists find true knowledge after many failures, twists and turns. This process, in nature’s view, is actually random. Human civilization is indeed a complex system, but this complex system is developed from very simple rules, and the number of rule possibilities is not much. We now recognize that human civilization is not the design of a wise man, but originated from spontaneous order. This is a random choice.

For example, when Wolfram, the author of A New Kind of Science, experimented with the rules of one-dimensional ternary two-state Cellular Automata, he found a total of 256 possible rules. He was able to experiment one by one and found three interesting complex structures, and the rest rules were of little significance. When the rules are very simple, the number of possibilities of rules is relatively small. People can find them through experiments one by one. Although they feel that they are looking for them according to rational design, in fact, they later found that true knowledge is an accident, a random probability. Just as Michael Faraday found that electromagnetic induction was an accident. Usually, scientists put forward various hypotheses, then confirm or falsify them. If false, continue to improve the hypothesis or give way to another hypothesis. For example, the heliocentric theory negates the geocentric theory, and the oxidation theory negates the phlogiston theory. Or some complementarity is formed between several hypotheses, such as the wave theory of light and the particle theory of light, and finally the wave particle duality theory is formed. These practices, in a pure sense, are random choices.

Stephen Wolfram: A New Kind of Science | Online—Table of Contents

The rules formed in civilized society seem complex, but almost all of them are formed in the minimalist initial rules. For example, for a specific situation at each moment, people have only two choices as to what to do next. Yes or no, as time goes on, they form observable and orderly rules. This situation is similar to the one-dimensional ternary two-state model of cellular automata, and the number of possibility rules is relatively small. For example, in the securities market, everyone just decides according to the current situation whether to buy or sell or not to change the status quo (if they have held and continue to hold, if they have been short, continue to be short). If everyone goes on like this step by step, a complex price and quantity structure and time fluctuation series of the securities market will be formed. What human beings can do now is to know through experience which rules are effective, which are invalid and which are negative. Market rules are the rules discovered and mastered through the early trial and error process of mankind. This process is random. Hayek said the market was “discovered by chance”.

The basic rules of some complex systems are not as simple as the one-dimensional ternary two-state cellular automata model. For example, plants in nature should be said to be three-dimensional. Every step of plant growth must decide how to change in three directions: length, width and height. Once cellular automata is a little more complex, such as changing into one-dimensional ternary three states, there are 7625597484987 possible rules (Wolfram, 2002, page 60). When entering the two-dimensional level, the number of rule possibilities will further increase. There are 4294967296 kinds of rules for two-dimensional nine element two states only (Wolfram, page 213). If “in systems containing more than just a few cells, the total number of possible patterns is absolutely astronomical, and so enumerating them becomes completely impractical.” (Wolfram, page 217) It can be imagined that if it is a three-dimensional system, the number of possible rules is even more uncountable.

In this case, the method of sequential experiment by experiment is obviously not feasible. According to the above statement, it is not feasible to “rationally” design a screening procedure. Wolfram once designed a program to automatically search a huge number of possible rules of mobile cellular automata, added various parameters of how he expected mobile cellular automata to behave, and quickly searched 1 million possible rules, followed by 10 million. But he “got nothing”. So he removed these parameters. After a short time, he found two interesting patterns. He concluded, “The main challenge was always to avoid assumptions, and set up experiments that were simple and direct enough that they did not miss important new phenomena.” (Pages 112-113) This confirms our previous conclusion that random selection is better than “rational” selection.

Even in the aspect of minimalist behavior rules, as long as people can’t observe directly, people can’t find true knowledge through rational design in a large number of possible rules faster than random selection. For example, in human society, which is much simpler than animals and plants, no rule or system has been man-made so far. On the contrary, even when some organisms or self-organizations are formed according to these rules, such as market, village community or social order, people cannot know how the original basic rules were found because they did not see their origin. Hayek once said that the formation of spontaneous orders such as customs is ancient. People do not know when and how they were formed. The information contained in them is not fully understood and grasped by us. This is enough for Hayek to emphasize that most of the rules of human society are formed spontaneously, not artificially designed.

In comparison, plants and animals on earth were originated much earlier and grew in three-dimensional space. It is impossible for human beings to see their origin, and because the possibility of their formation rules is astronomical, it is impossible to find the true rules of their formation. Therefore, human beings are doomed to be impossible to create life. They can plant seeds and let them germinate, blossom and bear fruit. They can change plant characters by grafting. They can implant chromosomes in one cell into another cell. They can also imitate the formed life to synthesize a life, but they can’t create a new life type. It is said that Craig Venter’s team synthesized organisms in 2010, but bioethicist Arthur Kaplan commented that “Venter did not really create life.” because the formation of any life type is as if win a trillion awards, the probability is very low. If human beings look for it in order, it will take billion years, which is almost impossible in human society; to “rationally” design a screening program to speed up this process, as mentioned earlier, the result will be worse than random selection.

However, such time-consuming and permanent work can be done by Nature. It has plenty of time. The best way is to choose at random. We can imagine that in the huge experimental field of the universe, random choices are taking place in trillions of different places all the time. Because they are random, it is equivalent to trillions of experiments at the same time. Because the rules are simple, the experiment is not difficult. We can imagine that the vast majority and nearly all of the experiments failed. But after a long time, one random selection succeeded. From the perspective of time, successful behavior will show an orderly structure, just like the 30th, 90th and 110th rules of one-dimensional ternary two-state cellular automata, showing the pattern formed by one-dimensional space over time, which is a complex structure. The characteristics of this structure are: first, it can be repeated, but it is not simple repetition; second, it will form some coordination among multiple individuals. Due to the “use and disuse” evolutionary mechanisms, the structure of behavior in the time dimension will be transformed into the structure in the space dimension.

Behavior is the different forms of energy, and matter is the spatial structure of energy. This is very similar to the explanation of string theory. At the most basic level of matter, there are only some energy forms expressed as strings, or “energy strings”. Elementary particles are the spatial structure of strings. This spatial structure is mainly manifested in the repetition and coordination of energy behavior. “Repetition” is expressed as movement along a circular orbit or fluctuation at a certain frequency; “Synergy” is the connection and structure between strings. According to the same meta rule, countless elementary particles randomly choose the true rule to form quarks, electrons, protons and so on; atoms and molecules are formed according to the same meta rules. This is the emergence of the material universe, the solar system and the earth. The appearance of all these substances is related to the behavior of the basic individuals who form them, that is, the selected behavior. In the same way, a more complex spatial structure has emerged, which is the original organism.

This means that it is life and it can copy itself. Although the replication process is not very fast, it is a moment compared with the process of randomly selecting true rules. These primitive lives still follow the simplest meta rules and make random selection of behavior rules, that is, make behavior choices of yes or no, black or white, 0 or 1, and also conduct trillions of experiments simultaneously on the huge experimental field of the earth. However, this round of random selection is not a simple repetition of the initial selection, but is quite different. First, individuals are different. The original is still an inanimate individual, and the space of choice is relatively narrow; now it is life that makes choices, and the capacity of behavior is different. The space of choice becomes larger, and the complexity of behavior rules increases. At the same time, the possibility of behavior rules also increases sharply. There are many such levels from the simplest individual to the most complex life. For each level of complexity, the loop described above is repeated.

In the random selection at each level, most of the behavior rules are proved to be invalid or failed. Occasionally, one or two behavior rules are effective, so they are affirmed and continued. The organisms that adopt this behavior rule will develop. The organisms that do not adopt this behavior rule either stay in the original state or go to extinction. Once again, the time structure of the new selected behavior rules is transformed into the spatial structure of the organism, and the organism becomes more complex, and this organism change is conducive to the implementation of the selected behavior rules. Like Darwin’s theory of evolution, cattle ruminate, so they grow two stomachs; people have to think, so the brain has a large capacity. This rule has been like this from the beginning. Organisms that have evolved more complex will randomly explore the rules of behavior to adapt to new organisms in all directions, which has carried out a new cycle. The new behavior rules are determined, which leads to new changes in the body.

Of course, the behavior rules affirmed by nature are not necessarily one, but many. For example, even among the 256 possible rules of the one-dimensional ternary two-state cellular automata model, there are still three showing complex structures. So you can see all kinds of microorganisms, plants and animals, even people. These are the successful schemes of organisms with complex structures affirmed by nature in random exploration. In each change bifurcation, there will be a variety of choices. For example, although modern plants are ever-changing, most of their branches and leaves diverge at an angle of 137.5 º, indicating that after the plant ancestors chose this appropriate angle, a large number of different species appeared. Another possible situation is that a behavior rule with time structure and its system structure have been selected, and there is no opponent for a long time, but it does not mean that it is optimal, because a better rule and structure have not yet been selected. For example, the angle of antelope horn bending does not need to be the best, as long as it is better than its competitors, it can dominate. Moreover, there is no so-called pure optimization, only the optimization in a specific environment. This is what Darwin’s theory of evolution has revealed.

Some people argue that if it is a random choice, “it will take nearly 100 billion years to produce even a very simple pronuclear life.” This is absolutely impossible for the earth with a history of only 4.6 billion years. However, this view takes the probability of the emergence of known organic structures as the object of discussion. In fact, this does not refute Darwin’s theory of evolution. Evolution theory precisely means that all kinds of biological species in today’s universe have evolved from the simplest structure. That is, the evolution process is from simplicity to complexity. This is much easier than creating a complex system at once. According to Wolfram’s “new science”, nature begins with the simplest rules of behavior. That is, any individual randomly selects behavior rules. This rule allows the individual to make the next choice according to the current state. There are only two choices, yes or no, black or white, 0 or 1. Although the number of possible rules is also huge, it is much less than the number of possible combinations of alternatives to a prokaryotic life structure, and the probability of being selected is much higher. For example, there is only 10 to the 14th power.

This hierarchical evolution path greatly reduces the number of possible forms of behavior rules if there is no stratification, which itself saves the cost and time of random search and selection. This path was discovered by another author, Brian Arthur. In his book Complexity Economics, Arthur pointed out that the evolution of complexity has three forms. The first is called “growth of coevolutionary diversity”, that is, due to the formation and emergence of new individuals or “species”, it creates “niche”, which makes other individuals or “species” easier to generate, and they will produce new “niche”; there are interactive cycles as such (2018, P. 227). In other words, when the simplest individual is generated, it will bring convenience to the more complex individual generation in the future. Each level of complexity will have a similar effect in the future. Therefore, the most complex structure or “species” is not a super rational sacred design, nor can it be formed randomly out of thin air, but is formed step by step from simplicity to complexity, and it is the result of random selection in each level.

Current Address

In this way, the random selection of behavior rules generates life and intelligent creatures on earth. We have proved earlier that this method is more efficient than the “rational” choice method, that is, the method of rationalist constructivism. It is also much simpler than the random selection of the organism structure to prove creationism, so that the work of forming life and even human beings can be completed within the age of the earth. From the perspective of occurrence, the random selection of organism structure is fundamentally against common sense. If you can’t do simple machines, but directly do complex machines, it will be much more difficult than the step-by-step way. Comparing with the organism, behavior is easier to change. Therefore, evolution must start with behavior change, and then the organism adapts to behavior, not behavior to adapt to the organism. More clearly, the random choice of behavior rules is between “rational” choice and random choice of organism structure. Its success probability is so small that rational choice cannot be better than it, but often not as good as it; it is so  simple that it can form life and intelligent creatures in the  time-length of earth.

So, if rational choice is not as good as random choice, what is the use of rationality? The first and most important thing is to know that “rational choice is not as good as random choice”. This is not nonsense. This is the most important knowledge in human cultural tradition. Through his search for behavior rules, Wolfram found that randomness is the most important property to ensure the emergence of orderly structure. He said, “For essentially all purposes the only reasonable model for important new features of organisms is that they come from programs selected purely at random.” (Wolfram, Pp. 399). Randomness here is expressed as “freedom”, “nature” and “spontaneous order” in the existing human culture. We found that this is the most valuable part of human culture. Look closely again, they all contain the meaning of randomness. For example, “freedom” means not excluding any possibility, that is, it can develop in any direction, that is, there should be no restrictions, including those that seem “rational”. And “nature”, which directly excludes man’s deliberate action, is to make nature work, which is the role of random change. “Spontaneous order” is the order that excludes human rational design.

Here, we should distinguish between two kinds of randomness. Randomness is that, first, the probability of any choice is equal; second, the possibility of any individual changing in any direction is equal. However, the differences between individuals are different. For example, in the case of maximum entropy, the temperature at different points is the same, that is, the difference is very small, so the random selection here will not have any results, because each individual is almost the same. In another case, such as the big bang, the universe continues to expand, and the differences between individuals, including temperature and spatial location, are different or even expand. At this time, random selection is meaningful, because different choices have different results. Therefore, these two randomness can be called Big Bang randomness and Big Collapse randomness, or expansion randomness and contraction randomness respectively. In this paper, “randomness” refers to the former randomness. In this case, random selection is better than rational selection.

What else can reason do? First, the behavior rules of simple system are discussed. This is what science has done since modern times. Physical or chemical laws are the “rules of behavior” of simple systems. Because of the small number of possible rules, many scientists’ trial and error and Einstein’s genius conjecture are still useful. Second, imitate the simple system itself. Man-made simple systems can directly imitate natural simple systems, such as ancient houses; It can also be built according to the “rules of behavior” of simple systems, such as modern architecture. Third, explore the behavior rules of organisms, such as the origin of customs and DNA; simple imitation of the structure of an organism itself, such as a bionic machine. The so-called “simple imitation” is a much more rough imitation, which can only realize one or two simple functions of the imitated object, and must not be equivalent to the imitated object. If we cross this border, it will lead to problems and even disasters. For example, a simple imitation of the existing social order and structure and the establishment of a planned economy is a much worse imitation.

After organisms and social systems have been generated, random selection will be limited by the selection results. That is, it will exclude the negation of the selected organism or social system. That’s what Hayek meant when he said that freedom and order are two sides of the same coin. Order is the selected behavior rule. Following order is not the result of denying freedom, but the result of freedom. The human brain is the organic structure of the selected behavior rules, and its thinking is the affirmation of the brain structure. Therefore, the brain’s exploration of all behavior rules and system structures and following the results of exploration seem to be a negation of randomness, but in fact it is an affirmation of the results of randomness. Of course, any formed behavior rules or system structures also face variation and new choices. Therefore, nature does not rule out the changes of existing rules or structures. Therefore, when affirming the selected rules and structures, the brain will also hold a more tolerant attitude, that is, follow the rules but do not rule out a certain degree of deviation.

Then, is there still a God when we have proven that random selection can generate humans? In fact, God is another way of saying that man is limited. As mentioned earlier, the random selection of behavior rules is much simpler than the random selection of organism structure, so the earth has enough time to develop people; however, the random selection of behavior rules is more effective than rational selection. The possible number of behavior rules greatly exceeds the ability of human beings to choose one by one, so it is beyond human rationality. So it still highlights the limitations of human beings. And who arranged the real rules of behavior hidden in hundreds of millions of possibilities? What rules does this arrangement follow? This is still a bigger mystery. Those random choices in nature, those social orders formed spontaneously when human beings are ignorant, human beings only know these things, but do not know how they are formed; it can only be named “spontaneous order”. The visible spontaneous order may not be the best. The natural law hidden behind may never be known by mankind. Therefore, after proving that random selection is more effective than rational selection, people should still be in awe of the omniscient existence.

(Stephen Wolfram, A New Kind of Science, Wolfram Media Inc., 2002.

Brian Arthur, Complex Economics, Zhejiang People’s publishing house, 2018.)

At Five-woods Study on January 30, 2019

The Chinese version of this essay was published firstly in the fourth issue of 《读书》(Readings)in 2019

Author: flourishflood

Economist, Confucianist

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: