Saturday, December 05, 2015

What shall we do with the peer review?

Only one day ago I felt myself happy as I saw that Lubos had written something interesting and even hilarious without any personal attacks or boasting about superstrings as the only possible theory. I did not anticipate that the disappointment would comes the next day. Today Lubos had a long rant attacking Sabine Hossenfelder who had had the courage to prepare a nice slides about the problems of peer review. It became clear that the main motivation of Lubos to argue against anything that Sabine says is that she is Sabine rather than say Jerry. From "Sabine" Lubos eventually deduces that Sabine must be leftist, communist, and totalitarian, and all related evil things except being moslem terrorist and black (implies lower IQ according to Lubos).

The title of Sabine's talk was Peer Review and its Discontents. I studied the slides through two times yesterday and found the representation analytic and entertaining. I agreed about what the situation is since I have long experience about the sad state of recent day theoretical physics (not crisis but stagnation). I would even talk about the breakdown of ethical and moral standards in Big Science. Bee did not talk use these concepts but preferred to talk about self-organization and "misalignment of collective goals and individual incentives". They certainly sound more "scientific" terms in the framework of the physicalism.

  • Bee sees peer review as necessary for a good science. I would see communication as the most important prerequisite for optimizing the rate for the development of science. At best peer review can make possible rapid precisely targeted communication. The problem however is that peer reviewer is often the basic obstacle for the communication of new and far reaching ideas. If new idea goes over the head of the typical peer reviewer, the reaction is "rejected". Peer review is a tool of academic power.

    Not all peer reviewers are mediocrits. I had good luck with my first article about TGD. It was reviewed by Wheeler and he regarded it as brilliant: without his review my work would not have become my thesis. However, 10 years later two finnish theoretical physicists told that my work fails to satisfy all imaginable criteria of good science! This example shows how important it is that peer reviewer satisfies certain intellectual and ethical standards. The formal status of professor is not enough.

  • Bee stressed the fact that scientists can blame only themselves for the situation. Funding agencies, publishers, etc, trust on peer review and peer reviers are scientists themselves. This is true. Only science community can cure the problem if its is curable at all.

  • Bee sees the basic problem as misaligment of collective goals and individual incentives leading to a loss of objectivity. Lubos raged against the idea of collective goals since he tends to see scientists more like individuals in market economy trying to maximize their personal profits. Ideal gas would be the simplest thermodynamical analogy. Bacteria fighting for metabolic resources looks at first a better analogy but it is known that bacteria collaborate and communicate and are far from lonely market economy survivers or aggressice builders of academic career. As a consciousness theorists I see collective consciousness as something very real and individual scientists are for me more like neurons in brain. Negentropy Maximization Principle is the basic variational of TGD inspired theory of consciousness and states that the basic goal of the Universe is to understand itself better - to increases its negentropy recources. For me science as part of universal evolution is much more than an an activity controlled and guided by rational decision makers using some optimization strategy.

  • Bee seems to believe on social engineering of science. I don't know. I have been witnessing the development of theoretical physics for about four decades and on basis of this experience I am not at all convinced about the idea of socially engineered science. All kinds of monitoring and controlling adds new levels of bureaucracy and loopholes and the number of people who do not really understand what is the target of their decisions increases and the decisions are eventually made without any referral to contents using external cues.

    I think that science must be able to do without social engineering. Social engineering could perhaps fasten the breakthrough of really important new ideas by few decades or maybe even centuries - but this is short time in the time scale of evolution. Peer review is not absolutely necessary and good peer review is very rare in the situation in which it would be important. If someone has the idea of century, he or she certainly realizes it and is therefore motivated enough to live outside the academic community and even as a social dropout - often the only option. If there is no "This is it!" feeling, it takes few years to forget the idea and choose the "business as usual" option and live happy family life as a professor. This is quite ok since the alternative would be probably fatal: the life outside the society having bad idea as the only companion is a horrible fate.

  • Bee emphasizes scientific method and objectivity. I do not believe that there is any method for discovery. Discovery means the emergence of something genuinely new and cannot follow by deduction from what already exists. Of course, after the discovery one can apply scientific method and logic to deduce conclusions. Data are important but from data alone one cannot build anything. The endless and more or less useless model building in recent day particle physics demonstrates this rather convincingly. In theoretical physics great vision is necessary but is missing.
Bee modelled the situation using the language of self-organization theory modelling the development of science as a feedback loop involving four basic steps: tracking the change, assessing it, responding to it, and modifying the behavior.
  • According to Bee tracking the change requires channels making it possible to debate and criticize. There is however a further and even more important mechanism involved: new ideas are needed for change and is must be possible to communicate them somehow. In theoretical particle physics and related fields the situation has been especially problematic in this respect. During the first decades of almost 40 years that I have developed TGD I learned that it is impossible to publish in so called respected journals. After the emergence of superstring hegemony also archive begun to apply censorship. Luckily, Research Gate and comparable institutions have now come to rescue and there are reasons for optimism. The possiblity to communicate might eventually induce dramatic change also in theoretical particle physics, quantum gravity, and cosmology. Of course, one must not forget the enormous insitutional arrogance of researches belonging to organizations like CERN.

  • Assessing the change means interpreting the tracked change. Peer review is the basic institutional tool for this. But the problems of peer review have become obvious during decades of superstring revolutions. Group think and wishful thinking make realistic evaluation impossible and fads and fashions are amplified. Funding of fads and fashions continues even after it has become clear that they were failures. The communication barrier becomes plays a key role: peers refuse to take seriously anything, which comes outside the hegemony. One must not forget that all this is also about fight for staying in power.

    There are also human limitations. Peers do not have the needed time and resources to read all the materials involved and therefore they use criteria, which do do not rely on contents such as number of publications and journals in which works have been published. Those who are unknown remain unknown and those who are names become even more namey. Not surprisingly, getting name and funding and building networks have become the primary goals of young scientists instead of doing good science.

    Most of us are mediocrits in the branch of life we live and so are peer reviewers. Mediocrits cannot foresee whether some idea is far reaching so that the those having this kind of ideas must trust their own judgement and find the ways to survive in the environment dominated by mediocrits.

  • Responding to the assessments of peer reviews - and most importantly - to the new ideas represented by others managed to get through the iron curtain of peer review, is far from an unconscious reflex action. It can be a painful process of giving up some belief. Bee mentioned the term loss aversion: if one has worked for decades with some idea, one loses ability to see realistically its status. The tragic situation of SUSYs is mentioned by Bee is indeed a school example about loss aversion.

  • As the fourth step would come the modification. In science it would mean giving up the idea that did not work and starting self re-training. Giving up an idea is not easy. Re-training would take years and in practice various pressures (financial-, time -, peer -, social pressures) make this impossible unless one is willling to leave the academic environment. I do not of course believe that pressures are bad: they serve also as measures for how good your idea is. If it is good enough you can are able to make the choices making possible to avoid the pressures. Isolation can be in some situations the best thing that can happen to a scientist and I must say that I have not experience the loss of contact to most colleagues as an intellectual loss.

Bee argues that modern academic community is too tightly connected to allow the feedback cycle to work. This is true. Typical theoretical physicist talking seriously about unified theories knows nothing about the world outside physics since he has safely enclosed himself in the box of reductionism and physicalism. Biology and neuroscience would provide huge number of poorly understood phenomena - treasure trove for anyone seriously trying to gain unified understanding - but these people have decided that these systems are just "dirty": complexity theory wil take care of all this stuff.

Could social engineering help here? People can good science, when they feel free to do what they regard as interesting and if they are able to be objective. This the definition of good science proposed by Bee and I agree at least partially. I know the incredible importance of motivation. I have however no recipe for how to be objective.

  • The pessimistic side of me is afraid that there is already now too much of social engineering and control to allow good science. Maybe the solution of the problem is at the level of individual rather than social engineering. Individual scientist must become fully conscious of the fact that career building and good science are usually mutually exclusive and accept that the goal to understand how the Universe works is not the path to money and fame. But how can he/she become conscious of this if he/she does not have the idea of century!?

  • Optimistic side of me suggests that situation might not be so gloomy. Maybe the connectedness is the key thing. Too tight connectedness is bad as also too loose connectedness. In condensed matter physics ice is analog for too tightly connected systems. Ice is indeed rather boring system. Gas is is loosely connected but not very interesting either. Liquid is between them and crritical systems are really interesting: long range correlations, correlations, several phases present. If social engineering of science is to be successful it must be able drive the scientists or even science community to a critical state (rather than madness). Energy and information feeds are certainly necessary but in too tightly connected system peer review has become the basic obstacle although it should serve as a selective communication channel.

  • Bee suggests that adaptation at the level of individual scientist and knowledge about knowledge discovery could be the cure of the situation. I cannot say about adaptation: it does not look very promising idea to me. I also doubt that there are some laws governing knowledge discovery allowing to deduce a recipe of good science. I remember that for decades ago many finnish colleagues said publicly that the age of individuals is over in particle physics: we live the era of Big Science. The recent situation in theoretical particle physics demonstrates how wrong these people were.
Personally I believe that better science can come only from improved ethical and moral standards. Most of the problems are after all related to the greed for power and misuse of power. Know yourself: this might be the only advice for those who want to heal the world. Second advice is also simple: learn to listen. I repeat since it seems that you did not listen;-): LISTEN!

For a summary of earlier postings see Links to the latest progress in TGD.

No comments: