How reliable is peer review

[shunyadragon]

You might start here. I posted the link that has all reference/sources. It's all there for further in depth reviews by the reader if they choose so.

This is not meaningful at all. Actually some reference are old, and I acknowledged that the problem exists primarily in social sciences and some medical fields like survey analysis. You still have not acknowledged the corrective process of repeated research and the over all corrective processes of the scientific methodes of which the peer review process is only a part of.

You still HAVE NOT referenced where the problem of peer reviewed articles from reputable journals in the basic sciences like Nature and American Science have been a problem in arguments on RF

 

[sayak83]

Not a good point. I was a science major, and the course was required in my major.

Really? The scientific method philosophy course was a required course for a science degree? I did not think it would be common.

 

[David T]

I take it you don't agree with the study and the conclusion.

Do you agree or disagree that something that is highly subjective, something of a lottery, prone to bias, and easily abused belongs in science?

What would be the "objective" measurement to determine that exactly? One could say cosmology itself is problematic. We dont have a math of math although we certainly could create one i suppose.

Science is a narrative and thus it comes under certain limitations like all the rest of the things we do. Ironically there are deeply religious folk off the deep end who might read this as justification for their fantasies.Science is good at pointing out fantasies although it can create fantasies as well. But its all very "normal".

 

[We Never Know]

This is not meaningful at all. Actually some reference are old, and I acknowledged that the problem exists primarily in social sciences and some medical fields like survey analysis. You still have not acknowledged the corrective process of repeated research and the over all corrective processes of the scientific methodes of which the peer review process is only a part of.

You still HAVE NOT referenced where the problem of peer reviewed articles from reputable journals in the basic sciences like Nature and American Science have been a problem in arguments on RF

The whole study is from 2006 as I noted in the OP.

"You still HAVE NOT referenced where the problem of peer reviewed articles from reputable journals in the basic sciences like Nature and American Science have been a problem in arguments on RF"

I never made that claim. I said several here won't accept anything if it isn't peer reviewed and that's the only standard they use. I simply posted a study showing peer review is not the gold standard they think it is.

 

[shunyadragon]

I believe this article goes into the 'peer review process, it's flaws, and issues in science. Please note that this article describes peer review is a part of the scientific method process, and needs to read in full.

From: How Scientific Peer Review Works

Most people didn't know much about scientific peer review five years ago. Then, in December 2005, South Korean scientist Dr. Hwang Woo Suk stunned the world by admitting that his stem cell research -- research that was supposed to revolutionize health care by helping to cure diseases ranging from diabetes to Parkinson's -- used fabricated data. Although the revelation brought disgrace to Hwang and poured fuel on the stem cell controversy, it had a more damaging effect on the public's perception of science itself. Suddenly, there were reports questioning how Science, the prestigious U.S. journal that published Hwang's findings, could have been so easily duped. Other reports condemned the process of science itself as antiquated and flawed.

At the heart of that process is scientific peer review, a quality-control system that requires all new scientific discoveries, ideas and implications to be scrutinized and critiqued by expert scientists before they become widely accepted. Peer review has been around for nearly 300 years, so it is not new. It just seems that way sometimes because of the attention it has received in the wake of the stem cell scandal. Unfortunately, increased awareness does not always translate into increased understanding. Many myths and misconceptions about peer review still exist, and many average citizens don't see how a system of checks and balances is important either to science or to their day-to-day decision making.

To make matters worse, the conventions of peer review, just like the conventions of many traditional systems and processes, are being challenged by Web 2.0 technologies. A new generation of scientists is turning to the Web as a vehicle to post raw experimental results, fledgling theories and draft papers. Proponents of "open access" practices argue that science is improved in a more democratic and collaborative atmosphere. Critics warn that such promiscuity with research data undermines the very integrity of the scientific endeavor.

These are some of the issues we will explore in this article. We'll cover the basics -- what is scientific peer review, how does it work and what is its historical context -- before moving on to an analysis of what it can and can't do. Finally, we'll examine some of the current trends in peer review to understand how the system is evolving and may continue to evolve.

But first, let's expand a bit on the basic definition of peer review."

 

[We Never Know]

What would be the "objective" measurement to determine that exactly? One could say cosmology itself is problematic. We dont have a math of math although we certainly could create one i suppose.

Science is a narrative and thus it comes under certain limitations like all the rest of the things we do. Ironically there are deeply religious folk off the deep end who might read this as justification for their fantasies.Science is good at pointing out fantasies although it can create fantasies as well. But its all very "normal".

Agreed. Religious people don't like anything that doesn't agree with their reality.
I'm seeing that happen in some non-religious people as well.

 

[shunyadragon]

Really? The scientific method philosophy course was a required course for a science degree? I did not think it would be common.

The course is indeed taught in virtually all major universes, but yes requirements may vary. This where I learned of the important contributions by Popper to the modern development of the scientific method. The course I took was primarily addressing science.

 

[shunyadragon]

Agreed. Religious people don't like anything that doesn't agree with their reality.
I'm seeing that happen in some non-religious people as well.

Careful your posts in this thread are replete with generalizations, and often vague.

 

[We Never Know]

I believe this article goes into the 'peer review process, it's flaws, and issues in science. Please note that this article describes peer review is a part of the scientific method process, and needs to read in full.

From: How Scientific Peer Review Works

Most people didn't know much about scientific peer review five years ago. Then, in December 2005, South Korean scientist Dr. Hwang Woo Suk stunned the world by admitting that his stem cell research -- research that was supposed to revolutionize health care by helping to cure diseases ranging from diabetes to Parkinson's -- used fabricated data. Although the revelation brought disgrace to Hwang and poured fuel on the stem cell controversy, it had a more damaging effect on the public's perception of science itself. Suddenly, there were reports questioning how Science, the prestigious U.S. journal that published Hwang's findings, could have been so easily duped. Other reports condemned the process of science itself as antiquated and flawed.

At the heart of that process is scientific peer review, a quality-control system that requires all new scientific discoveries, ideas and implications to be scrutinized and critiqued by expert scientists before they become widely accepted. Peer review has been around for nearly 300 years, so it is not new. It just seems that way sometimes because of the attention it has received in the wake of the stem cell scandal. Unfortunately, increased awareness does not always translate into increased understanding. Many myths and misconceptions about peer review still exist, and many average citizens don't see how a system of checks and balances is important either to science or to their day-to-day decision making.

To make matters worse, the conventions of peer review, just like the conventions of many traditional systems and processes, are being challenged by Web 2.0 technologies. A new generation of scientists is turning to the Web as a vehicle to post raw experimental results, fledgling theories and draft papers. Proponents of "open access" practices argue that science is improved in a more democratic and collaborative atmosphere. Critics warn that such promiscuity with research data undermines the very integrity of the scientific endeavor.

These are some of the issues we will explore in this article. We'll cover the basics -- what is scientific peer review, how does it work and what is its historical context -- before moving on to an analysis of what it can and can't do. Finally, we'll examine some of the current trends in peer review to understand how the system is evolving and may continue to evolve.

But first, let's expand a bit on the basic definition of peer review."

There are many articles about peer review. The one I post was a study on peer review from ncbi.
On the web if one searches enough, they will eventually find something that they like better.

 

[PoetPhilosopher]

So we have little evidence on the effectiveness of peer review

I think it works well in the porn industry. They have different labels for the professional stuff, and it gets more hits. So a case of peer review actually working.

A friend told me that.

 

[We Never Know]

Careful your posts in this thread are replete with generalizations, and often vague.

Many religious people don't like science that shows for example the flood didn't happen because it goes against their reality of the biblical accounts.

 

[shunyadragon]

And to address your last statement. It clearly is talking about the amount of belief placed into peer review

Scientists are inherently skeptical, and do not place 'belief' in the peer review process. You're sounding more and more like a Creationist anti-science advocate that science is based on 'belief.'

 

[shunyadragon]

Many religious people don't like science that shows for example the flood didn't happen because it goes against their reality of the biblical accounts.

This is true, but does not address the issue.

 

[We Never Know]

Careful your posts in this thread are replete with generalizations, and often vague.

And to this part of my post "I'm seeing that happen in some non-religious people as well."

It's my opinion if the study would have said the opposite of peer review like it's the gold standard, not flawed, the best, not bias, etc., there would be no argument about the study from people of no religion.

 

[We Never Know]

This is true, but does not address the issue.

It addressed the post that was replied to

 

[David T]

Agreed. Religious people don't like anything that doesn't agree with their reality.
I'm seeing that happen in some non-religious people as well.

Yes. We are a culture deeply steeped in a fable? A myth? A fantasy? of the intellect. Generally trees often times make more sense to me than people. We tend to get lost in the things we think and debate that which this forum is about. Rarely do we puncture that illusion and get down into to how we are experiencing and that effects how we see ourselves and nature. I treate it as a jeruselem effect on spectrum related to cult-ure.

 

[We Never Know]

Scientists are inherently skeptical, and do not place 'belief' in the peer review process. You're sounding more and more like a Creationist anti-science advocate that science is based on 'belief.'

I didn't write the study or anything in it. I did find it interesting. On the last sentence about "belief"(which is the way it was worded in the study) I simply suggested maybe they weren't saying science is about belief, but scientists have belief in peer review.

 

[sayak83]

The course is indeed taught in virtually all major universes, but yes requirements may vary. This where I learned of the important contributions by Popper to the modern development of the scientific method. The course I took was primarily addressing science.

Hmm, but scientific researchers rarely think about scientific method while doing research in any field that I know. It remains a topic of consideration only in the circles of philosophy.

 

[Subduction Zone]

Sure. The point is I've seen it voiced many times here that only peer reviewed links will be accepted as support. If peer review is highly subjective, something of a lottery, prone to bias, and easily abused as the study says it is, how reliable is it?

Peer review is a minimal standard when it comes to scientific claims. I am sure that this has been explained to you before, but let's go over it one more time.:

Yes, peer review is quite often wrong. New ideas have to be tested and retested to be sure that they are valid. But ideas that cannot even pass peer review, a relatively low hurdles, are almost always wrong. Peer reviewed articles may be wrong as high as two thirds of the time, depending upon the branch of science. Ideas that failed peer review usually failed because they are obviously wrong. I would put ideas that failed peer review at being wrong far over 90% of the time. What is wrong with demanding a minimal standard when it comes to support for one's beliefs?

 

[We Never Know]

And to this part of my post "I'm seeing that happen in some non-religious people as well."

It's my opinion if the study would have said the opposite of peer review like it's the gold standard, not flawed, the best, not bias, etc., there would be no argument about the study.

Oh and to add, I would think this study critiquing peer review was probably peer reviewed itself. That's irony.