• Welcome to Religious Forums, a friendly forum to discuss all religions in a friendly surrounding.

    Your voice is missing! You will need to register to get access to the following site features:
    • Reply to discussions and create your own threads.
    • Our modern chat room. No add-ons or extensions required, just login and start chatting!
    • Access to private conversations with other members.

    We hope to see you as a part of our community soon!

How reliable is peer review

We Never Know

No Slack
Not a bad article. I see many times in discussions here where it is voiced only peer reviewed links are acceptable. It also goes into ways to improve peer review. Granted the study is from 2006 but it still interesting. I posted one paragraph and the conclusion of the complete article/study. If you want to read more click the link.

Peer review: a flawed process at the heart of science and journals.

The defects of peer review.
So we have little evidence on the effectiveness of peer review, but we have considerable evidence on its defects. In addition to being poor at detecting gross defects and almost useless for detecting fraud it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused.

CONCLUSION
So peer review is a flawed process, full of easily identified defects with little evidence that it works. Nevertheless, it is likely to remain central to science and journals because there is no obvious alternative, and scientists and editors have a continuing belief in peer review. How odd that science should be rooted in belief.

Peer review: a flawed process at the heart of science and journals
 
Last edited:

shunyadragon

shunyadragon
Premium Member
Not a bad article. I see many times in discussions here where it is voiced only peer reviewed links are acceptable. It also goes into ways to improve peer review. Granted the study is from 2006 but it still interesting. I posted one paragraph and the conclusion of the complete article/study. If you want to read more click the link.

Peer review: a flawed process at the heart of science and journals.

The defects of peer review.
So we have little evidence on the effectiveness of peer review, but we have considerable evidence on its defects. In addition to being poor at detecting gross defects and almost useless for detecting fraud it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused.

CONCLUSION
So peer review is a flawed process, full of easily identified defects with little evidence that it works. Nevertheless, it is likely to remain central to science and journals because there is no obvious alternative, and scientists and editors have a continuing belief in peer review. How odd that science should be rooted in belief.

Peer review: a flawed process at the heart of science and journals
First, Peer review is not central to science. The whole scope of methodological naturalism where research is repeated many times to verify the reproducibility and predictability of every hypothesis, and published research over time forms the foundation of science. Second, peer review is only one step in a longer process of research in any one science field of research. Third peer review does represent a first step in the process of confirming the validity of research. A number of articles on research are turned down on the basis of the peer review process,
 

Kenny

Face to face with my Father
Premium Member
Not a bad article. I see many times in discussions here where it is voiced only peer reviewed links are acceptable. It also goes into ways to improve peer review. Granted the study is from 2006 but it still interesting. I posted one paragraph and the conclusion of the complete article/study. If you want to read more click the link.

Peer review: a flawed process at the heart of science and journals.

The defects of peer review.
So we have little evidence on the effectiveness of peer review, but we have considerable evidence on its defects. In addition to being poor at detecting gross defects and almost useless for detecting fraud it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused.

CONCLUSION
So peer review is a flawed process, full of easily identified defects with little evidence that it works. Nevertheless, it is likely to remain central to science and journals because there is no obvious alternative, and scientists and editors have a continuing belief in peer review. How odd that science should be rooted in belief.

Peer review: a flawed process at the heart of science and journals
Quoted by the Journal of the Royal Society of Medicine and duly supported:

References
1. Lock S. A Difficult Balance: Editorial Peer Review In Medicine. London: Nuffield Provincials Hospital Trust, 1985
2. Jefferson T, Alderson P, Wager E, Davidoff F. Effects of editorial peer review: a systematic review. JAMA 2002;287: 2784-6 [PubMed] [Google Scholar]
3. Godlee F, Gale CR, Martyn CN. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA 1998;280: 237-40 [PubMed] [Google Scholar]
4. Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: randomised controlled trial. BMJ 2004;328: 673. [PMC free article] [PubMed] [Google Scholar]
5. Wennerås C, Wold A. Sexism and nepotism in peer-review. Nature 1997;387: 341-3 [PubMed] [Google Scholar]
6. Peters D, Ceci S. Peer-review practices of psychological journals: the fate of submitted articles, submitted again. Behav Brain Sci 1982;5: 187-255 [Google Scholar]
7. McIntyre N, Popper K. The critical attitude in medicine: the need for a new ethics. BMJ 1983;287: 1919-23 [PMC free article] [PubMed] [Google Scholar]
8. Horton R. Pardonable revisions and protocol reviews. Lancet 1997; 349: 6. [PubMed] [Google Scholar]
9. Rennie D. Misconduct and journal peer review. In: Godlee F, Jefferson T, eds. Peer Review In Health Sciences, 2nd edn. London: BMJ Books, 2003: 118-29
10. McNutt RA, Evans AT, Fletcher RH, Fletcher SW. The effects of blinding on the quality of peer review. A randomized trial. JAMA 1990;263: 1371-6 [PubMed] [Google Scholar]
11. Justice AC, Cho MK, Winker MA, Berlin JA, Rennie D, the PEER investigators. Does masking author identity improve peer review quality: a randomised controlled trial. JAMA 1998;280: 240-2 [PubMed] [Google Scholar]
12. van Rooyen S, Godlee F, Evans S, Smith R, Black N. Effect of blinding and unmasking on the quality of peer review: a randomised trial. JAMA 1998;280: 234-7 [PubMed] [Google Scholar]
13. van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial. BMJ 1999;318: 23-7 [PMC free article] [PubMed] [Google Scholar]
 

We Never Know

No Slack
First, Peer review is not central to science. The whole scope of methodological naturalism where research is repeated many times to verify the reproducibility and predictability of every hypothesis, and published research over time forms the foundation of science. Second, peer review is only one step in a longer process of research in any one science field of research. Third peer review does represent a first step in the process of confirming the validity of research. A number of articles on research are turned down on the basis of the peer review process,

I take it you don't agree with the study and the conclusion.

Do you agree or disagree that something that is highly subjective, something of a lottery, prone to bias, and easily abused belongs in science?
 

shunyadragon

shunyadragon
Premium Member
I take it you don't agree with the study and the conclusion.

Do you agree or disagree that something that is highly subjective, something of a lottery, prone to bias, and easily abused belongs in science?

No, peer review is only one step in a comprehensive process for the falsification of theories and hypothesis, and the advancement of the knowledge of science. It is not a lottery nor highly subjective, but yes some peer review is prone to bias, but the over all system has numerous checks and balances, is inherently skeptical, and challenged continuously not only by the peer review process, but by further research confirming or refuting previous research.

As I stated before the peer review process is only a first step in the process of weeding out flawed research, and heart of the process is further research confirming or refuting previous research. The scientific process world wide is extremely redundant.

There is also a wide range of differences between disciplines concerning the problems with research that fails the test of predictability and reproducibility. The test of the peer review process is the reproducibility and predictability when the research is repeated. This part of the process is more important than peer review. The applied sciences like social sciences, and survey based research have high rates of failure in reproducibility and predictability in the confirmation process of future research. The basic sciences have much better records in confirmation of predictability, and reproducibility.
 
Last edited:

Audie

Veteran Member
Not a bad article. I see many times in discussions here where it is voiced only peer reviewed links are acceptable. It also goes into ways to improve peer review. Granted the study is from 2006 but it still interesting. I posted one paragraph and the conclusion of the complete article/study. If you want to read more click the link.

Peer review: a flawed process at the heart of science and journals.

The defects of peer review.
So we have little evidence on the effectiveness of peer review, but we have considerable evidence on its defects. In addition to being poor at detecting gross defects and almost useless for detecting fraud it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused.

CONCLUSION
So peer review is a flawed process, full of easily identified defects with little evidence that it works. Nevertheless, it is likely to remain central to science and journals because there is no obvious alternative, and scientists and editors have a continuing belief in peer review. How odd that science should be rooted in belief.

Peer review: a flawed process at the heart of science and journals

Do you have a point here, a reason for bringing up
something well known and much discussed in academic
circles?
The Academic Journal such as found in every faculty
lounge is a good place for articles on the topic.

How odd that science should be rooted in belief.

Whoever came up with that quote,
it is kind of a dumb statement.

 

We Never Know

No Slack
Do you have a point here, a reason for bringing up
something well known and much discussed in academic
circles?
The Academic Journal such as found in every faculty
lounge is a good place for articles on the topic.

How odd that science should be rooted in belief.

Whoever came up with that quote,
it is kind of a dumb statement.

Sure. The point is I've seen it voiced many times here that only peer reviewed links will be accepted as support. If peer review is highly subjective, something of a lottery, prone to bias, and easily abused as the study says it is, how reliable is it?
 

sayak83

Veteran Member
Staff member
Premium Member
Not a bad article. I see many times in discussions here where it is voiced only peer reviewed links are acceptable. It also goes into ways to improve peer review. Granted the study is from 2006 but it still interesting. I posted one paragraph and the conclusion of the complete article/study. If you want to read more click the link.

Peer review: a flawed process at the heart of science and journals.

The defects of peer review.
So we have little evidence on the effectiveness of peer review, but we have considerable evidence on its defects. In addition to being poor at detecting gross defects and almost useless for detecting fraud it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused.

CONCLUSION
So peer review is a flawed process, full of easily identified defects with little evidence that it works. Nevertheless, it is likely to remain central to science and journals because there is no obvious alternative, and scientists and editors have a continuing belief in peer review. How odd that science should be rooted in belief.

Peer review: a flawed process at the heart of science and journals
The most reliable method yet devised by humans.
 

metis

aged ecumenical anthropologist
Peer review is a process, not a conclusion. If some misuse or abuse that process, the onus is on them, not the process.

Much the same can be said about religion and all science as well since they also are a process that can be misused and have at times.
 

We Never Know

No Slack
Do you have a point here, a reason for bringing up
something well known and much discussed in academic
circles?
The Academic Journal such as found in every faculty
lounge is a good place for articles on the topic.

How odd that science should be rooted in belief.

Whoever came up with that quote,
it is kind of a dumb statement.

And to address your last statement. It clearly is talking about the amount of belief placed into peer review
 

sayak83

Veteran Member
Staff member
Premium Member
Not a bad article. I see many times in discussions here where it is voiced only peer reviewed links are acceptable. It also goes into ways to improve peer review. Granted the study is from 2006 but it still interesting. I posted one paragraph and the conclusion of the complete article/study. If you want to read more click the link.

Peer review: a flawed process at the heart of science and journals.

The defects of peer review.
So we have little evidence on the effectiveness of peer review, but we have considerable evidence on its defects. In addition to being poor at detecting gross defects and almost useless for detecting fraud it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused.

CONCLUSION
So peer review is a flawed process, full of easily identified defects with little evidence that it works. Nevertheless, it is likely to remain central to science and journals because there is no obvious alternative, and scientists and editors have a continuing belief in peer review. How odd that science should be rooted in belief.

Peer review: a flawed process at the heart of science and journals
You said this before in my recollection. It was pointed out to you that the problems exist only in soft sciences like psychology etc. Further, peer review makes an article worthy of publication only, whether the ideas in the paper are found to be validated by the wider community is a long drawn multi-decadal process involving much much follow up research.
 

Audie

Veteran Member
Sure. The point is I've seen it voiced many times here that only peer reviewed links will be accepted as support. If peer review is highly subjective, something of a lottery, prone to bias, and easily abused as the study says it is, how reliable is it?

Lets try it this way.

The current Ms Universe has a lopsided face! Her
teeth are crooked, her skin is not exactly "perfect',
one eye is bigger than the other, one lower than the other,
her nose.....
So how pretty could she be???

You know? All these things are true, but,
are what I call "lying with the truth".

As for the use of "peer review" in this forum,
there is only one poster noted for doing that,
and I find it tiresome.

But, still there is for sure a point to this-
that only peer reviewed links will be accepted as support.

Actual researchers to not publish their work in the Enquirer.
Actual professional journals all do peer review.
Of course t he process is not perfect. Nothing on
gods green earth is perfect. It happens to be the
best we have.

You are invited to show us one (1) single research
articled from any non-peer reviewed source that
has any legitimacy to it at all.

Your argument seems to boil down to something
like "what is the difference between woo woo and
science, if science is not perfect?"






 

We Never Know

No Slack
You said this before in my recollection. It was pointed out to you that the problems exist only in soft sciences like psychology etc. Further, peer review makes an article worthy of publication only, whether the ideas in the paper are found to be validated by the wider community is a long drawn multi-decadal process involving much much follow up research.

I don't recall it but I am getting a little forgetful :).

Yes peer review is for publication, not neccasaraly for validity. It seems some use it/think it is for validity.
 

sayak83

Veteran Member
Staff member
Premium Member
Uhm no. That would be the scientific method which peer review isn't a part of.
There is no such method. It's a idealized generalization used for explanatory purpose only. There are no peer reviewed papers validating the scientific method. Science is what scientists do in universities and research center's. Scientists are never given any course on "The scientific method" as there is no such thing.
 

sayak83

Veteran Member
Staff member
Premium Member
I don't recall it but I am getting a little forgetful :).

Yes peer review is for publication, not neccasaraly for validity. It seems some use it/think it is for validity.
Not. Validity. The idea is that the paper is good enough to be disseminated among other scientists if it has passed peer review.
 

We Never Know

No Slack
Lets try it this way.

The current Ms Universe has a lopsided face! Her
teeth are crooked, her skin is not exactly "perfect',
one eye is bigger than the other, one lower than the other,
her nose.....
So how pretty could she be???

You know? All these things are true, but,
are what I call "lying with the truth".

As for the use of "peer review" in this forum,
there is only one poster noted for doing that,
and I find it tiresome.

But, still there is for sure a point to this-
that only peer reviewed links will be accepted as support.

Actual researchers to not publish their work in the Enquirer.
Actual professional journals all do peer review.
Of course t he process is not perfect. Nothing on
gods green earth is perfect. It happens to be the
best we have.

You are invited to show us one (1) single research
articled from any non-peer reviewed source that
has any legitimacy to it at all.

Your argument seems to boil down to something
like "what is the difference between woo woo and
science, if science is not perfect?"






I've seen several like that here. You are invited to read the article and refute its points and I never claimed non-peer reviewed articles are correct in their information.
 

Audie

Veteran Member
You said this before in my recollection. It was pointed out to you that the problems exist only in soft sciences like psychology etc. Further, peer review makes an article worthy of publication only, whether the ideas in the paper are found to be validated by the wider community is a long drawn multi-decadal process involving much much follow up research.

Well, not only. Lots of med research is bad, the linked
article was from a med journal. And no doubt some
ill considered stuff gets out in chemistry and physics too.

In the event, your observation is true, that if bad data
is published-and the topic is of any interest or relevance
to anyone, which i likely will never be- then the problem
will come out before long.
 

shunyadragon

shunyadragon
Premium Member
Sure. The point is I've seen it voiced many times here that only peer reviewed links will be accepted as support. If peer review is highly subjective, something of a lottery, prone to bias, and easily abused as the study says it is, how reliable is it?

Yes the links used here should be from only reputable sources, but it remains that like in all science research used in references should be subject to rebutal and reputable sources from scientific sources when your dealing with science.

When the debates involve Social Sciences, Politics, and Theology and Philosophy I can see many problems when selectively used as references in supporting arguments here. In the basic sciences of Biology, Chemistry, Geology, and Physics the only problem I see is misrepresentation of science by those with a religious agenda or a bias against good science.

I would like to see examples of problems of citations from the main reputable science journals concerning peer review, and flawed research where cited here on RF.
 
Top