1. Welcome to Religious Forums, a friendly forum to discuss all religions in a friendly surrounding.

    Your voice is missing! You will need to register to get access to the following site features:
    • Reply to discussions and create your own threads.
    • Our modern chat room. No add-ons or extensions required, just login and start chatting!
    • Access to private conversations with other members.

    We hope to see you as a part of our community soon!

How reliable is peer review

Discussion in 'General Debates' started by We Never Know, Jun 6, 2019.

  1. We Never Know

    We Never Know Well-Known Member

    Joined:
    Jan 20, 2019
    Messages:
    2,201
    Ratings:
    +717
    Religion:
    It exists
    Not a bad article. I see many times in discussions here where it is voiced only peer reviewed links are acceptable. It also goes into ways to improve peer review. Granted the study is from 2006 but it still interesting. I posted one paragraph and the conclusion of the complete article/study. If you want to read more click the link.

    Peer review: a flawed process at the heart of science and journals.

    The defects of peer review.
    So we have little evidence on the effectiveness of peer review, but we have considerable evidence on its defects. In addition to being poor at detecting gross defects and almost useless for detecting fraud it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused.

    CONCLUSION
    So peer review is a flawed process, full of easily identified defects with little evidence that it works. Nevertheless, it is likely to remain central to science and journals because there is no obvious alternative, and scientists and editors have a continuing belief in peer review. How odd that science should be rooted in belief.

    Peer review: a flawed process at the heart of science and journals
     
    #1 We Never Know, Jun 6, 2019
    Last edited: Jun 6, 2019
    • Like Like x 3
  2. shunyadragon

    shunyadragon Veteran Member
    Premium Member

    Joined:
    Mar 23, 2017
    Messages:
    11,107
    Ratings:
    +4,855
    Religion:
    Baha'i Faith
    First, Peer review is not central to science. The whole scope of methodological naturalism where research is repeated many times to verify the reproducibility and predictability of every hypothesis, and published research over time forms the foundation of science. Second, peer review is only one step in a longer process of research in any one science field of research. Third peer review does represent a first step in the process of confirming the validity of research. A number of articles on research are turned down on the basis of the peer review process,
     
    • Like Like x 2
    • Winner Winner x 1
    • Informative Informative x 1
  3. KenS

    KenS Well-Known Member

    Joined:
    Sep 3, 2013
    Messages:
    7,551
    Ratings:
    +2,024
    Religion:
    Judeo/Christian
    Quoted by the Journal of the Royal Society of Medicine and duly supported:

    References
    1. Lock S. A Difficult Balance: Editorial Peer Review In Medicine. London: Nuffield Provincials Hospital Trust, 1985
    2. Jefferson T, Alderson P, Wager E, Davidoff F. Effects of editorial peer review: a systematic review. JAMA 2002;287: 2784-6 [PubMed] [Google Scholar]
    3. Godlee F, Gale CR, Martyn CN. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA 1998;280: 237-40 [PubMed] [Google Scholar]
    4. Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: randomised controlled trial. BMJ 2004;328: 673. [PMC free article] [PubMed] [Google Scholar]
    5. Wennerås C, Wold A. Sexism and nepotism in peer-review. Nature 1997;387: 341-3 [PubMed] [Google Scholar]
    6. Peters D, Ceci S. Peer-review practices of psychological journals: the fate of submitted articles, submitted again. Behav Brain Sci 1982;5: 187-255 [Google Scholar]
    7. McIntyre N, Popper K. The critical attitude in medicine: the need for a new ethics. BMJ 1983;287: 1919-23 [PMC free article] [PubMed] [Google Scholar]
    8. Horton R. Pardonable revisions and protocol reviews. Lancet 1997; 349: 6. [PubMed] [Google Scholar]
    9. Rennie D. Misconduct and journal peer review. In: Godlee F, Jefferson T, eds. Peer Review In Health Sciences, 2nd edn. London: BMJ Books, 2003: 118-29
    10. McNutt RA, Evans AT, Fletcher RH, Fletcher SW. The effects of blinding on the quality of peer review. A randomized trial. JAMA 1990;263: 1371-6 [PubMed] [Google Scholar]
    11. Justice AC, Cho MK, Winker MA, Berlin JA, Rennie D, the PEER investigators. Does masking author identity improve peer review quality: a randomised controlled trial. JAMA 1998;280: 240-2 [PubMed] [Google Scholar]
    12. van Rooyen S, Godlee F, Evans S, Smith R, Black N. Effect of blinding and unmasking on the quality of peer review: a randomised trial. JAMA 1998;280: 234-7 [PubMed] [Google Scholar]
    13. van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial. BMJ 1999;318: 23-7 [PMC free article] [PubMed] [Google Scholar]
     
  4. We Never Know

    We Never Know Well-Known Member

    Joined:
    Jan 20, 2019
    Messages:
    2,201
    Ratings:
    +717
    Religion:
    It exists
    I take it you don't agree with the study and the conclusion.

    Do you agree or disagree that something that is highly subjective, something of a lottery, prone to bias, and easily abused belongs in science?
     
    • Like Like x 2
    • Friendly Friendly x 1
  5. shunyadragon

    shunyadragon Veteran Member
    Premium Member

    Joined:
    Mar 23, 2017
    Messages:
    11,107
    Ratings:
    +4,855
    Religion:
    Baha'i Faith
    No, peer review is only one step in a comprehensive process for the falsification of theories and hypothesis, and the advancement of the knowledge of science. It is not a lottery nor highly subjective, but yes some peer review is prone to bias, but the over all system has numerous checks and balances, is inherently skeptical, and challenged continuously not only by the peer review process, but by further research confirming or refuting previous research.

    As I stated before the peer review process is only a first step in the process of weeding out flawed research, and heart of the process is further research confirming or refuting previous research. The scientific process world wide is extremely redundant.

    There is also a wide range of differences between disciplines concerning the problems with research that fails the test of predictability and reproducibility. The test of the peer review process is the reproducibility and predictability when the research is repeated. This part of the process is more important than peer review. The applied sciences like social sciences, and survey based research have high rates of failure in reproducibility and predictability in the confirmation process of future research. The basic sciences have much better records in confirmation of predictability, and reproducibility.
     
    #5 shunyadragon, Jun 6, 2019
    Last edited: Jun 6, 2019
    • Winner Winner x 2
    • Informative Informative x 1
  6. Audie

    Audie Veteran Member

    Joined:
    Feb 18, 2018
    Messages:
    12,370
    Ratings:
    +5,899
    Religion:
    None
    Do you have a point here, a reason for bringing up
    something well known and much discussed in academic
    circles?
    The Academic Journal such as found in every faculty
    lounge is a good place for articles on the topic.

    How odd that science should be rooted in belief.

    Whoever came up with that quote,
    it is kind of a dumb statement.

     
    • Like Like x 2
    • Winner Winner x 1
  7. We Never Know

    We Never Know Well-Known Member

    Joined:
    Jan 20, 2019
    Messages:
    2,201
    Ratings:
    +717
    Religion:
    It exists
    Sure. The point is I've seen it voiced many times here that only peer reviewed links will be accepted as support. If peer review is highly subjective, something of a lottery, prone to bias, and easily abused as the study says it is, how reliable is it?
     
  8. sayak83

    sayak83 Well-Known Member
    Staff Member Premium Member

    Joined:
    May 7, 2012
    Messages:
    8,542
    Ratings:
    +7,972
    Religion:
    Pluralist Hindu
    The most reliable method yet devised by humans.
     
    • Like Like x 1
  9. metis

    metis aged ecumenical anthropologist

    Joined:
    Aug 27, 2013
    Messages:
    28,536
    Ratings:
    +11,970
    Religion:
    Catholic-- liberal & ecumenical
    Peer review is a process, not a conclusion. If some misuse or abuse that process, the onus is on them, not the process.

    Much the same can be said about religion and all science as well since they also are a process that can be misused and have at times.
     
    • Like Like x 1
  10. We Never Know

    We Never Know Well-Known Member

    Joined:
    Jan 20, 2019
    Messages:
    2,201
    Ratings:
    +717
    Religion:
    It exists
    And to address your last statement. It clearly is talking about the amount of belief placed into peer review
     
  11. We Never Know

    We Never Know Well-Known Member

    Joined:
    Jan 20, 2019
    Messages:
    2,201
    Ratings:
    +717
    Religion:
    It exists
    Uhm no. That would be the scientific method which peer review isn't a part of.
     
  12. sayak83

    sayak83 Well-Known Member
    Staff Member Premium Member

    Joined:
    May 7, 2012
    Messages:
    8,542
    Ratings:
    +7,972
    Religion:
    Pluralist Hindu
    You said this before in my recollection. It was pointed out to you that the problems exist only in soft sciences like psychology etc. Further, peer review makes an article worthy of publication only, whether the ideas in the paper are found to be validated by the wider community is a long drawn multi-decadal process involving much much follow up research.
     
    • Like Like x 1
  13. Audie

    Audie Veteran Member

    Joined:
    Feb 18, 2018
    Messages:
    12,370
    Ratings:
    +5,899
    Religion:
    None
    Lets try it this way.

    The current Ms Universe has a lopsided face! Her
    teeth are crooked, her skin is not exactly "perfect',
    one eye is bigger than the other, one lower than the other,
    her nose.....
    So how pretty could she be???

    You know? All these things are true, but,
    are what I call "lying with the truth".

    As for the use of "peer review" in this forum,
    there is only one poster noted for doing that,
    and I find it tiresome.

    But, still there is for sure a point to this-
    that only peer reviewed links will be accepted as support.

    Actual researchers to not publish their work in the Enquirer.
    Actual professional journals all do peer review.
    Of course t he process is not perfect. Nothing on
    gods green earth is perfect. It happens to be the
    best we have.

    You are invited to show us one (1) single research
    articled from any non-peer reviewed source that
    has any legitimacy to it at all.

    Your argument seems to boil down to something
    like "what is the difference between woo woo and
    science, if science is not perfect?"






     
    • Like Like x 1
  14. We Never Know

    We Never Know Well-Known Member

    Joined:
    Jan 20, 2019
    Messages:
    2,201
    Ratings:
    +717
    Religion:
    It exists
    I don't recall it but I am getting a little forgetful :).

    Yes peer review is for publication, not neccasaraly for validity. It seems some use it/think it is for validity.
     
  15. Audie

    Audie Veteran Member

    Joined:
    Feb 18, 2018
    Messages:
    12,370
    Ratings:
    +5,899
    Religion:
    None
    Now you are just getting silly.
     
  16. sayak83

    sayak83 Well-Known Member
    Staff Member Premium Member

    Joined:
    May 7, 2012
    Messages:
    8,542
    Ratings:
    +7,972
    Religion:
    Pluralist Hindu
    There is no such method. It's a idealized generalization used for explanatory purpose only. There are no peer reviewed papers validating the scientific method. Science is what scientists do in universities and research center's. Scientists are never given any course on "The scientific method" as there is no such thing.
     
  17. sayak83

    sayak83 Well-Known Member
    Staff Member Premium Member

    Joined:
    May 7, 2012
    Messages:
    8,542
    Ratings:
    +7,972
    Religion:
    Pluralist Hindu
    Not. Validity. The idea is that the paper is good enough to be disseminated among other scientists if it has passed peer review.
     
  18. We Never Know

    We Never Know Well-Known Member

    Joined:
    Jan 20, 2019
    Messages:
    2,201
    Ratings:
    +717
    Religion:
    It exists
    I've seen several like that here. You are invited to read the article and refute its points and I never claimed non-peer reviewed articles are correct in their information.
     
  19. Audie

    Audie Veteran Member

    Joined:
    Feb 18, 2018
    Messages:
    12,370
    Ratings:
    +5,899
    Religion:
    None
    Well, not only. Lots of med research is bad, the linked
    article was from a med journal. And no doubt some
    ill considered stuff gets out in chemistry and physics too.

    In the event, your observation is true, that if bad data
    is published-and the topic is of any interest or relevance
    to anyone, which i likely will never be- then the problem
    will come out before long.
     
  20. shunyadragon

    shunyadragon Veteran Member
    Premium Member

    Joined:
    Mar 23, 2017
    Messages:
    11,107
    Ratings:
    +4,855
    Religion:
    Baha'i Faith
    Yes the links used here should be from only reputable sources, but it remains that like in all science research used in references should be subject to rebutal and reputable sources from scientific sources when your dealing with science.

    When the debates involve Social Sciences, Politics, and Theology and Philosophy I can see many problems when selectively used as references in supporting arguments here. In the basic sciences of Biology, Chemistry, Geology, and Physics the only problem I see is misrepresentation of science by those with a religious agenda or a bias against good science.

    I would like to see examples of problems of citations from the main reputable science journals concerning peer review, and flawed research where cited here on RF.
     
Loading...