Scientific publishing platform to crowdsource review using reputation

Hacker News

Share card

Scientific publishing platform to crowdsource review using reputation

Hey HN, Academic and scientific publishing is badly broken with impacts that affect all of us. It's still operating more or less the same way it did prior to the internet, but it is exactly the sort of thing the internet was supposed to make faster, more transparent, and more freely available. The platform I built is an attempt to replace the mostly closed academic journals with something open -- open source, open access, and user governed. It enables crowdsourced peer review of academic papers by using a reputation system (similar to StackExchange) and tying reputation to a field/concept tagging system. Submitted papers must be tagged with 1 - n fields, and only peers who have passed a reputation threshold in one of the tagged fields may offer review. Review is also split into two phases: pre-publish and post-publish. Pre-publish review is author driven. It's focused on collaborative, constructive feedback and uses an interface heavily inspired by both Github Pull Requests and Google Docs. Post-publish review is much closer to traditional review, and is focused on maintaining the integrity of the literature by filtering out spam, misinformation, fraud, and poorly done work. Reputation is mostly gained and lost through voting that happens during post-publish review. Reputation can also be gained by offering particularly constructive pre-publish reviews. All reviews are open and published alongside the papers. Post-publish review is on-going. In theory, this system could allow review to happen among members of each academic field with out the need for editors to organize it. It also allows authors to have a dialog with reviewers during the pre-publish review stage, allowing for much better feedback and potentially contributing to better final papers. As much as I believe review could and should be crowdsourced, it seems pretty clear that going straight from what we have to this platform would be a huge leap for a lot of people. So I have ideas for how to build a journal overlay on top of the crowdsourced review system that would allow editors to manage teams of reviewers and run their journals through the platform. Authors would then be able to choose to submit their papers to one or more journals, crowdsourced review, or both. Building that out is the next project. I would love feedback on all aspects of this project - both the current iteration and attempt to crowdsource and the idea to build a generic, open platform for diamond open access journals on top of that iteration. You can find the platform here: https://peer-review.io The source here: https://github.com/danielbingham/peerreview And more details about exactly how it works (in its current iteration) here: https://peer-review.io/about Maintaining an open roadmap here: https://github.com/users/danielBingham/projects/6/views/1