Originally Published on the Peer Review Blog.

Scientific and academic publishing is broken. The vast majority of the journals have been privatized by publishers who charge astronomical fees for access to the literature.

The fees have gotten so high that even well funded universities have started to walk away from negotiations. Universities with less funding have long been unable to afford them. The average citizen has no hope of affording them.

With the results of the scientific process locked away behind paywalls, science is no longer an open and transparent process. Worse, the ultimate deciders of policy in a democracy, average citizens, are being denied access to the primary source materials necessary to make good policy decisions.

The Open Access movement has been trying to solve this problem, but it has mostly stuck to the existing model of hiring editors to manually match reviewers to papers, creating high overhead. To fund their operations, most Open Access journals have flipped the business model from fee for access to fee for publish. This has created a whole host of new problems, including the rise of predatory journals that are willing to publish almost anything by anyone who can pay.

The ultimate effect of pay-to-play is that the traditional peer review and refereeing process has broken down. If a dishonest researcher gets rejected from a reputable journal, they can take their paper to a pay-to-play journal and have it published there. With over 10,000 academic journals, it’s impossible for the lay public to track which journals are reputable and which are not. As far as the public is concerned, a published paper is a valid paper.

This is a proposal for a software platform that may help the academic community solve these problems, and more.

Peer Review - A Proposed Publishing Platform

Peer Review is a diamond open access (free to publish, free to access) academic publishing platform with the potential to replace the entire journal system.

Peer Review allows scholars, scientists, academics, and researchers to self organize their own peer review and refereeing, without needing journal editors to manually mediate it. The platform allows review and refereeing to be crowdsourced, using a reputation system tied to academic fields to determine who should be able to offer review and to referee.

The platform splits pre-publish peer review from post-publish refereeing. Pre-publish review then becomes completely about helping authors polish their work and decide if their articles are ready to publish. Refereeing happens post-publish, and in a way which is easily understandable to the lay reader, helping the general public sort solid studies from shakey ones.

Peer Review is being developed open source. The hope is to form a non-profit to develop it which would be governed by the community of academics who use the platform in collaboration with the team of software professionals who build it (a multi-stakeholder cooperative).

Since the platform crowdsources the work of review and refereeing, and because it can potentially handle all academic and scientific fields on a single platform, we could eliminate most of the overhead of academic publishing. Meaning Peer Review could be initially funded with small donations from the scholars using it. If the platform were to eventually replace the entire journal system, it could be funded by the universities for a tiny fraction (1% or less) of what they are paying for publishing now.

Peer Review is currently at the alpha stage of development, most of the core features are functional in a proof of concept. They need testing and hardening, and a number of functionality gaps need to be filled in. I hope to reach a closed beta in the next few months. And an open beta several months after that. I’m seeking academics to give feedback on the concept, help test the alpha, and help us prioritize the roadmap for the closed and open betas.

If Peer Review succeeds, there are any number of ways we could take it. It could potentially solve the file drawer problem from the beginning, by simply giving scholars a place to submit, get immediate feedback on, and publish file drawered papers. We could explore building systems to help incentivize and highlight replications - linking replications to the studies they are replicating and giving replications a reputation bonus. We could build systems to assist with data sharing and funding transparency. We could even explore automating some of the grunt work of maintaining the academic literature - such as generating automated literature reviews. And we could work to make the academic literature more accessible and understandable to general public.

How It Works - The Overview

Here’s how the platform works in detail. When you have a draft of a paper you’re ready to get feedback on, you submit it to the platform. You give it a title, add your co-authors, and tag it with up to five fields or disciplines (eg. “biology”, “biochemistry”, “economics”, etc).

The fields exist in a hierarchical graph, which is intended to be evolutionary. Each field may have multiple parents and many children - eg. “biochemistry” which is a child of both “biology” and “chemistry”. The hierarchy can go as deep as it needs to. We’re initializing the field hierarchy using Wikipedia’s outlines of academic disciplines, but the intention is for the 1.0 version to include the ability for scholars to propose new fields or edits to existing fields, along with a proposed place in the hierarchy, and for their peers to confirm the proposals.

When reputation is gained in a child field, it is also gained in all of that field’s parents. So a paper tagged “astrophysics” also gives reputation in “physics” and “space-science”. Reputation is primarily gained through publishing and receiving positive feedback from your peers during post publish-refereeing - more on that later.

When you hit submit, the draft goes into the review queue. Here other scholars who have enough reputation in any of the fields you tagged the paper with can see the draft and offer feedback on it.

This system gives scholars an enormous amount of control over who they solicit feedback from. By choosing how high up the field tree they go, they choose how wide to cast their net. Because they can add up to five fields (which don’t have to be related) they can easily request interdisciplinary feedback.

Reviewers can click anywhere on the document to leave comments.

When they are ready, reviewers submit their review with a summary and a recommendation. The possible recommendations are:

• “Recommend Approval” meaning that this draft is ready to publish.
• “Recommend Changes” meaning that they think it’ll be ready to publish after the recommended changes are made.
• “Recommend Rejection” meaning that they don’t think this paper is worthy of publishing, or could be made publishable.
• “Commentary” which is just a way to offer feedback and commentary with out a specific recommendation.

Authors can then mark reviews as “accepted”, indicating that they found them helpful, or “rejected” indicating that they were not helpful. Reviews that are “accepted” grant the reviewer reputation in the fields the paper is tagged with (and their parents). Reviews that are “rejected” grant no reputation, but don’t remove it either.

As the review process goes along, authors may upload additional versions of their paper and request new rounds of review feedback for each version uploaded. Reviewers may offer as many reviews to each version as needed, but only gain reputation for a single accepted review on each version.

When the authors are ready, they hit “Publish” and their paper is published and live to the world.

This puts the pre-publish review process entirely in the hands of the authors. It gives reviewers an incentive to give solid, constructive review feedback - and rewards good reviewers for their efforts with recognition of their contributions. It treats review work as a valuable contribution to an academic field alongside publishing.

Refereeing begins once the work is published.

At that point, peer scholars with enough reputation in the fields the paper is tagged with can vote the paper up or down. Up votes increase the paper’s score and grant the authors reputation in the tagged fields. Down votes decrease the paper’s score and the author’s reputation in the tagged fields.

Up votes should be given based on an objective assessment of the paper’s quality. Is this good science? A well constructed argument? Much of the same criteria currently used to determine whether a paper should be published in a well refereed journal, should be used to determine whether a paper should be upvoted, downvoted, or simply left with no score. Instead of that judgment being passed by a handful of reviewers selected by a journal’s editor, it will be collected from the entire community of the fields the paper was submitted in.

Peers can also post a single response to each paper, outlining their feedback and reasons for voting how they did (or not voting at all) in public. Down voters will be strongly encouraged to post a response explaining their downvote.

In this way, the refereeing process is made transparent and easy for the public to follow. A down voted paper should be treated with skepticism. An up voted one is trustworthy. The responses help add context and clarity.

All papers submitted to Peer Review are published under the Creative Commons Attribution License, meaning the work can be freely distributed, remixed, and reused as long as the authors of the original work are attributed.

It’s important to emphasize: Peer Review is intended to be the final publish step. It is not a pre-print server. It is an attempt to replace the journal system with something open to its core, scholar lead, and collectively managed by the scholar community.

Who Are You?

My name is Daniel Bingham and I’m the developer of Peer Review. I grew up in an academic family (my mother and father are both professors and my brother got his PhD), but went into software engineering. I taught myself to code at age 12 and have been in professional software development for well over a decade.

My most recent role was Director of DevOps at Ceros, a mid-sized software company. I built and lead the department which developed and maintained the cloud infrastructure and deployment pipelines for the Ceros Studio and MarkUp. Before building the DevOps team, I was a full stack developer at Ceros helping to build the Ceros Studio.

When I’m not writing code, I’m very involved in local policy. I’ve worked closely with representatives of Bloomington, Indiana’s municipal government on climate, transportation, and housing policy. I’ve served on government task forces and on the boards of local non-profits, including a three year term as president of the board of our local 501(c)3 affordable housing cooperative.

I’ve been dreaming about Peer Review for years. In my role as a policy advocate, I needed access to the research literature, but struggled to get it. As I pondered potential solutions to the problem of open access, the tools I used on a daily basis as a software engineer inspired the concept that became Peer Review.

I have a deep commitment to democracy, and I am as excited about the potential to build a scholar and worker governed organization around the platform as I am to build the platform.

Where Do Things Stand and Where Are They Going?

I am currently the only developer working on Peer Review full time. There are a small handful of volunteers who’ve offered part time help.

I have the alpha version of Peer Review up on a staging server. I’m looking for scientists, researchers, scholars, and academics from all disciplines who are interested in exploring the alpha and giving me feedback on the concept and where to go next.

Could this work the way I think it might? Are there things I’m not thinking of? Problems I haven’t foreseen? Aspects of the academic publishing system I am unaware of and that Peer Review doesn’t account for? Broadly speaking, am I going in the right direction?

I have a roadmap of features I need to put in place before we can begin a closed beta, and a significant amount of hardening and bug fixing to do as well. I hope to reach a closed beta in the next couple months. I’m also soliciting sign ups for the closed beta. Peer Review will only be as good as the community that forms around it, so - if the direction I’m exploring does indeed hold promise - I’m hoping we can start building that community now.

After the closed beta period I intend to do an open beta, where the platform is opened to any who would like to use it (while still understanding that it is not completely finished or polished).

You can view the roadmap on GitHub. If you’re unfamiliar with the process of software development, please keep in mind that the roadmap is a very rough estimate and that it is constantly in flux.

I’m also looking for feedback on that roadmap, are there features which aren’t currently included that should be considered for the closed beta? For the open beta?

If you’re interested in exploring the alpha and giving feedback, want to participate in the closed beta, or want to be notified when we reach open beta, please fill out this form! You can also use that form to give feedback on the initial concept with out signing up for anything.

If you think I’m on to something and you want to see Peer Review grow, please consider supporting the development effort! Right now my family (my wife, my daughter, and I) are living on savings. I’m hoping to raise enough through donations to be able to commit to working on it full time, indefinitely.

I need to raise $8000 / month to make that commitment: $5500 of monthly living expenses, $1500 to cover health care, and $1000 to cover the initial cloud infrastructure costs for site hosting.

You can support me on GitHub Sponsors. You’ll need to make a GitHub Account, but it’s free.

If we successfully raise that much, I will form the non-profit. If we raise substantially more than that I will hire additional engineers, designers, product managers, devops, and QE to help with development. I will also be pursuing grant funding once we reach the closed beta period. Any leads in that direction would be much appreciated.

If you have questions, ideas, suggestions, criticisms, feedback of any kind, grant leads, or offers of help that don’t fit into the form linked above, you can contact me at contact@peer-review.io

Today is my last day at Ceros. I’ve been at Ceros since November of 2014. Minus a 10 month sabbatical in 2016, I’ve worked at Ceros for nearly a full 7 years of my life. That’s almost 10% of an average American lifespan.

Ceros is a great company with a great product team. I got to solve some really fun (and some less fun) problems, worked with incredible people, had some wild adventures, and learned a ton. I got to build a DevOps department from the ground up to now three teams totalling 13 people and growing. I got to help a startup grow from 37 people, when I joined, to now approaching 400. It’s been a hell of a ride.

Now I’m on to the next adventure, which I couldn’t be more excited about.

For the last 3 years, I’ve been dreaming about a web platform that I think has the potential to drastically improve scientific and academic publishing for everyone.

If you’re not in, or adjacent to, academia you may not realize just how broken academic publishing is.

Academic publishing has been privatized and monopolized. Five major publishing houses own the majority (close to 80%) of the academic journals. They charge absolutely enormous fees to sell the output of academia back to the universities.

While this has wildly negative impacts in a lot of areas, the worst of it is in science.

Most scientific research is funded by the public in one way or another. The way scientists share the results of their research is by publishing papers in academic journals. The quality control process is performed by other scientists for free - meaning their (often public) university salaries are funding that work. The private journals then take the results of this work and sell it back to the universities for billions of dollars.

The vast majority of the public can’t even begin to afford to access the scientific and academic literature. The public who, in a democracy, ultimately decide which government policies get implemented and which ones do not.

Science is supposed to be an open process, and that’s why we should trust it. But with the results of scientific research being privatized and hidden behind paywalls, that stopped being true for the vast majority of people in the world. And this is a major contributor to the growing crisis of disbelief in scientific fact. A crisis that contributed to our struggle to manage the pandemic world wide.

The Open Access movement has tried any number of approaches to fixing the access side of it. But they’re mostly still sticking to the same journal model that was developed in the 1680s, when the papers were being bundled into booklets and sent through the mail. This means they have really high overhead, and most open access journals have had to fund themselves by charging a fee to publish. This creates a whole new host of problems, and has lead to the rise of pay to play journals with little or no refereeing.

What this means, practically, is that the peer review process has broken down. A dishonest researcher who gets rejected from a reputable journal can take their paper to a disreputable one and simply pay to have it published. There are over 10,000 journals. It’s impossible for the general public to track which ones are reputable and which ones are not. So as far as the public is concerned, once it’s published, it’s published.

For three years, I’ve been dreaming of a web platform that I believe could fix all of this. For the past three months, I’ve been building it on nights and weekends.

It would allow academics to self-organize the publishing process, without needing the journals or the publishing houses. It uses a reputation system to determine who can peer review and referee papers. It splits pre-publish editorial peer review from post publish refereeing to make sure the quality signal is preserved for the public. And it puts full control of the publishing process back in the hands of academic authors, while allowing their peers to offer helpful pre-publish feedback and to exercise post publish, public quality control.

This web platform would cut the overhead of scientific publishing by several orders of magnitude. I believe it could easily be funded by donations from the academics who use it, and eventually by the institutions (the universities) paying a tiny fraction of what they are now paying. That would allow it to be diamond open access - free to publish, free to access. If successful, it could open up the scientific and academic literature completely.

My hope is to build a non-profit to develop, operate, and maintain the web platform and have it governed by the scholars who use the platform and the workers who build it.

I’m very close to an alpha prototype of the platform. I hope to have one running on cloud infrastructure in the next week or two.

I’m seeking academics and scholars to serve as alpha testers and give me early feedback on the prototype. Could this work the way I think it could? Will this solve the problems of access in academic publishing, while preserving what’s good about the journal system? What am I missing? What else needs to be included in an open, public beta? If you or anyone you know might be interested in helping, reach out to me here and I’ll let you or them know when the alpha is ready!

I’m going to take a moment to talk about Elizabeth Warren. This post has been bouncing around my head since the Nevada debate - particularly since Warren started taking real shots at Bernie.

I want to preface this, I’m a huge fan of Warren. I’ve said from the beginning that I would enthusiastically support Warren in the general, if she were to beat Bernie. My dream outcome is Warren as Senate Majority Leader in a Sanders presidency. I want to see Warren in a position of real power in the movement.

But I don’t think Warren is the right person to lead the movement. And the more Warren goes on the attack against Bernie, the more firmly I believe that.

American Oligarchy

In my last post, I talked about Page and Gilens - the study that showed that America is an oligarchy controlled by the wealthy. Most nationally elected Democrats are wealthy. Most nationally elected Republicans are wealthy. Most national media pundits are wealthy. Most national scale businesses are owned and run by the wealthy.

For the past several decades, these wealthy oligarchs have completely controlled policy in this country - to their own benefit. This is how we’ve wound up with massive income and wealth inequality, with wage stagnation, poor healthcare, student loan crises, mass incarceration, and decaying infrastructure. So many policies that benefit or don’t harm the wealthy oligarchs, while badly harming the rest of us.

When people refer to the “establishment” they’re talking about the oligarchy.

The progressive movement is, and always has been, about overturning the oligarchy and taking power back. It’s about a return to democracy (small “d”), equity, and justice.

In a battle between an oligarchy that has seized power in a democracy, and the people of that democracy who want to take that power back, there is no unity lane. Being willing to work with the oligarchs is not a positive trait, because the oligarchs will not give up power willingly. It has to be taken, by electoral and political force.

This is what Elizabeth Warren has never seemed to understand, and what Bernie has understood from the beginning. When people say that Bernie doesn’t play well with others, the “others” are the oligarchs.

Carrying the Banner

In 2016, the progressive movement begged Warren to carry the banner. There was a powerful Draft Warren campaign trying to get her to challenge Clinton. I was among those who desperately wanted her to run. Bernie himself begged her to run and offered his full support.

Had Warren chosen to run against Clinton, I believe she would have won. If Bernie came as close as he did, Warren would have clobbered Clinton. The lanes of attack open to Clinton against Bernie were not available to her with Warren. It would have been Warren against Trump and she would have cleaned his clock. (Don’t think so? Did you see her eviscerate Bloomberg?)

It would have been Warren who built the progressive movement instead of Bernie.

But she declined. Instead, she lead her senate colleagues in whipping endorsements for Clinton. In that moment, she showed that she doesn’t understand the battle currently unfolding between oligarchy and democracy.

And so, it was Bernie who challenged Clinton. The progressive movement found its champion for democracy in Bernie.

That campaign was a tipping point for the movement. A moment when those who recognize the oligarchy for what it is saw that we were not alone. A moment when Occupy went electoral. A moment when the battle for the soul of the democratic party began.

And Warren sat on the sidelines.

In the four years after that campaign, Bernie worked tirelessly to fuel the flames of the movement that had ignited during his presidential campaign. He built Our Revolution to recruit and support pro-democracy, anti-oligarchy candidates. He personally inspired the next generation of progressive candidates to run for office, many under the Democratic Socialist label. Alexandria Ocasio-Cortez and Ilhan Omar both credit Bernie with inspiring them.

He endorsed. He spoke. He raised money. He carried the banner.

Warren didn’t.

Defeating Trump and Fascism

Now in 2020, she’s been trying to make the case that she should be leading the movement that Bernie’s efforts helped ignite, empower, and coalesce. But in making the case, she’s arguing that the movement should unify with the oligarchy to defeat Trump and fascism.

Don’t get me wrong, defeating Trump and fascism is of vital import, but if we do not also overturn the oligarchy and reinstate democracy, then we will have failed. And it will only be a matter of time before a new Trump emerges.

When Warren tries to run in that unity lane, she shows she still doesn’t understand this. We cannot unite with the oligarchs, we have to defeat them. It was the desire to overturn them that empowered Trump in the first place. He did the classic thing that oligarchs do when they are trying to undermine nascent movements against them - he redirected the anger and frustration at the oligarchy towards immigrants and people of color. He displaced class anger with racism.

We cannot defeat him with out giving that justified anger at the seizure of our democracy by wealthy oligarchs a true voice. And we can’t do that if we’re trying to unify and work with the oligarchs.

Which is why Warren’s attacks on Bernie for not being willing to work with the oligarchs are frustrating and reinforce my belief that she is not the right person to lead the movement.

It could have been her. It should have been her. But she missed the moment. Now we need the person who did build the movement, who understands what it takes to over turn oligarchy, and has never wavered from that goal. And that’s Bernie.

In 2014, Benjamin Page and Martin Gilens reported the results of a multi-year study of American politics, trying to figure out who’s really pulling the levers of power in America.

They tested 4 hypothesis - that it was the broad majority of American voters, that it was issue focused special interest groups, that it was business interests, or that it was the wealthy (defined as the top 10% of the income spectrum).

They looked at polling data from 1981 to 2002 to see which groups wanted what policies, and then looked at which policies actually passed to determine who was calling the shots.

They found that the wealthy were ruling the country.

They found no correlation between the desires of the vast majority of Americans and the policies that got passed. They found no correlation between the desires of special interest groups and what got passed. They found some correlation between the desires of business interests and what policies passed. (As it happens, the wealthy own the businesses.)

They found strong correlation between the desires of the wealthy and the policies that actually became law. The wealthy rule this country.

Most people in political power (in both parties) are well up in the top 10%. Most media commentators are in the top 10%. Most media outlets are owned by stockholders - and 80% of stock is owned by the top 20% of the income spectrum.

When you understand this reality, what is happening with Bernie Sanders’ campaign snaps into clear focus.

The 90% want their power back. They want a true democracy. And the wealthy oligarchy who currently control both political parties and most of the media really don’t want to give that power back.

In his inauguration day speech, the Mayor proposed a .5% increase in the LIT to create a climate justice fund. This would generate about $8 million for the city. He didn’t propose how that money would be spent.

Here’s the thing about the LIT - it’s a flat tax, which means it’s regressive. The sting of that tax is going to be felt much more deeply by low income people who can ill afford any tax increase.

That means, if this is truly going to be a climate justice initiative, then the money needs to be spent first and foremost on programs that will help those who are struggling.

It’s important to know, in this discussion, that a progressive income tax is off the table for us. Like so many things, it’s banned at the state level. The battle for home rule is one we need to fight at some point, but we need climate response now - it cannot wait.

The idea of a 0.25% increase in the LIT to support the public transit system has been on the table since the Spring’s Democratic primary campaign. Everyone in those conversations is very hesitant to propose raising the LIT, because they all recognize its a regressive tax.

With that in mind, I spent all summer desperately trying to find stuff we could cut to create money for climate programs. I came up blank. There just isn’t much in the city budget, at least with the granularity that I’ve got access to, that we can cut with out directly impacting programs people rely on.

We can bond for climate programs, but bonds are not free money. We have to pay them back, with interest, out of the budget. Most of the stuff on the table that we could cut - parking garages, for instance - is bond funded. I believe we should cut those and spend the money on busses and bike lanes, but we would still need annual income to run those busses.

If we’re going to hit the IPCC’s targets for 2030, we need to install $1 billion worth of solar just in Bloomington (or weatherize to cut emissions in half), in the next 10 years. In addition to that $1 billion, we need to cut our transportation emissions by 45%. We need to cut our natural gas use by 45%.

The entire annual city budget is $90 million / year. The whole thing.

Obviously, we’re not going to hit that target on our own. But we also cannot pretend someone else can do this for us.

We’re not going to survive this with out making some sacrifices.

Prior to the Mayor’s proposal, having spent a summer and fall desperately trying to come up with funding for a climate justice program, I had started quietly exploring the idea of using the LIT to fund a wide range climate justice initiatives. Since the Mayor’s kick started the conversation, I’m going to go ahead and put this idea/proposal out there. A counter proposal to the Mayor’s. And where the Mayor didn’t propose a use for that money - I will.

Call it a Green New Deal for Bloomington.

The Proposal

Here’s what I propose, don’t stop at 0.5%, we’ve got a lot to do and we need the money to do it, so max it out. We can go to 2.5% total LIT, we’re at about 1.25%, so raise it the remaining 1.25%. That generates about $20 million /year in revenue for the city.

Here’s how we use it:

Transit: Give $10 million to the transit system. Make transit fare free. That doubles the Bloomington Transit system budget. It allows them to implement full Saturday and Sunday service. It would allow them to massively expand service - more frequent busses, more routes. And it would be free - saving low income people $300 / year (the current cost of an annual bus pass).

Housing: Put $5 million into the city’s affordable housing fund. Require that it be spent on non-profit housing and public housing that helps low income and homeless people. By restricting it to non-profit and public housing, it assures that it’s going to permanently affordable housing and building capacity in organizations that will continue to work towards affordable housing outside of the city government. Eventually, we could build a strong enough cooperative, community land trust, and public housing sector that they could begin to force the overall cost of housing down.

Solar: Put $2.5 million into a zero interest solar loan program that prioritizes low income families. Structure the loans so that they are zero down payment and so that the monthly payment is equal to the average previous electric bill. For those in the lowest income brackets, offer a grant program. Last year, Indiana Solar for All helped install 12 solar systems. This money could fund upwards of 125 per year. Since part of it is structured as a loan program, the amount of new systems installed would increase every year, as the recipients paid back their loans through their utility bills.

Weatherization: Put the remaining $2.5 million into weatherization grants. Prioritize low income and marginalized people. Weatherization can have massive returns in emissions reduction, and poorly insulated housing is a huge cost for a lot of low income people. So give grants to low income home-owners to weatherize and insulate their homes. This will help them cut their utility bills and their emissions.

For someone making $20,000 a year, a 1.25% tax costs them $250 /year. They make that back on the bus fares alone. If we can help reduce the costs of housing, and their utility bills, at the same time, then this tax will be a net benefit to low income people, rather than a burden. And that’s a start for local climate justice.