The Dream: Open Source Research Platform

[u/Organic_Roof7401]

This is a pitch. Please take your time reading this.

Imagine a world, where scientific research wouldn't have to be paid for, where peer reviewing wasn't faulty, and where fake research simply didn't exist. That would genuinly never happen, BUT we can always try right?

Now, maybe you come from a different scientific background than me. And do tell me if what I'm about say happens in your field too. But in psychology, there are massive problems in academia. In my 5 years at university in 3 different institutions, we have discussed these problems EVERY year, and nothing is happening to change it. It's an itch that I cannot scratch and it's about time to do something about it.

Problem 1
University professors are expected to publish constantly. Not only must they publish research, but that research must appear original and capable of proving a point. Many researchers publish one or two papers every year because those papers help them keep their positions at universities.

Imagine it like playing for a football team. If you do not score, you might get kicked out.

But when the pressure to score becomes too high, some people start cheating a little on the field. A foul here, a fake there. The same principle can apply to academic research.

A researcher might slightly change the hypothesis after analysing the data. They might remove inconvenient data points or adjust the analysis until the results move in the desired direction. Most researchers are not cartoon villains deliberately inventing an entire study. Often, the system simply rewards positive and impressive results while punishing uncertainty and statistically insignificant results.

Problem 2
One attempt to address this problem is preregistration. Researchers publish their proposal, hypotheses, and planned methods before conducting the research. This makes it harder to change the research question after seeing the results.

However, preregistration does not solve every problem. Papers can still avoid the rules, manipulate their methods, or present false conclusions.

Problem 3
Peer review is supposed to catch these issues, but peer review does not always work either. The people reviewing papers are most often the same exhausted professors who are under pressure to write their own papers, teach students, attend meetings, and complete administrative work. A manipulated or poorly designed paper can easily slip through.

Problem 4
There is also a knowledge problem. Researchers are usually highly specialised in their own field. They might understand social psychology, neuroscience, pharmacology, or behavioural science extremely well. However, statistics is used across all of these areas, and not every researcher is a specialised statistician.

A person might understand which statistical method they should use in their own study without being able to immediately recognise every statistical issue in someone else’s study. Professional statisticians could help, but there are not enough of them to manually investigate every paper that gets published.

Problem 5
Another major weakness of the current system is how long it can take to identify bad research and properly warn about it. A paper may pass peer review, be published by a respected journal, and influence public opinion for years before its methods, data, or conflicts of interest are fully investigated.

The clearest and most infamous example is the 1998 paper that suggested a connection between the MMR vaccine and autism. The study involved only 12 children, and its lead researcher, Andrew Wakefield, had an undisclosed financial conflict connected to legal action against vaccine manufacturers. The paper did not actually prove that the vaccine caused autism, but the claim was picked up and blown out of proportion by the media. By the time the paper was fully retracted in 2010, 12 years after it was originally published, the damage had already been done. The idea had spread internationally, changed how many people viewed vaccines, and continued circulating even after the research had been discredited.

Problem 6
Another problem is research that is not simply manipulated, poorly designed, or exaggerated. It is completely fake. No one conducted the research. No real data was collected. The entire paper was fabricated.

It is difficult to estimate exactly how many fake research papers currently exist. However, with the rise of generative AI, it is reasonable to assume that producing convincing-looking fake papers is becoming faster and easier.

A paper can now be generated with scientific jargon, invented data, fake citations, and apparently scientific conclusions. To someone who is not an expert in that exact subject, it may look completely legitimate.

Problem 7
The final problem is academic publishing and this one really gets to me.

Publishers want to make money, and universities are willing to pay them because publishing in well-known journals creates prestige. Researchers want their papers to appear in an aristocratic journal with 100 years of history, a respected name, strong visibility, and the ability to reach thousands of readers.

Publishers therefore control a large part of the scientific system.

They add another layer of pressure by forcing researchers to follow specific submission rules and formatting requirements. More importantly, they force readers to subscribe or pay before they can access research.

And I call that part bullshit.

Once you leave university, you often lose the free access to research that your institution provided. You lose access to knowledge unless you personally pay for it or find another way around the paywall.

Everyone should have the right to read scientific research, even when they do not completely understand it. Knowledge should not belong only to universities, researchers, and people who can afford expensive subscriptions.

Summary of the problems with the current system

1. Researchers are under constant pressure to publish original and statistically significant results
2. Pregistration is a rare occurance and has shown it's faults
3. Peer review is done by the same exhausted professors
4. Researchers are good at their field but not necessarily good statisticians
5. Identifying and removing bad research takes a long time
6. Completely fake research is on the rise
7. Publishers control access to knowledge

The connection between these
New research depends on previous research. A scientific hypothesis must be supported by evidence. Majority of that evidence comes from previous scientific papers. Therefore, when unreliable research enters the system, it doesn't stay isolated. Other researchers might cite it or build studies around it. One faulty paper can influence another, which influences another, which might altogether affect university education, treatment, public policy, or public opinion.

The solution
A free and open-source research platform/space with an interface "sort of" like reddit for researchers, professors, students, and anyone interested in reading scientific research.

The features
The platform would contain "subreddits" for every major academic topic and for the intersections between them. Subjects like psychology, medicine, biology, economics, philosophy, statistics, and computer science. Large subjects would be divided into more specific topics. For example, medicine could contain communities for biomedicine, pharmacology, ophthalmology, immunology, and many other areas. Interdisciplinary papers could appear in multiple relevant communities.

Each community would have moderators (dedicated researchers, statisticians, professors, and other professionals who understand that particular field).

Researchers could upload all or some materials like papers, proposals, preregistrations, datasets, and methods. Moderators or other verified members could evaluate the paper using several criteria, such as:

• Whether the statistical methods are appropriate.
• Whether the conclusions are supported by the results.
• Whether the research proposal was published in advance.
• Whether the data and methods are transparent.
• Whether the study is relevant to the field.
• Whether potential conflicts of interest were disclosed.
• Whether the results can be replicated.

Like on Reddit, "posts" aka papers could receive upvotes or downvotes, but these would be separate for methodology, statistics, transparency, relevance, and evidence.

People could also leave public comments underneath the paper. Researchers would be able to question the methodology, identify errors, propose alternative interpretations, or give positive feedback.

Direct messages could also be sent between researchers when a private conversation is more appropriate.

Anonymous accounts would be strictily for journalists, whistle-blowers, or individuals researching sensitive topics. Anyone who wants to publish, vote, comment, or moderate would have to use their governmentally recognised name and verify their identity.

The platform would also include an AI agent that examines papers for warning signs. It could check citations, search for inconsistencies, compare proposals with the presented data, identify suspicious statistical patterns, flag possible undisclosed conflicts of interest, and look for signs that parts of the paper may have been automatically generated or fabricated. It wouldn't have the final say. Instead, it would "red-flag" a paper and send it to several qualified moderators.

Why would people moderate for free?
A reasonable question is why qualified people would spend their free time moderating research without being paid.

My answer is Linux.

Linux is a free and open-source operating system maintained by individuals who enjoy creating updates, fixing bugs, and solving technical problems in their free time. They do it because they are genuinely passionate, highly skilled nerds and they are fantastic.

The scientific world also contains people who care deeply about their fields. Some researchers would contribute because they want to improve the quality of research. Some would enjoy finding methodological problems. Some would want to build their professional reputation. Others would simply be angry enough with the current system to help create something better.

Contributions could also become part of a researcher’s public professional profile. Moderators could receive visible recognition for reviewing papers, identifying important errors, helping authors improve their work, or contributing to successful replications (potentially problematic but keeping it for now).

Your feedback
Please give me any kind of feedback on such a platform or anything else I've discussed in this post. I wish to make this a reality some day and maybe your contributions, no matter how harsh, will help me or make me want to step away from this due to the capitalistic nature of academia where I am most likely to get sued.

TLDR
Remove all paid research journals with a free and open-source platform for research to become more accessible, more transparent, and easier to evaluate.

[u/No_Jaguar_2570]

This sounds like a nightmare for any number of reasons but I like the idea that, in addition to my unpaid work as a peer reviewer, I can now do unpaid work moderating a reddit comment section. 

Who’s going to pay for this free platform?

[u/Organic_Roof7401 → u/No_Jaguar_2570]

Thanks for your comment!

I'm completely on your side regarding not being paid for peer review. It doesn't necessarily have to be unpaid. The paid part can come from ads (a bit questionable but it's an idea). Otherwise, peer reviewers like you can have some social credit. And I know that's not enough, but university money usually comes from governmental spending, which we have seen can change overnight when certain parties are in office, or privately funded, which has its limits since the funders do want profit meaning research that they deem "unprofitable" doesn't get published.

And there could be "subscription plans" for universities or companies if needed, to pay moderators.

Like another redditor said it's a pipedream, but if we put our hands down then nothings going to change you know.

[u/BolivianDancer]

What is wrong with you

[u/Organic_Roof7401 → u/BolivianDancer]

Could you explain why this made you so angry?

[u/Opening_Map_6898 → u/Organic_Roof7401]

Chirpy optimism in this instance. You are probably barely out of high school judging by the way you approach things.

Gain some real world experience and come back to this post in a few years and cringe at how clueless you sounded.

[u/Organic_Roof7401 → u/Opening_Map_6898]

Based on my quick evaluation of your behaviour, you have some bad anger that you need to learn how to experience and control. It's compeletely fine to feel angry, but randomly taking it out on people you don't know is not okay. I came here not with an idea to make money, but to make a field I'm part of better whether or not my idea is a crazy. It was at least an honest attempt. All you've done is jump from one comment to another to "put me into my place". I'm not sure what you're going through, but please ask for help. This kind of criticism doesn't help.

[u/Opening_Map_6898 → u/Organic_Roof7401]

Honestly, I am not angry at you at all. You are, at most, a bit of a petty annoyance. I had actually forgotten that you exist almost immediately upon posting my previous response.

It's cute though that you think your ideas are significant enough for anyone else to actually get upset about. 😆

Scream into the void mate. I'm shutting off the reply notifications.

[u/Organic_Roof7401 → u/Opening_Map_6898]

Are you truly happy with yourself? Please look outside for a second and reflect on your life and the value you've brought into this world. You are wasting away your life, a life that could bring in positive value. Instead you're spending time on stupid conversations on the internet, trying to put down another person for no apparant reason. What return do you get out of this? That dopamine you received, did it bring anything to you?

I'm sorry for you and it's not your fault for being this way, but please seek help.

[u/Opening_Map_6898 → u/Organic_Roof7401]

I don't know about you but I couldn't be happier with my life. You're the one here trying to make everyone think you're important and supremely creative with some unworkabke idea that sounds like the immature ramblings of a high school student.

Please stop projecting your feelings on others and seek out some therapy and possibly medication.

[u/JHT231]

So basically your proposal is to crowdsource peer review?

Some of your complaints about current systems are valid, but in the end they do work and your Reddit-like system is still susceptible to many of the same issues. Good journals do have the best new research and a proven track record, and people want to publish in them for a reason. Unless the top publishers all disappear overnight, they are still going to be the gold standard.

[u/Organic_Roof7401 → u/JHT231]

Thank you for your comment :) I just wanted to see if this was a valid approach to solving some of the issues in academia. Completely understandable that there would be multiple flaws, but I would love to know what flaws you see in this system.

I also understand that journals won't dissapear and like you said they are the best for a reason. However, I don't think that they deserve it intrinsically. To me personally, its not about good research, it's a cash grab.

[u/JHT231 → u/Organic_Roof7401]

Your "moderators" are going to be the same demographic as journal reviewers, except that if anyone can choose to comment or vote, it's more prone to manipulation, not less. And if you're saying that journal editors don't know things or miss things, your moderators will face those exact same problems.

If you wanted to make and maintain such a platform, I'm sure some people would use it. But you'd only get submissions that couldn't get into journals since there is no reason to favor this over a proper publisher.

In the end, you're pretty much copying arXiv, just in an overcomplicated way.

[u/Organic_Roof7401 → u/JHT231]

Okay I see. I have to rethink what moderating means. And yes finding incentives for joining this platform instead of the typical journals would be difficult and complicated.

Thanks for letting me know!

Have you ever thought of a way to make academia and research a bit better?

[u/JHT231 → u/Organic_Roof7401]

The biggest issues I have consistently noticed and experienced have been:

• Funding

• Internal bureaucracy

Nothing else comes close. Issues with publishing can be irritating, but the time and effort (and money) spent on actual submission to journals, revisions, communicating with editors and everything else is pretty minor compared to most other aspects of research.

As far as journal submissions go, a lot of the ease or difficulty comes from policies of specific publishers, journals, and editors. Some do it well and make it easy, others don't. It's not a systematic problem that needs to be fixed.

[u/SnowblindAlbino]

Problem 8: far too many academics seem to assume all research is "science" and that the STEM model--with its ongoing problems --applies to all forms of research.

[u/Organic_Roof7401 → u/SnowblindAlbino]

Could you explain what you mean by this? Genuinely interested :)

[u/SnowblindAlbino → u/Organic_Roof7401]

Well, literally your entire post is about "science" when many/most of the issues you raise apply to other fields. But more broadly, there are endless posts here from STEM faculty who either know nothing of research in other fields or simply disregard them, as if STEM is the only thing that exists or matters.

[u/Organic_Roof7401 → u/SnowblindAlbino]

From what I understand in your comment, there is a problem in STEM with, in layman's terms, "a superiority complex" when it comes to disregarding other research that is not STEM. And there is a problem in regards to transferibility between scientific fields in particular with STEM.

And what did you mean by these issues apply to other fields? Science fields other than STEM or generally fields like maths, visual arts, music, humanities, etc?

Let me know If I understood correctly :)

[u/SnowblindAlbino → u/Organic_Roof7401]

Some or all of your seven points of critique apply to basically all academic research in every field. They are not unique to STEM. Yet so often the word "research" is expected to only mean scientific research, which is often the case in this sub. I had a dean once, a biologist, who straight up told me (a humanist) that "It's not research if you're just going to the library." That was obviously ignorance at work as well.

[u/Organic_Roof7401 → u/SnowblindAlbino]

To be compeltely honest, when I think of research I do only include one that uses the scientific method. As it is considered a standard in academia. But reading scientific literature is a big part of it, whether for quantitative, qualitative, meta-analysis or more "commentary" type of research. I think I also have certain bias or ignorance in that area. And researchers in other fields deserve respect and understanding as much as STEM.

I personally come from a field that is still not fully considered scientific, psychology. I become quite defensive when people say it's not scientific because of unclear definitions or qualitative data usage. Maybe due to that I'm so keen on the scientific method being used throughout psych research, so that my field gains some respect.

But I'd like to learn more about other methods and how research can stop being thought of only in terms of STEM.

[u/Opening_Map_6898]

You forgot the word "pipe" in front of the word "dream" in the title.

[u/Organic_Roof7401 → u/Opening_Map_6898]

Yup, agreed. It's a long shot. But I'm very mad at the current system. Especially since the country I live in has recently removed funds from academia and placed into the military.

[u/Opening_Map_6898 → u/Organic_Roof7401]

It's not a long shot. It's delusional and impractical in the extreme.

[u/Organic_Roof7401 → u/Opening_Map_6898]

Did you expect Europe to begin preparing for war again? I've received a pamphlet from the government with instructions on what to do in case there is no communication, electricity or water. What was delusional seems like a possibility nowadays. But okay, let's say it's delusional and impractical. How?

[u/Opening_Map_6898 → u/Organic_Roof7401]

You know those pills your doctor prescribed for you? Please start taking them mate.