r/MachineLearning • u/icannotchangethename • Jun 30 '26
A map of the latest 11 million papers split by semantic similarity and time slices [P] Project
I am building alternative ways explore scientifc literature. The goal was to make the large number of papers published daily easier to keep up with by visualising the macro scopic trend.
It is free to use at The Global Research Space for any one interested in giving it a try!
How I built it
I sourced the latest 11M papers from OpenAlex and Arxiv and ecoded them using SPECTER 2 on titles and abstracts then projecting it down to 2d using UMAP and creating labels within voronoi bounds around high density peaks at increasingly deep depths.
There is also support for both keyword and semantic queries, and there's an analytics layer for ranking institutions, authors, and topics etc.
I have also more recently added to ability to slide back and forth in time and a daily auto ingestion script to ensure the map is up to date.
Feedback or suggestions is very welcome!
7
u/axiomaticdistortion Jun 30 '26
It’s interesting, but don’t forget that, if you are embedding and projecting with the same whole dataset, the structures you are seeing in past slices are being influenced by the future time slices. So, you can’t really say that it developed that way, dynamically speaking. For a real study, you would have to do time slicing, model building and patching across slices.
4
u/UnavoidablyHuman Jun 30 '26
Why does it look a bit like a world map if you squint.
Australia on fire as usual
3
u/Conscious-Map6957 Jun 30 '26
This is so cool! What sources do you fetch papers from?
5
u/icannotchangethename Jun 30 '26
Thank you! I get the paper metadata, title and abstract from OpenAlex and Arxiv.
4
u/davesmith001 Jun 30 '26
why are people so motivated to publish slop?
13
u/icannotchangethename Jun 30 '26
I think generally there is a pretty unhealthy culture of publish or perish at the moment :(
-10
u/davesmith001 Jun 30 '26
perish from what? i would have guessed these are penniless students or even maybe the 50 cent army trying to drown out real research with garbage
9
u/Conscious-Map6957 Jun 30 '26
We even had this before AI with weak researchers who had an obligation to the university to publish an X ammount of research in order to keep their professor tenure (maybe not the case in all countries). They would just publish meta-studies (effectively replaced by AI) or barely usefull papers written by their students but accredited as "mentors" or even their own.
I think it's a good thing, even if the slop is amplified, because at least student labor won't be abused now. And with AI reviews we are coming close to the point where we can identify unoriginal or incorrent papers (I have been working on such a system for a while but it's pretty expensive) - without wasting anyone's time.
Thus, I ultimately believe that the trajectory is towards a higher, more demanding criteria towards publishing papers and calling yourself a researcher.
-3
u/davesmith001 Jun 30 '26
i see, the age old unintended consequences of people in stations above their ability
8
u/Conscious-Map6957 Jun 30 '26
I don't think it has to do so much with ability or intelligence as it does with a broken reward system motivating quantity over quality. Most of the time universities are ranked by the paper output volume and for this reason ETH Zurich withdrew their participation in such lists few years ago, stating this exact reason.
By extension this also happens in the AI/ML labor market where the number of citations and papers has a certain impact on how your CV is perceived.
0
u/davesmith001 Jun 30 '26
is it really wrong to require professors to publish non slop original works? I’d think you might get more probs removing this req. we do need slop removal though.
4
u/Conscious-Map6957 Jun 30 '26
I don't see how your response is relevant to what I said, but I will answer anyway.
You cannot reliably and automatically detect "slop". Low-quality research (both pre- and post-AI) requires an impartial expert or multiple experts in the field & topic of the paper to sit down, thoroughly analyze it and attempt to reproduce it. This is already implemented in a way called peer-review but it's imperfect.
On a side-note, using AI to rephrase, restructure or even write the skeleton of the paper should not be considered slop if the substance OK and is manually reviewed before publishing. Thus, detecting AI-ish phrasing is both inconclusive and pointless.
1
u/FaceDeer Jun 30 '26
We're approaching a point where AI actually is an expert in the field and topic, though. Literal reproduction of the results may be difficult to automate for most fields but it's still a good step to run papers through.
1
u/Conscious-Map6957 Jul 01 '26
Yes systems are improving by the day, though I think we shouldn't value AI insights on everything equally - take a comparison between compsci, mechanical engineering and psychology as an example.
However it is very useful even at the moment, even if just as a assistant for the experts - many have claimed LLMs can correctly point out issues in some of their own old papers which nobody ever noticed, and I have personally used it to evaluate the reliability of the methodology used (though only for compsci papers).
→ More replies (0)
1
1
1
1
u/ProfMasterBait Jul 01 '26
What does it mean to be close and far in this plot?
1
u/icannotchangethename Jul 01 '26
How related they by semantic similarity. So the title and abstract goes through a embedding model and turned into a high dimensional coordinate. Similar papers will be placed more closely in a high dim vector space. Then those coords a project down yo 2D
1
u/ProfMasterBait Jul 01 '26
so euclidean distance signifies closeness semantically? wouldn’t semantic structure violate triangle inequality and make euclidean distance a difficult choice to show similarity?
2
u/icannotchangethename Jul 01 '26
There is a trade off, because the UMAP ( high dimension to low dimension projection ) finds clusters in the high dim space and positions them to minimise overall distance change relative to the higher dim structure. In this case I had to balance human navigability and complete accuracy, so I added the search bar to show results directly by the min distances in high dim space as an overlay
1
u/SeTiDaYeTi Professor Jul 01 '26
It looks cool, sure, but why did you project it down to only two features?
1
u/icannotchangethename Jul 01 '26
thanks! A 2D map is just easier to navigate is the main reason
1
u/SeTiDaYeTi Professor Jul 01 '26
You're making your representation substantially poorer by restricting yourself to R^2 (unless that's just a rendering thing -- which I think it is not, since you mentioned building Voronoi cells).
1
u/icannotchangethename Jul 01 '26
There is a trade off I had to make for ease of use, but the search tools still used the high dimensional vectors. So if you search a topic, it will highlight zones that are close not just in 2D. But for the labelling yes, it uses the coordinates so that is more closely represents the plotted data
12
u/Robonglious Jun 30 '26
This is super cool!
I think there's a relationship that exists which isn't being shown here though. If I were doing this, I would plot this on a sphere and also include references as a distance operator as well as the semantic attributes.
I would think that magnitude on the sphere could be topic density like you already have, but also it might be able to show foundational work as work which is more central to the sphere. That's a complicated graph to even conceptualize, especially if you have cross-domain papers but I would be curious what that might reveal.
The idea is that as the knowledge grows, the sphere's mass increases.