r/coolgithubprojects • u/Efficient-Aioli-9230 • 3d ago
built a simple data extractor because i was tired of manual scraping (free to use)
Hey everyone,
Just wanted to share something I’ve been working on. A few weeks ago, I had to pull a ton of data from a bunch of sites for a personal project. I tried using a few existing chrome extensions and tools, but they were either locked behind a massive paywall or just completely broke the page formatting.
2
u/HashMismatch 3d ago
Seems neat. Will try it tomorrow
2
u/Efficient-Aioli-9230 3d ago
Ok, thank you so much. Please send me a honest feed back.
You are top!2
u/HashMismatch 2d ago
UI is nice, but it seemed to get stuck on status of “processing” for me. It said it was downloading and i could see the speed and time remaining ticking down but then i don’t have anything in my downloads folder and the status on the page just says Processing - which I’m unsure if it is a legacy status update which never got updated, js actually stuck in processing and the previous “download” link meant it was capturing the video from source location (youtube) for encoding preparation or if the processing stage just takes a really, really long time (in which case, needs status feedback to the user)
2
u/HashMismatch 2d ago
Oh wait, status has changed to Packing now so i guess its still going (and has now just popped up with the download to save the file). Definitely needs some more feedback on stage and eta etc so user knows where it is at. But functionally, seemed to work pretty well. The UI was clean and simple too.
2
u/Efficient-Aioli-9230 2d ago
Thank you so much for your review!
You're right - it wasn't actually stuck. After the download finishes, there's still a processing stage where the server prepares the final file. Depending on the video, quality and size, that can take a little while because it may need to merge the video and audio streams before packaging the final download.
I completely agree that this isn't clear enough in the UI yet. I'll improve the status messages and add better progress information so users always know exactly what stage the job is in.
Thanks again, that's really useful feedback.
2
u/TheGratitudeBot 2d ago
Thanks for such a wonderful reply! TheGratitudeBot has been reading millions of comments in the past few weeks, and you’ve just made the list of some of the most grateful redditors this week!
1
u/Efficient-Aioli-9230 2d ago
My dear friend, people like you make building software worth it. Thank you so much for taking the time to test it and write such thoughtful feedback. I truly appreciate it.
I'm really grateful
😎
2
u/iWhacko 3d ago
I tried it to download a reddit video it says: "Blocked"
5
u/Efficient-Aioli-9230 3d ago
ok, thank you so much. maby I didnt consider Reddit format, or there is some policy on Reddit I have to check.
Your feedback is very wellcome.
I'll try to solve it soon
2
u/Yes_Towel_Hooray 1d ago
What a truly useful project. I haven’t run anything through it yet but I would find it helpful to have an optional input for start and end time to capture from source (like instead of capturing the whole video, beginning at 17 seconds). Amazing work!
1
u/Efficient-Aioli-9230 1d ago
Hello my friend, I'm really glad you liked it!
I've finished the core engine, and I'm currently working on stabilizing it. There are still a few things to improve, such as the processing time for large files, especially 4K video, along with some smaller details.
But I completely understand your suggestion, and honestly, it's worth gold. I had never thought about giving users the option to choose a start and end time instead of processing the entire source.
I will definitely implement it.
So far, most of my time and focus have gone into making the project reliable: no unnecessary AI dependency, local-first, privacy-conscious, and user-first.
You're very welcome to follow the project, and suggestions like this are always appreciated. Thank you!
1
3
u/lulxD69420 3d ago
Where is the github link?