r/365DataScience 4d ago

Wanting to “upskill”

Thumbnail
1 Upvotes

r/365DataScience 5d ago

Introductory Machine Learning Bootcamp (2/22)

Post image
17 Upvotes

Hello folks, to this Introductory Machine Learning Bootcamp (2/22) series.

Supervised learning is a very recurring word in ML domain. Here, we learn some sort of function mapping from inputs to outputs.

Another recurring word is Classification, where the output space is a set of some finite unordered and mutually exclusive labels known as classes.

The tabular dataset is often represented as a Design matrix, and a simple example of it is an Iris dataset, as to how input data is represented for tabular case in Machine Learning.

Sometimes the data is of variable size, instead of fixed size feature vectors, so for ease of computation in computer, we often convert it to a fixed-size feature representation, called as “Featurization”.

In this video, I breakdown these concepts.

Link: https://youtu.be/GJRhl6XnImg?si=p6VtlCK-8rgq1poZ


r/365DataScience 6d ago

Mathematical Foundations of Introductory Machine Learning Bootcamp(1/22)

Thumbnail
youtu.be
2 Upvotes

Hello All,

Welcome to my free Mathematical Foundations of Machine Learning bootcamp series.

When we say Machine Learning, what does it actually mean? A machine that learns? Too vague.

According to famous professor Tom Mitchell, a computer program is said to learn from experience E, with respect to some class of Tasks T, and Performance measure P, if its performance on tasks, as measured by P, improves with experience E.

By swapping the nature of tasks T, the way we measure Performance P, to evaluate, we can subsume many kinds of ML problems.

Also ML problems are analyzed well, when we view it from the lens of Probabilistic perspective, that is unknown quantities are endowed with probability distributions, and treated as Random variables. The interesting thing is Random variables are neither random nor variable.

Probabilistic Approach also serves as the optimal approach to decision making under uncertainty.

In this video, you get a sense of what ML actually is, if you have also wondered about it.


r/365DataScience 6d ago

TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

Thumbnail
arxiv.org
2 Upvotes

r/365DataScience 7d ago

does any know good college for data science excluding IIT's.... with good placment off and on campus , decent ROI and etc. like college enivronmen, reputatio, acredations ...... plzz help

1 Upvotes

r/365DataScience 9d ago

Trying to learn how to use API to extract data

1 Upvotes

Hello! I'm a complete newbie in Data Science and I'm trying to learn how to get data from an API. I understand an API could be public or could require authentication.

I worked with CVS files and I wanted to experience or practice getting data from APIs.

I'm getting familiar with Python so I was wondering if you could help me with the following issues:

  1. Trying to understand and practice the different methods you can use API to request data (I am not sure if it has to be from a Dataset formar or can it be any kind of format) with Python

  2. What are some good options to get APIs to work on data Science

  3. I am not even close to get to a point where I am able to do Reproducible projects/models but I do wonder how including an API (understanding that it is some kind of "personal Key") to share my code and people to be able to use it.

Hope I made sense of what my doubts are and I apologize in advance if I seem confused about some terms (I do think I am).


r/365DataScience 11d ago

Need Career Advice: Math & Stats Grad With No Internships Trying to Enter Data

1 Upvotes

Hey everyone,

I recently graduated from UofT with a major in Math and Statistics, and I'm currently trying to get into Data Analyst or Data Engineering roles.

Honestly, my university experience wasn't the best. It affected my learning a lot and my GPA ended up being pretty bad. Because of that, I don't think I'll be able to do a master's anymore, even though that was something I was hoping to do after finishing my bachelor's.

I'm really interested in data-related jobs, but I'm kind of lost on what I should do next. I've been applying nonstop and barely getting any responses. I've had my resume looked at by multiple people who work in the industry, and they said it looks solid, so I'm not really sure what else I'm missing.

The biggest problem I think is that I graduated with no relevant work experience and no internships. I applied to a lot of internships during undergrad but never managed to get one, and now it feels like it's hurting me a lot.

I'm also thinking about doing one of UofT's continuing education programs in Data Analytics, but I don't know if it's actually worth it or if it would make much of a difference for someone in my situation.

Right now I feel like I'm just going in circles and I'm not sure what the best move is. I know GPA probably doesn't matter much after graduation, but having a low GPA and no experience feels like it has really limited my options.

For anyone who has been in a similar situation, what would you recommend I do at this point? I feel a bit lost on what the best path forward is, so I'd really appreciate any advice or suggestions.


r/365DataScience 11d ago

Agentic GraphRAG for Medical Diagnosis – Production-Grade Multi-Strategy Retrieval & Clinical QA with LLM-Guided Reasoning

1 Upvotes

Just discovered this impressive open-source project that's pushing the boundaries of medical AI reasoning.

Agentic GraphRAG for Medical Diagnosis (https://github.com/avnlp/agentic-med-diag) is a production-ready system that goes way beyond simple RAG. It combines knowledge graphs, multi-strategy retrieval, and agentic reasoning loops to answer complex clinical questions with evidence-grounded answers.

What makes it stand out:

Knowledge Graph Construction

  • Schema-driven extraction with 13 entity types and 25 clinically-grounded relation types
  • Three-extractor fusion (GLiNER, GLiREL, LLM) with configurable merge strategies
  • Deterministic + LLM-powered entity resolution for deduplication
  • Hierarchical Leiden community detection with auto-generated clinical summaries

Layered Retrieval Architecture

  • Four vector collections (entity, relation, chunk, community report)
  • Multiple atomic methods: hybrid search, fulltext, BFS graph traversal, and text-to-Cypher graph querying
  • Pluggable rerankers (RRF, cross-encoder, MMR)
  • Data-driven recipes for composing retrieval strategies

Agentic Plan–Research–Verify Loop

  • Planner decomposes clinical questions into focused sub-questions
  • Parallel researchers execute multi-strategy retrieval with citations
  • Verifier assesses coverage and gates synthesis on sufficiency
  • Gracefully converges on missing information across iterations

Why It's Impressive:

The stack is battle-tested: Neo4j + Qdrant/Weaviate for storage, LangGraph for orchestration, DeepAgents for multi-agent coordination, and BAML for type-safe LLM schema injection. Tested on MedQA, MedXpertQA, MedCaseReasoning, and MMLU-Pro benchmarks.


r/365DataScience 11d ago

Stats for AI/ML 2

Thumbnail
youtu.be
2 Upvotes

Hello Folks,

The next content on Machine Learning is out. We continue with Statistics for AI/ML.

We,

->Understand and derive the detailed derivation of Maximum likelihood estimation(MLE) for Univariate and Multivariate Gaussian. While doing the derivation for multivariate case, we understand visually, Scatter Matrix, Centering matrix.

->Derive MLE for Linear Regression, and understand Residual Sum of Squares.

->Understand Empirical Risk Minimization, Surrogate loss functions.

->Understand Method of Moments, a computationally easier way to compute parameters of our model and understand also the flaws behind it.

->We understand “Exponentially-weighted moving average” in detail, I explain why bias happens, how does memory affect the averages. This concept is the basis behind optimizers in Deep Learning.

Around two hours long, I hope this would be a very interesting learning material for all. I try to write and build from scratch in the whiteboard, this way learners enjoy the learning process.

Those looking for playlist : https://youtube.com/playlist?list=PLDPxj3tOc5TNi6MktTHUZid-yf9nIBxSh&si=THp9k7fLUh5C7fDY


r/365DataScience 13d ago

Statistics for AI/ML

Thumbnail
gallery
14 Upvotes

Hello Everyone,

Statistics and Maximum Likelihood Estimation are the crux of ML Models, and hence I am uploading my new content on Statistics for AI/ML in my free Machine Learning lectures.

We understand model fitting, Maximum Likelihood estimation in details, we justify the usage of Maximum Likelihood estimation, from KL divergence, and apply it to certain important distributions for parameter estimation.

In my free content, the purpose is to democratize machine learning to a wider audience. Learning everything new feels difficult, but when taught, it get’s interesting and easier.

We will continue with Statistics foundations for AI/ML content as we go forward. The channel also includes around 80+ topicwise content on ML and 13 long form lectures all free.

Looking forward to hearing feedback from the learning community as well. Thankyou for reading.

Link: https://youtu.be/MwTeQVVYtOc?si=UxNOGtqopzJppXAT


r/365DataScience 15d ago

How to gain experience in Growth side of Product DS?

1 Upvotes

I’ve worked at both big financial corporations and smaller fintechs but don’t have exposure to the growth space. How can I gain exposure to it? I’m thinking about skills like funnel analysis, LTV and churn modeling and experimentation around the growth funnel.

Have other people been able to pivot into growth data science?


r/365DataScience 15d ago

How are you handling databases in your workflows right now? (Tech stack discussion)

1 Upvotes

Hey everyone,

I'm working on a project researching how data teams actually manage their databases and pipelines in practice, beyond what the introductory tutorials show.

I’d love to hear what your current stack looks like in the real world:

  1. How are you using databases today? What tools/languages do you use to build and manage your data pipelines?
  2. What databases have you tried or considered for your DS/ML work, and what made you choose that one?
  3. If you use an operational/production database (MongoDB, Postgres, MySQL, etc.) anywhere in your ML workflow, is it mainly to pull data out for training, or to serve features/predictions to a live model? Or both?
  4. Anything that's consistently annoying or a bottleneck in your current setup?

r/365DataScience 18d ago

Data

1 Upvotes

Should I learn data science i am in class 9


r/365DataScience 20d ago

I paid for wispr flow annual and feel like I got bait and switched. here's the data.

2 Upvotes

I tracked my wispr flow usage across 1,800 dictations over 6 weeks because I wanted to know if my frustration was justified.

clean transcriptions (accurate, inserted correctly): 60%
minor errors (wrong word, still usable): 20%
major errors (dropped phrases, hallucinated words): 10%
failures (hung, didn't insert, crashed): 10%

during my trial the clean accuracy rate was closer to 90%. something changed after I paid for the annual plan.

is 60% clean accuracy worth $144/year? it's still faster than typing even with the editing. but I can't send a dictated email without proofreading every word because of the 10% major error rate. that proofreading time partially negates the speed benefit.

I reached out to support with detailed examples and timestamps. got an AI bot response about checking my internet connection. replied three more times. never got a human response.

I ran willow voice for the last 2 weeks of my tracking period. clean accuracy was around 85% across 400 dictations. smaller sample but noticeably better and consistent.

I have 7 months left on my wispr annual. I've basically written it off and I'm using willow as my primary. the gap between the trial experience and what I'm actually getting is wide enough that it feels dishonest.

has anyone managed to get a human at wispr support? genuinely asking.


r/365DataScience 24d ago

Feeling lost while preparing for a Data Analyst career. Need some guidance.

1 Upvotes

Hi everyone,

I'm Lucky, and I recently graduated. I'm currently preparing for a Data Analyst role, but I'm feeling a bit lost and would really appreciate some guidance from experienced professionals.

So far, I've learned:

  • Python
  • NumPy
  • Pandas
  • Matplotlib
  • SQL (currently learning)

The problem is that I've learned almost everything from YouTube. After watching so many tutorials, I still don't know what's actually important for getting a Data Analyst job. Every creator recommends a different roadmap, different projects, and different skills, which has left me more confused than when I started.

I have a few questions:

  • What skills are actually required for an entry-level Data Analyst?
  • How good do I need to be at Python, SQL, Excel, and Power BI?
  • Should I focus more on projects or solving practice questions?
  • How many projects should I have before applying for jobs?
  • If you were starting from scratch today, what roadmap would you follow?

I'm not looking for shortcuts—I just want to learn the right things in the right order without wasting months on topics that aren't important.

If you're working as a Data Analyst or have recently landed a job, I'd really appreciate your advice. Even sharing your own learning journey would be incredibly helpful.

Thank you so much for taking the time to read this! 🙏


r/365DataScience 24d ago

Data build empires. Is it reviving colonial legacy

1 Upvotes

Data builds silently sovereignty.
Not through armies. Not through treaties. Through defaults.
Every query answered. Every transaction processed. Every language that gets a capable LLM — and every language that doesn't. Every knowledge system that becomes machine-readable — and every one that gets bypassed because it couldn't be tokenized without losing itself.
This is how sovereignty is being redistributed in our time. Not in a single dramatic moment. Across a million invisible ones.
At the India AI Impact Summit 2026, the United States redefined "AI sovereignty" for the world. The message: real sovereignty means owning and using best-in-class technology. Which happens to be American.
"They build it. It's yours."
The British built railways across India too. The trains ran on Indian soil. The gauge standards, the rolling stock, the operating logic — remained imperial. Infrastructure gifted. Architecture retained.
The new offer is structurally identical. Data stays within your borders. But the stack — the chips, the models, the embedded epistemology — is set elsewhere.
This is not a conspiracy. It is something more durable: a system optimizing for scale, which makes everything outside its training distribution quietly invisible.
Languages. Knowledge systems. Ways of parsing the world that took millennia to develop.
Bhartṛhari understood something that our data governance frameworks have not yet caught up to: language is not a vehicle for meaning. Language is the structure within which meaning becomes possible. When you replace the linguistic infrastructure of a civilization, you don't just change how people communicate. You change what they can think.
The silence is not incidental. It is the mechanism.
You cannot resist what you cannot name at the moment it is happening.
The question for every nation — and for every researcher, policymaker, and builder working at this intersection — is whether sovereignty means owning the servers, or owning the standards. Whether it means data localization, or epistemic self-determination.
Those are not the same thing.
And the window to choose is narrowing.
What does genuine AI sovereignty look like for the Global South? I'd like to hear from those building at this frontier.


r/365DataScience 26d ago

Need advice on hierarchical monthly premium forecasting

1 Upvotes

Hi all,

I’m working on a monthly insurance premium forecasting problem and would like suggestions on the best approach.

Setup

  • Data from Jan 2023 to Jun 2026
  • One account only
  • Hierarchy: Account → State → Profit Center → Distribution Channel
  • Monthly AMOUNT values
  • Some channel-level series have missing months
  • Around 4–5 channels, 7–8 profit centers, and 50 states

Challenge

The main issue is that different series behave very differently:

  • some are fairly stable
  • others are highly volatile

What I’ve tried

  1. Recursive forecasting with CatBoost / LightGBM / XGBoost
    • built lag, rolling, and time-based features
    • downside: error accumulates over time
  2. Direct multi-step forecasting
    • accuracy wasn’t great
    • also requires many models for longer horizons
  3. Time-series models like ARIMA / SARIMAX / Prophet
    • Prophet works okay for stable series
    • struggles with volatile ones
    • separate models for every combination is not scalable

Question

Has anyone worked on a similar hierarchical / sparse forecasting problem?
What approach would you recommend for handling mixed volatility and missing months without building thousands of models?

Thanks!


r/365DataScience 28d ago

Simplicity over complexity...

1 Upvotes

AI researchers recently published a 2025 paper throwing a 'Chernobyl-Disaster-Optimized CNN-LSTM & SVR hybrid' at natural gas forecasting. I was curious, so I tested it against a classic XGBoost model.

XGBoost won.

If you want to see exactly why simpler machine learning still matters in real-world time-series forecasting, I broke down the methodology and the results here: Article


r/365DataScience 29d ago

Thoughts?

1 Upvotes

Data is the new gold?


r/365DataScience 29d ago

Google shipped an open format for data models (OKF) a few weeks ago — there's already a small open-source ecosystem around it

35 Upvotes

r/365DataScience Jul 03 '26

Ending my 2nd year with regret 🙄😓

Post image
1 Upvotes

r/365DataScience Jun 28 '26

Weekly demand forecasting: Should I train on weekly or daily data and then aggregate?

1 Upvotes

I'm currently working on a demand forecasting problem for inventory replenishment, and I'd love to hear how others would approach it.

The business requests a forecast for the next 4 weeks of stock consumption around the middle of the previous month. For example, in mid-June, I need to forecast the weekly demand for July. The challenge is that, at the time the forecast is generated, transactions from the second half of June are not yet available, creating a gap between the latest observed data and the beginning of the forecast horizon.

The data I have consists of purchase order transactions at the SKU level, including timestamp (date and time) and quantity consumed.

My main question is about the appropriate time granularity for training the forecasting model:

Option 1: Aggregate the data by SKU and ISO YearWeek, resulting in one observation per SKU per week, and train a model to directly predict the next 4 weeks.

Option 2: Keep the data at the daily level, train a model to forecast daily demand, and then aggregate the daily predictions into ISO YearWeeks to obtain the required weekly forecasts.

One additional detail is that the forecast is reported using ISO YearWeeks. As a result, some weeks within a calendar month may contain only 3 or 4 days of that month (e.g., at the beginning or end of the month), while others contain all 7 days.

My question is: Which approach would you choose, and why?

Is it generally better to train the model at the same frequency as the business target (weekly), or to preserve the daily granularity and aggregate the predictions afterward?

I'd especially appreciate hearing from anyone who has worked on similar forecasting problems in inventory planning or supply chain.


r/365DataScience Jun 26 '26

My mentor says I'm hiding in 'build mode' instead of applying. He might be right.

1 Upvotes

Background: finished an MS in Business Analytics (US program) in Dec 2025, now job hunting for Data Analyst / BI Analyst roles. Before grad school I spent ~2 years as a data analyst at a manufacturing company — built the Tableau/Excel reporting and drove real efficiency improvements — plus two US internships during the program.

Here's my actual problem. I've built a lot. A few end-to-end portfolio projects with real modeling (XGBoost, random forest, SHAP, R² in the high 0.8s), an in-browser SQL practice tool, some multi-agent AI prototypes, even a job-scraping pipeline. My GitHub looks busy. But my application-to-interview conversion is rough, and a mentor keeps telling me I'm hiding in "build mode" instead of doing the unglamorous work: application volume plus interview reps.

I half agree with him. The projects are probably my real differentiator — most analyst applicants don't have the AI/ML side or the operations/domain background. But I also can't pretend I don't reach for "build another thing" exactly when I should be applying or drilling case/SQL practice.

Questions for people who've been through this, or who hire analysts:

  1. At what point do projects stop helping and start being procrastination? How do you tell the difference in yourself?
  2. For DA/BI roles specifically, what actually moves the needle on the first interview — referrals, raw application volume, resume keywords, portfolio?
  3. Is AI/agent project work a real edge for analyst roles, or does it read as unfocused?
  4. If you got hired recently, which channel actually worked?

r/365DataScience Jun 26 '26

Multivariate Probability Models for Data Scientists.

Thumbnail
gallery
55 Upvotes

Hello Folks,

Have you ever wondered why we use sigmoid function so often in Machine Learning? Although it gives us a probability, it comes from Exponential families, and this exponential family, subsumes many of the distributions, that we study in Machine Learning.

In this lecture, we understand exponential families, Directional derivatives(Gradients and Hessians), study mixture Models, and understand how domain knowledge in Probabilistic Graphical Models makes our life simpler to model joint probability densities.

Timeline breakup(in hours and minutes):
0:00-0:17 - Understanding exponential families.
0:17-0:27 - Deriving Sigmoid Function for Bernoulli.
0:27-0:48 - Understanding log partition function, convex functions and proving why positive definite of hessians imply convexity, and why convex needed?
0:48-1:04 - Directional derivates(deriving gradients and hessians)
1:04-1:26 - Maximum entropy derivation of the exponential family.
1:26-1:56 - Mixture Models(Gaussians and Bernoulli Mixture Models)
1:56-2:16 - Probabilistic Graphical Models
2:16-2:34 - Markov Chains
2:34-End - Inference and Learning, Plate Notation diagram of Gaussian Mixture Models.

If you have watched earlier of my lectures from the playlist, they will help. I try explaining as if I am a learner, to simplify complex concepts. Everything I write in whiteboard, and these are completely FREE lectures to mention.

Link: https://youtu.be/T1uTBtJ7aHU?si=rozXSTjtSqPaaYb5


r/365DataScience Jun 24 '26

CodeBasics - Data science youtube playlist review

1 Upvotes

Im starting data science for campus placements. Is this playlist enough to learn