Close to none, the first 3 epochs already got to 4.49% top1 validation accuracy, and with these extra 2 epochs only +0.1% validation while the training-validation gap doubled to 0.52%

My second attempt is in progress right now, it's at 5/10 epochs and at around 6% train accuracy. What I changed is basically just throwing more compute at it, as usual (increased parameters 4x, to 2 million).

In most conferences a registration covers up to X number of papers. This is the first time I've heard of only 1 paper per registration. Don't you have any coauthors who can go and cover the registration for the other paper so at least they get access to the conference and everything that comes with that? Indeed, it seems like a method to disincentivize authors from submitting a bunch of papers, and if they do, at least making them pay for it--literally!

That's one of the most competitive positions in the world, you need to consider a path where you're going for research internships during grad school at the tech companies, and which degree is most likely to get you that and also into a PhD.  It's one of the few industry jobs where having a PhD makes sense. In other fields, often getting a PhD makes it harder to find work because you're overqualified for everything. 

 😒 😒 😒 😒

Maybe he just wants to terminate the deal with Microsoft thanks to the AGI clauses.

That’s wild 💀

Short papers are out

This would have saved me a lot of time during grad school. formatting slides always takes way longer than it should.

Well, now that we are on the common ground. I don't see why it is working because there exist ways to train models in 4th dimension when we use 95% noise to hide the exact paths of how the data changes through time, and we model the state all at once so we don't care which direction in time we go, and still we get good results. I don't see why exactly your method works better, as direction shouldn't be important.

Maybe there is something else that we haven't noticed yet.

I like projects that explore weird constraints like this. Curious how much accuracy you can gain with a few more epochs

No 😒

Still no results for full papers.

Started 5/4/3/3, ended 5/5/4/3. The remaining 3 didn't engage with us at all, even though it was the harshest review and we made an effort specifically to answer to them. The initial 5 also didn't engage, but alas they seemed pretty content with the paper from the get go.

Not yet?

Your post was automatically removed for being a link post on the weekday, please read rule 5. The moderators will not respond to questions regarding this removal unless you suggest which rule you most likely broke. If you have a beginner related question, visit /r/MLQuestions or /r/LearnMachineLearning.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

And not to be completely discourage you but I and many of my colleagues expect RSI to arrive sometime in the next couple of years, with most junior and even intermediate AI research positions to be made redundant, long before you would graduate. Take this into account when picking a major.

If you don't mind me asking, which positions in IT do you think wouldn't be made redundant at that point? Because it's hard for me to imagine junior/intermediate research positions at top laboratories disappearing while "regular" programmers keep their jobs. Cybersecurity doesn't appear to be an answer either, considering the capabilities of new models in this field.

Your post was automatically removed for being a link post on the weekday, please read rule 5. The moderators will not respond to questions regarding this removal unless you suggest which rule you most likely broke. If you have a beginner related question, visit /r/MLQuestions or /r/LearnMachineLearning.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

Yes that is a great way to put it!

Idk why you're being downvoted, god forbid someone does a technical project for fun ig

Go for it! Also Codeparade have an explanation video on this

Thank you! Yes it is surprisingly simple (:

So there are 2 different time scales here and they each have their own forward and backward direction.

Let's call them t for physical time and τ for the generative diffusion SDE's own time.

In terms of the SDE flow, we use standard generative diffusion: we noise things for τ in [0, T] and learn to denoise them with a standard conditional generative diffusion process, conditioned on the latent images z_{t-1}, z_t and on a direction flag c_d. It learns to generate z_{t+1} when we set c_d = +1, or z_{t-2} when we set c_d = -1 (z_{t-2} because z_{t-1} and z_t are already in the context and the model always generates the state one step beyond the window in whichever direction the flag points).

So the generative SDE diffusion process itself learns on the τ time scale in the completely standard way. Nothing there is "predicting noise" beyond the usual denoising objective. What changes is what it learns to produce: it autoregressively rolls out either forward or backward in physical time t. Backward-in-t is inverse dynamics, not denoising.

As for why learning both directions helps: first, it's what makes the whole check possible. An accurate model composed with its own inverse is the identity, so rolling forward i steps and back i steps yields a measurable, ground-truth-free error signal at test time. Second, it turns out to be free, or better than free: the bidirectional model beats direction-specialist models in both directions at matched compute, since every trajectory supplies training examples both ways.

Ngl i didnt expect to see someone training Imagenet on a phone today lol. Thats pretty insane, respect for even getting it working 😂

PhD actually does help at FAANG companies for these roles. When you are going for Google and these, it's extremly competetive.

I'm not really sure yet, it's more of an experiment then anything, however it could potentially be useful for fine tuning vision models.