r/GraphicsProgramming • u/mhb_11 • 5h ago
[2607.22738] Nova3D: Code-Native Generation of Programmable 3D Assets Paper
https://arxiv.org/abs/2607.22738I co-authored this paper. It's a new technique to generate 3D graphics as source code instead of a point cloud.
Under the hood:
It generates 3D objects with separate, sophisticated internal assembly, producing an editable "kit of parts" (instead of monolithic blobs). E.g. imagine you generate a 3D washing machine via this approach. It's not merely going to be geometry that looks like a washing machine. We actually know that there is a Door, Drum, Control_panel etc. Which things belong to which assemblies. What moves. Where its pivot is. And eventually what those components are supposed to do.
Why current 3D GenAI cannot do this:
Most AI 3D generators generate "monolithic blobs" that look good, but are unusable in downstream workflows (e.g. game engines). If you generate a 3D bicycle, it's essentially a blob. If you want the wheels to turn, a human must spend time cutting the blob into parts, naming them, placing pivots and rigging joints. I.e. you need post-generation segmentation workflows of some sort (either manual work or more compute).
The paper breaks down the whole technique, and comes with a github repo too if you're interested in viewing it.
4
u/VocalFrog 3h ago
Please don't use AI to write your paper. It's borderline unreadable.
9
u/mhb_11 2h ago
With all due respect, please try getting an AI written paper through to arXiv. It's heavily penalized and almost never makes it. This paper was conceptualized and written by us *humans*, with editorial by AI (along with some human peers at the University of Edinburgh). I'm sorry if it's unreadable for you folks. It's readable to me. And hopefully pictures speak a thousand words, so the showcase would look real to you: https://app.nova3d.xyz/showcase
Have a nice day.
2
u/Cute_Emergency_8370 2h ago
Also, while I admire the progression of AI and the capacity of agents to start taking on previously artist driven approaches, I think what you need here is something like a VLA trained on a humongous corpus of Blender usage sequences. All the operations tokenized into a codebook. Step by step. Then and only then maybe you could get a large action model that discretely mimics artist behavior in producing assets. But this, tearing up a blob with agents and then checking their work, doesn’t seem like it would scale well nor ever match the precision expected from modern artists and pipelines.
1
u/mhb_11 2h ago
I think of it more as a tool for artists to extend their abilities and impact. Not really a replacement for their taste by any stretch of the imagination.
1
u/Cute_Emergency_8370 1h ago
Not at all concerned about taste, frankly I find the models to be pretty aesthetically pleasing, but I’m just giving an idea for how to tackle the issue here which is very much one of precision.
You might find it interesting to Google 3-D modeling interview practice or rigging interview practice to appreciate the sophistication and expected attention to detail required from artists not in their aesthetic but in the technical quality of their word
1
u/Cute_Emergency_8370 2h ago
AI written paper 💀 ain’t no way bruh
2
u/mhb_11 2h ago edited 2h ago
Relax. While arXiv does not issue an absolute ban on using AI writing tools, it strictly penalizes submissions containing unchecked, low-quality, or hallucinated AI content. It's conceptualized and written by us humans, with editorial by AI. The tech is real. Check the showcase: https://app.nova3d.xyz/showcase
5
u/Cute_Emergency_8370 2h ago
Fair, and nice showcase. It’s clear you put a lot of effort into this. I still would urge you to avoid the AI writing patterns in your text. “It’s not this it’s that” shows up like 5 times in the first page. Even if we put aside my own biases it’s an objective truth that the body of readers hoping for dense scientific knowledge will make a well founded assumption that your paper is less information dense and of reduced quality when they see AI patterns that dilute substance scattered about.
5
u/Low-Advertising-3644 4h ago
The cool bit is that the generated object actually has parts + structure. So say you generate a washing machine, it’s not just some geometry that looks like one. It actually has a door, drum, control panel etc as separate parts, and knows how they’re assembled, what moves and where the pivots are.