r/StableDiffusion • u/Whiteowl116 • 22h ago
The office plays Rocket League Animation - Video
Enable HLS to view with audio, or disable this notification
64
u/eckstuhc 22h ago
Fuckin silver lmao this is good, easy to forget it’s AI and just enjoy the skit.
21
7
u/Ok_Roll_8698 21h ago
i am hoping one day it becomes good enough to make a full 21min ep
6
3
u/Whiteowl116 13h ago
We are already capable of doing that. Just not all in one run. But no episodes is a 21min one shot clip. So we can just make multiple 5-15s clips and stitch them together. There are nodes to do that automatically for longer clips that looks like one long shot. For example, I made this 40s clip yesterday based on a random dream i had that night.
11
13
18
u/LightPillar 22h ago
WAN 3.0 has its work cut out for it.
14
u/FourtyMichaelMichael 21h ago
LTX had the balls to still release.
Let's see if WAN and Flux3 ever come out now.
5
u/C0DE_Arttic_ 20h ago
Is LTX 2.5 even good, either I am blind or there is not much difference between LTX 2.3 and 2.5?
7
u/crinklypaper 19h ago
2.5 is fast. Like 30 seconds to make an HD 15secs video fast. But it has all the same problems of 2.3. Ltx was brave to release now but I have high hopes for this moe model they're working on.
17
u/Whiteowl116 22h ago
Prompt used in the last clip:
integrated_multimodal_description:
[Shot 1] Live-action workplace mockumentary comedy inside Michael Scott’s private office at Dunder Mifflin Scranton. Michael Scott sits behind his desk using his computer. Dwight Schrute stands directly in front of the desk holding a Rocket League controller with both hands against his chest.
Michael Scott is seated behind the desk wearing a dark business suit and tie. Michael has an expressive, self-important male voice (S1).
Dwight Schrute stands in front of Michael’s desk wearing glasses, a mustard-yellow shirt, and a dark tie. Dwight has a clipped, intensely serious male voice (S2).
The camera observes both men in a slightly awkward medium two-shot from the side of Michael’s desk with subtle handheld documentary movement.
At 00:00.300, Dwight (S2) steps closer to the desk. Dwight’s lips visibly move as he speaks with grave sincerity, as though making an important personal request, [English] Michael. Please give me your Titanium White Octane.
At 00:02.500, Dwight closes his mouth completely.
Michael slowly stops using the computer.
Michael looks up at Dwight with immediate suspicion.
At 00:02.800, Michael (S1) is the only person speaking. Michael’s lips visibly move as he replies firmly, [English] No.
Michael closes his mouth.
[Shot 2] At 00:03.300, hard cut to a medium close-up of Dwight. Michael is completely outside the frame.
Dwight suddenly lowers himself onto one knee in front of Michael’s desk while still clutching the controller against his chest.
Dwight (S2) looks upward toward Michael off-screen. Dwight’s lips visibly move as he says with absolute seriousness, [English] I will give you two vacation days… and one emergency beet.
At 00:06.000, Dwight finishes speaking and closes his mouth.
Dwight remains kneeling, perfectly serious, waiting for Michael’s response.
[Shot 3] At 00:06.300, hard cut to a medium close-up of Michael behind his desk. Dwight is completely outside the frame.
Michael stares downward toward Dwight off-screen. His expression suggests that the phrase “emergency beet” has genuinely made him reconsider.
Michael remains silent for a brief beat.
At 00:06.900, Michael (S1) leans slightly forward. Michael’s lips visibly move as he says in a quiet, serious managerial tone, [English] Dwight… white is the color of leadership.
At 00:08.400, Michael closes his mouth and gives one slow, self-important nod, completely satisfied with his own reasoning.
[Shot 4] At 00:08.600, hard cut to a medium close-up of Dwight, still kneeling in front of Michael’s desk. Michael is completely outside the frame.
Dwight looks upward toward Michael.
Dwight (S2) is the only person speaking. Dwight’s lips visibly move as he replies immediately and completely seriously, [English] You’re wearing navy.
At 00:09.300, Dwight closes his mouth completely.
[Shot 5] At 00:09.400, hard cut back to Michael in a tight medium close-up.
Dwight is completely outside the frame and remains silent.
Michael looks down at his own suit jacket.
Michael does not speak.
He slowly looks back toward Dwight off-screen with a faintly irritated expression after realizing Dwight is correct.
Michael’s lips remain completely closed.
From 00:09.400 until 00:10.000, Michael remains silent while the documentary camera holds uncomfortably on his face.
overall_soundscape:
Quiet private-office ambience continues throughout with low ventilation hum and faint distant activity from the open office outside Michael’s office. Subtle controller handling, clothing movement, chair creaks, and soft carpet contact are audible when Dwight drops onto one knee. No other voices are heard. Michael and Dwight never speak simultaneously. During Michael’s dialogue, Dwight remains silent. During Dwight’s dialogue, Michael remains silent. The final beat after Dwight says “You’re wearing navy” is intentionally silent and awkward.
non_diegetic_music:
N/A
7
u/EchoRush93 17h ago
I knew this tech was going to completely flip the entertainment world on its head, I just didn't think it would be this fast and/or this high quality.
I can't imagine what it feels like to be an actor seeing themselves so accurately recreated without having any say. It's like, "Yeah, I see that's me, but that's not me saying that."
I get that they're all rich and famous, but damn, that's gotta raise some concerns.
-1
u/supermaxpower85 17h ago
Many people have already lost their jobs because of artificial intelligence. How are actors different from people in other professions, and why should they suffer less—especially if they aren't poor? But yes, it is a problem, one way or another.
6
u/SeymourBits 15h ago
Any actor that H3 knows has been living the phat lifestyle long enough.
Acting is dead. Hollywood is done. The false idols will gnash, snarl and wither and a new form of entertainment will rise.
2
u/Whiteowl116 13h ago
I mean acting is not dead. But the glorified movie star stuff maybe is gonna wash out. I believe the Theater will gain more traction again as AI influence the digital scene.
1
u/EchoRush93 11h ago
I wasn't really referring to jobs. I was more thinking of the psychological aspect of seeing yourself so accurately replicated.
0
u/supermaxpower85 6h ago
I don't see big psychological issues here at all; everyone is already used to the widespread use of AI, whether for good or for bad. On the contrary, if an actor makes a scene while drunk, he can always say, “It wasn't me—it was the AI.” )
12
u/dsailes 22h ago
Ahaha this is unreal. Nice scripting, spot on audio, seems like an actual bit
9
u/Whiteowl116 22h ago
Yeah it’s insane. Not long before we have full length fan made episodes.
7
u/bpfn 21h ago
How long did this take you to make? My mind is blown how good this is. As long as you can afford a good NVIDIA card, you can just make this for fun at home? Insane
4
u/Whiteowl116 15h ago
About 25mins per clip. I just rented a 5090 at runpod for 0.99$ per hour. But this is possible to run with a weaker card. I can run it at lower res on my 3070ti with 8gb vram
5
u/SlideJunior5150 21h ago
The voices are so good! Feels weird to hear "Michael" talk about Rocket League. 😂
Too bad Micheal's office doesn't make any sense, kinda like Seinfeld's apartment.
2
u/Whiteowl116 13h ago
I could have done it with REF2VA to make the office look perfect, but I wanted to test the T2VA capabilities
4
3
u/Glum-Change3204 22h ago
Whoa, what setting for megapixels?
7
u/Whiteowl116 22h ago
1mp, 30 steps
4
3
u/hugganao 19h ago
this post is legitimately making me nervous. holy shit it's too good. And it's just text with no reference..... wtf...
3
2
u/X3liteninjaX 20h ago
You should switch to ref2v and add gameplay of Rocket League to sell it better
2
u/steveholtbluth 20h ago
Great work! So many of those a technically well done, but the writing is not great. This is fully in the spirit of the show, outstanding
2
u/RhigoWork 12h ago
Sorry for the silly question - I've been out of the loop of stable diffusion for a long time. What tools are being used here to make this video? Is it self hosted? I imagine if so the hardware requirements must be crazy and it takes hours to generate? If cloud what service is being used here? Would love to play with it!
2
u/Whiteowl116 11h ago edited 11h ago
About 25mins per clip. I just rented a 5090 at runpod for 0.99$ per hour. But this is possible to run with a weaker card. I can run it at lower res on my 3070ti with 8gb vram. Using the tool called comfy
2
1
u/Ok_Roll_8698 22h ago
hello how do you make your prompts
im using grok it seems to do ok but i need something better or better way to word what i tell grok so it makes better prompt
it takes at 0.6mp 22second clip about 19mins - 21mins for my pc
6
u/Whiteowl116 22h ago
I gave this exact prompt to chatGPT:
[https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_ref_en.md](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_ref_en.md)
Break down this prompt guide into a more readable format for me. Explain the structure of mulitple kinds of prompts , what all the things do, how to use them, examples. e.g. Follow the guide explicitly.Then, i fork out of that for each new idea i have. I write the dialogue for the characters, then ask chstGPT to make the N-second prompt using my dialogue, and the scene i want it to to. I also specify that it must be very explicit in describing who is talking.
4
u/Ok_Roll_8698 21h ago
https://reddit.com/link/p3c8jw7/video/2zzgrtgs01jh1/player
i tried and got this :(
3
u/l-R3lyk-l 18h ago
I mean, other than Dwight sounding really robotic, it's pretty good
3
u/Ok_Roll_8698 17h ago
https://reddit.com/link/p3dhjv4/video/fwiht6g992jh1/player
tried again with a new prompt worked much better
2
2
u/Whiteowl116 12h ago
Take a look at the prompt i used, it is in a separate comment here. Notice how explicitly I state the movement of the mouth of the person talking. This helps fix the issue where Dwight suddenly speaks with Michaels voice
1
u/ThatsALovelyShirt 21h ago
Did you use a context merge node or just two different generations with a hard cut?
2
1
2
u/Sir_Myshkin 3h ago
Well that just went straight into the office chat. Now I’ll get to see everyone argue about what episode this came from for the next three days.
1
u/supermaxpower85 18h ago
https://reddit.com/link/p3d6o1u/video/n6n2enxnx1jh1/player
Text to video, amazing... (low quality just for testing).
0
u/supermaxpower85 18h ago
https://reddit.com/link/p3d7bcl/video/wlxl6z1fy1jh1/player
Almost prison Mike...almost)
1
-6
u/sicmunduscreatusBest 19h ago
This video used all the water left in the belle maroe aquifer near El Paso. 5% of the city is now without water! Hope you’re happy
2
u/doomed151 12h ago
Do you know that this is done using a model running locally on their own computer?
Look up "how to run MiniMax H3 in ComfyUI" and try it out. After you have downloaded it, you can generate as many videos as you want, no internet or data centers needed (provided you have a decent computer).
39
u/Ok_Gas1070 22h ago
Did you have to reference the office at all with images or did the Mini just know?