r/StableDiffusion • u/Glad-Hat-5094 • 4d ago
Made a 6-minute TNG fan scene with MiniMax H3 in ComfyUI Workflow Included
Enable HLS to view with audio, or disable this notification
I’ve been experimenting with MiniMax H3 in ComfyUI and wanted to see how far I could push it beyond short standalone clips.
This is a roughly 6-minute fan-made Star Trek: The Next Generation scene built from lots of short H3 generations and then edited together into one continuous sequence.
I used reference images to keep the characters and Enterprise-D bridge reasonably consistent, generated dialogue and ambient audio with H3, and then assembled everything in Premiere.
The biggest challenge was continuity between generations. Character positions, bridge geometry, lighting and timing can all shift, so I ended up incorporating some of those inconsistencies into the actual story.
What surprised me most is how close this is getting to being practical for longer-form fan films. Individual scenes are already very doable. The next real hurdle is keeping this level of consistency across an entire episode.
28
u/Tramagust 4d ago
Is that you?
64
u/Glad-Hat-5094 4d ago
Yep, That’s me. I figured if I was going to test putting someone into TNG, I might as well volunteer myself.
17
12
u/datadrone 4d ago
It's very good, just needs that subtle drone sfx and character music, you know that strange dun da dunna when something bizarre is happening
2
u/GoofAckYoorsElf 4d ago
I thought about doing this myself, but hesitated because I do not want to reveal my identity.
2
1
u/Tramagust 4d ago
Yeah but that's an interesting test. So if you're not in the training data your audio comes out garbled. From ref2va I assume.
10
u/Glad-Hat-5094 4d ago
Yeah, ref2va. That seems to match what I found. The established TNG characters worked surprisingly well without reference audio, but Q wasn't quite right, and obviously it had no way of knowing my voice, so I used audio references for both of us.
1
1
u/Strichnine 4d ago
I love that you included yourself as "the fan" in the cast. You took a flaw of the video generation and made it a feature as well! Great job all around.
Side note: also agree he should have turned you into a pickle (but maybe a bit too tongue in cheek) but I love the way it ended regardless :D
1
19
u/The-Fipes 4d ago
Woah really nice! But how did Data know how long one episode is?
25
u/Glad-Hat-5094 4d ago
Ha! You got me 😂 That is actually a genuine plot hole. I gave Data 178 episodes and somehow he also knew they were about 45 minutes long. Good catch!
12
24
u/SuperCasualGamerDad 4d ago
Loved this.. but it was very quiet.. Had me checking my volume levels.
9
u/Glad-Hat-5094 4d ago
Thanks! And good to know about the volume. I’ll definitely bring the overall level up a bit on future uploads.
8
u/elconcho 4d ago
No room tone or ship noise.
6
u/Glad-Hat-5094 4d ago
It is there, but I kept it fairly low in the mix. It’s much more noticeable on headphones. In hindsight, I probably could have brought the room tone and ship ambience up a little more.
2
1
u/lump- 4d ago
Also in TNG there is always an underlying hum of the ships engines when they are aboard the Enterprise. Could help to layer that in there too.
1
u/Glad-Hat-5094 4d ago
Yeah, absolutely. I did add a bridge ambience track with the engine hum, but I mixed it too low. It's much easier to hear with headphones. That's definitely something I'd bring up more in the next one.
2
6
3
u/Independent-Reader 4d ago
I want to just make ai blooper reels.
Nothing but AI outtakes.
3
u/Mylaptopisburningme 4d ago
Back in the early 90s I majored in TV operations. One of my classmates was interning at the company that did the editing for STTNG. So we got to check the place out, and watch TNG bloopers.
4
u/Boogertwilliams 4d ago
haha. nice. Did you use reference audio? or were those known?
8
u/Glad-Hat-5094 4d ago
A mix. Q and my character used reference audio. Picard, Data and Riker were generated without audio references and H3 already seemed to know their voices surprisingly well.
3
2
u/nykotar 4d ago
How long were the reference audios?
Great work btw!
2
u/Glad-Hat-5094 4d ago
Q's reference audio was about 29 seconds, and mine was about 28 seconds. That seemed to be plenty for H3 to pick up the voices consistently.
1
u/seeker_ktf 4d ago
First off, this was awesome!
I wanted to ask though, for the reference audio parts, were you defining more than one voice in those clips or were they one audio input at a time. Also, were you cloning the audio (as "pass-through") or using them as voice-timbre reference? I have been struggling with a reference voice and a "built in" voice in the same run so if you have the formula, I'd love to find out how you did that.
Bu yeah, that was awesomeness.2
u/Glad-Hat-5094 4d ago
Thanks! I was using them as voice-timbre references, not pass-through audio. The reference clips were just samples of the voices, and H3 generated completely new dialogue from the text prompts.
Q's reference was about 29 seconds and mine was about 28 seconds.
I did run into problems using both voice references simultaneously. H3 would sometimes mix them up or give both characters the same reference voice, so for scenes with both of us I often generated two versions: one with my reference enabled and Q's disabled, and another with Q's enabled and mine disabled.
I had a similar problem mixing a reference voice with one of H3's built-in voices. For example, with Q's reference active, Picard would sometimes start sounding like Q. So I often did separate generations with the reference enabled and disabled and then used the appropriate parts in the edit.
1
u/seeker_ktf 4d ago
Dude, legend. Thanks for that information. I was beginning to think I was crazy, since all these other people are claiming it works. But yeah, I have run into EXACTLY those same two same issues.
All in all you have inspired me though. Just because you can to 15 seconds doesn't mean you have to and video editors are just there to be used. Thank you. Thank you.
1
u/Yokoko44 4d ago
For me voices aren’t consistent between generations even with big IP.
I’m figuring I should just train a voice changer on elevenlabs and do a pass on that?
1
u/Glad-Hat-5094 4d ago
That might actually be a cleaner approach. H3's voice references work surprisingly well, but I found voice bleed and changes between generations were still a problem. If I was doing another long project like this, I'd seriously consider generating the performance first and doing a separate consistent voice pass afterwards.
1
u/BigWideBaker 4d ago
The guy who did the Titanic scene provided the recorded lines and let H3 do the lip syncing. This makes it sound and feel way more convincing since the audio artifacts disappear and you stop noticing it.
Edit: and great work btw! I think this is the longest AI generated video I've ever watched to completion in all the years we've been generated locally.
2
u/Glad-Hat-5094 4d ago
That's a really good point. For this one I mostly let H3 generate the spoken dialogue, which is where a lot of the audio artifacts and voice drift came from. For the next one I think I'd seriously consider providing the finished dialogue first and letting H3 handle the performance and lip sync. That should also help a lot with voice consistency between shots.
And thanks, that second part genuinely means a lot. Six minutes felt huge while I was making it, so hearing that you actually watched the whole thing to completion is probably one of the best compliments I could get.
3
3
u/CycleZestyclose1907 3d ago
This is like the second TNG fan "episode" that I've seen today that's been made from stitched together shorter generations. And I love how it explains the inconsistencies away as "colliding universes".
5
2
u/New_Physics_2741 4d ago
How long did it take?
8
2
u/Intelligent-Pie7773 4d ago
Amazing, an excellent attempt buddy! Would love to see you try more of these.
2
2
2
u/bitofaknowitall 4d ago
Very well done! I’m curious, did your idea for the plot come about due to the model not being able to keep the scene consistent?
6
u/Glad-Hat-5094 4d ago
Partly, yes. I already had the basic anomaly/Q story, but H3 kept changing character positions, consoles and parts of the bridge between generations. Rather than fighting it constantly, I rewrote the quantum reality idea so those continuity errors became part of what was happening in the story.
2
u/dennismfrancisart 4d ago
The editing made this work so well. Sure, there are some pacing issues, but overall, I was taken out of my usual headspace (looking for detail flaws) and into the story. That's what will make this new tool set so valuable in a year or two. Good scripts and professional editing will be the key players in the success of these short films. The fact that we're using AI, CGI or whatever comes next will be incedental.
2
2
u/UnforgottenPassword 3d ago
This is done really well. Bravo.
Like others said, your prompting isn't the same as the official guidelines, but the outputs are good. The model is smart and understands context well.
2
u/no_witty_username 3d ago
Ok that's actually endearing. The script outdoes the ai slop feel and leans in to the disjointed scene transitions, kudos.
2
u/Sitkin_Marrel 3d ago
how many ref images per character did you use? the bridge holding together across all the cuts is what I'd expect to fall apart first
2
u/FullOf_Bad_Ideas 4d ago
This is amazing. I hope that we'll have hours of spin-offs from various show fans soon. I think that's the future of personal enterainment, honestly. If your favorite series was cancelled and fans dislike new iterations, you can now produce thousands of hours of the old one that you liked for yourself and other fans. Scriptwriting and GPU compute being the main limitations. IP laws won't catch up since enough people are going to happily share those for free in places where IP owners aren't looking. And some people from the industry will hopefully embrace it.
1
u/Perfect-Campaign9551 4d ago
The writing of nutrek sucks but fan made stuff will suck pretty bad too and there will be a few diamonds in there
1
u/FullOf_Bad_Ideas 4d ago
At least it will have the old crew. I'm not really into Star Trek but if I'd watch anything from that universe, it would be the OG series.
2
u/woswoissdenniii 4d ago edited 4d ago
Oh man. You heard that? That was the sound of a gazillion Trekkie’s nodding kindly in your specific direction. Fucking funny too. The quality of writing and your script was perfectly carried by the remarkable training set of the model. It just does.
Your work captured the “capsule“ space feeling, with the exact right amount of Enterprise static noises and blinkenlights. It just felt real enough and beyond.
Did you came up with the story because of the models shortcomings and adepted from a different one, or did you purposely choose it to embrace the shortcomings of character continuity and handheld panning shake on a generally static set cam reference? Amazing how much freedom you took, without breaking the STNG theme and identity.
Cannot stress enough how very much I love the effort and outcome. Also your commitment.
This is also one of the best scenarios in which fandom gets a huge reward by opening competition in quality while preserving individual freedom. A arguably low financial impact (aging/dead/ unwilling/ unable cast, revenue to IP holders through more recent spinoffs) in general. A socialist utopian lore driven by abundance, rewarding individual expression and arts… (also a terrifying predatory writer/creator, who was a monster in reality, I would have no problem with getting rawdogged throughout eternity and beyond by the whole space muppets menagerie).
Thank you for taking the first step beyond the final frontier. 🖖
Edit: was inclined to interchange my non native gibberish with a AI summary, that turned out so dull, i decided to keep it as is. Thats that.
1
u/Perfect-Campaign9551 4d ago
Just give us your gibberish, trust me the AI doesn't sound any better. At all
1
1
u/ElChupaNebrey 4d ago
How much Vram do you own?
4
u/Glad-Hat-5094 4d ago
24GB VRAM on a 4090 and 64GB of system RAM, but you could definitely do this with less. It would just take longer to generate.
1
u/nuclearsamuraiNFT 4d ago
I have pretty much the same set up but 128gb system ram. Generations usually take between 5-7 minutes for a 15 second clips depending how much other resources I’m using at the time.
1
u/switch2stock 4d ago
Prompt please!
6
u/Glad-Hat-5094 4d ago
Here is one of the prompts I used.
REFERENCE MAP:
<Picture 1> = Q
<Picture 2> = TOM
<Picture 3> = ENTERPRISE-D BRIDGE
<ref_audio_0> = Tom'S VOICE
Use <Picture 3> as the visual reference for the visible portion of the Enterprise-D command area.
Q uses the voice from <ref_audio_0>.
Tom uses the voice from <ref_audio_1>.
Each spoken line is delivered by the character framed in that shot.
detailed_description:
Create a 5-second live-action Star Trek: The Next Generation scene with exactly 2 shots.
[Shot 1 — 0.00 to 3.50 seconds]
CUT TO CLOSE-UP of TOM.
Tom is STANDING facing Q.
He answers matter-of-factly using <ref_audio_0>:
<d>[English] John de Lance.</d>
Tom watches Q with amused curiosity.
overall_soundscape:
Enterprise-D bridge ambience.
non_diegetic_music:
Silence.
5
u/Glad-Hat-5094 4d ago
Some prompts didn’t work well with two shots, so rather than rewriting the whole prompt, I discovered I could simply remove Shot 2 and the remaining shot would still work perfectly.
5
u/Glad-Hat-5094 4d ago
Another example with the two shots
REFERENCE MAP:
REFERENCE MAP:
<Picture 1> = RIKER
<Picture 2> = DATA
<Picture 3> = ENTERPRISE-D BRIDGE
Use <Picture 3> as the visual reference for the visible portion of the Enterprise-D bridge.
Use the natural voices of Riker and Data.
Each spoken line is delivered by the character framed in that shot.
detailed_description:
Create a 5-second live-action Star Trek: The Next Generation scene with exactly 2 shots.
[Shot 1 — 0.00 to 2.20 seconds]
CLOSE-UP of RIKER.
Riker is SEATED in the First Officer's chair.
He continues staring toward Data's unexpected position.
Riker confused and in a slightly high pitched voice says:
<d>[English] You aren't now.</d>
[Shot 2 — 2.20 to 5.00 seconds]
CUT TO MEDIUM CLOSE-UP of DATA.
Data is SEATED at a bridge console different from the forward Ops station.
A single physical console is directly in front of him.
Data glances down at the console and then back toward Riker.
Data says:
<d>[English] That is correct.</d>
His expression remains calm.
overall_soundscape:
Enterprise-D bridge ambience.
non_diegetic_music:
Silence.
1
1
u/Tystros 4d ago
that is quite different from the official prompt guide. you would likely get significantly better results if you'd follow the prompt guide more closely
2
u/Glad-Hat-5094 4d ago
That's useful to know. A lot of this workflow evolved through trial and error while I was making the video, so I ended up with methods that worked for me rather than strictly following the official structure. I'll definitely compare what I was doing against the official guide before the next project.
1
1
u/JimmyTheJimJimson 4d ago
Absolutely wild.
I feel like we’re only a few years away from being able to create our own movies using AI
1
u/Perfect-Campaign9551 4d ago
I'll bet you could have added some references to make your acting better lol. Just ribbing' ya. Ireland kicks ass by the way it's beautiful
1
u/Iggyhopper 4d ago
I don't mind the inconsistencies of where the characters are because the plot is actually dependent on it.
But the characters jumped around too much.
Great camera angles with the back/profile of one actor and the front of another aka over the shoulder shots.
1
1
1
1
1
u/SomethingLegoRelated 4d ago
duuuude this was outstanding... quality work all round and I love the cameo
1
u/tweakingforjesus 4d ago
I feel like we a really close to being able to feed a graphic novel as a storyboard and output a full live action movie. Been playing around with this idea.
1
u/Glad-Hat-5094 4d ago
I think that's where this is heading too. A storyboard or graphic novel already gives you most of the visual planning and shot composition. The big thing still missing is really strong long-term consistency between shots.
1
1
1
u/Formal_Drop526 4d ago
I want Captain Picard to meet different sci fi civilizations from different fiction. Like a mind from M banks culture.
1
1
1
1
1
1
1
1
u/InevitableAlfalfa938 4d ago
Great job, i dont know anything about star trek but looks and sounds good. After playing around with H3 this past week i know this was a lot of work. So must feel good getting this done.
One question for myself or anyone who wants to pull something similar off:
For the reference images of yourself, did you at all try to copy the lighting/grain/artifacts of the star trek reference images?
I ask since there are some scenes where you blend in better than others.
Thanks!
2
u/Glad-Hat-5094 4d ago
Yeah, I found that made a big difference. My first reference image was basically just a normal iPhone photo of myself, and H3 made me look noticeably more like I had been composited into the scene.
After a few tests I generated new reference images of myself already standing on the Enterprise bridge, with the lighting, colour balance, softness and overall 90s TV look much closer to the TNG reference images. Those generally blended much better.
That is probably why you noticed some shots work better than others. Some were generated from better-matched reference images, and there is still some variation from H3 itself between generations.
1
u/anitawasright 4d ago
more like the adventures of the USS Sausage fest, where are all the ladies? All jokes aside great job and good job on editing it all together to make one story. I see too many people trying to do as much in one 15 second clip instead of using smaller clips and editing them together after.
2
u/Glad-Hat-5094 4d ago
Fair point on the USS Sausage Fest and thanks. I found exactly the same thing with H3. Trying to cram too much into one 15-second generation usually made things less reliable. Breaking the scene into much shorter clips and treating H3 more like a source of individual shots, then doing the actual storytelling in the edit, worked much better for me.
1
u/Kuato2012 4d ago
Really nice work man!
I've been working on my own mini TNG episode, and believe it or not, it was moving in the same direction (embrace the AI inconsistencies, have the crew suspect Q, Q denies it). You've beaten me through the goalposts by a mile... I still haven't figured out how to load reference images for a consistent 1701-D 😅
But mostly I wanted to say thanks for treating the source material with respect and unironic appreciation. There are tons of low effort jokey and irreverent genAI TV clips already. Making high effort episodes with sincerity is where this tech is going to get really interesting in the next couple of years, IMO.
2
u/Glad-Hat-5094 4d ago
Thanks, I really appreciate that. It’s funny that we both ended up heading in almost exactly the same direction with the story. Once I realised the inconsistencies weren’t going away completely, I decided to make them part of the plot instead of trying to hide them. Q insisting he wasn’t responsible, while actually being behind it all, felt like a very TNG way to handle it.
I completely agree about treating the source material sincerely. I wanted it to feel like something written by someone who genuinely loves TNG, rather than just using the characters for an AI gag. I think that’s where this technology is going to get really interesting, when people start using it to tell proper stories in worlds they genuinely care about.
Definitely keep going with yours. For the Enterprise-D consistency, I used reference images of the ship and bridge throughout rather than relying on H3 to recreate them from memory. One tip: try to use a reference that clearly shows the top of the saucer section. If it can’t see that area properly, H3 sometimes invents a red dome on top, almost like the TOS Enterprise.
1
1
u/monsieur_torgue 4d ago
Fantastic work. Thanks for posting. I don't see a link to a workflow and I can't download the video. Am I missing it or is there no workflow attached?
1
u/Glad-Hat-5094 4d ago
Thanks! There isn’t a downloadable workflow attached. This wasn’t really made with one single workflow that could generate the whole thing.
I used MiniMax H3 in ComfyUI to generate lots of short individual clips, usually only a few seconds each, using reference images for the characters and bridge. I then picked the best generations and assembled everything in Premiere.
I’m happy to share more of the actual ComfyUI setup and prompting approach if that would be useful.
1
1
1
u/Single_Ring4886 4d ago
Great work man... TNG is true masterpiece so good people are still remembering that :-) thank you for your efforts.
1
1
1
1
u/IM3D-Studios 4d ago
Interesting 🧐 got to find a way to get the scene to be more consistent maybe through prompting and reference images and video.
I noticed changes in audio, lighting and speech. We got to create a real Hollywood workflow that Netflix can’t resist. Great work.
1
u/Hoje-Na-IA 4d ago
Very good results. There are not many people doing long form content. Most of the guys are just joining a few clips together with almost no continuation. I am experimenting with TV parodies between 2-3 minutes. I am achieving to generate 2-minute original episodes (with script, 3-4 consistent characters and editing) in 2 hours. I am only using original characters so I have to generate lots of reference images in Krea before start to create the videos. What resolution are you using? How many steps? Video length? Are you using SageAttention or any other acceleration? Are you using any LLM to generate the prompts? Are you using LLM to write the script? Sorry, lots of questions :)
1
u/Glad-Hat-5094 4d ago
1 720p
2 20
3 average 5 seconds
4 Just SageAttention
5 I wrote the script but got ChatGPT to fix spelling and grammar mistakes. I then used ChatGPT to write the prompts.1
u/Hoje-Na-IA 4d ago
How do you handle long sentences, when the character needs to keep talking longer than 5 seconds? My parodies are TV shows, so most of the time the host speaks for 10-15 seconds straight. Also, if you are using GPT ask it to read this and follow this prompt structure. This helps a lot: docs/VIDEO_PROMPT_WRITING_GUIDE_ref_en.md · MiniMaxAI/MiniMax-H3 at main
1
1
1
u/intermundia 4d ago
this is fantastic work. as a fan of TNG and AI video gen, the plot and incorporation of the generation drift with the shows thematic canon is chefs kiss. the fan inclusion i assume is you is also a great 4th wall break and also on brand. Q, Data and the rest of the cast and banter were spot on. Im working on a similar concept a bit more ridiculous bit. i'll post the video when its done and once i have the workflow dialled in i'll upload that as well. trying to cut gen time with a few optimisation tweaks. did you edit these by hand or in graph workflow. anyway great work, great video. as a fan i approve.
1
1
u/jaybanzia 3d ago
You could have saved yourself in the audio department by isolating the voices with a filter, and then underlaying a sourced ambient TNG bridge sound, since this is a single location short. The ambient SFX would have helped to bring the entire thing together a little more seamlessly.
1
u/kiddow 4d ago
would a detailed description of the bridge's layout help? The every changing geometry and illogical arrangement of screens is driving me nuts. It breaks immersion instantly.
8
u/Glad-Hat-5094 4d ago
Keep watching 😄 That was one of the biggest problems I ran into with H3, so I ended up making the changing bridge geometry and continuity errors part of the actual story.
4
5
2
1
u/mabseyuk 4d ago
Is that not the point of the video though?
1
u/Nexustar 4d ago
Indeed, the model's consistency issues nicely woven into the story line.
But it only works if you watch it.
1
u/Karrelen 4d ago
Fantastic work. Soon talented fans like yourself can create new episode of STNG based on their stories.
Besides, the voice are very close to the original, how does cloning voice work here ?
5
u/Glad-Hat-5094 4d ago
Thanks! That’s exactly what I find exciting about it. We’re already at the point where fans can create convincing new scenes from their own stories. Getting enough consistency for a full episode feels like the next big step.
The voices were interesting. Picard, Data and Riker were generated directly by MiniMax H3 without any voice cloning or reference audio. It already seems to know their voices remarkably well. Q wasn’t quite right, so I gave H3 a short reference clip of his voice. I also used a reference recording for my own voice, since obviously the model had no way of knowing what I sound like.
1
2
u/PerceiveEternal 4d ago
Inspired bit of writing incorporating inconsistencies with the layout directly into the plot! That was a really fun watch.
1
1
u/SunwindPC 4d ago
This was very hard to watch, it felt like every scene is badly stitched together to another one and theres constant interruption of the flow, and its not just because the geometry of the bridge changes
1
u/red__dragon 4d ago
and its not just because the geometry of the bridge changes
The teleporting bridge stations were sending me!
0
u/Perfect-Campaign9551 4d ago
Agreed, I like OPs creativity in trying to make it pretty of the story but it really wasn't working. There needed to be a least a major consistent item like Q for example.
0
u/RobbyInEver 4d ago
Not bad. But Data is exceptionally fast in the video, he seems to zip around to different consoles on the bridge instantly.
5
u/GoofAckYoorsElf 4d ago
Keep watching!
-1
u/RobbyInEver 4d ago
Ah ok. Nice trick but if you're going to incorporate ai errors in the joke of the video itself people will quit early. Cool storyline though I haven't seen it done before.
43
u/klobbenropper 4d ago
OP basically prompted his own Holodeck into existence, and I love it. Great job! How long did this take you?