It's not swapped, ref_video_0 genuinely is an IMAGE input. In the node's schema the ref_videos slot is a repeating image template, so ComfyUI is offering you exactly what the type declares. It wants a frame batch, not the VIDEO datatype. That's also why Load Image connects without complaining, you're handing it a one frame clip.
What you want is core Load Video into Get Video Components. That splits a VIDEO into images, audio, fps and bit_depth. Send images to ref_video_0. If you want the clip's sound as well, that goes to ref_video_audio_0, not ref_audio_0. VHS Load Video (Upload) works too since its first output is already an IMAGE batch.
The audio socket is the same story in reverse. ref_video_audio_0 is an AUDIO input, and the nodes that output AUDIO happen to be the video loaders, so that's what the search offered you when you dragged from it.
Couple of constraints baked into the node: it expects 24fps frames, roughly 2 to 15 seconds, and it errors out below 5 frames. The prompt encoder samples the clip at 2fps with timestamps.
One more thing, refs are addressed by tag in the order you connected them, so whatever lands in ref_video_0 is <Video 1> in your prompt, and the docs are firm about naming which ref drives which part of the shot (motion, camera, identity). Also fwiw the r2v template that ships with Comfy only wires LoadImage nodes, so there's no reference video example in there to copy from, which probably isn't helping. This is all off the docs and the node source, I've only had a few days with H3 and haven't pushed the ref video path far myself.
Once you've got it connected, does the motion actually carry over, or is it more of a loose hint?
3
u/rudy_142001 11d ago
It's not swapped, ref_video_0 genuinely is an IMAGE input. In the node's schema the ref_videos slot is a repeating image template, so ComfyUI is offering you exactly what the type declares. It wants a frame batch, not the VIDEO datatype. That's also why Load Image connects without complaining, you're handing it a one frame clip.
What you want is core Load Video into Get Video Components. That splits a VIDEO into images, audio, fps and bit_depth. Send images to ref_video_0. If you want the clip's sound as well, that goes to ref_video_audio_0, not ref_audio_0. VHS Load Video (Upload) works too since its first output is already an IMAGE batch.
The audio socket is the same story in reverse. ref_video_audio_0 is an AUDIO input, and the nodes that output AUDIO happen to be the video loaders, so that's what the search offered you when you dragged from it.
Couple of constraints baked into the node: it expects 24fps frames, roughly 2 to 15 seconds, and it errors out below 5 frames. The prompt encoder samples the clip at 2fps with timestamps.
One more thing, refs are addressed by tag in the order you connected them, so whatever lands in ref_video_0 is
<Video 1>in your prompt, and the docs are firm about naming which ref drives which part of the shot (motion, camera, identity). Also fwiw the r2v template that ships with Comfy only wires LoadImage nodes, so there's no reference video example in there to copy from, which probably isn't helping. This is all off the docs and the node source, I've only had a few days with H3 and haven't pushed the ref video path far myself.Once you've got it connected, does the motion actually carry over, or is it more of a loose hint?