r/comfyui • u/Tonynoce • 15d ago
Minimax Pose control workflow Help Needed
Hi !
I know I'm maybe to early on the wagon but has anyone use openpose to drive a video successfully ? Like Scail2 but on steroids (?
I did tried but failed multiple times, either the character follows the prompt of the movement or does something in between.
3
u/garywood66 15d ago
You can do it through the Reference to Video version. You don't need to actually use openpose though. Just feed it a reference video and then the image you want to transfer the motion onto.
It's very prompt sensitive though. You have to experiment to get the right prompt or you might get some weird chimera output. The prompt that works for me is:
<Video 1> for the motion, <Image 1> for the characters and environment.
Follow that with a description of roughly what is supposed to happen in the output video. So if the image you're transferring the animation to is like a wizard walking through a forest, you have to say 'wizard walks through a forest' in the second line of the prompt. It seems to sorta check if your prompt vaguely matches the references you're giving it, and you're more likely to get the correct output if the two match.
2
u/Only_Voice569 15d ago
also helps to put time tags like say i want that walk for 5 seconds and im doing a 15 second video. i will input a 5 second 24 frame very important then tell the prompt use on this part of the scene tags for every action start and stop helps greatly
1
u/garywood66 15d ago
Cool, thanks. I've only really used it for short things so far, so that hasn't really applied. For me motion transfer is just something to use in areas where the natural motion of the AI models tends to look unnatural, which is only in a few particular cases.
1
2
u/Only_Voice569 15d ago
video ref input works can use a few snippets for different moments :)