Keypose estimation

[rezkanas]

Hello,

Thank you for your valuable contribution. I'm eager to test it out.

I'm planning to utilize it for training on images of multiple people. However, I'm currently exploring the feasibility of initiating this process solely through a textual prompt. The key hurdle is the key pose estimation. Do you have any suggestions or recommendations in this regard for how to get a key pose estimation from a prompt?

@yxgeee @xinntao @yeliudev @guyuchao @ARCer

[guyuchao]

The keypoint is not obtained from text prompt. You need to create multiple character pose images by PoseMaker or pose generator by sd-webui extension.

[rezkanas]

it would great also to define the specific revision for the modules required for your module. . now it is accelerate
diffusers, transformers (with no revision)

[rezkanas]

@guyuchao @yxgeee @xinntao @yeliudev
I get the model now to work... inference is fine for one character but when fusing them... it looks like the model does not recognize characters/ tokens at all. what parameters do I need to finetune?

I am playing around with optimize_textenc_iters [700, 1000, 500] and optimize_unet_iters [50, 70, 100] for fusing only 2 EDLORA.. and I would like to try it on more characters.

what do you advise? do you have some emprical ideas of how many steps required to fuse 2/3/4 EDLORA?