This week I wanted to try to add some more motion. I wasn’t quite happy with the 2 1/2D motion that Cuebric gave me and tried putting some images through Runway to see if I could get something more cinematic. Overall I’m impressed with what I got out of Runway Gen2. I got some “usable” shots (don’t look too close) using both archive and AI generated shots. The archive shots I coloured using Photoshops Neural filter.
One issue, which was surprising, given that these softwares all are targeting filmmakers, was how hard it was to get an OTS (over the shoulder shot). It is a widely used term that most people in cinema get instantly but the AI doesn’t seem to understand this term the same way. I kept getting profile shots like the ones below - all generated in Cuebric - except the last one - that one is from Runway (I like the composition and lighting better in Cuebric but the runway one looks more like a period image)
All nice images, but not “over the shoulder.” I am finding that I am missing the control I get when drawing and would love to be able to start with a rough sketch - I eventually tried with ComfyUI and the image to image generation by drawing an image to start with. I used their own example which is drawn very much like the image below using different colours for different parts of the body - which gave me a very strange result. (The short version of the prompt is over the shoulder of Chinese boy doing calligraphy - in 1932)
Here is my photoshop drawing
…and the somewhat scary result
So, I am learning my drawing needs to be better but I still get very strange hands and I cannot seem to get the OTS shot I drew
ComfyUI - image to image
with the image it got better, but still not what I had drawn (or prompted)- and there are some strange extra hands.
I’m also finding funny things happening when for example giving the AI a picture of the historical Bund in Shanghai. When you run it through the AI generators you get modern artifacts such as the Shanghai World Financial Center and the Oriental Pearl TV tower - both buildings that are definitely not from before WWII. I guess changing the CFG would help this
