I’ll show how to get Blender to EBSynth Workflow
Let me show you something. Right, listen up. This is what you’ve got to do:
You need to have Blender, EBSynth, and Stable Diffusion installed. Make sure to have EBSynth Utility installed with a quadruple one, and create a projects folder with underscores, not spaces. If you use spaces, EBSynth Utility will not work correctly. Create a new folder within your projects folder. Call it whatever you want. Within this folder, create the following subfolders:
- Video Frame
- Video Key
- Video Mask
- Image to Image Key
- Image to Image Normal
Now, let’s move to Blender and start rendering the image sequence, for image prompts get them from da prompts.com. I’m rendering at 4K, but you can do this at 1080p. Ensure that in your output properties, you have RGBA selected. The output needs to be in your projects folder, and you need to select “Video Frame” as your output for the image sequence. In EV, make sure in your render properties, you have transparent selected. Render a frame, add a zero to your output, and hit Ctrl+F12 to render out the frames. Check if the frames are being rendered into the Video Frame folder.
Now, move on to rendering the video masks. In the render properties, change from the render engine Eevee to Workbench. Select “Flat” and “Single” for lighting. Set this to Y. In color management, change “View Transform” from Filmic to Standard. Also, set the world properties viewport display color to black. Change back from transparent, select “Video Masks,” and render them out.
Next, render the normal maps. Change from Flat to Mat Cap and select the normal map version. Turn on transparency, change the output to “Image to Image Normal,” and render the frames. Now that you have all the frames needed, create a video for EBSynth Utility. Start another Blender file for video editing, add an image sequence, select your Video Frame in the projects folder, ensure it’s in order, and set the render size. Render the video and save it in your projects folder.
Once the video is rendered, move to EBSynth Utility. Add your project directory and original movie path. Choose the keyframes or let EBSynth Utility choose based on settings. Copy and paste the first and last frames into the keyframes section. Now, move to Quadruple One and start rendering frames for image to image key in Stable Diffusion. Select your first video key and add the corresponding normal map. Adjust settings as needed and render the frames.
Upscale the keyframes back to 4K, open them in Photoshop, and mask out the background using the Video Mask. Save the masked images in the projects folder under “Image to Image Key.” Set up the EBS file for EBSynth Utility, select stage five, rename keyframes, and generate EBS file. Open EBSynth, select the file, and click run all to render out the frames.
Export the frames to After Effects, set the frame rate to 30 fps, and add a background layer. Render out an EV pass and Mat Cap for the background. Add these to your After Effects file. If needed, add a shadow pass and render it out separately. Precompose the EBSynth sequence in After Effects and apply adjustments as required, such as brightness, contrast, soft matte, and matte choker. Consider adding a shadow effect using blending modes like hard light or multiply.