"CAFE"

"Cafe" AI music video by Mark DK Berry.

WORKFLOWS & PROJECT DETAILS

🎥 Description

A woman walking through life, finds the solution to fix what she cannot. She isn't the first to figure it out.

Music: "Cafe" by Mark DK Berry – Available on Bandcamp
Date Video Published: 17th Jan 2025.

🎬 About the Project

This was the first AI Video I did with Comfyui, and so the other workflows are probably better as they improved over time. The Hunyuan text-to-video model had been out since 5th Dec 2024, so I found enough info to help me get going with it. I then set myself a limit of 5 days to see what I could create. I had spent 3 months working on "Fallen Angel" in UE5 and the result had been disappointing, so was aware of the need for restricting myself going in. If it worked out, I could always do another. It worked out.

⚠️ Key Challenges

AI generated max 2 seconds per prompt on my PC else it fell over, and prompts hit character limits around 350 tokens with that Hunyuan t2v model and didn't obey the requests very easily. While the results were clear, stretching 2 seconds into 8 using FFmpeg worked to buy time, but added blur and distortion.

Prompts worked best when kept general and simple, e.g., “hot female model in a red pencil dress walking away at an old English train station, realistic and cinematic, daytime.”

🔧 Workflows & Tools Used

The below zip file contains x1 Comfyui workflows:
- hunyuan_comfyui_workflow_for__cafe__music_video.json
Right-Click and "Save link as" to download the ZIP containing Comfyui json workflows

⏱️ Time & Energy Investment

Day 1 (and part of the night) was for main content, Day 2 for fixing ideas, Day 3 for tidying in DaVinci, and Day 4 for final edits and a go at color grading (likely overdone—sorry, colorists).

💻 Hardware

GPU: RTX 3060 (12GB VRAM)
RAM: 32GB
OS: Windows 10

All of it was done on a regular home PC.

🧰 Software Stack

ComfyUI (Hunyuan text-to-video model)
Ffmpeg to smooth interpolation and stretch time.
Davinci Resolve 19 – Final cut and colour grade

🎨 Loras Used & Trained

N/A

📺 Resolution & Rendering Details

512 x 416 resolution balanced quality with the model and my PC's capabilities at that time. Bigger sizes caused issues, and smaller ones lost clarity.

😵‍💫 Final Thoughts

As a first attempt I was blown away by what was possible. It had lots of issues, but I knew it was headed somewhere incredible, and I had jumped in right on time - nice and early.