← sources

I had Claude direct a full music video for $37.88 of fal credits (and fix its own tooling along the way)

source: reddit (reddit:ClaudeAI)published 2026-09-30 15:18ingested 2026-10-01 02:38

retained text

**Video:** https://youtu.be/XQgSZGIOleg

On Sunday a tune popped into my head in the car. After a day of iterating in Suno (64 versions, lyrics rewritten by hand), I pointed Claude (Opus 5.5, High effort) at my open-source app, [PortOS](https://github.com/atomantic/PortOS). I asked it to make a real music video through the app's Music Video autopilot, and to fix anything in the pipeline that got in its way.

**What it cost**

- fal.ai: $37.88 (Minimax H3 for lip-sync and image-to-video)
- Claude: about 10 hours of runtime, 5% of a weekly 20x plan
- Images: Codex image gen for the character sheet, sets and keyframes

**How it worked**

1. It designed the singer and the sets from my mood board, then stopped for my sign-off.
2. It mapped the beats and lyrics: which lines get lip-sync, where the cuts and glitch hits land.
3. It built an HTML/canvas animatic first (every frame is a pure function of song time) and used it as the storyboard.
4. It generated keyframes, then lip-sync and image-to-video clips through fal.
5. The final cut is that same HTML composition with the real takes dropped in: VHS/CRT treatment, a lab HUD, kinetic type.

**What it learned (now built into the autopilot)**

- Feed the lip-sync model the isolated vocal stem, not the full mix. With the mix it misheard sung words: "swarm" came out as "swan", "run" as "vun".
- Whole-song transcription drifts after silences, so lyrics get anchored to vocal-stem phrases.
- Motion prompts must pin the keyframe's lighting, and must not inherit mood-board captions (those triggered provider rejections).
- Wide shots with tiny faces lose identity when the model pushes in, and crowd shots need an explicit "every dancer is a clone of her" clause.
- Overlapping lead and backing vocals can close the lips on n/d sounds ("need" became "meeb"). Roll two takes and compare mouth crops.

Along the way it closed 4 open issues and merged 5 PRs in PortOS. I even changed the song's ending lyric after the video was finished, and only the last segment had to be re-rendered.

The full starter prompt is in this thread: https://x.com/antic/status/2105308911452790926

document fetched 2026-10-01 00:24 · canonical https://reddit.com/r/ClaudeAI/comments/1wu7p0v/i_had_claude_direct_a_full_music_video_for_3788