When a fix for hallucinations made our agentic video creation system’s reels more technically accurate but lifeless, I used Claude Code to trace the cause through 20+ agents’ instructions and ship a fix that brought the energy back without hallucinations.
Overview
Before
After
Energy and action restored to videos, balancing technical and experiential quality.
Our multimodal agentic system turned a mobile game’s images, gameplay clips, and descriptions into short-form videos for social media promotion. After a quality sprint raised its pass rate from 3% to 31%, avatar accuracy and hallucinations were the biggest gaps left to fix.
One of our engineers went deep on it in a solo sprint. His fix worked technically. Avatars became far more accurate, with almost no hallucinations. But it drained the energy out of the videos and made action scenes feel stilted.
In user research, we showed young gamers videos from before his fix, which were energetic but prone to hallucinations, alongside videos from after it, which were accurate but stilted. They preferred the energetic videos, despite the hallucinations. I see this tug-of-war often in AI systems design, between technical quality and the experiential quality of how something feels.
BeforeLess technically accurate but energetic
AfterMore technically accurate but stilted
Young gamers preferred the energetic version, despite its hallucinations. My job was to keep the technical accuracy and bring back the energy.
The engineer’s fix for better avatar accuracy grew our agentic system from 8 agents to more than 20. The system was now too complex to debug session logs by eye to find what was killing the motion. So I pointed Claude Code at every agent’s instructions and a few sets of full session traces, including the reasoning, the hand-offs, and the prompts each agent generated.
8 agents · previous system
20+ agents · new system
Claude Code found the cause in the video-generator agent for action scenes. A “motion happens in place” rule, written to stamp out hallucinations, was suppressing all motion.
The bug wasn’t in the code. It was in the instructions somebody wrote. The engineer who wrote the rule had just rolled off the team, so the fix was on me because the rest of our engineers were improving our input asset pipeline.
Agent trace
Video generator output
Instead of one rule for every action scene, I added subject-type classification with different rules for 3 types of action scenes—character action, object action, and static scenes.
Fix to restore motion
Video generator instructions
# Subject Type Classification
Before constructing the prompt, classify the scene by reading `motion_summary` and `scene_spec.description`. Pick ONE subject type:
- **CHARACTER-ACTION** — A character is the primary actor performing visible action (running, attacking, mining, swinging an axe, sprinting through an obby, celebrating). Motion is BY the character.
- **OBJECT-ACTION** — Primary motion is a non-character object (asteroid, vehicle, projectile, explosion, falling debris). Character may be present but is not the moving focus.
- **STATIC** — Primary subject is at rest or held: avatar speaking to camera, posed celebration, dialogue settle, environment hold. The `DIALOGUE SETTLE`, `DOLLY PUSH-IN`, and `DOLLY PULL-OUT` motion patterns are static.
This classification governs three rules below: Camera framing, Environment lock, and Negative prompt.
----
- **Camera framing for fast subjects:** Apply based on SUBJECT TYPE:
- **CHARACTER-ACTION:** No camera-framing override. Use the camera angle specified in the cinematic vision (chase, side, overhead, push-in, etc. all valid). Character has internal animation (leg cycle, arm swing, body bob) that conveys motion even from a chase view.
- **OBJECT-ACTION:** Prefer a fixed external camera (side-view, overhead, or stationary perspective) or a chase or POV camera. Chase or POV cameras pin the subject in the center of the frame, hiding its motion—the subject stays still while only the background moves. A fixed camera lets the subject visibly travel across the frame, which reads as speed.
- **STATIC:** No camera-framing override.What each rule does
Fast, dynamic movement
Motion happens in place
Action comes from camera movement
Before and after the instruction fix
BeforeBefore with stilted motion
AfterAfter with energetic motion
I made the fix with Claude Code too, and I worked with it the way I work with my engineering partners. I gave creative direction, asked hard questions, and ran experiments. I’m not an engineer, but I understood the system well enough to know when its recommendations were on the mark and when they’d add complexity or sneak in a trade-off.
In testing, the classification broke on games where the object you control is the character — like an asteroid rolling through a town, smashing everything to rack up points. My object-action rule still rendered that as “motion happens in place,” making the asteroid remain still.
So I went one step deeper and added a sub-case rule for object scenes. The rule separated player-controlled objects that move from other objects in the environment, so it could apply to any game where you play as an object.
What changed
Player-controlled objects get their own sub-case instead of being treated like other object-action scenes.
A chase camera works when the prompt shows the object spinning and the mess it leaves behind.
Before
AfterMy fix brought the energy back without bringing the hallucinations back. I shipped it through code review days before I left Meta, so I don’t have post-launch numbers or follow-up research to share, but the before-and-after shows the difference. It was the last of 17+ changes I landed in the production pipeline in 2026, across orchestrators, agent prompts, Python validation tools, and YAML schemas.
The fix also confirmed 2 things about my model design practice. First, classification schemas that give each category its own rules have been central to how I’ve designed language systems for years, long before agents. I used the same approach when encoding a brand voice into a system prompt.
Second, you have to measure model quality through both objective and subjective lenses. The release before mine passed on accuracy and still lost with players. That idea is at the core of the AI quality framework I built for Meta Horizon.