18 Shots in 3 Days: How Higgsfield's Animation Team Brings AI into a Traditional Animation Pipeline

With AI animation, the hard part usually isn't getting one good-looking shot. It's keeping the people, the car and the lighting consistent across 18 shots, so the headphones don't suddenly vanish and the taxi doesn't turn into a different car.

An animation team at Higgsfield, all from traditional animation, finished a car chase like that in 3 days. Their answer wasn't a stronger model. It was bringing over the old rules of traditional animation: build the reference images first, test first, previs the complex shots in Blender, and write every prompt to a fixed structure.

The sequence comes from Passport Rush, a 5-minute action comedy they're making: a girl gets to the airport, realizes her passport is on the other side of town, and jumps into a taxi to race back. None of the team had touched AI before joining the company. In the first episode of their new YouTube channel, Higgsfield Animation, they spend 28 minutes laying out how they went from storyboard to final shot, including the actual prompts and three failures.

Here's what came out of it:

If you want the whole process, here is the full 28-minute breakdown with Chinese and English subtitles:

The host, Amina, sums up the team's approach in one line: they want to bring AI into the animation pipeline without losing the craft. What follows goes in the order they actually worked: storyboard, then assets, then testing, then video prompts, complex shots through Blender first, and finally three failures and how each was solved.

Decide at the storyboard: which shots you make by hand, which go to the model

Pre-production starts the same way as traditional animation: idea, script, storyboard. What they wanted to explore in this sequence was how 3D previs and AI can work together in action scenes. Previs (previsualization) means blocking out the action and camera with very simple 3D models before real production, a bit like a walk-through rehearsal before a live-action shoot.

The difference is that they assigned every shot a route at the storyboard stage. Panels marked orange go through Blender previs; unmarked panels go straight to text-to-video.

The rule for sorting is simple: shots with very specific action, carefully planned camera work, or anything that would take ages to explain to a model in words go into Blender; shots with simple action get generated directly. In other words, if doing it yourself is faster than describing it, do it yourself. They're visual artists, and some things are much easier to draw and animate than to describe in a wall of text.

Another attitude worth noting: the storyboard is a reference, not a contract. Video generation is unpredictable by nature, so they locked only the story beats and main locations instead of planning every shot down to the frame, leaving room for surprises and adjusting as they went.

The team structure mostly mirrors a traditional animation pipeline, just with fewer people each wearing several hats. One thing stands out: they have no dedicated prompt engineer. Whoever builds the characters writes the character prompts, whoever builds the environments writes the environment prompts, and the animators write the shot prompts. You own a piece, you write its prompts.

Asset-first: sloppy references make a sloppy film

The whole pipeline rests on one rule they call the asset-first approach: the quality of your image inputs determines the quality of the film. Blurry, sloppy, or inconsistent image inputs lead to a sloppy film.

"Image inputs" here means character sheets, prop sheets, and background plates. When generating video, these are attached as references that tell the model what everything looks like. To manage this, they created an asset manager role, responsible for making sure characters, props, and backgrounds look like they belong to the same film and the same world. It's roughly the art director's job in a traditional pipeline.

The art direction for this film is stylized 3D characters on watercolor backgrounds. Even in a traditional pipeline, getting those two textures to sit together harmoniously is hard, and it became the project's biggest challenge.

Characters: AI drafts, humans choose and fix

Their project timelines are usually tight, sometimes just a month, so for character design they generate a few first drafts from prompts and then refine them by hand to save time. The process goes like this:

1. Write a short brief and have Claude expand it into a full character sheet prompt covering appearance, outfit, and pose, plus composition and lighting. They use Claude, though the host says any AI chat tool works. How short is the brief? This is the one they wrote for the driver in the video:

   Write a prompt for this character: an elderly heavyset man, short, round kind face… big fluffy grey mustache, round glasses in a thin gold frame… Outfit: burgundy short-sleeve polo, light blue jeans, brown leather sandals.

   A request line, then one chunk for looks and one for the outfit. The heroine's brief had the same two parts: long straight red hair with a cowlick sticking up, big eyes looking upward; an oversized light-blue zip hoodie with red letter patches on the chest, a beige pleated miniskirt, white headphones around her neck. Composition, lighting, and background were left for Claude to fill in.

2. Review it yourself before generating, checking that details and design are consistent, and fix what's off. These choices directly shape the output, so be clear about what you want.

3. In Higgsfield's Cinema Studio, pick the in-house image model Soul 2.0, paste in the prompt, set the aspect ratio, and generate a batch of 4.

4. Choose, and combine the good elements from several images.

How they picked the heroine is telling. The first was nice but a bit too goofy; they wanted funny but also a little cool. The second was cute but too realistic; they wanted more exaggerated features and proportions. The third was closest, and the orange-red hair really popped against the watercolor backgrounds, but the proportions needed work and she was dressed too formally. Amina says she usually goes to the airport in sweatpants and whatever's most comfortable. The final version brought back the hoodie and sneakers from the first image and added a short skirt to improve the silhouette.

Then came the hand edits, which Amina calls "the most fun part":

• The original colors were too gray to carry comedic energy, so they raised brightness and saturation in Photoshop.
• They added a few hand-painted brush strokes around the character's silhouette to give a slightly "hybrid" feel while keeping the character itself 3D.

With the lead settled, the other characters followed the same method, such as the taxi driver. One driver option was thrown out immediately because his hair was different in every view of the character sheet. The sheet will be used as a reference for the video model; if the views don't match each other, later shots can't be consistent either, so check multi-view consistency first when choosing. The final driver is a round-faced, kindly-looking guy with a flat cap and a big mustache who drives like he has nothing to lose. Amina calls him the "driving force" behind most of the action. They added the same hand-painted strokes around his face, cap, and clothes. Those strokes make the two 3D characters look like they belong in the same film.

Environments: watercolor nearly got cut, and lighting saved it

Environments follow the same process with one extra step: short brief → Claude expands it into a full environment prompt → style test. The setting for this sequence is a long highway leading from the airport into the city, with no buildings around.

The environments taught them their biggest style lesson. They wanted pure watercolor, but in tests it looked dirty, muddy, and lifeless. The team hit a dead end and seriously discussed dropping watercolor and taking the whole film in a 3D direction.

They didn't give up right away. During style tests they ran watercolor and 3D in parallel, trying to find a balance between the two. The versions that didn't make it still had value: they helped the team rule out wrong directions early and lock the art direction before carrying it into production. Eventually they found the problem was the lighting. The film had been set at noon, and flat light made the image flat. Switching to warm late-afternoon light with long, hard shadows suddenly brought out volume and depth.

That fix became the visual formula for the whole film: generate environment images in watercolor style, then add golden-hour lighting in the video prompt during production.

They use this to stress that assets need to be tested in video during pre-production. It's the equivalent of the animation tests traditional pre-production does for characters, except that in this pipeline the backgrounds have to be tested too. Given how AI behaves, backgrounds can affect the whole image in unexpected ways, and they can pull the props off course as well.

Props: only make sheets for things that recur

The taxi plays a big role in the film, so they designed it like a character: it has to bounce, have plenty of squash and stretch, and match its owner's personality. Amina compares it to the relationship between a horse and its rider.

The taxi started out in an obviously watercolor style. The short brief they gave Claude was: "A watercolour drawing of an old yellow taxi. 3/4 front left view. Single car in frame. Rich watercolour washes with drips and highlights. Soft shadow under the car." Claude expanded it into a full prop sheet prompt, which they generated with Soul 2.0. In video, the trouble started: the texture didn't seem to sit in the right place, and the whole car looked like a PNG stretched over a 3D model. It looked cheap.

After many attempts, they worked out why: heavy watercolor texture is a huge amount of information for the model to hold together in motion, and putting a watercolor prop on a watercolor background doubles the load, with two layers of dense texture fighting in every frame. The fix was to lighten the car: make the car clean 3D and keep the watercolor in the world around it.

They had a few more practical habits for props:

• Not everything gets a sheet. Only props that appear across multiple shots and must stay consistent get their own. For example, three trucks show up in more than one shot, so each got a sheet; everything else is generated from text. A truck that flashes by kept its watercolor style, since its screen time was too short to be worth extra effort.
• Editing an existing image is faster than generating from scratch. Once the taxi was final, the convertible version and the back-seat interior were both made by feeding the main car design into Seedream 5.0 Pro and editing it.
• Watch out for the model taking words literally. The brief for the final explosion originally said "mushroom-shaped explosion," and the model really did generate a smoking mushroom. Switching to "nuclear blast" got it close, then they hand-added painted texture to match the characters' strokes, and that explosion made it into the final cut.

Test before production, and turn the lessons into rules

Between pre-production and production they set aside a dedicated testing phase: drop the freshly made assets straight into video generation and see whether they hold up in a real shot. The reasoning is simple: problems found now are cheap to fix; going back halfway through production is painful.

They had no rules yet when testing started. Most of the rules they use now came from three lessons in this round:

Lesson one: how much texture to strip from props. This is the watercolor taxi story above. Testing helped them decide how much texture to keep on the car body, and they wrote that direction into the prompts so the car's surface stays consistent, with no photorealistic car suddenly appearing in the middle of a hand-painted scene.

Lesson two: the model fills empty spots on its own. They started testing the car before the character sheets were done, so the car was empty at first. Random passersby kept showing up inside it, even though they weren't in the references or the prompt. Once the characters were added, the problem went away.

Lesson three: fix the lighting before scrapping the art direction. A colleague casually added "late afternoon light" to a prompt just to see what would happen, and the image instantly got warmer and deeper. Low warm sun with long shadows became the direction they've used ever since.

They went into production with that lighting setup plus the tested car and character references.

Why write an essay for a 4-second shot

Most of the time, "production" here means one thing: text-to-video with the assets attached as image inputs. That's how all the straightforward shots are done. The key to this step is the prompt.

One of their shot prompts is very long. Amina expects viewers to ask: who writes an essay for a 4-second shot? Her answer: every detail you don't write, the model will decide for you, and it's a terrible guesser.

There's one overarching principle before you write: order matters. The model reads top to bottom, like a person, and the further down it goes, the more loosely it follows. So the first 20% of the prompt is especially critical, and the most important rules go at the very top.

Their video prompts are always split into five blocks. The example below is a shot they actually showed: a taxi with its roof torn off tears down a mountain road with a traffic cop standing on the hood, glaring at the driver; the driver slams on the brakes, and the cop is flung off, landing back-first on the rear seat of his partner's motorcycle far ahead.

Block one: scene and style

Describe in plain language what happens in the shot, along with the style rules and the physics of this world.

Notice how much is packed into the opening paragraph: stylized 3D animated sequence, slapstick action comedy, 2 shots, 1 hard cut, each shot one continuous take, real-time speed, no slow motion, animating on twos (every key pose held exactly 2 frames, pose to pose, no mushy in-betweens), the full 12 principles of animation applied, 16:9, 6 seconds total, 3 seconds per shot. The things a model is most likely to improvise on are all locked at the very top.

Block two: reference descriptions

Spell out what each attached image and video is, one by one, like a production call sheet. Leave it vague and the model starts guessing again.

Two points for this block:

• You can describe personality and texture, but don't describe appearance; leave appearance to the references. From the screenshot, they did still include a few key identifying features, such as the taxi's "667" plate, the "T" in a circle on the doors, and "the roof stays torn off," to keep the model from getting critical details wrong.
• Every line ends with "100% matches the reference". Amina says consistency hinges on that one line.

Block three: shot structure

Split shots at the hard cuts, give each shot its own section, and write the action and timing beat by beat: what happens first, what happens next, how long each action lasts. It's the storyboard panels written out as text.

Their approach follows a clear pattern, with every shot written in four layers:

• Shot job: what the shot needs to convey (speed, the hard stop, the violent launch into depth).
• Camera: very specific, e.g. "about 25 cm above the ground, just behind the left rear wheel, tire and fender filling the left foreground, looking slightly up along the car's flank, rigidly mounted to the car, only tiny road vibration."
• First frame: where everything is and what state it's in on the first frame, e.g. the cop standing on the hood glaring at the driver, the heroine's hair streaming above the back seat, the other cop on his motorcycle far off at lower right.
• Action timing: down to the beat, e.g. "Beat 1 (~1.1 s) full speed... Beat 2 (~0.15 s) the brakes bite, and at the same moment the cop is launched..."

Block four: locks and constraints

Video models love to improvise, and constraints pull them back by stating what must be followed exactly. Constraints vary by project, but there's one they write every time: no music, SFX only, because music makes post-production harder.

From the screenshot, their constraints have two features. First, they restate the entire action from start to finish in "must" language, tying every character and prop back to its reference number. Second, they pin down the spots most likely to go wrong: the driver and the heroine can't leave their seats, the roof stays torn off, only small non-verbal grunts are allowed, no music and only diegetic sound, no text, logos, or subtitles.

Block five: animation principles

The 12 principles of animation get their own section in the prompt: squash and stretch, anticipation, follow-through, and so on, all required. Amina admits the model doesn't reliably follow every one, but writing them in does get the motion closer to the feel they want.

For spatially complex shots, add a "location map"

The voiceover only covers the five blocks above, but another prompt shown in the video (for the in-car grab-handle shot discussed later) has an extra LOCATION MAP block devoted to spatial relationships, e.g. "the camera is inside the taxi cabin, mounted at the front seats, looking back at the full rear bench..." Cramped shots like a car interior, with people and objects packed close together, are where the model most easily scrambles positions, and this block sets up the camera and layout first.

The same prompt shows another technique worth borrowing: pin down cause and effect. They wrote that she never deliberately yanks the handle and doesn't grab it to steady herself; it's the inertia of the car swerving hard left and right that tears the handle off its mount. They also specified that after it falls, the handle keeps its original design and "does not turn into a loop, bar, rope, rail, hook, or anything else." The model will fill in "why did the handle break" and "what does it look like broken" on its own; leave that unwritten and it may well have the character pull it deliberately or draw the handle as something else.

Write the structure once, share it across the team

You don't have to type all this from scratch every time. They gave the structure to Claude and turned it into a skill, essentially a system prompt, and every shot prompt starts from it. The team shares the same system prompt, so everyone's prompts come out with roughly the same structure.

If you want to reuse it, the structure boils down to the template below (organized by their five blocks; replace the angle-bracket parts with your own content):

[SCENE & STYLE]
<film type and style>, <tone>. <N> shots, <hard cut / one continuous take>, real-time speed, no slow motion.
<physics and style rules of this world>. <aspect ratio>, total <X> s: Shot 1 = <a> s, Shot 2 = <b> s.
Scene: <in plain language, what happens in this shot>

[ACTIVE REFERENCES]
<<<image_1>>> — <identity: location / character / prop>, <personality or texture; essential identifying features>. 100% matches the reference.
<<<image_2>>> — ...
<<<video_1>>> — for camera movement, action rhythm, and timing only; do not copy its look.

[SHOT STRUCTURE]
Shot 1 — <title> (<seconds>)
Shot job: <what this shot needs to convey>
Camera: <height, position, direction, whether it follows the subject>
First frame: <what is in the first frame and where>
Action timing: Beat 1 (~x s) — ... Beat 2 (~y s) — ...
Hard cut.
Shot 2 — ...

[LOCKS & CONSTRAINTS]
All character and vehicle designs 100% match their references. <who must not move / what must stay unchanged>.
No music, SFX only. No text, no logos, no subtitles.

[LOCATION MAP (optional, for spatially complex shots)]
The camera is at <exact position> in <<<image_N>>>, looking <direction>; <relative positions of people and objects>.

[ANIMATION PRINCIPLES]
Full 12 principles applied: squash & stretch, anticipation, staging, follow-through & overlap, ease in-out, arcs, secondary action, timing, exaggeration, appeal.

Complex shots: the Blender grayblock handles motion, references handle looks

Back to those orange marks on the storyboard. Take the hardest shot as an example: the taxi is blocked by two trucks side by side with nowhere to go, so it tips up onto two wheels, squeezes through the narrow gap between the trucks, and lands back on all four once it's out.

Shots like this take several steps:

1. Read the storyboard and confirm the sequence: approach the gap, tip onto two wheels, squeeze through, land only once fully clear. It all happens fast, so the order has to read at a glance.
2. Build the action and camera in Blender. This is the animators' favorite part: when the tilt starts, how long it stays on two wheels, how it rocks back after landing, all of it can be tuned over and over. The camera has to keep up with the action without losing the subject, and that gets tuned here alongside the animation. They also added a little squash and stretch to the wheels, just enough to remove the stiffness and give the car some "personality." The previs locked down two things for them: camera and timing, the latter especially.

3. Export a plain grayblock preview: no color, no materials, just gray geometry, motion, and camera. At this stage it only shows motion and composition; the look comes from the references.

4. Assemble in Higgsfield: attach the grayblock preview as the video input, then attach references for the taxi, the two trucks, the characters, the highway, and the lighting, and have Claude write the prompt using the five-block structure above, stating what each reference is for: the video handles motion and camera, the images handle looks. From the panel screenshot, the settings were: generation mode set to References, attached inputs numbered in order and referred to in the prompt as @Video 1 and @Image 1 through @Image 8; 16:9, 1080p, 7 s duration, a batch of 4.

   You may have noticed that the five-block screenshots earlier used <<<image_1>>>, while this panel uses @Image 1. The video doesn't explain how the two relate, but they do the same job: give each attached reference a number so the prompt can call it by name. When you try this, use whatever reference syntax your tool supports.

A few key lines of this shot's prompt are readable on the panel: replace the car in the original video with the compact car from @Image 1, replace the two trucks with the semi-trailer trucks from @Image 2 and @Image 3 respectively; retain no environmental elements from the original video and rebuild the whole setting as a highway; base the lighting and atmosphere on @Image 4, with the sun direction strictly aligned with @Video 1; use @Video 1 only for camera movement, rhythm, and sun direction, and do not reproduce its multiple cuts. The model selector at the bottom of the panel shows Seedance 2.5, which the voiceover doesn't mention.

5. Pin down the details most likely to go wrong. For example, both characters must stay in their seats the whole time, and the taxi must not land until it's completely clear of the trucks. Leave these out and the output keeps breaking.
6. After generating, check against the previs: timing, camera, and whether the car and characters still match their designs. This shot got all of it right.

The comparison below shows what "grayblock handles motion, references handle looks" means in practice: the top half is the storyboard or Blender previs, the bottom half is the final generated image.

Simple shots skip Blender and go straight from text to video. Checking and adjusting work the same way, just without the previs step.

Three failures, three troubleshooting rules

Of the 18 shots in this sequence, Amina picked three that took extra work. The first two were eventually solved; the third wasn't.

Model can't read your reference? Pull the camera back

In one shot, the girl is pressed right up against a truck door. They had already built the reference in Blender, but no matter what, the generations came out wrong.

It turned out the camera in the Blender previs was too close to the action. It looked fine to them because they knew what every gray block was; but to the model, with no textures and no faces, the frame was mostly big gray shapes with almost no surrounding environment visible, so it struggled to tell where people and objects were relative to each other.

They went back into Blender, pulled the camera back a bit to show more of the scene and make the action easier to read, and the next generation worked. Rule one: if the model can't read your reference, you're probably showing it too little. Pull back.

Check even the screenshots you added just for composition

In another shot, the girl buckles two seatbelts into an X across her chest. They expected it to be easy, since earlier tests had already worked. But those tests used an early character design. When they swapped in the final design and regenerated with the prompt and other references unchanged, the camera moved every single time and the composition they wanted just wouldn't hold. All they meant to do was update the character's look, and they ended up spending time getting the camera right all over again.

Their fix: take screenshots from earlier generations where the camera was right and attach them as references. The composition did lock in, but a new problem appeared: the girl's headphones vanished in every generation.

Looking back at that screenshot, the headphones were never in it. They had picked it purely for the camera angle and missed the detail. The character sheet had headphones, the screenshot didn't, and the model was getting contradictory references.

The fix was to bring the screenshot in line with the character sheet: open it in Photoshop, paint the headphones back in by hand, and use that version as the reference. On the next generation, the headphones were back and the camera was still right.

Rule two: check every reference image, including the ones you added only for composition.

Subtle performance is much harder than big set pieces

The last one is Amina's favorite shot, and it still hasn't been made, so it didn't make the final cut.

It's a small gag inside the taxi: amid the chaos, the girl grabs the grab handle on the roof and it comes off in her hand; she panics, reaches for the other one, and that breaks too. Looking at the storyboard, they weren't worried at all: they'd already done flying logs and explosions, so how hard could two breaking handles be?

It turned out to be an easy shot to underestimate, because it depends on the tiniest details.

• Prop consistency: where the handle is mounted, how it comes off, and what it looks like in her hand all have to be described clearly in the prompt.
• Performance: she has to grab the first handle in a panic, then freeze for a beat, realize it broke, and reach for the other one. The whole gag rests on that little pause, and as animators they know it.

Text prompts alone couldn't produce the pause, so they turned to Blender to animate the performance themselves and use it as a video reference. That didn't go smoothly either: expressions and hand motion from a simple gray character didn't come through in the generations; and when they added more detail, the model started copying the Blender geometry instead of drawing her from the character sheet.

They call this the performance gap: they know how she should react and where she should pause, but they can't get the performance they want while also keeping her final character design consistent in an AI-generated shot.

What surprised them most was that a shot that would be fairly straightforward with traditional methods turned out this hard. They had assumed big action scenes would be the difficult part, and found instead that what AI struggles with is subtle character performance. Rule three: test subtle performance moments early.

Where this pipeline stands now

Looking back, what this team did was keep each stage of the traditional animation pipeline and then decide, stage by stage, which part to hand to the model:

• Handled by people: storyboarding and route decisions, choices on characters and environments, hand color correction and brush strokes, action and camera in Blender, checking every reference, and the final review.
• Handled by models: expanding prompts (Claude), generating sheets (Soul 2.0), editing images (Seedream 5.0 Pro), and generating video under the constraints of the references and previs.

18 shots in 3 days is genuinely fast. But the film is still in production, and they say they haven't cracked every shot yet. Once it's finished, they'll publish the full production numbers: how many generations in total, how much it cost, how long the whole film took. Those numbers don't exist yet, so "3 days" only tells you the pace of this sequence; it can't be converted into the cost of the whole film.

For anyone looking to animate with AI, the most valuable part of this case is the concrete judgment calls: reference quality determines output quality, textures need to leave headroom for motion, fix the lighting before the art direction, put the most important rules at the top of the prompt, pull back when the model can't read you, check every reference, and subtle performance is still where AI falls short.