Case Study — A Real, Unedited Session

We Let Claude Re-Cut an Ad 6 Times. The Original Won.

This is a real chat session, start to finish. A user typed "i want to score an ad" into Claude with the PreTestAds MCP connected. Over the next hour, Claude scored a TikTok ad, watched it frame by frame, re-cut it four different ways, generated brand-new AI footage with a second MCP, and scored every version. Six scores later, the data delivered a verdict nobody expected — and a lesson worth more than any single edit.

The whole story in four lines

  • The original ad scored 47/100 with a "slow" opening everyone would cut.
  • Claude cut it. Score dropped to 42. Cut harder: 30. Cut the ending too: 14.
  • Claude generated an AI opening shot with Higgsfield and grafted it on: 42, then 22.
  • The untouched original beat every edit. The "slow" part was doing invisible work.

Round 0 — score the original

The ad: a 10.5-second LED strip-light install for car interiors. Hands coil a glowing wire, press it into the console trim, and then — payoff — the whole dashboard glows through a night drive. Claude uploaded it through the MCP's upload widget and got the first score back in about three minutes: 47/100, moderate — better than 47% of the 76 top-performing TikTok ads in the benchmark library.

Opening shot: hands coiling LED wire with scissors
0s — the static open. Attention: 14.
Hands tucking the LED wire into car console trim
4s — the install moment. Attention: 100.
Car dashboard glowing blue during a night drive
6s — the payoff. Attention: 68.
Daytime car interior with amber LED glow
9s+ — the daytime closer. Attention: 55.
02550751000s1s2s3s4s5s6s7s8s9s
Original — 47/100

The report flagged the obvious problem: hook strength 25/100. The first two seconds — a static shot of coiled wire and scissors — hit literal zero at second 1. Then Claude did something a dashboard can't do: it called get_ad_frames and watched the ad, one frame per second, mapping each moment against the curve. Its diagnosis: "coiled-wire-with-scissors is prep, not a hook — cut the first seconds and open on the install moment." Textbook advice. Every hook guide on the internet agrees. So it cut.

Rounds 2–4 — the trims (it gets worse)

Claude re-cut the file with ffmpeg — no re-shoot, same footage — and pushed each version back through the MCP for a fresh score:

  • v2 — open on the peak (cut first 3.3s): 30/100. The cold open landed for exactly one second, then crashed. Dropped mid-install with no context, viewers had no idea what they were looking at. The 2–4s "boring" buildup was what made the payoff land.
  • v3 — gentler trim (cut only the truly static 1.8s): 42/100. Hook strength more than doubled (25 → 56). But trimming the front shifted the weakest footage into the closing window — purchase signal fell from 60 to 39, and the final second flatlined. Better hook, worse ad.
  • v4 — v3 plus cutting the "weak" daytime ending: 14/100. The worst of the run. Removing the ending gutted the arc entirely. An ad that dies at the end scores worse than one that starts slow.

Three edits, three drops. And note what happened each time: some individual metric improved — hook strength up, peak earlier — while the overall score fell. If you were optimizing a single metric instead of the whole ad, you'd have shipped a loser and felt good about it.

Rounds 5–6 — Claude generates new footage (two MCPs, one loop)

Trimming had hit its ceiling, so the user said: "use the higgsfield mcp to get the footage." Claude extracted a reference frame from the scored ad — the glowing night-drive dashboard — and sent it to the Higgsfield MCP with a prompt: dark dashboard, then the LED strip snaps on in a burst of color. The generated clip matched the real footage almost perfectly.

AI-generated frame: dark car dashboard before lights turn on
AI clip, 0s — dark beat before the snap.
AI-generated frame: dashboard fully lit in purple LED glow
AI clip, 1.6s — lights snapped on.

Claude grafted the AI clip onto the front of the untouched original (scaling, frame-rate matching, and audio handled with ffmpeg) and scored the hybrid. v5: 42/100 — with the best back-half numbers of the entire experiment (purchase signal 87, strongest 5-second run averaging 82). The failure was surgical: the dark beat before the snap read as two seconds of nothing. So Claude cut the dark beat and kept only the snap. v6: 22/100. The hard cut from a glowing finished dash back to dark prep footage read as a broken promise — the model punished the whiplash at the seam.

02550751000s1s2s3s4s5s6s7s8s9s10s11s
v5 — AI hook, 42/100v6 — tighter AI hook, 22/100

The final scoreboard

Six versions. Six real scores. The untouched original beat every single edit — including the ones with objectively better hooks and stronger endings.

What the data actually taught us

  • Continuity beats components. The model consistently rewarded the unbroken arc — prep → install → payoff — and punished every splice at the seam, no matter how strong the pieces were individually. "Slow" footage can be load-bearing.
  • A single metric will lie to you. v3 doubled hook strength and scored worse. v5 posted the best purchase signal of the run and still lost. The overall percentile is the only number that integrates the whole story.
  • Editing intuition needs a referee. "Cut the slow opening" is the most standard advice in short-form video. Here it was wrong four times in a row — and it only cost a few testing credits to find out, instead of a live media budget.
  • The loop is the product. Score → watch frames → diagnose → edit → re-score, with every step in one chat. Claude was wrong about the first edit, and it didn't matter — the loop caught it in minutes. That's what pre-testing is for.

What the session actually looked like

No dashboards, no exports, no tab-switching. The entire experiment was a conversation:

You: i want to score an ad

Claude: Drop your video into the upload box — I'll take it from there. [3 minutes later] Score: 47/100, moderate. Your hook is holding it back — the first two seconds are a static shot. Want me to re-cut it and score the new version?

You: Sure lets see it

Claude: The re-cut scored 30 — I have to be straight with you: my edit made it worse, and the curves show why. The buildup I cut was setup, not filler…

You: use the higgsfield mcp to get the footage

Claude: Generated a matching "lights snap on" opening from a frame of your real footage, grafted it on, scored it: 42 — best back half of any version, but the dark beat killed the hook…

Note the second exchange: Claude reported that its own edit failed. That's the point of wiring a scoring model into the chat — the number is the referee, not the AI's confidence and not your gut.

Run this loop on your own ad

Connect the PreTestAds MCP to Claude and say "score my ad." Setup takes two minutes.

How to Connect the MCP

No account? Sign up — your first analysis is free.

Frequently asked questions

Can Claude score and edit ads by itself with the PreTestAds MCP?

Yes. With the PreTestAds MCP connected, Claude can upload a video, get a 0-100 attention score with a second-by-second engagement curve, extract and watch frames from the scored video, re-cut the file with ffmpeg, and score the new version — all in one chat. In this case study Claude ran that loop six times on a real TikTok ad.

Does trimming a slow opening always improve an ad's attention score?

No — in this test it made the score worse every time. The original scored 47; cutting the slow first seconds produced 42, 30, and 14. The 'slow' opening was buildup that made the payoff land. Attention models reward narrative arc, not just fast openings.

Can Claude combine the PreTestAds MCP with a video generation MCP like Higgsfield?

Yes. In this session Claude extracted a reference frame from the scored ad, sent it to the Higgsfield MCP to generate a matching 'lights snap on' opening shot, grafted the AI clip onto the original with ffmpeg, and scored the hybrid — a full make → test → fix loop across two MCPs in one chat.

What did the six-version experiment prove?

Every edit that broke the ad's narrative continuity scored below the untouched original, even when individual metrics like hook strength or purchase signal improved. The practical lesson: pre-test before you re-cut, because intuitive edits — even data-inspired ones — can quietly destroy a working ad.

By Nina Krecicki · Published